@lenne.tech/nest-server 11.22.1 → 11.23.0

package/migration-guides/11.22.x-to-11.23.0.md

@@ -0,0 +1,235 @@
+# Migration Guide: 11.22.x → 11.23.0
+
+## Overview
+
+| Category | Details |
+|----------|---------|
+| **Breaking Changes** | `mainDbModel.collection` and `mainDbModel.db` blocked via TypeScript (`SafeModel`) |
+| **New Features** | `process()` depth-based optimization, `debugProcessInput` config, `getNativeCollection(reason)`, `getNativeConnection(reason)`, restricted metadata cache |
+| **Performance** | ~70% less memory on nested service cascades, eliminated redundant `JSON.stringify` and recursive `process()` calls |
+| **Migration Effort** | ~5 minutes if using `mainDbModel.db`; 0 minutes otherwise |
+
+---
+
+## Quick Migration (No `mainDbModel.db` Usage)
+
+If your project does **not** access `this.mainDbModel.collection` or `this.mainDbModel.db` directly:
+
+```bash
+# Update package
+pnpm add @lenne.tech/nest-server@11.23.0
+
+# Verify build
+pnpm run build
+
+# Run tests
+pnpm test
+```
+
+No code changes required. All performance optimizations activate automatically.
+
+---
+
+## Breaking Changes
+
+### `mainDbModel` Type Changed to `SafeModel` (TypeScript Only)
+
+`mainDbModel` is now typed as `SafeModel<T>` which is `Omit<Model<T>, 'collection' | 'db'>`. This is a **compile-time-only** change — runtime behavior is identical.
+
+**What breaks:** Direct access to `.collection` or `.db` on `this.mainDbModel` produces a TypeScript error.
+
+**What does NOT break:** All Mongoose Model methods (`find`, `findById`, `insertMany`, `updateOne`, `aggregate`, `bulkWrite`, etc.) continue to work unchanged.
+
+#### If You Use `mainDbModel.collection`
+
+**Before:**
+```typescript
+// Direct native collection access — bypasses ALL Mongoose plugins
+await this.mainDbModel.collection.insertOne(doc);
+```
+
+**After (Option A — use Mongoose Model method):**
+```typescript
+// Mongoose method — plugins (Tenant, Audit, RoleGuard, Password) fire correctly
+await this.mainDbModel.insertMany([doc]);
+```
+
+**After (Option B — intentional native access with logging):**
+```typescript
+// Logged escape hatch — requires justification
+const col = this.getNativeCollection('Migration: bulk-import historical data without tenant context');
+await col.insertOne(doc);
+```
+
+#### If You Use `mainDbModel.db`
+
+**Before:**
+```typescript
+// Native DB access via model — bypasses ALL Mongoose plugins
+const count = await this.mainDbModel.db.db.collection('chatmessages').countDocuments({ ... });
+```
+
+**After (Option A — inject the target model):**
+```typescript
+// Inject the model via @InjectModel and use Mongoose methods
+constructor(@InjectModel('ChatMessage') private chatMessageModel: Model<ChatMessageDocument>) {}
+
+const count = await this.chatMessageModel.countDocuments({ ... });
+```
+
+**After (Option B — intentional native access with logging):**
+```typescript
+// Logged escape hatch — requires justification
+const conn = this.getNativeConnection('Statistics: count chatmessages across all tenants');
+const count = await conn.db.collection('chatmessages').countDocuments({ ... });
+```
+
+#### If You Use `getModel()`
+
+`getModel()` continues to return the full `Model` type (including `.collection` and `.db`). No change needed.
+
+---
+
+## What's New in 11.23.0
+
+### 1. process() Pipeline Performance Optimization
+
+The `process()` method in `ModuleService` now tracks nesting depth via `RequestContext`. On nested calls (service cascades like A.create → B.create → C.create), redundant pipeline steps are skipped:
+
+| Step | Outermost Call | Nested Calls |
+|------|---------------|--------------|
+| prepareInput | Runs | Runs |
+| checkRights (input) | Runs | Runs |
+| serviceFunc | Runs | Runs |
+| processFieldSelection (populate) | Runs | **Skipped** (unless explicit `populate` is set) |
+| prepareOutput (model mapping) | Runs | **Skipped** (secret removal still active) |
+| checkRights (output) | Runs | **Skipped** |
+
+**Security is maintained** through three layers:
+1. Input checkRights always runs at every depth
+2. Output checkRights runs at depth 0 (outermost call)
+3. `CheckSecurityInterceptor` (Safety Net) runs on the final HTTP response
+
+**Estimated savings for an 8-level service cascade:** ~70% less memory (~120 KB instead of ~400 KB).
+
+No configuration needed — this activates automatically.
+
+### 2. Lean Query for Rights Checking
+
+The `process()` pipeline previously called `this.get()` to resolve `dbObject` for authorization checks, which triggered a **recursive** `process()` call. It now uses a direct lean query:
+
+```typescript
+// Before: recursive process() call with full pipeline
+const dbObject = await this.get(id);
+
+// After: single lean query + map
+const rawDoc = await this.mainDbModel.findById(id).lean().exec();
+const dbObject = mainModelConstructor.map(rawDoc);
+```
+
+This preserves ALL fields (including `createdBy`) needed for `S_CREATOR` and `S_SELF` checks, while avoiding the overhead of a full pipeline pass.
+
+### 3. `debugProcessInput` Configuration Option
+
+The previous `JSON.stringify` debug comparison that ran on **every** `process()` call is now behind a config flag:
+
+```typescript
+// config.env.ts — enable only for debugging
+{
+  debugProcessInput: true,  // default: false
+}
+```
+
+When disabled (default), two `JSON.stringify` calls per `process()` invocation are eliminated.
+
+### 4. Restricted Metadata Cache
+
+`getRestricted()` results (from `@Restricted()` decorators) are now cached per class + property. Since decorator metadata is static (set at compile time and never changes), this eliminates hundreds of `Reflect.getMetadata()` lookups per request for objects with many properties.
+
+### 5. Native Driver Security (`SafeModel`, `getNativeCollection`, `getNativeConnection`)
+
+Two new protected methods in `CrudService` provide logged escape hatches for legitimate native MongoDB driver access:
+
+```typescript
+// Get the native MongoDB Collection for this model's collection
+protected getNativeCollection(reason: string): Collection
+
+// Get the Mongoose Connection (for cross-collection access, schema-less collections)
+protected getNativeConnection(reason: string): Connection
+```
+
+Both methods:
+- Require a non-empty justification string
+- Log a `[SECURITY]` warning on every call
+- Throw an `Error` if no reason is provided
+
+The `SafeModel<T>` type is exported for use in custom service implementations.
+
+### 6. RequestContext.processDepth API
+
+New methods for tracking `process()` nesting depth:
+
+```typescript
+// Get current depth (0 = outermost or no process() call)
+RequestContext.getProcessDepth(): number
+
+// Run a function with incremented depth
+RequestContext.runWithIncrementedProcessDepth<T>(fn: () => T): T
+```
+
+These are used internally by `process()` but are also available for custom pipeline implementations.
+
+---
+
+## Compatibility Notes
+
+| Pattern | Status |
+|---------|--------|
+| All `CoreModule.forRoot()` patterns | Works unchanged |
+| `CrudService` subclasses | Works unchanged (SafeModel is transparent for Mongoose methods) |
+| Custom `process()` calls with `populate` option | Works unchanged (explicit populate overrides nested skip) |
+| `getModel()` callers | Works unchanged (returns full Model type) |
+| `@Restricted()` decorators | Works unchanged (cache is transparent) |
+| Services using `this.mainDbModel.find/save/update/etc.` | Works unchanged |
+| Services using `this.mainDbModel.collection` | **TypeScript error** — use `getNativeCollection(reason)` or Mongoose methods |
+| Services using `this.mainDbModel.db` | **TypeScript error** — use `getNativeConnection(reason)` or inject target model |
+| Services using `@InjectConnection` | Works unchanged (Connection is not affected by SafeModel) |
+
+---
+
+## Troubleshooting
+
+| Issue | Solution |
+|-------|----------|
+| TypeScript error on `mainDbModel.collection` | Replace with Mongoose Model method or `this.getNativeCollection('reason')` |
+| TypeScript error on `mainDbModel.db` | Replace with `this.getNativeConnection('reason')` or inject the target model via `@InjectModel` |
+| `getModel()` return type mismatch | `getModel()` returns full `Model` — no change needed |
+| `processFieldSelection` type error in custom override | Update parameter type to accept `SafeModel` (see `ModuleService.processFieldSelection`) |
+| Debug logs from `prepareInput` disappeared | Set `debugProcessInput: true` in config to re-enable |
+
+---
+
+## Module Documentation
+
+### Native Driver Security
+
+- **Security Policy:** [`docs/native-driver-security.md`](../docs/native-driver-security.md)
+- **Performance Optimization:** [`docs/process-performance-optimization.md`](../docs/process-performance-optimization.md)
+
+### Affected Source Files
+
+| File | Changes |
+|------|---------|
+| `src/core/common/services/module.service.ts` | `SafeModel` type, `mainDbModel` type change, `process()` depth optimization, lean query, debug config |
+| `src/core/common/services/crud.service.ts` | `getNativeCollection(reason)`, `getNativeConnection(reason)`, `getModel()` cast |
+| `src/core/common/services/request-context.service.ts` | `processDepth` field, `getProcessDepth()`, `runWithIncrementedProcessDepth()` |
+| `src/core/common/decorators/restricted.decorator.ts` | Metadata cache, `_.uniq()` optimization |
+| `src/core/common/interfaces/server-options.interface.ts` | `debugProcessInput` config option |
+
+---
+
+## References
+
+- [Native Driver Security Policy](../docs/native-driver-security.md)
+- [process() Performance Optimization](../docs/process-performance-optimization.md)
+- [nest-server-starter](https://github.com/lenneTech/nest-server-starter) (reference implementation)

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@lenne.tech/nest-server",
-  "version": "11.22.1",
+  "version": "11.23.0",
   "description": "Modern, fast, powerful Node.js web framework in TypeScript based on Nest with a GraphQL API and a connection to MongoDB (or other databases).",
   "keywords": [
     "node",
@@ -92,13 +92,14 @@
     "@nestjs/websockets": "11.1.18",
     "@tus/file-store": "2.0.0",
     "@tus/server": "2.3.0",
+    "@types/supertest": "7.2.0",
     "bcrypt": "6.0.0",
     "better-auth": "1.5.5",
     "class-transformer": "0.5.1",
     "class-validator": "0.15.1",
     "compression": "1.8.1",
     "cookie-parser": "1.4.7",
-    "dotenv": "17.4.0",
+    "dotenv": "17.4.1",
     "ejs": "5.0.1",
     "express": "5.2.1",
     "graphql": "16.13.2",
@@ -112,18 +113,19 @@
     "mongoose": "9.4.1",
     "multer": "2.1.1",
     "node-mailjet": "6.0.11",
-    "nodemailer": "8.0.4",
+    "nodemailer": "8.0.5",
     "passport": "0.7.0",
     "passport-jwt": "4.0.1",
     "reflect-metadata": "0.2.2",
     "rfdc": "1.4.1",
     "rxjs": "7.8.2",
+    "supertest": "7.2.2",
     "ts-morph": "27.0.2",
     "yuml-diagram": "1.2.0"
   },
   "devDependencies": {
     "@compodoc/compodoc": "1.2.1",
-    "@nestjs/cli": "11.0.17",
+    "@nestjs/cli": "11.0.18",
     "@nestjs/schematics": "11.0.10",
     "@nestjs/testing": "11.1.18",
     "@swc/cli": "0.8.1",
@@ -135,30 +137,28 @@
     "@types/lodash": "4.17.24",
     "@types/multer": "2.1.0",
     "@types/node": "25.5.2",
-    "@types/nodemailer": "7.0.11",
+    "@types/nodemailer": "8.0.0",
     "@types/passport": "1.0.17",
-    "@types/supertest": "7.2.0",
-    "@vitest/coverage-v8": "4.1.2",
-    "@vitest/ui": "4.1.2",
+    "@vitest/coverage-v8": "4.1.3",
+    "@vitest/ui": "4.1.3",
     "ansi-colors": "4.1.3",
     "find-file-up": "2.0.1",
     "husky": "9.1.7",
     "nodemon": "3.1.14",
     "npm-watch": "0.13.0",
     "otpauth": "9.5.0",
-    "oxfmt": "0.43.0",
-    "oxlint": "1.58.0",
+    "oxfmt": "0.44.0",
+    "oxlint": "1.59.0",
     "rimraf": "6.1.3",
-    "supertest": "7.2.2",
     "ts-node": "10.9.2",
     "tsconfig-paths": "4.2.0",
     "tus-js-client": "4.3.1",
     "typescript": "5.9.3",
     "unplugin-swc": "1.5.9",
-    "vite": "7.3.1",
+    "vite": "7.3.2",
     "vite-plugin-node": "7.0.0",
     "vite-tsconfig-paths": "6.1.1",
-    "vitest": "4.1.2"
+    "vitest": "4.1.3"
   },
   "main": "dist/index.js",
   "types": "dist/index.d.ts",
@@ -179,13 +179,12 @@
   "watch": {
     "build:dev": "src"
   },
-  "packageManager": "pnpm@10.29.2",
+  "packageManager": "pnpm@10.33.0",
   "pnpm": {
     "overrides": {
       "minimatch@<3.1.5": "3.1.5",
       "minimatch@>=9.0.0 <9.0.9": "9.0.9",
       "minimatch@>=10.0.0 <10.2.5": "10.2.5",
-      "rollup@>=4.0.0 <4.60.1": "4.60.1",
       "ajv@<6.14.0": "6.14.0",
       "ajv@>=7.0.0-alpha.0 <8.18.0": "8.18.0",
       "undici@>=7.0.0 <7.24.7": "7.24.7",
@@ -198,7 +197,8 @@
       "path-to-regexp@>=8.0.0 <8.4.2": "8.4.2",
       "kysely@>=0.26.0 <0.28.15": "0.28.15",
       "lodash@>=4.0.0 <4.18.0": "4.18.1",
-      "defu@<=6.1.4": "6.1.6"
+      "defu@<=6.1.6": "6.1.7",
+      "vite@>=7.0.0 <7.3.2": "7.3.2"
     },
     "onlyBuiltDependencies": [
       "bcrypt",

package/src/core/common/decorators/restricted.decorator.ts

@@ -44,6 +44,19 @@ export const Restricted = (...rolesOrMember: RestrictedType): ClassDecorator & P
   return Reflect.metadata(restrictedMetaKey, rolesOrMember);
 };
 
+/**
+ * Cache for Restricted metadata — decorators are static, metadata never changes at runtime.
+ * WeakMap<CacheTarget, Map<propertyKey | '__class__', RestrictedType>>
+ *
+ * Uses WeakMap so that dynamically-generated or hot-reloaded class constructors can be GC'd
+ * when no longer reachable (prevents unbounded growth in test suites and dev watch mode).
+ *
+ * CacheTarget is the class constructor (for instances) or the class itself (when object IS a constructor).
+ * This distinction is critical: getRestricted(data.constructor) passes a class as `object`,
+ * and (classFunction).constructor === Function for ALL classes — so we must use the class itself.
+ */
+const restrictedMetadataCache = new WeakMap<object, Map<string, RestrictedType>>();
+
 /**
  * Get restricted data for (property of) object
  */
@@ -51,10 +64,36 @@ export const getRestricted = (object: unknown, propertyKey?: string): Restricted
   if (!object) {
     return null;
   }
-  if (!propertyKey) {
-    return Reflect.getMetadata(restrictedMetaKey, object);
+
+  // Determine cache target: use the class constructor for instances, the object itself for classes.
+  // When object IS a constructor (typeof === 'function'), using object.constructor would give Function
+  // for ALL classes, causing cache collisions.
+  const cacheTarget: object | undefined =
+    typeof object === 'function' ? (object as object) : (object as any).constructor;
+  if (!cacheTarget) {
+    return propertyKey
+      ? Reflect.getMetadata(restrictedMetaKey, object, propertyKey)
+      : Reflect.getMetadata(restrictedMetaKey, object);
+  }
+
+  let classCache = restrictedMetadataCache.get(cacheTarget);
+  if (!classCache) {
+    classCache = new Map();
+    restrictedMetadataCache.set(cacheTarget, classCache);
+  }
+
+  const cacheKey = propertyKey || '__class__';
+  if (classCache.has(cacheKey)) {
+    return classCache.get(cacheKey);
   }
-  return Reflect.getMetadata(restrictedMetaKey, object, propertyKey);
+
+  // Cache miss: perform Reflect lookup and cache the result
+  const metadata = propertyKey
+    ? Reflect.getMetadata(restrictedMetaKey, object, propertyKey)
+    : Reflect.getMetadata(restrictedMetaKey, object);
+
+  classCache.set(cacheKey, metadata);
+  return metadata;
 };
 
 /**
@@ -257,7 +296,8 @@ export const checkRestricted = (
 
     // Check restricted
     const restricted = getRestricted(data, propertyKey) || [];
-    const concatenatedRestrictions = config.mergeRoles ? _.uniq(objectRestrictions.concat(restricted)) : restricted;
+    const concatenatedRestrictions =
+      config.mergeRoles && objectRestrictions.length ? _.uniq(objectRestrictions.concat(restricted)) : restricted;
     const valid = validateRestricted(concatenatedRestrictions);
 
     // Check rights

package/src/core/common/interfaces/server-options.interface.ts

@@ -1109,6 +1109,14 @@ export interface IServerOptions {
     CronExpression | CronJobConfigWithTimeZone | CronJobConfigWithUtcOffset | Date | Falsy | string
   >;
 
+  /**
+   * When true, logs a debug message when prepareInput() changes the input type during process().
+   * Enable only for debugging — has performance cost due to JSON.stringify on every process() call.
+   *
+   * @default false
+   */
+  debugProcessInput?: boolean;
+
   /**
    * SMTP and template configuration for sending emails
    */

package/src/core/common/services/crud.service.ts

@@ -1,6 +1,8 @@
-import { NotFoundException } from '@nestjs/common';
+import { Logger, NotFoundException } from '@nestjs/common';
 import {
   AggregateOptions,
+  Collection,
+  Connection,
   Document,
   Model as MongooseModel,
   PipelineStage,
@@ -76,7 +78,11 @@ export abstract class CrudService<
     return this.process(
       async (data) => {
         const currentUserId = serviceOptions?.currentUser?.id;
-        return new this.mainDbModel({ ...data.input, createdBy: currentUserId, updatedBy: currentUserId }).save();
+        return new (this.mainDbModel as MongooseModel<Document & Model>)({
+          ...data.input,
+          createdBy: currentUserId,
+          updatedBy: currentUserId,
+        }).save();
       },
       { input, serviceOptions },
     );
@@ -314,7 +320,7 @@ export abstract class CrudService<
     filter?: FilterArgs | { filterQuery?: QueryFilter<any>; queryOptions?: QueryOptions; samples?: number },
     serviceOptions: ServiceOptions = {},
   ): Promise<{ items: Model[]; pagination: PaginationInfo; totalCount: number }> {
-    serviceOptions.raw = true;
+    serviceOptions.force = true;
     return this.findAndCount(filter, serviceOptions);
   }
 
@@ -443,11 +449,77 @@ export abstract class CrudService<
   }
 
   /**
-   * Get service model to process queries directly
+   * Get service model to process queries directly.
    * See https://mongoosejs.com/docs/api/model.html
+   *
+   * Note: Returns the full Mongoose Model (including `.collection`).
+   * Prefer using CrudService methods or Mongoose Model methods directly
+   * to ensure plugins (Tenant, Audit, RoleGuard) fire correctly.
    */
   getModel(): MongooseModel<Document & Model> {
-    return this.mainDbModel;
+    return this.mainDbModel as unknown as MongooseModel<Document & Model>;
+  }
+
+  /**
+   * Get the native MongoDB Collection, bypassing all Mongoose plugins.
+   *
+   * **WARNING:** Native driver access bypasses Tenant-Isolation, Audit-Fields,
+   * RoleGuard, and Password-Hashing plugins. Only use this when Mongoose
+   * Model methods cannot achieve the goal (e.g., bulk imports, migrations).
+   *
+   * Every call logs a `[SECURITY]` warning with the provided reason.
+   *
+   * @param reason - Mandatory justification for native access (logged for audit trail)
+   * @throws Error if reason is empty
+   *
+   * @example
+   * ```typescript
+   * const col = this.getNativeCollection('Migration: bulk-import historical data without tenant context');
+   * await col.insertMany(legacyDocs);
+   * ```
+   */
+  protected getNativeCollection(reason: string): Collection {
+    this.validateNativeAccessReason(reason, 'getNativeCollection');
+
+    const modelName = (this.mainDbModel as any)?.modelName || 'Unknown';
+    Logger.warn(`[SECURITY] Native collection access: ${reason} (Model: ${modelName})`, this.constructor.name);
+
+    return (this.mainDbModel as unknown as MongooseModel<Document & Model>).collection;
+  }
+
+  /**
+   * Get the Mongoose Connection (which provides access to the native MongoDB Db and MongoClient).
+   *
+   * **WARNING:** Via `connection.db` you get the native MongoDB Db instance,
+   * and via `connection.getClient()` the native MongoClient. Both bypass ALL
+   * Mongoose plugins (Tenant-Isolation, Audit-Fields, RoleGuard, Password-Hashing).
+   *
+   * Every call logs a `[SECURITY]` warning with the provided reason.
+   *
+   * @param reason - Mandatory justification (min 20 chars) for native access (logged for audit trail)
+   * @throws Error if reason is empty or too short
+   *
+   * @example
+   * ```typescript
+   * // Read-only cross-collection count (no Mongoose schema for target collection)
+   * const conn = this.getNativeConnection('Statistics: count chatmessages across all tenants');
+   * const count = await conn.db.collection('chatmessages').countDocuments({ ... });
+   * ```
+   */
+  protected getNativeConnection(reason: string): Connection {
+    this.validateNativeAccessReason(reason, 'getNativeConnection');
+
+    const modelName = (this.mainDbModel as any)?.modelName || 'Unknown';
+    Logger.warn(`[SECURITY] Native connection access: ${reason} (Model: ${modelName})`, this.constructor.name);
+
+    return (this.mainDbModel as unknown as MongooseModel<Document & Model>).db;
+  }
+
+  private validateNativeAccessReason(reason: string, method: string): void {
+    const trimmed = reason?.trim();
+    if (!trimmed || trimmed.length < 20) {
+      throw new Error(`${method} requires a meaningful reason (min 20 chars) — explain why native access is needed`);
+    }
   }
 
   /**