@rws-framework/db 3.7.3 → 3.9.0

package/LOADING_CONTEXT_FIX.md

@@ -0,0 +1,139 @@
+# LoadingContext Implementation - Solution 1 Documentation (Updated)
+
+## Problem Statement
+
+The RWS framework had two related issues in the postLoad mechanism:
+
+### 1. Original Circular Reference Issue
+- **User model postLoad** calls `userGroup.reload(true)` 
+- **UserGroup model postLoad** calls `User.find(owner, { cancelPostLoad: true })`
+- If that User gets loaded elsewhere without `cancelPostLoad: true`, it could trigger an infinite cycle
+
+### 2. Cross-Request Stack Persistence Issue (Discovered)
+- The LoadingContext had a **global loading stack** that persisted across HTTP requests
+- Stack built up and never got cleared between requests
+- Caused false positive "already being loaded" errors for subsequent requests
+- Led to models not loading properly and authentication failures
+
+## Solution: Request-Scoped Loading Context
+
+### Key Fix: Execution Context Isolation
+
+The critical fix was changing from a **global static loading stack** to **per-execution-context stacks**:
+
+**Before (Problematic):**
+```typescript
+class LoadingContext {
+    private static loadingStack: Set<string> = new Set(); // GLOBAL - persisted across requests
+}
+```
+
+**After (Fixed):**
+```typescript
+class LoadingContext {
+    private static executionContexts = new WeakMap<object, Set<string>>(); // Per-context isolation
+    
+    static withNewExecutionContext<T>(fn: () => Promise<T>): Promise<T> {
+        // Creates fresh context for each operation
+    }
+}
+```
+
+### Implementation Components
+
+#### 1. LoadingContext Class (`/models/utils/LoadingContext.ts`) - Updated
+
+**Key Changes:**
+- **Execution Context Isolation**: Each operation gets its own loading stack
+- **WeakMap for Context Storage**: Associates loading stacks with execution contexts
+- **Automatic Context Creation**: `withNewExecutionContext()` creates fresh contexts
+- **Per-Request Isolation**: Each HTTP request gets its own clean loading context
+
+#### 2. FindUtils Integration (`/models/utils/LoadingUtils.ts`) - Critical Fix
+
+**All top-level methods now wrap operations in new execution contexts:**
+
+```typescript
+public static async findOneBy<T extends RWSModel<T>>(
+    opModel: OpModelType<T>,
+    findParams?: FindByType
+): Promise<T | null> {
+    // CRITICAL: Wrap in new execution context to ensure clean loading stack
+    return LoadingContext.withNewExecutionContext(async () => {
+        // ... existing logic with circular reference protection
+    });
+}
+```
+
+### How The Fix Works
+
+#### Before Fix (Problematic):
+```
+Request 1: User.find(11) -> adds "User:11" to global stack
+Request 1: UserGroup.reload() -> adds "UserGroup:10" to global stack  
+Request 1: Completes, but stack NOT cleared
+Request 2: User.find(11) -> "User:11" already in global stack -> FALSE POSITIVE error
+```
+
+#### After Fix (Working):
+```
+Request 1: LoadingContext.withNewExecutionContext() -> creates fresh context
+Request 1: User.find(11) -> adds "User:11" to context-specific stack
+Request 1: UserGroup.reload() -> adds "UserGroup:10" to same context stack
+Request 1: Completes -> context automatically cleaned up
+
+Request 2: LoadingContext.withNewExecutionContext() -> creates NEW fresh context  
+Request 2: User.find(11) -> adds "User:11" to NEW context stack -> works normally
+```
+
+### Request Isolation Benefits
+
+1. **No Cross-Request Interference**: Each HTTP request gets its own loading context
+2. **Automatic Cleanup**: No manual stack management needed
+3. **Proper Circular Detection**: Still prevents infinite loops within single operations
+4. **No False Positives**: Eliminates "already loading" errors between requests
+
+### Configuration
+
+```typescript
+// Set maximum loading depth for new contexts (default: 10)
+LoadingContext.setDefaultMaxDepth(15);
+
+// Get current loading stack for debugging current context
+console.log(LoadingContext.getLoadingStack());
+
+// Manual context creation (usually not needed)
+await LoadingContext.withNewExecutionContext(async () => {
+    // Database operations here get fresh context
+});
+```
+
+### Testing Results
+
+**Before Fix - Error Logs:**
+```
+Circular reference detected: User:11 is already being loaded. Breaking cycle.
+UnauthorizedException: Unauthorized
+Circular reference detected: User:11 is already being loaded. Breaking cycle.
+Circular reference detected: UserGroup:10 is already being loaded. Breaking cycle.
+Circular reference detected: UserGroup:10 is already being loaded. Breaking cycle.
+// Repeated across multiple requests - FALSE POSITIVES
+```
+
+**After Fix:**
+- Clean loading per HTTP request
+- No false positive circular reference detections  
+- Proper circular reference protection within individual operations
+- No authentication failures due to loading issues
+- No cross-request interference
+
+### Backward Compatibility
+
+- **100% backward compatible**: No changes needed to existing model code
+- **Existing `cancelPostLoad` still works**: Provides additional layer of protection
+- **All existing APIs preserved**: No breaking changes to any interfaces
+- **Automatic context management**: Developers don't need to manage contexts manually
+
+### Summary
+
+This fix resolves both the original circular reference issue AND the newly discovered cross-request stack persistence problem. The key insight was that the loading context needs to be **request-scoped** rather than **application-global**. Each database operation chain now gets its own clean loading context, preventing both infinite recursion within operations and false positive detection across operations.

package/LOADING_CONTEXT_IMPLEMENTATION.md

@@ -0,0 +1,122 @@
+# LoadingContext Implementation - Solution 1 Documentation
+
+## Problem Statement
+
+The RWS framework had a potential infinite recursion issue in the postLoad mechanism where:
+
+1. **User model postLoad** calls `userGroup.reload(true)` 
+2. **UserGroup model postLoad** calls `User.find(owner, { cancelPostLoad: true })`
+3. If that User gets loaded elsewhere without `cancelPostLoad: true`, it could trigger an infinite cycle
+
+The existing anti-recursion mechanism was **per-instance** rather than **per-loading-context**, meaning it only prevented the same object from running postLoad multiple times, but didn't prevent circular loading between different instances of related models.
+
+## Solution: Context-Based Loading Stack
+
+### Implementation Components
+
+#### 1. LoadingContext Class (`/models/utils/LoadingContext.ts`)
+
+A utility class that tracks the chain of models currently being loaded to prevent circular references:
+
+**Key Features:**
+- **Static loading stack**: Tracks `ModelType:ID` combinations currently being loaded
+- **Circular reference detection**: Prevents loading a model that's already in the loading chain
+- **Maximum depth protection**: Prevents infinite chains with configurable depth limit
+- **Automatic cleanup**: Uses try/finally blocks to ensure proper cleanup
+- **Debugging support**: Provides stack inspection and clear error messages
+
+**Key Methods:**
+- `isLoading(modelType, id)`: Check if a model is currently being loaded
+- `startLoading(modelType, id)`: Mark a model as being loaded
+- `finishLoading(modelType, id)`: Mark a model as finished loading
+- `withLoadingContext(modelType, id, fn)`: Execute function with automatic context management
+
+#### 2. FindUtils Integration (`/models/utils/FindUtils.ts`)
+
+Modified all find methods to use LoadingContext:
+
+**Changes in `find()`, `findOneBy()`, `findBy()`, and `paginate()`:**
+- Check if model is already being loaded before creating instance
+- Use `LoadingContext.withLoadingContext()` to wrap model loading
+- Return `null` or skip items that are already being loaded
+- Provide warning messages for detected circular references
+
+#### 3. RWSModel Integration (`/models/core/RWSModel.ts`)
+
+- Added LoadingContext import
+- The reload method now uses FindUtils which automatically gets the protection
+
+### How It Works
+
+#### Normal Loading Flow
+```
+1. User.find(1) -> LoadingContext.startLoading('User', 1)
+2. User._asyncFill() -> loads userGroup relation
+3. UserGroup.reload() -> LoadingContext.startLoading('UserGroup', 2)  
+4. UserGroup._asyncFill() -> loads owner relation
+5. User.find(owner) -> LoadingContext.startLoading('User', 3)
+6. Complete normally -> LoadingContext.finishLoading() for each
+```
+
+#### Circular Reference Prevention
+```
+1. User.find(1) -> LoadingContext.startLoading('User', 1)
+2. User._asyncFill() -> loads userGroup relation
+3. UserGroup.reload() -> LoadingContext.startLoading('UserGroup', 2)  
+4. UserGroup._asyncFill() -> tries to load User.find(1) again
+5. LoadingContext.isLoading('User', 1) returns true
+6. Returns null instead of creating infinite loop
+7. Warning logged: "Circular reference detected"
+```
+
+#### Maximum Depth Protection
+```
+If loading chain exceeds configurable limit (default 10):
+- Throws descriptive error with full loading stack
+- Shows exact chain: "Model1:1 -> Model2:2 -> Model3:3 -> ..."
+- Helps identify complex circular dependencies
+```
+
+### Benefits
+
+1. **Complete Circular Reference Prevention**: Unlike the previous per-instance flag, this prevents all types of circular loading
+2. **Transparent Integration**: Existing code doesn't need changes - works automatically through FindUtils
+3. **Performance Optimized**: Minimal overhead - just Set operations for tracking
+4. **Debugging Friendly**: Clear error messages and stack inspection capabilities
+5. **Configurable**: Adjustable maximum depth and behavior
+6. **Fail-Safe**: Always cleans up loading stack even if errors occur
+
+### Configuration Options
+
+```typescript
+// Set maximum loading depth (default: 10)
+LoadingContext.setMaxDepth(15);
+
+// Get current loading stack for debugging
+console.log(LoadingContext.getLoadingStack());
+
+// Clear stack in emergency situations
+LoadingContext.clearStack();
+```
+
+### Backward Compatibility
+
+- **100% backward compatible**: No changes needed to existing model code
+- **Existing `cancelPostLoad` still works**: Provides additional layer of protection
+- **All existing APIs preserved**: No breaking changes to FindUtils or RWSModel
+
+### Testing
+
+A test file (`LoadingContext.test.ts`) demonstrates:
+- Basic loading detection
+- Circular reference prevention  
+- Maximum depth protection
+- Automatic cleanup
+
+### Error Handling
+
+The implementation provides clear error messages:
+- **Circular Reference**: "Circular reference detected: User:1 is already being loaded. Breaking cycle."
+- **Maximum Depth**: "Maximum loading depth (10) exceeded. Possible circular reference detected. Loading stack: User:1 -> UserGroup:2 -> ..."
+
+This solution provides robust protection against infinite recursion while maintaining full backward compatibility and excellent debugging capabilities.

package/dist/models/core/RWSModel.d.ts

@@ -73,6 +73,6 @@ declare class RWSModel<T> implements IModel {
         [k: string]: any;
     }): Promise<number>;
     static getDb(): DBService;
-    reload(): Promise<RWSModel<T> | null>;
+    reload(inPostLoad?: boolean): Promise<RWSModel<T> | null>;
 }
 export { RWSModel };

package/dist/models/core/RWSModel.js

@@ -12,11 +12,7 @@ Object.defineProperty(exports, "__esModule", { value: true });
 exports.RWSModel = void 0;
 const decorators_1 = require("../../decorators");
 const FieldsHelper_1 = require("../../helper/FieldsHelper");
-const RelationUtils_1 = require("../utils/RelationUtils");
-const TimeSeriesUtils_1 = require("../utils/TimeSeriesUtils");
-const ModelUtils_1 = require("../utils/ModelUtils");
-const HydrateUtils_1 = require("../utils/HydrateUtils");
-const FindUtils_1 = require("../utils/FindUtils");
+const utils_1 = require("../utils");
 class RWSModel {
     static services = {};
     id;
@@ -87,13 +83,13 @@ class RWSModel {
         return this;
     }
     async hasRelation(key) {
-        return RelationUtils_1.RelationUtils.hasRelation(this.constructor, key);
+        return utils_1.RelationUtils.hasRelation(this.constructor, key);
     }
     async getRelationKey(key) {
-        return RelationUtils_1.RelationUtils.getRelationKey(this.constructor, key);
+        return utils_1.RelationUtils.getRelationKey(this.constructor, key);
     }
     bindRelation(key, relatedModel) {
-        return RelationUtils_1.RelationUtils.bindRelation(relatedModel);
+        return utils_1.RelationUtils.bindRelation(relatedModel);
     }
     async _asyncFill(data, fullDataMode = false, allowRelations = true, postLoadExecute = true) {
         const collections_to_models = {};
@@ -108,48 +104,49 @@ class RWSModel {
         });
         const seriesHydrationfields = [];
         if (allowRelations) {
-            await HydrateUtils_1.HydrateUtils.hydrateRelations(this, relManyData, relOneData, seriesHydrationfields, fullDataMode, data);
+            await utils_1.HydrateUtils.hydrateRelations(this, relManyData, relOneData, seriesHydrationfields, fullDataMode, data);
         }
         // Process regular fields and time series
-        await HydrateUtils_1.HydrateUtils.hydrateDataFields(this, collections_to_models, relOneData, seriesHydrationfields, fullDataMode, data);
+        await utils_1.HydrateUtils.hydrateDataFields(this, collections_to_models, relOneData, seriesHydrationfields, fullDataMode, data);
         if (!this.isPostLoadExecuted() && postLoadExecute) {
             await this.postLoad();
-            this.setPostLoadExecuted();
         }
         return this;
     }
     getModelScalarFields(model) {
-        return ModelUtils_1.ModelUtils.getModelScalarFields(model);
+        return utils_1.ModelUtils.getModelScalarFields(model);
     }
     async getRelationOneMeta(classFields) {
-        return RelationUtils_1.RelationUtils.getRelationOneMeta(this, classFields);
+        return utils_1.RelationUtils.getRelationOneMeta(this, classFields);
     }
     static async getRelationOneMeta(model, classFields) {
-        return RelationUtils_1.RelationUtils.getRelationOneMeta(model, classFields);
+        return utils_1.RelationUtils.getRelationOneMeta(model, classFields);
     }
     async getRelationManyMeta(classFields) {
-        return RelationUtils_1.RelationUtils.getRelationManyMeta(this, classFields);
+        return utils_1.RelationUtils.getRelationManyMeta(this, classFields);
     }
     static async getRelationManyMeta(model, classFields) {
-        return RelationUtils_1.RelationUtils.getRelationManyMeta(model, classFields);
+        return utils_1.RelationUtils.getRelationManyMeta(model, classFields);
     }
     static async paginate(paginateParams, findParams) {
-        return await FindUtils_1.FindUtils.paginate(this, paginateParams, findParams);
+        return await utils_1.FindUtils.paginate(this, paginateParams, findParams);
     }
     async toMongo() {
         const data = {};
-        const timeSeriesIds = TimeSeriesUtils_1.TimeSeriesUtils.getTimeSeriesModelFields(this);
+        const timeSeriesIds = utils_1.TimeSeriesUtils.getTimeSeriesModelFields(this);
         const timeSeriesHydrationFields = [];
         for (const key in this) {
             if (await this.hasRelation(key)) {
-                data[key] = this.bindRelation(key, this[key]);
-                if (data[key] === null) {
-                    const relationKey = await this.getRelationKey(key);
-                    if (relationKey) {
-                        data[relationKey] = null;
-                        delete data[key];
-                    }
+                if (this[key] === null) {
+                    // For null relations, use disconnect or set to null
+                    data[key] = {
+                        disconnect: true
+                    };
+                }
+                else {
+                    data[key] = this.bindRelation(key, this[key]);
                 }
+                // Don't try to set the foreign key directly anymore
                 continue;
             }
             if (!(await this.isDbVariable(key))) {
@@ -178,10 +175,10 @@ class RWSModel {
     async save() {
         const data = await this.toMongo();
         let updatedModelData = data;
-        const entryExists = await ModelUtils_1.ModelUtils.entryExists(this);
+        const entryExists = await utils_1.ModelUtils.entryExists(this);
         if (entryExists) {
             await this.preUpdate();
-            const pk = ModelUtils_1.ModelUtils.findPrimaryKeyFields(this.constructor);
+            const pk = utils_1.ModelUtils.findPrimaryKeyFields(this.constructor);
             updatedModelData = await this.dbService.update(data, this.getCollection(), pk);
             await this._asyncFill(updatedModelData);
             await this.postUpdate();
@@ -196,12 +193,13 @@ class RWSModel {
         return this;
     }
     static async getModelAnnotations(constructor) {
-        return ModelUtils_1.ModelUtils.getModelAnnotations(constructor);
+        return utils_1.ModelUtils.getModelAnnotations(constructor);
     }
     async preUpdate() {
         return;
     }
     async postLoad() {
+        this.setPostLoadExecuted();
         return;
     }
     async postUpdate() {
@@ -214,19 +212,19 @@ class RWSModel {
         return;
     }
     static isSubclass(constructor, baseClass) {
-        return ModelUtils_1.ModelUtils.isSubclass(constructor, baseClass);
+        return utils_1.ModelUtils.isSubclass(constructor, baseClass);
     }
     hasTimeSeries() {
-        return TimeSeriesUtils_1.TimeSeriesUtils.checkTimeSeries(this.constructor);
+        return utils_1.TimeSeriesUtils.checkTimeSeries(this.constructor);
     }
     static checkTimeSeries(constructor) {
-        return TimeSeriesUtils_1.TimeSeriesUtils.checkTimeSeries(constructor);
+        return utils_1.TimeSeriesUtils.checkTimeSeries(constructor);
     }
     async isDbVariable(variable) {
-        return ModelUtils_1.ModelUtils.checkDbVariable(this.constructor, variable);
+        return utils_1.ModelUtils.checkDbVariable(this.constructor, variable);
     }
     static async checkDbVariable(constructor, variable) {
-        return ModelUtils_1.ModelUtils.checkDbVariable(constructor, variable);
+        return utils_1.ModelUtils.checkDbVariable(constructor, variable);
     }
     sanitizeDBData(data) {
         const dataKeys = Object.keys(data);
@@ -244,13 +242,13 @@ class RWSModel {
         return await this.services.dbService.watchCollection(collection, preRun);
     }
     static async findOneBy(findParams) {
-        return await FindUtils_1.FindUtils.findOneBy(this, findParams);
+        return await utils_1.FindUtils.findOneBy(this, findParams);
     }
     static async find(id, findParams = null) {
-        return await FindUtils_1.FindUtils.find(this, id, findParams);
+        return await utils_1.FindUtils.find(this, id, findParams);
     }
     static async findBy(findParams) {
-        return await FindUtils_1.FindUtils.findBy(this, findParams);
+        return await utils_1.FindUtils.findBy(this, findParams);
     }
     static async delete(conditions) {
         const collection = Reflect.get(this, '_collection');
@@ -277,7 +275,7 @@ class RWSModel {
         return RWSModel.loadModels();
     }
     checkRelDisabled(key) {
-        return RelationUtils_1.RelationUtils.checkRelDisabled(this, key);
+        return utils_1.RelationUtils.checkRelDisabled(this, key);
     }
     static setServices(services) {
         this.allModels = services.configService.get('db_models');
@@ -292,8 +290,8 @@ class RWSModel {
     static getDb() {
         return this.services.dbService;
     }
-    async reload() {
-        const pk = ModelUtils_1.ModelUtils.findPrimaryKeyFields(this.constructor);
+    async reload(inPostLoad = false) {
+        const pk = utils_1.ModelUtils.findPrimaryKeyFields(this.constructor);
         const where = {};
         if (Array.isArray(pk)) {
             for (const pkElem of pk) {
@@ -303,7 +301,7 @@ class RWSModel {
         else {
             where[pk] = this[pk];
         }
-        return await FindUtils_1.FindUtils.findOneBy(this.constructor, { conditions: where });
+        return await utils_1.FindUtils.findOneBy(this.constructor, { conditions: where, cancelPostLoad: inPostLoad });
     }
 }
 exports.RWSModel = RWSModel;

package/dist/models/interfaces/IModel.d.ts

@@ -9,7 +9,7 @@ export interface IModel {
     save: () => Promise<this>;
     getDb: () => DBService;
     getCollection: () => string | null;
-    reload: () => Promise<RWSModel<any>>;
+    reload: (inPostLoad: boolean) => Promise<RWSModel<any>>;
     delete: () => Promise<void>;
     hasTimeSeries: () => boolean;
     _asyncFill: (data: any, fullDataMode?: boolean, allowRelations?: boolean) => Promise<any>;