bulltrackers-module 1.0.490 → 1.0.492

package/functions/generic-api/user-api/helpers/FALLBACK_CONDITIONS.md

@@ -0,0 +1,98 @@
+# Profile Page Fallback Conditions
+
+## Overview
+
+This document explains when and why the profile page uses fallback mode instead of computation data.
+
+## Data Storage Locations
+
+### Signed-In User Computations
+- **Category**: `popular-investor` (NOT `signed_in_user`)
+- **Reason**: ResultCommitter ignores category metadata when computation is in a non-core folder
+- **Computation**: `SignedInUserProfileMetrics` is in `calculations/popular-investor/` folder
+- **Path**: `/unified_insights/{date}/results/popular-investor/computations/SignedInUserProfileMetrics`
+
+### Signed-In User Raw Data
+- **Portfolio**: `signed_in_users/{blockId}/snapshots/{date}/parts/part_*`
+- **History**: `signed_in_user_history/{blockId}/snapshots/{date}/parts/part_*` (or `signed_in_users_history` depending on config)
+- **Social**: `signed_in_users_social/{cid}/posts/{postId}`
+
+## Fallback Conditions
+
+### Frontend Fallback Logic (UserProfile.tsx)
+
+The profile page uses fallback mode when:
+
+1. **Computation Not Found**: 
+   - API call to `/user/me/computations?computation=SignedInUserProfileMetrics` returns empty `data: {}`
+   - OR the computation exists but doesn't contain data for the user's CID
+
+2. **API Error**:
+   - Any error fetching the computation (network error, 500, etc.)
+
+3. **Data Structure Issues**:
+   - Computation exists but `metricsResponse.data[latestDate]?.SignedInUserProfileMetrics` is undefined/null
+
+### Backend Fallback Logic (getUserComputations)
+
+The API returns `isFallback: false` but `data: {}` when:
+
+1. **Wrong Category Lookup** (FIXED):
+   - Was looking in `signed_in_user` category
+   - Should look in `popular-investor` category
+
+2. **No Data for User**:
+   - Computation document exists but doesn't contain `data[String(effectiveCid)]`
+   - This can happen if:
+     - Computation ran but user wasn't included (shouldn't happen with targetCid optimization)
+     - Data is sharded and shard doesn't contain user's data
+     - Data exists but in wrong format
+
+3. **Date Mismatch**:
+   - Computation exists for different date than requested
+   - `mode=latest` tries to find latest date, but if none found, returns empty
+
+## Current Issue
+
+The user reported:
+- API returns `"isFallback": false, "data": {}`
+- Profile page shows fallback mode
+- Computation exists at `/unified_insights/2025-12-29/results/popular-investor/computations/SignedInUserProfileMetrics/_shards/shard_0`
+
+**Root Cause**: The `getUserComputations` endpoint was looking in the wrong category (`signed_in_user` instead of `popular-investor`).
+
+**Fix Applied**: Updated `getUserComputations` to look in `popular-investor` category for `SignedInUserProfileMetrics`.
+
+## Verification Steps
+
+1. Check computation exists:
+   ```
+   /unified_insights/2025-12-29/results/popular-investor/computations/SignedInUserProfileMetrics
+   ```
+
+2. Check if data is sharded:
+   - If `_sharded: true`, check `_shards` subcollection
+   - Merge all shards to get full data
+
+3. Check if user's CID exists in data:
+   - Data structure: `{ "29312236": { ...metrics... } }`
+   - Must use `String(cid)` as key
+
+4. Check API response:
+   - Should return `data: { "2025-12-29": { "SignedInUserProfileMetrics": { ... } } }`
+   - If empty, check logs for why lookup failed
+
+## History Data Storage
+
+Signed-in user trade history is stored in:
+- Collection: `signed_in_user_history` (or `signed_in_users_history` depending on config)
+- Path: `{collection}/{blockId}/snapshots/{date}/parts/part_*`
+- Block ID: Typically `19M` (canary block)
+
+The task engine stores history via:
+```javascript
+await batchManager.addToTradingHistoryBatch(String(cid), blockId, today, historyData, 'signed_in_user');
+```
+
+This uses `getHistoryCollection('signed_in_user')` which returns `signedInHistory` collection.
+

package/functions/generic-api/user-api/helpers/HISTORY_STORAGE_LOCATION.md

@@ -0,0 +1,66 @@
+# Signed-In User History Storage Location
+
+## Current Issue
+
+History data is being stored in the **wrong collection** because `FIRESTORE_COLLECTION_SIGNED_IN_HISTORY` is not defined in the task engine config, causing it to fall back to `NormalUserTradeHistory`.
+
+## Storage Path Structure
+
+For signed-in users, history data should be stored at:
+```
+{signed_in_user_history}/{blockId}/snapshots/{date}/parts/part_{shardIndex}
+```
+
+Where:
+- `signed_in_user_history` = Collection name (default: `signed_in_user_history`)
+- `blockId` = `'19M'` (canary block ID)
+- `date` = Date string in format `YYYY-MM-DD` (e.g., `2025-12-29`)
+- `shardIndex` = `cid % TOTAL_SHARDS` (modulo sharding for even distribution)
+
+## Example Path
+
+For user CID `29312236` on date `2025-12-29`:
+```
+signed_in_user_history/19M/snapshots/2025-12-29/parts/part_{shardIndex}
+```
+
+Where `shardIndex = 29312236 % TOTAL_SHARDS` (typically 7 shards, so `29312236 % 7 = 1`)
+
+## Fix Applied
+
+Added `FIRESTORE_COLLECTION_SIGNED_IN_HISTORY` to `taskEngine_config.js`:
+```javascript
+FIRESTORE_COLLECTION_SIGNED_IN_HISTORY: getEnvVar('FIRESTORE_COLLECTION_SIGNED_IN_HISTORY', 'signed_in_user_history'),
+```
+
+## Before Fix
+
+Without this config, the batch manager was using:
+- `config.FIRESTORE_COLLECTION_SIGNED_IN_HISTORY` = `undefined`
+- Fallback: `config.FIRESTORE_COLLECTION_NORMAL_HISTORY` = `'NormalUserTradeHistory'`
+
+So data was being stored at:
+```
+NormalUserTradeHistory/19M/snapshots/2025-12-29/parts/part_{shardIndex}
+```
+
+## After Fix
+
+With the config added, data will be stored at:
+```
+signed_in_user_history/19M/snapshots/2025-12-29/parts/part_{shardIndex}
+```
+
+## Verification
+
+To verify history data exists for a user:
+1. Calculate shard index: `userCid % 7` (or whatever TOTAL_SHARDS is)
+2. Check path: `signed_in_user_history/19M/snapshots/2025-12-29/parts/part_{shardIndex}`
+3. Look for user's CID as a key in the document: `{ "29312236": { ...historyData... } }`
+
+## Related Collections
+
+- **Portfolio**: `signed_in_users/19M/snapshots/{date}/parts/part_{shardIndex}`
+- **History**: `signed_in_user_history/19M/snapshots/{date}/parts/part_{shardIndex}` (FIXED)
+- **Social**: `signed_in_users_social/{cid}/posts/{postId}`
+

package/functions/generic-api/user-api/helpers/data_helpers.js

@@ -870,8 +870,10 @@ async function getUserComputations(req, res, dependencies, config) {
         const resultsSub = config.resultsSubcollection || 'results';
         const compsSub = config.computationsSubcollection || 'computations';
         
-        // For signed-in users, computations are stored in the 'signed_in_user' category
-        const category = 'signed_in_user';
+        // NOTE: ResultCommitter ignores category metadata if computation is in a non-core folder
+        // SignedInUserProfileMetrics is in popular-investor folder, so it's stored in popular-investor category
+        // For signed-in users, computations are stored in the 'popular-investor' category (not 'signed_in_user')
+        const category = 'popular-investor';
         const today = new Date().toISOString().split('T')[0];
         
         const computationNames = computation ? computation.split(',') : [];
@@ -889,6 +891,7 @@ async function getUserComputations(req, res, dependencies, config) {
         
         // If mode is 'latest', try to find the latest available date for the first computation
         // Use effectiveCid for computation lookup
+        // CRITICAL: Only look back 7 days as per requirements
         if (mode === 'latest') {
             const firstCompName = computationNames[0];
             const latestDate = await findLatestComputationDate(
@@ -899,7 +902,7 @@ async function getUserComputations(req, res, dependencies, config) {
                 category, 
                 firstCompName, 
                 effectiveCid, 
-                30
+                7 // Only look back 7 days as per requirements
             );
             
             if (latestDate) {
@@ -908,14 +911,15 @@ async function getUserComputations(req, res, dependencies, config) {
                     logger.log('INFO', `[getUserComputations] Using fallback date ${latestDate} for effective CID ${effectiveCid} (today: ${today})`);
                 }
             } else {
-                // No data found, return empty
+                // No data found after 7 days - return empty (frontend will use fallback)
+                logger.log('WARN', `[getUserComputations] No computation data found for CID ${effectiveCid} in last 7 days. Frontend will use fallback.`);
                 return res.status(200).json({
                     status: 'success',
                     userCid: String(effectiveCid),
                     mode,
                     computations: computationNames,
                     data: {},
-                    isFallback: false,
+                    isFallback: true, // Mark as fallback since no data found
                     requestedDate: today,
                     isImpersonating: isImpersonating || false,
                     actualCid: Number(userCid)
@@ -953,10 +957,38 @@ async function getUserComputations(req, res, dependencies, config) {
                     if (doc.exists) {
                         // Decompress if needed (handles byte string storage)
                         const rawData = doc.data();
-                        const data = tryDecompress(rawData);
+                        let data = tryDecompress(rawData);
+                        
+                        // Handle string decompression result
+                        if (typeof data === 'string') {
+                            try {
+                                data = JSON.parse(data);
+                            } catch (e) {
+                                logger.log('WARN', `[getUserComputations] Failed to parse decompressed string for ${compName} on ${date}:`, e.message);
+                                data = null;
+                            }
+                        }
+                        
+                        // Check if data is sharded
+                        if (data && data._sharded === true && data._shardCount) {
+                            // Data is stored in shards - read all shards and merge
+                            const shardsCol = docRef.collection('_shards');
+                            const shardsSnapshot = await shardsCol.get();
+                            
+                            if (!shardsSnapshot.empty) {
+                                data = {};
+                                for (const shardDoc of shardsSnapshot.docs) {
+                                    const shardData = shardDoc.data();
+                                    Object.assign(data, shardData);
+                                }
+                            } else {
+                                data = null; // Sharded but no shards found
+                            }
+                        }
+                        
                         // Filter by user CID - computation results are stored as { cid: result }
                         // Use effectiveCid for lookup
-                        let userResult = data[String(effectiveCid)];
+                        let userResult = data && typeof data === 'object' ? data[String(effectiveCid)] : null;
                         
                         // Apply dev override for computations that include copied PIs (use actual userCid for override check)
                         if (isDevOverrideActive && (compName === 'SignedInUserProfileMetrics' || compName === 'SignedInUserCopiedPIs')) {

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "bulltrackers-module",
-  "version": "1.0.490",
+  "version": "1.0.492",
   "description": "Helper Functions for Bulltrackers.",
   "main": "index.js",
   "files": [