bulltrackers-module 1.0.504 → 1.0.506

package/functions/generic-api/user-api/ADDING_LEGACY_ROUTES_GUIDE.md

@@ -0,0 +1,345 @@
+# Adding New Legacy Routes for Auto-Migration
+
+## How Easy Is It?
+
+**Answer: Very Easy!** Adding a new legacy route requires updating **just 1 location** in most cases!
+
+**Even Easier:** The system now automatically reads legacy paths from the collection registry, so you typically only need to add one line to the registry!
+
+---
+
+## Quick Overview
+
+The auto-migration system works by:
+1. **Trying new path first** - Reads from the new CID-based path
+2. **Falling back to legacy** - If not found, tries legacy path(s) from registry
+3. **Auto-migrating** - If found in legacy, automatically migrates to new path
+
+To add a new legacy route, you typically only need to:
+1. **`collection_registry.js`** - Add to `legacyPaths` array (that's it!)
+
+**Only if you need complex conditional logic:**
+2. **`path_resolution_helpers.js`** - Add to `legacyPathMap` object (fallback)
+
+---
+
+## Step-by-Step Guide
+
+### ✅ EASIEST WAY: Just Add to Collection Registry (Recommended)
+
+**File:** `Backend/Entrypoints/BullTrackers/Backend/Core/config/collection_registry.js`
+
+**Location:** Find your collection definition and add to `legacyPaths` array
+
+**Example:**
+```javascript
+notifications: {
+    path: 'SignedInUsers/{cid}/notifications',
+    description: 'User notifications',
+    dynamicSegments: ['{cid}'],
+    status: 'active',
+    legacyPaths: [
+        'user_notifications/{firebaseUid}/notifications',  // Existing
+        'user_notifications/{userCid}/notifications',        // Existing
+        'old_notifications/{userId}/items'                   // NEW - Just add here!
+    ]
+}
+```
+
+**That's it!** The system automatically uses the first legacy path from the registry. No other changes needed!
+
+**How it works:**
+- `readWithMigration()` automatically passes `category` and `subcategory` to `getLegacyPath()`
+- `getLegacyPath()` first tries to get legacy paths from the registry
+- If found, it automatically resolves dynamic segments and returns the path
+- No need to update `path_resolution_helpers.js`!
+
+---
+
+### Alternative: Add to Legacy Path Map (For Complex Logic)
+
+**Only needed if:** You need conditional logic or special handling that can't be expressed in a simple path template.
+
+**File:** `Backend/Entrypoints/BullTrackers/Backend/Core/bulltrackers-module/functions/generic-api/user-api/helpers/core/path_resolution_helpers.js`
+
+**Location:** Inside `getLegacyPath()` function, add to `legacyPathMap` object
+
+**Example:**
+```javascript
+function getLegacyPath(dataType, userCid, config = {}, params = {}) {
+    // ... registry lookup happens first ...
+    
+    // Fallback to hardcoded map (only if registry doesn't have it)
+    const legacyPathMap = {
+        // ... existing mappings ...
+        
+        // NEW - Add your new data type here (only if you need complex logic)
+        newDataType: params.someCondition 
+            ? `old_collection/${userId}/items/${params.someParam}`
+            : `old_collection/${userId}/items`
+    };
+    
+    return legacyPathMap[dataType] || null;
+}
+```
+
+**Note:** This is only needed for complex conditional logic. For simple paths, just use the registry!
+
+---
+
+## Real-World Example
+
+Let's say you want to add support for a legacy "user preferences" collection:
+
+### Step 1: Update Collection Registry
+
+```javascript
+// In collection_registry.js
+signedInUsers: {
+    // ... other collections ...
+    
+    preferences: {
+        path: 'SignedInUsers/{cid}/preferences/data',
+        description: 'User preferences (single document)',
+        dynamicSegments: ['{cid}'],
+        status: 'active',
+        legacyPaths: [
+            'user_preferences/{userCid}',           // Add this
+            'old_prefs/{userId}/settings'           // And this if multiple legacy formats exist
+        ]
+    }
+}
+```
+
+### Step 2: (Optional) Update Path Resolution Helper
+
+**Only needed if:** You need conditional logic that can't be expressed in a simple path template.
+
+```javascript
+// In path_resolution_helpers.js
+// This is only needed for complex conditional logic
+// The registry lookup happens first, so this is just a fallback
+```
+
+### Step 3: Use It in Your API
+
+```javascript
+// In your API helper file
+const { readWithMigration } = require('./helpers/core/path_resolution_helpers');
+
+async function getUserPreferences(req, res, dependencies, config) {
+    const { db, logger } = dependencies;
+    const { userCid } = req.query;
+    
+    const result = await readWithMigration(
+        db,
+        'signedInUsers',      // category
+        'preferences',        // subcategory
+        { cid: userCid },     // params
+        {
+            isCollection: false,  // It's a document
+            dataType: 'preferences',  // This triggers legacy path lookup
+            config: config,
+            logger: logger
+        }
+    );
+    
+    if (!result) {
+        return res.status(404).json({ error: "Preferences not found" });
+    }
+    
+    // result.data contains the preferences
+    // result.source tells you if it came from 'new' or 'legacy'
+    // result.migrated tells you if it was auto-migrated
+    
+    return res.status(200).json({
+        preferences: result.data,
+        source: result.source,
+        migrated: result.migrated || false
+    });
+}
+```
+
+**That's it!** The auto-migration happens automatically.
+
+---
+
+## Advanced: Multiple Legacy Formats
+
+If you have multiple legacy formats for the same data type:
+
+### Option 1: Try Multiple Legacy Paths (Recommended)
+
+Update `getLegacyPath()` to return an array or try multiple paths:
+
+```javascript
+function getLegacyPath(dataType, userCid, config = {}, params = {}) {
+    const cid = String(userCid);
+    
+    const legacyPathMap = {
+        // ... existing ...
+        
+        preferences: [
+            `user_preferences/${cid}`,           // Try first
+            `old_prefs/${cid}/settings`,        // Try second
+            `legacy_preferences/${cid}/data`     // Try third
+        ]
+    };
+    
+    const path = legacyPathMap[dataType];
+    return Array.isArray(path) ? path[0] : path;  // Return first for now
+}
+```
+
+Then update `readWithMigration()` to try all paths in the array (this would require a small code change).
+
+### Option 2: Use Conditional Logic
+
+```javascript
+const legacyPathMap = {
+    preferences: params.useOldFormat 
+        ? `old_prefs/${userId}/settings`
+        : `user_preferences/${cid}`
+};
+```
+
+---
+
+## Dynamic Segment Resolution
+
+The system automatically resolves common dynamic segments:
+
+- `{cid}` → User CID
+- `{userCid}` → User CID
+- `{firebaseUid}` → Firebase UID (from params)
+- `{username}` → Username (from params)
+- `{date}` → Date (from params)
+- `{requestId}` → Request ID (from params or documentId)
+- `{piCid}` → PI CID (from params)
+- `{postId}` → Post ID (from params or documentId)
+- `{watchlistId}` → Watchlist ID (from params)
+- `{version}` → Version (from params)
+- `{viewId}` → View ID (from params or documentId)
+- `{reviewId}` → Review ID (from params or documentId)
+- `{timestamp}` → Timestamp (from params or auto-generated)
+
+**Just use these placeholders in your legacy path string!**
+
+---
+
+## Testing Your New Legacy Route
+
+1. **Create test data in legacy path:**
+   ```javascript
+   await db.collection('user_preferences').doc('29312236').set({
+       theme: 'dark',
+       notifications: true
+   });
+   ```
+
+2. **Call your API:**
+   ```javascript
+   GET /user/me/preferences?userCid=29312236
+   ```
+
+3. **Check the response:**
+   ```json
+   {
+       "preferences": { "theme": "dark", "notifications": true },
+       "source": "legacy",
+       "migrated": true
+   }
+   ```
+
+4. **Verify migration:**
+   ```javascript
+   const newDoc = await db.collection('SignedInUsers/29312236/preferences/data').get();
+   // Should exist with _migratedAt and _migratedFrom fields
+   ```
+
+---
+
+## Common Patterns
+
+### Pattern 1: Simple CID-based Legacy Path
+```javascript
+legacyPaths: ['old_collection/{userCid}']
+legacyPathMap: { newDataType: `old_collection/${cid}` }
+```
+
+### Pattern 2: Firebase UID-based Legacy Path
+```javascript
+legacyPaths: ['old_collection/{firebaseUid}']
+legacyPathMap: { 
+    newDataType: firebaseUid 
+        ? `old_collection/${firebaseUid}` 
+        : `old_collection/${cid}` 
+}
+```
+
+### Pattern 3: Nested Legacy Path
+```javascript
+legacyPaths: ['old_collection/{userCid}/subcollection/{itemId}']
+legacyPathMap: { 
+    newDataType: `old_collection/${cid}/subcollection/${params.itemId || '{itemId}'}` 
+}
+```
+
+### Pattern 4: Date-based Legacy Path
+```javascript
+legacyPaths: ['old_collection/{date}/{userCid}']
+legacyPathMap: { 
+    newDataType: `old_collection/${params.date || '{date}'}/${cid}` 
+}
+```
+
+---
+
+## Troubleshooting
+
+### Legacy Path Not Found?
+
+1. **Check `dataType` parameter** - Must match key in `legacyPathMap`
+2. **Check `params`** - Ensure all required dynamic segments are provided
+3. **Check path resolution** - Use logger to see what path is being tried
+
+### Migration Not Happening?
+
+1. **Check if new path exists** - Migration only happens if new path is available
+2. **Check logger** - Look for migration success/error messages
+3. **Check permissions** - Ensure write permissions to new path
+
+### Multiple Legacy Formats?
+
+1. **Add all to `legacyPaths` array** in registry
+2. **Use conditional logic** in `getLegacyPath()` to choose which to try first
+3. **Or modify `readWithMigration()`** to try multiple legacy paths sequentially
+
+---
+
+## Summary
+
+**Adding a new legacy route is as simple as:**
+
+### ✅ EASIEST (Recommended):
+1. ✅ Add one line to `legacyPaths` array in `collection_registry.js`
+2. ✅ Use `readWithMigration()` in your API code (pass `category` and `subcategory`)
+
+**Total time: ~30 seconds per legacy route!**
+
+### Alternative (For Complex Logic):
+1. ✅ Add one line to `legacyPaths` array in `collection_registry.js` (optional, for documentation)
+2. ✅ Add one line to `legacyPathMap` in `path_resolution_helpers.js` (if you need conditional logic)
+3. ✅ Use `readWithMigration()` in your API code
+
+**Total time: ~2 minutes per legacy route!**
+
+The system handles:
+- ✅ Path resolution
+- ✅ Fallback logic
+- ✅ Auto-migration
+- ✅ Error handling
+- ✅ Logging
+
+You just need to define the paths! 🎉
+

package/functions/generic-api/user-api/CODE_REORGANIZATION_PLAN.md

@@ -0,0 +1,320 @@
+# User API Code Reorganization Plan
+
+## Current State Analysis
+
+### File Sizes and Complexity
+- **data_helpers.js**: 2573 lines, 29 functions - **TOO LARGE**
+- **review_helpers.js**: 545 lines, 4 functions - OK
+- **watchlist_helpers.js**: 697 lines, 8 functions - OK
+- **on_demand_fetch_helpers.js**: 554 lines, 3 functions - OK
+- **user_sync_helpers.js**: 593 lines, 2 functions - OK
+- **alert_helpers.js**: 355 lines, 7 functions - OK
+- **subscription_helpers.js**: 328 lines, 5 functions - OK
+- **verification_helpers.js**: 231 lines, 2 functions - OK
+- **notification_helpers.js**: 131 lines, 3 functions - OK
+- **dev_helpers.js**: 316 lines, 6 functions - OK
+- **collection_helpers.js**: 216 lines, 8 functions - OK
+
+### Functions in data_helpers.js (29 total)
+1. `tryDecompress` - Utility
+2. `findLatestPortfolioDate` - Data lookup utility
+3. `findLatestComputationDate` - Data lookup utility
+4. `checkIfUserIsPI` - User status check
+5. `findLatestRankingsDate` - Data lookup utility
+6. `findLatestPiPortfolioDate` - Data lookup utility
+7. `findLatestPiHistoryDate` - Data lookup utility
+8. `getPiAnalytics` - PI analytics endpoint
+9. `getUserRecommendations` - User recommendations endpoint
+10. `getUserDataStatus` - User data status endpoint
+11. `getWatchlist` - Watchlist endpoint (legacy)
+12. `updateWatchlist` - Watchlist endpoint (legacy)
+13. `getUserPortfolio` - User portfolio endpoint
+14. `getUserSocialPosts` - User social posts endpoint
+15. `getInstrumentMappings` - Instrument mappings endpoint
+16. `getUserVerification` - User verification endpoint
+17. `getUserComputations` - User computations endpoint
+18. `autoGenerateWatchlist` - Watchlist generation endpoint
+19. `searchPopularInvestors` - PI search endpoint
+20. `requestPiAddition` - PI request endpoint
+21. `getWatchlistTriggerCounts` - Watchlist analytics endpoint
+22. `checkPisInRankings` - PI rankings check endpoint
+23. `checkPiInComputationDate` - Computation check utility
+24. `getPiProfile` - PI profile endpoint
+25. `checkIfUserIsPopularInvestor` - User status endpoint
+26. `trackProfileView` - Profile view tracking endpoint
+27. `generateSamplePIPersonalizedMetrics` - Sample data generator
+28. `getSignedInUserPIPersonalizedMetrics` - Personalized metrics endpoint
+
+## Proposed Reorganization
+
+### New File Structure
+
+```
+user-api/
+├── index.js (router - unchanged)
+├── helpers/
+│   ├── core/
+│   │   ├── compression_helpers.js (tryDecompress)
+│   │   ├── data_lookup_helpers.js (findLatest* functions)
+│   │   ├── user_status_helpers.js (checkIfUserIsPI, checkIfUserIsPopularInvestor)
+│   │   └── path_resolution_helpers.js (enhanced collection_helpers)
+│   ├── data/
+│   │   ├── portfolio_helpers.js (getUserPortfolio, getUserDataStatus)
+│   │   ├── social_helpers.js (getUserSocialPosts)
+│   │   ├── computation_helpers.js (getUserComputations, checkPiInComputationDate)
+│   │   └── instrument_helpers.js (getInstrumentMappings)
+│   ├── profile/
+│   │   ├── pi_profile_helpers.js (getPiProfile, getPiAnalytics)
+│   │   ├── user_profile_helpers.js (getUserVerification, checkIfUserIsPopularInvestor)
+│   │   └── profile_view_helpers.js (trackProfileView)
+│   ├── watchlist/
+│   │   ├── watchlist_data_helpers.js (getWatchlist, updateWatchlist - legacy)
+│   │   ├── watchlist_generation_helpers.js (autoGenerateWatchlist)
+│   │   └── watchlist_analytics_helpers.js (getWatchlistTriggerCounts)
+│   ├── recommendations/
+│   │   └── recommendation_helpers.js (getUserRecommendations)
+│   ├── search/
+│   │   ├── pi_search_helpers.js (searchPopularInvestors)
+│   │   └── pi_request_helpers.js (requestPiAddition, checkPisInRankings)
+│   ├── metrics/
+│   │   └── personalized_metrics_helpers.js (getSignedInUserPIPersonalizedMetrics, generateSamplePIPersonalizedMetrics)
+│   ├── notifications/
+│   │   └── notification_helpers.js (unchanged)
+│   ├── alerts/
+│   │   └── alert_helpers.js (unchanged)
+│   ├── watchlists/
+│   │   └── watchlist_helpers.js (unchanged)
+│   ├── subscriptions/
+│   │   └── subscription_helpers.js (unchanged)
+│   ├── verification/
+│   │   └── verification_helpers.js (unchanged)
+│   ├── reviews/
+│   │   └── review_helpers.js (unchanged)
+│   ├── sync/
+│   │   └── user_sync_helpers.js (unchanged)
+│   ├── fetch/
+│   │   └── on_demand_fetch_helpers.js (unchanged)
+│   ├── dev/
+│   │   └── dev_helpers.js (unchanged)
+│   └── collection_helpers.js (keep for backward compatibility, re-export from core/)
+```
+
+## Detailed Breakdown
+
+### 1. core/compression_helpers.js
+**Purpose:** Compression/decompression utilities
+**Functions:**
+- `tryDecompress(data)` - Decompress computation results
+
+**Dependencies:** `zlib`
+
+### 2. core/data_lookup_helpers.js
+**Purpose:** Find latest available data dates
+**Functions:**
+- `findLatestPortfolioDate(db, signedInUsersCollection, userCid, maxDaysBack)`
+- `findLatestComputationDate(db, insightsCollection, resultsSub, compsSub, category, computationName, userCid, maxDaysBack)`
+- `findLatestRankingsDate(db, rankingsCollection, maxDaysBack)`
+- `findLatestPiPortfolioDate(db, piPortfoliosCollection, userCid, maxDaysBack)`
+- `findLatestPiHistoryDate(db, piHistoryCollection, userCid, maxDaysBack)`
+
+**Dependencies:** `@google-cloud/firestore`
+
+### 3. core/user_status_helpers.js
+**Purpose:** Check user status (PI, signed-in, etc.)
+**Functions:**
+- `checkIfUserIsPI(db, userCid, config, logger)`
+- `checkIfUserIsPopularInvestor(req, res, dependencies, config)` (endpoint handler)
+
+**Dependencies:** `dev_helpers`, `data_lookup_helpers`
+
+### 4. core/path_resolution_helpers.js
+**Purpose:** Enhanced path resolution with migration support
+**Functions:**
+- `getCidFromFirebaseUid(db, firebaseUid)` - from collection_helpers
+- `getCollectionPath(collectionRegistry, category, subcategory, params)` - enhanced
+- `readWithMigration(db, newPath, legacyPath, options)` - new, with auto-migration
+- `writeDual(db, newPath, legacyPath, data, options)` - enhanced
+- `migrateUserData(db, userCid, dataType, logger)` - new, batch migration
+
+**Dependencies:** `collection_registry`, `@google-cloud/firestore`
+
+### 5. data/portfolio_helpers.js
+**Purpose:** User portfolio data endpoints
+**Functions:**
+- `getUserPortfolio(req, res, dependencies, config)`
+- `getUserDataStatus(req, res, dependencies, config)`
+
+**Dependencies:** `core/data_lookup_helpers`, `core/path_resolution_helpers`, `core/compression_helpers`
+
+### 6. data/social_helpers.js
+**Purpose:** User social posts endpoints
+**Functions:**
+- `getUserSocialPosts(req, res, dependencies, config)`
+
+**Dependencies:** `core/data_lookup_helpers`, `core/path_resolution_helpers`, `core/compression_helpers`
+
+### 7. data/computation_helpers.js
+**Purpose:** Computation result endpoints
+**Functions:**
+- `getUserComputations(req, res, dependencies, config)`
+- `checkPiInComputationDate(db, insightsCollection, resultsSub, compsSub, category, computationName, dateStr, cidStr, logger)`
+
+**Dependencies:** `core/data_lookup_helpers`, `core/compression_helpers`, `core/path_resolution_helpers`
+
+### 8. data/instrument_helpers.js
+**Purpose:** Instrument mappings endpoint
+**Functions:**
+- `getInstrumentMappings(req, res, dependencies, config)`
+
+**Dependencies:** `@google-cloud/firestore`
+
+### 9. profile/pi_profile_helpers.js
+**Purpose:** Popular Investor profile endpoints
+**Functions:**
+- `getPiProfile(req, res, dependencies, config)`
+- `getPiAnalytics(req, res, dependencies, config)`
+
+**Dependencies:** `core/data_lookup_helpers`, `core/compression_helpers`, `review_helpers`, `data/computation_helpers`
+
+### 10. profile/user_profile_helpers.js
+**Purpose:** Signed-in user profile endpoints
+**Functions:**
+- `getUserVerification(req, res, dependencies, config)`
+- `checkIfUserIsPopularInvestor(req, res, dependencies, config)` - moved from data_helpers
+
+**Dependencies:** `core/user_status_helpers`, `verification_helpers`
+
+### 11. profile/profile_view_helpers.js
+**Purpose:** Profile view tracking
+**Functions:**
+- `trackProfileView(req, res, dependencies, config)`
+
+**Dependencies:** `@google-cloud/firestore`
+
+### 12. watchlist/watchlist_data_helpers.js
+**Purpose:** Legacy watchlist endpoints (for backward compatibility)
+**Functions:**
+- `getWatchlist(req, res, dependencies, config)` - legacy single watchlist
+- `updateWatchlist(req, res, dependencies, config)` - legacy single watchlist
+
+**Dependencies:** `watchlist_helpers` (new multi-watchlist system)
+
+### 13. watchlist/watchlist_generation_helpers.js
+**Purpose:** Watchlist auto-generation
+**Functions:**
+- `autoGenerateWatchlist(req, res, dependencies, config)`
+
+**Dependencies:** `core/data_lookup_helpers`, `core/compression_helpers`, `watchlist_helpers`
+
+### 14. watchlist/watchlist_analytics_helpers.js
+**Purpose:** Watchlist analytics
+**Functions:**
+- `getWatchlistTriggerCounts(req, res, dependencies, config)`
+
+**Dependencies:** `core/data_lookup_helpers`, `core/compression_helpers`, `watchlist_helpers`
+
+### 15. recommendations/recommendation_helpers.js
+**Purpose:** User recommendations
+**Functions:**
+- `getUserRecommendations(req, res, dependencies, config, type)`
+
+**Dependencies:** `@google-cloud/firestore`
+
+### 16. search/pi_search_helpers.js
+**Purpose:** Popular Investor search
+**Functions:**
+- `searchPopularInvestors(req, res, dependencies, config)`
+
+**Dependencies:** `core/data_lookup_helpers`, `@google-cloud/firestore`
+
+### 17. search/pi_request_helpers.js
+**Purpose:** PI addition requests and rankings checks
+**Functions:**
+- `requestPiAddition(req, res, dependencies, config)`
+- `checkPisInRankings(req, res, dependencies, config)`
+
+**Dependencies:** `core/data_lookup_helpers`, `@google-cloud/firestore`
+
+### 18. metrics/personalized_metrics_helpers.js
+**Purpose:** Personalized metrics for signed-in users
+**Functions:**
+- `getSignedInUserPIPersonalizedMetrics(req, res, dependencies, config)`
+- `generateSamplePIPersonalizedMetrics(userCid, rankEntry)` - internal helper
+
+**Dependencies:** `core/data_lookup_helpers`, `core/compression_helpers`, `core/user_status_helpers`, `review_helpers`
+
+## Migration Strategy
+
+### Phase 1: Create Core Helpers
+1. Create `helpers/core/` directory
+2. Extract compression utilities → `compression_helpers.js`
+3. Extract data lookup utilities → `data_lookup_helpers.js`
+4. Extract user status checks → `user_status_helpers.js`
+5. Enhance path resolution → `path_resolution_helpers.js`
+
+### Phase 2: Create Data Helpers
+1. Create `helpers/data/` directory
+2. Extract portfolio functions → `portfolio_helpers.js`
+3. Extract social functions → `social_helpers.js`
+4. Extract computation functions → `computation_helpers.js`
+5. Extract instrument functions → `instrument_helpers.js`
+
+### Phase 3: Create Profile Helpers
+1. Create `helpers/profile/` directory
+2. Extract PI profile functions → `pi_profile_helpers.js`
+3. Extract user profile functions → `user_profile_helpers.js`
+4. Extract profile view functions → `profile_view_helpers.js`
+
+### Phase 4: Create Feature-Specific Helpers
+1. Create `helpers/watchlist/` directory
+2. Create `helpers/recommendations/` directory
+3. Create `helpers/search/` directory
+4. Create `helpers/metrics/` directory
+5. Extract respective functions
+
+### Phase 5: Update Imports
+1. Update `index.js` to import from new locations
+2. Update all helper files to use new core utilities
+3. Keep `collection_helpers.js` for backward compatibility (re-export from core)
+
+### Phase 6: Cleanup
+1. Delete old `data_helpers.js`
+2. Update any external references
+3. Run tests to ensure everything works
+
+## Benefits
+
+1. **Modularity**: Each file has a clear, single responsibility
+2. **Maintainability**: Easier to find and update specific functionality
+3. **Testability**: Smaller files are easier to test
+4. **Reusability**: Core utilities can be shared across modules
+5. **Readability**: Clear organization makes code easier to understand
+6. **Scalability**: Easy to add new features without bloating existing files
+
+## File Size Targets
+
+- **Core helpers**: 100-300 lines each
+- **Data helpers**: 200-400 lines each
+- **Profile helpers**: 200-500 lines each
+- **Feature helpers**: 200-400 lines each
+- **Total**: Better organized, easier to navigate
+
+## Implementation Order
+
+1. ✅ Create migration plan (done)
+2. ⏳ Create core helpers (compression, data lookup, user status, path resolution)
+3. ⏳ Create data helpers (portfolio, social, computation, instrument)
+4. ⏳ Create profile helpers (PI profile, user profile, profile view)
+5. ⏳ Create feature helpers (watchlist, recommendations, search, metrics)
+6. ⏳ Update index.js imports
+7. ⏳ Update all helper files to use new structure
+8. ⏳ Test all endpoints
+9. ⏳ Remove old data_helpers.js
+
+## Notes
+
+- Keep backward compatibility during migration
+- Use re-exports where needed to avoid breaking changes
+- Update tests as we go
+- Document new structure in README
+