bulltrackers-module 1.0.591 → 1.0.592

package/functions/generic-api/user-api/COMPLETE_REFACTORING_PLAN.md

@@ -1,116 +0,0 @@
-# Complete Refactoring Plan
-
-## Overview
-This document outlines the complete refactoring of `data_helpers.js` (2550 lines) into organized, modular helper files with full migration support.
-
-## Function Mapping
-
-### ✅ Already Migrated
-- `getUserPortfolio` → `data/portfolio_helpers.js`
-- `getUserDataStatus` → `data/portfolio_helpers.js`
-- `getUserSocialPosts` → `data/social_helpers.js`
-- `getUserComputations` → `data/computation_helpers.js`
-- `checkPiInComputationDate` → `data/computation_helpers.js`
-- `getInstrumentMappings` → `data/instrument_helpers.js`
-- Core utilities → `core/` directory
-
-### 🔄 To Be Migrated
-
-#### Profile Helpers (`profile/`)
-1. **`profile/pi_profile_helpers.js`**
-   - `getPiProfile` - PI profile endpoint
-   - `getPiAnalytics` - PI analytics endpoint
-
-2. **`profile/user_profile_helpers.js`**
-   - `getUserVerification` - User verification endpoint (needs migration to use `SignedInUsers/{cid}/verification`)
-   - `checkIfUserIsPopularInvestor` - Check if user is PI
-
-3. **`profile/profile_view_helpers.js`**
-   - `trackProfileView` - Profile view tracking (needs migration to use `PopularInvestors/{piCid}/profileViews` and `PopularInvestors/{piCid}/views`)
-
-#### Watchlist Helpers (`watchlist/`)
-4. **`watchlist/watchlist_data_helpers.js`** (Legacy)
-   - `getWatchlist` - Legacy single watchlist endpoint
-   - `updateWatchlist` - Legacy single watchlist endpoint
-
-5. **`watchlist/watchlist_generation_helpers.js`**
-   - `autoGenerateWatchlist` - Auto-generate watchlist from copied PIs
-
-6. **`watchlist/watchlist_analytics_helpers.js`**
-   - `getWatchlistTriggerCounts` - Watchlist analytics
-
-#### Search & Recommendations (`search/`, `recommendations/`)
-7. **`search/pi_search_helpers.js`**
-   - `searchPopularInvestors` - PI search endpoint
-
-8. **`search/pi_request_helpers.js`**
-   - `requestPiAddition` - Request PI addition
-   - `checkPisInRankings` - Check PIs in rankings
-
-9. **`recommendations/recommendation_helpers.js`**
-   - `getUserRecommendations` - User recommendations (hedges/similar)
-
-#### Metrics (`metrics/`)
-10. **`metrics/personalized_metrics_helpers.js`**
-    - `getSignedInUserPIPersonalizedMetrics` - Personalized metrics endpoint
-    - `generateSamplePIPersonalizedMetrics` - Sample data generator (internal)
-
-## Legacy Path Mappings
-
-### Signed-In User Data
-| New Path | Legacy Path | Migration Type |
-|----------|------------|----------------|
-| `SignedInUsers/{cid}/notifications` | `user_notifications/{firebaseUid}/notifications` | Auto-migrate on read |
-| `SignedInUsers/{cid}/watchlists` | `user_watchlists/{cid}/lists` | Auto-migrate on read |
-| `SignedInUsers/{cid}/verification/data` | `user_verifications/{username}` | Auto-migrate on read |
-| `SignedInUsers/{cid}/syncRequests` | `user_sync_requests/{cid}/requests` | Auto-migrate on read |
-| `SignedInUsers/{cid}/syncStatus/latest` | `user_sync_requests/{cid}/global/latest` | Auto-migrate on read |
-| `SignedInUsers/{cid}/portfolio/latest` | `{signedInUsersCollection}/19M/snapshots/{date}/parts` | Read from root data, write to user-centric |
-| `SignedInUsers/{cid}/posts` | `signed_in_users_social/{cid}/posts` | Auto-migrate on read |
-
-### Popular Investor Data
-| New Path | Legacy Path | Migration Type |
-|----------|------------|----------------|
-| `PopularInvestors/{piCid}/fetchRequests/{requestId}` | `pi_fetch_requests/{piCid}/requests/{requestId}` | Auto-migrate on read |
-| `PopularInvestors/{piCid}/fetchStatus/latest` | `pi_fetch_requests/{piCid}/global/latest` | Auto-migrate on read |
-| `PopularInvestors/{piCid}/userFetchRequests/{userCid}` | `pi_fetch_requests/{piCid}/user_requests/{userCid}` | Auto-migrate on read |
-| `PopularInvestors/{piCid}/reviews/{reviewId}` | `pi_reviews/{piCid}_{userCid}` | Auto-migrate on read |
-| `PopularInvestors/{piCid}/posts/{postId}` | `pi_social_posts/{piCid}/posts/{postId}` | Auto-migrate on read |
-| `PopularInvestors/{piCid}/profileViews/{date}` | `profile_views/{piCid}_{date}` | Auto-migrate on read |
-| `PopularInvestors/{piCid}/views/{viewId}` | `profile_views/individual_views/views/{piCid}_{viewerCid}_{timestamp}` | Auto-migrate on read |
-
-### Root Data (No Migration - Task Engine Writes Here)
-- `SignedInUserPortfolioData/{date}/{cid}`
-- `SignedInUserTradeHistoryData/{date}/{cid}`
-- `SignedInUserSocialPostData/{date}/{cid}`
-- `PopularInvestorPortfolioData/{date}/{cid}`
-- `PopularInvestorTradeHistoryData/{date}/{cid}`
-- `PopularInvestorSocialPostData/{date}/{cid}`
-
-## Implementation Steps
-
-1. ✅ Update `path_resolution_helpers.js` with all legacy path mappings
-2. ✅ Update `collection_registry.js` with correct new paths
-3. ⏳ Create profile helper files
-4. ⏳ Create watchlist helper files
-5. ⏳ Create search/recommendations helper files
-6. ⏳ Create metrics helper files
-7. ⏳ Update all functions to use `readWithMigration` and `writeWithMigration`
-8. ⏳ Update `data_helpers.js` to only re-export
-9. ⏳ Test all endpoints
-
-## Migration Strategy
-
-All functions should:
-1. Use `readWithMigration()` to read data (tries new path, falls back to legacy, auto-migrates)
-2. Use `writeWithMigration()` to write data (writes to both paths during transition)
-3. Use `getNewPath()` from registry for new paths
-4. Use `getLegacyPath()` for legacy path resolution
-
-## Notes
-
-- Root data collections remain unchanged (populated by task engines)
-- Public watchlists remain in `public_watchlists` (not user-centric)
-- System collections like `user_gcid_mappings` remain unchanged
-- Firebase UID → CID mapping stays in `signedInUsers/{firebaseUid}` main document
-

package/functions/generic-api/user-api/FIRESTORE_PATHS_INVENTORY.md

@@ -1,171 +0,0 @@
-# Firestore Paths Inventory - User API
-
-## Complete List of All Firestore Paths Used in User-API
-
-### User-Centric Data (To Be Migrated)
-
-#### 1. Notifications
-- **Legacy:** `user_notifications/{userCid}/notifications/{notificationId}`
-- **Legacy:** `user_notifications/{userCid}/counters/{date}`
-- **Legacy:** `user_notifications/{firebaseUid}/notifications/{notificationId}` (test_alert_helpers - needs CID conversion)
-- **New (CID-only):** `SignedInUsers/{cid}/notifications/{notificationId}`
-- **New (CID-only):** `SignedInUsers/{cid}/notifications/counters/{date}` (if needed)
-
-#### 2. Alerts
-- **Legacy:** `user_alerts/{userCid}/alerts/{alertId}`
-- **Legacy:** `user_alerts/{userCid}/counters/{date}`
-- **New:** `SignedInUsers/{cid}/alerts/{alertId}`
-- **New:** `SignedInUsers/{cid}/alerts/counters/{date}` (if needed)
-
-#### 3. Watchlists (Private)
-- **Legacy:** `watchlists/{userCid}/lists/{watchlistId}`
-- **New:** `SignedInUsers/{cid}/watchlists/{watchlistId}`
-
-#### 4. Watchlist Subscriptions
-- **Legacy:** `watchlist_subscriptions/{userCid}/alerts/{piCid}`
-- **New (per-watchlist):** `SignedInUsers/{cid}/watchlists/{watchlistId}/subscriptions/{piCid}`
-- **New (global):** `SignedInUsers/{cid}/subscriptions/{piCid}`
-
-#### 5. Verification
-- **Legacy (pending):** `{verificationsCollection}/{username}` (before CID is known, 2 segments)
-- **New (CID-based):** `SignedInUsers/{cid}/verification/data` (4 segments: collection/document/collection/document)
-- **Migration:** After `finalizeVerification` gets `realCID` from bio check API, migrate to CID-based path
-- **Firebase UID:** Store `firebaseUid` in verification document once Firebase account is created
-- **Note:** Using subcollection with single document named "data" to comply with Firestore even-segment rule
-
-#### 6. User Sync Requests
-- **Legacy:** `user_sync_requests/{userCid}/requests/{requestId}`
-- **Legacy:** `user_sync_requests/{userCid}/global/latest`
-- **New:** `SignedInUsers/{cid}/syncRequests/{requestId}`
-- **New:** `SignedInUsers/{cid}/syncStatus/latest`
-
-#### 7. Portfolio Data (Latest Snapshot)
-- **Legacy:** `{signedInUsersCollection}/{blockId}/snapshots/{date}/parts/part_{N}`
-- **Root Data:** `SignedInUserPortfolioData/{date}/{cid}/{cid}` (4 segments - task engine format, document ID is CID)
-- **New (CID-only):** `SignedInUsers/{cid}/portfolio/latest` (4 segments: collection/document/collection/document)
-
-#### 8. Trade History (Latest Snapshot)
-- **Legacy:** `{signedInHistoryCollection}/{blockId}/snapshots/{date}/parts/part_{N}`
-- **Root Data:** `SignedInUserTradeHistoryData/{date}/{cid}/{cid}` (4 segments - task engine format, document ID is CID)
-- **New (CID-only):** `SignedInUsers/{cid}/tradeHistory/latest` (4 segments: collection/document/collection/document)
-
-#### 9. Social Posts (Latest Snapshot)
-- **Legacy:** `signed_in_users_social/{cid}/posts/{postId}` (4 segments)
-- **Root Data:** `SignedInUserSocialPostData/{date}/{cid}/{cid}` (4 segments - task engine format, document ID is CID)
-- **New (CID-only):** `SignedInUsers/{cid}/posts/{postId}` (4 segments: collection/document/collection/document)
-
-#### 10. Signed-In User Main Document
-- **Firebase Auth Mapping:** `signedInUsers/{firebaseUid}` (unchanged - ONLY for UID→CID lookup)
-- **Legacy:** `{signedInUsersCollection}/{cid}`
-- **Legacy:** `signed_in_users/{cid}`
-
-**Note:** `signedInUsers/{firebaseUid}` is ONLY for Firebase Auth UID → eToro CID mapping. All user data should use CID-based paths.
-
-### Public/System Collections (No Migration Needed)
-
-#### 11. Public Watchlists
-- `public_watchlists/{watchlistId}` (unchanged)
-- `public_watchlists/{watchlistId}/versions/{version}` (unchanged)
-
-#### 12. PI Fetch Requests (PI-Centric, Not User-Centric)
-- `pi_fetch_requests/{piCid}/requests/{requestId}` (unchanged - PI-specific)
-- `pi_fetch_requests/{piCid}/user_requests/{userCid}` (unchanged - rate limiting)
-- `pi_fetch_requests/{piCid}/global/latest` (unchanged - PI-specific)
-
-#### 13. Reviews (PI-Centric, Not User-Centric)
-- `{reviewsCollection}/{reviewId}` (unchanged - reviews of PIs)
-- Should be: `popularInvestors/{piCid}/reviews/{reviewId}` (per registry)
-
-#### 14. Dev Overrides (System Configuration)
-- `dev_overrides/{etoroCid}` (unchanged - system-level)
-
-#### 15. Computation Results (Read-Only, Date-Based)
-- `unified_insights/{date}/results/{category}/computations/{computationName}`
-- `unified_insights/{date}/results/{category}/computations/{computationName}/_shards/{shardId}`
-
-#### 16. Rankings Data (Read-Only, Date-Based)
-- `popular_investor_rankings/{date}`
-
-## Summary by Migration Priority
-
-### High Priority (User Actions/Creations)
-1. ✅ Notifications - User receives notifications
-2. ✅ Alerts - User receives alerts
-3. ✅ Watchlists - User creates watchlists
-4. ✅ Subscriptions - User subscribes to alerts
-5. ✅ Verification - User verification data
-6. ✅ Sync Requests - User requests syncs
-
-### Medium Priority (User Data Snapshots)
-7. ✅ Portfolio (latest) - Fallback data for profile pages
-8. ✅ Trade History (latest) - Fallback data for profile pages
-9. ✅ Social Posts (latest) - Fallback data for profile pages
-
-### Low Priority (System/Public Data)
-10. ⚠️ Public Watchlists - Public data, not user-centric
-11. ⚠️ PI Fetch Requests - PI-centric, not user-centric
-12. ⚠️ Reviews - PI-centric, not user-centric
-13. ⚠️ Dev Overrides - System configuration
-14. ⚠️ Computation Results - Date-based, not user-centric
-15. ⚠️ Rankings - Date-based, not user-centric
-
-## Files That Need Updates
-
-### High Priority Files
-1. `helpers/notification_helpers.js` - Notifications
-2. `helpers/alert_helpers.js` - Alerts
-3. `helpers/watchlist_helpers.js` - Watchlists
-4. `helpers/subscription_helpers.js` - Subscriptions
-5. `helpers/verification_helpers.js` - Verification
-6. `helpers/user_sync_helpers.js` - Sync Requests
-7. `helpers/data_helpers.js` - Portfolio, History, Social Posts
-
-### Medium Priority Files
-8. `helpers/collection_helpers.js` - Path resolution helpers
-9. `helpers/test_alert_helpers.js` - Test notifications
-
-### Low Priority Files (No Changes Needed)
-10. `helpers/on_demand_fetch_helpers.js` - PI fetch (PI-centric)
-11. `helpers/review_helpers.js` - Reviews (PI-centric)
-12. `helpers/dev_helpers.js` - Dev overrides (system-level)
-
-## Path Resolution Notes
-
-### CID vs Firebase UID
-- **Always use CID** for user data paths (`SignedInUsers/{cid}/...`)
-- **Firebase UID** is ONLY used in `signedInUsers/{firebaseUid}` for UID→CID mapping
-- **Helper function:** `getCidFromFirebaseUid()` - already exists in `collection_helpers.js`
-- **Migration requirement:** Convert Firebase UID to CID before accessing user data
-
-### Collection Name Variations
-- `signedInUsers` vs `SignedInUsers` - case sensitivity matters
-- `signed_in_users` vs `signedInUsers` - legacy variations
-- Config collections: `{signedInUsersCollection}`, `{verificationsCollection}`, etc.
-
-### Block ID Pattern (Legacy)
-- Legacy portfolio/history uses: `{collection}/{blockId}/snapshots/{date}/parts/part_{N}`
-- Block ID is typically `'19M'` (CANARY_BLOCK_ID)
-- New format eliminates block ID structure
-
-## Migration Checklist
-
-- [ ] Create `helpers/path_mapping.js` with path resolution logic
-- [ ] Create `helpers/migration_helpers.js` with auto-migration functions
-- [ ] Update `collection_registry.js` with migration metadata
-- [ ] Update `helpers/notification_helpers.js`
-- [ ] Update `helpers/alert_helpers.js`
-- [ ] Update `helpers/watchlist_helpers.js`
-- [ ] Update `helpers/subscription_helpers.js`
-- [ ] Update `helpers/verification_helpers.js`
-- [ ] Update `helpers/user_sync_helpers.js`
-- [ ] Update `helpers/data_helpers.js` (portfolio, history, social)
-- [ ] Update `helpers/collection_helpers.js` (enhance existing helpers)
-- [ ] Update `helpers/test_alert_helpers.js`
-- [ ] Test dual read/write functionality
-- [ ] Test auto-migration on read
-- [ ] Test legacy fallback
-- [ ] Deploy with feature flag
-- [ ] Monitor migration progress
-- [ ] Remove legacy read fallback (after all users migrated)
-- [ ] Stop dual writes (after verification period)
-