aiden-shared-calculations-unified 1.0.0 → 1.0.2

package/README.MD

@@ -0,0 +1,78 @@
+# Unified Calculations Package (`aiden-shared-calculations-unified`)
+
+**Version:** 1.0.0
+
+## Overview
+
+This package centralizes all data calculation logic for the BullTrackers project. It provides a standardized structure for calculations run by the unified Computation System and includes shared utility functions. Calculations are dynamically loaded and categorized.
+
+## Package Structure
+
+### `/utils`
+
+Shared utility functions used by calculations or the systems consuming them.
+
+* `firestore_utils.js`: Provides a resilient `withRetry` wrapper for Firestore operations using exponential backoff.
+* `sector_mapping_provider.js`: Provides functions (`loadInstrumentMappings`, `getInstrumentSectorMap`) to fetch and cache instrument-to-ticker and instrument-to-sector mappings from Firestore.
+
+### `/calculations`
+
+Contains the core calculation logic, organized into subdirectories representing categories.
+
+* **Calculation Class Structure:** Each `.js` file defines a class responsible for a specific metric. Every class **must** implement:
+    * `constructor()`: Initializes any internal state needed for aggregation.
+    * `process(portfolioData, userId, context)` OR `process(todayPortfolio, yesterdayPortfolio, userId)`: Processes a single user's data. The signature depends on whether the calculation requires historical comparison. `context` provides shared data like mappings.
+    * `getResult()`: Returns the final, calculated result for the aggregation period. **Crucially, this method must perform any final averaging (e.g., sum/count) itself.** It should return the final value or object ready for storage, not raw components.
+    * `reset()`: (Optional but recommended) Resets the internal state, often used by the calling system between processing batches or days.
+
+* **Categories (Examples based on your files):**
+    * `/asset_metrics`: Calculations focused on individual assets (e.g., `asset_dollar_metrics`, `asset_position_size`).
+    * `/behavioural`: Calculations analyzing user trading patterns (e.g., `drawdown_response`, `gain_response`, `paper_vs_diamond_hands`, `position_count_pnl`, `smart_money_flow`).
+    * `/pnl`: Profit and Loss related calculations (e.g., `asset_pnl_status`, `average_daily_pnl_all_users`, `average_daily_pnl_per_sector`, `average_daily_pnl_per_stock`, `average_daily_position_pnl`, `pnl_distribution_per_stock`, `profitability_migration`, `profitability_ratio_per_stock`, `profitability_skew_per_stock`, `user_profitability_tracker`).
+    * `/sanity`: Basic checks and counts (e.g., `users_processed`).
+    * `/sectors`: Calculations aggregated by market sector (e.g., `diversification_pnl`, `sector_dollar_metrics`, `sector_rotation`, `total_long_per_sector`, `total_short_per_sector`).
+    * `/sentiment`: Calculations related to market sentiment (e.g., `crowd_conviction_score`).
+    * `/short_and_long_stats`: Specific counts for short and long positions (e.g., `long_position_per_stock`, `sentiment_per_stock`, `short_position_per_stock`, `total_long_figures`, `total_short_figures`).
+    * `/speculators`: Calculations **specifically** for the 'speculator' user type, often involving leverage, stop-loss, or take-profit data (e.g., `distance_to_stop_loss_per_leverage`, `distance_to_tp_per_leverage`, `entry_distance_to_sl_per_leverage`, `entry_distance_to_tp_per_leverage`, `holding_duration_per_asset`, `leverage_per_asset`, `leverage_per_sector`, `risk_appetite_change`, `risk_reward_ratio_per_asset`, `speculator_asset_sentiment`, `speculator_danger_zone`, `stop_loss_distance_by_sector_short_long_breakdown`, `stop_loss_distance_by_ticker_short_long_breakdown`, `stop_loss_per_asset`, `take_profit_per_asset`, `tsl_effectiveness`, `tsl_per_asset`).
+
+* **Output Formats:** Calculations should adhere to the standardized output formats defined in `docs/Notes/output_formats.md`.
+
+## Usage
+
+This package is intended to be consumed primarily by the unified Computation System.
+
+```javascript
+// Example usage within Computation System
+const { calculations, utils } = require('aiden-shared-calculations-unified');
+
+const CalculationClass = calculations.pnl.average_daily_pnl_per_stock;
+const calculator = new CalculationClass();
+
+// ... load data ...
+
+// In a loop for each user:
+calculator.process(portfolioData, userId, context);
+
+// After processing all users:
+const results = await calculator.getResult();
+
+// ... store results ...
+````
+
+## Contributing
+
+*(Outline the process for adding new calculations)*
+
+1.  **Determine Category:** Decide which subdirectory (`/calculations/<category>`) the new metric belongs to. Create a new category if necessary.
+2.  **Create File:** Create a new `.js` file using kebab-case (e.g., `my-new-metric.js`).
+3.  **Implement Class:** Define the calculation class, ensuring it has `constructor`, `process`, and `getResult` methods adhering to the standards. Remember `getResult` must return the *final* computed value.
+4.  **Add to Manifest:** The main `index.js` uses `require-all`, so the new calculation should be automatically included in the exports upon the next package build/publish, assuming the file naming and structure are correct.
+5.  **Publish:** Bump the package version (`npm version patch` or minor/major as appropriate) and publish (`npm publish --access public`).
+6.  **Update Consumer:** Update the version dependency in the Computation System's `package.json`.
+
+<!-- end list -->
+
+```
+
+---
+```

package/calculations/capital_flow/deposit_withdrawal_percentage.js

@@ -0,0 +1,71 @@
+/**
+ * @fileoverview Estimates the average percentage deposited into accounts between two days.
+ */
+
+class DepositPercentage {
+    constructor() {
+        this.totalDepositPercentage = 0;
+        this.userCount = 0;
+    }
+
+    /**
+     * Calculates total PnL from aggregated or public positions.
+     * @param {object} portfolio - Portfolio data object.
+     * @returns {number|null} Total PnL or null if positions are missing.
+     */
+    calculateTotalPnl(portfolio) {
+        // Use the same logic as in ProfitabilityMigration
+        if (portfolio && portfolio.AggregatedPositions) {
+            return portfolio.AggregatedPositions.reduce((sum, pos) => sum + (pos.NetProfit || 0), 0);
+        } else if (portfolio && portfolio.PublicPositions) {
+            // PublicPositions might lack NetProfit, adjust if necessary based on your data
+            return portfolio.PublicPositions.reduce((sum, pos) => sum + (pos.NetProfit || 0), 0);
+        }
+        return null;
+    }
+
+    process(todayPortfolio, yesterdayPortfolio, userId) {
+        if (!todayPortfolio || !yesterdayPortfolio || !yesterdayPortfolio.PortfolioValue || yesterdayPortfolio.PortfolioValue === 0) {
+            // Need both portfolios and a non-zero yesterday value for calculation
+            return;
+        }
+
+        const valueChange = (todayPortfolio.PortfolioValue || 0) - yesterdayPortfolio.PortfolioValue;
+
+        const todayPnl = this.calculateTotalPnl(todayPortfolio);
+        const yesterdayPnl = this.calculateTotalPnl(yesterdayPortfolio);
+
+        // If PnL can't be calculated for either day, we can't reliably estimate cash flow
+        if (todayPnl === null || yesterdayPnl === null) {
+            return;
+        }
+
+        const pnlChange = todayPnl - yesterdayPnl;
+        const cashFlow = valueChange - pnlChange;
+
+        // Check for deposit (positive cash flow)
+        if (cashFlow > 0) {
+            const depositPercentage = (cashFlow / yesterdayPortfolio.PortfolioValue) * 100;
+            this.totalDepositPercentage += depositPercentage;
+            this.userCount++;
+        }
+    }
+
+    getResult() {
+        if (this.userCount === 0) {
+            return {}; // Return empty object if no users contributed
+        }
+
+        return {
+            // Calculate the final average directly
+            average_deposit_percentage: this.totalDepositPercentage / this.userCount
+        };
+    }
+
+    reset() {
+        this.totalDepositPercentage = 0;
+        this.userCount = 0;
+    }
+}
+
+module.exports = DepositPercentage;

package/calculations/capital_flow/new_allocation_percentage.js

@@ -0,0 +1,49 @@
+/**
+ * @fileoverview Calculates the average percentage of total portfolio value
+ * newly allocated to assets not held on the previous day.
+ */
+
+class NewAllocationPercentage {
+    constructor() {
+        this.accumulatedNewAllocationPercentage = 0;
+        this.userCount = 0;
+    }
+
+    process(todayPortfolio, yesterdayPortfolio, userId) {
+        const todayPositions = todayPortfolio?.AggregatedPositions;
+        const yesterdayPositions = yesterdayPortfolio?.AggregatedPositions || [];
+
+        if (!todayPositions || todayPositions.length === 0) return;
+
+        const yesterdayIds = new Set(yesterdayPositions.map(p => p.InstrumentID));
+        let userNewAllocation = 0;
+
+        for (const pos of todayPositions) {
+            if (!yesterdayIds.has(pos.InstrumentID)) {
+                const invested = typeof pos.Invested === 'number' ? pos.Invested : 0;
+                userNewAllocation += invested;
+            }
+        }
+
+        // Guard against data rounding or eToro API drift
+        if (userNewAllocation > 100) userNewAllocation = 100;
+
+        this.accumulatedNewAllocationPercentage += userNewAllocation;
+        this.userCount++;
+    }
+
+    getResult() {
+        if (this.userCount === 0) return {};
+        return {
+            average_new_allocation_percentage:
+                this.accumulatedNewAllocationPercentage / this.userCount
+        };
+    }
+
+    reset() {
+        this.accumulatedNewAllocationPercentage = 0;
+        this.userCount = 0;
+    }
+}
+
+module.exports = NewAllocationPercentage;

package/calculations/capital_flow/reallocation_increase_percentage.js

@@ -0,0 +1,63 @@
+/**
+ * @fileoverview Calculates the average percentage increase in allocation
+ * specifically towards assets already held on the previous day.
+ */
+
+class ReallocationIncreasePercentage {
+    constructor() {
+        this.accumulatedIncreasePercentage = 0;
+        this.userCount = 0;
+    }
+
+    process(todayPortfolio, yesterdayPortfolio, userId) {
+        if (!todayPortfolio || !yesterdayPortfolio || !todayPortfolio.AggregatedPositions || !yesterdayPortfolio.AggregatedPositions) {
+            // Requires AggregatedPositions which contain the 'Invested' percentage
+            return;
+        }
+
+        const yesterdayPositions = new Map(yesterdayPortfolio.AggregatedPositions.map(p => [p.InstrumentID, p]));
+        let userTotalIncreasePercentage = 0;
+
+        for (const todayPos of todayPortfolio.AggregatedPositions) {
+            const yesterdayPos = yesterdayPositions.get(todayPos.InstrumentID);
+
+            // Check if the asset was held yesterday
+            if (yesterdayPos) {
+                // Ensure 'Invested' property exists and is a number
+                const todayInvested = typeof todayPos.Invested === 'number' ? todayPos.Invested : 0;
+                const yesterdayInvested = typeof yesterdayPos.Invested === 'number' ? yesterdayPos.Invested : 0;
+
+                const deltaInvested = todayInvested - yesterdayInvested;
+
+                // Accumulate only the increases
+                if (deltaInvested > 0) {
+                    userTotalIncreasePercentage += deltaInvested;
+                }
+            }
+        }
+
+        // Only count users who had positions on both days for this metric
+        if (yesterdayPortfolio.AggregatedPositions.length > 0 && todayPortfolio.AggregatedPositions.length > 0) {
+           this.accumulatedIncreasePercentage += userTotalIncreasePercentage;
+           this.userCount++;
+        }
+    }
+
+    getResult() {
+        if (this.userCount === 0) {
+            return {};
+        }
+
+        return {
+             // Calculate the final average directly
+            average_reallocation_increase_percentage: this.accumulatedIncreasePercentage / this.userCount
+        };
+    }
+
+    reset() {
+        this.accumulatedIncreasePercentage = 0;
+        this.userCount = 0;
+    }
+}
+
+module.exports = ReallocationIncreasePercentage;

package/calculations/insights/daily_bought_vs_sold_count.js

@@ -0,0 +1,56 @@
+/**
+ * @fileoverview Calculates the total number of positions 'bought' vs 'sold' based on daily owner delta.
+ * Uses the 'daily_instrument_insights' collection (truth of source).
+ * 'Bought' = sum of positive deltas, 'Sold' = sum of absolute negative deltas.
+ */
+
+class DailyBoughtVsSoldCount {
+    constructor() {
+        this.totalBought = 0;
+        this.totalSold = 0;
+        this.processedDay = false; // Flag to run logic only once per day
+    }
+
+    // Needs insights from today and yesterday
+    async process(todayPortfolio, yesterdayPortfolio, userId, context, todayInsights, yesterdayInsights) {
+        if (this.processedDay) return;
+        this.processedDay = true;
+
+        if (!todayInsights || !todayInsights.insights || !yesterdayInsights || !yesterdayInsights.insights) {
+            console.warn('[DailyBoughtVsSoldCount] Missing insights data for today or yesterday.');
+            return;
+        }
+
+        const yesterdayTotals = new Map(yesterdayInsights.insights.map(i => [i.instrumentId, i.total]));
+
+        for (const instrument of todayInsights.insights) {
+            const instrumentId = instrument.instrumentId;
+            const todayTotal = instrument.total || 0;
+            const yesterdayTotal = yesterdayTotals.get(instrumentId) || 0;
+
+            const delta = todayTotal - yesterdayTotal;
+
+            if (delta > 0) {
+                this.totalBought += delta;
+            } else if (delta < 0) {
+                this.totalSold += Math.abs(delta); // Sum of absolute decreases
+            }
+        }
+    }
+
+    getResult() {
+        // Return the final aggregated counts
+        return {
+            total_positions_bought_delta: this.totalBought,
+            total_positions_sold_delta: this.totalSold
+        };
+    }
+
+    reset() {
+        this.totalBought = 0;
+        this.totalSold = 0;
+        this.processedDay = false;
+    }
+}
+
+module.exports = DailyBoughtVsSoldCount;

package/calculations/insights/daily_buy_sell_sentiment_count.js

@@ -0,0 +1,50 @@
+/**
+ * @fileoverview Calculates the total number of 'buy' vs 'sell' (short) positions across all instruments.
+ * Uses the 'daily_instrument_insights' collection (truth of source).
+ */
+
+class DailyBuySellSentimentCount {
+    constructor() {
+        this.totalBuyPositions = 0;
+        this.totalSellPositions = 0;
+        this.processedDay = false; // Flag to run logic only once per day
+    }
+
+    // Only needs today's insights
+    async process(todayPortfolio, yesterdayPortfolio, userId, context, todayInsights, yesterdayInsights) {
+        if (this.processedDay) return;
+        this.processedDay = true;
+
+        if (!todayInsights || !todayInsights.insights) {
+            console.warn('[DailyBuySellSentimentCount] Missing insights data for today.');
+            return;
+        }
+
+        for (const instrument of todayInsights.insights) {
+            const totalOwners = instrument.total || 0;
+            if (totalOwners > 0) {
+                const buyPercent = (instrument.buy || 0) / 100;
+                const sellPercent = (instrument.sell || 0) / 100; // 'sell' means short here
+
+                this.totalBuyPositions += (buyPercent * totalOwners);
+                this.totalSellPositions += (sellPercent * totalOwners);
+            }
+        }
+    }
+
+    getResult() {
+        // Return the final rounded sums
+        return {
+            total_buy_positions: Math.round(this.totalBuyPositions),
+            total_sell_positions: Math.round(this.totalSellPositions) // 'sell' are shorts
+        };
+    }
+
+    reset() {
+        this.totalBuyPositions = 0;
+        this.totalSellPositions = 0;
+        this.processedDay = false;
+    }
+}
+
+module.exports = DailyBuySellSentimentCount;

package/calculations/insights/daily_ownership_delta.js

@@ -0,0 +1,56 @@
+/**
+ * @fileoverview Calculates the daily change (delta) in total owners for each instrument.
+ * Uses the 'daily_instrument_insights' collection (truth of source).
+ */
+const { loadInstrumentMappings } = require('../../utils/sector_mapping_provider');
+
+class DailyOwnershipDelta {
+    constructor() {
+        this.deltaByTicker = {};
+        this.mappings = null;
+    }
+
+    // This calculation only needs insights data, not portfolio data.
+    // It runs daily but needs 'yesterday's insights to calculate the delta.
+    async process(todayPortfolio, yesterdayPortfolio, userId, context, todayInsights, yesterdayInsights) {
+        // We only need to run the core logic once per day, not per user.
+        // Use a flag to ensure it runs only on the first user processed for the day.
+        if (this.processedDay) return;
+        this.processedDay = true; // Set flag
+
+        if (!todayInsights || !todayInsights.insights || !yesterdayInsights || !yesterdayInsights.insights) {
+            console.warn('[DailyOwnershipDelta] Missing insights data for today or yesterday.');
+            return;
+        }
+
+        if (!this.mappings) {
+            this.mappings = await loadInstrumentMappings();
+        }
+
+        const yesterdayTotals = new Map(yesterdayInsights.insights.map(i => [i.instrumentId, i.total]));
+
+        for (const instrument of todayInsights.insights) {
+            const instrumentId = instrument.instrumentId;
+            const todayTotal = instrument.total || 0;
+            const yesterdayTotal = yesterdayTotals.get(instrumentId) || 0; // Default to 0 if not found yesterday
+
+            const delta = todayTotal - yesterdayTotal;
+            const ticker = this.mappings.instrumentToTicker[instrumentId] || `id_${instrumentId}`;
+
+            this.deltaByTicker[ticker] = delta;
+        }
+    }
+
+    getResult() {
+        // Return the final calculated delta for each ticker
+        return this.deltaByTicker;
+    }
+
+    reset() {
+        this.deltaByTicker = {};
+        this.processedDay = false; // Reset the flag for the next day
+        this.mappings = null;
+    }
+}
+
+module.exports = DailyOwnershipDelta;

package/calculations/insights/daily_total_positions_held.js

@@ -0,0 +1,40 @@
+/**
+ * @fileoverview Calculates the total number of positions held across all instruments for the day.
+ * Uses the 'daily_instrument_insights' collection (truth of source).
+ */
+
+class DailyTotalPositionsHeld {
+    constructor() {
+        this.totalPositions = 0;
+        this.processedDay = false; // Flag to run logic only once per day
+    }
+
+    // Only needs today's insights
+    async process(todayPortfolio, yesterdayPortfolio, userId, context, todayInsights, yesterdayInsights) {
+        if (this.processedDay) return;
+        this.processedDay = true;
+
+        if (!todayInsights || !todayInsights.insights) {
+            console.warn('[DailyTotalPositionsHeld] Missing insights data for today.');
+            return;
+        }
+
+        for (const instrument of todayInsights.insights) {
+            this.totalPositions += (instrument.total || 0);
+        }
+    }
+
+    getResult() {
+        // Return the final sum
+        return {
+            total_positions_held: this.totalPositions
+        };
+    }
+
+    reset() {
+        this.totalPositions = 0;
+        this.processedDay = false;
+    }
+}
+
+module.exports = DailyTotalPositionsHeld;

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "aiden-shared-calculations-unified",
-  "version": "1.0.0",
+  "version": "1.0.2",
   "description": "Shared calculation modules for the BullTrackers Computation System.",
   "main": "index.js",
   "files": [