bulltrackers-module 1.0.105 → 1.0.107

package/README.MD

@@ -1,223 +1,223 @@
-
------
-
-# bulltrackers-module
-
-**Version:** 1.0.53
-
-## Overview
-
-This package encapsulates the core backend logic for the BullTrackers platform. It is designed as a modular, stateless library to be used within a Google Cloud Functions environment.
-
-It promotes a clean, testable, and maintainable architecture by enforcing a clear separation of concerns and using a consistent dependency injection pattern. All business logic is exposed via a single, nested `pipe` object.
-
-## Core Concepts
-
-  * **eToro User IDs (CIDs):** Sequential integers assigned upon registration. This sequential nature allows for "Block Sampling".
-  * **Block Sampling:** The process of randomly sampling users within 1 million ID blocks (e.g., 1M-2M, 2M-3M) to ensure a representative dataset across different user registration cohorts.
-  * **User Types:** The system differentiates between two primary user types:
-      * **`normal`**: Standard users.
-      * **`speculator`**: Users identified by specific portfolio activities, such as holding certain high-risk assets.
-  * **Dependency Injection (DI):** No function in this module initializes its own clients (like Firestore or Pub/Sub) or accesses global configuration. All logic is exposed as stateless functions that receive two arguments: `(config, dependencies)`. This makes the entire module testable and portable.
-
-## Architecture: The `pipe` Object
-
-This module exports a single root object named `pipe`. This object organizes all application logic into distinct, self-contained "pipelines," each corresponding to a major microservice or domain.
-
-```javascript
-// Example of the module's structure
-const { pipe } = require('bulltrackers-module');
-
-// You initialize dependencies ONCE in your main function entry point
-const dependencies = {
-    db: new Firestore(),
-    pubsub: new PubSub(),
-    logger: myLogger,
-    headerManager: new pipe.core.IntelligentHeaderManager(...),
-    proxyManager: new pipe.core.IntelligentProxyManager(...),
-    // ...etc.
-};
-
-// You then pass those dependencies into any pipe function
-async function runDiscovery(config) {
-    await pipe.orchestrator.runDiscoveryOrchestrator(config, dependencies);
-}
-```
-
------
-
-## Module Structure
-
-### `pipe.core`
-
-Contains the fundamental, reusable classes and utilities that power the other pipes.
-
-  * **`IntelligentHeaderManager`**: A class that manages a pool of browser headers, rotating them based on historical success rates to avoid API blocks. Performance is tracked in Firestore.
-  * **`IntelligentProxyManager`**: A class that manages a pool of Google Apps Script proxy URLs. It selects available proxies, handles request fetching, and locks proxies that fail (e.g., due to quota errors).
-  * **`FirestoreBatchManager`**: A utility class used by the Task Engine to manage stateful, asynchronous batch writes to Firestore. It handles sharding portfolio data, batching timestamp updates, and flushing processed speculator IDs.
-  * **`firestoreUtils`**: A collection of stateless helper functions for common Firestore operations, such as resetting proxy locks, getting block capacities, fetching exclusion IDs, and querying users who need updates.
-  * **`pubsubUtils`**: A collection of stateless helper functions for publishing messages to Pub/Sub in batches.
-
-### `pipe.orchestrator`
-
-The "brain" of the system. This pipe is responsible for deciding *what* work needs to be done, but not *how* to do it. It's typically run on a schedule (e.g., via Cloud Scheduler).
-
-  * **`runDiscoveryOrchestrator(config, dependencies)`**: The main entry point for the discovery process. It checks which user blocks are under-capacity, gets candidate CIDs to scan, and dispatches `discover` tasks.
-  * **`runUpdateOrchestrator(config, dependencies)`**: The main entry point for the daily update process. It queries Firestore for all verified users who haven't been updated recently and dispatches `update` tasks.
-  * **Discovery Helpers** (`checkDiscoveryNeed`, `getDiscoveryCandidates`, `dispatchDiscovery`): Sub-pipes that handle the logic for finding underpopulated blocks, generating new CIDs (while respecting exclusion lists), and publishing discovery tasks.
-  * **Update Helpers** (`getUpdateTargets`, `dispatchUpdates`): Sub-pipes that find all `normal` and `speculator` users eligible for an update and publish tasks to the `Dispatcher`.
-
-### `pipe.dispatcher`
-
-The "throttle" of the system. Its sole purpose is to receive a large number of tasks (typically from the `orchestrator`) and slowly release them in small batches to a Pub/Sub topic. This prevents the `taskEngine` from scaling too quickly and incurring high costs.
-
-  * **`handleRequest(message, context, config, dependencies)`**: The main entry point, designed to be triggered by a Pub/Sub message containing an array of `tasks`.
-  * **`dispatchTasksInBatches(tasks, dependencies, config)`**: The core logic that loops through tasks, publishing them in batches with a configurable delay.
-
-### `pipe.taskEngine`
-
-The "factory" or "workhorse" of the system. This pipe is event-driven, triggered by individual Pub/Sub messages from the `orchestrator` or `dispatcher`. It executes the core data-fetching operations.
-
-  * **`handleRequest(message, context, config, dependencies)`**: The main entry point. It parses the Pub/Sub message, identifies the task `type`, and routes it to the correct sub-pipe handler.
-  * **`handleDiscover(task, taskId, dependencies, config)`**: Fetches public ranking data for a batch of CIDs. It filters out private/inactive users, applies heuristics (for speculators), and chains to the `verify` task for any promising candidates.
-  * **`handleVerify(task, taskId, dependencies, config)`**: Fetches the portfolio of a *single* user. It verifies they meet the criteria (e.g., holds a speculator asset) and, if so, saves their data and increments the block count.
-  * **`handleUpdate(task, taskId, dependencies, config)`**: Fetches the latest portfolio for an *existing, verified* user and stores the new data in a sharded Firestore document.
-
-### `pipe.computationSystem`
-
-The "refinery" of the system. This pipe is run on a schedule *after* data collection. It loads all the raw portfolio data, runs a suite of calculations (e.g., risk scores, profit migrations), and saves the results in a separate "unified insights" collection for the API to read.
-
-  * **`runOrchestration(config, dependencies)`**: The main entry point. It identifies which dates need processing (or backfilling), loads all data for those dates, initializes all calculation classes, and streams the data through them.
-  * **`dataLoader`**: A set of helpers for loading the sharded portfolio data from Firestore efficiently.
-  * **`computationUtils`**: A set of helpers for categorizing calculations (historical vs. daily), committing results in chunks, and finding the earliest available data date.
-
-### `pipe.api`
-
-The "tap" that serves the processed data. This pipe is an Express.js application designed to be run as a single Cloud Function, providing a public-facing API for the frontend.
-
-  * **`createApiApp(config, dependencies, unifiedCalculations)`**: The main entry point. It returns a fully configured Express app instance, complete with middleware (CORS, JSON) and all API routes.
-  * **API Helpers** (`buildCalculationMap`, `validateRequest`, `fetchUnifiedData`, etc.): Sub-pipes that handle request validation, fetching data from the "unified insights" collection, and generating debug manifests.
-
-### `pipe.maintenance`
-
-A collection of standalone utility and cleanup functions, typically run on a schedule.
-
-  * **`runSpeculatorCleanup(config, dependencies)`**: De-registers stale speculators who haven't held a qualifying asset for a set grace period, and decrements the block counts.
-  * **`handleInvalidSpeculator(message, context, config, dependencies)`**: An event-driven function that listens for Pub/Sub messages containing CIDs of users found to be private or invalid, adding them to an exclusion list.
-  * **`runFetchInsights(config, dependencies)`**: A scheduled function to fetch general eToro market insights (e.g., buy/sell percentages).
-  * **`runFetchPrices(config, dependencies)`**: A scheduled function to fetch daily closing prices for all instruments.
-
-### `pipe.proxy`
-
-The logic for the Google Apps Script web app that acts as the secure proxy layer. This code is *not* run in the Node.js environment; it's intended to be deployed directly into an Apps Script project.
-
-  * **`handlePost(e)`**: The main `doPost(e)` function for the Apps Script. It parses the request from the `IntelligentProxyManager`, executes it using `UrlFetchApp`, and returns the response.
-
------
-
-## Data Flow
-
-The system operates in four distinct, decoupled stages:
-
-1.  **Stage 1: Discovery (Scheduled)**
-
-      * `pipe.orchestrator.runDiscoveryOrchestrator` runs.
-      * It determines it needs 5,000 new `normal` users.
-      * It creates 50 `discover` tasks (100 CIDs each) and publishes them to the `task-engine` topic.
-      * `pipe.taskEngine` functions spin up, `handleDiscover` finds 200 promising users.
-      * It publishes 200 `verify` tasks back to the `task-engine` topic.
-      * `pipe.taskEngine` functions `handleVerify` these users, and 75 are saved to the database.
-
-2.  **Stage 2: Update (Scheduled)**
-
-      * `pipe.orchestrator.runUpdateOrchestrator` runs.
-      * It finds 50,000 existing users that need an update.
-      * It publishes *one* message containing all 50,000 tasks to the `dispatcher` topic.
-      * `pipe.dispatcher.handleRequest` receives the message.
-      * It loops, publishing 500 tasks at a time to the `task-engine` topic, with a 30-second delay between batches.
-      * `pipe.taskEngine` functions `handleUpdate` the portfolios, scaling gradually.
-
-3.  **Stage 3: Computation (Scheduled)**
-
-      * `pipe.computationSystem.runOrchestration` runs (e.g., at 1:00 AM).
-      * It loads all portfolio data collected in Stages 1 & 2.
-      * It runs all calculations and saves the results to the `unified_insights` collection.
-
-4.  **Stage 4: Serving (On-Demand)**
-
-      * A frontend user loads a chart.
-      * The browser sends a request to the `pipe.api` function.
-      * The API validates the request, reads the *pre-computed* data from `unified_insights`, and returns it.
-
-## Usage
-
-In your Google Cloud Function `index.js` file:
-
-```javascript
-/**
- * @fileoverview Unified Cloud Functions entry (Refactored for Pipe Architecture)
- */
-
-// Import FieldPath here
-const { Firestore, FieldValue, FieldPath } = require('@google-cloud/firestore'); 
-const { PubSub }                     = require('@google-cloud/pubsub');
-const { logger }                     = require('sharedsetup')(__filename);
-const { pipe }                       = require('bulltrackers-module');
-const { calculations }               = require('aiden-shared-calculations-unified');
-const fs                             = require('fs');
-const path                           = require('path');
-
-// --- Initialize Clients ---
-const db      = new Firestore();
-const pubsub  = new PubSub();
-
-// --- Load Configs ---
-const cfg = Object.fromEntries(fs.readdirSync(path.join(__dirname, 'config'))
-      .filter(f => f.endsWith('_config.js'))
-      .map(f => [path.basename(f, '_config.js'), require(path.join(__dirname, 'config', f))])
-);
-
-// --- Instantiate Core Managers ---
-const headerManager = new pipe.core.IntelligentHeaderManager(db, logger, cfg.core);
-const proxyManager  = new pipe.core.IntelligentProxyManager (db, logger, cfg.core);
-const batchManager  = new pipe.core.FirestoreBatchManager   (db, headerManager, logger, cfg.taskEngine);
-
-// --- Master Dependencies ---
-const allDependencies = {
-    db              : db,
-    pubsub          : pubsub,
-    logger          : logger,
-    FieldValue      : FieldValue,
-    FieldPath       : FieldPath,
-    headerManager   : headerManager,
-    proxyManager    : proxyManager,
-    batchManager    : batchManager,
-    firestoreUtils  : pipe.core.firestoreUtils,
-    pubsubUtils     : pipe.core.pubsubUtils
-};
-
-// --- Export Cloud Function Handlers ---
-const handlers = {
-    discoveryOrchestrator        : ()     => pipe.orchestrator     .runDiscoveryOrchestrator (cfg.orchestrator     , allDependencies),
-    updateOrchestrator           : ()     => pipe.orchestrator     .runUpdateOrchestrator    (cfg.orchestrator     , allDependencies),
-    speculatorCleanupOrchestrator: ()     => pipe.maintenance      .runSpeculatorCleanup     (cfg.cleanup          , allDependencies),
-    fetchEtoroInsights           : ()     => pipe.maintenance      .runFetchInsights         (cfg.fetchInsights    , allDependencies),
-    updateClosingPrices          : ()     => pipe.maintenance      .runFetchPrices           (cfg.priceFetcher     , allDependencies),
-    computationSystem            : ()     => pipe.computationSystem.runOrchestration         (cfg.computationSystem, allDependencies),
-
-    taskEngineHandler            : (m, c) => pipe.taskEngine       .handleRequest          (m, c, cfg.taskEngine       , allDependencies),
-    dispatcher                   : (m, c) => pipe.dispatcher       .handleRequest          (m, c, cfg.dispatcher       , allDependencies),
-    invalidSpeculatorHandler     : (m, c) => pipe.maintenance      .handleInvalidSpeculator(m, c, cfg.invalidSpeculator, allDependencies)
-};
-
-Object.entries(handlers).forEach(([name, fn]) => exports[name] = fn);
-
-// --- API Export ---
-exports.genericApiV2 = pipe.api.createApiApp(cfg.genericApiV2, allDependencies, calculations);
-
-// --- Local API Server ---
-if (require.main === module) {
-    const port = process.env.PORT || 8080;
-    exports.genericApiV2.listen(port, () => console.log(`API listening on port ${port}`));
-}
+
+-----
+
+# bulltrackers-module
+
+**Version:** 1.0.53
+
+## Overview
+
+This package encapsulates the core backend logic for the BullTrackers platform. It is designed as a modular, stateless library to be used within a Google Cloud Functions environment.
+
+It promotes a clean, testable, and maintainable architecture by enforcing a clear separation of concerns and using a consistent dependency injection pattern. All business logic is exposed via a single, nested `pipe` object.
+
+## Core Concepts
+
+  * **eToro User IDs (CIDs):** Sequential integers assigned upon registration. This sequential nature allows for "Block Sampling".
+  * **Block Sampling:** The process of randomly sampling users within 1 million ID blocks (e.g., 1M-2M, 2M-3M) to ensure a representative dataset across different user registration cohorts.
+  * **User Types:** The system differentiates between two primary user types:
+      * **`normal`**: Standard users.
+      * **`speculator`**: Users identified by specific portfolio activities, such as holding certain high-risk assets.
+  * **Dependency Injection (DI):** No function in this module initializes its own clients (like Firestore or Pub/Sub) or accesses global configuration. All logic is exposed as stateless functions that receive two arguments: `(config, dependencies)`. This makes the entire module testable and portable.
+
+## Architecture: The `pipe` Object
+
+This module exports a single root object named `pipe`. This object organizes all application logic into distinct, self-contained "pipelines," each corresponding to a major microservice or domain.
+
+```javascript
+// Example of the module's structure
+const { pipe } = require('bulltrackers-module');
+
+// You initialize dependencies ONCE in your main function entry point
+const dependencies = {
+    db: new Firestore(),
+    pubsub: new PubSub(),
+    logger: myLogger,
+    headerManager: new pipe.core.IntelligentHeaderManager(...),
+    proxyManager: new pipe.core.IntelligentProxyManager(...),
+    // ...etc.
+};
+
+// You then pass those dependencies into any pipe function
+async function runDiscovery(config) {
+    await pipe.orchestrator.runDiscoveryOrchestrator(config, dependencies);
+}
+```
+
+-----
+
+## Module Structure
+
+### `pipe.core`
+
+Contains the fundamental, reusable classes and utilities that power the other pipes.
+
+  * **`IntelligentHeaderManager`**: A class that manages a pool of browser headers, rotating them based on historical success rates to avoid API blocks. Performance is tracked in Firestore.
+  * **`IntelligentProxyManager`**: A class that manages a pool of Google Apps Script proxy URLs. It selects available proxies, handles request fetching, and locks proxies that fail (e.g., due to quota errors).
+  * **`FirestoreBatchManager`**: A utility class used by the Task Engine to manage stateful, asynchronous batch writes to Firestore. It handles sharding portfolio data, batching timestamp updates, and flushing processed speculator IDs.
+  * **`firestoreUtils`**: A collection of stateless helper functions for common Firestore operations, such as resetting proxy locks, getting block capacities, fetching exclusion IDs, and querying users who need updates.
+  * **`pubsubUtils`**: A collection of stateless helper functions for publishing messages to Pub/Sub in batches.
+
+### `pipe.orchestrator`
+
+The "brain" of the system. This pipe is responsible for deciding *what* work needs to be done, but not *how* to do it. It's typically run on a schedule (e.g., via Cloud Scheduler).
+
+  * **`runDiscoveryOrchestrator(config, dependencies)`**: The main entry point for the discovery process. It checks which user blocks are under-capacity, gets candidate CIDs to scan, and dispatches `discover` tasks.
+  * **`runUpdateOrchestrator(config, dependencies)`**: The main entry point for the daily update process. It queries Firestore for all verified users who haven't been updated recently and dispatches `update` tasks.
+  * **Discovery Helpers** (`checkDiscoveryNeed`, `getDiscoveryCandidates`, `dispatchDiscovery`): Sub-pipes that handle the logic for finding underpopulated blocks, generating new CIDs (while respecting exclusion lists), and publishing discovery tasks.
+  * **Update Helpers** (`getUpdateTargets`, `dispatchUpdates`): Sub-pipes that find all `normal` and `speculator` users eligible for an update and publish tasks to the `Dispatcher`.
+
+### `pipe.dispatcher`
+
+The "throttle" of the system. Its sole purpose is to receive a large number of tasks (typically from the `orchestrator`) and slowly release them in small batches to a Pub/Sub topic. This prevents the `taskEngine` from scaling too quickly and incurring high costs.
+
+  * **`handleRequest(message, context, config, dependencies)`**: The main entry point, designed to be triggered by a Pub/Sub message containing an array of `tasks`.
+  * **`dispatchTasksInBatches(tasks, dependencies, config)`**: The core logic that loops through tasks, publishing them in batches with a configurable delay.
+
+### `pipe.taskEngine`
+
+The "factory" or "workhorse" of the system. This pipe is event-driven, triggered by individual Pub/Sub messages from the `orchestrator` or `dispatcher`. It executes the core data-fetching operations.
+
+  * **`handleRequest(message, context, config, dependencies)`**: The main entry point. It parses the Pub/Sub message, identifies the task `type`, and routes it to the correct sub-pipe handler.
+  * **`handleDiscover(task, taskId, dependencies, config)`**: Fetches public ranking data for a batch of CIDs. It filters out private/inactive users, applies heuristics (for speculators), and chains to the `verify` task for any promising candidates.
+  * **`handleVerify(task, taskId, dependencies, config)`**: Fetches the portfolio of a *single* user. It verifies they meet the criteria (e.g., holds a speculator asset) and, if so, saves their data and increments the block count.
+  * **`handleUpdate(task, taskId, dependencies, config)`**: Fetches the latest portfolio for an *existing, verified* user and stores the new data in a sharded Firestore document.
+
+### `pipe.computationSystem`
+
+The "refinery" of the system. This pipe is run on a schedule *after* data collection. It loads all the raw portfolio data, runs a suite of calculations (e.g., risk scores, profit migrations), and saves the results in a separate "unified insights" collection for the API to read.
+
+  * **`runOrchestration(config, dependencies)`**: The main entry point. It identifies which dates need processing (or backfilling), loads all data for those dates, initializes all calculation classes, and streams the data through them.
+  * **`dataLoader`**: A set of helpers for loading the sharded portfolio data from Firestore efficiently.
+  * **`computationUtils`**: A set of helpers for categorizing calculations (historical vs. daily), committing results in chunks, and finding the earliest available data date.
+
+### `pipe.api`
+
+The "tap" that serves the processed data. This pipe is an Express.js application designed to be run as a single Cloud Function, providing a public-facing API for the frontend.
+
+  * **`createApiApp(config, dependencies, unifiedCalculations)`**: The main entry point. It returns a fully configured Express app instance, complete with middleware (CORS, JSON) and all API routes.
+  * **API Helpers** (`buildCalculationMap`, `validateRequest`, `fetchUnifiedData`, etc.): Sub-pipes that handle request validation, fetching data from the "unified insights" collection, and generating debug manifests.
+
+### `pipe.maintenance`
+
+A collection of standalone utility and cleanup functions, typically run on a schedule.
+
+  * **`runSpeculatorCleanup(config, dependencies)`**: De-registers stale speculators who haven't held a qualifying asset for a set grace period, and decrements the block counts.
+  * **`handleInvalidSpeculator(message, context, config, dependencies)`**: An event-driven function that listens for Pub/Sub messages containing CIDs of users found to be private or invalid, adding them to an exclusion list.
+  * **`runFetchInsights(config, dependencies)`**: A scheduled function to fetch general eToro market insights (e.g., buy/sell percentages).
+  * **`runFetchPrices(config, dependencies)`**: A scheduled function to fetch daily closing prices for all instruments.
+
+### `pipe.proxy`
+
+The logic for the Google Apps Script web app that acts as the secure proxy layer. This code is *not* run in the Node.js environment; it's intended to be deployed directly into an Apps Script project.
+
+  * **`handlePost(e)`**: The main `doPost(e)` function for the Apps Script. It parses the request from the `IntelligentProxyManager`, executes it using `UrlFetchApp`, and returns the response.
+
+-----
+
+## Data Flow
+
+The system operates in four distinct, decoupled stages:
+
+1.  **Stage 1: Discovery (Scheduled)**
+
+      * `pipe.orchestrator.runDiscoveryOrchestrator` runs.
+      * It determines it needs 5,000 new `normal` users.
+      * It creates 50 `discover` tasks (100 CIDs each) and publishes them to the `task-engine` topic.
+      * `pipe.taskEngine` functions spin up, `handleDiscover` finds 200 promising users.
+      * It publishes 200 `verify` tasks back to the `task-engine` topic.
+      * `pipe.taskEngine` functions `handleVerify` these users, and 75 are saved to the database.
+
+2.  **Stage 2: Update (Scheduled)**
+
+      * `pipe.orchestrator.runUpdateOrchestrator` runs.
+      * It finds 50,000 existing users that need an update.
+      * It publishes *one* message containing all 50,000 tasks to the `dispatcher` topic.
+      * `pipe.dispatcher.handleRequest` receives the message.
+      * It loops, publishing 500 tasks at a time to the `task-engine` topic, with a 30-second delay between batches.
+      * `pipe.taskEngine` functions `handleUpdate` the portfolios, scaling gradually.
+
+3.  **Stage 3: Computation (Scheduled)**
+
+      * `pipe.computationSystem.runOrchestration` runs (e.g., at 1:00 AM).
+      * It loads all portfolio data collected in Stages 1 & 2.
+      * It runs all calculations and saves the results to the `unified_insights` collection.
+
+4.  **Stage 4: Serving (On-Demand)**
+
+      * A frontend user loads a chart.
+      * The browser sends a request to the `pipe.api` function.
+      * The API validates the request, reads the *pre-computed* data from `unified_insights`, and returns it.
+
+## Usage
+
+In your Google Cloud Function `index.js` file:
+
+```javascript
+/**
+ * @fileoverview Unified Cloud Functions entry (Refactored for Pipe Architecture)
+ */
+
+// Import FieldPath here
+const { Firestore, FieldValue, FieldPath } = require('@google-cloud/firestore'); 
+const { PubSub }                     = require('@google-cloud/pubsub');
+const { logger }                     = require('sharedsetup')(__filename);
+const { pipe }                       = require('bulltrackers-module');
+const { calculations }               = require('aiden-shared-calculations-unified');
+const fs                             = require('fs');
+const path                           = require('path');
+
+// --- Initialize Clients ---
+const db      = new Firestore();
+const pubsub  = new PubSub();
+
+// --- Load Configs ---
+const cfg = Object.fromEntries(fs.readdirSync(path.join(__dirname, 'config'))
+      .filter(f => f.endsWith('_config.js'))
+      .map(f => [path.basename(f, '_config.js'), require(path.join(__dirname, 'config', f))])
+);
+
+// --- Instantiate Core Managers ---
+const headerManager = new pipe.core.IntelligentHeaderManager(db, logger, cfg.core);
+const proxyManager  = new pipe.core.IntelligentProxyManager (db, logger, cfg.core);
+const batchManager  = new pipe.core.FirestoreBatchManager   (db, headerManager, logger, cfg.taskEngine);
+
+// --- Master Dependencies ---
+const allDependencies = {
+    db              : db,
+    pubsub          : pubsub,
+    logger          : logger,
+    FieldValue      : FieldValue,
+    FieldPath       : FieldPath,
+    headerManager   : headerManager,
+    proxyManager    : proxyManager,
+    batchManager    : batchManager,
+    firestoreUtils  : pipe.core.firestoreUtils,
+    pubsubUtils     : pipe.core.pubsubUtils
+};
+
+// --- Export Cloud Function Handlers ---
+const handlers = {
+    discoveryOrchestrator        : ()     => pipe.orchestrator     .runDiscoveryOrchestrator (cfg.orchestrator     , allDependencies),
+    updateOrchestrator           : ()     => pipe.orchestrator     .runUpdateOrchestrator    (cfg.orchestrator     , allDependencies),
+    speculatorCleanupOrchestrator: ()     => pipe.maintenance      .runSpeculatorCleanup     (cfg.cleanup          , allDependencies),
+    fetchEtoroInsights           : ()     => pipe.maintenance      .runFetchInsights         (cfg.fetchInsights    , allDependencies),
+    updateClosingPrices          : ()     => pipe.maintenance      .runFetchPrices           (cfg.priceFetcher     , allDependencies),
+    computationSystem            : ()     => pipe.computationSystem.runOrchestration         (cfg.computationSystem, allDependencies),
+
+    taskEngineHandler            : (m, c) => pipe.taskEngine       .handleRequest          (m, c, cfg.taskEngine       , allDependencies),
+    dispatcher                   : (m, c) => pipe.dispatcher       .handleRequest          (m, c, cfg.dispatcher       , allDependencies),
+    invalidSpeculatorHandler     : (m, c) => pipe.maintenance      .handleInvalidSpeculator(m, c, cfg.invalidSpeculator, allDependencies)
+};
+
+Object.entries(handlers).forEach(([name, fn]) => exports[name] = fn);
+
+// --- API Export ---
+exports.genericApiV2 = pipe.api.createApiApp(cfg.genericApiV2, allDependencies, calculations);
+
+// --- Local API Server ---
+if (require.main === module) {
+    const port = process.env.PORT || 8080;
+    exports.genericApiV2.listen(port, () => console.log(`API listening on port ${port}`));
+}
 ```

package/functions/appscript-api/helpers/errors.js

@@ -1,19 +1,19 @@
-/**
- * Creates a structured JSON error response.
- * @param {string} message
- * @param {number} statusCode
- * @returns {GoogleAppsScript.Content.TextOutput}
- */
-function createErrorResponse(message, statusCode) {
-  const error = {
-    error: {
-      message,
-      code: statusCode,
-    },
-  };
-
-  return ContentService.createTextOutput(JSON.stringify(error))
-    .setMimeType(ContentService.MimeType.JSON);
-}
-
-module.exports = { createErrorResponse };
+/**
+ * Creates a structured JSON error response.
+ * @param {string} message
+ * @param {number} statusCode
+ * @returns {GoogleAppsScript.Content.TextOutput}
+ */
+function createErrorResponse(message, statusCode) {
+  const error = {
+    error: {
+      message,
+      code: statusCode,
+    },
+  };
+
+  return ContentService.createTextOutput(JSON.stringify(error))
+    .setMimeType(ContentService.MimeType.JSON);
+}
+
+module.exports = { createErrorResponse };

package/functions/appscript-api/index.js

@@ -1,58 +1,58 @@
-/**
- * @fileoverview Google Apps Script proxy logic (v2) as a module.
- * Can be imported and used in a doPost or other GAS handlers.
- * NOT intended to be used as a Google Cloud Function or Otherwise, and is included in this package as an example of a proxy implementation.
- */
-
-const { createErrorResponse } = require('./helpers/errors');
-
-/**
- * Handles the incoming POST request object from Apps Script.
- * @param {GoogleAppsScript.Events.DoPost} e
- * @returns {GoogleAppsScript.Content.TextOutput}
- */
-function handlePost(e) {
-  try {
-    // Parse the request details from the POST body
-    const requestDetails = JSON.parse(e.postData.contents);
-    const { url, headers, method, body } = requestDetails;
-
-    if (!url || !headers) {
-      return createErrorResponse("Invalid request: 'url' and 'headers' are required.", 400);
-    }
-
-    // Prepare the external request options
-    const options = {
-      headers: headers,
-      muteHttpExceptions: true,
-      method: method || 'GET',
-    };
-
-    if (body) {
-      options.payload = body;
-      if (!headers['Content-Type']) {
-        headers['Content-Type'] = 'application/json';
-      }
-    }
-
-    // Make the external request using UrlFetchApp
-    const response = UrlFetchApp.fetch(url, options);
-
-    // Package the response to send back
-    const responseData = {
-      statusCode: response.getResponseCode(),
-      headers: response.getHeaders(),
-      body: response.getContentText(),
-    };
-
-    return ContentService.createTextOutput(JSON.stringify(responseData))
-      .setMimeType(ContentService.MimeType.JSON);
-
-  } catch (error) {
-    return createErrorResponse(error.toString(), 500);
-  }
-}
-
-module.exports = {
-  handlePost,
-};
+/**
+ * @fileoverview Google Apps Script proxy logic (v2) as a module.
+ * Can be imported and used in a doPost or other GAS handlers.
+ * NOT intended to be used as a Google Cloud Function or Otherwise, and is included in this package as an example of a proxy implementation.
+ */
+
+const { createErrorResponse } = require('./helpers/errors');
+
+/**
+ * Handles the incoming POST request object from Apps Script.
+ * @param {GoogleAppsScript.Events.DoPost} e
+ * @returns {GoogleAppsScript.Content.TextOutput}
+ */
+function handlePost(e) {
+  try {
+    // Parse the request details from the POST body
+    const requestDetails = JSON.parse(e.postData.contents);
+    const { url, headers, method, body } = requestDetails;
+
+    if (!url || !headers) {
+      return createErrorResponse("Invalid request: 'url' and 'headers' are required.", 400);
+    }
+
+    // Prepare the external request options
+    const options = {
+      headers: headers,
+      muteHttpExceptions: true,
+      method: method || 'GET',
+    };
+
+    if (body) {
+      options.payload = body;
+      if (!headers['Content-Type']) {
+        headers['Content-Type'] = 'application/json';
+      }
+    }
+
+    // Make the external request using UrlFetchApp
+    const response = UrlFetchApp.fetch(url, options);
+
+    // Package the response to send back
+    const responseData = {
+      statusCode: response.getResponseCode(),
+      headers: response.getHeaders(),
+      body: response.getContentText(),
+    };
+
+    return ContentService.createTextOutput(JSON.stringify(responseData))
+      .setMimeType(ContentService.MimeType.JSON);
+
+  } catch (error) {
+    return createErrorResponse(error.toString(), 500);
+  }
+}
+
+module.exports = {
+  handlePost,
+};