backtest-kit 1.1.6 → 1.1.8

package/README.md

@@ -1,4 +1,4 @@
-# Backtest Kit
+# 🧿 Backtest Kit
 
 > A production-ready TypeScript framework for backtesting and live trading strategies with crash-safe state persistence, signal validation, and memory-optimized architecture.
 
@@ -19,10 +19,13 @@
 - ⚡ **Memory Optimized** - Prototype methods + memoization + streaming
 - 🔌 **Flexible Architecture** - Plug your own exchanges and strategies
 - 📝 **Markdown Reports** - Auto-generated trading reports with statistics (win rate, avg PNL, Sharpe Ratio, Standard Deviation, Certainty Ratio, Expected Yearly Returns, Risk-Adjusted Returns)
+- 📊 **Performance Profiling** - Built-in performance tracking with aggregated statistics (avg, min, max, stdDev, P95, P99) for bottleneck analysis
 - 🛑 **Graceful Shutdown** - Live.background() waits for open positions to close before stopping
 - 💉 **Strategy Dependency Injection** - addStrategy() enables DI pattern for trading strategies
 - 🔍 **Schema Reflection API** - listExchanges(), listStrategies(), listFrames() for runtime introspection
-- 🧪 **Comprehensive Test Coverage** - 45+ unit tests covering validation, PNL, callbacks, reports, and event system
+- 🧪 **Comprehensive Test Coverage** - 61 unit tests covering validation, PNL, callbacks, reports, performance tracking, and event system
+- 💾 **Zero Data Download** - Unlike Freqtrade, no need to download gigabytes of historical data - plug any data source (CCXT, database, API)
+- 🔒 **Safe Math & Robustness** - All metrics protected against NaN/Infinity with unsafe numeric checks, returns N/A for invalid calculations
 
 ## Installation
 
@@ -106,7 +109,7 @@ addStrategy({
   interval: "5m", // Throttling: signals generated max once per 5 minutes
   getSignal: async (symbol) => {
     // Your signal generation logic
-    // Validation happens automatically (prices, TP/SL logic, timestamps)
+    // Validation happens automatically (prices, TP/SL logic)
     return {
       position: "long",
       note: "BTC breakout",
@@ -114,14 +117,13 @@ addStrategy({
       priceTakeProfit: 51000,  // Must be > priceOpen for long
       priceStopLoss: 49000,     // Must be < priceOpen for long
       minuteEstimatedTime: 60,  // Signal duration in minutes
-      timestamp: Date.now(),
     };
   },
   callbacks: {
-    onOpen: (backtest, symbol, signal) => {
+    onOpen: (symbol, signal, currentPrice, backtest) => {
       console.log(`[${backtest ? "BT" : "LIVE"}] Signal opened:`, signal.id);
     },
-    onClose: (backtest, symbol, priceClose, signal) => {
+    onClose: (symbol, signal, priceClose, backtest) => {
       console.log(`[${backtest ? "BT" : "LIVE"}] Signal closed:`, priceClose);
     },
   },
@@ -348,7 +350,6 @@ All signals are validated automatically before execution:
   priceTakeProfit: 51000,  // ✅ 51000 > 50000
   priceStopLoss: 49000,     // ✅ 49000 < 50000
   minuteEstimatedTime: 60,  // ✅ positive
-  timestamp: Date.now(),    // ✅ positive
 }
 
 // ❌ Invalid long signal - throws error
@@ -465,7 +466,26 @@ const stopBacktest = Backtest.background("BTCUSDT", {
   frameName: "1d-backtest"
 });
 
-// Generate markdown report
+// Get raw statistical data (Controller)
+const stats = await Backtest.getData("my-strategy");
+console.log(stats);
+// Returns:
+// {
+//   signalList: [...],           // All closed signals
+//   totalSignals: 10,
+//   winCount: 7,
+//   lossCount: 3,
+//   winRate: 70.0,               // Percentage (higher is better)
+//   avgPnl: 1.23,                // Average PNL % (higher is better)
+//   totalPnl: 12.30,             // Total PNL % (higher is better)
+//   stdDev: 2.45,                // Standard deviation (lower is better)
+//   sharpeRatio: 0.50,           // Risk-adjusted return (higher is better)
+//   annualizedSharpeRatio: 9.55, // Sharpe × √365 (higher is better)
+//   certaintyRatio: 1.75,        // avgWin / |avgLoss| (higher is better)
+//   expectedYearlyReturns: 156   // Estimated yearly trades (higher is better)
+// }
+
+// Generate markdown report (View)
 const markdown = await Backtest.getReport("my-strategy");
 console.log(markdown);
 
@@ -476,29 +496,75 @@ await Backtest.dump("my-strategy");
 await Backtest.dump("my-strategy", "./custom/path");
 ```
 
-**Report includes:**
-- Total closed signals
+**getData() returns BacktestStatistics:**
+- `signalList` - Array of all closed signals
+- `totalSignals` - Total number of closed signals
+- `winCount` / `lossCount` - Number of winning/losing trades
+- `winRate` - Win percentage (higher is better)
+- `avgPnl` - Average PNL percentage (higher is better)
+- `totalPnl` - Total PNL percentage (higher is better)
+- `stdDev` - Standard deviation / volatility (lower is better)
+- `sharpeRatio` - Risk-adjusted return (higher is better)
+- `annualizedSharpeRatio` - Sharpe Ratio × √365 (higher is better)
+- `certaintyRatio` - avgWin / |avgLoss| (higher is better)
+- `expectedYearlyReturns` - Estimated number of trades per year (higher is better)
+
+**getReport() includes:**
+- All metrics from getData() formatted as markdown
 - All signal details (prices, TP/SL, PNL, duration, close reason)
 - Timestamps for each signal
+- "Higher is better" / "Lower is better" annotations
 
 ### Live Trading Reports
 
 ```typescript
 import { Live } from "backtest-kit";
 
-// Generate live trading report
+// Get raw statistical data (Controller)
+const stats = await Live.getData("my-strategy");
+console.log(stats);
+// Returns:
+// {
+//   eventList: [...],            // All events (idle, opened, active, closed)
+//   totalEvents: 15,
+//   totalClosed: 5,
+//   winCount: 3,
+//   lossCount: 2,
+//   winRate: 60.0,               // Percentage (higher is better)
+//   avgPnl: 1.23,                // Average PNL % (higher is better)
+//   totalPnl: 6.15,              // Total PNL % (higher is better)
+//   stdDev: 1.85,                // Standard deviation (lower is better)
+//   sharpeRatio: 0.66,           // Risk-adjusted return (higher is better)
+//   annualizedSharpeRatio: 12.61,// Sharpe × √365 (higher is better)
+//   certaintyRatio: 2.10,        // avgWin / |avgLoss| (higher is better)
+//   expectedYearlyReturns: 365   // Estimated yearly trades (higher is better)
+// }
+
+// Generate markdown report (View)
 const markdown = await Live.getReport("my-strategy");
 
 // Save to disk (default: ./logs/live/my-strategy.md)
 await Live.dump("my-strategy");
 ```
 
-**Report includes:**
-- Total events (idle, opened, active, closed)
-- Closed signals count
-- Win rate (% wins, wins/losses)
-- Average PNL percentage
+**getData() returns LiveStatistics:**
+- `eventList` - Array of all events (idle, opened, active, closed)
+- `totalEvents` - Total number of events
+- `totalClosed` - Total number of closed signals
+- `winCount` / `lossCount` - Number of winning/losing trades
+- `winRate` - Win percentage (higher is better)
+- `avgPnl` - Average PNL percentage (higher is better)
+- `totalPnl` - Total PNL percentage (higher is better)
+- `stdDev` - Standard deviation / volatility (lower is better)
+- `sharpeRatio` - Risk-adjusted return (higher is better)
+- `annualizedSharpeRatio` - Sharpe Ratio × √365 (higher is better)
+- `certaintyRatio` - avgWin / |avgLoss| (higher is better)
+- `expectedYearlyReturns` - Estimated number of trades per year (higher is better)
+
+**getReport() includes:**
+- All metrics from getData() formatted as markdown
 - Signal-by-signal details with current state
+- "Higher is better" / "Lower is better" annotations
 
 **Report example:**
 ```markdown
@@ -506,8 +572,14 @@ await Live.dump("my-strategy");
 
 Total events: 15
 Closed signals: 5
-Win rate: 60.00% (3W / 2L)
-Average PNL: +1.23%
+Win rate: 60.00% (3W / 2L) (higher is better)
+Average PNL: +1.23% (higher is better)
+Total PNL: +6.15% (higher is better)
+Standard Deviation: 1.85% (lower is better)
+Sharpe Ratio: 0.66 (higher is better)
+Annualized Sharpe Ratio: 12.61 (higher is better)
+Certainty Ratio: 2.10 (higher is better)
+Expected Yearly Returns: 365 trades (higher is better)
 
 | Timestamp | Action | Symbol | Signal ID | Position | ... | PNL (net) | Close Reason |
 |-----------|--------|--------|-----------|----------|-----|-----------|--------------|
@@ -639,6 +711,8 @@ Live.background("BTCUSDT", {
 - `listenSignalBacktestOnce(filter, callback)` - Subscribe to backtest signals once
 - `listenSignalLive(callback)` - Subscribe to live signals only
 - `listenSignalLiveOnce(filter, callback)` - Subscribe to live signals once
+- `listenPerformance(callback)` - Subscribe to performance metrics (backtest + live)
+- `listenProgress(callback)` - Subscribe to backtest progress events
 - `listenError(callback)` - Subscribe to background execution errors
 - `listenDone(callback)` - Subscribe to background completion events
 - `listenDoneOnce(filter, callback)` - Subscribe to background completion once
@@ -744,7 +818,7 @@ const quantity = await formatQuantity("BTCUSDT", 0.123456789);
 #### Backtest API
 
 ```typescript
-import { Backtest } from "backtest-kit";
+import { Backtest, BacktestStatistics } from "backtest-kit";
 
 // Stream backtest results
 Backtest.run(
@@ -762,7 +836,10 @@ Backtest.background(
   context: { strategyName, exchangeName, frameName }
 ): Promise<() => void> // Returns cancellation function
 
-// Generate markdown report
+// Get raw statistical data (Controller)
+Backtest.getData(strategyName: string): Promise<BacktestStatistics>
+
+// Generate markdown report (View)
 Backtest.getReport(strategyName: string): Promise<string>
 
 // Save report to disk
@@ -772,7 +849,7 @@ Backtest.dump(strategyName: string, path?: string): Promise<void>
 #### Live Trading API
 
 ```typescript
-import { Live } from "backtest-kit";
+import { Live, LiveStatistics } from "backtest-kit";
 
 // Stream live results (infinite)
 Live.run(
@@ -789,27 +866,133 @@ Live.background(
   context: { strategyName, exchangeName }
 ): Promise<() => void> // Returns cancellation function
 
-// Generate markdown report
+// Get raw statistical data (Controller)
+Live.getData(strategyName: string): Promise<LiveStatistics>
+
+// Generate markdown report (View)
 Live.getReport(strategyName: string): Promise<string>
 
 // Save report to disk
 Live.dump(strategyName: string, path?: string): Promise<void>
 ```
 
+#### Performance Profiling API
+
+```typescript
+import { Performance, PerformanceStatistics, listenPerformance } from "backtest-kit";
+
+// Get raw performance statistics (Controller)
+Performance.getData(strategyName: string): Promise<PerformanceStatistics>
+
+// Generate markdown report with bottleneck analysis (View)
+Performance.getReport(strategyName: string): Promise<string>
+
+// Save performance report to disk (default: ./logs/performance)
+Performance.dump(strategyName: string, path?: string): Promise<void>
+
+// Clear accumulated performance data
+Performance.clear(strategyName?: string): Promise<void>
+
+// Listen to real-time performance events
+listenPerformance((event) => {
+  console.log(`${event.metricType}: ${event.duration.toFixed(2)}ms`);
+  console.log(`Strategy: ${event.strategyName} @ ${event.exchangeName}`);
+  console.log(`Symbol: ${event.symbol}, Backtest: ${event.backtest}`);
+});
+```
+
 ## Type Definitions
 
+### Statistics Types
+
+```typescript
+// Backtest statistics (exported from "backtest-kit")
+interface BacktestStatistics {
+  signalList: IStrategyTickResultClosed[];  // All closed signals
+  totalSignals: number;
+  winCount: number;
+  lossCount: number;
+  winRate: number | null;               // Win percentage (higher is better)
+  avgPnl: number | null;                // Average PNL % (higher is better)
+  totalPnl: number | null;              // Total PNL % (higher is better)
+  stdDev: number | null;                // Standard deviation (lower is better)
+  sharpeRatio: number | null;           // Risk-adjusted return (higher is better)
+  annualizedSharpeRatio: number | null; // Sharpe × √365 (higher is better)
+  certaintyRatio: number | null;        // avgWin / |avgLoss| (higher is better)
+  expectedYearlyReturns: number | null; // Estimated yearly trades (higher is better)
+}
+
+// Live statistics (exported from "backtest-kit")
+interface LiveStatistics {
+  eventList: TickEvent[];               // All events (idle, opened, active, closed)
+  totalEvents: number;
+  totalClosed: number;
+  winCount: number;
+  lossCount: number;
+  winRate: number | null;               // Win percentage (higher is better)
+  avgPnl: number | null;                // Average PNL % (higher is better)
+  totalPnl: number | null;              // Total PNL % (higher is better)
+  stdDev: number | null;                // Standard deviation (lower is better)
+  sharpeRatio: number | null;           // Risk-adjusted return (higher is better)
+  annualizedSharpeRatio: number | null; // Sharpe × √365 (higher is better)
+  certaintyRatio: number | null;        // avgWin / |avgLoss| (higher is better)
+  expectedYearlyReturns: number | null; // Estimated yearly trades (higher is better)
+}
+
+// Performance statistics (exported from "backtest-kit")
+interface PerformanceStatistics {
+  strategyName: string;                 // Strategy name
+  totalEvents: number;                  // Total number of performance events
+  totalDuration: number;                // Total execution time (ms)
+  metricStats: Record<string, {         // Statistics by metric type
+    metricType: PerformanceMetricType;  // backtest_total | backtest_timeframe | backtest_signal | live_tick
+    count: number;                      // Number of samples
+    totalDuration: number;              // Total duration (ms)
+    avgDuration: number;                // Average duration (ms)
+    minDuration: number;                // Minimum duration (ms)
+    maxDuration: number;                // Maximum duration (ms)
+    stdDev: number;                     // Standard deviation (ms)
+    median: number;                     // Median duration (ms)
+    p95: number;                        // 95th percentile (ms)
+    p99: number;                        // 99th percentile (ms)
+  }>;
+  events: PerformanceContract[];        // All raw performance events
+}
+
+// Performance event (exported from "backtest-kit")
+interface PerformanceContract {
+  timestamp: number;                    // When metric was recorded (epoch ms)
+  metricType: PerformanceMetricType;    // Type of operation measured
+  duration: number;                     // Operation duration (ms)
+  strategyName: string;                 // Strategy name
+  exchangeName: string;                 // Exchange name
+  symbol: string;                       // Trading symbol
+  backtest: boolean;                    // true = backtest, false = live
+}
+
+// Performance metric types (exported from "backtest-kit")
+type PerformanceMetricType =
+  | "backtest_total"      // Total backtest duration
+  | "backtest_timeframe"  // Single timeframe processing
+  | "backtest_signal"     // Signal processing (tick + getNextCandles + backtest)
+  | "live_tick";          // Single live tick duration
+```
+
 ### Signal Data
 
 ```typescript
 interface ISignalRow {
-  id: string;                    // Auto-generated
+  id: string;                     // UUID v4 auto-generated
   position: "long" | "short";
-  note: string;
+  note?: string;
   priceOpen: number;
   priceTakeProfit: number;
   priceStopLoss: number;
   minuteEstimatedTime: number;
-  timestamp: number;
+  exchangeName: string;
+  strategyName: string;
+  timestamp: number;              // Signal creation timestamp
+  symbol: string;                 // Trading pair (e.g., "BTCUSDT")
 }
 ```
 
@@ -817,9 +1000,27 @@ interface ISignalRow {
 
 ```typescript
 type IStrategyTickResult =
-  | { action: "idle"; signal: null }
-  | { action: "opened"; signal: ISignalRow }
-  | { action: "active"; signal: ISignalRow; currentPrice: number }
+  | {
+      action: "idle";
+      signal: null;
+      strategyName: string;
+      exchangeName: string;
+      currentPrice: number;
+    }
+  | {
+      action: "opened";
+      signal: ISignalRow;
+      strategyName: string;
+      exchangeName: string;
+      currentPrice: number;
+    }
+  | {
+      action: "active";
+      signal: ISignalRow;
+      currentPrice: number;
+      strategyName: string;
+      exchangeName: string;
+    }
   | {
       action: "closed";
       signal: ISignalRow;
@@ -827,10 +1028,12 @@ type IStrategyTickResult =
       closeReason: "take_profit" | "stop_loss" | "time_expired";
       closeTimestamp: number;
       pnl: {
-        priceOpenWithCosts: number;
-        priceCloseWithCosts: number;
         pnlPercentage: number;
+        priceOpen: number;        // Entry price adjusted with slippage and fees
+        priceClose: number;       // Exit price adjusted with slippage and fees
       };
+      strategyName: string;
+      exchangeName: string;
     };
 ```
 
@@ -909,6 +1112,72 @@ Backtest.background("BTCUSDT", {
 });
 ```
 
+### Performance Profiling
+
+```typescript
+import { Performance, listenPerformance, Backtest } from "backtest-kit";
+
+// Listen to real-time performance metrics
+listenPerformance((event) => {
+  console.log(`[${event.metricType}] ${event.duration.toFixed(2)}ms`);
+  console.log(`  Strategy: ${event.strategyName}`);
+  console.log(`  Symbol: ${event.symbol}, Backtest: ${event.backtest}`);
+});
+
+// Run backtest
+await Backtest.background("BTCUSDT", {
+  strategyName: "my-strategy",
+  exchangeName: "binance",
+  frameName: "1d-backtest"
+});
+
+// Get aggregated performance statistics
+const perfStats = await Performance.getData("my-strategy");
+console.log("Performance Statistics:");
+console.log(`  Total events: ${perfStats.totalEvents}`);
+console.log(`  Total duration: ${perfStats.totalDuration.toFixed(2)}ms`);
+console.log(`  Metrics tracked: ${Object.keys(perfStats.metricStats).join(", ")}`);
+
+// Analyze bottlenecks
+for (const [type, stats] of Object.entries(perfStats.metricStats)) {
+  console.log(`\n${type}:`);
+  console.log(`  Count: ${stats.count}`);
+  console.log(`  Average: ${stats.avgDuration.toFixed(2)}ms`);
+  console.log(`  Min/Max: ${stats.minDuration.toFixed(2)}ms / ${stats.maxDuration.toFixed(2)}ms`);
+  console.log(`  P95/P99: ${stats.p95.toFixed(2)}ms / ${stats.p99.toFixed(2)}ms`);
+  console.log(`  Std Dev: ${stats.stdDev.toFixed(2)}ms`);
+}
+
+// Generate and save performance report
+const markdown = await Performance.getReport("my-strategy");
+await Performance.dump("my-strategy"); // Saves to ./logs/performance/my-strategy.md
+```
+
+**Performance Report Example:**
+```markdown
+# Performance Report: my-strategy
+
+**Total events:** 1440
+**Total execution time:** 12345.67ms
+**Number of metric types:** 3
+
+## Time Distribution
+
+- **backtest_timeframe**: 65.4% (8074.32ms total)
+- **backtest_signal**: 28.3% (3493.85ms total)
+- **backtest_total**: 6.3% (777.50ms total)
+
+## Detailed Metrics
+
+| Metric Type | Count | Total (ms) | Avg (ms) | Min (ms) | Max (ms) | Std Dev (ms) | Median (ms) | P95 (ms) | P99 (ms) |
+| --- | --- | --- | --- | --- | --- | --- | --- | --- | --- |
+| backtest_timeframe | 1440 | 8074.32 | 5.61 | 2.10 | 12.45 | 1.85 | 5.20 | 8.90 | 10.50 |
+| backtest_signal | 45 | 3493.85 | 77.64 | 45.20 | 125.80 | 18.32 | 75.10 | 110.20 | 120.15 |
+| backtest_total | 1 | 777.50 | 777.50 | 777.50 | 777.50 | 0.00 | 777.50 | 777.50 | 777.50 |
+
+**Note:** All durations are in milliseconds. P95/P99 represent 95th and 99th percentile response times.
+```
+
 ### Early Termination
 
 **Using async generator with break:**