@backtest-kit/cli 5.11.0 → 5.11.1

package/template/project/docs/backtest_risk_async.md

@@ -0,0 +1,153 @@
+# Async Risk Validation with resolve()
+
+## Overview
+
+Risk validators can be async and call `resolve()` on any graph node. This lets you reuse the same source nodes (garch, trend, fear & greed) in both signal generation and risk filtering — without duplicate fetches.
+
+---
+
+## Sync vs async validation
+
+`backtest_strategy_structure.md` shows the basic sync pattern:
+
+```ts
+// Sync — receives pre-computed payload fields
+validations: [
+  {
+    validate: ({ currentSignal, currentPrice }) => {
+      if (currentSignal.position === "short" && currentPrice > 50000) {
+        throw new Error("Price too high for short");
+      }
+    },
+  },
+],
+```
+
+Async pattern — resolve a graph node:
+
+```ts
+// Async — fetch fresh data at validation time
+validations: [
+  async () => {
+    const garch = await resolve(garchSource);
+    if (!garch.reliable || garch.movePercent < 1.0) {
+      throw new Error(`GARCH volatility too low: ${garch.movePercent.toFixed(2)}%`);
+    }
+  },
+],
+```
+
+Both forms can coexist in the same `validations` array.
+
+---
+
+## GARCH volatility gate
+
+The canonical use case: block signals when predicted volatility is too low to cover transaction costs and generate profit.
+
+```ts
+import { predict } from "garch";
+import { sourceNode, resolve } from "@backtest-kit/graph";
+import { Cache, getCandles, addRiskSchema } from "backtest-kit";
+
+const CANDLES_FOR_GARCH = 1_000;
+const GARCH_CONFIDENCE  = 0.95;
+const MIN_MOVE_PERCENT  = 1.0;
+
+const garchSource = sourceNode(
+  Cache.fn(
+    async (symbol: string) => {
+      const candles = await getCandles(symbol, "8h", CANDLES_FOR_GARCH);
+      return predict(candles, "8h", null, GARCH_CONFIDENCE);
+    },
+    { interval: "8h", key: ([symbol]: [string]) => symbol },
+  ),
+);
+
+addRiskSchema({
+  riskName: "garch_volatility_risk",
+  validations: [
+    async () => {
+      const garch = await resolve(garchSource);
+      if (!garch.reliable || garch.movePercent < MIN_MOVE_PERCENT) {
+        throw new Error(
+          `GARCH volatility too low: ${garch.movePercent.toFixed(2)}% (need ≥${MIN_MOVE_PERCENT}%, confidence=${GARCH_CONFIDENCE})`,
+        );
+      }
+    },
+  ],
+});
+```
+
+**`garch.reliable`** — false when the model didn't converge or has insufficient data. Always check this before using `movePercent`.
+
+**`garch.movePercent`** — predicted price move as percentage of current price over one candle (one 8h bar here). At confidence=0.95 (±1.96σ), a value of 1.0 means the model predicts ≥1% move with 95% probability.
+
+---
+
+## Fear & Greed gate (directional)
+
+From `feb_2024.strategy.ts` — block longs in fear and shorts in greed:
+
+```ts
+addRiskSchema({
+  riskName: "fear_greed_directional",
+  validations: [
+    async ({ currentSignal }) => {
+      const fearGreed = await resolve(fearGreedSource);
+      if (currentSignal.position === "short" && fearGreed > 50) {
+        throw new Error(`Still greed (${fearGreed}): short not allowed`);
+      }
+    },
+    async ({ currentSignal }) => {
+      const fearGreed = await resolve(fearGreedSource);
+      if (currentSignal.position === "long" && fearGreed < 50) {
+        throw new Error(`Still fear (${fearGreed}): long not allowed`);
+      }
+    },
+  ],
+});
+```
+
+`fearGreedSource` uses `Cache.file` with `interval: "8h"` — fetched once per 8h from `api.alternative.me`.
+
+---
+
+## Node deduplication in risk
+
+When `garchSource` is a dependency of both `enterSignal` (output node) and the risk validator, the graph deduplicates resolution within the same tick. If `getSignal()` already resolved `garchSource`, the risk validator gets the cached result — no second fetch.
+
+This only holds if the validator uses `resolve(garchSource)` with the **same node reference**. Do not create a new `sourceNode` inside the validator — that would be a different node with its own cache.
+
+---
+
+## `validate` vs bare async function
+
+Both forms are accepted in `validations`:
+
+```ts
+// Object form — supports `note` field
+validations: [
+  {
+    note: "GARCH volatility must be at least 1%",
+    validate: async () => {
+      const garch = await resolve(garchSource);
+      if (!garch.reliable || garch.movePercent < 1.0) {
+        throw new Error("...");
+      }
+    },
+  },
+]
+
+// Bare async function — more concise
+validations: [
+  async () => {
+    const garch = await resolve(garchSource);
+    if (!garch.reliable || garch.movePercent < 1.0) {
+      throw new Error("...");
+    }
+  },
+]
+```
+
+Throw `Error` to reject the signal. Return normally (or return `void`) to allow it.

package/template/project/docs/backtest_strategy_structure.md

@@ -0,0 +1,250 @@
+# Strategy Structure Guide
+
+## Overview
+
+A backtest-kit strategy file registers four schemas and wires them together. The CLI reads all `.ts` / `.mjs` files in the target directory and calls the registered schemas at runtime.
+
+```
+addExchangeSchema  — data source (candles, trades, order book)
+addFrameSchema     — backtest time window
+addStrategySchema  — signal generation logic
+addRiskSchema      — optional position filters
+```
+
+---
+
+## Minimal Strategy File
+
+```ts
+import { addExchangeSchema, addFrameSchema, addStrategySchema, getCandles, Log } from "backtest-kit";
+import { randomString } from "functools-kit";
+
+// 1. Data source
+addExchangeSchema({
+  exchangeName: "ccxt-exchange",
+  getCandles: async (symbol, interval, since, limit) => { ... },
+});
+
+// 2. Backtest window
+addFrameSchema({
+  frameName: "feb_2024_frame",
+  interval: "1m",                              // tick granularity
+  startDate: new Date("2024-02-01T00:00:00Z"),
+  endDate:   new Date("2024-02-29T23:59:59Z"),
+});
+
+// 3. Signal logic
+addStrategySchema({
+  strategyName: "my_strategy",
+  interval: "15m",                             // getSignal call frequency
+  getSignal: async (symbol, when) => {
+    // ... compute signal
+    return {
+      id: randomString(),
+      position: "long",
+      priceTakeProfit: 50000,
+      priceStopLoss:   48000,
+      minuteEstimatedTime: 480,
+    };
+    // return null → no signal this tick
+  },
+});
+```
+
+---
+
+## ISignalDto — Signal Fields
+
+```ts
+interface ISignalDto {
+  id?:                  string;   // auto-generated UUID if omitted
+  position:             "long" | "short";
+  priceTakeProfit:      number;   // for long: > priceOpen; for short: < priceOpen
+  priceStopLoss:        number;   // for long: < priceOpen; for short: > priceOpen
+  minuteEstimatedTime:  number;   // expected hold duration in minutes
+  priceOpen?:           number;   // omit → opens immediately at current price
+                                  // provide → creates a scheduled signal
+  note?:                string;   // human-readable description
+}
+```
+
+**Immediate vs scheduled signal:**
+- `priceOpen` omitted → position opens at current VWAP immediately
+- `priceOpen` set → signal waits until price reaches that level, then opens
+
+---
+
+## Signal Lifecycle
+
+```
+idle
+  │  getSignal() returns ISignalDto (no priceOpen)
+  ▼
+opened → active → closed (TP or SL hit, or time expired)
+                 ↓
+              pnlPercentage in result
+
+  │  getSignal() returns ISignalDto (with priceOpen)
+  ▼
+scheduled → waiting → opened → active → closed
+                    ↓
+                 cancelled (if price never reached)
+```
+
+Lifecycle callbacks (all optional):
+
+```ts
+addStrategySchema({
+  strategyName: "...",
+  interval: "15m",
+  getSignal: ...,
+  callbacks: {
+    onOpen:    (symbol, signal, price, backtest) => { ... },
+    onClose:   (symbol, signal, closePrice, backtest) => { ... },
+    onActive:  (symbol, signal, price, backtest) => { ... },
+    onIdle:    (symbol, price, backtest) => { ... },
+    onCancel:  (symbol, signal, price, backtest) => { ... },
+    onSchedule:(symbol, signal, price, backtest) => { ... },
+    onPartialProfit: (symbol, signal, price, revenuePercent, backtest) => { ... },
+    onPartialLoss:   (symbol, signal, price, lossPercent, backtest) => { ... },
+    onBreakeven:     (symbol, signal, price, backtest) => { ... },
+  },
+});
+```
+
+---
+
+## Interval Types
+
+| Schema | Parameter | Valid values |
+|---|---|---|
+| `addFrameSchema` | `interval` | `"1m"` `"3m"` `"5m"` `"15m"` `"30m"` `"1h"` `"2h"` `"4h"` `"6h"` `"8h"` `"12h"` `"1d"` `"3d"` |
+| `addStrategySchema` | `interval` | `"1m"` `"3m"` `"5m"` `"15m"` `"30m"` `"1h"` |
+| `getCandles` / `addExchangeSchema` | `interval` | `"1m"` `"3m"` `"5m"` `"15m"` `"30m"` `"1h"` `"2h"` `"4h"` `"6h"` `"8h"` |
+
+**Rule:** `frameSchema.interval` controls tick granularity; `strategySchema.interval` throttles how often `getSignal` is called. Strategy interval must be ≥ frame interval.
+
+---
+
+## Exchange Schema — Binance via CCXT
+
+Full boilerplate with candles + aggregated trades:
+
+```ts
+import { singleshot } from "functools-kit";
+import ccxt from "ccxt";
+
+const getExchange = singleshot(async () => {
+  const exchange = new ccxt.binance({
+    options: {
+      defaultType: "spot",
+      adjustForTimeDifference: true,
+      recvWindow: 60000,
+    },
+    enableRateLimit: true,
+  });
+  await exchange.loadMarkets();
+  return exchange;
+});
+
+addExchangeSchema({
+  exchangeName: "ccxt-exchange",
+  getCandles: async (symbol, interval, since, limit) => {
+    const exchange = await getExchange();
+    const candles = await exchange.fetchOHLCV(symbol, interval, since.getTime(), limit);
+    return candles.map(([timestamp, open, high, low, close, volume]) => ({
+      timestamp, open, high, low, close, volume,
+    }));
+  },
+  getAggregatedTrades: async (symbol, from, to) => {
+    const exchange = await getExchange();
+    const response = await exchange.publicGetAggTrades({
+      symbol,
+      startTime: from.getTime(),
+      endTime:   to.getTime(),
+    });
+    return response.map((t: any) => ({
+      id:           String(t.a),
+      price:        parseFloat(t.p),
+      qty:          parseFloat(t.q),
+      timestamp:    t.T,
+      isBuyerMaker: t.m,
+    }));
+  },
+});
+```
+
+Notes:
+- `singleshot` from `functools-kit` memoizes the exchange init — called once, reused everywhere
+- `getAggregatedTrades` is optional; only needed if strategy uses `volume-anomaly`
+- In backtest mode the `backtest: boolean` flag is passed as 4th arg — use it to switch between live API and cached data
+
+---
+
+## Risk Schema
+
+Filters that run before a signal is accepted. If any validation returns a rejection string, the signal is blocked.
+
+```ts
+import { addRiskSchema } from "backtest-kit";
+
+addRiskSchema({
+  riskName: "max_positions",
+  validations: [
+    {
+      note: "Allow max 1 active position",
+      validate: ({ activePositionCount }) => {
+        if (activePositionCount >= 1) return "Too many open positions";
+        return null; // null = allowed
+      },
+    },
+  ],
+});
+
+// Attach to strategy:
+addStrategySchema({
+  strategyName: "my_strategy",
+  interval: "15m",
+  riskName: "max_positions",  // single risk profile
+  getSignal: ...,
+});
+// or multiple:
+// riskList: ["max_positions", "funding_filter"]
+```
+
+`IRiskValidationPayload` fields available in `validate`:
+
+| Field | Type | Description |
+|---|---|---|
+| `currentSignal` | `IRiskSignalRow` | Signal about to open |
+| `activePositionCount` | `number` | Currently open positions |
+| `activePositions` | `IRiskActivePosition[]` | Details of each open position |
+| `currentPrice` | `number` | Current market price |
+| `symbol` | `string` | Trading pair |
+| `backtest` | `boolean` | Whether running in backtest mode |
+
+---
+
+## PNL Calculation
+
+Transaction costs are automatically applied:
+- Slippage: **0.1%** per side
+- Fee: **0.1%** per side
+- Round-trip cost: **0.4%**
+
+`priceOpen` and `priceClose` in `IStrategyPnL` are **adjusted** (slippage + fee already subtracted). `pnlPercentage` is the net result.
+
+Minimum `movePercent` to cover costs: `> 0.4%`. Recommended filter: `> 0.7%` to leave margin.
+
+---
+
+## File Naming Convention
+
+Strategy files must be importable by the CLI. Use either:
+- `content/my_strategy.ts` (TypeScript, requires compilation)
+- `strategies/my_strategy/index.mjs` (compiled ESM)
+
+The CLI entry point from `package.json`:
+```
+npx @backtest-kit/cli ./strategies/feb_2024/index.mjs --backtest --ui --noCache
+```

package/template/project/docs/pine_debug.md

@@ -0,0 +1,174 @@
+# Pine Script Debugging Guide
+
+## Context
+
+This project runs Pine Script indicators via `@backtest-kit/pinets` against real exchange data fetched through `ccxt`. Use `@backtest-kit/cli --pine` to execute any `.pine` file and dump results to a JSONL file for inspection.
+
+Key files:
+- `math/*.pine` — Pine Script indicators
+
+---
+
+## Debug Workflow
+
+### Step 1 — Add debug plots to Pine
+
+Add named `plot()` calls for internal variables you want to inspect. Use `display=display.data_window` to mark bot-facing outputs (convention only — the CLI captures all named plots regardless):
+
+```pine
+// === OUTPUTS FOR BOT ===
+plot(close,         "Close",          display=display.data_window)
+plot(active_signal, "Signal",         display=display.data_window)
+
+// === DEBUG ===
+plot(ema_fast,             "EmaFast",        display=display.data_window)
+plot(ema_slow,             "EmaSlow",        display=display.data_window)
+plot(ema_fast - ema_slow,  "EmaGap",         display=display.data_window)
+plot(bars_since_signal,    "BarsSinceSignal", display=display.data_window)
+plot(last_signal,          "LastSignal",      display=display.data_window)
+```
+
+### Step 2 — Run and dump to JSONL
+
+Use `--jsonl` to write output to a file instead of stdout. JSONL is preferred over Markdown for large outputs — each row is a self-contained JSON object, so an AI agent can read only the rows it needs without loading the full table into context.
+
+Output is written to `<pine-dir>/dump/<output>.jsonl` — the directory is created automatically. By default `<output>` equals the `.pine` file name (without extension). Override with `--output`.
+
+```bash
+npx @backtest-kit/cli --pine ./math/my_indicator.pine \
+  --symbol BTCUSDT \
+  --timeframe 15m \
+  --limit 180 \
+  --when "2025-09-24T12:00:00.000Z" \
+  --jsonl
+# → ./math/dump/my_indicator.jsonl
+```
+
+Override the output name:
+
+```bash
+npx @backtest-kit/cli --pine ./math/my_indicator.pine \
+  --jsonl \
+  --output debug
+# → ./math/dump/debug.jsonl
+```
+
+Or add to `package.json`:
+
+```json
+{
+  "scripts": {
+    "pine:debug": "npx @backtest-kit/cli --pine ./math/my_indicator.pine --symbol BTCUSDT --timeframe 15m --limit 180 --jsonl"
+  }
+}
+```
+
+```bash
+npm run pine:debug
+```
+
+### Step 3 — Read the JSONL file
+
+Each line is one bar:
+
+```jsonl
+{"Close":112871.28,"EmaFast":112500.10,"EmaGap":123.45,"BarsSinceSignal":0,"LastSignal":1,"Signal":1,"timestamp":"2025-09-22T15:00:00.000Z"}
+{"Close":112666.69,"EmaFast":112480.55,"EmaGap":98.12,"BarsSinceSignal":1,"LastSignal":1,"Signal":1,"timestamp":"2025-09-22T15:15:00.000Z"}
+```
+
+Scan rows where `BarsSinceSignal == 0` to find signal transitions — that's where the crossover fired.
+
+---
+
+## CLI Flags
+
+| Flag | Type | Description |
+|------|------|-------------|
+| `--pine` | boolean | Enable PineScript execution mode |
+| `--symbol` | string | Trading pair (default: `"BTCUSDT"`) |
+| `--timeframe` | string | Candle interval (default: `"15m"`) |
+| `--limit` | string | Number of candles to fetch (default: `250`) |
+| `--when` | string | End date for candle window — ISO 8601 or Unix ms (default: now) |
+| `--exchange` | string | Exchange name (default: first registered, falls back to CCXT Binance) |
+| `--output` | string | Output file base name without extension (default: `.pine` file name) |
+| `--jsonl` | boolean | Write plots as JSONL (one row per line) to `<pine-dir>/dump/{output}.jsonl` — **preferred for debug** |
+| `--json` | boolean | Write plots as a JSON array to `<pine-dir>/dump/{output}.json` |
+| `--markdown` | boolean | Write Markdown table to `<pine-dir>/dump/{output}.md` |
+
+**Important:** `limit` must cover indicator warmup bars — rows before warmup completes will show `N/A`.
+
+---
+
+## Exchange Configuration (pine.module)
+
+By default the CLI registers CCXT Binance automatically — no setup needed for Binance spot.
+
+To use a different exchange, create `modules/pine.module.ts` next to the `.pine` file (or at project root as fallback):
+
+```
+math/
+├── my_indicator.pine
+└── modules/
+    └── pine.module.ts    ← loaded automatically before running
+```
+
+```typescript
+// modules/pine.module.ts
+import { addExchangeSchema } from "backtest-kit";
+import ccxt from "ccxt";
+
+addExchangeSchema({
+  exchangeName: "my-exchange",
+  getCandles: async (symbol, interval, since, limit) => {
+    const exchange = new ccxt.bybit({ enableRateLimit: true });
+    const ohlcv = await exchange.fetchOHLCV(symbol, interval, since.getTime(), limit);
+    return ohlcv.map(([timestamp, open, high, low, close, volume]) => ({
+      timestamp, open, high, low, close, volume,
+    }));
+  },
+});
+```
+
+Then pass `--exchange my-exchange` to the CLI.
+
+---
+
+## Common Patterns
+
+### N/A values
+EMA of length N returns `N/A` for the first `N-1` bars — this is expected warmup behavior. `EmaFast` (len=8) starts at bar 8, `EmaSlow` (len=21) starts at bar 21.
+
+### Diagnosing whipsaw
+Look at `EmaGap` at the moment `LastSignal` changes (i.e. `BarsSinceSignal == 0`). If `|EmaGap|` is small (e.g. < 15), the crossover happened in a flat/noisy zone — likely a false signal.
+
+### Diagnosing stale signals
+`active_signal` goes to 0 when `bars_since_signal > signal_valid_bars`. If `Signal` is 0 but `LastSignal` is non-zero, the signal expired. Increase `signal_valid_bars` or check why the crossover didn't sustain.
+
+---
+
+## Adding Filters
+
+Filters go into the entry condition expressions:
+
+```pine
+min_gap = input.float(15.0, "Min EMA Gap Filter", minval=0.0)
+
+long_cond  = ta.crossover(ema_fast, ema_slow)  and math.abs(ema_gap) >= min_gap
+short_cond = ta.crossunder(ema_fast, ema_slow) and math.abs(ema_gap) >= min_gap
+```
+
+Add the filter threshold as a debug plot, then check in JSONL: when `BarsSinceSignal` resets to 0, does `EmaGap` meet the threshold?
+
+---
+
+## Pine Variables Worth Plotting for Debug
+
+| Variable | Purpose |
+|---|---|
+| `ema_fast - ema_slow` | Gap magnitude — key for noise filtering |
+| `bars_since_signal` | How many bars since last crossover |
+| `last_signal` | Raw last direction (1/-1/0), ignores expiry |
+| `active_signal` | Final output after expiry window |
+| `ta.rsi(close, 14)` | Momentum context |
+| `ta.atr(14)` | Volatility context for dynamic thresholds |
+| `volume` | Volume spike confirmation |

package/template/project/docs/pine_indicator_warmup.md

@@ -0,0 +1,88 @@
+# Pine Script Warmup & Limit Calculation
+
+## The Problem
+
+Pine functions that look back over N bars (`ta.supertrend`, `ta.ema`, `ta.highest`, etc.) return `na` / `null` until they have enough bars to compute. If `limit` is too small, the entire output is N/A.
+
+---
+
+## Warmup Formula
+
+```
+warmup_bars = max_lookback_period + any_secondary_lookback
+```
+
+Examples:
+
+```
+// EMA golden cross
+warmup = max(ema_slow_len=21) = 21 bars
+
+// MasterTrend
+warmup = atrPeriod(15) + confirmBars(15) = 30 bars
+```
+
+Rule: `limit` must be at least `warmup + desired_output_bars`.
+
+For a meaningful output of ~150 bars:
+```
+limit = warmup + 150
+```
+
+---
+
+## Timeframe → Limit Reference
+
+Warmup for `master_trend_15m.pine` (atrPeriod=15, confirmBars=15 → warmup=30):
+
+| Timeframe | Min limit (warmup only) | Limit for ~150 output bars | Real time covered |
+|---|---|---|---|
+| 5m  | 30 | 180 | ~15h |
+| **15m** | 30 | **180** | **~45h** |
+| 1h  | 30 | 180 | ~7.5 days |
+
+---
+
+## Diagnosing N/A Output
+
+Symptom: all columns except `Close` show `N/A` from bar 0.
+
+Check: count warmup bars in output — the index of the first non-null value.
+
+```js
+const posData = plots["Position"].data;
+const firstValid = posData.findIndex((d) => d.value !== null && !isNaN(d.value));
+console.log("Warmup bars:", firstValid);
+// if firstValid === posData.length → limit too small, zero valid output
+```
+
+If `firstValid === posData.length` → increase `limit` by at least `warmup - limit + desired_output`.
+
+---
+
+## Stacked Lookbacks
+
+When multiple lookback functions are chained, warmup is additive:
+
+```pine
+[supertrend, direction] = ta.supertrend(factor, atrPeriod)  // needs atrPeriod bars
+// confirmBars counter needs confirmBars more bars after supertrend is valid
+// total warmup = atrPeriod + confirmBars = 15 + 15 = 30
+```
+
+`ta.supertrend` itself uses ATR internally, so warmup = atrPeriod. The `confirmBars` confirmation window adds another `confirmBars` bars before `trend` stabilizes.
+
+---
+
+## Pine inputs cannot be set via run()
+
+The `inputs` parameter in `run()` is silently ignored — Pine `input.int()` / `input.float()` defaults are always used.
+
+To test different period values, change the defaults in the `.pine` file directly:
+
+```pine
+// change this line in the .pine file
+atrPeriod = input(15, "ATR Period")  // → input(20, "ATR Period")
+```
+
+This means warmup changes when tuning params — recalculate `limit` after any period change.

package/template/project/gitignore.mustache

@@ -0,0 +1,30 @@
+# Dependencies
+node_modules
+
+# Environment variables
+.env
+.env.local
+.env.*.local
+
+# Logs
+logs
+*.log
+npm-debug.log*
+error.txt
+
+# Runtime data
+tmp
+generated
+dump
+wwwroot
+
+# OS files
+.DS_Store
+Thumbs.db
+
+# IDE
+.vscode
+.idea
+*.swp
+*.swo
+*~