dexbot 1.4.19 → 1.4.20

package/CHANGELOG.md

@@ -2,6 +2,19 @@
 
 All notable changes to this project will be documented in this file.
 
+## [1.4.20] - 2026-08-23 - Grid Boundary Hardening, Recovery Poison Gate, Analysis Path Centralization
+
+### 2026-08-22
+
+- **Fix(grid)**: harden boundary promotion against gap-floor violations — the spread-correction promotion path could consume every gap slot in a single call (`maxPromotable` computed the opposite rail's span and constrained nothing; `resolveGapBand` re-derived `sellStartIdx` unconditionally from the mutable committed boundary), so any one-time overrun became permanent geometry with no spread floor. Promotion is now capped at band size − MIN_SPREAD_ORDERS (disabled entirely when band ≤ reserve) and the walk hard-stops reserve slots short of the opposite rail edge independent of quota; upstream depth caps (BUY P_s−G−1−B, SELL B−P_b) prevent the slide from stranding placed opposite-rail orders; new `validateBoundaryCommit()` gates commit-time proposals in `_commitWorkingGrid` (rejected proposals keep the last valid boundary); defense-in-depth detectors add pre-broadcast crossed-book refusal (`detectCrossedBookPlan`) and post-commit gap-band intrusion detection → structural resync; new shared `resolveGapSlots()` dedups the `_gapSlots ?? calculateGapSlots(config)` pattern (`modules/order/grid.ts`, `modules/order/utils/math.ts`, `modules/order/manager.ts`, `modules/dexbot_cow_runtime.ts`, `tests/test_spread_boundary_promotion.ts`, `tests/test_spread_check_orchestration.ts`).
+- **Fix(recovery)**: gate persisted boundary restore against overrun poison — a boundary committed and persisted by the pre-fix promotion overrun would legalize itself on every restart: `resolveGapBand` re-derives `sellStartIdx` from whatever value is restored and no runtime gate ran before restore, so structural resync "recovered" straight back into corrupted geometry. New `validatePersistedBoundary()` (stricter than commit-time validation — also rejects in-band placed orders, the poison signature only position-based geometry detects) is applied in `loadGrid` before `_restoreBoundary`; on rejection the structural center is re-derived from slot prices via `calculateIdealBoundary` (unmappable anchors degrade to a boundary-less load the next sync reconciles), and `recoverFromPersistedGrid` refuses poisoned snapshots BEFORE `loadGrid` so resync falls through to a clean `requestGridReset(refreshCenterPrice)` rebuild using config-derived `calculateGapSlots` (`modules/order/utils/math.ts`, `modules/order/grid.ts`, `modules/dexbot_state_recovery.ts`, new `tests/test_boundary_restore_validation.ts`, `tests/test_uncertain_broadcast.ts` fixtures updated).
+- **Feat(analysis)**: centralize dynamic weight chart slider ranges, kalS% default 1.0 — the interactive research chart hardcoded its 16 knob ranges independently in four places each (HTML range attrs, tooltip text, server-side clamps, paste-clamp code), and the amaS% paste clamp had already drifted from its slider span. New `SLIDER_RANGES` constant is the single source of truth feeding HTML attrs, tooltips, payload transport, and every paste/init/latch clamp; kalS% range 0.15–1.5 → 0.5–1.5 with `DYNAMIC_WEIGHT_KALMAN_MAX_SLOPE_PCT` raised 0.8 → 1.0; fixes a TDZ crash (`const SR` declared after first use threw ReferenceError on chart load, leaving all panels blank) (`analysis/trend_detection/dynamic_weight_chart_generator.ts`, `analysis/trend_detection/DYNAMIC_WEIGHT_RESEARCH.md`, `modules/constants.ts`).
+- **Fix(paths)**: centralize analysis output dirs and npm-install path resolution — research/chart tooling wrote generated artifacts into the package tree via `__dirname`/`import.meta.url`/cwd-relative defaults (wiped by `npm update -g`, blocked on read-only prefixes). New `PATHS.ANALYSIS.{DIR,CHARTS_DIR,RESULTS_DIR,ASSETS_DIR}` resolver keeps repo-local outputs only for repo-layout source checkouts and follows the resolved profiles dir otherwise, with a `DEXBOT_ANALYSIS_DIR` env override and `XDG_CONFIG_HOME` support; all chart generators and optimizer/backtest result writers rerouted (auto-discovery scans RESULTS_DIR first with legacy-dir fallback); `writeChartFile` materializes vendored uPlot assets next to relocated charts so relative refs keep resolving; claw profile/skill-doc resolution aligned with `modules/paths.ts` ordering; shell shim npm-package detection switched to exact-parent basename matching; generated charts/repo-stats excluded from the shipped tarball (`modules/paths.ts`, `modules/config.ts`, `analysis/*`, `scripts/lib/dexbot-paths.sh`, `claw/modules/*`, `package.json`).
+
+### 2026-08-23
+
+- **Fix(market-adapter)**: align dynamic weight clip logic between live service and research chart — the chart dropped zero-slope bars from the AMA clip pool, kept Kalman warmup bars that the live service sliced away, and fed different precision (rounded vs raw) into the velocity smoothing pipeline, so parameter research could mispredict live clipping behavior. The live service now feeds the smoothing pipeline unrounded velocity/displacement percentages and computes the Kalman clip percentile over the full series (no warmup slice); the chart keeps every finite slope including zeros via `Number.isFinite` and builds the payload-level initial smoothed series from raw fields to match the interactive recompute path; CLIP_PCT_MAX knob bound lowered 55 → 20. Note: at clip=10% totals shift ~−0.12% since warmup bars now count toward the percentile pool (`market_adapter/core/market_adapter_service.ts`, `analysis/trend_detection/dynamic_weight_chart_generator.ts`, `tests/test_market_adapter_service.ts`).
+
 ## [1.4.19] - 2026-08-21 - COW Broadcast Op Cap, Grid Divergence Rail Fix, UI Price Feedback
 
 ### 2026-08-21

package/analysis/ama_fitting/analyze_lambda_vs_slow.ts

@@ -186,8 +186,8 @@ function generateChartHtml(results: any, metricCache: any, fixEr: any, fixFast:
     const cacheSlow = metricCache.map((m: any) => m.slow);
     const cacheMove = metricCache.map((m: any) => m.movement);
 
-    const uplotCode = fs.readFileSync(path.resolve(__dirname, '..', 'uplot', 'uPlot.iife.min.js'), 'utf8');
-    const uplotCSS = fs.readFileSync(path.resolve(__dirname, '..', 'uplot', 'uPlot.min.css'), 'utf8');
+    const uplotCode = fs.readFileSync(path.join(PATHS.ANALYSIS.ASSETS_DIR, 'uPlot.iife.min.js'), 'utf8');
+    const uplotCSS = fs.readFileSync(path.join(PATHS.ANALYSIS.ASSETS_DIR, 'uPlot.min.css'), 'utf8');
 
     const amaAnnotations = [
         { label: 'AMA1', lambda: 0.0031, slow: 62.1, color: '#ef5350' },

package/analysis/ama_fitting/optimizer_high_resolution.ts

@@ -719,7 +719,8 @@ async function run() {
     const outName = dataFile
         ? `optimization_results_${path.basename(dataFile, '.json')}${lambdaSuffix}.json`
         : `optimization_results_high_resolution${lambdaSuffix}.json`;
-    const outPath = path.join(__dirname, outName);
+    const outPath = path.join(PATHS.ANALYSIS.RESULTS_DIR, outName);
+    ensureDir(path.dirname(outPath));
     writeJSON(outPath, {
         meta: {
             dataLabel,

package/analysis/ama_fitting/package.json

@@ -1,6 +1,6 @@
 {
   "name": "ama_fitting",
-  "version": "1.4.19",
+  "version": "1.4.20",
   "description": "Tools for fitting AMA parameters to market data",
   "main": "../../dist/analysis/ama_fitting/optimizer_high_resolution.js",
   "scripts": {

package/analysis/analyze_derivatives.ts

@@ -20,10 +20,12 @@
  */
 
 
+import path from 'node:path';
 import { pathToFileURL } from 'node:url';
 import { DerivativeAnalyzer } from './trend_detection/derivative_analyzer.js';
 import { generateHTML } from './derivative_chart_generator.js';
 import { writeChartFile } from './chart_utils.js';
+import { PATHS } from '../modules/paths.js';
 import { resolveSource, listAvailableBots, type SourceConfig } from './resolve_source.js';
 'use strict';
 
@@ -85,7 +87,7 @@ function parseArgs(): CliConfig {
         interpHoldBars: 3,
         rsiZone: 10,
         rsiExtreme: 90,
-        chartFile: 'analysis/charts/derivative_chart.html',
+        chartFile: path.join(PATHS.ANALYSIS.CHARTS_DIR, 'derivative_chart.html'),
         quiet: false,
         listBots: false,
     };
@@ -173,7 +175,7 @@ Analyzer options:
     --file PATH      JSON candle input
 
   Output:
-    --chart FILE   Chart output path (default: analysis/charts/derivative_chart.html)
+    --chart FILE   Chart output path (default: <analysis charts dir>/derivative_chart.html)
     --list-bots    List available bot keys from bots.json
     --quiet        Suppress log output
     `);

package/analysis/analyze_dynamic_weight.ts

@@ -12,6 +12,7 @@
  *     --file market_adapter/data/lp/<path>/<to>/<lp-candles>.json
  */
 
+import path from 'node:path';
 import { KalmanTrendAnalyzer } from './trend_detection/kalman_trend_analyzer.js';
 import { HurstAnalyzer } from './trend_detection/hurst_analyzer.js';
 import { PermutationEntropyAnalyzer } from './trend_detection/permutation_entropy_analyzer.js';
@@ -19,6 +20,7 @@ import { generateHTML } from './trend_detection/dynamic_weight_chart_generator.j
 import { calculateAMA } from '../market_adapter/core/strategies/ama.js';
 import { computeAmaSlopeWeights } from '../market_adapter/core/strategies/ama_slope_model.js';
 import { MARKET_ADAPTER } from '../modules/constants.js';
+import { PATHS } from '../modules/paths.js';
 import { writeChartFile } from './chart_utils.js';
 import { getCandleClose } from './math_utils.js';
 import { resolveSource, listAvailableBots, type SourceConfig } from './resolve_source.js';
@@ -64,7 +66,7 @@ function parseArgs() {
         dispScaleMinPct?: number;
     } = {
         source: { type: 'market_adapter', config: { botKey: '' } },
-        chartFile: 'analysis/charts/dynamic_weight_chart.html',
+        chartFile: path.join(PATHS.ANALYSIS.CHARTS_DIR, 'dynamic_weight_chart.html'),
         alpha: MARKET_ADAPTER.DYNAMIC_WEIGHT_ALPHA,
         gain: MARKET_ADAPTER.DYNAMIC_WEIGHT_GAIN,
         dispWeight: MARKET_ADAPTER.DYNAMIC_WEIGHT_DW,

package/analysis/analyze_kalman.ts

@@ -11,12 +11,14 @@
  *     --file market_adapter/data/lp/<path>/<to>/<lp-candles>.json
  */
 
+import path from 'node:path';
 import { KalmanTrendAnalyzer } from './trend_detection/kalman_trend_analyzer.js';
 import { generateHTML } from './trend_detection/kalman_chart_generator.js';
 import { calculateAMA } from '../market_adapter/core/strategies/ama.js';
 import { computeAverageAmaSlopePct } from '../market_adapter/core/strategies/dynamic_weight_series.js';
 import { getCandleClose } from './math_utils.js';
 import { writeChartFile } from './chart_utils.js';
+import { PATHS } from '../modules/paths.js';
 import { resolveSource, listAvailableBots, type SourceConfig } from './resolve_source.js';
 
 'use strict';
@@ -39,7 +41,7 @@ function parseArgs() {
         source: { type: 'market_adapter', config: { botKey: '' } },
         rNoise: 0.05,
         qNoise: 0.005,
-        chartFile: 'analysis/charts/kalman_chart.html',
+        chartFile: path.join(PATHS.ANALYSIS.CHARTS_DIR, 'kalman_chart.html'),
         quiet: false,
         listBots: false,
     };

package/analysis/analyze_regime.ts

@@ -18,7 +18,9 @@
  *     --file market_adapter/data/lp/<path>/<to>/<lp-candles>.json
  */
 
+import path from 'node:path';
 import { MARKET_ADAPTER }              from '../modules/constants.js';
+import { PATHS }                       from '../modules/paths.js';
 import { HurstAnalyzer }               from './trend_detection/hurst_analyzer.js';
 import { PermutationEntropyAnalyzer }  from './trend_detection/permutation_entropy_analyzer.js';
 import { generateRegimeHTML }          from './trend_detection/regime_chart_generator.js';
@@ -44,7 +46,7 @@ function parseArgs() {
         listBots:    boolean;
     } = {
         source:      { type: 'market_adapter', config: { botKey: '' } },
-        chartFile:   'analysis/charts/regime_chart.html',
+        chartFile:   path.join(PATHS.ANALYSIS.CHARTS_DIR, 'regime_chart.html'),
         hurstWindow: HURST_CONFIG.window,
         peWindow:    PE_CONFIG.window,
         peM:         PE_CONFIG.m,

package/analysis/analyze_regime_windows.ts

@@ -17,9 +17,11 @@
  *     --file market_adapter/data/lp/<path>/<to>/<lp-candles>.json
  */
 
+import path from 'node:path';
 import { HurstAnalyzer }              from './trend_detection/hurst_analyzer.js';
 import { PermutationEntropyAnalyzer } from './trend_detection/permutation_entropy_analyzer.js';
 import { MARKET_ADAPTER } from '../modules/constants.js';
+import { PATHS } from '../modules/paths.js';
 import { writeChartFile }            from './chart_utils.js';
 import { getCandleClose }            from './math_utils.js';
 import { roundTo } from '../modules/order/utils/math.js';
@@ -59,7 +61,7 @@ function parseArgs() {
         listBots:  boolean;
     } = {
         source:    { type: 'market_adapter', config: { botKey: '' } },
-        chartFile: 'analysis/charts/regime_windows_heatmap.html',
+        chartFile: path.join(PATHS.ANALYSIS.CHARTS_DIR, 'regime_windows_heatmap.html'),
         quiet:     false,
         listBots:  false,
     };

package/analysis/analyze_volatility.ts

@@ -21,7 +21,6 @@
  */
 
 import path from 'node:path';
-import { fileURLToPath } from 'node:url';
 
 import { calculateAMA } from '../market_adapter/core/strategies/ama.js';
 import { computeATRSeries } from '../market_adapter/core/strategies/atr/calculator.js';
@@ -29,6 +28,7 @@ import { normalizeAtrPeriod } from '../market_adapter/core/config_normalizers.js
 import { computeVolatilityShift } from '../market_adapter/core/strategies/volatility_shift.js';
 import { generateHTML } from './trend_detection/volatility_chart_generator.js';
 import { MARKET_ADAPTER } from '../modules/constants.js';
+import { PATHS } from '../modules/paths.js';
 import { getCandleClose } from './math_utils.js';
 import { writeChartFile } from './chart_utils.js';
 import { resolveSource, listAvailableBots, type SourceConfig } from './resolve_source.js';
@@ -40,7 +40,7 @@ const MIN_WEIGHT = MARKET_ADAPTER.DYNAMIC_WEIGHT_MIN_WEIGHT;
 const MAX_WEIGHT = MARKET_ADAPTER.DYNAMIC_WEIGHT_MAX_WEIGHT;
 const DEFAULT_THRESHOLD = MARKET_ADAPTER.DYNAMIC_WEIGHT_SYMMETRIC_SHIFT_THRESHOLD;
 const DEFAULT_CLAMP = MARKET_ADAPTER.DYNAMIC_WEIGHT_SYMMETRIC_SHIFT_CLAMP;
-const DEFAULT_CHART_DIR = path.join(path.dirname(fileURLToPath(import.meta.url)), 'charts');
+const DEFAULT_CHART_DIR = PATHS.ANALYSIS.CHARTS_DIR;
 const DEFAULT_CHART_FILE = path.join(DEFAULT_CHART_DIR, 'volatility_chart.html');
 
 function computeATRSeriesNormalized(candles: any[], period = DEFAULT_ATR_PERIOD) {

package/analysis/bot_fitting/backtest_ama_sweep.ts

@@ -7,6 +7,7 @@ import { calculateAMA } from '../../market_adapter/core/strategies/ama.js';
 import { range } from '../math_utils.js';
 import { parseListOrRange, loadLpData, fmt } from './shared_utils.js';
 import { getStorage } from '../../modules/storage/index.js';
+import { PATHS } from '../../modules/paths.js';
 const { readJSON, writeJSON } = getStorage();
 'use strict';
 
@@ -759,7 +760,7 @@ async function run() {
 
     // ── Save JSON ───────────────────────────────────────────────────────────
     const outName = `ama_sweep_results_${path.basename(cfg.dataPath!, '.json')}.json`;
-    const outPath = path.join(__dirname, outName);
+    const outPath = path.join(PATHS.ANALYSIS.RESULTS_DIR, outName);
     writeJSON(outPath, {
         meta: {
             generatedAt: new Date().toISOString(),

package/analysis/bot_fitting/backtest_bot_fitting.ts

@@ -4,6 +4,7 @@ import { calculateAMA } from '../../market_adapter/core/strategies/ama.js';
 import { range } from '../math_utils.js';
 import { parseListOrRange, loadLpData, fmt } from './shared_utils.js';
 import { getStorage } from '../../modules/storage/index.js';
+import { PATHS } from '../../modules/paths.js';
 const { ensureDir, readJSON, writeJSON } = getStorage();
 
 'use strict';
@@ -125,28 +126,32 @@ function parseArgs() {
  * The optimizer names results `optimization_results_<base>_w<λ1>_<λ2>_<λ3>_<λ4>.json`
  * (per-AMA distance weights appended as a `_w...` suffix). To stay robust against
  * weight overrides, pick the most recently written `optimization_results_<base>_w*.json`
- * in `ama_fitting/`; fall back to the legacy `optimization_results_<base>.json` name
- * if no suffixed file exists.
+ * in the analysis results dir (legacy ama_fitting location still scanned); fall
+ * back to the legacy `optimization_results_<base>.json` name if no suffixed file exists.
  */
 function resolveAutoResultsPath(dataPath: string): string {
     const base = path.basename(dataPath, '.json');
-    const dir = path.join(__dirname, '..', 'ama_fitting');
+    // Canonical results dir first, then the legacy ama_fitting location so
+    // previously generated files are still discovered.
+    const dirs = [PATHS.ANALYSIS.RESULTS_DIR, path.join(PATHS.PROJECT_ROOT, 'analysis', 'ama_fitting')];
     const prefix = `optimization_results_${base}_w`;
     let newest: { file: string; mtimeMs: number } | null = null;
     try {
-        for (const entry of fs.readdirSync(dir)) {
-            if (!entry.startsWith(prefix) || !entry.endsWith('.json')) continue;
-            const full = path.join(dir, entry);
-            const stat = fs.statSync(full);
-            if (!newest || stat.mtimeMs > newest.mtimeMs) {
-                newest = { file: full, mtimeMs: stat.mtimeMs };
+        for (const dir of dirs) {
+            for (const entry of fs.readdirSync(dir)) {
+                if (!entry.startsWith(prefix) || !entry.endsWith('.json')) continue;
+                const full = path.join(dir, entry);
+                const stat = fs.statSync(full);
+                if (!newest || stat.mtimeMs > newest.mtimeMs) {
+                    newest = { file: full, mtimeMs: stat.mtimeMs };
+                }
             }
         }
     } catch {
         // dir unreadable — fall through to legacy name check
     }
     if (newest) return newest.file;
-    return path.join(dir, `optimization_results_${base}.json`);
+    return path.join(PATHS.ANALYSIS.RESULTS_DIR, `optimization_results_${base}.json`);
 }
 
 function loadAmaStrategies(resultsPath: string) {
@@ -388,8 +393,8 @@ function run() {
     console.log();
 
     const outName = `bot_fitting_results_${path.basename(cfg.dataPath!, '.json')}.json`;
-    const outPath = path.join(__dirname, outName);
-    ensureDir(__dirname);
+    const outPath = path.join(PATHS.ANALYSIS.RESULTS_DIR, outName);
+    ensureDir(PATHS.ANALYSIS.RESULTS_DIR);
     writeJSON(outPath, {
         meta: {
             generatedAt: new Date().toISOString(),

package/analysis/chart_utils.ts

@@ -2,6 +2,7 @@
 import fs from 'node:fs';
 import path from 'node:path';
 import { getStorage } from '../modules/storage/index.js';
+import { PATHS } from '../modules/paths.js';
 const { ensureDir } = getStorage();
 'use strict';
 
@@ -9,6 +10,28 @@ const { ensureDir } = getStorage();
  * Chart utilities for analysis HTML generators.
  */
 
+// Runtime assets every generated chart references as ../uplot/* relative to
+// its own directory (charts live in <root>/charts, assets in <root>/uplot).
+const UPLOT_RUNTIME_FILES = ['uPlot.iife.min.js', 'uPlot.min.css'];
+
+/**
+ * Generated HTML loads uPlot from a sibling `uplot/` dir next to the charts
+ * dir. In a source checkout that dir is the vendored original; anywhere else
+ * (relocated analysis root under npm/home profiles, or a custom --chart path)
+ * materialize a copy so the relative refs keep resolving.
+ */
+function ensureSiblingUplotAssets(chartDir: any) {
+    const targetDir = path.join(path.dirname(path.resolve(chartDir)), 'uplot');
+    const sourceDir = PATHS.ANALYSIS.ASSETS_DIR;
+    if (targetDir === path.resolve(sourceDir)) return;
+    if (UPLOT_RUNTIME_FILES.every((f) => fs.existsSync(path.join(targetDir, f)))) return;
+    if (!fs.existsSync(sourceDir)) return;
+    fs.mkdirSync(targetDir, { recursive: true });
+    for (const f of UPLOT_RUNTIME_FILES) {
+        fs.copyFileSync(path.join(sourceDir, f), path.join(targetDir, f));
+    }
+}
+
 
 function escapeHtml(str: string) {
     const map: Record<string, string> = {
@@ -34,6 +57,7 @@ function toEpochSeconds(ts: any, fallbackIdx: any) {
 function writeChartFile(filePath: any, html: any) {
     const chartDir = path.dirname(filePath);
     if (!fs.existsSync(chartDir)) ensureDir(chartDir);
+    ensureSiblingUplotAssets(chartDir);
     fs.writeFileSync(filePath, html, 'utf8');
 }
 

package/analysis/derivative_chart_generator.ts

@@ -8,11 +8,12 @@ import { getStorage } from '../modules/storage/index.js';
 const { ensureDir, readJSON } = getStorage();
 import { fixedTo } from '../modules/order/utils/math.js';
 import { bindHoverStateFn, zoomResetScript } from './chart_ui.js';
+import { PATHS } from '../modules/paths.js';
 'use strict';
 function parseArgs(argv = process.argv.slice(2)) {
     const cfg: { inputFile: string | null; outputFile: string; title: string; quiet: boolean } = {
         inputFile: null,
-        outputFile: 'analysis/charts/derivative_chart.html',
+        outputFile: path.join(PATHS.ANALYSIS.CHARTS_DIR, 'derivative_chart.html'),
         title: 'Derivative Trend Analysis',
         quiet: false,
     };
@@ -40,7 +41,7 @@ Derivative Chart Generator (uPlot)
 Usage:
   tsx analysis/derivative_chart_generator.ts --input <file.json> [options]
 Options:
-  --output FILE   Output HTML (default: analysis/charts/derivative_chart.html)
+  --output FILE   Output HTML (default: <analysis charts dir>/derivative_chart.html)
   --title TEXT    Chart title
   --quiet         Suppress output
     `);

package/analysis/trend_detection/DYNAMIC_WEIGHT_RESEARCH.md

@@ -160,7 +160,7 @@ All panels share aligned vertical time grid lines, and the bottom output panel s
 | **nz%** | 0–1 | 0.00 | Neutral zone: dead-band below which offset is forced to 0 |
 | **lb** | 1–32 | 9 | Logarithmic. Lookback bars for AMA slope calculation |
 | **amaS%** | 0.04–0.12 | 0.085 | Logarithmic. Gear ratio for average per-bar AMA slope saturation |
-| **kalS%** | 0.15–1.5 | 0.8 | Logarithmic. Gear ratio for Kalman composite saturation |
+| **kalS%** | 0.5–1.5 | 1.0 | Logarithmic. Gear ratio for Kalman composite saturation |
 | **clip%** | 0–55 | 10 | Percentile clip: filters extreme inputs (0 = off) |
 
 ### Output Controls