dexbot 1.6.14 → 1.6.16

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (134) hide show
  1. package/CHANGELOG.md +60 -4
  2. package/README.md +4 -4
  3. package/analysis/README.md +6 -6
  4. package/analysis/tradingview/README.md +2 -1
  5. package/claw/README.md +7 -7
  6. package/claw/docs/DEXBOT2_TUNING_CHEAT_SHEET.md +3 -2
  7. package/claw/modules/chain_queries.ts +3 -3
  8. package/claw/package.json +1 -1
  9. package/claw/runtimes/openclaw-plugin/openclaw.plugin.json +1 -1
  10. package/claw/runtimes/openclaw-plugin/package.json +1 -1
  11. package/dist/analysis/ama_fitting/fetch_lp_candles.js +8 -1
  12. package/dist/analysis/ama_fitting/optimizer_high_resolution.d.ts +50 -1
  13. package/dist/analysis/ama_fitting/optimizer_high_resolution.js +33 -7
  14. package/dist/analysis/bot_fitting/backtest_bot_fitting.d.ts +8 -6
  15. package/dist/analysis/bot_fitting/backtest_bot_fitting.js +8 -10
  16. package/dist/analysis/bot_key_utils.js +9 -7
  17. package/dist/analysis/chart_utils.d.ts +9 -2
  18. package/dist/analysis/chart_utils.js +51 -1
  19. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.d.ts +2 -1
  20. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.js +99 -11
  21. package/dist/analysis/trend_detection/dynamic_weight_chart_generator.js +57 -4
  22. package/dist/analysis/trend_detection/kalman_chart_generator.js +1 -0
  23. package/dist/analysis/trend_detection/regime_chart_generator.js +1 -1
  24. package/dist/analysis/trend_detection/volatility_chart_generator.js +1 -0
  25. package/dist/dexbot.js +112 -141
  26. package/dist/market_adapter/core/strategies/dynamic_weight_series.js +26 -11
  27. package/dist/market_adapter/inputs/fetch_book_data.js +36 -12
  28. package/dist/market_adapter/inputs/fetch_lp_data.d.ts +6 -0
  29. package/dist/market_adapter/inputs/fetch_lp_data.js +65 -19
  30. package/dist/market_adapter/inputs/kibana_feed_source.js +45 -12
  31. package/dist/market_adapter/inputs/window_cache.d.ts +14 -1
  32. package/dist/market_adapter/inputs/window_cache.js +53 -17
  33. package/dist/market_adapter/lp_chart_core.js +1 -0
  34. package/dist/market_adapter/utils/chain.js +4 -3
  35. package/dist/market_adapter/utils/pair_orientation.d.ts +60 -0
  36. package/dist/market_adapter/utils/pair_orientation.js +142 -0
  37. package/dist/modules/account_bots.d.ts +40 -3
  38. package/dist/modules/account_bots.js +80 -37
  39. package/dist/modules/account_orders.d.ts +4 -1
  40. package/dist/modules/account_orders.js +4 -1
  41. package/dist/modules/bitshares-native/resolvers.js +3 -3
  42. package/dist/modules/bitshares-native/subscriptions.js +1 -1
  43. package/dist/modules/bitshares_client.d.ts +12 -10
  44. package/dist/modules/bitshares_client.js +12 -10
  45. package/dist/modules/bot_defaults.d.ts +13 -2
  46. package/dist/modules/bot_defaults.js +31 -2
  47. package/dist/modules/bot_settings.js +6 -3
  48. package/dist/modules/chain_keys.d.ts +5 -1
  49. package/dist/modules/chain_keys.js +5 -1
  50. package/dist/modules/chain_orders.d.ts +6 -2
  51. package/dist/modules/chain_orders.js +6 -2
  52. package/dist/modules/constants.d.ts +11 -0
  53. package/dist/modules/constants.js +31 -1
  54. package/dist/modules/credit_runtime.d.ts +35 -5
  55. package/dist/modules/credit_runtime.js +219 -38
  56. package/dist/modules/dexbot_class.d.ts +22 -2
  57. package/dist/modules/dexbot_class.js +61 -21
  58. package/dist/modules/dexbot_cow_runtime.js +19 -18
  59. package/dist/modules/dexbot_fill_runtime.d.ts +1 -1
  60. package/dist/modules/dexbot_fill_runtime.js +4 -7
  61. package/dist/modules/dexbot_maintenance_runtime.d.ts +7 -2
  62. package/dist/modules/dexbot_maintenance_runtime.js +52 -7
  63. package/dist/modules/dexbot_startup_runtime.js +3 -2
  64. package/dist/modules/dexbot_state_recovery.js +1 -1
  65. package/dist/modules/graceful_shutdown.d.ts +6 -2
  66. package/dist/modules/graceful_shutdown.js +6 -2
  67. package/dist/modules/launcher/bot_supervisor.js +4 -0
  68. package/dist/modules/launcher/launch_modes.js +6 -3
  69. package/dist/modules/launcher/monolithic_runtime.d.ts +39 -1
  70. package/dist/modules/launcher/monolithic_runtime.js +60 -1
  71. package/dist/modules/launcher/runtime_mode.d.ts +79 -0
  72. package/dist/modules/launcher/runtime_mode.js +223 -0
  73. package/dist/modules/market_adapter_whitelist.d.ts +8 -0
  74. package/dist/modules/market_adapter_whitelist.js +8 -0
  75. package/dist/modules/order/accounting.d.ts +15 -1
  76. package/dist/modules/order/accounting.js +15 -1
  77. package/dist/modules/order/async_lock.d.ts +9 -3
  78. package/dist/modules/order/async_lock.js +9 -3
  79. package/dist/modules/order/export.d.ts +0 -5
  80. package/dist/modules/order/export.js +2 -7
  81. package/dist/modules/order/format.d.ts +6 -4
  82. package/dist/modules/order/format.js +6 -4
  83. package/dist/modules/order/grid.d.ts +147 -19
  84. package/dist/modules/order/grid.js +321 -68
  85. package/dist/modules/order/grid_reconcile.js +4 -3
  86. package/dist/modules/order/grid_reconcile_internal.js +13 -18
  87. package/dist/modules/order/index.d.ts +4 -2
  88. package/dist/modules/order/index.js +1 -1
  89. package/dist/modules/order/logger_state.d.ts +1 -1
  90. package/dist/modules/order/logger_state.js +1 -1
  91. package/dist/modules/order/manager.d.ts +1 -0
  92. package/dist/modules/order/manager.js +5 -14
  93. package/dist/modules/order/sync_engine.d.ts +26 -11
  94. package/dist/modules/order/sync_engine.js +44 -20
  95. package/dist/modules/order/utils/math.d.ts +157 -47
  96. package/dist/modules/order/utils/math.js +236 -78
  97. package/dist/modules/order/utils/order.d.ts +64 -20
  98. package/dist/modules/order/utils/order.js +80 -24
  99. package/dist/modules/order/utils/system.d.ts +41 -11
  100. package/dist/modules/order/utils/system.js +43 -13
  101. package/dist/modules/order/utils/validate.d.ts +6 -3
  102. package/dist/modules/order/utils/validate.js +53 -60
  103. package/dist/modules/types.d.ts +7 -0
  104. package/dist/modules/utils/asset_symbols.d.ts +5 -5
  105. package/dist/modules/utils/asset_symbols.js +5 -5
  106. package/dist/modules/utils/errors.d.ts +2 -1
  107. package/dist/modules/utils/errors.js +2 -1
  108. package/dist/scripts/analyze-credit.js +44 -27
  109. package/dist/scripts/diagnose-pool-history.js +35 -30
  110. package/dist/scripts/pnl.js +1 -1
  111. package/dist/scripts/rich_list.d.ts +50 -0
  112. package/dist/scripts/rich_list.js +469 -0
  113. package/dist/scripts/run-tests.js +80 -4
  114. package/dist/unlock.d.ts +2 -1
  115. package/dist/unlock.js +53 -21
  116. package/docs/BITSHARES_ONBOARDING.md +7 -0
  117. package/docs/COW_INVARIANTS.md +7 -1
  118. package/docs/DEXBOT_COMPARISON.md +6 -6
  119. package/docs/EVOLUTION.md +11 -9
  120. package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md +3 -3
  121. package/docs/GRID_RECALCULATION.md +18 -6
  122. package/docs/LIFECYCLE.md +8 -8
  123. package/docs/LOGGING.md +16 -1
  124. package/docs/MPA_CREDIT_USAGE.md +10 -5
  125. package/docs/ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md +163 -13
  126. package/docs/README.md +1 -1
  127. package/docs/WORKFLOW.md +6 -5
  128. package/docs/architecture.md +56 -2
  129. package/docs/developer_guide.md +5 -1
  130. package/market_adapter/README.md +14 -7
  131. package/modules/README.md +1 -1
  132. package/package.json +4 -1
  133. package/scripts/README.md +59 -32
  134. package/tests/README.md +40 -2
package/scripts/README.md CHANGED
@@ -111,27 +111,9 @@ bash scripts/reset-settings.sh
111
111
 
112
112
  ## 📊 DIAGNOSTICS & VALIDATION
113
113
 
114
- ### Configuration Audit
115
- **File:** `validate_bots.ts`
116
- **Purpose:** Check `bots.json` for schema errors or missing required fields.
117
- ```bash
118
- # Validate the live bot configuration
119
- node dist/scripts/validate_bots.js
120
- ```
121
-
122
- ### Grid Divergence Audit
123
- **File:** `divergence-calc.ts`
124
- **Purpose:** Measure the "drift" between in-memory grid and disk state using RMS divergence metric.
125
- ```bash
126
- # Calculates RMS Error (hardcoded threshold: 1 promille ≈ 3.2% avg error)
127
- # RMS quadratically penalizes large errors - see docs/GRID_RECALCULATION.md
128
- node dist/scripts/divergence-calc.js
129
- ```
130
- **Reference:** RMS threshold interpretation in [Grid Recalculation docs](../docs/GRID_RECALCULATION.md)
131
-
132
- ### Grid Trading Analysis
133
- **File:** `analyze-orders.ts`
134
- **Purpose:** Analyze grid trading metrics and order distribution patterns.
114
+ ### Grid Order Analysis
115
+ **File:** `analyze-orders.ts` (CLI: `dexbot order`)
116
+ **Purpose:** Analyze grid order metrics and order distribution patterns.
135
117
  ```bash
136
118
  # Analyzes spread accuracy, geometric consistency, and fund distribution
137
119
  node dist/scripts/analyze-orders.js
@@ -155,24 +137,49 @@ node dist/scripts/analyze-credit.js <bot>
155
137
  ```
156
138
  Credit deals report per-pair and average CR, but only for pairs whitelisted in `bots.json` (`debtPolicy.lending`) and listed on the current credit offer; everything else is shown as ignored/unpriced and excluded from the average. A borrow-now preview per configured pair shows today's rate.
157
139
 
158
- ### Pool History Diagnostics
159
- **File:** `diagnose-pool-history.ts`
160
- **Purpose:** Inspect raw BitShares pool history API responses.
140
+ ### Asset Rich List / Top Holder
141
+ **File:** `rich_list.ts` (CLI: `dexbot rich`, npm: `npm run richlist`)
142
+ **Purpose:** Report the top holder(s) per asset using the chain's `asset_api`
143
+ (`get_asset_holders`). Holders come back balance-descending, so this is a live
144
+ rich list. Nodes must expose `asset_api` to anonymous logins; the script tries
145
+ `dex.iobanker.com` / `api.dex.trading` first and falls back through
146
+ `DEFAULT_NODES`.
161
147
  ```bash
162
- # Inspect pool history (default pool 1.19.133)
163
- node dist/scripts/diagnose-pool-history.js
148
+ # Rich list for one asset (default: node holder limit, usually 100)
149
+ dexbot rich <asset>
164
150
 
165
- # Custom pool, limit, and time range
166
- node dist/scripts/diagnose-pool-history.js --pool 1.19.x --limit 100 --hours 48 --maxPages 5
151
+ # Top 10 for several assets
152
+ dexbot rich <asset-a> <asset-b> <asset-c> --top 10
153
+
154
+ # Machine-readable
155
+ dexbot rich <asset-a> <asset-b> --json > rich.json
167
156
  ```
157
+ Balances reflect only liquid account balances, so they do not sum to total supply.
158
+ `--top` is clamped to the node's reported `api_limit_get_asset_holders` (shown
159
+ as a note when clamped). Legacy `bit*` MPA names were renamed on-chain without
160
+ the `bit` prefix; the script retries an unresolved `bit*` name stripped and notes
161
+ the rename. Note:
162
+ `get_asset_holders_count` is reported by the node as `distance(range) - 1`, so
163
+ the `holders=` figure can be off by one.
168
164
 
169
- ### Print Grid Sample
170
- **File:** `print_grid.ts`
171
- **Purpose:** Demonstrate the grid structure — shows consecutive price levels with percentage differences between adjacent slots.
165
+ ### Configuration Audit
166
+ **File:** `validate_bots.ts`
167
+ **Purpose:** Check `bots.json` for schema errors or missing required fields.
172
168
  ```bash
173
- node dist/scripts/print_grid.js
169
+ # Validate the live bot configuration
170
+ node dist/scripts/validate_bots.js
174
171
  ```
175
172
 
173
+ ### Grid Divergence Audit
174
+ **File:** `divergence-calc.ts`
175
+ **Purpose:** Measure the "drift" between in-memory grid and disk state using RMS divergence metric.
176
+ ```bash
177
+ # Calculates RMS Error (hardcoded threshold: 1 promille ≈ 3.2% avg error)
178
+ # RMS quadratically penalizes large errors - see docs/GRID_RECALCULATION.md
179
+ node dist/scripts/divergence-calc.js
180
+ ```
181
+ **Reference:** RMS threshold interpretation in [Grid Recalculation docs](../docs/GRID_RECALCULATION.md)
182
+
176
183
  ### Grid Calculation Runner
177
184
  **File:** `runner.ts`
178
185
  **Purpose:** Standalone order grid calculation debugger — loads a bot config, initializes the grid, and simulates sync cycles.
@@ -186,6 +193,25 @@ LIVE_BOT_NAME=my-bot CALC_CYCLES=10 CALC_DELAY_MS=1000 node dist/scripts/runner.
186
193
  Useful for verifying config produces the expected grid, testing price derivation, and debugging fund allocation.
187
194
  Requires a live BitShares connection (asset metadata lookups and price derivation are on-chain).
188
195
 
196
+ ### Print Grid Sample
197
+ **File:** `print_grid.ts`
198
+ **Purpose:** Demonstrate the grid structure — shows consecutive price levels with percentage differences between adjacent slots.
199
+ ```bash
200
+ node dist/scripts/print_grid.js
201
+ ```
202
+
203
+ ### Pool History Diagnostics
204
+ **File:** `diagnose-pool-history.ts`
205
+ **Purpose:** Inspect raw BitShares pool history API responses.
206
+ ```bash
207
+ # Inspect pool history (default pool 1.19.133)
208
+ node dist/scripts/diagnose-pool-history.js
209
+
210
+ # Custom pool, limit, and time range
211
+ node dist/scripts/diagnose-pool-history.js --pool 1.19.133 --limit 100 --hours 48 --maxPages 5
212
+ ```
213
+ `--pool` must be a concrete numeric liquidity-pool object ID (`1.19.N`). Placeholders such as `1.19.x` are rejected up front, because the node parses the instance part as a `uint64` and otherwise fails with `Parse Error: Couldn't parse uint64_t`.
214
+
189
215
  ### Native Release Gates
190
216
  **File:** `native_release_gates.ts` (+ `generate_mainnet_corpus_report.ts`)
191
217
  **Purpose:** Prove the native serialization layer matches the chain byte-for-byte before release.
@@ -374,6 +400,7 @@ The following scripts allow you to call `dexbot` commands directly from the `scr
374
400
  | `npm run analysis:trade-pnl` | Trade PnL analysis from fill data |
375
401
  | `npm run analysis:grid-check` | Last-fill-guard check (`grid_correction_check.ts`) |
376
402
  | `npm run analysis:resolve-accounts` | Batch-resolve on-chain bot accounts (`resolve_bot_accounts.ts`) |
403
+ | `npm run richlist` | Per-asset rich list / top holder (`rich_list.ts`) |
377
404
  | `npm run ama:chart:lp-local` | Generate local LP comparison chart |
378
405
  | `npm run lp:chart` | Generate uPlot LP chart |
379
406
  | `npm run test:credit-renewal` | Test credit offer renewal for a specific bot |
package/tests/README.md CHANGED
@@ -46,7 +46,7 @@ claw/tests/ # Claw integration tests (same conventions, run by npm test)
46
46
  `scripts/run-tests.ts` runs each compiled `test_*.js` file as its own `node` process (no shared state between tests). Knobs:
47
47
 
48
48
  - `RUN_LIVE_BITSHARES_TESTS=1` — also run the live-chain tests; without it they are skipped and counted as `skippedLive`. The live set is the `liveTestFiles` allow-list in `scripts/run-tests.ts`.
49
- - `DEXBOT_TEST_CONCURRENCY=N` — run N files in parallel (default `1`; several tests share `tests/tmp` paths, so parallelism must be requested deliberately).
49
+ - `DEXBOT_TEST_CONCURRENCY=N` — run N files in parallel (default `min(cores, 4)`; files are isolated with per-PID `tests/tmp` scratch space, so bounded parallelism is safe). Set `DEXBOT_TEST_CONCURRENCY=1` to force the old sequential order, which is useful when debugging an interleaved log.
50
50
  - `DEXBOT_TEST_TIMEOUT_MS=N` — per-file watchdog (default `240000`); on expiry the child is `SIGTERM`/`SIGKILL`ed and marked `TIMEOUT`.
51
51
  - `DEXBOT_SUPPRESS_WARNINGS=1` — add `--no-warnings` to children (hides the warnings the diagnostics summary is meant to surface).
52
52
 
@@ -108,9 +108,38 @@ PM2 lifecycle, startup ordering, bot supervision, launcher version reporting.
108
108
  Interactive tools and performance benchmarks (not part of CI).
109
109
  *Examples:* `connection_test.ts`, `diag_adapter_client.ts`, `diag_ws_nodes.ts`, `diag_ws_lifecycle.ts`, `benchmark_cow.ts`, `sim_batching.ts`, `repro_phantom_orders.ts`
110
110
 
111
+ ### Browser Charts (Playwright)
112
+ End-to-end tests for the generated HTML charts (currently the `dexbot tv`
113
+ TradingView exporter). These load a real chart page in Firefox and assert on
114
+ canvas draw work and pixels, so they live under `tests/playwright/` as
115
+ `.spec.mjs` files and are **not** picked up by the plain `node` runner.
116
+
117
+ ```bash
118
+ npm run test:playwright # builds, then runs tests/playwright
119
+ ```
120
+
121
+ The runner and browsers come from a global Playwright install (the repo keeps
122
+ zero runtime deps); `test:playwright` points `NODE_PATH` at `npm root -g`, and
123
+ the spec resolves `playwright/test` dynamically. That inline `NODE_PATH=` is
124
+ POSIX-shell syntax (Linux/macOS); on Windows set it in your shell
125
+ (`$env:NODE_PATH=(npm root -g)`) or add a local `playwright` devDependency.
126
+ **Firefox is the default target** (set in
127
+ `tests/playwright/playwright.config.mjs`) because that is the browser the
128
+ generated charts are hand-checked in. If Playwright or the Firefox build is
129
+ missing: `npm i -g playwright && npx playwright install firefox`.
130
+
131
+ `tv_chart_perf.spec.mjs` pins the range-band draw hook to the visible bar
132
+ window: it counts `CanvasRenderingContext2D.lineTo` calls to prove the band's
133
+ per-redraw cost scales with the viewport (not the full series), and diffs
134
+ canvas pixels to prove the band still spans the plot when zoomed in. By default
135
+ it generates a synthetic TV page; set `TV_CHART_HTML=<path>` to run the load
136
+ check against any already-generated chart HTML (the uPlot interception is
137
+ generic; the Range-band assertions are skipped when the page has no Range
138
+ toggle).
139
+
111
140
  ### Edge Cases & Regression
112
141
  Tests targeting specific bugs, race conditions, and failure modes.
113
- *Examples:* `test_critical_bug_fixes.ts`, `test_race_condition_fixes_batch1.ts`, `test_patch17_invariants.ts`, `test_shutdown_reentrancy.ts`, `test_multifill_opposite_partial.ts`, `test_correction_queue_staleness.ts`, `test_spread_pure_fund_driven.ts`, `test_sync_lock_id_verification.ts`
142
+ *Examples:* `test_critical_bug_fixes.ts`, `test_race_condition_fixes_batch1.ts`, `test_patch17_invariants.ts`, `test_shutdown_reentrancy.ts`, `test_multifill_opposite_partial.ts`, `test_correction_queue_staleness.ts`, `test_spread_pure_fund_driven.ts`, `test_spread_correction_resilience.ts`, `test_sync_lock_id_verification.ts`
114
143
 
115
144
  ### Utilities & Helpers
116
145
  Shared utility functions, precision handling, chain helpers.
@@ -129,6 +158,15 @@ Shared utility functions, precision handling, chain helpers.
129
158
 
130
159
  **Reference:** [docs/COPY_ON_WRITE_MASTER_PLAN.md](../docs/COPY_ON_WRITE_MASTER_PLAN.md)
131
160
 
161
+ ### Spread-Correction Resilience
162
+ - Refusal-cause classification and the transient/non-transient split are centralized (`SPREAD_CORRECTION_REFUSAL`)
163
+ - Plan-time CREATE dedupe withholds a plan whose destination was taken after planning
164
+ - The bounded fresh-master re-plan lands a transiently-refused correction on the retry
165
+ - The broadcast region is held across the plan/retry loop and released exactly once (including on throw)
166
+ - The out-of-spread watchdog tolerates transient refusals only up to its grace budget
167
+
168
+ **Reference:** [docs/architecture.md](../docs/architecture.md) (Spread Correction → Resilience)
169
+
132
170
  ### RMS Divergence Checking
133
171
  - Quadratic penalty for large errors
134
172
  - Concentrated errors raise RMS threshold