dexbot 1.4.20 → 1.4.21

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (234) hide show
  1. package/CHANGELOG.md +23 -0
  2. package/README.md +22 -12
  3. package/analysis/ama_fitting/calibrate_convergence_er.ts +1 -1
  4. package/analysis/ama_fitting/package.json +1 -1
  5. package/analysis/bot_fitting/shared_utils.ts +1 -1
  6. package/analysis/bot_key_utils.ts +1 -5
  7. package/analysis/bot_usage/kibana_bot_queries.ts +1 -256
  8. package/analysis/chart_css.ts +1 -1
  9. package/analysis/math_utils.ts +1 -8
  10. package/analysis/trend_detection/package.json +1 -1
  11. package/claw/ecosystem.config.cjs +2 -3
  12. package/claw/examples/memu_integration_example.ts +17 -17
  13. package/claw/modules/chain_actions.ts +51 -54
  14. package/claw/modules/chain_broadcast.ts +66 -104
  15. package/claw/modules/claw_bridge.ts +60 -80
  16. package/claw/modules/claw_catalog.ts +47 -16
  17. package/claw/modules/claw_infra.ts +1 -5
  18. package/claw/modules/claw_launcher.ts +47 -49
  19. package/claw/modules/claw_skill_md.ts +8 -11
  20. package/claw/modules/credit_runtime_adapter.ts +2 -18
  21. package/claw/modules/decision_loop.ts +4 -0
  22. package/claw/modules/dexbot_bridge.ts +1 -1
  23. package/claw/modules/dexbot_profiles.ts +5 -21
  24. package/claw/modules/honest_ecosystem.ts +24 -8
  25. package/claw/modules/liquidity_pools.ts +10 -3
  26. package/claw/modules/mcp_utils.ts +109 -0
  27. package/claw/modules/memu_bridge.ts +76 -54
  28. package/claw/modules/mpa_utils.ts +26 -3
  29. package/claw/modules/position_discovery.ts +16 -27
  30. package/claw/modules/position_manager.ts +32 -41
  31. package/claw/modules/position_manager_watch.ts +3 -2
  32. package/claw/modules/short_mpa_strategy.ts +1 -9
  33. package/claw/modules/skill_utils.ts +5 -3
  34. package/claw/modules/utils.ts +9 -1
  35. package/claw/package.json +1 -1
  36. package/claw/runtimes/openclaw-plugin/index.ts +27 -10
  37. package/claw/runtimes/openclaw-plugin/openclaw.plugin.json +1 -1
  38. package/claw/runtimes/openclaw-plugin/package.json +1 -1
  39. package/claw/scripts/claw_bridge.ts +1 -1
  40. package/claw/scripts/claw_mcp_server.ts +19 -78
  41. package/claw/scripts/honest_assets_report.ts +19 -84
  42. package/claw/scripts/memu_mcp_server.ts +57 -155
  43. package/claw/tests/test_claw_bridge.ts +6 -2
  44. package/claw/tests/test_claw_mcp_transport.ts +2 -2
  45. package/dist/analysis/ama_fitting/calibrate_convergence_er.js +1 -1
  46. package/dist/analysis/ama_fitting/calibrate_convergence_er.js.map +1 -1
  47. package/dist/analysis/bot_fitting/shared_utils.d.ts +1 -12
  48. package/dist/analysis/bot_fitting/shared_utils.d.ts.map +1 -1
  49. package/dist/analysis/bot_fitting/shared_utils.js +1 -1
  50. package/dist/analysis/bot_fitting/shared_utils.js.map +1 -1
  51. package/dist/analysis/bot_key_utils.d.ts +1 -1
  52. package/dist/analysis/bot_key_utils.d.ts.map +1 -1
  53. package/dist/analysis/bot_key_utils.js +1 -5
  54. package/dist/analysis/bot_key_utils.js.map +1 -1
  55. package/dist/analysis/bot_usage/kibana_bot_queries.d.ts +1 -337
  56. package/dist/analysis/bot_usage/kibana_bot_queries.d.ts.map +1 -1
  57. package/dist/analysis/bot_usage/kibana_bot_queries.js +1 -241
  58. package/dist/analysis/bot_usage/kibana_bot_queries.js.map +1 -1
  59. package/dist/analysis/chart_css.d.ts +1 -15
  60. package/dist/analysis/chart_css.d.ts.map +1 -1
  61. package/dist/analysis/chart_css.js +1 -1
  62. package/dist/analysis/chart_css.js.map +1 -1
  63. package/dist/analysis/math_utils.d.ts +3 -5
  64. package/dist/analysis/math_utils.d.ts.map +1 -1
  65. package/dist/analysis/math_utils.js +3 -5
  66. package/dist/analysis/math_utils.js.map +1 -1
  67. package/dist/bot.js +2 -2
  68. package/dist/bot.js.map +1 -1
  69. package/dist/credential-daemon.js +1 -1
  70. package/dist/credential-daemon.js.map +1 -1
  71. package/dist/dexbot.d.ts.map +1 -1
  72. package/dist/dexbot.js +18 -14
  73. package/dist/dexbot.js.map +1 -1
  74. package/dist/market_adapter/ama_signal_runner.js +3 -3
  75. package/dist/market_adapter/core/kibana_client.d.ts +1 -3
  76. package/dist/market_adapter/core/kibana_client.d.ts.map +1 -1
  77. package/dist/market_adapter/core/kibana_client.js +1 -1
  78. package/dist/market_adapter/core/kibana_client.js.map +1 -1
  79. package/dist/market_adapter/core/market_adapter_service.d.ts +1 -2
  80. package/dist/market_adapter/core/market_adapter_service.d.ts.map +1 -1
  81. package/dist/market_adapter/core/market_adapter_service.js +1 -1
  82. package/dist/market_adapter/core/market_adapter_service.js.map +1 -1
  83. package/dist/market_adapter/core/strategies/regime_gate.d.ts +1 -2
  84. package/dist/market_adapter/core/strategies/regime_gate.d.ts.map +1 -1
  85. package/dist/market_adapter/core/strategies/regime_gate.js +1 -1
  86. package/dist/market_adapter/core/strategies/regime_gate.js.map +1 -1
  87. package/dist/market_adapter/inputs/fetch_cex_synthetic_data.js +2 -2
  88. package/dist/market_adapter/inputs/fetch_cex_synthetic_data.js.map +1 -1
  89. package/dist/market_adapter/inputs/fetch_lp_data.d.ts +5 -5
  90. package/dist/market_adapter/inputs/fetch_lp_data.js +5 -5
  91. package/dist/market_adapter/inputs/kibana_source.d.ts +1 -2
  92. package/dist/market_adapter/inputs/kibana_source.d.ts.map +1 -1
  93. package/dist/market_adapter/inputs/kibana_source.js +1 -1
  94. package/dist/market_adapter/inputs/kibana_source.js.map +1 -1
  95. package/dist/market_adapter/lp_chart_strategy_loader.d.ts +1 -2
  96. package/dist/market_adapter/lp_chart_strategy_loader.d.ts.map +1 -1
  97. package/dist/market_adapter/lp_chart_strategy_loader.js +4 -4
  98. package/dist/market_adapter/lp_chart_strategy_loader.js.map +1 -1
  99. package/dist/market_adapter/market_adapter.d.ts.map +1 -1
  100. package/dist/market_adapter/market_adapter.js +2 -1
  101. package/dist/market_adapter/market_adapter.js.map +1 -1
  102. package/dist/modules/account_bots.d.ts +3 -0
  103. package/dist/modules/account_bots.d.ts.map +1 -1
  104. package/dist/modules/account_bots.js +156 -46
  105. package/dist/modules/account_bots.js.map +1 -1
  106. package/dist/modules/account_orders.d.ts +1 -7
  107. package/dist/modules/account_orders.d.ts.map +1 -1
  108. package/dist/modules/account_orders.js +9 -17
  109. package/dist/modules/account_orders.js.map +1 -1
  110. package/dist/modules/bitshares_client.d.ts +2 -14
  111. package/dist/modules/bitshares_client.d.ts.map +1 -1
  112. package/dist/modules/bitshares_client.js +22 -31
  113. package/dist/modules/bitshares_client.js.map +1 -1
  114. package/dist/modules/bot_settings.js +1 -1
  115. package/dist/modules/bot_settings.js.map +1 -1
  116. package/dist/modules/chain_keys.js +2 -2
  117. package/dist/modules/chain_keys.js.map +1 -1
  118. package/dist/modules/chain_orders.d.ts +2 -3
  119. package/dist/modules/chain_orders.d.ts.map +1 -1
  120. package/dist/modules/chain_orders.js +8 -25
  121. package/dist/modules/chain_orders.js.map +1 -1
  122. package/dist/modules/config.d.ts.map +1 -1
  123. package/dist/modules/config.js +4 -1
  124. package/dist/modules/config.js.map +1 -1
  125. package/dist/modules/constants.d.ts +0 -1
  126. package/dist/modules/constants.d.ts.map +1 -1
  127. package/dist/modules/constants.js +0 -4
  128. package/dist/modules/constants.js.map +1 -1
  129. package/dist/modules/credential_policy.js +1 -1
  130. package/dist/modules/credential_policy.js.map +1 -1
  131. package/dist/modules/dexbot_class.d.ts.map +1 -1
  132. package/dist/modules/dexbot_class.js +3 -2
  133. package/dist/modules/dexbot_class.js.map +1 -1
  134. package/dist/modules/dexbot_state_recovery.d.ts.map +1 -1
  135. package/dist/modules/dexbot_state_recovery.js +2 -1
  136. package/dist/modules/dexbot_state_recovery.js.map +1 -1
  137. package/dist/modules/fund_registry.d.ts.map +1 -1
  138. package/dist/modules/fund_registry.js +5 -2
  139. package/dist/modules/fund_registry.js.map +1 -1
  140. package/dist/modules/launcher/bot_supervisor.d.ts.map +1 -1
  141. package/dist/modules/launcher/bot_supervisor.js +6 -7
  142. package/dist/modules/launcher/bot_supervisor.js.map +1 -1
  143. package/dist/modules/launcher/foreign_cred_daemon.d.ts.map +1 -1
  144. package/dist/modules/launcher/foreign_cred_daemon.js +6 -0
  145. package/dist/modules/launcher/foreign_cred_daemon.js.map +1 -1
  146. package/dist/modules/launcher/market_adapter_runtime.d.ts +2 -3
  147. package/dist/modules/launcher/market_adapter_runtime.d.ts.map +1 -1
  148. package/dist/modules/launcher/market_adapter_runtime.js +7 -15
  149. package/dist/modules/launcher/market_adapter_runtime.js.map +1 -1
  150. package/dist/modules/launcher/monolithic_runtime.d.ts.map +1 -1
  151. package/dist/modules/launcher/monolithic_runtime.js +1 -0
  152. package/dist/modules/launcher/monolithic_runtime.js.map +1 -1
  153. package/dist/modules/launcher/status_reporting.d.ts +1 -2
  154. package/dist/modules/launcher/status_reporting.d.ts.map +1 -1
  155. package/dist/modules/launcher/status_reporting.js +1 -4
  156. package/dist/modules/launcher/status_reporting.js.map +1 -1
  157. package/dist/modules/order/accounting.js +2 -2
  158. package/dist/modules/order/accounting.js.map +1 -1
  159. package/dist/modules/order/format.d.ts +0 -8
  160. package/dist/modules/order/format.d.ts.map +1 -1
  161. package/dist/modules/order/format.js +0 -8
  162. package/dist/modules/order/format.js.map +1 -1
  163. package/dist/modules/order/logger.js +1 -1
  164. package/dist/modules/order/logger.js.map +1 -1
  165. package/dist/modules/order/logger_state.d.ts +1 -13
  166. package/dist/modules/order/logger_state.d.ts.map +1 -1
  167. package/dist/modules/order/logger_state.js +1 -11
  168. package/dist/modules/order/logger_state.js.map +1 -1
  169. package/dist/modules/order/processed_fill_store.d.ts +0 -5
  170. package/dist/modules/order/processed_fill_store.d.ts.map +1 -1
  171. package/dist/modules/order/processed_fill_store.js +1 -12
  172. package/dist/modules/order/processed_fill_store.js.map +1 -1
  173. package/dist/modules/order/sync_engine.d.ts.map +1 -1
  174. package/dist/modules/order/sync_engine.js +10 -3
  175. package/dist/modules/order/sync_engine.js.map +1 -1
  176. package/dist/modules/order/utils/math.d.ts +7 -1
  177. package/dist/modules/order/utils/math.d.ts.map +1 -1
  178. package/dist/modules/order/utils/math.js +19 -1
  179. package/dist/modules/order/utils/math.js.map +1 -1
  180. package/dist/modules/order/utils/order.d.ts.map +1 -1
  181. package/dist/modules/order/utils/order.js +10 -2
  182. package/dist/modules/order/utils/order.js.map +1 -1
  183. package/dist/modules/runtime_settings.js +3 -1
  184. package/dist/modules/runtime_settings.js.map +1 -1
  185. package/dist/modules/settings_merge.d.ts.map +1 -1
  186. package/dist/modules/settings_merge.js +3 -1
  187. package/dist/modules/settings_merge.js.map +1 -1
  188. package/dist/modules/storage/browser_adapter.d.ts +8 -3
  189. package/dist/modules/storage/browser_adapter.d.ts.map +1 -1
  190. package/dist/modules/storage/browser_adapter.js +53 -4
  191. package/dist/modules/storage/browser_adapter.js.map +1 -1
  192. package/dist/modules/utils/sanitize_key.d.ts +7 -0
  193. package/dist/modules/utils/sanitize_key.d.ts.map +1 -0
  194. package/dist/modules/utils/sanitize_key.js +15 -0
  195. package/dist/modules/utils/sanitize_key.js.map +1 -0
  196. package/dist/pm2.js +6 -6
  197. package/dist/pm2.js.map +1 -1
  198. package/dist/scripts/analyze-orders.d.ts +2 -2
  199. package/dist/scripts/analyze-orders.d.ts.map +1 -1
  200. package/dist/scripts/analyze-orders.js +4 -12
  201. package/dist/scripts/analyze-orders.js.map +1 -1
  202. package/dist/scripts/runner.js +1 -1
  203. package/dist/scripts/runner.js.map +1 -1
  204. package/dist/scripts/update.js +1 -1
  205. package/dist/scripts/update.js.map +1 -1
  206. package/docs/BITSHARES_ONBOARDING.md +63 -18
  207. package/docs/COPY_ON_WRITE_MASTER_PLAN.md +5 -14
  208. package/docs/COW_INVARIANTS.md +4 -4
  209. package/docs/CREDENTIAL_SECURITY.md +0 -11
  210. package/docs/DEXBOT_COMPARISON.md +11 -11
  211. package/docs/EVOLUTION.md +11 -10
  212. package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md +27 -83
  213. package/docs/GRID_RECALCULATION.md +1 -13
  214. package/docs/GRID_RECONCILE.md +24 -24
  215. package/docs/LIFECYCLE.md +4 -4
  216. package/docs/LOGGING.md +1 -1
  217. package/docs/MPA_CREDIT_USAGE.md +3 -3
  218. package/docs/README.md +2 -2
  219. package/docs/WORKFLOW.md +2 -2
  220. package/docs/architecture.md +43 -38
  221. package/docs/developer_guide.md +16 -18
  222. package/market_adapter/README.md +0 -2
  223. package/modules/README.md +2 -1
  224. package/package.json +1 -1
  225. package/scripts/README.md +1 -1
  226. package/claw/openclaw.plugin.json +0 -13
  227. package/dist/market_adapter/merge_lp_data.d.ts +0 -3
  228. package/dist/market_adapter/merge_lp_data.d.ts.map +0 -1
  229. package/dist/market_adapter/merge_lp_data.js +0 -125
  230. package/dist/market_adapter/merge_lp_data.js.map +0 -1
  231. package/dist/market_adapter/utils/paths.d.ts +0 -3
  232. package/dist/market_adapter/utils/paths.d.ts.map +0 -1
  233. package/dist/market_adapter/utils/paths.js +0 -5
  234. package/dist/market_adapter/utils/paths.js.map +0 -1
@@ -288,16 +288,6 @@ The failure path is consistent regardless of whether the daemon or the
288
288
  interactive password prompt handled the authentication, making the output
289
289
  predictable for monitoring and alerting.
290
290
 
291
- ### Legacy vault migration
292
-
293
- Older vaults stored a plain **SHA-256** hash of the master password for
294
- verification. This hash is deliberately weak by modern standards. The v2
295
- scrypt-derived vault format replaced it entirely: `masterPasswordHash` is no
296
- longer read, written, or migrated. A legacy vault that has not been converted
297
- is rejected by `unlockWithPassword` with "Unsupported key vault format" —
298
- recreate `profiles/keys.json` with the current key manager to adopt the v2
299
- format (HMAC-SHA256 vault verifier, scrypt-derived key).
300
-
301
291
  ---
302
292
 
303
293
  ## 6. Startup Path — Daemon-First, Interactive Fallback
@@ -391,7 +381,6 @@ accounts or after an unclean exit.
391
381
  | `probeBootstrapSocket` (live probe before cleanup) | Bootstrap dir cleanup | Prevents removing a live bootstrap directory |
392
382
  | `delete process.env.DEXBOT_CRED_BOOTSTRAP_PATH_FILE` | Daemon startup | Bootstrap path cannot be inherited by child processes or read from /proc |
393
383
  | Attempt limit (3) + immediate exit | Interactive auth | Limits online brute-force window |
394
- | `masterPasswordHash` removed | Legacy vault upgrade | Weak SHA-256 verifier no longer read, written, or migrated |
395
384
 
396
385
  ---
397
386
 
@@ -1,7 +1,7 @@
1
1
  # DEXBot vs DEXBot2 — Detailed Comparison Report
2
2
 
3
3
  > **Date:** 2026-08-19 *(metrics refreshed against local source trees)*
4
- > **Scope:** Full architectural, functional, and operational comparison between the original [DEXBot](https://github.com/Codaone/DEXBot) (Python, v1.0.0) and DEXBot2 (TypeScript, v1.4.20).
4
+ > **Scope:** Full architectural, functional, and operational comparison between the original [DEXBot](https://github.com/Codaone/DEXBot) (Python, v1.0.0) and DEXBot2 (TypeScript, v1.4.21).
5
5
  > **Audience:** Developers, contributors, and operators evaluating or migrating between the two projects.
6
6
 
7
7
  ---
@@ -36,7 +36,7 @@
36
36
 
37
37
  | Attribute | DEXBot (original) | DEXBot2 |
38
38
  |---|---|---|
39
- | **Release Track** | 1.0.0 | v1.4.20 |
39
+ | **Release Track** | 1.0.0 | v1.4.21 |
40
40
  | **Language** | Python 3.6+ | TypeScript 5.x |
41
41
  | **Status** | Released 1.0.0, unmaintained | Active development |
42
42
  | **Last Repo Activity** | May 23, 2020 | 2026-08-19 |
@@ -46,7 +46,7 @@
46
46
  | **Target Exchange** | BitShares DEX | BitShares DEX |
47
47
  | **Lines of Code** | ~10,846 Python LOC in `dexbot/` | Large TypeScript codebase; core runtime, adapter, analysis, Claw, and test modules |
48
48
  | **Source Files** | 72 Python files in `dexbot/` | ~527 TS files across the repo |
49
- | **Test Files** | 16 Python test files | 249 `test_*.ts` files (249 auto-discovered via `globSync`) |
49
+ | **Test Files** | 16 Python test files | 252 `test_*.ts` files (252 auto-discovered via `globSync`) |
50
50
 
51
51
  ### Summary
52
52
 
@@ -70,7 +70,7 @@ DEXBot2 is a ground-up rewrite in TypeScript that prioritizes production correct
70
70
  | **Process Manager** | Systemd service (Linux) | `dexbot start` (native monolithic launcher, default; `unlock` runtime) — PM2 optional |
71
71
  | **External APIs** | CoinGecko, CCXT, Waves | On-chain/pool/Kibana candle inputs; optional CEX synthetic seed generator (`fetch_cex_synthetic_data.ts`) for adapter bootstrap |
72
72
  | **Container** | Docker (Ubuntu 18.04) | Docker (multi-stage) |
73
- | **Testing** | pytest + Docker testnet | Native Node assert (249 `test_*.ts` files; auto-discovered via `globSync`) |
73
+ | **Testing** | pytest + Docker testnet | Native Node assert (252 `test_*.ts` files; auto-discovered via `globSync`) |
74
74
  | **CI/CD** | Travis CI, AppVeyor | GitHub Actions / local deterministic script suite |
75
75
  | **Packaging** | PyInstaller (Win/Mac/Linux binaries) | npm / `unlock` + optional PM2 ecosystem |
76
76
 
@@ -616,7 +616,7 @@ Where:
616
616
  ### DEXBot2
617
617
 
618
618
  - **Framework:** Native Node `assert` module (no external test framework)
619
- - **249 `test_*.ts` files** auto-discovered via `globSync` (`tests/test_*.ts` + `claw/tests/test_*.ts`), covering:
619
+ - **252 `test_*.ts` files** auto-discovered via `globSync` (`tests/test_*.ts` + `claw/tests/test_*.ts`), covering:
620
620
  - Unit tests: accounting, strategy, grid, manager logic
621
621
  - Copy-on-Write semantics: COW commits, guards, concurrent fills
622
622
  - Edge cases: authoritative full-fill resolution, partial fills, BTS fee accounting, precision
@@ -634,7 +634,7 @@ Where:
634
634
  | Feature | DEXBot | DEXBot2 |
635
635
  |---|---|---|
636
636
  | **Framework** | pytest | Native Node assert |
637
- | **Test Count** | 16 Python test files | 249 `test_*.ts` files; auto-discovered via `globSync` |
637
+ | **Test Count** | 16 Python test files | 252 `test_*.ts` files; auto-discovered via `globSync` |
638
638
  | **Test Types** | Unit + integration | Unit + integration + edge-case + runtime regression |
639
639
  | **Testnet Integration** | Yes (Docker) | No (mocks) |
640
640
  | **External Dependency** | pytest, Docker | None |
@@ -780,13 +780,13 @@ Where:
780
780
 
781
781
  | Metric | DEXBot | DEXBot2 |
782
782
  |---|---|---|
783
- | **Release Track** | 1.0.0 | v1.4.20 |
783
+ | **Release Track** | 1.0.0 | v1.4.21 |
784
784
  | **Active Since** | ~2018 | December 2025 |
785
785
  | **Last Commit** | May 23, 2020 | 2026-08-19 |
786
786
  | **Total Commits** | 2281 | 2,034 (v1.4.17) |
787
787
  | **Lines of Code** | ~10,846 Python LOC in `dexbot/` | Large TypeScript runtime + adapter + Claw + analysis + tests |
788
788
  | **Source Files** | 72 Python files in `dexbot/` | ~527 TS files across the repo |
789
- | **Test Files** | 16 Python test files | 249 `test_*.ts` files (249 auto-discovered via `globSync`) |
789
+ | **Test Files** | 16 Python test files | 252 `test_*.ts` files (252 auto-discovered via `globSync`) |
790
790
  | **Documentation** | Sphinx docs + README | 50+ Markdown docs plus Claw skills/references |
791
791
  | **Strategies** | 3 + plugins | 1 |
792
792
  | **Max Concurrent Bots** | Many (one process) | Many (one monolithic process by default; per-bot via `--isolated`/PM2) |
@@ -814,7 +814,7 @@ Each bottleneck in DEXBot compounds because they run *serially in sequence* —
814
814
  | # | Bottleneck | DEXBot | DEXBot2 (with references) | Multiplier |
815
815
  |---|------------|--------|---------------------------|------------|
816
816
  | 1 | **RPC queries** | Per-order `get_objects` loop called twice per cycle → 2×N sequential RPCs. | Single batch `get_objects([…])` in `fetchRefBlock` — **`modules/bitshares-native/tx/builder.ts`**; parallel account refresh via `Promise.all` — **`modules/bitshares-native/subscriptions.ts`** (`refreshTasks`, `scanTasks`). | **~400×** |
817
- | 2 | **Order counting** | Geometric while-loop iterating `price /= 1+increment` ~920 times per call. | O(1) `Math.log` spread-step formula in `calculateSpreadSteps` — **`modules/order/utils/math.ts`**; O(1) spread check in `shouldFlagOutOfSpread` — **`modules/order/utils/order.ts`**. | **~920× CPU** |
817
+ | 2 | **Order counting** | Geometric while-loop iterating `price /= 1+increment` ~920 times per call. | O(1) `Math.log` spread-step formula in `calculateGapSlots` — **`modules/order/utils/math.ts`**; O(1) spread check in `shouldFlagOutOfSpread` — **`modules/order/utils/order.ts`**. | **~920× CPU** |
818
818
  | 3 | **Market price** | Fresh `ticker()` RPC inside every order placement. | Cached `centerPrice` served from `botState`, zero RPC per placement — **`market_adapter/market_adapter.ts`** + **`market_adapter/core/market_adapter_service.ts`**. | **~∞ (eliminated)** |
819
819
  | 4 | **Account refresh** | Full `_account.refresh()` fetches all orders + balances + history every cycle. | Targeted `set_subscribe_callback` pushing only `OP_FILL_ORDER` ops — **`modules/bitshares-native/subscriptions.ts`** (`refreshSubscriptions`); filtered per-account; no full re-read — **`modules/chain_orders.ts`**. | **~50×** |
820
820
  | 5 | **Thread blocking** | `time.sleep(2–6)` on retry blocks the GIL thread entirely. | Async `await sleep()` + `AsyncLock` queue — **`modules/order/async_lock.ts`**; lock guards in **`modules/order/manager.ts`**; backoff in **`modules/order/utils/system.ts`**. | **~100× I/O utilization** |
@@ -838,7 +838,7 @@ DEXBot2 parallel path: [batch RPC] ─┐
838
838
  [JSON write] ┘
839
839
  ```
840
840
 
841
- The Python runtime overhead (≈2× slower than V8 on equivalent CPU work) is the *least* impactful factor here — but it still compounds with everything else. The geometric while-loops and `_calc_increase` iterations all run at Python bytecode speed, thousands of iterations per cycle. DEXBot2 eliminates the iterations entirely with O(1) formulas (`calculateSpreadSteps` in `math.ts`, `shouldFlagOutOfSpread` in `order.ts`) — so the 2× language factor is just insurance on top of the architectural gains.
841
+ The Python runtime overhead (≈2× slower than V8 on equivalent CPU work) is the *least* impactful factor here — but it still compounds with everything else. The geometric while-loops and `_calc_increase` iterations all run at Python bytecode speed, thousands of iterations per cycle. DEXBot2 eliminates the iterations entirely with O(1) formulas (`calculateGapSlots` in `math.ts`, `shouldFlagOutOfSpread` in `order.ts`) — so the 2× language factor is just insurance on top of the architectural gains.
842
842
 
843
843
  The 500× figure is not theoretical: it materializes in production when higher order counts, slower public nodes, transient block-expiration retries, and wide geometric ranges all hit at once — a scenario DEXBot handles by piling seconds onto seconds, while DEXBot2 absorbs each factor with negligible marginal cost.
844
844
 
@@ -886,7 +886,7 @@ The 500× figure is not theoretical: it materializes in production when higher o
886
886
  | **Security** | ★★★☆☆ | ★★★★★ (AES-256-GCM, credential daemon, authority resolution) | DEXBot2 |
887
887
  | **Ease of Setup** | ★★☆☆☆ (PyQt5/PyInstaller/Systemd dependency hell) | ★★★★★ (`npm i -g dexbot`, zero deps) | DEXBot2 |
888
888
  | **Accessibility** | ★★★★★ (GUI) | ★★☆☆☆ (CLI only) | DEXBot |
889
- | **Testing Depth** | ★★★☆☆ | ★★★★★ (249 test files; focused regressions) | DEXBot2 |
889
+ | **Testing Depth** | ★★★☆☆ | ★★★★★ (252 test files; focused regressions) | DEXBot2 |
890
890
  | **Documentation** | ★★★☆☆ | ★★★★★ (architecture/accounting/security/adapter docs) | DEXBot2 |
891
891
  | **Dependency Footprint** | ★★☆☆☆ (heavy) | ★★★★★ (0 runtime deps) | DEXBot2 |
892
892
  | **Extensibility** | ★★★★★ (plugins) | ★☆☆☆☆ | DEXBot |
package/docs/EVOLUTION.md CHANGED
@@ -2,14 +2,14 @@
2
2
 
3
3
  ## Executive Summary
4
4
 
5
- DEXBot2 is a sophisticated decentralized exchange trading bot for the BitShares blockchain. This report documents the complete evolution of the project from its inception in December 2025 through the current 1.4.20 stable release.
5
+ DEXBot2 is a sophisticated decentralized exchange trading bot for the BitShares blockchain. This report documents the complete evolution of the project from its inception in December 2025 through the current 1.4.21 stable release.
6
6
 
7
7
  ### Key Milestones
8
8
  - **Project Inception**: December 2, 2025
9
- - **Growth Phase**: 2,024 commits over ~8 active months
9
+ - **Growth Phase**: 2,068 commits over ~9 active months
10
10
  - **Code Maturity**: Evolution from basic utilities to a ~70,000+ LoC intelligent TypeScript system
11
- - **Stability**: Progression from manual testing to a suite of 249 automated test files
12
- - **Releases**: 97 release entries (v0.1.0 to v1.4.20)
11
+ - **Stability**: Progression from manual testing to a suite of 252 automated test files
12
+ - **Releases**: 98 release entries (v0.1.0 to v1.4.21)
13
13
 
14
14
  ---
15
15
 
@@ -61,13 +61,13 @@ v1.4.12 completed the module transition to native ES modules (root + claw `"type
61
61
 
62
62
  ### Phase 9: Post-ESM Cleanup, Consolidation & Hardening (August 2026)
63
63
 
64
- v1.4.16 centralized all user/runtime state onto a single resolver-derived profiles dir (`~/.config/dexbot2/profiles`) so it survives re-clones and npm updates and never lands in a read-only package dir, turned divergence surplus-cancel + hole-create pairs into in-place order rotations, and added an npm auto-update flow. v1.4.17 consolidated duplicate code (EC math, Base58Check, settings merge, asset resolution), trimmed `modules/types.ts` from 875 lines to the Order union, purged dead exports, and centralized the analysis tooling under strict TypeScript. v1.4.19 capped COW broadcasts at `MAX_OPS_PER_BROADCAST` (4) with chunked retry-on-uncertain broadcasting, fixed a spread-collapse regression via the shared `isSlotInRail` helper, lowered the AMA slope grid-reset threshold to 8, and added editor price feedback plus a `docs/LIFECYCLE.md` onboarding walkthrough. v1.4.20 hardened spread-correction boundary promotion with a MIN_SPREAD_ORDERS reserve floor and commit-time validation, gated persisted-boundary restore against self-legalizing overrun poison, aligned dynamic-weight clip parity between the live service and the research chart, centralized chart slider ranges, and moved all analysis outputs onto the central path resolver.
64
+ v1.4.16 centralized all user/runtime state onto a single resolver-derived profiles dir (`~/.config/dexbot2/profiles`) so it survives re-clones and npm updates and never lands in a read-only package dir, turned divergence surplus-cancel + hole-create pairs into in-place order rotations, and added an npm auto-update flow. v1.4.17 consolidated duplicate code (EC math, Base58Check, settings merge, asset resolution), trimmed `modules/types.ts` from 875 lines to the Order union, purged dead exports, and centralized the analysis tooling under strict TypeScript. v1.4.19 capped COW broadcasts at `MAX_OPS_PER_BROADCAST` (4) with chunked retry-on-uncertain broadcasting, fixed a spread-collapse regression via the shared `isSlotInRail` helper, lowered the AMA slope grid-reset threshold to 8, and added editor price feedback plus a `docs/LIFECYCLE.md` onboarding walkthrough. v1.4.20 hardened spread-correction boundary promotion with a MIN_SPREAD_ORDERS reserve floor and commit-time validation, gated persisted-boundary restore against self-legalizing overrun poison, aligned dynamic-weight clip parity between the live service and the research chart, centralized chart slider ranges, and moved all analysis outputs onto the central path resolver. v1.4.21 fixed silent-failure runtime defects surfaced by a modules-wide audit (NaN fund-invariant tolerance, always-flush fill store, double-decremented fill guard, config num() defaults), aligned the two boundary writers on a shared sell-rail ceiling enforced at commit and restore time, deduplicated claw shared logic while hardening error paths (secret redaction, EPIPE guards, spawn-error surfacing), corrected launcher supervisor/runtime lifecycle bugs, made the browser storage adapter persist deletions and flush debounced, extended editor green/red live color feedback to funds and price inputs, stopped docs/error messages from hardcoding repo-relative profile paths, and pruned dead code across analysis/market_adapter/claw/logging.
65
65
 
66
66
  ---
67
67
 
68
68
  ## Development Statistics
69
69
 
70
- The project has accumulated 250 automated test files across 97 release entries. See the **Version History** below for a per-release commit breakdown.
70
+ The project has accumulated 252 automated test files across 98 release entries. See the **Version History** below for a per-release commit breakdown.
71
71
 
72
72
  ---
73
73
 
@@ -86,7 +86,7 @@ The project has accumulated 250 automated test files across 97 release entries.
86
86
 
87
87
  ## Documentation & Testing
88
88
 
89
- Evolved from a basic README to a comprehensive framework (50+ docs entries, 80%+ JSDoc coverage, AGENTS.md). Testing matured from manual blockchain trials → Jest → lightweight Node.js assert across a 249-file suite covering unit, integration, simulation, and COW architectural guard tests.
89
+ Evolved from a basic README to a comprehensive framework (50+ docs entries, 80%+ JSDoc coverage, AGENTS.md). Testing matured from manual blockchain trials → Jest → lightweight Node.js assert across a 252-file suite covering unit, integration, simulation, and COW architectural guard tests.
90
90
 
91
91
  ---
92
92
 
@@ -132,11 +132,12 @@ Compact, era-level view; per-release commit detail lives in [CHANGELOG.md](../CH
132
132
  | v1.4.8 → v1.4.13 | 45 | Native ESM runtime, broadcast serialization, onboarding |
133
133
  | v1.4.13 → v1.4.19 | 36 | Profile-state centralization, code consolidation, per-broadcast op cap |
134
134
  | v1.4.19 → v1.4.20 | 5 | Grid boundary promotion hardening, recovery poison gate, analysis output centralization |
135
+ | v1.4.20 → v1.4.21 | 15 | Runtime audit fixes, claw dedup hardening, boundary ceiling alignment, editor color feedback |
135
136
 
136
137
  ---
137
138
 
138
139
  **Report Originally Generated**: February 19, 2026
139
- **Last Updated**: August 23, 2026 (v1.4.20)
140
- **Total Commits**: 2,051
141
- **Date Range**: December 2, 2025 – August 23, 2026
140
+ **Last Updated**: August 24, 2026 (v1.4.21)
141
+ **Total Commits**: 2,068
142
+ **Date Range**: December 2, 2025 – August 24, 2026
142
143
  **Repository**: DEXBot2 (BitShares DEX Trading Bot)
@@ -52,15 +52,13 @@ funds.allocated → _getSizingContext() (budget for spread correction)
52
52
  **Key points:**
53
53
  - `botFunds` percentage applies to **total** capital (free + locked in orders), not just free. A bot at `"50%"` gets half of everything, not half of what's currently idle.
54
54
  - `funds.allocated` is the ceiling for each side. Existing orders already consume part of it; the remaining free portion is available for new placements.
55
- - The downstream `applyBotFundsAllocation()` (`manager.ts:953`) also caps `funds.available` to `<= allocated` as a safety net, but the primary budget chokepoint is `getSideBudget` / `_getSizingContext` reading `allocated` directly (v1.2.6).
55
+ - The downstream `applyBotFundsAllocation()` (`manager.ts:949`) also caps `funds.available` to `<= allocated` as a safety net, but the primary budget chokepoint is `getSideBudget` / `_getSizingContext` reading `allocated` directly (v1.2.6).
56
56
 
57
57
  ---
58
58
 
59
59
  ### 1.4 Mixed Order Fund Validation
60
60
 
61
- **Problem Fixed**: Early batch builders ran a single fund check keyed off the first order's side, so a mixed BUY/SELL batch could trip a false "insufficient funds" warning even when each side had ample capital — the BUY check was applied to SELL orders (or vice versa).
62
-
63
- **Solution**: Per-asset validation using a signed-delta water-mark. The current validator tracks the **peak** running requirement per asset (not a side lump sum), so BUY and SELL ops in the same batch are validated independently against their own free balance.
61
+ Mixed BUY/SELL batches are validated per asset using a signed-delta **peak** running requirement (not a side lump sum), so BUY and SELL ops in the same batch are checked independently against their own free balance.
64
62
 
65
63
  #### Fund Availability Checks by Asset
66
64
 
@@ -69,7 +67,7 @@ funds.allocated → _getSizingContext() (budget for spread correction)
69
67
 
70
68
  #### Implementation Location
71
69
 
72
- File: `modules/dexbot_cow_runtime.ts` — `validateOperationFunds()` (line 1309), called from the COW batch broadcast path at line 2648. `modules/dexbot_class.ts` exposes a thin wrapper `_validateOperationFunds()` (line 1165).
70
+ File: `modules/dexbot_cow_runtime.ts` — `validateOperationFunds()` (line 1488), called from the COW batch broadcast path at line 2908. `modules/dexbot_class.ts` exposes a thin wrapper `_validateOperationFunds()` (line 1122).
73
71
 
74
72
  ```javascript
75
73
  // Per-asset peak requirement vs. quantized chain-free snapshot.
@@ -112,19 +110,9 @@ See [developer_guide.md#order-state-helper-functions](developer_guide.md#order-s
112
110
 
113
111
  ### 1.5 Fill Batch Processing & Timeline
114
112
 
115
- #### Problem Solved
116
-
117
- Previously, fills were processed one-at-a-time (~3s per broadcast). A burst of 29 fills in the Feb 7 market crash took ~90 seconds, during which:
118
- - Market prices moved significantly
119
- - Orders became stale (filled on-chain but not yet synced)
120
- - Orphan fills were created (fill events for orders no longer on-chain)
121
- - Fund tracking diverged from blockchain reality
122
-
123
- **Impact**: The extended 90s window meant the bot couldn't react to market moves, creating a cascading failure.
124
-
125
- #### Solution: Fixed-Cap Batch Fill Processing
113
+ #### Fixed-Cap Batch Fill Processing
126
114
 
127
- **Mechanism**: Fill events arrive via `modules/dexbot_fill_runtime.ts` (the fill-runtime module), which pushes them into `bot._incomingFillQueue` (declared in `modules/dexbot_class.ts`). The drain loop in `dexbot_fill_runtime.ts` then chunks the queue into capped batches and calls `modules/order/manager.ts::processFilledOrders` (line 1442) once per chunk to run the full rebalance pipeline.
115
+ **Mechanism**: Fill events arrive via `modules/dexbot_fill_runtime.ts` (the fill-runtime module), which pushes them into `bot._incomingFillQueue` (declared in `modules/dexbot_class.ts`). The drain loop in `dexbot_fill_runtime.ts` then chunks the queue into capped batches and calls `modules/order/manager.ts::processFilledOrders` (line 1438) once per chunk to run the full rebalance pipeline.
128
116
 
129
117
  **Batch Sizing Algorithm**: A single cap-based batch size (`FILL_PROCESSING.MAX_FILL_BATCH_SIZE`): a queue depth of 4 or fewer is processed as one unified batch; deeper queues are chunked into repeated batches of 4 (the last chunk may be smaller).
130
118
 
@@ -176,9 +164,7 @@ IF ratio >= GRID_REGENERATION_PERCENTAGE (default: 3%):
176
164
 
177
165
  #### Recovery Retry System
178
166
 
179
- **Problem**: One-shot `_recoveryAttempted` boolean flag meant permanent lockup if recovery failed once.
180
-
181
- **New Behavior**: Count+time-based retry system with periodic reset.
167
+ Recovery uses a count+time-based retry system with periodic reset, so a single failed recovery attempt never locks out future retries.
182
168
 
183
169
  **State Machine**:
184
170
  ```
@@ -221,25 +207,14 @@ When a batch fails because an on-chain order no longer exists, the cleanup relea
221
207
 
222
208
  ### 1.6 Remainder Accuracy During Capped Resize
223
209
 
224
- #### Problem Fixed
225
-
226
- When grid resize was capped by available funds, the accounting system needed to track what portion of the ideal grid went unallocated. This required careful per-slot tracking to distinguish between:
227
- - **Fully allocated slots**: received their ideal size (no remainder)
228
- - **Fund-capped slots**: received less than ideal because available funds ran out mid-allocation
210
+ When grid resize is capped by available funds, accounting tracks what portion of the ideal grid went unallocated via per-slot tracking:
211
+ - **Fully allocated slots**: receive their ideal size (no remainder)
212
+ - **Fund-capped slots**: receive less than ideal because available funds ran out mid-allocation
229
213
 
230
- Without per-slot tracking, the remainder was computed from totals, which overstated it when some slots were fully allocated and others were capped.
214
+ Computing the remainder from totals instead overstates it when some slots are fully allocated and others are capped.
231
215
 
232
- #### Solution: Per-Slot Tracking
233
-
234
- **Old Behavior** (Incorrect):
235
- ```javascript
236
- // Compute unallocated remainder from ideal sizes
237
- const remainder = totalIdealSizes - totalAllocatedSizes;
238
- // Problem: If actual allocation capped at 80% due to insufficient funds,
239
- // this uses 100% ideal in calculation → remainder overstated
240
- ```
216
+ #### Per-Slot Tracking
241
217
 
242
- **New Behavior** (Correct):
243
218
  ```javascript
244
219
  // Track per-slot applied sizes
245
220
  const appliedSizes = [];
@@ -415,25 +390,9 @@ targetSlot.orderId = newOrderId;
415
390
 
416
391
  ### 3.6 Orphan-Fill Deduplication & Double-Credit Prevention
417
392
 
418
- **Location**: `modules/dexbot_class.ts` — constructor, `_recoverExplicitStaleOrders()` (line 445), orphan-fill guard in the fill drain loop (line ~1633), and pruning pass after each cycle (line ~1919).
419
-
420
- #### Problem Solved
421
-
422
- During Feb 7 market crash, stale-order batch failures cascaded into double-crediting:
423
-
424
- **Scenario**:
425
- 1. Batch operation scheduled with 12 orders
426
- 2. Order X is on-chain, included in batch
427
- 3. Between sync and broadcast, order X fills on market (stale order)
428
- 4. Batch execution fails: "Limit order X does not exist"
429
- 5. Error handler: Clean up grid slot X, release funds to `chainFree`
430
- 6. Meanwhile, fill event arrives: "Order X was filled at price Y for amount Z"
431
- 7. Orphan-fill handler: Credits proceeds to `chainFree` AGAIN
432
- 8. **Result**: Double-credit of proceeds, inflated `chainTotal`, fund drift
393
+ **Location**: `modules/dexbot_class.ts` — constructor, `_recoverExplicitStaleOrders()` (line 448); orphan-fill guard in the fill drain loop and pruning pass after each cycle now live in `modules/dexbot_fill_runtime.ts` (guard ~lines 628-646, pruning ~lines 934-946).
433
394
 
434
- **In Crash Numbers**: 7 orphan fills × ~700 BTS = ~4,600 BTS inflated → cascaded to 47,842 BTS total drift.
435
-
436
- #### Solution: Stale-Cleaned Order ID Tracking with TTL + Recycled-Slot Guard
395
+ #### Stale-Cleaned Order ID Tracking with TTL + Recycled-Slot Guard
437
396
 
438
397
  **Mechanism**: Track which orders were cleaned up during batch failure recovery using timestamp + grid-slot retention.
439
398
 
@@ -566,7 +525,7 @@ BitShares charges fees for `limit_order_create` and `limit_order_cancel`.
566
525
  2. If sufficient `chainFree` available: deduct full amount atomically.
567
526
  3. If insufficient: defer settlement and retry when funds become available.
568
527
 
569
- - **Adoption fee parity (1.4.8):** COW chain-adoption paths charge fees exactly like the normal open-orders loop. `adoptPlacedBatchFromChain` (refused-commit and poll-confirmed paths) and the startup uncertain-create adoption apply the create/cancel/update fees via `_applySync` — previously create-only charging let optimistic BTS drift when a batch's orders were adopted without fee accounting.
528
+ - **Adoption fee parity:** COW chain-adoption paths charge fees exactly like the normal open-orders loop. `adoptPlacedBatchFromChain` (refused-commit and poll-confirmed paths) and the startup uncertain-create adoption apply the create/cancel/update fees via `_applySync`.
570
529
 
571
530
  - **Safe fee lookup:** `processBatchResults` uses `getAssetFeesSafe('BTS')` with zero-fee fallbacks — the throwing variant can no longer hard-fail a whole batch after a successful commit (`modules/dexbot_cow_runtime.ts`).
572
531
 
@@ -588,7 +547,7 @@ These are deducted from the *proceeds* of a fill.
588
547
 
589
548
  For BTS fees, the system returns a structured object (not a simple number) with multiple fields for accounting precision.
590
549
 
591
- **Location**: `modules/order/utils/math.ts::getAssetFees()` (line 312). The fee cache itself is populated by `modules/order/utils/system.ts::initializeFeeCache()` (line 631).
550
+ **Location**: `modules/order/utils/math.ts::getAssetFees()` (line 312). The fee cache itself is populated by `modules/order/utils/system.ts::initializeFeeCache()` (line 665).
592
551
 
593
552
  #### BTS Fee Object (Always Object)
594
553
 
@@ -651,10 +610,6 @@ const createFee = feeInfo.createFee; // BTS only
651
610
 
652
611
  ### 5.4 BUY Side Sizing & Fee Accounting
653
612
 
654
- **Problem Fixed**: BUY side fee calculations incorrectly applied fees to base asset instead of quote asset.
655
-
656
- **Solution**: Corrected fee accounting with proper asset assignment.
657
-
658
613
  #### Fee Application by Side
659
614
 
660
615
  | Side | Asset | Calculation | Notes |
@@ -702,13 +657,11 @@ For BUY orders that are makers:
702
657
 
703
658
  ### 5.5 Precision & Quantization
704
659
 
705
- **Problem**: Floating-point arithmetic accumulates rounding errors over many calculations. After dozens of order size calculations, price derivations, and fund allocations, float values drift from their true blockchain integer representations, causing mismatches between internal state and on-chain reality.
706
-
707
- **Solution**: Centralized quantization utilities that eliminate float accumulation by round-tripping through blockchain integer representation.
660
+ Floating-point arithmetic drifts from true blockchain integer representations over many order-size calculations, price derivations, and fund allocations. Quantization eliminates this accumulation by round-tripping every value through its blockchain integer form.
708
661
 
709
662
  #### 5.5.1 Core Quantization Functions
710
663
 
711
- **Location**: `modules/order/utils/math.ts` (line 260)
664
+ **Location**: `modules/order/utils/math.ts` (line 265)
712
665
 
713
666
  ##### `quantizeFloat(value, precision)` - Eliminate Accumulation Errors
714
667
 
@@ -771,21 +724,13 @@ const normalized = normalizeInt(currentSizeInt, 8);
771
724
  - Normalizing fund totals before invariant checks
772
725
  - Preparing sizes for blockchain transaction encoding
773
726
 
774
- #### 5.5.2 Consolidation Impact
727
+ #### 5.5.2 Consolidated Quantization
775
728
 
776
- Previously, five separate quantization implementations existed:
777
- - `dexbot_class.ts` - Manual rounding logic
778
- - `order.ts` - Custom precision handling
779
- - `strategy.ts` - Divergent rounding approach
780
- - `chain_orders.ts` - Different quantization pattern
781
- - `export.ts` - Isolated float conversions
729
+ Quantization has a single source of truth: `quantizeFloat()` in `modules/order/utils/math.ts`.
782
730
 
783
- **After Consolidation:**
784
- ✅ Single source of truth (`math.ts`)
785
731
  ✅ Consistent precision handling across all modules
786
732
  ✅ Reduced regression risk (tested once, used everywhere)
787
- ✅ Eliminated subtle float accumulation bugs
788
- ✅ All 34+ test suites pass with zero regressions
733
+ ✅ No subtle float accumulation bugs from divergent rounding paths
789
734
 
790
735
  #### 5.5.3 Precision Best Practices
791
736
 
@@ -818,7 +763,7 @@ This prevents the bug where `available = chainFree + required` created a tautolo
818
763
 
819
764
  ## 6. Safety & Invariants
820
765
 
821
- The `Accountant` enforces strict mathematical invariants to detect bugs or manual interference. Invariants are checked by `_verifyFundInvariants()` (`modules/order/accounting.ts` line 502) after every blockchain sync cycle. The verification reads from a snapshot captured under `_fundLock` — `actualBuy`/`actualSell` are captured at snapshot time, not read live outside the lock, closing a TOCTOU window. When a violation is detected, the system logs a `CRITICAL` error and attempts automatic recovery via `manager.accountant.recalculateFunds()` (`modules/order/accounting.ts` line 347, delegated from `modules/order/manager.ts` lines 981–990) — resetting internal state to match on-chain reality. If the grid lock is held (mid-rebalance), recovery is deferred until the lock is released. The bot continues operating throughout; it does **not** halt on invariant violations.
766
+ The `Accountant` enforces strict mathematical invariants to detect bugs or manual interference. Invariants are checked by `_verifyFundInvariants()` (`modules/order/accounting.ts` line 501) after every blockchain sync cycle. The verification reads from a snapshot captured under `_fundLock` — `actualBuy`/`actualSell` are captured at snapshot time, not read live outside the lock, closing a TOCTOU window. When a violation is detected, the system logs a `CRITICAL` error and attempts automatic recovery via `manager.accountant.recalculateFunds()` (`modules/order/accounting.ts` line 346, delegated from `modules/order/manager.ts` lines 981–990) — resetting internal state to match on-chain reality. If the grid lock is held (mid-rebalance), recovery is deferred until the lock is released. The bot continues operating throughout; it does **not** halt on invariant violations.
822
767
 
823
768
  ### 6.1 The Equality Invariant
824
769
  Total funds on chain must equal free plus committed.
@@ -834,18 +779,17 @@ A violation here means the grid has allocated more capital than actually exists
834
779
 
835
780
  ### 6.3 Race Condition Protection (TOCTOU)
836
781
  To prevent "Time-of-Check to Time-of-Use" errors:
837
- 1. **Locking:** `AsyncLock` (re-entrant) prevents concurrent updates to the same order. Nested `acquire()` from the same execution context runs the callback directly instead of queueing, eliminating the `fillLockAlreadyHeld` parameter that previously threaded through 25+ call sites.
782
+ 1. **Locking:** `AsyncLock` (re-entrant) prevents concurrent updates to the same order. Nested `acquire()` from the same execution context runs the callback directly instead of queueing.
838
783
  2. **Atomic Deduct:** `tryDeductFromChainFree` checks *and* subtracts in a single synchronous step.
839
784
  3. **Bootstrapping:** Fills arriving during startup (`isBootstrapping=true`) are queued until the grid is fully reconciled ([GRID_RECONCILE.md](GRID_RECONCILE.md)).
840
785
 
841
- ### 6.4 Stale Accounting & Fee Over-Credit Guards (v1.2.1)
842
- Two additional accounting hardening measures added in v1.2.1:
786
+ ### 6.4 Stale Accounting & Fee Over-Credit Guards
843
787
 
844
- **Stale `accountTotals` no longer HARD-ABORTs COW commit.** Transient staleness (e.g., the periodic balance fetch overlaps with a COW commit) logs a `WARN` and schedules recovery instead of throwing `ACCOUNTING_COMMITMENT_FAILED`. Totals are also refreshed after bootstrap to prevent a spurious full recovery on the first maintenance cycle.
788
+ **Stale `accountTotals` does not abort COW commit.** Transient staleness (e.g., the periodic balance fetch overlaps with a COW commit) logs a `WARN` and schedules recovery instead of throwing `ACCOUNTING_COMMITMENT_FAILED`. Totals are also refreshed after bootstrap to prevent a spurious full recovery on the first maintenance cycle.
845
789
 
846
- **Fee-deduction failure escalates to `error`.** When `getAssetFees` throws during fill processing (e.g., network blip), `_deductFeesFromProceeds` previously returned raw proceeds without deduction and logged at `warn` level — silently inflating `accountTotals` over time. The log is now `error` with explicit "fund tracking will over-credit" language so operators can detect the drift source in production logs.
790
+ **Fee-deduction failure logs at `error`.** When `getAssetFees` throws during fill processing (e.g., network blip), `_deductFeesFromProceeds` skips the deduction and logs at `error` with explicit "fund tracking will over-credit" language so operators can detect the drift source in production logs.
847
791
 
848
- **TOCTOU in `processFillAccounting`.** `_buildBtsDeferredRefundAdjustment` reads `btsFeeState` from `mgr.orders`, but the order lock was acquired after accounting ran. Fixed by acquiring the lock first, then running `processFillAccounting` under the lock. This fix was also ported to POST-RESET and BOOTSTRAP tracked-fill accounting paths.
792
+ **TOCTOU protection in `processFillAccounting`.** `_buildBtsDeferredRefundAdjustment` reads `btsFeeState` from `mgr.orders` while the order lock is held — the lock is acquired before accounting runs, and the POST-RESET and BOOTSTRAP tracked-fill accounting paths follow the same locking pattern.
849
793
 
850
794
  ---
851
- *Technical Reference for DEXBot2 v1.4.20 release*
795
+ *Technical Reference for DEXBot2 v1.4.21 release*
@@ -397,7 +397,7 @@ and dedupes while one resync is already pending or running.
397
397
  - Fill/rotation operations are causing noticeable size errors
398
398
 
399
399
  **Disable (set to 0) if:**
400
- - You want to rely ONLY on AMA triggers (Issue #5: RMS Divergence Check Disabling)
400
+ - You want to rely ONLY on AMA triggers
401
401
  - You want to prevent automatic full resync from divergence alone
402
402
  - You manually trigger full grid resyncs through other mechanisms
403
403
 
@@ -596,18 +596,6 @@ Removed trigger file.
596
596
 
597
597
  ---
598
598
 
599
- ## Related Issues
600
-
601
- > Issue numbers below refer to the project's historical internal tracker and
602
- > are kept here for archival context. They are not always resolvable against
603
- > external issue trackers.
604
-
605
- - **Issue #5:** RMS Divergence Check Disabling — Ability to set `RMS_PERCENTAGE: 0` to disable checks
606
- - **Feature:** AMA Integration — AMA-derived center snapshots are already used for market-adapter-triggered grid recentering
607
- - **Issue #1:** Fund Validation Bug — Fixed validation logic for order batch placement
608
-
609
- ---
610
-
611
599
  ## References
612
600
 
613
601
  - `modules/constants.ts` — Default configuration values
@@ -78,16 +78,16 @@ Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations a
78
78
 
79
79
  ### Phase 1 — Pure Planning Under `_gridLock`
80
80
 
81
- **`grid_reconcile.ts:205-356`**
81
+ **`grid_reconcile.ts:208-375`**
82
82
 
83
- 1. **Phantom order sanitization** (lines 211-244): For each `isOrderPlaced()` order whose `orderId` is not in the chain snapshot, reset it to VIRTUAL with `skipAccounting` to prevent fund inflation. Two absence-decision guards make this safe:
84
- - **Freshly-assigned deferral** (lines 215-233): an `orderId` assigned within `TIMING.SYNC_LOCK_TIMEOUT_MS` (via `manager._orderIdAssignedAt`) may be an in-flight create/adopt whose broadcast has not landed or is not yet visible to a lagging/truncated read — virtualizing it and re-creating would duplicate a real live order (the reconcile-timeout death-spiral root cause). It is skipped (deferred) rather than virtualized.
85
- - **Ghost heuristic** (line 225): an order with `size <= 0` && `PARTIAL` (a known filled ghost) still passes through so known fills get cleaned up.
83
+ 1. **Phantom order sanitization** (lines 216-245): For each `isOrderPlaced()` order whose `orderId` is not in the chain snapshot, reset it to VIRTUAL with `skipAccounting` to prevent fund inflation. Two absence-decision guards make this safe:
84
+ - **Freshly-assigned deferral** (lines 226-235): an `orderId` assigned within `TIMING.SYNC_LOCK_TIMEOUT_MS` (via `manager._orderIdAssignedAt`) may be an in-flight create/adopt whose broadcast has not landed or is not yet visible to a lagging/truncated read — virtualizing it and re-creating would duplicate a real live order (the reconcile-timeout death-spiral root cause). It is skipped (deferred) rather than virtualized.
85
+ - **Ghost heuristic** (line 229): an order with `size <= 0` && `PARTIAL` (a known filled ghost) still passes through so known fills get cleaned up.
86
86
  - Virtualization always uses `{ skipAccounting: true }` so startup cleanup never inflates `ChainFree`.
87
87
 
88
- 2. **Duplicate detection** (lines 246-311): For each unmatched chain order, find the nearest active same-side grid order. If `priceDiff ≤ tolerance × 5`, flag it as a suspected duplicate and queue for a Phase 2 cancel (never cancelled under lock). Tolerance is computed from price impact via `calculatePriceTolerance`: capped at `PRICE_TOLERANCE_MAX_PERCENT` (1%) with a `PRICE_TOLERANCE_MIN_ABSOLUTE` (0.0001) floor. Duplicate IDs are removed from the unmatched set so they aren't also paired for updates/creates.
88
+ 2. **Duplicate detection** (lines 258-331): For each unmatched chain order, find the nearest active same-side grid order. If `priceDiff ≤ tolerance × 5`, flag it as a suspected duplicate and queue for a Phase 2 cancel (never cancelled under lock). Tolerance is computed from price impact via `calculatePriceTolerance`: capped at `PRICE_TOLERANCE_MAX_PERCENT` (1%) with a `PRICE_TOLERANCE_MIN_ABSOLUTE` (0.0001) floor. Duplicate IDs are removed from the unmatched set so they aren't also paired for updates/creates.
89
89
 
90
- 3. **Per-side reconciliation** via `_reconcileStartupSide(planOnly=true)` (lines 323-353):
90
+ 3. **Per-side reconciliation** via `_reconcileStartupSide(planOnly=true)` (lines 343-372):
91
91
  - Count `matchedOnGrid` (active grid orders with `orderId`)
92
92
  - `neededSlots = targetCount - matchedOnGrid`; pick virtual slots to activate
93
93
  - Match sorted unmatched chain orders to virtual slots → `plannedUpdates`
@@ -99,29 +99,29 @@ Returns `{ plannedCreates, plannedUpdates, plannedCancels, chainSellCount, chain
99
99
 
100
100
  ### Phase 2 — Blockchain Execution Outside Lock
101
101
 
102
- **`grid_reconcile.ts:358-480`**
102
+ **`grid_reconcile.ts:376-500`**
103
103
 
104
104
  Each sub-phase releases `_gridLock` before starting and re-acquires it per operation (through `synchronizeWithChain` in individual helpers). No single long-held lock blocks fills, sync, or divergence checks — but each operation still runs under the lock for consistency.
105
105
 
106
- **Cancellations** (lines 366-391): Execute `plannedCancels`. Each `_cancelChainOrder` acquires `_gridLock` internally. This covers duplicate cancels, edge-release cancels, and excess-order cancels.
106
+ **Cancellations** (lines 384-411): Execute `plannedCancels`. Each `_cancelChainOrder` acquires `_gridLock` internally. This covers duplicate cancels, edge-release cancels, and excess-order cancels.
107
107
 
108
- **Updates** (lines 393-465):
108
+ **Updates** (lines 413-485):
109
109
  - Batch via `_executeStartupUpdateBatch` when `supportsBatchUpdate` is available
110
110
  - Retry up to 3× (`maxBatchAttempts = 3`)
111
111
  - On each failure: `_recoverStartupSyncFailure()` re-fetches open orders from chain (guarded read) and re-syncs `manager` state via `manager.syncFromOpenOrders()`, then `_refreshStartupUpdatePlans()` rebuilds plans against the fresh chain state
112
112
  - If retries exhausted or batch helpers are unavailable → `_executeStartupSequentialUpdateFallback()` one-by-one with per-failure recovery
113
113
 
114
- **Creates** (lines 467-480): `_executePlannedStartupCreates` runs with the outside-in pair grouping — grouped from the outermost grid slots toward the center, BUY descending / SELL ascending, so the most price-critical orders are placed first. BitShares DEX batch-create operations are used where supported. Every created chain ID is captured into `phase2CreatedOrderIds` so Phase 3 cannot later cancel the freshly-created orders.
114
+ **Creates** (lines 487-500): `_executePlannedStartupCreates` runs with the outside-in pair grouping — grouped from the outermost grid slots toward the center, BUY descending / SELL ascending, so the most price-critical orders are placed first. BitShares DEX batch-create operations are used where supported. Every created chain ID is captured into `phase2CreatedOrderIds` so Phase 3 cannot later cancel the freshly-created orders.
115
115
 
116
116
  ### Phase 3 — Fresh Re-read, Adoption, Stale Surplus Cleanup
117
117
 
118
- **`grid_reconcile.ts:502-631`** (guarded by `if (!dryRun)` at line 504)
118
+ **`grid_reconcile.ts:502-641`** (guarded by `if (!dryRun)` at line 504)
119
119
 
120
120
  1. **Guarded fresh re-read** (lines 512-516): `readOpenOrdersGuarded` re-fetches all open orders. On a truncated/empty read it returns early (defers), keeping the pre-Phase-2 counts for the summary log — a capped window omits exactly the freshest Phase-2 creates.
121
121
 
122
122
  2. **Adopt uncertain-landed creates** (lines 533-578): For any fresh chain order not matching a grid `orderId` and not created by a slot, it attempts targeted slot adoption — matching a VIRTUAL slot by type+price+size (within tolerance) and registering it via `_applySync(..., 'createOrder')` with the create-fee deduction. Full `syncFromOpenOrders` is deliberately **not** used here (its pass-1 virtualizes ACTIVE slots missing from the snapshot, and a lagging read right after the Phase-2 broadcast would destroy the confirmed grid). If adoption fails, the ID is still protected from surplus-cancel; the next sync loop's orphan adoption registers it.
123
123
 
124
- 3. **Stale surplus cancellation** (lines 579-599): Per side, count orders exceeding `targetCount` that no grid slot holds via `orderId` (including the phase-2 created IDs). Cancel only these untracked surplus orders, sorted by chain ID for determinism. This catches orphans lost during grid reinitialization — on-chain orders with no corresponding grid slot.
124
+ 3. **Stale surplus cancellation** (lines 579-625): Per side, count orders exceeding `targetCount` that no grid slot holds via `orderId` (including the phase-2 created IDs). Cancel only these untracked surplus orders, sorted by chain ID for determinism. This catches orphans lost during grid reinitialization — on-chain orders with no corresponding grid slot.
125
125
 
126
126
  ### Partial Failure State
127
127
 
@@ -130,8 +130,8 @@ If Phase 2 partially succeeds (some cancels, some creates fail), there is no rol
130
130
  ### Timeouts and Read Coverage
131
131
 
132
132
  - **No per-attempt race** around the reconcile itself — the 1.4.8 change removed it to avoid orphaning mid-batch broadcasts (see the [`recalculateGrid`](../modules/order/grid.ts) call site in `modules/order/grid.ts`).
133
- - The whole resync is bounded by a **10-minute total timeout** (`PIPELINE_TIMING.TIMEOUT_MS * 2` at `grid.ts:1122`), applied via `Promise.race` at `grid.ts:1217`.
134
- - Every internal chain read goes through `readOpenOrdersGuarded` (`chain_orders.ts:611`) with the 30s / 3-retry / node-failover standard, and empty/truncated reads are treated as **ambiguous** — never as authoritative absence.
133
+ - The whole resync is bounded by a **10-minute total timeout** (`PIPELINE_TIMING.TIMEOUT_MS * 2` at `grid.ts:1184`), applied via `Promise.race` at `grid.ts:1279`.
134
+ - Every internal chain read goes through `readOpenOrdersGuarded` (`chain_orders.ts:610`) with the 30s / 3-retry / node-failover standard, and empty/truncated reads are treated as **ambiguous** — never as authoritative absence.
135
135
 
136
136
  ---
137
137
 
@@ -139,7 +139,7 @@ If Phase 2 partially succeeds (some cancels, some creates fail), there is no rol
139
139
 
140
140
  ### Fresh Grid Guard (`matchedOnGrid > 0`)
141
141
 
142
- **`grid_reconcile_internal.ts:1604-1616`**
142
+ **`grid_reconcile_internal.ts:1600`**
143
143
 
144
144
  When a brand-new grid is generated, every slot is VIRTUAL — `matchedOnGrid = 0`. Without a guard, every on-chain order appears "unmatched" and would be cancelled as excess:
145
145
 
@@ -153,11 +153,11 @@ When `matchedOnGrid === 0` AND scaling up (`neededSlots > 0`), excess cancellati
153
153
 
154
154
  ### Grid-Edge Lock
155
155
 
156
- **`grid_reconcile_internal.ts:259`** — `_isGridEdgeFullyActive` detects when the grid boundary is fully active (all slots on-chain) before cancelling excess orders.
156
+ **`grid_reconcile_internal.ts:244`** — `_isGridEdgeFullyActive` detects when the grid boundary is fully active (all slots on-chain) before cancelling excess orders.
157
157
 
158
- When all outermost orders of a side are ACTIVE with `orderId`, all balance is committed to the edges. Cancel the **largest** order among the update candidates (`_cancelLargestOrder`, line 329) to free maximum funds with minimum operations, since the DEX does not expose partial-reduce in one operation. The cancelled slot gets a replacement create.
158
+ When all outermost orders of a side are ACTIVE with `orderId`, all balance is committed to the edges. Cancel the **largest** order among the update candidates (`_cancelLargestOrder`, line 314) to free maximum funds with minimum operations, since the DEX does not expose partial-reduce in one operation. The cancelled slot gets a replacement create.
159
159
 
160
- Detection (`_isGridEdgeFullyActive`, line 259): sort orders by price (BUY descending, SELL ascending), and check the outermost ones are all `isOrderPlaced()`.
160
+ Detection (`_isGridEdgeFullyActive`, line 244): sort orders by price (BUY descending, SELL ascending), and check the outermost ones are all `isOrderPlaced()`.
161
161
 
162
162
  ### Duplicate Tolerance (5× Multiplier)
163
163
 
@@ -181,10 +181,10 @@ Reconcile Phase 1 runs under `_gridLock` with no side effects on the frozen mast
181
181
 
182
182
  ### Truncated-Read Ambiguity (since 1.4.8)
183
183
 
184
- Every chain read feeding an absence/surplus decision goes through `readOpenOrdersGuarded` (`chain_orders.ts:611`) and treats an empty or truncated snapshot as **unreadable** — never as "nothing landed" or "nothing to cancel":
184
+ Every chain read feeding an absence/surplus decision goes through `readOpenOrdersGuarded` (`chain_orders.ts:610`) and treats an empty or truncated snapshot as **unreadable** — never as "nothing landed" or "nothing to cancel":
185
185
 
186
- - `_recoverSyncFromChain` (`grid_reconcile_internal.ts:607`) — plus its three recovery sites in `_createOrderFromGrid` / `_cancelChainOrder` — defers on empty/truncated reads (`deferEmpty: true`). A pass-1 phantom cleanup would otherwise virtualize live slots from a partial window.
187
- - `_adoptPossiblyLandedCreate` (`grid_reconcile_internal.ts:947`) defers to an uncertain outcome on truncated reads, and the startup group batch uncertain verification follows the same rule.
186
+ - `_recoverSyncFromChain` (`grid_reconcile_internal.ts:592`) — plus its three recovery sites in `_createOrderFromGrid` / `_cancelChainOrder` — defers on empty/truncated reads (`deferEmpty: true`). A pass-1 phantom cleanup would otherwise virtualize live slots from a partial window.
187
+ - `_adoptPossiblyLandedCreate` (`grid_reconcile_internal.ts:932`) defers to an uncertain outcome on truncated reads, and the startup group batch uncertain verification follows the same rule.
188
188
  - Phase 3 final refresh (`grid_reconcile.ts:512`) skips adoption/surplus-cancel on a truncated read, keeping the pre-phase-2 counts for the summary log.
189
189
  - Adoption paths (`_adoptPossiblyLandedCreate`, grouping path, reconcile adoption loop) apply the create-fee deduction via `_applySync` for accounting parity.
190
190
 
@@ -194,7 +194,7 @@ The underlying rule is `INV-BROADCAST-004`: a capped `get_full_accounts` window
194
194
 
195
195
  ## Lock Hierarchy
196
196
 
197
- **`manager.ts:471-486`** — canonical reference in [`developer_guide.md`](developer_guide.md#lock-ordering-for-deadlock-prevention).
197
+ **`manager.ts:474-489`** — canonical reference in [`developer_guide.md`](developer_guide.md#lock-ordering-for-deadlock-prevention).
198
198
 
199
199
  ```
200
200
  Level 0: _fillProcessingLock Level 1: _divergenceLock
@@ -224,7 +224,7 @@ Commit `e64db685` replaced 6 single-value boolean state fields with refcounts/st
224
224
  | `maxBatchAttempts` | `3` | `grid_reconcile.ts:415` | Update-batch retry limit |
225
225
  | `PRICE_TOLERANCE_MAX_PERCENT` | `0.01` (1%) | `constants.ts:452` | Cap on price tolerance |
226
226
  | `PRICE_TOLERANCE_MIN_ABSOLUTE` | `0.0001` | `constants.ts:456` | Floor for price tolerance |
227
- | `PIPELINE_TIMING.TIMEOUT_MS` | `300000` (5min) | `constants.ts:805` | Base pipeline timing; resync uses 2× (10 min) |
227
+ | `PIPELINE_TIMING.TIMEOUT_MS` | `300000` (5min) | `constants.ts:800` | Base pipeline timing; resync uses 2× (10 min) |
228
228
 
229
229
  ---
230
230
 
@@ -250,7 +250,7 @@ Commit `e64db685` replaced 6 single-value boolean state fields with refcounts/st
250
250
  | File | Role |
251
251
  |------|------|
252
252
  | `modules/order/grid_reconcile.ts` | Public API + 3-phase orchestrator (642 lines) |
253
- | `modules/order/grid_reconcile_internal.ts` | Internal helpers — `_reconcileStartupSide`, grid detection, recovery, uncertainty (1690 lines) |
253
+ | `modules/order/grid_reconcile_internal.ts` | Internal helpers — `_reconcileStartupSide`, grid detection, recovery, uncertainty (1675 lines) |
254
254
  | `modules/order/manager.ts` | Lock hierarchy definition, `_applyOrderUpdate`, phantom guard, `reconcileGrid` entry, COW integration |
255
255
  | `modules/order/async_lock.ts` | AsyncLock engine with ALS re-entrancy (424 lines) |
256
256
  | `modules/order/sync_engine.ts` | Blockchain sync pipeline |
package/docs/LIFECYCLE.md CHANGED
@@ -134,7 +134,7 @@ sequenceDiagram
134
134
  participant STORE as Storage
135
135
 
136
136
  MA->>CL: AMA center snapshot updated
137
- CL->>CL: _performPeriodicGridChecks() (dexbot_class.ts:1664)
137
+ CL->>CL: _performPeriodicGridChecks() (dexbot_class.ts:1694)
138
138
  CL->>RT: performPeriodicGridChecks() (dexbot_maintenance_runtime.ts:1043)
139
139
  RT->>RT: runGridMaintenance(bot,'periodic') (dexbot_maintenance_runtime.ts:1845)
140
140
  RT->>RT: executeMaintenanceLogic() (dexbot_maintenance_runtime.ts:1452)
@@ -158,10 +158,10 @@ the parameters.
158
158
 
159
159
  > Note: `runMaintenance()` is a **different** subsystem — the credit/MPA debt
160
160
  > runtime (`modules/credit_runtime.ts:3041`, reached via
161
- > `_runCreditRuntimeMaintenance` at `dexbot_class.ts:1769`). The grid maintenance
161
+ > `_runCreditRuntimeMaintenance` at `dexbot_class.ts:1794`). The grid maintenance
162
162
  > chain above is the one that matters for order/price upkeep.
163
163
 
164
- References: `modules/dexbot_class.ts:1664` (`_performPeriodicGridChecks`) →
164
+ References: `modules/dexbot_class.ts:1694` (`_performPeriodicGridChecks`) →
165
165
  `modules/dexbot_maintenance_runtime.ts:1043` (`performPeriodicGridChecks`) →
166
166
  `:1845` (`runGridMaintenance`) → `:1452` (`executeMaintenanceLogic`),
167
167
  `docs/GRID_RECALCULATION.md`, `docs/GRID_RECONCILE.md`.
@@ -191,7 +191,7 @@ enforced*, not compiler-enforced — learn them or you will introduce fund bugs.
191
191
  | How a fill becomes orders | `modules/dexbot_fill_runtime.ts` → `modules/order/manager.ts` |
192
192
  | Grid math / recalculation | `modules/order/grid.ts`, `docs/GRID_RECALCULATION.md` |
193
193
  | Funds & accounting | `modules/order/accounting.ts`, `docs/FUND_MOVEMENT_AND_ACCOUNTING.md` |
194
- | Periodic loop & AMA hook | `modules/dexbot_maintenance_runtime.ts`, `modules/dexbot_class.ts:1664` |
194
+ | Periodic loop & AMA hook | `modules/dexbot_maintenance_runtime.ts`, `modules/dexbot_class.ts:1694` |
195
195
  | Market signal source | `market_adapter/market_adapter.ts`, `analysis/README.md` |
196
196
  | Startup & orchestration | `modules/dexbot_class.ts`, `docs/developer_guide.md` §"Startup Sequence" |
197
197
  | Why COW exists | `docs/architecture.md` §"Copy-on-Write (COW) Grid Pattern", `docs/COW_INVARIANTS.md` |