dexbot 1.6.2 → 1.6.4

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (212) hide show
  1. package/CHANGELOG.md +53 -3
  2. package/README.md +11 -7
  3. package/analysis/README.md +20 -14
  4. package/analysis/account_resolver.ts +171 -0
  5. package/analysis/ama_fitting/analyze_lambda_vs_slow.ts +2 -5
  6. package/analysis/ama_fitting/package.json +1 -1
  7. package/analysis/bot_usage/discover_bot_accounts.ts +20 -27
  8. package/analysis/chain_pool.ts +47 -0
  9. package/analysis/chart_utils.ts +33 -20
  10. package/analysis/derivative_chart_generator.ts +2 -3
  11. package/analysis/fills_source.ts +234 -0
  12. package/analysis/grid_correction_check.ts +104 -244
  13. package/analysis/results/ama_sweep_results_lp_pool_133_1h.json +2455 -0
  14. package/analysis/results/bot_fitting_results_lp_pool_133_1h.json +218 -0
  15. package/analysis/trade_profitability.ts +169 -304
  16. package/analysis/tradingview/README.md +2 -2
  17. package/analysis/tradingview/h-bts_tradingview.html +1570 -0
  18. package/analysis/tradingview/t-bts_tradingview.html +1570 -0
  19. package/analysis/tradingview/tradingview_uplot_chart_generator.ts +3 -4
  20. package/analysis/trend_detection/dynamic_weight_chart_generator.ts +2 -3
  21. package/analysis/trend_detection/kalman_chart_generator.ts +2 -3
  22. package/analysis/trend_detection/package.json +1 -1
  23. package/analysis/trend_detection/regime_chart_generator.ts +2 -3
  24. package/analysis/trend_detection/volatility_chart_generator.ts +2 -3
  25. package/claw/package.json +1 -1
  26. package/claw/runtimes/openclaw-plugin/openclaw.plugin.json +1 -1
  27. package/claw/runtimes/openclaw-plugin/package.json +1 -1
  28. package/claw/tests/test_claw_mcp_transport.ts +2 -2
  29. package/dist/analysis/account_resolver.d.ts +43 -0
  30. package/dist/analysis/account_resolver.d.ts.map +1 -0
  31. package/dist/analysis/account_resolver.js +137 -0
  32. package/dist/analysis/account_resolver.js.map +1 -0
  33. package/dist/analysis/ama_fitting/analyze_lambda_vs_slow.d.ts.map +1 -1
  34. package/dist/analysis/ama_fitting/analyze_lambda_vs_slow.js +2 -4
  35. package/dist/analysis/ama_fitting/analyze_lambda_vs_slow.js.map +1 -1
  36. package/dist/analysis/bot_usage/discover_bot_accounts.js +20 -28
  37. package/dist/analysis/bot_usage/discover_bot_accounts.js.map +1 -1
  38. package/dist/analysis/chain_pool.d.ts +10 -0
  39. package/dist/analysis/chain_pool.d.ts.map +1 -0
  40. package/dist/analysis/chain_pool.js +47 -0
  41. package/dist/analysis/chain_pool.js.map +1 -0
  42. package/dist/analysis/chart_utils.d.ts +14 -1
  43. package/dist/analysis/chart_utils.d.ts.map +1 -1
  44. package/dist/analysis/chart_utils.js +32 -22
  45. package/dist/analysis/chart_utils.js.map +1 -1
  46. package/dist/analysis/derivative_chart_generator.d.ts.map +1 -1
  47. package/dist/analysis/derivative_chart_generator.js +2 -3
  48. package/dist/analysis/derivative_chart_generator.js.map +1 -1
  49. package/dist/analysis/fills_source.d.ts +79 -0
  50. package/dist/analysis/fills_source.d.ts.map +1 -0
  51. package/dist/analysis/fills_source.js +186 -0
  52. package/dist/analysis/fills_source.js.map +1 -0
  53. package/dist/analysis/grid_correction_check.d.ts +4 -20
  54. package/dist/analysis/grid_correction_check.d.ts.map +1 -1
  55. package/dist/analysis/grid_correction_check.js +95 -248
  56. package/dist/analysis/grid_correction_check.js.map +1 -1
  57. package/dist/analysis/trade_profitability.d.ts +18 -19
  58. package/dist/analysis/trade_profitability.d.ts.map +1 -1
  59. package/dist/analysis/trade_profitability.js +136 -280
  60. package/dist/analysis/trade_profitability.js.map +1 -1
  61. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.d.ts.map +1 -1
  62. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.js +3 -4
  63. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.js.map +1 -1
  64. package/dist/analysis/trend_detection/dynamic_weight_chart_generator.d.ts.map +1 -1
  65. package/dist/analysis/trend_detection/dynamic_weight_chart_generator.js +2 -3
  66. package/dist/analysis/trend_detection/dynamic_weight_chart_generator.js.map +1 -1
  67. package/dist/analysis/trend_detection/kalman_chart_generator.d.ts.map +1 -1
  68. package/dist/analysis/trend_detection/kalman_chart_generator.js +2 -3
  69. package/dist/analysis/trend_detection/kalman_chart_generator.js.map +1 -1
  70. package/dist/analysis/trend_detection/regime_chart_generator.d.ts.map +1 -1
  71. package/dist/analysis/trend_detection/regime_chart_generator.js +2 -3
  72. package/dist/analysis/trend_detection/regime_chart_generator.js.map +1 -1
  73. package/dist/analysis/trend_detection/volatility_chart_generator.d.ts.map +1 -1
  74. package/dist/analysis/trend_detection/volatility_chart_generator.js +2 -3
  75. package/dist/analysis/trend_detection/volatility_chart_generator.js.map +1 -1
  76. package/dist/market_adapter/lp_chart_core.d.ts.map +1 -1
  77. package/dist/market_adapter/lp_chart_core.js +2 -3
  78. package/dist/market_adapter/lp_chart_core.js.map +1 -1
  79. package/dist/modules/account_bots.d.ts +1 -18
  80. package/dist/modules/account_bots.d.ts.map +1 -1
  81. package/dist/modules/account_bots.js +1 -1
  82. package/dist/modules/account_bots.js.map +1 -1
  83. package/dist/modules/bitshares-native/serial/operations.d.ts +1 -84
  84. package/dist/modules/bitshares-native/serial/operations.d.ts.map +1 -1
  85. package/dist/modules/bitshares-native/serial/operations.js +1 -38
  86. package/dist/modules/bitshares-native/serial/operations.js.map +1 -1
  87. package/dist/modules/cli_colors.d.ts +0 -1
  88. package/dist/modules/cli_colors.d.ts.map +1 -1
  89. package/dist/modules/constants.js +5 -5
  90. package/dist/modules/constants.js.map +1 -1
  91. package/dist/modules/credential_policy.d.ts +1 -25
  92. package/dist/modules/credential_policy.d.ts.map +1 -1
  93. package/dist/modules/credential_policy.js +1 -1
  94. package/dist/modules/credential_policy.js.map +1 -1
  95. package/dist/modules/credential_runtime.d.ts +1 -6
  96. package/dist/modules/credential_runtime.d.ts.map +1 -1
  97. package/dist/modules/credential_runtime.js +1 -1
  98. package/dist/modules/credential_runtime.js.map +1 -1
  99. package/dist/modules/credential_session_cache.d.ts +1 -5
  100. package/dist/modules/credential_session_cache.d.ts.map +1 -1
  101. package/dist/modules/credential_session_cache.js +1 -1
  102. package/dist/modules/credential_session_cache.js.map +1 -1
  103. package/dist/modules/crypto/pure_secp256k1.d.ts +1 -2
  104. package/dist/modules/crypto/pure_secp256k1.d.ts.map +1 -1
  105. package/dist/modules/crypto/pure_secp256k1.js +1 -1
  106. package/dist/modules/crypto/pure_secp256k1.js.map +1 -1
  107. package/dist/modules/dexbot_class.d.ts +9 -8
  108. package/dist/modules/dexbot_class.d.ts.map +1 -1
  109. package/dist/modules/dexbot_class.js +9 -8
  110. package/dist/modules/dexbot_class.js.map +1 -1
  111. package/dist/modules/dexbot_cow_runtime.d.ts +87 -7
  112. package/dist/modules/dexbot_cow_runtime.d.ts.map +1 -1
  113. package/dist/modules/dexbot_cow_runtime.js +489 -14
  114. package/dist/modules/dexbot_cow_runtime.js.map +1 -1
  115. package/dist/modules/dexbot_fill_runtime.d.ts +1 -17
  116. package/dist/modules/dexbot_fill_runtime.d.ts.map +1 -1
  117. package/dist/modules/dexbot_fill_runtime.js +1 -1
  118. package/dist/modules/dexbot_fill_runtime.js.map +1 -1
  119. package/dist/modules/dexbot_maintenance_runtime.d.ts +1 -1
  120. package/dist/modules/dexbot_maintenance_runtime.d.ts.map +1 -1
  121. package/dist/modules/dexbot_maintenance_runtime.js +21 -5
  122. package/dist/modules/dexbot_maintenance_runtime.js.map +1 -1
  123. package/dist/modules/dexbot_state_recovery.d.ts +1 -7
  124. package/dist/modules/dexbot_state_recovery.d.ts.map +1 -1
  125. package/dist/modules/dexbot_state_recovery.js +1 -1
  126. package/dist/modules/dexbot_state_recovery.js.map +1 -1
  127. package/dist/modules/launcher/adapter_requirement.d.ts +1 -5
  128. package/dist/modules/launcher/adapter_requirement.d.ts.map +1 -1
  129. package/dist/modules/launcher/adapter_requirement.js +1 -1
  130. package/dist/modules/launcher/adapter_requirement.js.map +1 -1
  131. package/dist/modules/launcher/bot_supervisor.d.ts +1 -6
  132. package/dist/modules/launcher/bot_supervisor.d.ts.map +1 -1
  133. package/dist/modules/launcher/bot_supervisor.js +1 -1
  134. package/dist/modules/launcher/bot_supervisor.js.map +1 -1
  135. package/dist/modules/launcher/foreign_cred_daemon.d.ts +1 -3
  136. package/dist/modules/launcher/foreign_cred_daemon.d.ts.map +1 -1
  137. package/dist/modules/launcher/foreign_cred_daemon.js +1 -1
  138. package/dist/modules/launcher/foreign_cred_daemon.js.map +1 -1
  139. package/dist/modules/launcher/launch_modes.d.ts +1 -2
  140. package/dist/modules/launcher/launch_modes.d.ts.map +1 -1
  141. package/dist/modules/launcher/launch_modes.js +1 -1
  142. package/dist/modules/launcher/launch_modes.js.map +1 -1
  143. package/dist/modules/launcher/market_adapter_runtime.d.ts +1 -3
  144. package/dist/modules/launcher/market_adapter_runtime.d.ts.map +1 -1
  145. package/dist/modules/launcher/market_adapter_runtime.js +1 -1
  146. package/dist/modules/launcher/market_adapter_runtime.js.map +1 -1
  147. package/dist/modules/launcher/monolithic_runtime.d.ts +1 -5
  148. package/dist/modules/launcher/monolithic_runtime.d.ts.map +1 -1
  149. package/dist/modules/launcher/monolithic_runtime.js +1 -1
  150. package/dist/modules/launcher/monolithic_runtime.js.map +1 -1
  151. package/dist/modules/launcher/status_reporting.d.ts +1 -2
  152. package/dist/modules/launcher/status_reporting.d.ts.map +1 -1
  153. package/dist/modules/launcher/status_reporting.js +1 -1
  154. package/dist/modules/launcher/status_reporting.js.map +1 -1
  155. package/dist/modules/order/export.d.ts +1 -8
  156. package/dist/modules/order/export.d.ts.map +1 -1
  157. package/dist/modules/order/export.js +1 -1
  158. package/dist/modules/order/export.js.map +1 -1
  159. package/dist/modules/order/grid.d.ts +7 -0
  160. package/dist/modules/order/grid.d.ts.map +1 -1
  161. package/dist/modules/order/grid.js +52 -98
  162. package/dist/modules/order/grid.js.map +1 -1
  163. package/dist/modules/order/grid_reconcile_internal.d.ts +1 -10
  164. package/dist/modules/order/grid_reconcile_internal.d.ts.map +1 -1
  165. package/dist/modules/order/grid_reconcile_internal.js +2 -2
  166. package/dist/modules/order/grid_reconcile_internal.js.map +1 -1
  167. package/dist/modules/order/manager.d.ts +10 -2
  168. package/dist/modules/order/manager.d.ts.map +1 -1
  169. package/dist/modules/order/manager.js +20 -7
  170. package/dist/modules/order/manager.js.map +1 -1
  171. package/dist/modules/order/sync_engine.d.ts.map +1 -1
  172. package/dist/modules/order/sync_engine.js +25 -20
  173. package/dist/modules/order/sync_engine.js.map +1 -1
  174. package/dist/modules/order/utils/math.d.ts +1 -10
  175. package/dist/modules/order/utils/math.d.ts.map +1 -1
  176. package/dist/modules/order/utils/math.js +1 -1
  177. package/dist/modules/order/utils/math.js.map +1 -1
  178. package/dist/modules/order/utils/order.d.ts +45 -7
  179. package/dist/modules/order/utils/order.d.ts.map +1 -1
  180. package/dist/modules/order/utils/order.js +160 -15
  181. package/dist/modules/order/utils/order.js.map +1 -1
  182. package/dist/modules/order/utils/system.d.ts +0 -15
  183. package/dist/modules/order/utils/system.d.ts.map +1 -1
  184. package/dist/modules/order/utils/system.js +2 -2
  185. package/dist/modules/order/utils/system.js.map +1 -1
  186. package/dist/modules/process_discovery.d.ts +0 -3
  187. package/dist/modules/process_discovery.d.ts.map +1 -1
  188. package/dist/modules/process_discovery.js +1 -1
  189. package/dist/modules/process_discovery.js.map +1 -1
  190. package/dist/modules/utils/chain_logs.d.ts +1 -2
  191. package/dist/modules/utils/chain_logs.d.ts.map +1 -1
  192. package/dist/modules/utils/chain_logs.js +1 -1
  193. package/dist/modules/utils/chain_logs.js.map +1 -1
  194. package/dist/scripts/analyze-git.js +2 -2
  195. package/dist/scripts/analyze-git.js.map +1 -1
  196. package/dist/scripts/update.js +46 -31
  197. package/dist/scripts/update.js.map +1 -1
  198. package/docs/BITSHARES_ONBOARDING.md +110 -40
  199. package/docs/COPY_ON_WRITE_MASTER_PLAN.md +10 -207
  200. package/docs/COW_INVARIANTS.md +16 -10
  201. package/docs/DEXBOT_COMPARISON.md +3 -3
  202. package/docs/EVOLUTION.md +75 -29
  203. package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md +6 -6
  204. package/docs/GRID_RECALCULATION.md +14 -6
  205. package/docs/GRID_RECONCILE.md +28 -25
  206. package/docs/LIFECYCLE.md +12 -11
  207. package/docs/README.md +48 -42
  208. package/docs/architecture.md +19 -30
  209. package/docs/developer_guide.md +3 -3
  210. package/package.json +1 -1
  211. package/scripts/git-viewer.sh +1 -1
  212. package/tests/README.md +10 -1
@@ -79,11 +79,11 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
79
79
  - `INV-PROJ-002` Preserve on-chain PARTIAL size in projection
80
80
  - If identity is retained (`keepOrderId=true`) and current state is `PARTIAL`, projected size must preserve current on-chain remaining size.
81
81
  - It must not be overwritten by ideal geometric `targetSize`.
82
- - Exception: a `PARTIAL` with a rotation/size-update action targeting its `orderId` does use `targetSize` (the explicit-UPDATE path at `modules/order/utils/validate.ts:959`).
82
+ - Exception: a `PARTIAL` with a rotation/size-update action targeting its `orderId` does use `targetSize` (the explicit-UPDATE path at `modules/order/utils/validate.ts:1129`).
83
83
  - Preserve-path size must be normalized to finite, non-negative value.
84
84
 
85
85
  - `INV-PROJ-003` ACTIVE on-chain projection preserves current size (same as PARTIAL)
86
- - If identity is retained and state is `ACTIVE`, projection preserves current on-chain size via the same `shouldPreserveSize` path as `PARTIAL` (`validate.ts:958-972`).
86
+ - If identity is retained and state is `ACTIVE`, projection preserves current on-chain size via the same `shouldPreserveSize` path as `PARTIAL` (`validate.ts:1129`).
87
87
  - An explicit UPDATE action targeting the `orderId` is required to apply `targetSize`.
88
88
 
89
89
  - `INV-ID-001` Order identity retention rule
@@ -102,7 +102,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
102
102
  - `INV-ACC-003` Cross-bot fund registry invariant (INVARIANT 3)
103
103
  - Shared-account per-bot commitment must not exceed the bot's proportional share of chain balance.
104
104
  - Checked with widened tolerance `max(PERCENT_TOLERANCE * 3, 0.15)`.
105
- - Registry failure logs an error (`accounting.ts:554-563`, with a "CRITICAL FIX: Log as ERROR instead of WARN" comment), not a silent skip.
105
+ - Registry failure logs an error (`order/accounting.ts:574-590`, with a "CRITICAL FIX: Log as ERROR instead of WARN" comment), not a silent skip.
106
106
 
107
107
  ---
108
108
 
@@ -195,6 +195,13 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
195
195
  - Top-of-window partials remain always eligible.
196
196
  - Two PARTIALs sharing a price with no active sibling do not qualify (left to rebalancer).
197
197
 
198
+ - `INV-GRID-004` Slot price equals its genesis level ([GRID_PRICE_INVARIANT.md](GRID_PRICE_INVARIANT.md))
199
+ - `order.price` for a slot-`idx` order must equal `priceForSlot(idx, genesis)`; the genesis ladder is the only authoritative price for a slot.
200
+ - Enforced at all six emission sites (CREATE / UPDATE / CREATE-FALLBACK, RECONCILE-CREATE / RECONCILE-UPDATE, STARTUP-CREATE): an off-grid emission is blocked, never broadcast.
201
+ - Range guards (`isChainPriceOutOfGrid`) are bounds checks, not membership checks — they cannot substitute for this invariant.
202
+ - Adoption keeps the slot's own level (a fill/chain price is metadata, not the slot's price); `loadGrid` repairs a pre-existing off-grid slot price at load.
203
+ - Tests: GPI-001..015 (`tests/test_grid_price_invariant_guard.ts`), GPI-WIRE-001..009 (`tests/test_grid_price_invariant_wiring.ts`), LEGACY-ADOPT/MATERIALIZE/ADOPT-NAME (`tests/test_sync_out_of_grid_defer.ts`).
204
+
198
205
  ---
199
206
 
200
207
  ## Reconcile ([GRID_RECONCILE.md](GRID_RECONCILE.md))
@@ -208,18 +215,17 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
208
215
  - Dust health thresholding applies consistently to both CREATE and rotation destination holes.
209
216
 
210
217
  - `INV-RECON-003` Reconcile cancels duplicate chain orders unconditionally
211
- - When an unmatched order is within `looseTolerance` of an active grid order, it must be cancelled on chain via `_cancelChainOrder` with `releaseUntrackedFunds: true`.
218
+ - An unmatched chain order whose price equals an active same-type grid slot's price (exact slot-price equality via `priceSlotEqual` at the asset precision) is a suspected duplicate and must be cancelled on chain via `_cancelChainOrder` with `releaseUntrackedFunds: true`.
212
219
  - Cancelled IDs are filtered out of `unmatchedParsed` to prevent reprocessing.
213
220
  - No size guard — any duplicate at the same price is a violation.
214
- - `SUSPECTED_DUPLICATE_TOLERANCE_FLOOR` (absolute price floor) is removed — only `tolerance * SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER` is used.
215
- - `SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER` is a file-local constant (`modules/order/grid_reconcile.ts`, value `5`), not a centralized `constants.ts` entry.
221
+ - The earlier fuzzy `SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER` (5× `calculatePriceTolerance`) and `SUSPECTED_DUPLICATE_TOLERANCE_FLOOR` are removed; only exact price-level equality triggers a reconcile cancel.
216
222
 
217
223
  - `INV-RECON-004` Rebalance must not convert on-chain slots to SPREAD via CREATE
218
224
  - `performSafeRebalance` must not emit `CREATE` actions that convert existing on-chain slots into SPREAD orders.
219
225
  - On-chain mid-slot must keep its BUY/SELL type before commit.
220
226
 
221
227
  - `INV-RECON-005` Extreme placement ordering
222
- - BUY placements must use nearest available free slots first (descending price, so the nearest-to-center slots fill first — `validate.ts:465-468`).
228
+ - BUY placements must use nearest available free slots first (descending price, so the nearest-to-center slots fill first — `order/utils/order.ts` `buildOutsideInPairGroups`).
223
229
  - SELL placements must use nearest available free slots first (ascending price).
224
230
 
225
231
  ---
@@ -228,7 +234,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
228
234
 
229
235
  - `INV-BATCH-001` Illegal state batch abort
230
236
  - `executeBatch` throws `ILLEGAL_SPREAD_STATE` on an illegal grid layout (emitted at `modules/order/utils/validate.ts`, propagated via `modules/order/manager.ts` `_throwOnIllegalState`).
231
- - The `_handleBatchHardAbort` catch for `ILLEGAL_ORDER_STATE` (`dexbot_state_recovery.ts:133`) is a test-only dead branch — production never emits that code; only the test stub at `tests/test_patch17_invariants.ts:396` uses it.
237
+ - The `_handleBatchHardAbort` catch for `ILLEGAL_ORDER_STATE` (`dexbot_class.ts:455`) is a test-only dead branch — production never emits that code; only a test stub uses it.
232
238
  - In production, recovery + cooldown are armed on the next maintenance tick via `_abortFlowIfIllegalState` (the `INV-MAINT-002` path), returning `abortedForIllegalState: true` to the caller. The caller does not need to return immediately; the maintenance tick handles recovery.
233
239
  - Hard abort triggers one immediate recovery sync (`_triggerStateRecoverySync`) plus arms one maintenance cooldown cycle (`_maintenanceCooldownCycles = Math.max(current, 1)`).
234
240
 
@@ -247,7 +253,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
247
253
  - NOT virtualize the slot.
248
254
  - Preserve `orderId` until sync reconciles it.
249
255
  - NOT mark the order as stale-cleaned.
250
- - Fast path: if the batch result indicates `ORDER_SIZE_DRIFT_TARGETED` (`dexbot_state_recovery.ts:263`), a targeted repair applies the correction directly and skips `_triggerStateRecoverySync`.
256
+ - Fast path: if the batch result indicates `ORDER_SIZE_DRIFT_TARGETED` (`dexbot_state_recovery.ts:269`), a targeted repair applies the correction directly and skips `_triggerStateRecoverySync`.
251
257
 
252
258
  ---
253
259
 
@@ -271,7 +277,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
271
277
  - `INV-REG-001` Cross-bot allocation ≤ proportional share
272
278
  - Per-bot committed amounts (sum of on-chain orders) must not exceed `totalChainBalance × allocatedPercent`.
273
279
  - Violation triggers an error-level log entry (not silent), with tolerance `max(PERCENT_TOLERANCE * 3, 0.15)`.
274
- - Registry registration is pre-flight + atomic; only shared-account bots register (`dexbot.ts:535` filters `accountGroups[a].length > 1`), and registration completes before any shared-account bot starts.
280
+ - Registry registration is pre-flight + atomic; only shared-account bots register (`dexbot.ts:611` filters `accountGroups[a].length > 1`), and registration completes before any shared-account bot starts.
275
281
  - Release happens in `DEXBot.shutdown`.
276
282
 
277
283
  - `INV-REG-002` Async-locked registry writes
@@ -1,7 +1,7 @@
1
1
  # DEXBot vs DEXBot2 — Detailed Comparison Report
2
2
 
3
3
  > **Date:** 2026-09-02 *(metrics refreshed against local source trees)*
4
- > **Scope:** Full architectural, functional, and operational comparison between the original [DEXBot](https://github.com/Codaone/DEXBot) (Python, v1.0.0) and DEXBot2 (TypeScript, v1.6.2).
4
+ > **Scope:** Full architectural, functional, and operational comparison between the original [DEXBot](https://github.com/Codaone/DEXBot) (Python, v1.0.0) and DEXBot2 (TypeScript, v1.6.4).
5
5
  > **Audience:** Developers, contributors, and operators evaluating or migrating between the two projects.
6
6
 
7
7
  ---
@@ -36,7 +36,7 @@
36
36
 
37
37
  | Attribute | DEXBot (original) | DEXBot2 |
38
38
  |---|---|---|
39
- | **Release Track** | 1.0.0 | v1.6.2 |
39
+ | **Release Track** | 1.0.0 | v1.6.4 |
40
40
  | **Language** | Python 3.6+ | TypeScript 5.x |
41
41
  | **Status** | Released 1.0.0, unmaintained | Active development |
42
42
  | **Last Repo Activity** | May 23, 2020 | 2026-09-02 |
@@ -776,7 +776,7 @@ Where:
776
776
 
777
777
  | Metric | DEXBot | DEXBot2 |
778
778
  |---|---|---|
779
- | **Release Track** | 1.0.0 | v1.6.2 |
779
+ | **Release Track** | 1.0.0 | v1.6.4 |
780
780
  | **Active Since** | ~2018 | December 2025 |
781
781
  | **Last Commit** | May 23, 2020 | 2026-09-02 |
782
782
  | **Total Commits** | 2281 | 2,125 (v1.4.25) |
package/docs/EVOLUTION.md CHANGED
@@ -2,14 +2,19 @@
2
2
 
3
3
  ## Executive Summary
4
4
 
5
- DEXBot2 is a sophisticated decentralized exchange trading bot for the BitShares blockchain. This report documents the complete evolution of the project from its inception in December 2025 through the current 1.6.2 stable release.
5
+ DEXBot2 is a sophisticated decentralized exchange trading bot for the BitShares blockchain. This report documents the complete evolution of the project from its inception in December 2025 through the current 1.6.4 stable release.
6
6
 
7
7
  ### Key Milestones
8
8
  - **Project Inception**: December 2, 2025
9
- - **Growth Phase**: 2,223 commits over ~9 active months
9
+ - **Growth Phase**: 2,247 commits over ~9 active months
10
10
  - **Code Maturity**: Evolution from basic utilities to a ~100,000+ LoC intelligent TypeScript system
11
- - **Stability**: Progression from manual testing to a suite of 284 automated test files
12
- - **Releases**: 106 release entries (v0.1.0 to v1.6.2)
11
+ - **Stability**: Progression from manual testing to a suite of 306 automated test files
12
+ - **Releases**: 108 release entries (v0.1.0 to v1.6.4)
13
+
14
+ > **Post-1.0.0 "why":** the thematic story behind the hardening releases — root cause, recurring
15
+ > bug families, and lessons — lives in
16
+ > [ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md](ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md).
17
+ > This report stays chronological.
13
18
 
14
19
  ---
15
20
 
@@ -61,10 +66,62 @@ v1.4.12 completed the module transition to native ES modules (root + claw `"type
61
66
 
62
67
  ### Phase 9: Post-ESM Cleanup, Consolidation & Hardening (August 2026)
63
68
 
64
- The post-ESM releases consolidated state, code, and tooling while hardening the grid engine. **State & packaging** (v1.4.14–v1.4.17): all user/runtime state centralized on a resolver-derived profiles dir (`~/.config/dexbot2/profiles`) safe from re-clones, read-only prefixes, and npm wipes; divergence surplus/hole pairs became in-place order rotations; npm auto-update shipped; duplicated EC-crypto/settings/asset-resolution code collapsed; dead exports purged; analysis tooling moved under strict TypeScript. **Grid hardening** (v1.4.19–v1.4.21): COW broadcasts capped at `MAX_OPS_PER_BROADCAST` (4) with chunked retry-on-uncertain; boundary promotion and persisted-boundary restore gated against gap-floor overrun poison behind a shared sell-rail ceiling enforced at commit and restore time; spread-collapse fixed via the shared `isSlotInRail` filter; silent-failure runtime defects from a modules-wide audit fixed (NaN fund-invariant tolerance, always-flush fill store, double-decremented fill guard). **Tooling & UX** (v1.4.20–v1.4.22): live/research clip parity with centralized chart sliders and analysis outputs on the central path resolver; claw logic deduplicated with hardened error paths; browser storage adapter persists deletions; editor green/red input feedback extended to funds and prices; compile-first runtime completed — tsx removed entirely, every entry point and the test suite running compiled dist under plain node through frozen-ESM-safe seams, plus exact AMA cold-start bootstrap sizing and research tools unified onto production slope/bounds math. Post-v1.4.22 work followed the same themes: bot-fitting backtests re-modeled on the production grid lifecycle, analysis tooling consolidated onto market_adapter sources, Kibana proxy-reset hardening, and TradingView price-axis interaction. **v1.4.24** fixed native fill-gap recovery and LP pricing; **v1.4.25** freezes genesis price-slots and hardens grid orphan/self-trade/fill-guard/shutdown paths, plus whitelist scoped `--bot` overwrite and Range quality legend.
69
+ The post-ESM releases consolidated state, packaging, and tooling while hardening the grid engine — profile state on a resolver-derived `~/.config/dexbot2` dir, in-place order rotations, npm auto-update, dead-code purge, and a compile-first runtime (tsx removed; every entry point and the test suite run compiled `dist` under plain node). Grid work capped COW broadcasts and chunked retry-on-uncertain, gated boundary promotion and persisted-restore against gap-floor poison, fixed spread-collapse via the shared `isSlotInRail` filter, and cleared silent-failure defects from a modules-wide audit. **v1.4.24** fixed native fill-gap recovery and LP pricing; **v1.4.25** froze genesis price-slots and hardened orphan/self-trade/fill-guard/shutdown paths.
65
70
 
66
71
  ---
67
72
 
73
+ ## Architecture Deep-Dive: COW & Memory Tracking
74
+
75
+ Two mechanisms shaped the order engine after the browser/TypeScript era: the Copy-on-Write grid
76
+ and the memory-only integer tracking model. The construction detail that used to live in
77
+ `COPY_ON_WRITE_MASTER_PLAN.md` and `architecture.md` is recorded here; those docs now describe
78
+ only the current design.
79
+
80
+ ### Copy-on-Write: three eras
81
+
82
+ - **Era 0 — original optimistic state (pre-v1.0):** the master grid was mutated directly during
83
+ planning, with no isolation or rollback. A sudden market move corrupted in-flight state (the
84
+ "Price Jump" incident — planning mutations applied straight to the master grid).
85
+ - **Era 1 — frozen master state (v1.0):** `Object.freeze()` on the master Map and `deepFreeze()`
86
+ on order objects; every `_applyOrderUpdate` creates a new frozen Map via the immutable-swap
87
+ pattern. Retained as defense-in-depth — it catches accidental in-place mutation of
88
+ `manager.orders`.
89
+ - **Era 2 — Copy-on-Write (v2.0, current):** clone the master into a `WorkingGrid`, plan and
90
+ broadcast on the clone, commit atomically on blockchain success (discard on failure). True
91
+ transactional semantics; the master is never in an intermediate state.
92
+
93
+ The production code layers Era 1 and Era 2: freeze provides runtime mutation enforcement, COW
94
+ provides the plan → broadcast → commit/discard lifecycle.
95
+
96
+ ### COW construction milestones (February–April 2026)
97
+
98
+ - `modules/order/working_grid.ts` (`WorkingGrid`: clone, delta, stale tracking) and
99
+ `COW_PERFORMANCE` thresholds added.
100
+ - `performSafeRebalance` → `_applySafeRebalanceCOW`; `buildDelta`; `_commitWorkingGrid` atomic
101
+ swap.
102
+ - COW broadcast path (`_updateOrdersOnChainBatchCOW`); legacy rollback code removed.
103
+ - Selective-abort fill strategy: individual fills continue, full-side updates block.
104
+ - Divergence corrections and `updateGridFromBlockchainSnapshot` migrated to the COW pattern.
105
+ - Atomic boundary shifts (patch 20): `pendingBoundaryIdx` carries boundary changes through the
106
+ pipeline and applies them only at `_commitWorkingGrid`, so boundary position and slot BUY/SELL
107
+ roles never transiently disagree during blockchain execution.
108
+ - Validation suites added: COW core, commit-guard, concurrent-fill, divergence-correction, and
109
+ stale-plan/stack-discipline tests.
110
+
111
+ ### Memory-only integer tracking
112
+
113
+ - **Raw order cache (`rawOnChain`):** grid slots store the exact blockchain order integers
114
+ (satoshis); seeded from broadcast arguments on placement, updated in place on partial fills,
115
+ refreshed on updates/rotations.
116
+ - **Chain-free planning:** redundant `readOpenOrders()` calls were removed from the size-update
117
+ and rotation builders (`_buildSizeUpdateOps()`, `_buildRotationOps()`), and the
118
+ `computeVirtualOpenOrders()` virtual-order computation was dropped; `buildUpdateOrderOp`
119
+ gained an optional `cachedOrder` and returns `finalInts`.
120
+ - **Result:** batch updates and rotations run without blockchain fetches; only placements and
121
+ recovery syncs query the chain (~10–20× faster high-frequency operations).
122
+ - **Self-healing:** a failed memory-driven transaction triggers a full state-recovery sync so the
123
+ internal ledger stays consistent with the chain.
124
+
68
125
  ## Technical Challenges & Solutions
69
126
 
70
127
  | Challenge | Solution | Impact |
@@ -86,41 +143,30 @@ Evolved from a basic README to a comprehensive framework (50+ docs entries, 80%+
86
143
 
87
144
  ## Post-1.0.0 Status
88
145
 
89
- **Completed**: browser-safe core (140+ files, portable abstractions, pure-JS crypto); credit/MPA runtime (multi-asset collateral, oversize deal splitting); storage-adapter I/O centralization; self-healing (structural resync, subscription watchdog, bot/daemon broadcast-deadlock recovery); Kibana-driven trade PnL analytics; multi-round AMA refits; documented subsystem invariants; bot identity enforcement; credit-only mode; Docker support; npm package with lockfile sync; hardened grid order engine (COW pipeline, zero-amount prevention, fill batching, dust detection, spread correction).
146
+ **Completed**: browser-safe core; credit/MPA runtime; storage-adapter I/O centralization; self-healing recovery; Kibana PnL analytics; credit-only mode; Docker support; npm package. For the grid order engine arc specifically (COW pipeline, orphan/self-trade/fill-guard hardening, invariants) see [ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md](ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md) §R4.
90
147
 
91
148
  **Planned**: backtesting engine (historical candle replay via exchange abstraction); injectable interfaces at call boundaries; SQLite persistence + Zod validation at the blockchain boundary; Telegram bot (**not yet implemented**) — owner-gated monitoring (`/status`, `/orders`, `/grid`, `/balance`) and opt-in+confirm gated control (`/start`, `/stop`, `/pause`); DEXBot is the only writer, private keys never reach the module (`TELEGRAM` block + `DEXBOT_TELEGRAM_TOKEN` env).
92
149
 
93
150
  ## Version History
94
151
 
95
- Compact, era-level view; per-release commit detail lives in [CHANGELOG.md](../CHANGELOG.md).
152
+ Compact, era-level view. Per-release detail lives in [CHANGELOG.md](../CHANGELOG.md); the thematic post-1.0.0 story in [ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md](ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md).
96
153
 
97
154
  | Era | Commits | Theme |
98
155
  |-----|--------:|-------|
99
156
  | v0.1.0 → v0.6.0 | 1,217 | Foundation → COW architecture, strategy/sync engine, credential daemon, AMA prototype, credit/MPA runtime |
100
- | v0.6.0 → v1.0.0 | 309 | Zero-dependency & TS migration, native BitShares, fill detection overhaul, first stable release |
101
- | v1.0.0 → v1.1.0 | 85 | Post-stable hardening, PnL analytics, auto-update, broadcast deadlock fixes |
102
- | v1.1.0 → v1.3.3 | 114 | AMA refits, credit-only mode, COW recovery hardening, runtime extraction |
103
- | v1.3.3 → v1.4.8 | 74 | CJS→ESM completion, concurrency correction, uncertain-broadcast safety, truncated-read ambiguity |
104
- | v1.4.8 → v1.4.13 | 45 | Native ESM runtime, broadcast serialization, onboarding |
105
- | v1.4.13 → v1.4.19 | 36 | Profile-state centralization, code consolidation, per-broadcast op cap |
106
- | v1.4.19 → v1.4.20 | 5 | Grid boundary promotion hardening, recovery poison gate, analysis output centralization |
107
- | v1.4.20 → v1.4.21 | 15 | Runtime audit fixes, claw dedup hardening, boundary ceiling alignment, editor color feedback |
108
- | v1.4.21 → v1.4.22 | 4 | tsx removal completion (dist-only runtime + tests), exact AMA bootstrap sizing, research-tool production parity |
109
- | v1.4.22 → v1.4.23 | 12 | Even geometric AMA ladder, BTS fee-carve fix, sub-1x price-bound rejection, tradingview axis restore, doc realignment |
110
- | v1.4.23 → v1.4.24 | 3 | Native fill gap recovery with eager coalesced retry, LP collateral offer-first pricing |
111
- | v1.4.24 → v1.4.25 | 26 | Genesis-frozen price-slots, self-trade & fill-guard hardening, orphan & gap-band fixes, trigger/shutdown hardening, bot poll, grid monotonicity gate, whitelist scoped overwrite, Range legend |
112
- | v1.4.25 → v1.5.0 | 13 | Credit overview CLI + whitelist-scoped CR on shared pricing math, one-step TradingView chart, daemon-safe reload, offline account-ID cache, case-insensitive bot identity, stale-pivot guard fix, partial-surplus rotation clamp, adapter ownership centralization, op-77 bot discovery split |
113
- | v1.5.0 → v1.5.1 | 11 | Gap-evacuation guard allowance + rail-typed holes, persisted streaks with cancel-only teeth, vacated-rail refill, adoption/accounting/duplicate-guard hardening |
114
- | v1.5.1 → v1.5.2 | 10 | Sync rejection handling across pass-1/pass-2 adoption, crossing-guard candidate sharing, empty-read confirmation, broadcast-price CREATE validation, stamped gap-evacuation re-proof, credit whole-account display + expiry, offline export fill-block derivation |
115
- | v1.5.2 → v1.5.3 | 6 | Boundary ownership hardening (fund-driven sync removal, guard-skipped refill hold), COW broadcast/reconcile dedup, TradingView bot-grid range highlight, createOrder unknown-id materialize-or-error, curve-comparison docs |
116
- | v1.5.3 → v1.6.0 | 37 | Node-failure strike ledger and broadcast-deferred fill rebalancing, trust-chain free-balance heal with deferred-drain tolerance, bidirectional grid-regeneration trigger, TradingView order overlay and chart pref namespacing, credit short-offer id display, live-config pickup (issue #27), reserve ladder anchored at resolved bounds + live-grid rail edges with single-source ordering and exact-size activation, owed-crawl persistence across refused broadcasts/restarts + hold-aware reload-safe lifecycle, fill-anchored boundary recovery + poisoned persisted-boundary erase, startup rail gate + static-center crawl fold, all `*-deferred` holds non-blocking + hold metrics surfaced, opt-in MPA price-feed charts + range-aware shared candle cache, range-band span parity on grid-less charts, orange range zone widened to 1.40x, docs reserve-ladder sweep, reserve-deficit targeted-sync trigger with window-exclusion counting, matched-surplus startup excess planning, shelf-order guards across reserve classification/placement/startup cancels/size recalc (issue #27 follow-ups), unified Kibana candle cache on runCachedWindows with fetch retry budgets, genuine-coverage LP window reuse, feed volume/AMA timeframe alignment, TradingView monthly candles/stat badges/rigid pan/volume toggle/feed affordance, credit full offer id + empty-pair Curr. CR hiding, live-config onboarding note |
117
- | v1.6.0 → v1.6.1 | 3 | Never-run-stale hardening (level-triggered deferred-fill retry, stale-totals fill parking, out-of-spread watchdog, region-end fan-out, one-sided spread honesty), whitelist range-scaling opt-in defaults, live-save vs reset vs reload docs + power-law paper restructure |
118
- | v1.6.1 → v1.6.2 | 6 | Grid-price invariant + resync escalation, gap-slot batch sizing, shard candle cache + chart fixes, recovery stranding tolerance, fill-counter hygiene, dead-node prune + trace single-sourcing, obsolete-doc removal |
157
+ | v0.6.0 → v1.0.0 | 309 | Zero-dependency & TS migration, native BitShares, fill-detection overhaul, first stable release |
158
+ | v1.0.0 → v1.3.3 | 199 | Post-stable hardening, PnL analytics, auto-update, broadcast-deadlock fixes, AMA refits, credit-only mode, COW recovery hardening, runtime extraction |
159
+ | v1.3.3 → v1.4.13 | 119 | CJS→ESM completion, concurrency correction, uncertain-broadcast & truncated-read safety, native ESM runtime, broadcast serialization, onboarding |
160
+ | v1.4.13 → v1.4.25 | 101 | Profile-state centralization, consolidation, per-broadcast op cap, grid boundary/recovery hardening, tsx removal, genesis-frozen price-slots, self-trade & orphan fixes |
161
+ | v1.4.25 → v1.5.3 | 40 | Credit overview + whitelist-scoped CR, TradingView tooling, daemon-safe reload, gap-evacuation/rail-hole hardening, sync adoption hardening, boundary ownership |
162
+ | v1.5.3 → v1.6.0 | 37 | Node-failure ledger, grid regeneration, reserve ladder, live-config pickup, owed-crawl persistence, fill-anchored boundary recovery, TradingView overlay |
163
+ | v1.6.0 → v1.6.3 | 12 | Never-run-stale hardening, whitelist range-scaling opt-in, grid-price invariant, shard candle cache, correction-queue staleness, final pre-broadcast pivot gate |
164
+ | v1.6.3 → v1.6.4 | 18 | Fund-driven spread correction, gapSlots+1 batch cap, VIRTUAL RMS divergence, sync-lock log fix, invariant-doc contract, analysis shared modules, window-aware profitability annualisation, portable chart exports, range-threshold restore, dead-code purge, doc consolidation |
119
165
 
120
166
  ---
121
167
 
122
168
  **Report Originally Generated**: February 19, 2026
123
- **Last Updated**: September 14, 2026
124
- **Total Commits**: 2,223
125
- **Date Range**: December 2, 2025 – September 14, 2026
169
+ **Last Updated**: September 22, 2026
170
+ **Total Commits**: 2,247
171
+ **Date Range**: December 2, 2025 – September 22, 2026
126
172
  **Repository**: DEXBot2 (BitShares DEX Trading Bot)
@@ -67,7 +67,7 @@ Mixed BUY/SELL batches are validated per asset using a signed-delta **peak** run
67
67
 
68
68
  #### Implementation Location
69
69
 
70
- File: `modules/dexbot_cow_runtime.ts` — `validateOperationFunds()` (line 1488), called from the COW batch broadcast path at line 2908. `modules/dexbot_class.ts` exposes a thin wrapper `_validateOperationFunds()` (line 1122).
70
+ File: `modules/dexbot_cow_runtime.ts` — `validateOperationFunds()` (line 1655), called from the COW batch broadcast path at line 4457.
71
71
 
72
72
  ```javascript
73
73
  // Per-asset peak requirement vs. quantized chain-free snapshot.
@@ -114,9 +114,9 @@ See [developer_guide.md#order-state-helper-functions](developer_guide.md#order-s
114
114
 
115
115
  **Mechanism**: Fill events arrive via `modules/dexbot_fill_runtime.ts` (the fill-runtime module), which pushes them into `bot._incomingFillQueue` (declared in `modules/dexbot_class.ts`). The drain loop in `dexbot_fill_runtime.ts` then chunks the queue into capped batches and calls `modules/order/manager.ts::processFilledOrders` (line 1438) once per chunk to run the full rebalance pipeline.
116
116
 
117
- **Batch Sizing Algorithm**: Batch size is derived from the grid gap-slot count (`DEXBot._getGapSlotBatchSize`): a queue depth at or below gapSlots is processed as one unified batch; deeper queues are chunked into repeated batches of gapSlots (the last chunk may be smaller). The same gap-slot size caps order operations per broadcast transaction (oversized op batches are split into sequential broadcasts).
117
+ **Batch Sizing Algorithm**: Batch size is derived from the grid gap-slot count + 1 (`DEXBot._getGapSlotBatchSize`): a queue depth at or below gapSlots+1 is processed as one unified batch; deeper queues are chunked into repeated batches of gapSlots+1 (the last chunk may be smaller). The same gapSlots+1 size caps order operations per broadcast transaction (oversized op batches are split into sequential broadcasts).
118
118
 
119
- **Configuration**: no fixed constant — both `FILL_PROCESSING.MAX_FILL_BATCH_SIZE` and `COW_PERFORMANCE.MAX_OPS_PER_BROADCAST` were removed; batch sizing follows the grid gap-slot count.
119
+ **Configuration**: no fixed constant — both `FILL_PROCESSING.MAX_FILL_BATCH_SIZE` and `COW_PERFORMANCE.MAX_OPS_PER_BROADCAST` were removed; batch sizing follows the grid gap-slot count + 1.
120
120
 
121
121
  #### Fill Batch Processing Timeline
122
122
 
@@ -299,7 +299,7 @@ When a fill occurs, the boundary shifts to "follow" the price.
299
299
 
300
300
  ### 3.2 Global Side Capping
301
301
 
302
- Budgets are dynamic. The bot calculates `TotalSideBudget` from `funds.allocated.{buy,sell}` (the `botFunds`-capped capital per side — see §1.3). This ensures the bot never attempts to deploy more than its configured share of account capital, even when the account holds additional free balance for other bots or manual trading.
302
+ Budgets are dynamic. The bot calculates the per-side budget via `getSideBudget` from `funds.allocated.{buy,sell}` (the `botFunds`-capped capital per side — see §1.3). This ensures the bot never attempts to deploy more than its configured share of account capital, even when the account holds additional free balance for other bots or manual trading.
303
303
 
304
304
  **Safety Check:**
305
305
  If the calculated ideal grid requires more capital than available in the allocation, the *increase* is capped.
@@ -751,7 +751,7 @@ Quantization has a single source of truth: `quantizeFloat()` in `modules/order/u
751
751
 
752
752
  #### 5.5.4 Relationship to Fund Validation
753
753
 
754
- The corrected fund validation in `_validateOperationFunds()` uses quantized values:
754
+ The corrected fund validation in `validateOperationFunds()` uses quantized values:
755
755
 
756
756
  ```javascript
757
757
  // Check: Does required amount fit in available balance?
@@ -801,4 +801,4 @@ To prevent "Time-of-Check to Time-of-Use" errors:
801
801
  **TOCTOU protection in `processFillAccounting`.** `_buildBtsDeferredRefundAdjustment` reads `btsFeeState` from `mgr.orders` while the order lock is held — the lock is acquired before accounting runs, and the POST-RESET and BOOTSTRAP tracked-fill accounting paths follow the same locking pattern.
802
802
 
803
803
  ---
804
- *Technical Reference for DEXBot2 v1.6.2 release*
804
+ *Technical Reference for DEXBot2 v1.6.4 release*
@@ -25,7 +25,7 @@ rebuild around that snapshot.
25
25
  | **Initial AMA Snapshot** | Market adapter | Bot has no accepted AMA `gridCenterPrice` yet | Write `dynamicgrid.json`, then write a trigger file | Full grid resync around the first accepted AMA center |
26
26
  | **AMA Center Move** | Market adapter | Current AMA center moves past the configured delta threshold | Write `dynamicgrid.json`, then write a trigger file | Full grid resync around the new accepted AMA center |
27
27
  | **AMA Slope Range Move** | Market adapter | Range-scaling bot's accepted AMA-slope baseline moves past threshold | Write range-scaling fields to `dynamicgrid.json`, then write a trigger file | Full grid resync with updated asymmetric range/offset data |
28
- | **RMS Structural Divergence** | Bot runtime maintenance | Current grid shape diverges from persisted/on-chain grid by RMS threshold | Refresh `gridCenterPrice` from latest `amaCenterPrice`, then run full grid resync | Full grid resync from latest market-adapter snapshot |
28
+ | **RMS Structural Divergence** | Bot runtime maintenance | Current grid shape diverges from persisted grid (ACTIVE + VIRTUAL) by RMS threshold | Refresh `gridCenterPrice` from latest `amaCenterPrice`, then run full grid resync | Full grid resync from latest market-adapter snapshot |
29
29
  | **Available-Funds Resize** | Bot runtime maintenance | Filled-order proceeds exceed `GRID_REGENERATION_PERCENTAGE` (grow), or grid-tracked size exceeds allocation by that threshold after fund removal (shrink) | Recalculate affected side/order sizes through maintenance logic | Order-size/grid maintenance update, not an AMA recenter trigger |
30
30
 
31
31
  Each source is evaluated independently. Market-adapter full-resync requests are
@@ -313,17 +313,25 @@ so the adapter converts them when loading overrides. New settings should use
313
313
 
314
314
  ### What It Does
315
315
  Compares the **calculated grid** currently held by the bot with the
316
- **persisted/on-chain grid state**. When structural divergence exceeds the
317
- threshold, the bot performs a full grid resync.
316
+ **persisted grid state** (ACTIVE on-chain orders plus VIRTUAL planned
317
+ reservations). When structural divergence exceeds the threshold, the bot
318
+ performs a full grid resync.
319
+
320
+ **Scope (per side):** the metric covers **ACTIVE + VIRTUAL** orders. PARTIAL
321
+ orders are excluded (expected to deviate mid-fill) and SPREAD placeholders are
322
+ excluded (size-0). VIRTUAL slots carry the planned reservation for unplaced
323
+ rail slots (`funds.virtual`), so a persisted-vs-ideal drift there moves
324
+ `Available = ChainFree − Virtual − fees` and is treated as structural.
318
325
 
319
326
  **Why it matters:** Order fills, rotations, and fee deductions can make the
320
- active grid shape drift away from the stored/on-chain picture. RMS divergence
327
+ grid shape drift away from the stored picture. RMS divergence
321
328
  detects that structural drift. Once it crosses the threshold, DEXBot rebuilds
322
329
  from the latest market-adapter snapshot instead of trying to keep patching the
323
330
  old shape.
324
331
 
325
332
  The RMS calculation compares the runtime grid (calculated from the bot's
326
- config and live dynamic weights) against the persisted/on-chain grid state.
333
+ config and live dynamic weights) against the persisted grid state
334
+ (ACTIVE + VIRTUAL).
327
335
  Crossing the threshold only changes the follow-up action: the bot refreshes
328
336
  `gridCenterPrice` from the latest `amaCenterPrice` in `dynamicgrid.json`,
329
337
  then runs the full resync path.
@@ -376,7 +384,7 @@ and dedupes while one resync is already pending or running.
376
384
  ### How It Works
377
385
 
378
386
  1. **Grid Engine** (`modules/order/grid.ts`) calculates the ideal grid state
379
- 2. Compares with the actual blockchain grid state after fills/rotations
387
+ 2. Compares with the persisted grid state — ACTIVE on-chain orders plus VIRTUAL planned reservations — after fills/rotations
380
388
  3. Computes RMS divergence metric:
381
389
  ```
382
390
  RMS = √(mean of ((calculated - persisted) / persisted)²)
@@ -13,7 +13,7 @@ Reconciliation aligns the bot's model with on-chain reality. It runs at startup
13
13
 
14
14
  ### Why Not Cancel Everything
15
15
 
16
- - No atomic cancel+create on BitShares — `cancel_order` cancels the full order; there is no partial size reduction
16
+ - No atomic cancel+create on BitShares — `limit_order_cancel` cancels the full order; there is no partial size reduction
17
17
  - A full teardown leaves the bot unable to trade during the rebuild window
18
18
 
19
19
  ---
@@ -132,8 +132,8 @@ If Phase 2 partially succeeds (some cancels, some creates fail), there is no rol
132
132
  ### Timeouts and Read Coverage
133
133
 
134
134
  - **No per-attempt race** around the reconcile itself — the 1.4.8 change removed it to avoid orphaning mid-batch broadcasts (see the [`recalculateGrid`](../modules/order/grid.ts) call site in `modules/order/grid.ts`).
135
- - The whole resync is bounded by a **10-minute total timeout** (`PIPELINE_TIMING.TIMEOUT_MS * 2` at `grid.ts:1184`), applied via `Promise.race` at `grid.ts:1279`.
136
- - Every internal chain read goes through `readOpenOrdersGuarded` (`chain_orders.ts:610`) with the 30s / 3-retry / node-failover standard, and empty/truncated reads are treated as **ambiguous** — never as authoritative absence.
135
+ - The whole resync is bounded by a **10-minute total timeout** (`PIPELINE_TIMING.TIMEOUT_MS * 2` at `grid.ts:1511`), applied via `Promise.race` at `grid.ts:1617`.
136
+ - Every internal chain read goes through `readOpenOrdersGuarded` (`chain_orders.ts:608`) with the 30s / 3-retry / node-failover standard, and empty/truncated reads are treated as **ambiguous** — never as authoritative absence.
137
137
 
138
138
  ---
139
139
 
@@ -141,7 +141,7 @@ If Phase 2 partially succeeds (some cancels, some creates fail), there is no rol
141
141
 
142
142
  ### Fresh Grid Guard (`matchedOnGrid > 0`)
143
143
 
144
- **`grid_reconcile_internal.ts:1600`**
144
+ **`grid_reconcile_internal.ts:1738`**
145
145
 
146
146
  When a brand-new grid is generated, every slot is VIRTUAL — `matchedOnGrid = 0`. Without a guard, every on-chain order appears "unmatched" and would be cancelled as excess:
147
147
 
@@ -155,27 +155,27 @@ When `matchedOnGrid === 0` AND scaling up (`neededSlots > 0`), excess cancellati
155
155
 
156
156
  ### Grid-Edge Lock
157
157
 
158
- **`grid_reconcile_internal.ts:244`** — `_isGridEdgeFullyActive` detects when the grid boundary is fully active (all slots on-chain) before cancelling excess orders.
158
+ **`grid_reconcile_internal.ts:355`** — `_isGridEdgeFullyActive` detects when the grid boundary is fully active (all slots on-chain) before cancelling excess orders.
159
159
 
160
- When all outermost orders of a side are ACTIVE with `orderId`, all balance is committed to the edges. Cancel the **largest** order among the update candidates (`_cancelLargestOrder`, line 314) to free maximum funds with minimum operations, since the DEX does not expose partial-reduce in one operation. The cancelled slot gets a replacement create.
160
+ When all outermost orders of a side are ACTIVE with `orderId`, all balance is committed to the edges. Cancel the **largest** order among the update candidates (`_cancelLargestOrder`, line 425) to free maximum funds with minimum operations, since the DEX does not expose partial-reduce in one operation. The cancelled slot gets a replacement create.
161
161
 
162
- Detection (`_isGridEdgeFullyActive`, line 244): sort orders by price (BUY descending, SELL ascending), and check the outermost ones are all `isOrderPlaced()`.
162
+ Detection (`_isGridEdgeFullyActive`, line 355): sort orders by price (BUY descending, SELL ascending), and check the outermost ones are all `isOrderPlaced()`.
163
163
 
164
- ### Duplicate Tolerance (5× Multiplier)
164
+ ### Duplicate Cancellation
165
165
 
166
- **`grid_reconcile.ts:246-311`**
166
+ **`grid_reconcile.ts:278`**
167
167
 
168
- `SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER = 5`. An unmatched chain order within 5× price tolerance of an active same-type grid slot is a suspected duplicate → queued for Phase 2 cancellation (not cancelled under lock). The base tolerance comes from `calculatePriceTolerance`, which estimates the maximum acceptable price deviation for the order's size and the grid's price step.
168
+ An unmatched chain order whose price equals an active same-type grid slot's price — exact slot-price equality via `priceSlotEqual` at the asset precision — is a **suspected duplicate** → queued for Phase 2 cancellation (not cancelled under lock). Non-equal neighbours are only logged with nearest-same-side diagnostics, never cancelled here. The earlier fuzzy `SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER` (5× `calculatePriceTolerance`) was removed.
169
169
 
170
170
  ### Batch Update Failure Recovery
171
171
 
172
- **`grid_reconcile.ts:393-465`**
172
+ **`grid_reconcile.ts:436`**
173
173
 
174
174
  Up to 3 batch attempts. Each failure triggers a recovery sync + plan refresh. If all plans are empty → resolved early. After 3× → sequential fallback with per-plan recovery (each individual failure triggers a recovery sync + queue refresh).
175
175
 
176
176
  ### Phantom Orders via Reconcile
177
177
 
178
- **`grid_reconcile.ts:211-244`** — Reconcile's role in the defense-in-depth: during Phase 1, any ACTIVE/PARTIAL order whose `orderId` is not found on-chain is reset to VIRTUAL with `skipAccounting`. The freshly-assigned deferral protects in-flight broadcasts, and the ghost heuristic lets known fills pass. See [`developer_guide.md`](developer_guide.md#phantom-orders-prevention-defense-in-depth) for the full 3-layer defense.
178
+ **`grid_reconcile.ts:227-253`** — Reconcile's role in the defense-in-depth: during Phase 1, any ACTIVE/PARTIAL order whose `orderId` is not found on-chain is reset to VIRTUAL with `skipAccounting`. The freshly-assigned deferral protects in-flight broadcasts, and the ghost heuristic lets known fills pass. See [`developer_guide.md`](developer_guide.md#phantom-orders-prevention-defense-in-depth) for the full 3-layer defense.
179
179
 
180
180
  ### Shelf Orders (Fork-Kept Manual Orders)
181
181
 
@@ -189,13 +189,17 @@ Live on-chain orders with non-slot-N ids below the rail (e.g. `deep-*` manuals k
189
189
 
190
190
  Reconcile Phase 1 runs under `_gridLock` with no side effects on the frozen master Map. The working grid is not involved — reconcile is a startup operation that runs before the COW pipeline is active. See [`COPY_ON_WRITE_MASTER_PLAN.md`](COPY_ON_WRITE_MASTER_PLAN.md#safety-guardrails) and [`COW_INVARIANTS.md`](COW_INVARIANTS.md#reconcile-grid_reconcilemd) for COW rules.
191
191
 
192
+ ### Slot-Price Invariant at the Reconcile Emission Sites
193
+
194
+ Three of the six guarded emission sites are reconcile sites (`RECONCILE-CREATE`, `RECONCILE-UPDATE`, `STARTUP-CREATE` in `grid_reconcile_internal.ts`): every op reconcile emits is checked against the slot's genesis level and an off-grid emission is skipped, not broadcast — see [`GRID_PRICE_INVARIANT.md`](GRID_PRICE_INVARIANT.md). The coupling is bidirectional: the invariant guard's persistent-rejection escalation and the deferred-hold escalation both exit through the same structural resync (debounced reload → full reset) described here, and a full reset's update-first reconcile emits the rail's genesis level, so the guard does not block its own resolution.
195
+
192
196
  ### Truncated-Read Ambiguity (since 1.4.8)
193
197
 
194
- Every chain read feeding an absence/surplus decision goes through `readOpenOrdersGuarded` (`chain_orders.ts:610`) and treats an empty or truncated snapshot as **unreadable** — never as "nothing landed" or "nothing to cancel":
198
+ Every chain read feeding an absence/surplus decision goes through `readOpenOrdersGuarded` (`chain_orders.ts:608`) and treats an empty or truncated snapshot as **unreadable** — never as "nothing landed" or "nothing to cancel":
195
199
 
196
- - `_recoverSyncFromChain` (`grid_reconcile_internal.ts:592`) — plus its three recovery sites in `_createOrderFromGrid` / `_cancelChainOrder` — defers on empty/truncated reads (`deferEmpty: true`). A pass-1 phantom cleanup would otherwise virtualize live slots from a partial window.
197
- - `_adoptPossiblyLandedCreate` (`grid_reconcile_internal.ts:932`) defers to an uncertain outcome on truncated reads, and the startup group batch uncertain verification follows the same rule.
198
- - Phase 3 final refresh (`grid_reconcile.ts:512`) skips adoption/surplus-cancel on a truncated read, keeping the pre-phase-2 counts for the summary log.
200
+ - `_recoverSyncFromChain` (`grid_reconcile_internal.ts:726`) — plus its three recovery sites in `_createOrderFromGrid` / `_cancelChainOrder` — defers on empty/truncated reads (`deferEmpty: true`). A pass-1 phantom cleanup would otherwise virtualize live slots from a partial window.
201
+ - `_adoptPossiblyLandedCreate` (`grid_reconcile_internal.ts:1112`) defers to an uncertain outcome on truncated reads, and the startup group batch uncertain verification follows the same rule.
202
+ - Phase 3 final refresh (`grid_reconcile.ts:533`) skips adoption/surplus-cancel on a truncated read, keeping the pre-phase-2 counts for the summary log.
199
203
  - Adoption paths (`_adoptPossiblyLandedCreate`, grouping path, reconcile adoption loop) apply the create-fee deduction via `_applySync` for accounting parity.
200
204
 
201
205
  The underlying rule is `INV-BROADCAST-004`: a capped `get_full_accounts` window omits the freshest orders (fresh creates sort last), so absence can never be authoritative on a truncated read.
@@ -204,7 +208,7 @@ The underlying rule is `INV-BROADCAST-004`: a capped `get_full_accounts` window
204
208
 
205
209
  ## Lock Hierarchy
206
210
 
207
- **`manager.ts:474-489`** — canonical reference in [`developer_guide.md`](developer_guide.md#lock-ordering-for-deadlock-prevention).
211
+ **`manager.ts:593-597`** — canonical reference in [`developer_guide.md`](developer_guide.md#lock-ordering-for-deadlock-prevention).
208
212
 
209
213
  ```
210
214
  Level 0: _fillProcessingLock Level 1: _divergenceLock
@@ -230,11 +234,10 @@ Commit `e64db685` replaced 6 single-value boolean state fields with refcounts/st
230
234
 
231
235
  | Constant | Value | File | Role |
232
236
  |----------|-------|------|------|
233
- | `SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER` | `5` | `grid_reconcile.ts:20` | Amplifies base tolerance for duplicate detection |
234
- | `maxBatchAttempts` | `3` | `grid_reconcile.ts:415` | Update-batch retry limit |
235
- | `PRICE_TOLERANCE_MAX_PERCENT` | `0.01` (1%) | `constants.ts:452` | Cap on price tolerance |
236
- | `PRICE_TOLERANCE_MIN_ABSOLUTE` | `0.0001` | `constants.ts:456` | Floor for price tolerance |
237
- | `PIPELINE_TIMING.TIMEOUT_MS` | `300000` (5min) | `constants.ts:800` | Base pipeline timing; resync uses 2× (10 min) |
237
+ | `maxBatchAttempts` | `3` | `grid_reconcile.ts:436` | Update-batch retry limit |
238
+ | `PRICE_TOLERANCE_MAX_PERCENT` | `0.01` (1%) | `constants.ts:566` | Cap on price tolerance |
239
+ | `PRICE_TOLERANCE_MIN_ABSOLUTE` | `0.0001` | `constants.ts:570` | Floor for price tolerance |
240
+ | `PIPELINE_TIMING.TIMEOUT_MS` | `300000` (5min) | `constants.ts:973` | Base pipeline timing; resync uses 2× (10 min) |
238
241
 
239
242
  ---
240
243
 
@@ -260,10 +263,10 @@ Commit `e64db685` replaced 6 single-value boolean state fields with refcounts/st
260
263
 
261
264
  | File | Role |
262
265
  |------|------|
263
- | `modules/order/grid_reconcile.ts` | Public API + 3-phase orchestrator (642 lines) |
264
- | `modules/order/grid_reconcile_internal.ts` | Internal helpers — `_reconcileStartupSide`, grid detection, recovery, uncertainty (1675 lines) |
266
+ | `modules/order/grid_reconcile.ts` | Public API + 3-phase orchestrator (670 lines) |
267
+ | `modules/order/grid_reconcile_internal.ts` | Internal helpers — `_reconcileStartupSide`, grid detection, recovery, uncertainty (2132 lines) |
265
268
  | `modules/order/manager.ts` | Lock hierarchy definition, `_applyOrderUpdate`, phantom guard, `reconcileGrid` entry, COW integration |
266
- | `modules/order/async_lock.ts` | AsyncLock engine with ALS re-entrancy (424 lines) |
269
+ | `modules/order/async_lock.ts` | AsyncLock engine with ALS re-entrancy (435 lines) |
267
270
  | `modules/order/sync_engine.ts` | Blockchain sync pipeline |
268
271
  | `modules/order/grid.ts` | Grid creation, `recalculateGrid` (full resync) calls reconcile |
269
272
  | `modules/chain_orders.ts` | `readOpenOrdersGuarded` / guarded read infrastructure |
package/docs/LIFECYCLE.md CHANGED
@@ -90,7 +90,7 @@ sequenceDiagram
90
90
  participant CO as chain_orders (broadcast)
91
91
  participant STORE as Storage
92
92
 
93
- CH->>Q: enqueue fill(s) (dexbot_fill_runtime.ts:325)
93
+ CH->>Q: enqueue fill(s) (dexbot_fill_runtime.ts:414)
94
94
  FR->>FR: drain queue -> _processFillsWithBatching
95
95
  FR->>OM: processFilledOrders() (manager.ts)
96
96
  OM->>AC: processFillAccounting() (single call, batch)
@@ -105,7 +105,7 @@ sequenceDiagram
105
105
  ```
106
106
 
107
107
  Why it matters:
108
- - **Gap-slot batching** (batch size = grid gap-slot count, `DEXBot._getGapSlotBatchSize`) keeps bursts
108
+ - **Gap-slot batching** (batch size = grid gap-slot count + 1, `DEXBot._getGapSlotBatchSize`) keeps bursts
109
109
  deterministic — see `docs/architecture.md` §"Fill Processing Pipeline".
110
110
  - **Single rebalance cycle**: all fills in a batch share one broadcast, so proceeds
111
111
  are immediately available for replacement sizing (no split-across-cycles delay).
@@ -134,10 +134,10 @@ sequenceDiagram
134
134
  participant STORE as Storage
135
135
 
136
136
  MA->>CL: AMA center snapshot updated
137
- CL->>CL: _performPeriodicGridChecks() (dexbot_class.ts:1694)
138
- CL->>RT: performPeriodicGridChecks() (dexbot_maintenance_runtime.ts:1043)
139
- RT->>RT: runGridMaintenance(bot,'periodic') (dexbot_maintenance_runtime.ts:1845)
140
- RT->>RT: executeMaintenanceLogic() (dexbot_maintenance_runtime.ts:1452)
137
+ CL->>CL: _performPeriodicGridChecks() (dexbot_class.ts:1673)
138
+ CL->>RT: performPeriodicGridChecks() (dexbot_maintenance_runtime.ts:1554)
139
+ RT->>RT: runGridMaintenance(bot,'periodic') (dexbot_maintenance_runtime.ts:2770)
140
+ RT->>RT: executeMaintenanceLogic() (dexbot_maintenance_runtime.ts:2270)
141
141
  RT->>AC: recalculate funds from balances
142
142
  RT->>GD: promote AMA center -> grid center
143
143
  RT->>GD: recalculateGrid() (grid.ts)
@@ -157,12 +157,12 @@ The AMA signal stack (AMA/Kalman/Hurst/PE) is *research-tuned* in `analysis/` an
157
157
  the parameters.
158
158
 
159
159
  > Note: `runMaintenance()` is a **different** subsystem — the credit/MPA debt
160
- > runtime (`modules/credit_runtime.ts:3041`, reached via
161
- > `_runCreditRuntimeMaintenance` at `dexbot_class.ts:1794`). The grid maintenance
160
+ > runtime (`modules/credit_runtime.ts:3019`, reached via
161
+ > `_runCreditRuntimeMaintenance` at `dexbot_class.ts:1790`). The grid maintenance
162
162
  > chain above is the one that matters for order/price upkeep.
163
163
 
164
- References: `modules/dexbot_class.ts:1694` (`_performPeriodicGridChecks`) →
165
- `modules/dexbot_maintenance_runtime.ts:1043` (`performPeriodicGridChecks`) →
164
+ References: `modules/dexbot_class.ts:1673` (`_performPeriodicGridChecks`) →
165
+ `modules/dexbot_maintenance_runtime.ts:1554` (`performPeriodicGridChecks`) →
166
166
  `:1845` (`runGridMaintenance`) → `:1452` (`executeMaintenanceLogic`),
167
167
  `docs/GRID_RECALCULATION.md`, `docs/GRID_RECONCILE.md`.
168
168
 
@@ -179,6 +179,7 @@ enforced*, not compiler-enforced — learn them or you will introduce fund bugs.
179
179
  | **Fund SSOT** | `Accounting` owns every fund number. Nothing else computes available funds. | `docs/architecture.md` §"Fund Flow Architecture" |
180
180
  | **Replay-safe fills** | A fill is credited exactly once via processed-fill keys; retries are idempotent. | `modules/dexbot_fill_runtime.ts` |
181
181
  | **Single broadcast per cycle** | One `updateOrdersOnChainBatch` per rebalance — never scatter writes. | `docs/architecture.md` §"Fill Processing Pipeline" |
182
+ | **Slot price = genesis level** | Every emitted order's price must equal `priceForSlot(idx, genesis)` for its slot — range guards cannot substitute for grid membership; off-grid emissions are blocked. | `docs/GRID_PRICE_INVARIANT.md` |
182
183
  | **Browser/Node split** | Heavy runtime is Node-only; never import it from a browser bundle. | `AGENTS.md` "Browser-Safe Surface", `package.json` "browser" field |
183
184
  | **Lock ordering** | Fill drain and maintenance must not run a rebalance concurrently. | `docs/developer_guide.md` §"Startup Sequence & Lock Ordering" |
184
185
 
@@ -191,7 +192,7 @@ enforced*, not compiler-enforced — learn them or you will introduce fund bugs.
191
192
  | How a fill becomes orders | `modules/dexbot_fill_runtime.ts` → `modules/order/manager.ts` |
192
193
  | Grid math / recalculation | `modules/order/grid.ts`, `docs/GRID_RECALCULATION.md` |
193
194
  | Funds & accounting | `modules/order/accounting.ts`, `docs/FUND_MOVEMENT_AND_ACCOUNTING.md` |
194
- | Periodic loop & AMA hook | `modules/dexbot_maintenance_runtime.ts`, `modules/dexbot_class.ts:1694` |
195
+ | Periodic loop & AMA hook | `modules/dexbot_maintenance_runtime.ts`, `modules/dexbot_class.ts:1673` |
195
196
  | Market signal source | `market_adapter/market_adapter.ts`, `analysis/README.md` |
196
197
  | Startup & orchestration | `modules/dexbot_class.ts`, `docs/developer_guide.md` §"Startup Sequence" |
197
198
  | Why COW exists | `docs/architecture.md` §"Copy-on-Write (COW) Grid Pattern", `docs/COW_INVARIANTS.md` |