dexbot 1.5.3 → 1.6.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (231) hide show
  1. package/CHANGELOG.md +62 -0
  2. package/README.md +4 -3
  3. package/analysis/README.md +1 -2
  4. package/analysis/ama_fitting/fetch_lp_candles.ts +1 -9
  5. package/analysis/ama_fitting/package.json +1 -1
  6. package/analysis/analyze_derivatives.ts +1 -1
  7. package/analysis/bot_usage/discover_bot_accounts.ts +12 -6
  8. package/analysis/resolve_source.ts +9 -2
  9. package/analysis/tradingview/README.md +29 -11
  10. package/analysis/tradingview/analyze_tradingview.ts +92 -3
  11. package/analysis/tradingview/tradingview_uplot_chart_generator.ts +1120 -182
  12. package/analysis/trend_detection/package.json +1 -1
  13. package/claw/docs/DEXBOT2_TUNING_CHEAT_SHEET.md +1 -1
  14. package/claw/modules/dexbot_profiles.ts +22 -2
  15. package/claw/package.json +1 -1
  16. package/claw/runtimes/openclaw-plugin/openclaw.plugin.json +1 -1
  17. package/claw/runtimes/openclaw-plugin/package.json +1 -1
  18. package/claw/skills/launcher-ops/references/launcher-workflow.md +1 -1
  19. package/claw/tests/test_claw_mcp_transport.ts +2 -2
  20. package/claw/tests/test_dexbot_profiles.ts +65 -0
  21. package/dist/analysis/ama_fitting/fetch_lp_candles.js +1 -9
  22. package/dist/analysis/ama_fitting/fetch_lp_candles.js.map +1 -1
  23. package/dist/analysis/analyze_derivatives.js +1 -1
  24. package/dist/analysis/bot_usage/discover_bot_accounts.js +11 -6
  25. package/dist/analysis/bot_usage/discover_bot_accounts.js.map +1 -1
  26. package/dist/analysis/bot_usage/kibana_bot_queries.d.ts +2 -0
  27. package/dist/analysis/bot_usage/kibana_bot_queries.d.ts.map +1 -1
  28. package/dist/analysis/resolve_source.d.ts +1 -0
  29. package/dist/analysis/resolve_source.d.ts.map +1 -1
  30. package/dist/analysis/resolve_source.js +6 -2
  31. package/dist/analysis/resolve_source.js.map +1 -1
  32. package/dist/analysis/tradingview/analyze_tradingview.d.ts +5 -0
  33. package/dist/analysis/tradingview/analyze_tradingview.d.ts.map +1 -1
  34. package/dist/analysis/tradingview/analyze_tradingview.js +112 -3
  35. package/dist/analysis/tradingview/analyze_tradingview.js.map +1 -1
  36. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.d.ts +5 -1
  37. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.d.ts.map +1 -1
  38. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.js +1118 -182
  39. package/dist/analysis/tradingview/tradingview_uplot_chart_generator.js.map +1 -1
  40. package/dist/bot.js +1 -1
  41. package/dist/bot.js.map +1 -1
  42. package/dist/dexbot.js +3 -3
  43. package/dist/dexbot.js.map +1 -1
  44. package/dist/market_adapter/ama_signal_runner.js +2 -2
  45. package/dist/market_adapter/candle_utils.d.ts +6 -0
  46. package/dist/market_adapter/candle_utils.d.ts.map +1 -1
  47. package/dist/market_adapter/candle_utils.js +6 -0
  48. package/dist/market_adapter/candle_utils.js.map +1 -1
  49. package/dist/market_adapter/core/kibana_candles.d.ts +17 -1
  50. package/dist/market_adapter/core/kibana_candles.d.ts.map +1 -1
  51. package/dist/market_adapter/core/kibana_candles.js +101 -19
  52. package/dist/market_adapter/core/kibana_candles.js.map +1 -1
  53. package/dist/market_adapter/core/kibana_client.d.ts +2 -0
  54. package/dist/market_adapter/core/kibana_client.d.ts.map +1 -1
  55. package/dist/market_adapter/core/kibana_client.js +31 -4
  56. package/dist/market_adapter/core/kibana_client.js.map +1 -1
  57. package/dist/market_adapter/core/market_adapter_service.d.ts.map +1 -1
  58. package/dist/market_adapter/core/market_adapter_service.js +0 -6
  59. package/dist/market_adapter/core/market_adapter_service.js.map +1 -1
  60. package/dist/market_adapter/inputs/fetch_book_data.d.ts +14 -0
  61. package/dist/market_adapter/inputs/fetch_book_data.d.ts.map +1 -0
  62. package/dist/market_adapter/inputs/fetch_book_data.js +138 -0
  63. package/dist/market_adapter/inputs/fetch_book_data.js.map +1 -0
  64. package/dist/market_adapter/inputs/fetch_cex_synthetic_data.js +1 -1
  65. package/dist/market_adapter/inputs/fetch_lp_data.d.ts +16 -2
  66. package/dist/market_adapter/inputs/fetch_lp_data.d.ts.map +1 -1
  67. package/dist/market_adapter/inputs/fetch_lp_data.js +109 -281
  68. package/dist/market_adapter/inputs/fetch_lp_data.js.map +1 -1
  69. package/dist/market_adapter/inputs/kibana_feed_source.d.ts +87 -0
  70. package/dist/market_adapter/inputs/kibana_feed_source.d.ts.map +1 -0
  71. package/dist/market_adapter/inputs/kibana_feed_source.js +570 -0
  72. package/dist/market_adapter/inputs/kibana_feed_source.js.map +1 -0
  73. package/dist/market_adapter/inputs/kibana_source.d.ts.map +1 -1
  74. package/dist/market_adapter/inputs/kibana_source.js +10 -5
  75. package/dist/market_adapter/inputs/kibana_source.js.map +1 -1
  76. package/dist/market_adapter/inputs/window_cache.d.ts +166 -0
  77. package/dist/market_adapter/inputs/window_cache.d.ts.map +1 -0
  78. package/dist/market_adapter/inputs/window_cache.js +500 -0
  79. package/dist/market_adapter/inputs/window_cache.js.map +1 -0
  80. package/dist/market_adapter/interval_utils.d.ts +8 -1
  81. package/dist/market_adapter/interval_utils.d.ts.map +1 -1
  82. package/dist/market_adapter/interval_utils.js +14 -1
  83. package/dist/market_adapter/interval_utils.js.map +1 -1
  84. package/dist/modules/account_bots.d.ts +1 -0
  85. package/dist/modules/account_bots.d.ts.map +1 -1
  86. package/dist/modules/account_bots.js +21 -5
  87. package/dist/modules/account_bots.js.map +1 -1
  88. package/dist/modules/account_orders.d.ts +25 -4
  89. package/dist/modules/account_orders.d.ts.map +1 -1
  90. package/dist/modules/account_orders.js +67 -4
  91. package/dist/modules/account_orders.js.map +1 -1
  92. package/dist/modules/bitshares-native/chain_client.d.ts +2 -0
  93. package/dist/modules/bitshares-native/chain_client.d.ts.map +1 -1
  94. package/dist/modules/bitshares-native/chain_client.js +3 -1
  95. package/dist/modules/bitshares-native/chain_client.js.map +1 -1
  96. package/dist/modules/bitshares-native/subscriptions.d.ts.map +1 -1
  97. package/dist/modules/bitshares-native/subscriptions.js +13 -3
  98. package/dist/modules/bitshares-native/subscriptions.js.map +1 -1
  99. package/dist/modules/bitshares-native/transport.d.ts +2 -0
  100. package/dist/modules/bitshares-native/transport.d.ts.map +1 -1
  101. package/dist/modules/bitshares-native/transport.js +122 -28
  102. package/dist/modules/bitshares-native/transport.js.map +1 -1
  103. package/dist/modules/bitshares_client.d.ts.map +1 -1
  104. package/dist/modules/bitshares_client.js +59 -8
  105. package/dist/modules/bitshares_client.js.map +1 -1
  106. package/dist/modules/bot_settings.d.ts.map +1 -1
  107. package/dist/modules/bot_settings.js +11 -0
  108. package/dist/modules/bot_settings.js.map +1 -1
  109. package/dist/modules/constants.d.ts +13 -0
  110. package/dist/modules/constants.d.ts.map +1 -1
  111. package/dist/modules/constants.js +68 -4
  112. package/dist/modules/constants.js.map +1 -1
  113. package/dist/modules/dexbot_class.d.ts +8 -0
  114. package/dist/modules/dexbot_class.d.ts.map +1 -1
  115. package/dist/modules/dexbot_class.js +57 -2
  116. package/dist/modules/dexbot_class.js.map +1 -1
  117. package/dist/modules/dexbot_cow_runtime.d.ts +26 -1
  118. package/dist/modules/dexbot_cow_runtime.d.ts.map +1 -1
  119. package/dist/modules/dexbot_cow_runtime.js +157 -28
  120. package/dist/modules/dexbot_cow_runtime.js.map +1 -1
  121. package/dist/modules/dexbot_fill_runtime.d.ts +41 -1
  122. package/dist/modules/dexbot_fill_runtime.d.ts.map +1 -1
  123. package/dist/modules/dexbot_fill_runtime.js +135 -5
  124. package/dist/modules/dexbot_fill_runtime.js.map +1 -1
  125. package/dist/modules/dexbot_maintenance_runtime.d.ts +79 -12
  126. package/dist/modules/dexbot_maintenance_runtime.d.ts.map +1 -1
  127. package/dist/modules/dexbot_maintenance_runtime.js +554 -46
  128. package/dist/modules/dexbot_maintenance_runtime.js.map +1 -1
  129. package/dist/modules/dexbot_startup_runtime.d.ts.map +1 -1
  130. package/dist/modules/dexbot_startup_runtime.js +13 -0
  131. package/dist/modules/dexbot_startup_runtime.js.map +1 -1
  132. package/dist/modules/dexbot_state_recovery.d.ts +15 -1
  133. package/dist/modules/dexbot_state_recovery.d.ts.map +1 -1
  134. package/dist/modules/dexbot_state_recovery.js +36 -7
  135. package/dist/modules/dexbot_state_recovery.js.map +1 -1
  136. package/dist/modules/fund_registry.d.ts +1 -1
  137. package/dist/modules/fund_registry.js +1 -1
  138. package/dist/modules/node_connect_policy.d.ts +52 -0
  139. package/dist/modules/node_connect_policy.d.ts.map +1 -0
  140. package/dist/modules/node_connect_policy.js +35 -0
  141. package/dist/modules/node_connect_policy.js.map +1 -0
  142. package/dist/modules/node_manager.d.ts +32 -0
  143. package/dist/modules/node_manager.d.ts.map +1 -1
  144. package/dist/modules/node_manager.js +76 -9
  145. package/dist/modules/node_manager.js.map +1 -1
  146. package/dist/modules/order/accounting.d.ts +30 -0
  147. package/dist/modules/order/accounting.d.ts.map +1 -1
  148. package/dist/modules/order/accounting.js +176 -3
  149. package/dist/modules/order/accounting.js.map +1 -1
  150. package/dist/modules/order/export.d.ts.map +1 -1
  151. package/dist/modules/order/export.js +1 -0
  152. package/dist/modules/order/export.js.map +1 -1
  153. package/dist/modules/order/grid.d.ts.map +1 -1
  154. package/dist/modules/order/grid.js +143 -15
  155. package/dist/modules/order/grid.js.map +1 -1
  156. package/dist/modules/order/grid_reconcile.d.ts.map +1 -1
  157. package/dist/modules/order/grid_reconcile.js +5 -3
  158. package/dist/modules/order/grid_reconcile.js.map +1 -1
  159. package/dist/modules/order/grid_reconcile_internal.d.ts.map +1 -1
  160. package/dist/modules/order/grid_reconcile_internal.js +182 -6
  161. package/dist/modules/order/grid_reconcile_internal.js.map +1 -1
  162. package/dist/modules/order/manager.d.ts +64 -2
  163. package/dist/modules/order/manager.d.ts.map +1 -1
  164. package/dist/modules/order/manager.js +231 -14
  165. package/dist/modules/order/manager.js.map +1 -1
  166. package/dist/modules/order/strategy.d.ts +3 -0
  167. package/dist/modules/order/strategy.d.ts.map +1 -1
  168. package/dist/modules/order/strategy.js +79 -7
  169. package/dist/modules/order/strategy.js.map +1 -1
  170. package/dist/modules/order/sync_engine.d.ts.map +1 -1
  171. package/dist/modules/order/sync_engine.js +16 -1
  172. package/dist/modules/order/sync_engine.js.map +1 -1
  173. package/dist/modules/order/utils/math.d.ts +11 -1
  174. package/dist/modules/order/utils/math.d.ts.map +1 -1
  175. package/dist/modules/order/utils/math.js +24 -1
  176. package/dist/modules/order/utils/math.js.map +1 -1
  177. package/dist/modules/order/utils/order.d.ts +258 -13
  178. package/dist/modules/order/utils/order.d.ts.map +1 -1
  179. package/dist/modules/order/utils/order.js +662 -12
  180. package/dist/modules/order/utils/order.js.map +1 -1
  181. package/dist/modules/order/utils/system.d.ts +36 -0
  182. package/dist/modules/order/utils/system.d.ts.map +1 -1
  183. package/dist/modules/order/utils/system.js +102 -6
  184. package/dist/modules/order/utils/system.js.map +1 -1
  185. package/dist/modules/order/utils/validate.d.ts.map +1 -1
  186. package/dist/modules/order/utils/validate.js +6 -2
  187. package/dist/modules/order/utils/validate.js.map +1 -1
  188. package/dist/modules/paths.d.ts +2 -0
  189. package/dist/modules/paths.d.ts.map +1 -1
  190. package/dist/modules/paths.js +1 -0
  191. package/dist/modules/paths.js.map +1 -1
  192. package/dist/modules/runtime_settings.d.ts +13 -0
  193. package/dist/modules/runtime_settings.d.ts.map +1 -1
  194. package/dist/modules/runtime_settings.js +20 -0
  195. package/dist/modules/runtime_settings.js.map +1 -1
  196. package/dist/modules/utils/errors.d.ts +16 -0
  197. package/dist/modules/utils/errors.d.ts.map +1 -1
  198. package/dist/modules/utils/errors.js +31 -0
  199. package/dist/modules/utils/errors.js.map +1 -1
  200. package/dist/pm2.js +7 -7
  201. package/dist/scripts/analyze-credit.d.ts.map +1 -1
  202. package/dist/scripts/analyze-credit.js +13 -3
  203. package/dist/scripts/analyze-credit.js.map +1 -1
  204. package/dist/scripts/analyze-orders.js +2 -2
  205. package/dist/scripts/analyze-orders.js.map +1 -1
  206. package/dist/scripts/diagnose-kibana-candles.d.ts.map +1 -1
  207. package/dist/scripts/diagnose-kibana-candles.js +0 -1
  208. package/dist/scripts/diagnose-kibana-candles.js.map +1 -1
  209. package/dist/scripts/test-credit-renewal.js +1 -1
  210. package/dist/scripts/test-credit-renewal.js.map +1 -1
  211. package/dist/scripts/tv.d.ts +24 -2
  212. package/dist/scripts/tv.d.ts.map +1 -1
  213. package/dist/scripts/tv.js +190 -65
  214. package/dist/scripts/tv.js.map +1 -1
  215. package/docs/BITSHARES_ONBOARDING.md +9 -2
  216. package/docs/COW_INVARIANTS.md +18 -1
  217. package/docs/DEXBOT_COMPARISON.md +3 -3
  218. package/docs/EVOLUTION.md +8 -7
  219. package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md +21 -9
  220. package/docs/GRID_RECALCULATION.md +5 -6
  221. package/docs/GRID_RECONCILE.md +12 -2
  222. package/docs/MPA_CREDIT_USAGE.md +2 -2
  223. package/docs/README.md +5 -6
  224. package/docs/architecture.md +0 -58
  225. package/docs/developer_guide.md +2 -1
  226. package/docs/docker.md +1 -1
  227. package/market_adapter/README.md +4 -0
  228. package/modules/README.md +1 -1
  229. package/package.json +1 -1
  230. package/scripts/README.md +5 -3
  231. package/tests/README.md +2 -2
@@ -144,24 +144,36 @@ FILL_PROCESSING: {
144
144
 
145
145
  **Residual Dust Cancellation** (post-1.4.12): After a sub-dust fill leaves a residual order on chain (e.g. the quote-side value truncates to 0 on `bitshares-core` `maybe_cull_small_order`), the fill runtime explicitly cancels those residuals via `cancelResidualOrders()` (`dexbot_fill_runtime.ts:74`, `[RESIDUAL]` tag) so a leftover of ≥1 base unit cannot be re-adopted into a grid slot by maintenance.
146
146
 
147
- #### Grid Regeneration Trigger (Available Funds Ratio)
147
+ #### Grid Regeneration Trigger (Available Funds Ratio — bidirectional)
148
148
 
149
- The grid regenerates when accumulated proceeds create a significant funding imbalance. This is detected using the **Available Funds Ratio**:
149
+ The grid regenerates when accumulated proceeds create a significant funding imbalance — in either direction. This is detected using the **Available Funds Ratio** (grow leg) plus an over-allocation leg (shrink leg), all sharing `GRID_REGENERATION_PERCENTAGE` (default: 3%):
150
150
 
151
151
  ```
152
- ratio = (availableFunds / allocatedCapital) * 100
153
-
154
- IF ratio >= GRID_REGENERATION_PERCENTAGE (default: 3%):
155
- → Trigger grid regeneration
152
+ GROW: ratio = (availableFunds / allocatedCapital) * 100
153
+ IF ratio >= GRID_REGENERATION_PERCENTAGE (default: 3%):
154
+ → Trigger grid regeneration (deploy proceeds)
155
+
156
+ SHRINK: overAlloc = (gridTracked - allocatedCapital) / allocatedCapital * 100
157
+ (`gridTracked` = funds.total.grid: ACTIVE + PARTIAL + VIRTUAL planned size)
158
+ IF overAlloc >= threshold:
159
+ → Trigger grid regeneration (resize affected orders down)
156
160
  ```
157
161
 
158
- **How It Works**:
162
+ Deliberately no per-side chain-total-drop leg: a normal fill moves value across sides (pays one asset, receives the other — see `recordFillBalances`), so one side's total routinely drops ≥3% on ordinary fills, and the fill pipeline already re-sizes from the post-fill budget.
163
+
164
+ **How It Works (grow)**:
159
165
  1. Fill occurs → proceeds added to `chainFree`
160
166
  2. `calculateAvailableFundsValue()` computes true spending power (chainFree minus reservations)
161
167
  3. Grid divergence check compares this ratio against allocated capital in active orders
162
168
  4. If ratio exceeds 3%, the grid has accumulated enough proceeds to warrant redeployment
163
169
  5. Grid regeneration recalculates all order sizes and applies new placements
164
170
 
171
+ **How It Works (shrink)**:
172
+ 1. External removal (manual transfer/withdrawal) → `chainTotal` drops → `funds.allocated` (chainTotal × `botFunds`%) drops while the grid-tracked size stays put
173
+ 2. Divergence check flags the side when grid-tracked size exceeds the allocation by ≥ 3% (slow bleeds accumulate across ticks: the grid stays fixed while the allocation sinks)
174
+ 3. The same regeneration path runs: `_recalculateGridOrderSizesFromBlockchain` recomputes geometric ideals on the smaller budget and queues `UPDATE` actions shrinking affected on-chain orders (`delta < 0` releases the difference back to free balance on chain). Fork-kept shelf orders (live non-slot-N ids below the rail) are skipped by the per-slot loop — their manual sizes survive the resize while still counting in the budget denominator.
175
+ 4. Under-deployed grids (grid-tracked size still within the shrunken allocation) correctly do NOT trigger — their orders remain fully funded
176
+
165
177
  #### Recovery Retry System
166
178
 
167
179
  Recovery uses a count+time-based retry system with periodic reset, so a single failed recovery attempt never locks out future retries.
@@ -518,7 +530,7 @@ BitShares charges fees for `limit_order_create` and `limit_order_cancel`.
518
530
 
519
531
  - **Reservation** (`BTS_RESERVATION_MULTIPLIER` in `constants.ts::FEE_PARAMETERS`):
520
532
  $$Reserve = N_{active} \times BTS\_RESERVATION\_MULTIPLIER$$
521
- *(Default: 5× per order — covers create, rotate (cancel+place), update, and cancel over the order's lifetime)*
533
+ *(Default: 5× per order — covers create, rotate (cancel+place), update, and cancel over the order's lifetime. $N_{active}$ counts window orders plus the `reserveOrders` edge ladder — reserves rest live on-chain and pay fees like window orders)*
522
534
 
523
535
  - **Settlement (`deductBtsFees`):**
524
536
  1. Check `Funds.btsFeesOwed`.
@@ -794,4 +806,4 @@ To prevent "Time-of-Check to Time-of-Use" errors:
794
806
  **TOCTOU protection in `processFillAccounting`.** `_buildBtsDeferredRefundAdjustment` reads `btsFeeState` from `mgr.orders` while the order lock is held — the lock is acquired before accounting runs, and the POST-RESET and BOOTSTRAP tracked-fill accounting paths follow the same locking pattern.
795
807
 
796
808
  ---
797
- *Technical Reference for DEXBot2 v1.5.3 release*
809
+ *Technical Reference for DEXBot2 v1.6.0 release*
@@ -26,7 +26,7 @@ rebuild around that snapshot.
26
26
  | **AMA Center Move** | Market adapter | Current AMA center moves past the configured delta threshold | Write `dynamicgrid.json`, then write a trigger file | Full grid resync around the new accepted AMA center |
27
27
  | **AMA Slope Range Move** | Market adapter | Range-scaling bot's accepted AMA-slope baseline moves past threshold | Write range-scaling fields to `dynamicgrid.json`, then write a trigger file | Full grid resync with updated asymmetric range/offset data |
28
28
  | **RMS Structural Divergence** | Bot runtime maintenance | Current grid shape diverges from persisted/on-chain grid by RMS threshold | Refresh `gridCenterPrice` from latest `amaCenterPrice`, then run full grid resync | Full grid resync from latest market-adapter snapshot |
29
- | **Available-Funds Resize** | Bot runtime maintenance | Filled-order proceeds exceed `GRID_REGENERATION_PERCENTAGE` | Recalculate affected side/order sizes through maintenance logic | Order-size/grid maintenance update, not an AMA recenter trigger |
29
+ | **Available-Funds Resize** | Bot runtime maintenance | Filled-order proceeds exceed `GRID_REGENERATION_PERCENTAGE` (grow), or grid-tracked size exceeds allocation by that threshold after fund removal (shrink) | Recalculate affected side/order sizes through maintenance logic | Order-size/grid maintenance update, not an AMA recenter trigger |
30
30
 
31
31
  Each source is evaluated independently. Market-adapter full-resync requests are
32
32
  serialized through `profiles/recalculate.<botKey>.trigger`. Runtime maintenance
@@ -433,11 +433,9 @@ GRID_LIMITS: {
433
433
  ```
434
434
 
435
435
  **Parameters:**
436
- - `GRID_REGENERATION_PERCENTAGE`: Percentage of allocated capital that can accumulate as free funds before triggering a size recalculation
437
- - Default: `3%`
438
- - Example: 20 orders × 100 BTS = 2000 BTS grid
439
- - Triggers when availableFunds ≥ 60 BTS (3% of 2000)
440
- - Allows ~3 fill-proceeds to accumulate before resize
436
+ - `GRID_REGENERATION_PERCENTAGE`: Bidirectional threshold for size recalculation (default: `3%`)
437
+ - Grow: free funds accumulate to ≥3% of allocated capital (e.g. availableFunds ≥ 60 BTS on a 2000 BTS grid)
438
+ - Shrink: grid-tracked size (ACTIVE + PARTIAL + VIRTUAL) exceeds allocation by ≥3% (external fund removal resizes affected orders down; per-side chain totals are not used — fills move value across sides)
441
439
 
442
440
  ### How It Works
443
441
 
@@ -448,6 +446,7 @@ GRID_LIMITS: {
448
446
  - Recalculates order sizes to incorporate freed capital
449
447
  - Maintains asymmetric fills (BUY fills don't trigger SELL resize)
450
448
  4. After the resize, available funds are re-allocated into active orders
449
+ 5. Mirror (shrink): when `(gridTracked - allocated) / allocated × 100 >= threshold` (external fund removal), the same path resizes affected orders down
451
450
 
452
451
  ### When to Adjust
453
452
 
@@ -26,7 +26,7 @@ Phase 1 does all reasoning in memory under `_gridLock` (fast); Phases 2 and 3 ex
26
26
 
27
27
  Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations are attempted — plans are logged but not executed.
28
28
 
29
- `targetCount` (per side, `targetSell`/`targetBuy`) is sourced from bot config and determines how many active orders each side should maintain. The internal `planOnly` flag controls whether `_reconcileStartupSide` records plans for Phase 2 or executes inline — Phase 1 always calls with `planOnly=true`.
29
+ `targetCount` (per side, `targetSell`/`targetBuy`) is sourced from bot config (`activeOrders` window plus `reserveOrders` edge ladder) and determines how many live orders each side should maintain. The internal `planOnly` flag controls whether `_reconcileStartupSide` records plans for Phase 2 or executes inline — Phase 1 always calls with `planOnly=true`.
30
30
 
31
31
  ```
32
32
  Grid generated
@@ -93,7 +93,8 @@ Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations a
93
93
  - Match sorted unmatched chain orders to virtual slots → `plannedUpdates`
94
94
  - Detect grid-edge lock and plan a largest-order cancel
95
95
  - Plan creates for remaining slots
96
- - Plan excess cancellations (guarded by `matchedOnGrid > 0`)
96
+ - Plan excess cancellations (guarded by `matchedOnGrid > 0`): orphans first, then **matched surplus** (`chainCount - targetCount`, reserve edge slots last) — the matched-excess selection is shared by the planOnly and execute branches so planning can never drift from execution; planOnly omits `releaseUntrackedFunds` for matched slots (their funds are tracked on the grid slot)
97
+ - Fork-kept **shelf orders** (live non-slot-N ids below the rail, e.g. `deep-*`) are never cancel candidates or reserve members — see [Shelf Orders](#shelf-orders-fork-kept-manual-orders)
97
98
  - **Vacated-rail refill**: each PROCEEDING update whose vacated price exactly matches (`priceSlotEqual`) an empty, sized, in-rail slot of the same side queues a refill CREATE in the same plan (`source startupVacatedRailRefill`) — skipped updates, ghost prices (lattice moved), in-band slots, and already-desired slots never refill; refill targets require VIRTUAL state with no `orderId`
98
99
 
99
100
  Returns `{ plannedCreates, plannedUpdates, plannedCancels, chainSellCount, chainBuyCount }`.
@@ -176,6 +177,14 @@ Up to 3 batch attempts. Each failure triggers a recovery sync + plan refresh. If
176
177
 
177
178
  **`grid_reconcile.ts:211-244`** — Reconcile's role in the defense-in-depth: during Phase 1, any ACTIVE/PARTIAL order whose `orderId` is not found on-chain is reset to VIRTUAL with `skipAccounting`. The freshly-assigned deferral protects in-flight broadcasts, and the ghost heuristic lets known fills pass. See [`developer_guide.md`](developer_guide.md#phantom-orders-prevention-defense-in-depth) for the full 3-layer defense.
178
179
 
180
+ ### Shelf Orders (Fork-Kept Manual Orders)
181
+
182
+ Live on-chain orders with non-slot-N ids below the rail (e.g. `deep-*` manuals kept across a fork) are **shelf orders**: they sit outside the grid contract and must survive every startup path untouched. Three gates enforce this (all no-ops on grids that only mint slot-N ids):
183
+
184
+ - **Reserve classification/placement** (`order.ts`, `grid_reconcile_internal.ts`, `manager.ts`): `reserveEdgeIdSet`, the Tier-2 live-anchor scan, `_pickEdgeReserveSlots`, and `pickEdgeReserves` all gate to `parseSlotIndex(id) !== null`, so a shelf can never count as the reserve edge (which would mask a real reserve deficit) nor be activated as a reserve it would never be counted as. The live-reserve count additionally excludes window members via `liveWindowIdSet`, so an edge-reaching window cannot masquerade as dedicated reserves.
185
+ - **Startup excess cancels** (`grid_reconcile_internal.ts`): the matched-excess selection filters to slot-N ids in both planOnly and execute branches — the cheapest-first sort would otherwise wipe the shelf on the next boot.
186
+ - **Geometric size recalc** (`grid.ts`): `_recalculateGridOrderSizesFromBlockchain` skips non-slot-N slots in the per-slot loop, so divergence-triggered resizing never overwrites manual shelf sizes on-chain (the shelf stays in the budget denominator, so allocation math is unchanged).
187
+
179
188
  ### COW Interaction
180
189
 
181
190
  Reconcile Phase 1 runs under `_gridLock` with no side effects on the frozen master Map. The working grid is not involved — reconcile is a startup operation that runs before the COW pipeline is active. See [`COPY_ON_WRITE_MASTER_PLAN.md`](COPY_ON_WRITE_MASTER_PLAN.md#safety-guardrails) and [`COW_INVARIANTS.md`](COW_INVARIANTS.md#reconcile-grid_reconcilemd) for COW rules.
@@ -242,6 +251,7 @@ Commit `e64db685` replaced 6 single-value boolean state fields with refcounts/st
242
251
  | `tests/test_race_condition_fixes_batch1.ts` | ABBA deadlock (RC-1B) |
243
252
  | `tests/test_async_lock_force_release.ts` | Nested multi-lock re-entrancy |
244
253
  | `tests/test_targeted_drift_reconcile.ts` | Active-order shortfall triggers sync |
254
+ | `tests/test_reserve_orders.ts` | Reserve startup coverage: fully-placed matched-surplus cancels, orphan+matched ordering, at-target silence, plan/execute parity, shelf-order survival, reserve-deficit trigger (`buy reserves 0/2`) with filled/disabled/empty-budget silence, window-exclusion counting |
245
255
  | `tests/repro_phantom_orders.ts` | Phantom order prevention |
246
256
 
247
257
  ---
@@ -139,11 +139,11 @@ Set `creditOnly: true` on a bot entry to run only the credit runtime — no grid
139
139
  "preferredAccount": "my-account", "debtPolicy": { "lending": [...] } }
140
140
  ```
141
141
 
142
- No trading fields (`assetA`, `assetB`, `startPrice`, `incrementPercent`, `activeOrders`, `botFunds`) are needed.
142
+ No trading fields (`assetA`, `assetB`, `startPrice`, `incrementPercent`, `activeOrders`, `reserveOrders`, `botFunds`) are needed.
143
143
 
144
144
  ```bash
145
145
  dexbot start credit # Background daemon — runs only the credit worker
146
- dexbot start <bot-name> # Start a named bot directly
146
+ dexbot start <bot> # Start a named bot directly
147
147
  ```
148
148
 
149
149
  ### Collateral Increase Thresholds
package/docs/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  This directory contains the comprehensive technical documentation for the DEXBot2 trading bot. It is designed to guide developers from high-level architecture down to the nuances of fund accounting and state management.
4
4
 
5
- **Version context:** v1.5.3 (released).
5
+ **Version context:** v1.6.0 (released).
6
6
 
7
7
  ---
8
8
 
@@ -100,7 +100,6 @@ This directory contains the comprehensive technical documentation for the DEXBot
100
100
  - **Module Responsibilities**: Detailed breakdown of the **Manager**, **Accountant**, **Strategy**, **Grid**, **FillRuntime**, and **MaintenanceRuntime** modules.
101
101
  - **Copy-on-Write Pattern**: Safe concurrent rebalancing with isolated working grids (see [COPY_ON_WRITE_MASTER_PLAN.md](COPY_ON_WRITE_MASTER_PLAN.md))
102
102
  - **Fill Processing Pipeline**: Fixed-cap batch fill processing (1-4 fills per broadcast; documented Feb 7 29-fill scenario: ~24s)
103
- - **Fund-Driven Boundary Sync**: Automatic grid alignment with inventory distribution
104
103
  - **Spread Correction**: Conservative, fund-aware maintenance of constant spread width
105
104
  - **Periodic Market Price Refresh**: Background 4-hour price updates
106
105
  - **Pipeline Safety & Diagnostics**: 5-minute timeout safeguard and health monitoring
@@ -130,7 +129,7 @@ This directory contains the comprehensive technical documentation for the DEXBot
130
129
  - **Signal Concepts**: Dynamic weights, regime detection, derivative signals, and market adapter integration
131
130
  - **Debt Policy**: Native MPA and credit offer configuration and runtime rules
132
131
  - **Common Tasks**: Practical "how-to" guides for adding features or fixing bugs.
133
- - **Glossary**: Definitions of project-specific terminology (e.g., "Virtual Orders", "Rotation", "Pipeline Safety", "Fund-Driven Boundary", "WorkingGrid", "COW Commit", "Dynamic Weight", "Regime Detection").
132
+ - **Glossary**: Definitions of project-specific terminology (e.g., "Virtual Orders", "Rotation", "Pipeline Safety", "WorkingGrid", "COW Commit", "Dynamic Weight", "Regime Detection").
134
133
 
135
134
  ### 🔄 [Workflow](WORKFLOW.md)
136
135
  *How we build and release.*
@@ -144,12 +143,12 @@ This directory contains the comprehensive technical documentation for the DEXBot
144
143
 
145
144
  ### 🧭 [Evolution Report](EVOLUTION.md)
146
145
  *Project timeline and major architecture phases.*
147
- - **Coverage**: Historical milestones from the initial December 2025 bootstrap through the v1.5.3 stable release (grid persistence safety, dust pipeline fix, net inventory lots); credit/debt runtime and maintenance hardening are covered under Phase 5 (Mar–Jun 2026)
148
- - **Focus**: Architecture evolution, release history, test growth, and documentation changes
146
+ - **Coverage**: Historical milestones from bootstrap through the current stable release; per-release detail lives in [CHANGELOG.md](../CHANGELOG.md)
147
+ - **Focus**: Architecture evolution, release history, and test growth
149
148
 
150
149
  ### 🗒️ [Changelog](../CHANGELOG.md)
151
150
  *Release notes and documentation history.*
152
- - **Scope**: Versioned notes for patch releases and the current unreleased documentation refresh
151
+ - **Scope**: Versioned notes per release
153
152
 
154
153
  ### 🧩 [Copy-on-Write Master Plan](COPY_ON_WRITE_MASTER_PLAN.md)
155
154
  *COW design, phases, and state machine details.*
@@ -454,64 +454,6 @@ roughly 3 seconds per broadcast; see
454
454
 
455
455
  ---
456
456
 
457
- ## Fund-Driven Boundary Sync
458
-
459
- The grid boundary (which separates BUY, SPREAD, and SELL zones) automatically aligns with the bot's actual inventory distribution.
460
-
461
- ### Why This Matters
462
-
463
- By default, the grid is centered around `startPrice`. However, if the bot has asymmetric capital (e.g., more assetB than assetA), the boundary should shift to favor the "heavier" side.
464
-
465
- **Example**: If 70% of capital is in assetB (buying power), the BUY zone should be expanded.
466
-
467
- ### Boundary Calculation
468
-
469
- **Location**: `modules/dexbot_class.ts::_performPeriodicGridChecks()` → Boundary Sync step
470
-
471
- **Algorithm**:
472
- ```javascript
473
- // 1. Scan all grid slots and their current assignments
474
- const buyTotal = sum(orders with type === BUY);
475
- const sellTotal = sum(orders with type === SELL);
476
- const totalAllocated = buyTotal + sellTotal;
477
-
478
- // 2. Calculate target allocation based on available funds
479
- const buyAvailable = manager.funds.available.buy;
480
- const sellAvailable = manager.funds.available.sell;
481
- const totalAvailable = buyAvailable + sellAvailable;
482
-
483
- // 3. Determine ideal boundary position
484
- const buyTargetRatio = buyAvailable / totalAvailable; // e.g., 0.7
485
- const slots = grid.length;
486
- const targetBuySlots = Math.round(slots * buyTargetRatio * 0.5); // Apply centering factor
487
-
488
- // 4. Adjust boundary to new position
489
- newBoundaryIdx = calculateNewBoundary(targetBuySlots);
490
-
491
- // 5. Re-assign slot roles (BUY/SPREAD/SELL) based on new boundary
492
- reassignSlotRoles(newBoundaryIdx);
493
- ```
494
-
495
- ### Three Rotation Cases
496
-
497
- Once the new boundary is determined, existing on-chain orders are matched to desired slots:
498
-
499
- | Case | Condition | Action |
500
- |------|-----------|--------|
501
- | **MATCH** | Existing order price matches desired slot | Update size if needed |
502
- | **ACTIVATE** | Desired slot is empty | Place new order at this price |
503
- | **DEACTIVATE** | Existing order exceeds target count | Cancel excess orders |
504
-
505
- **Target Count**: `activeOrders` from config, applied uniformly to both sides.
506
-
507
- ### Impact
508
-
509
- - **Automatic Capital Repositioning**: Grid follows capital distribution without manual intervention
510
- - **Fund Respect**: Never exceeds available funds when activating slots
511
- - **Smooth Transitions**: Rotations happen gradually, not all at once
512
-
513
- ---
514
-
515
457
  ## Spread Correction (Fund-Aware Approach)
516
458
 
517
459
  Simplified spread maintenance that keeps the gap consistent and fund-driven, avoiding complex split/merge mechanics.
@@ -143,7 +143,8 @@ A **phantom order** is an order in ACTIVE/PARTIAL state WITHOUT a valid `orderId
143
143
  | **Crawl Candidate** | Furthest active order eligible for rotation |
144
144
  | **Shortage** | Empty slot in the active window that needs an order |
145
145
  | **Surplus** | Order outside the active window that can be rotated |
146
- | **Hard Surplus** | Order beyond the configured `activeOrders` count |
146
+ | **Hard Surplus** | Order beyond the configured window + reserve (`activeOrders` + `reserveOrders`) count |
147
+ | **Reserve** | Edge-pinned live order outside the active window (buys at grid floor, sells at grid ceiling); never moves the boundary |
147
148
  | **Dust** | Partial order < 5% of ideal size |
148
149
  | **Dust Cancel** | Immediate on-chain cancel of dust partials on detection (no delay, no timer). Slot freed for a fresh counter-order. |
149
150
 
package/docs/docker.md CHANGED
@@ -35,7 +35,7 @@ dexbot start
35
35
  To start only one bot:
36
36
 
37
37
  ```bash
38
- dexbot start <bot-name>
38
+ dexbot start <bot>
39
39
  ```
40
40
 
41
41
  ### Headless (non-interactive) startup
@@ -849,6 +849,10 @@ suppress writes via `unresolved_candle_gaps` until repaired on a future cycle.
849
849
  The adapter prunes old candles to the required AMA window and acts only on
850
850
  closed 1h candles.
851
851
 
852
+ #### Shared Chunk Cache and Fetch Robustness
853
+
854
+ Pool, book, and feed candle fetches share one cache entry point (`runCachedWindows` in `market_adapter/inputs/window_cache.ts`): sibling chunk files load once, only missing buckets plus a bounded 48h tail refresh are queried, and chunk metas record the ranges actually queried (`meta.queriedRanges`). A missing range is pruned only when recorded query coverage genuinely covers it — the absence of local buckets alone never certifies history as empty. Partial windows merge into the run output but are never persisted, and orphan chunks are deleted after complete runs only. Every range fetch runs through `fetchRangeWithRetry` (per-range attempts + linear backoff + abort-signal timeout; the LP path keeps a 4-attempt budget), one-shot Kibana queries retry transient errors (3 attempts), paged fetchers cap at `kibanaMaxPages` (500), and bidirectional fetches tolerate a one-direction failure.
855
+
852
856
  #### AMA Warmup Window — Why Candle Length Matters
853
857
 
854
858
  The AMA is a recursive (infinite impulse response) filter. On cold start, the adapter uses an initial warmup phase: it calculates an **SMA (Simple Moving Average)** over the first `erPeriod` candles to establish a stable seed price, while simultaneously building the price history needed to calculate the first valid Efficiency Ratio (ER).
package/modules/README.md CHANGED
@@ -159,7 +159,7 @@ External consumers — other parts of DEXBot2 interact with modules through thes
159
159
 
160
160
  - **Zero runtime dependencies** — no npm trading/blockchain libraries; the native client, crypto, and serialization are all in `bitshares-native/`. This eliminates supply-chain risk and keeps the bot fully self-contained.
161
161
  - **Copy-on-write** — `order/working_grid.ts` provides isolated COW mutations; master grid is immutable during rebalance. The grid is never modified in-place; a working copy is built, mutated, and committed atomically only after blockchain operations succeed.
162
- - **Fund-driven boundary sync** — the grid rebalances based on available funds, not arbitrary triggers; no forced allocations. When a fill arrives, the system calculates what it can actually afford and adjusts the grid around that.
162
+ - **Fund-driven sizing, fill-driven boundary** — available funds set order sizing and budget allocation, not the boundary position; no forced allocations. The boundary moves only through boundary crawl on fills or spread promotion onto orders placed in the same atomic batch.
163
163
  - **Replay-safe accounting** — fill processing in `dexbot_fill_runtime.ts` uses `processed_fill_store.ts` to prevent double-counting. If the bot restarts mid-fill, it can safely replay without creating duplicate orders.
164
164
  - **Daemon-backed signing** — the credential daemon holds decrypted keys; modules never handle raw private keys. If the main bot crashes, keys stay encrypted on disk — only the small daemon process sees them.
165
165
  - **Fixed-cap batch processing** — fill batches are capped (default 4) to keep blockchain broadcasts predictable. Even if 20 fills arrive at once, they're processed in small chunks to avoid overwhelming the chain.
package/package.json CHANGED
@@ -1,6 +1,6 @@
1
1
  {
2
2
  "name": "dexbot",
3
- "version": "1.5.3",
3
+ "version": "1.6.0",
4
4
  "description": "The first open source trading bot with zero runtime dependencies and a fully adaptive market making strategy.",
5
5
  "main": "dist/modules/dexbot_class.js",
6
6
  "exports": {
package/scripts/README.md CHANGED
@@ -253,7 +253,7 @@ npm run verify:browser-bundle
253
253
 
254
254
  ### Create PM2 Bot Symlinks
255
255
  **File:** `create-bot-symlinks.sh`
256
- **Purpose:** Create `profiles/<bot-name>.config.cjs` symlinks pointing to `profiles/ecosystem.config.cjs` so you can run `pm2 start <bot-name>` directly.
256
+ **Purpose:** Create `profiles/<bot>.config.cjs` symlinks pointing to `profiles/ecosystem.config.cjs` so you can run `pm2 start <bot>` directly.
257
257
  ```bash
258
258
  bash scripts/create-bot-symlinks.sh
259
259
  ```
@@ -387,14 +387,16 @@ The following scripts allow you to call `dexbot` commands directly from the `scr
387
387
 
388
388
  ### TradingView (`dexbot tv`)
389
389
  **File:** `tv.ts`
390
- **Purpose:** One-step TradingView-style 1h chart for a bot (with AMA overlay), pool, or pair. Fetches candles in monthly Kibana chunks, then renders via `analysis/tradingview/`.
391
- **Output:** `analysis/charts/tv_<bot|pool_<id>|<a>_<b>>_1h_<N>m.html`
390
+ **Purpose:** One-step TradingView-style 1h chart for a bot (with AMA + order overlay), pool, or pair. Fetches candles in monthly Kibana chunks (pool-first with order-book fallback; `--feed` for MPA price-feed history), then renders via `analysis/tradingview/`. Bot charts pick up the order overlay from `profiles/orders/<botKey>.json` automatically.
391
+ **Output:** `analysis/charts/tv_<bot|pool_<id>|<a>_<b>>_1h_<N>m.html` (`_feed` suffix for feed charts)
392
392
  ```bash
393
393
  # Bot chart (default: 3 months)
394
394
  dexbot tv <bot>
395
395
  # Pool or pair, custom window
396
396
  dexbot tv 133 --month 6
397
397
  dexbot tv TOKENA/TOKENB --month 1 --chart analysis/charts/custom.html
398
+ # MPA price-feed history instead of market candles (opt-in)
399
+ dexbot tv BTS/HONEST.USD --feed --month 1
398
400
  ```
399
401
 
400
402
  ### LP Chart
package/tests/README.md CHANGED
@@ -48,7 +48,7 @@ Key validation, balance queries, account selection.
48
48
 
49
49
  ### Market Data & Pricing
50
50
  Price derivation, orderbook inspection, tolerance checks.
51
- *Examples:* `test_market_price.ts`, `test_price_derive.ts`, `test_price_tolerance.ts`, `test_any_pair.ts`, `test_kibana_candles.ts`
51
+ *Examples:* `test_market_price.ts`, `test_price_derive.ts`, `test_price_tolerance.ts`, `test_any_pair.ts`, `test_kibana_candles.ts`, `test_fetch_book_data.ts`, `test_window_cache.ts`
52
52
 
53
53
  ### Market Adapter
54
54
  AMA signal processing, price offset, bound clamping, signal gates.
@@ -60,7 +60,7 @@ Order lifecycle, fill processing, trade history, batch execution.
60
60
 
61
61
  ### Strategy & Grid Logic
62
62
  Grid generation, sizing, rotation, divergence detection, reconciliation.
63
- *Examples:* `test_order_grid.ts`, `test_strategy_logic.ts`, `test_grid_reconcile.ts`, `test_working_grid.ts`, `test_rotation_order_sizing.ts`, `test_strategy_edge_cases.ts`
63
+ *Examples:* `test_order_grid.ts`, `test_strategy_logic.ts`, `test_grid_reconcile.ts`, `test_working_grid.ts`, `test_rotation_order_sizing.ts`, `test_strategy_edge_cases.ts`, `test_reserve_orders.ts`
64
64
 
65
65
  ### Copy-on-Write (COW) Rebalancing
66
66
  Concurrent-safe rebalancing with isolated working grids — dedicated test suite.