dexbot 1.4.20 → 1.4.21
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +23 -0
- package/README.md +22 -12
- package/analysis/ama_fitting/calibrate_convergence_er.ts +1 -1
- package/analysis/ama_fitting/package.json +1 -1
- package/analysis/bot_fitting/shared_utils.ts +1 -1
- package/analysis/bot_key_utils.ts +1 -5
- package/analysis/bot_usage/kibana_bot_queries.ts +1 -256
- package/analysis/chart_css.ts +1 -1
- package/analysis/math_utils.ts +1 -8
- package/analysis/trend_detection/package.json +1 -1
- package/claw/ecosystem.config.cjs +2 -3
- package/claw/examples/memu_integration_example.ts +17 -17
- package/claw/modules/chain_actions.ts +51 -54
- package/claw/modules/chain_broadcast.ts +66 -104
- package/claw/modules/claw_bridge.ts +60 -80
- package/claw/modules/claw_catalog.ts +47 -16
- package/claw/modules/claw_infra.ts +1 -5
- package/claw/modules/claw_launcher.ts +47 -49
- package/claw/modules/claw_skill_md.ts +8 -11
- package/claw/modules/credit_runtime_adapter.ts +2 -18
- package/claw/modules/decision_loop.ts +4 -0
- package/claw/modules/dexbot_bridge.ts +1 -1
- package/claw/modules/dexbot_profiles.ts +5 -21
- package/claw/modules/honest_ecosystem.ts +24 -8
- package/claw/modules/liquidity_pools.ts +10 -3
- package/claw/modules/mcp_utils.ts +109 -0
- package/claw/modules/memu_bridge.ts +76 -54
- package/claw/modules/mpa_utils.ts +26 -3
- package/claw/modules/position_discovery.ts +16 -27
- package/claw/modules/position_manager.ts +32 -41
- package/claw/modules/position_manager_watch.ts +3 -2
- package/claw/modules/short_mpa_strategy.ts +1 -9
- package/claw/modules/skill_utils.ts +5 -3
- package/claw/modules/utils.ts +9 -1
- package/claw/package.json +1 -1
- package/claw/runtimes/openclaw-plugin/index.ts +27 -10
- package/claw/runtimes/openclaw-plugin/openclaw.plugin.json +1 -1
- package/claw/runtimes/openclaw-plugin/package.json +1 -1
- package/claw/scripts/claw_bridge.ts +1 -1
- package/claw/scripts/claw_mcp_server.ts +19 -78
- package/claw/scripts/honest_assets_report.ts +19 -84
- package/claw/scripts/memu_mcp_server.ts +57 -155
- package/claw/tests/test_claw_bridge.ts +6 -2
- package/claw/tests/test_claw_mcp_transport.ts +2 -2
- package/dist/analysis/ama_fitting/calibrate_convergence_er.js +1 -1
- package/dist/analysis/ama_fitting/calibrate_convergence_er.js.map +1 -1
- package/dist/analysis/bot_fitting/shared_utils.d.ts +1 -12
- package/dist/analysis/bot_fitting/shared_utils.d.ts.map +1 -1
- package/dist/analysis/bot_fitting/shared_utils.js +1 -1
- package/dist/analysis/bot_fitting/shared_utils.js.map +1 -1
- package/dist/analysis/bot_key_utils.d.ts +1 -1
- package/dist/analysis/bot_key_utils.d.ts.map +1 -1
- package/dist/analysis/bot_key_utils.js +1 -5
- package/dist/analysis/bot_key_utils.js.map +1 -1
- package/dist/analysis/bot_usage/kibana_bot_queries.d.ts +1 -337
- package/dist/analysis/bot_usage/kibana_bot_queries.d.ts.map +1 -1
- package/dist/analysis/bot_usage/kibana_bot_queries.js +1 -241
- package/dist/analysis/bot_usage/kibana_bot_queries.js.map +1 -1
- package/dist/analysis/chart_css.d.ts +1 -15
- package/dist/analysis/chart_css.d.ts.map +1 -1
- package/dist/analysis/chart_css.js +1 -1
- package/dist/analysis/chart_css.js.map +1 -1
- package/dist/analysis/math_utils.d.ts +3 -5
- package/dist/analysis/math_utils.d.ts.map +1 -1
- package/dist/analysis/math_utils.js +3 -5
- package/dist/analysis/math_utils.js.map +1 -1
- package/dist/bot.js +2 -2
- package/dist/bot.js.map +1 -1
- package/dist/credential-daemon.js +1 -1
- package/dist/credential-daemon.js.map +1 -1
- package/dist/dexbot.d.ts.map +1 -1
- package/dist/dexbot.js +18 -14
- package/dist/dexbot.js.map +1 -1
- package/dist/market_adapter/ama_signal_runner.js +3 -3
- package/dist/market_adapter/core/kibana_client.d.ts +1 -3
- package/dist/market_adapter/core/kibana_client.d.ts.map +1 -1
- package/dist/market_adapter/core/kibana_client.js +1 -1
- package/dist/market_adapter/core/kibana_client.js.map +1 -1
- package/dist/market_adapter/core/market_adapter_service.d.ts +1 -2
- package/dist/market_adapter/core/market_adapter_service.d.ts.map +1 -1
- package/dist/market_adapter/core/market_adapter_service.js +1 -1
- package/dist/market_adapter/core/market_adapter_service.js.map +1 -1
- package/dist/market_adapter/core/strategies/regime_gate.d.ts +1 -2
- package/dist/market_adapter/core/strategies/regime_gate.d.ts.map +1 -1
- package/dist/market_adapter/core/strategies/regime_gate.js +1 -1
- package/dist/market_adapter/core/strategies/regime_gate.js.map +1 -1
- package/dist/market_adapter/inputs/fetch_cex_synthetic_data.js +2 -2
- package/dist/market_adapter/inputs/fetch_cex_synthetic_data.js.map +1 -1
- package/dist/market_adapter/inputs/fetch_lp_data.d.ts +5 -5
- package/dist/market_adapter/inputs/fetch_lp_data.js +5 -5
- package/dist/market_adapter/inputs/kibana_source.d.ts +1 -2
- package/dist/market_adapter/inputs/kibana_source.d.ts.map +1 -1
- package/dist/market_adapter/inputs/kibana_source.js +1 -1
- package/dist/market_adapter/inputs/kibana_source.js.map +1 -1
- package/dist/market_adapter/lp_chart_strategy_loader.d.ts +1 -2
- package/dist/market_adapter/lp_chart_strategy_loader.d.ts.map +1 -1
- package/dist/market_adapter/lp_chart_strategy_loader.js +4 -4
- package/dist/market_adapter/lp_chart_strategy_loader.js.map +1 -1
- package/dist/market_adapter/market_adapter.d.ts.map +1 -1
- package/dist/market_adapter/market_adapter.js +2 -1
- package/dist/market_adapter/market_adapter.js.map +1 -1
- package/dist/modules/account_bots.d.ts +3 -0
- package/dist/modules/account_bots.d.ts.map +1 -1
- package/dist/modules/account_bots.js +156 -46
- package/dist/modules/account_bots.js.map +1 -1
- package/dist/modules/account_orders.d.ts +1 -7
- package/dist/modules/account_orders.d.ts.map +1 -1
- package/dist/modules/account_orders.js +9 -17
- package/dist/modules/account_orders.js.map +1 -1
- package/dist/modules/bitshares_client.d.ts +2 -14
- package/dist/modules/bitshares_client.d.ts.map +1 -1
- package/dist/modules/bitshares_client.js +22 -31
- package/dist/modules/bitshares_client.js.map +1 -1
- package/dist/modules/bot_settings.js +1 -1
- package/dist/modules/bot_settings.js.map +1 -1
- package/dist/modules/chain_keys.js +2 -2
- package/dist/modules/chain_keys.js.map +1 -1
- package/dist/modules/chain_orders.d.ts +2 -3
- package/dist/modules/chain_orders.d.ts.map +1 -1
- package/dist/modules/chain_orders.js +8 -25
- package/dist/modules/chain_orders.js.map +1 -1
- package/dist/modules/config.d.ts.map +1 -1
- package/dist/modules/config.js +4 -1
- package/dist/modules/config.js.map +1 -1
- package/dist/modules/constants.d.ts +0 -1
- package/dist/modules/constants.d.ts.map +1 -1
- package/dist/modules/constants.js +0 -4
- package/dist/modules/constants.js.map +1 -1
- package/dist/modules/credential_policy.js +1 -1
- package/dist/modules/credential_policy.js.map +1 -1
- package/dist/modules/dexbot_class.d.ts.map +1 -1
- package/dist/modules/dexbot_class.js +3 -2
- package/dist/modules/dexbot_class.js.map +1 -1
- package/dist/modules/dexbot_state_recovery.d.ts.map +1 -1
- package/dist/modules/dexbot_state_recovery.js +2 -1
- package/dist/modules/dexbot_state_recovery.js.map +1 -1
- package/dist/modules/fund_registry.d.ts.map +1 -1
- package/dist/modules/fund_registry.js +5 -2
- package/dist/modules/fund_registry.js.map +1 -1
- package/dist/modules/launcher/bot_supervisor.d.ts.map +1 -1
- package/dist/modules/launcher/bot_supervisor.js +6 -7
- package/dist/modules/launcher/bot_supervisor.js.map +1 -1
- package/dist/modules/launcher/foreign_cred_daemon.d.ts.map +1 -1
- package/dist/modules/launcher/foreign_cred_daemon.js +6 -0
- package/dist/modules/launcher/foreign_cred_daemon.js.map +1 -1
- package/dist/modules/launcher/market_adapter_runtime.d.ts +2 -3
- package/dist/modules/launcher/market_adapter_runtime.d.ts.map +1 -1
- package/dist/modules/launcher/market_adapter_runtime.js +7 -15
- package/dist/modules/launcher/market_adapter_runtime.js.map +1 -1
- package/dist/modules/launcher/monolithic_runtime.d.ts.map +1 -1
- package/dist/modules/launcher/monolithic_runtime.js +1 -0
- package/dist/modules/launcher/monolithic_runtime.js.map +1 -1
- package/dist/modules/launcher/status_reporting.d.ts +1 -2
- package/dist/modules/launcher/status_reporting.d.ts.map +1 -1
- package/dist/modules/launcher/status_reporting.js +1 -4
- package/dist/modules/launcher/status_reporting.js.map +1 -1
- package/dist/modules/order/accounting.js +2 -2
- package/dist/modules/order/accounting.js.map +1 -1
- package/dist/modules/order/format.d.ts +0 -8
- package/dist/modules/order/format.d.ts.map +1 -1
- package/dist/modules/order/format.js +0 -8
- package/dist/modules/order/format.js.map +1 -1
- package/dist/modules/order/logger.js +1 -1
- package/dist/modules/order/logger.js.map +1 -1
- package/dist/modules/order/logger_state.d.ts +1 -13
- package/dist/modules/order/logger_state.d.ts.map +1 -1
- package/dist/modules/order/logger_state.js +1 -11
- package/dist/modules/order/logger_state.js.map +1 -1
- package/dist/modules/order/processed_fill_store.d.ts +0 -5
- package/dist/modules/order/processed_fill_store.d.ts.map +1 -1
- package/dist/modules/order/processed_fill_store.js +1 -12
- package/dist/modules/order/processed_fill_store.js.map +1 -1
- package/dist/modules/order/sync_engine.d.ts.map +1 -1
- package/dist/modules/order/sync_engine.js +10 -3
- package/dist/modules/order/sync_engine.js.map +1 -1
- package/dist/modules/order/utils/math.d.ts +7 -1
- package/dist/modules/order/utils/math.d.ts.map +1 -1
- package/dist/modules/order/utils/math.js +19 -1
- package/dist/modules/order/utils/math.js.map +1 -1
- package/dist/modules/order/utils/order.d.ts.map +1 -1
- package/dist/modules/order/utils/order.js +10 -2
- package/dist/modules/order/utils/order.js.map +1 -1
- package/dist/modules/runtime_settings.js +3 -1
- package/dist/modules/runtime_settings.js.map +1 -1
- package/dist/modules/settings_merge.d.ts.map +1 -1
- package/dist/modules/settings_merge.js +3 -1
- package/dist/modules/settings_merge.js.map +1 -1
- package/dist/modules/storage/browser_adapter.d.ts +8 -3
- package/dist/modules/storage/browser_adapter.d.ts.map +1 -1
- package/dist/modules/storage/browser_adapter.js +53 -4
- package/dist/modules/storage/browser_adapter.js.map +1 -1
- package/dist/modules/utils/sanitize_key.d.ts +7 -0
- package/dist/modules/utils/sanitize_key.d.ts.map +1 -0
- package/dist/modules/utils/sanitize_key.js +15 -0
- package/dist/modules/utils/sanitize_key.js.map +1 -0
- package/dist/pm2.js +6 -6
- package/dist/pm2.js.map +1 -1
- package/dist/scripts/analyze-orders.d.ts +2 -2
- package/dist/scripts/analyze-orders.d.ts.map +1 -1
- package/dist/scripts/analyze-orders.js +4 -12
- package/dist/scripts/analyze-orders.js.map +1 -1
- package/dist/scripts/runner.js +1 -1
- package/dist/scripts/runner.js.map +1 -1
- package/dist/scripts/update.js +1 -1
- package/dist/scripts/update.js.map +1 -1
- package/docs/BITSHARES_ONBOARDING.md +63 -18
- package/docs/COPY_ON_WRITE_MASTER_PLAN.md +5 -14
- package/docs/COW_INVARIANTS.md +4 -4
- package/docs/CREDENTIAL_SECURITY.md +0 -11
- package/docs/DEXBOT_COMPARISON.md +11 -11
- package/docs/EVOLUTION.md +11 -10
- package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md +27 -83
- package/docs/GRID_RECALCULATION.md +1 -13
- package/docs/GRID_RECONCILE.md +24 -24
- package/docs/LIFECYCLE.md +4 -4
- package/docs/LOGGING.md +1 -1
- package/docs/MPA_CREDIT_USAGE.md +3 -3
- package/docs/README.md +2 -2
- package/docs/WORKFLOW.md +2 -2
- package/docs/architecture.md +43 -38
- package/docs/developer_guide.md +16 -18
- package/market_adapter/README.md +0 -2
- package/modules/README.md +2 -1
- package/package.json +1 -1
- package/scripts/README.md +1 -1
- package/claw/openclaw.plugin.json +0 -13
- package/dist/market_adapter/merge_lp_data.d.ts +0 -3
- package/dist/market_adapter/merge_lp_data.d.ts.map +0 -1
- package/dist/market_adapter/merge_lp_data.js +0 -125
- package/dist/market_adapter/merge_lp_data.js.map +0 -1
- package/dist/market_adapter/utils/paths.d.ts +0 -3
- package/dist/market_adapter/utils/paths.d.ts.map +0 -1
- package/dist/market_adapter/utils/paths.js +0 -5
- package/dist/market_adapter/utils/paths.js.map +0 -1
|
@@ -288,16 +288,6 @@ The failure path is consistent regardless of whether the daemon or the
|
|
|
288
288
|
interactive password prompt handled the authentication, making the output
|
|
289
289
|
predictable for monitoring and alerting.
|
|
290
290
|
|
|
291
|
-
### Legacy vault migration
|
|
292
|
-
|
|
293
|
-
Older vaults stored a plain **SHA-256** hash of the master password for
|
|
294
|
-
verification. This hash is deliberately weak by modern standards. The v2
|
|
295
|
-
scrypt-derived vault format replaced it entirely: `masterPasswordHash` is no
|
|
296
|
-
longer read, written, or migrated. A legacy vault that has not been converted
|
|
297
|
-
is rejected by `unlockWithPassword` with "Unsupported key vault format" —
|
|
298
|
-
recreate `profiles/keys.json` with the current key manager to adopt the v2
|
|
299
|
-
format (HMAC-SHA256 vault verifier, scrypt-derived key).
|
|
300
|
-
|
|
301
291
|
---
|
|
302
292
|
|
|
303
293
|
## 6. Startup Path — Daemon-First, Interactive Fallback
|
|
@@ -391,7 +381,6 @@ accounts or after an unclean exit.
|
|
|
391
381
|
| `probeBootstrapSocket` (live probe before cleanup) | Bootstrap dir cleanup | Prevents removing a live bootstrap directory |
|
|
392
382
|
| `delete process.env.DEXBOT_CRED_BOOTSTRAP_PATH_FILE` | Daemon startup | Bootstrap path cannot be inherited by child processes or read from /proc |
|
|
393
383
|
| Attempt limit (3) + immediate exit | Interactive auth | Limits online brute-force window |
|
|
394
|
-
| `masterPasswordHash` removed | Legacy vault upgrade | Weak SHA-256 verifier no longer read, written, or migrated |
|
|
395
384
|
|
|
396
385
|
---
|
|
397
386
|
|
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
# DEXBot vs DEXBot2 — Detailed Comparison Report
|
|
2
2
|
|
|
3
3
|
> **Date:** 2026-08-19 *(metrics refreshed against local source trees)*
|
|
4
|
-
> **Scope:** Full architectural, functional, and operational comparison between the original [DEXBot](https://github.com/Codaone/DEXBot) (Python, v1.0.0) and DEXBot2 (TypeScript, v1.4.
|
|
4
|
+
> **Scope:** Full architectural, functional, and operational comparison between the original [DEXBot](https://github.com/Codaone/DEXBot) (Python, v1.0.0) and DEXBot2 (TypeScript, v1.4.21).
|
|
5
5
|
> **Audience:** Developers, contributors, and operators evaluating or migrating between the two projects.
|
|
6
6
|
|
|
7
7
|
---
|
|
@@ -36,7 +36,7 @@
|
|
|
36
36
|
|
|
37
37
|
| Attribute | DEXBot (original) | DEXBot2 |
|
|
38
38
|
|---|---|---|
|
|
39
|
-
| **Release Track** | 1.0.0 | v1.4.
|
|
39
|
+
| **Release Track** | 1.0.0 | v1.4.21 |
|
|
40
40
|
| **Language** | Python 3.6+ | TypeScript 5.x |
|
|
41
41
|
| **Status** | Released 1.0.0, unmaintained | Active development |
|
|
42
42
|
| **Last Repo Activity** | May 23, 2020 | 2026-08-19 |
|
|
@@ -46,7 +46,7 @@
|
|
|
46
46
|
| **Target Exchange** | BitShares DEX | BitShares DEX |
|
|
47
47
|
| **Lines of Code** | ~10,846 Python LOC in `dexbot/` | Large TypeScript codebase; core runtime, adapter, analysis, Claw, and test modules |
|
|
48
48
|
| **Source Files** | 72 Python files in `dexbot/` | ~527 TS files across the repo |
|
|
49
|
-
| **Test Files** | 16 Python test files |
|
|
49
|
+
| **Test Files** | 16 Python test files | 252 `test_*.ts` files (252 auto-discovered via `globSync`) |
|
|
50
50
|
|
|
51
51
|
### Summary
|
|
52
52
|
|
|
@@ -70,7 +70,7 @@ DEXBot2 is a ground-up rewrite in TypeScript that prioritizes production correct
|
|
|
70
70
|
| **Process Manager** | Systemd service (Linux) | `dexbot start` (native monolithic launcher, default; `unlock` runtime) — PM2 optional |
|
|
71
71
|
| **External APIs** | CoinGecko, CCXT, Waves | On-chain/pool/Kibana candle inputs; optional CEX synthetic seed generator (`fetch_cex_synthetic_data.ts`) for adapter bootstrap |
|
|
72
72
|
| **Container** | Docker (Ubuntu 18.04) | Docker (multi-stage) |
|
|
73
|
-
| **Testing** | pytest + Docker testnet | Native Node assert (
|
|
73
|
+
| **Testing** | pytest + Docker testnet | Native Node assert (252 `test_*.ts` files; auto-discovered via `globSync`) |
|
|
74
74
|
| **CI/CD** | Travis CI, AppVeyor | GitHub Actions / local deterministic script suite |
|
|
75
75
|
| **Packaging** | PyInstaller (Win/Mac/Linux binaries) | npm / `unlock` + optional PM2 ecosystem |
|
|
76
76
|
|
|
@@ -616,7 +616,7 @@ Where:
|
|
|
616
616
|
### DEXBot2
|
|
617
617
|
|
|
618
618
|
- **Framework:** Native Node `assert` module (no external test framework)
|
|
619
|
-
- **
|
|
619
|
+
- **252 `test_*.ts` files** auto-discovered via `globSync` (`tests/test_*.ts` + `claw/tests/test_*.ts`), covering:
|
|
620
620
|
- Unit tests: accounting, strategy, grid, manager logic
|
|
621
621
|
- Copy-on-Write semantics: COW commits, guards, concurrent fills
|
|
622
622
|
- Edge cases: authoritative full-fill resolution, partial fills, BTS fee accounting, precision
|
|
@@ -634,7 +634,7 @@ Where:
|
|
|
634
634
|
| Feature | DEXBot | DEXBot2 |
|
|
635
635
|
|---|---|---|
|
|
636
636
|
| **Framework** | pytest | Native Node assert |
|
|
637
|
-
| **Test Count** | 16 Python test files |
|
|
637
|
+
| **Test Count** | 16 Python test files | 252 `test_*.ts` files; auto-discovered via `globSync` |
|
|
638
638
|
| **Test Types** | Unit + integration | Unit + integration + edge-case + runtime regression |
|
|
639
639
|
| **Testnet Integration** | Yes (Docker) | No (mocks) |
|
|
640
640
|
| **External Dependency** | pytest, Docker | None |
|
|
@@ -780,13 +780,13 @@ Where:
|
|
|
780
780
|
|
|
781
781
|
| Metric | DEXBot | DEXBot2 |
|
|
782
782
|
|---|---|---|
|
|
783
|
-
| **Release Track** | 1.0.0 | v1.4.
|
|
783
|
+
| **Release Track** | 1.0.0 | v1.4.21 |
|
|
784
784
|
| **Active Since** | ~2018 | December 2025 |
|
|
785
785
|
| **Last Commit** | May 23, 2020 | 2026-08-19 |
|
|
786
786
|
| **Total Commits** | 2281 | 2,034 (v1.4.17) |
|
|
787
787
|
| **Lines of Code** | ~10,846 Python LOC in `dexbot/` | Large TypeScript runtime + adapter + Claw + analysis + tests |
|
|
788
788
|
| **Source Files** | 72 Python files in `dexbot/` | ~527 TS files across the repo |
|
|
789
|
-
| **Test Files** | 16 Python test files |
|
|
789
|
+
| **Test Files** | 16 Python test files | 252 `test_*.ts` files (252 auto-discovered via `globSync`) |
|
|
790
790
|
| **Documentation** | Sphinx docs + README | 50+ Markdown docs plus Claw skills/references |
|
|
791
791
|
| **Strategies** | 3 + plugins | 1 |
|
|
792
792
|
| **Max Concurrent Bots** | Many (one process) | Many (one monolithic process by default; per-bot via `--isolated`/PM2) |
|
|
@@ -814,7 +814,7 @@ Each bottleneck in DEXBot compounds because they run *serially in sequence* —
|
|
|
814
814
|
| # | Bottleneck | DEXBot | DEXBot2 (with references) | Multiplier |
|
|
815
815
|
|---|------------|--------|---------------------------|------------|
|
|
816
816
|
| 1 | **RPC queries** | Per-order `get_objects` loop called twice per cycle → 2×N sequential RPCs. | Single batch `get_objects([…])` in `fetchRefBlock` — **`modules/bitshares-native/tx/builder.ts`**; parallel account refresh via `Promise.all` — **`modules/bitshares-native/subscriptions.ts`** (`refreshTasks`, `scanTasks`). | **~400×** |
|
|
817
|
-
| 2 | **Order counting** | Geometric while-loop iterating `price /= 1+increment` ~920 times per call. | O(1) `Math.log` spread-step formula in `
|
|
817
|
+
| 2 | **Order counting** | Geometric while-loop iterating `price /= 1+increment` ~920 times per call. | O(1) `Math.log` spread-step formula in `calculateGapSlots` — **`modules/order/utils/math.ts`**; O(1) spread check in `shouldFlagOutOfSpread` — **`modules/order/utils/order.ts`**. | **~920× CPU** |
|
|
818
818
|
| 3 | **Market price** | Fresh `ticker()` RPC inside every order placement. | Cached `centerPrice` served from `botState`, zero RPC per placement — **`market_adapter/market_adapter.ts`** + **`market_adapter/core/market_adapter_service.ts`**. | **~∞ (eliminated)** |
|
|
819
819
|
| 4 | **Account refresh** | Full `_account.refresh()` fetches all orders + balances + history every cycle. | Targeted `set_subscribe_callback` pushing only `OP_FILL_ORDER` ops — **`modules/bitshares-native/subscriptions.ts`** (`refreshSubscriptions`); filtered per-account; no full re-read — **`modules/chain_orders.ts`**. | **~50×** |
|
|
820
820
|
| 5 | **Thread blocking** | `time.sleep(2–6)` on retry blocks the GIL thread entirely. | Async `await sleep()` + `AsyncLock` queue — **`modules/order/async_lock.ts`**; lock guards in **`modules/order/manager.ts`**; backoff in **`modules/order/utils/system.ts`**. | **~100× I/O utilization** |
|
|
@@ -838,7 +838,7 @@ DEXBot2 parallel path: [batch RPC] ─┐
|
|
|
838
838
|
[JSON write] ┘
|
|
839
839
|
```
|
|
840
840
|
|
|
841
|
-
The Python runtime overhead (≈2× slower than V8 on equivalent CPU work) is the *least* impactful factor here — but it still compounds with everything else. The geometric while-loops and `_calc_increase` iterations all run at Python bytecode speed, thousands of iterations per cycle. DEXBot2 eliminates the iterations entirely with O(1) formulas (`
|
|
841
|
+
The Python runtime overhead (≈2× slower than V8 on equivalent CPU work) is the *least* impactful factor here — but it still compounds with everything else. The geometric while-loops and `_calc_increase` iterations all run at Python bytecode speed, thousands of iterations per cycle. DEXBot2 eliminates the iterations entirely with O(1) formulas (`calculateGapSlots` in `math.ts`, `shouldFlagOutOfSpread` in `order.ts`) — so the 2× language factor is just insurance on top of the architectural gains.
|
|
842
842
|
|
|
843
843
|
The 500× figure is not theoretical: it materializes in production when higher order counts, slower public nodes, transient block-expiration retries, and wide geometric ranges all hit at once — a scenario DEXBot handles by piling seconds onto seconds, while DEXBot2 absorbs each factor with negligible marginal cost.
|
|
844
844
|
|
|
@@ -886,7 +886,7 @@ The 500× figure is not theoretical: it materializes in production when higher o
|
|
|
886
886
|
| **Security** | ★★★☆☆ | ★★★★★ (AES-256-GCM, credential daemon, authority resolution) | DEXBot2 |
|
|
887
887
|
| **Ease of Setup** | ★★☆☆☆ (PyQt5/PyInstaller/Systemd dependency hell) | ★★★★★ (`npm i -g dexbot`, zero deps) | DEXBot2 |
|
|
888
888
|
| **Accessibility** | ★★★★★ (GUI) | ★★☆☆☆ (CLI only) | DEXBot |
|
|
889
|
-
| **Testing Depth** | ★★★☆☆ | ★★★★★ (
|
|
889
|
+
| **Testing Depth** | ★★★☆☆ | ★★★★★ (252 test files; focused regressions) | DEXBot2 |
|
|
890
890
|
| **Documentation** | ★★★☆☆ | ★★★★★ (architecture/accounting/security/adapter docs) | DEXBot2 |
|
|
891
891
|
| **Dependency Footprint** | ★★☆☆☆ (heavy) | ★★★★★ (0 runtime deps) | DEXBot2 |
|
|
892
892
|
| **Extensibility** | ★★★★★ (plugins) | ★☆☆☆☆ | DEXBot |
|
package/docs/EVOLUTION.md
CHANGED
|
@@ -2,14 +2,14 @@
|
|
|
2
2
|
|
|
3
3
|
## Executive Summary
|
|
4
4
|
|
|
5
|
-
DEXBot2 is a sophisticated decentralized exchange trading bot for the BitShares blockchain. This report documents the complete evolution of the project from its inception in December 2025 through the current 1.4.
|
|
5
|
+
DEXBot2 is a sophisticated decentralized exchange trading bot for the BitShares blockchain. This report documents the complete evolution of the project from its inception in December 2025 through the current 1.4.21 stable release.
|
|
6
6
|
|
|
7
7
|
### Key Milestones
|
|
8
8
|
- **Project Inception**: December 2, 2025
|
|
9
|
-
- **Growth Phase**: 2,
|
|
9
|
+
- **Growth Phase**: 2,068 commits over ~9 active months
|
|
10
10
|
- **Code Maturity**: Evolution from basic utilities to a ~70,000+ LoC intelligent TypeScript system
|
|
11
|
-
- **Stability**: Progression from manual testing to a suite of
|
|
12
|
-
- **Releases**:
|
|
11
|
+
- **Stability**: Progression from manual testing to a suite of 252 automated test files
|
|
12
|
+
- **Releases**: 98 release entries (v0.1.0 to v1.4.21)
|
|
13
13
|
|
|
14
14
|
---
|
|
15
15
|
|
|
@@ -61,13 +61,13 @@ v1.4.12 completed the module transition to native ES modules (root + claw `"type
|
|
|
61
61
|
|
|
62
62
|
### Phase 9: Post-ESM Cleanup, Consolidation & Hardening (August 2026)
|
|
63
63
|
|
|
64
|
-
v1.4.16 centralized all user/runtime state onto a single resolver-derived profiles dir (`~/.config/dexbot2/profiles`) so it survives re-clones and npm updates and never lands in a read-only package dir, turned divergence surplus-cancel + hole-create pairs into in-place order rotations, and added an npm auto-update flow. v1.4.17 consolidated duplicate code (EC math, Base58Check, settings merge, asset resolution), trimmed `modules/types.ts` from 875 lines to the Order union, purged dead exports, and centralized the analysis tooling under strict TypeScript. v1.4.19 capped COW broadcasts at `MAX_OPS_PER_BROADCAST` (4) with chunked retry-on-uncertain broadcasting, fixed a spread-collapse regression via the shared `isSlotInRail` helper, lowered the AMA slope grid-reset threshold to 8, and added editor price feedback plus a `docs/LIFECYCLE.md` onboarding walkthrough. v1.4.20 hardened spread-correction boundary promotion with a MIN_SPREAD_ORDERS reserve floor and commit-time validation, gated persisted-boundary restore against self-legalizing overrun poison, aligned dynamic-weight clip parity between the live service and the research chart, centralized chart slider ranges, and moved all analysis outputs onto the central path resolver.
|
|
64
|
+
v1.4.16 centralized all user/runtime state onto a single resolver-derived profiles dir (`~/.config/dexbot2/profiles`) so it survives re-clones and npm updates and never lands in a read-only package dir, turned divergence surplus-cancel + hole-create pairs into in-place order rotations, and added an npm auto-update flow. v1.4.17 consolidated duplicate code (EC math, Base58Check, settings merge, asset resolution), trimmed `modules/types.ts` from 875 lines to the Order union, purged dead exports, and centralized the analysis tooling under strict TypeScript. v1.4.19 capped COW broadcasts at `MAX_OPS_PER_BROADCAST` (4) with chunked retry-on-uncertain broadcasting, fixed a spread-collapse regression via the shared `isSlotInRail` helper, lowered the AMA slope grid-reset threshold to 8, and added editor price feedback plus a `docs/LIFECYCLE.md` onboarding walkthrough. v1.4.20 hardened spread-correction boundary promotion with a MIN_SPREAD_ORDERS reserve floor and commit-time validation, gated persisted-boundary restore against self-legalizing overrun poison, aligned dynamic-weight clip parity between the live service and the research chart, centralized chart slider ranges, and moved all analysis outputs onto the central path resolver. v1.4.21 fixed silent-failure runtime defects surfaced by a modules-wide audit (NaN fund-invariant tolerance, always-flush fill store, double-decremented fill guard, config num() defaults), aligned the two boundary writers on a shared sell-rail ceiling enforced at commit and restore time, deduplicated claw shared logic while hardening error paths (secret redaction, EPIPE guards, spawn-error surfacing), corrected launcher supervisor/runtime lifecycle bugs, made the browser storage adapter persist deletions and flush debounced, extended editor green/red live color feedback to funds and price inputs, stopped docs/error messages from hardcoding repo-relative profile paths, and pruned dead code across analysis/market_adapter/claw/logging.
|
|
65
65
|
|
|
66
66
|
---
|
|
67
67
|
|
|
68
68
|
## Development Statistics
|
|
69
69
|
|
|
70
|
-
The project has accumulated
|
|
70
|
+
The project has accumulated 252 automated test files across 98 release entries. See the **Version History** below for a per-release commit breakdown.
|
|
71
71
|
|
|
72
72
|
---
|
|
73
73
|
|
|
@@ -86,7 +86,7 @@ The project has accumulated 250 automated test files across 97 release entries.
|
|
|
86
86
|
|
|
87
87
|
## Documentation & Testing
|
|
88
88
|
|
|
89
|
-
Evolved from a basic README to a comprehensive framework (50+ docs entries, 80%+ JSDoc coverage, AGENTS.md). Testing matured from manual blockchain trials → Jest → lightweight Node.js assert across a
|
|
89
|
+
Evolved from a basic README to a comprehensive framework (50+ docs entries, 80%+ JSDoc coverage, AGENTS.md). Testing matured from manual blockchain trials → Jest → lightweight Node.js assert across a 252-file suite covering unit, integration, simulation, and COW architectural guard tests.
|
|
90
90
|
|
|
91
91
|
---
|
|
92
92
|
|
|
@@ -132,11 +132,12 @@ Compact, era-level view; per-release commit detail lives in [CHANGELOG.md](../CH
|
|
|
132
132
|
| v1.4.8 → v1.4.13 | 45 | Native ESM runtime, broadcast serialization, onboarding |
|
|
133
133
|
| v1.4.13 → v1.4.19 | 36 | Profile-state centralization, code consolidation, per-broadcast op cap |
|
|
134
134
|
| v1.4.19 → v1.4.20 | 5 | Grid boundary promotion hardening, recovery poison gate, analysis output centralization |
|
|
135
|
+
| v1.4.20 → v1.4.21 | 15 | Runtime audit fixes, claw dedup hardening, boundary ceiling alignment, editor color feedback |
|
|
135
136
|
|
|
136
137
|
---
|
|
137
138
|
|
|
138
139
|
**Report Originally Generated**: February 19, 2026
|
|
139
|
-
**Last Updated**: August
|
|
140
|
-
**Total Commits**: 2,
|
|
141
|
-
**Date Range**: December 2, 2025 – August
|
|
140
|
+
**Last Updated**: August 24, 2026 (v1.4.21)
|
|
141
|
+
**Total Commits**: 2,068
|
|
142
|
+
**Date Range**: December 2, 2025 – August 24, 2026
|
|
142
143
|
**Repository**: DEXBot2 (BitShares DEX Trading Bot)
|
|
@@ -52,15 +52,13 @@ funds.allocated → _getSizingContext() (budget for spread correction)
|
|
|
52
52
|
**Key points:**
|
|
53
53
|
- `botFunds` percentage applies to **total** capital (free + locked in orders), not just free. A bot at `"50%"` gets half of everything, not half of what's currently idle.
|
|
54
54
|
- `funds.allocated` is the ceiling for each side. Existing orders already consume part of it; the remaining free portion is available for new placements.
|
|
55
|
-
- The downstream `applyBotFundsAllocation()` (`manager.ts:
|
|
55
|
+
- The downstream `applyBotFundsAllocation()` (`manager.ts:949`) also caps `funds.available` to `<= allocated` as a safety net, but the primary budget chokepoint is `getSideBudget` / `_getSizingContext` reading `allocated` directly (v1.2.6).
|
|
56
56
|
|
|
57
57
|
---
|
|
58
58
|
|
|
59
59
|
### 1.4 Mixed Order Fund Validation
|
|
60
60
|
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
**Solution**: Per-asset validation using a signed-delta water-mark. The current validator tracks the **peak** running requirement per asset (not a side lump sum), so BUY and SELL ops in the same batch are validated independently against their own free balance.
|
|
61
|
+
Mixed BUY/SELL batches are validated per asset using a signed-delta **peak** running requirement (not a side lump sum), so BUY and SELL ops in the same batch are checked independently against their own free balance.
|
|
64
62
|
|
|
65
63
|
#### Fund Availability Checks by Asset
|
|
66
64
|
|
|
@@ -69,7 +67,7 @@ funds.allocated → _getSizingContext() (budget for spread correction)
|
|
|
69
67
|
|
|
70
68
|
#### Implementation Location
|
|
71
69
|
|
|
72
|
-
File: `modules/dexbot_cow_runtime.ts` — `validateOperationFunds()` (line
|
|
70
|
+
File: `modules/dexbot_cow_runtime.ts` — `validateOperationFunds()` (line 1488), called from the COW batch broadcast path at line 2908. `modules/dexbot_class.ts` exposes a thin wrapper `_validateOperationFunds()` (line 1122).
|
|
73
71
|
|
|
74
72
|
```javascript
|
|
75
73
|
// Per-asset peak requirement vs. quantized chain-free snapshot.
|
|
@@ -112,19 +110,9 @@ See [developer_guide.md#order-state-helper-functions](developer_guide.md#order-s
|
|
|
112
110
|
|
|
113
111
|
### 1.5 Fill Batch Processing & Timeline
|
|
114
112
|
|
|
115
|
-
####
|
|
116
|
-
|
|
117
|
-
Previously, fills were processed one-at-a-time (~3s per broadcast). A burst of 29 fills in the Feb 7 market crash took ~90 seconds, during which:
|
|
118
|
-
- Market prices moved significantly
|
|
119
|
-
- Orders became stale (filled on-chain but not yet synced)
|
|
120
|
-
- Orphan fills were created (fill events for orders no longer on-chain)
|
|
121
|
-
- Fund tracking diverged from blockchain reality
|
|
122
|
-
|
|
123
|
-
**Impact**: The extended 90s window meant the bot couldn't react to market moves, creating a cascading failure.
|
|
124
|
-
|
|
125
|
-
#### Solution: Fixed-Cap Batch Fill Processing
|
|
113
|
+
#### Fixed-Cap Batch Fill Processing
|
|
126
114
|
|
|
127
|
-
**Mechanism**: Fill events arrive via `modules/dexbot_fill_runtime.ts` (the fill-runtime module), which pushes them into `bot._incomingFillQueue` (declared in `modules/dexbot_class.ts`). The drain loop in `dexbot_fill_runtime.ts` then chunks the queue into capped batches and calls `modules/order/manager.ts::processFilledOrders` (line
|
|
115
|
+
**Mechanism**: Fill events arrive via `modules/dexbot_fill_runtime.ts` (the fill-runtime module), which pushes them into `bot._incomingFillQueue` (declared in `modules/dexbot_class.ts`). The drain loop in `dexbot_fill_runtime.ts` then chunks the queue into capped batches and calls `modules/order/manager.ts::processFilledOrders` (line 1438) once per chunk to run the full rebalance pipeline.
|
|
128
116
|
|
|
129
117
|
**Batch Sizing Algorithm**: A single cap-based batch size (`FILL_PROCESSING.MAX_FILL_BATCH_SIZE`): a queue depth of 4 or fewer is processed as one unified batch; deeper queues are chunked into repeated batches of 4 (the last chunk may be smaller).
|
|
130
118
|
|
|
@@ -176,9 +164,7 @@ IF ratio >= GRID_REGENERATION_PERCENTAGE (default: 3%):
|
|
|
176
164
|
|
|
177
165
|
#### Recovery Retry System
|
|
178
166
|
|
|
179
|
-
|
|
180
|
-
|
|
181
|
-
**New Behavior**: Count+time-based retry system with periodic reset.
|
|
167
|
+
Recovery uses a count+time-based retry system with periodic reset, so a single failed recovery attempt never locks out future retries.
|
|
182
168
|
|
|
183
169
|
**State Machine**:
|
|
184
170
|
```
|
|
@@ -221,25 +207,14 @@ When a batch fails because an on-chain order no longer exists, the cleanup relea
|
|
|
221
207
|
|
|
222
208
|
### 1.6 Remainder Accuracy During Capped Resize
|
|
223
209
|
|
|
224
|
-
|
|
225
|
-
|
|
226
|
-
|
|
227
|
-
- **Fully allocated slots**: received their ideal size (no remainder)
|
|
228
|
-
- **Fund-capped slots**: received less than ideal because available funds ran out mid-allocation
|
|
210
|
+
When grid resize is capped by available funds, accounting tracks what portion of the ideal grid went unallocated via per-slot tracking:
|
|
211
|
+
- **Fully allocated slots**: receive their ideal size (no remainder)
|
|
212
|
+
- **Fund-capped slots**: receive less than ideal because available funds ran out mid-allocation
|
|
229
213
|
|
|
230
|
-
|
|
214
|
+
Computing the remainder from totals instead overstates it when some slots are fully allocated and others are capped.
|
|
231
215
|
|
|
232
|
-
####
|
|
233
|
-
|
|
234
|
-
**Old Behavior** (Incorrect):
|
|
235
|
-
```javascript
|
|
236
|
-
// Compute unallocated remainder from ideal sizes
|
|
237
|
-
const remainder = totalIdealSizes - totalAllocatedSizes;
|
|
238
|
-
// Problem: If actual allocation capped at 80% due to insufficient funds,
|
|
239
|
-
// this uses 100% ideal in calculation → remainder overstated
|
|
240
|
-
```
|
|
216
|
+
#### Per-Slot Tracking
|
|
241
217
|
|
|
242
|
-
**New Behavior** (Correct):
|
|
243
218
|
```javascript
|
|
244
219
|
// Track per-slot applied sizes
|
|
245
220
|
const appliedSizes = [];
|
|
@@ -415,25 +390,9 @@ targetSlot.orderId = newOrderId;
|
|
|
415
390
|
|
|
416
391
|
### 3.6 Orphan-Fill Deduplication & Double-Credit Prevention
|
|
417
392
|
|
|
418
|
-
**Location**: `modules/dexbot_class.ts` — constructor, `_recoverExplicitStaleOrders()` (line
|
|
419
|
-
|
|
420
|
-
#### Problem Solved
|
|
421
|
-
|
|
422
|
-
During Feb 7 market crash, stale-order batch failures cascaded into double-crediting:
|
|
423
|
-
|
|
424
|
-
**Scenario**:
|
|
425
|
-
1. Batch operation scheduled with 12 orders
|
|
426
|
-
2. Order X is on-chain, included in batch
|
|
427
|
-
3. Between sync and broadcast, order X fills on market (stale order)
|
|
428
|
-
4. Batch execution fails: "Limit order X does not exist"
|
|
429
|
-
5. Error handler: Clean up grid slot X, release funds to `chainFree`
|
|
430
|
-
6. Meanwhile, fill event arrives: "Order X was filled at price Y for amount Z"
|
|
431
|
-
7. Orphan-fill handler: Credits proceeds to `chainFree` AGAIN
|
|
432
|
-
8. **Result**: Double-credit of proceeds, inflated `chainTotal`, fund drift
|
|
393
|
+
**Location**: `modules/dexbot_class.ts` — constructor, `_recoverExplicitStaleOrders()` (line 448); orphan-fill guard in the fill drain loop and pruning pass after each cycle now live in `modules/dexbot_fill_runtime.ts` (guard ~lines 628-646, pruning ~lines 934-946).
|
|
433
394
|
|
|
434
|
-
|
|
435
|
-
|
|
436
|
-
#### Solution: Stale-Cleaned Order ID Tracking with TTL + Recycled-Slot Guard
|
|
395
|
+
#### Stale-Cleaned Order ID Tracking with TTL + Recycled-Slot Guard
|
|
437
396
|
|
|
438
397
|
**Mechanism**: Track which orders were cleaned up during batch failure recovery using timestamp + grid-slot retention.
|
|
439
398
|
|
|
@@ -566,7 +525,7 @@ BitShares charges fees for `limit_order_create` and `limit_order_cancel`.
|
|
|
566
525
|
2. If sufficient `chainFree` available: deduct full amount atomically.
|
|
567
526
|
3. If insufficient: defer settlement and retry when funds become available.
|
|
568
527
|
|
|
569
|
-
- **Adoption fee parity
|
|
528
|
+
- **Adoption fee parity:** COW chain-adoption paths charge fees exactly like the normal open-orders loop. `adoptPlacedBatchFromChain` (refused-commit and poll-confirmed paths) and the startup uncertain-create adoption apply the create/cancel/update fees via `_applySync`.
|
|
570
529
|
|
|
571
530
|
- **Safe fee lookup:** `processBatchResults` uses `getAssetFeesSafe('BTS')` with zero-fee fallbacks — the throwing variant can no longer hard-fail a whole batch after a successful commit (`modules/dexbot_cow_runtime.ts`).
|
|
572
531
|
|
|
@@ -588,7 +547,7 @@ These are deducted from the *proceeds* of a fill.
|
|
|
588
547
|
|
|
589
548
|
For BTS fees, the system returns a structured object (not a simple number) with multiple fields for accounting precision.
|
|
590
549
|
|
|
591
|
-
**Location**: `modules/order/utils/math.ts::getAssetFees()` (line 312). The fee cache itself is populated by `modules/order/utils/system.ts::initializeFeeCache()` (line
|
|
550
|
+
**Location**: `modules/order/utils/math.ts::getAssetFees()` (line 312). The fee cache itself is populated by `modules/order/utils/system.ts::initializeFeeCache()` (line 665).
|
|
592
551
|
|
|
593
552
|
#### BTS Fee Object (Always Object)
|
|
594
553
|
|
|
@@ -651,10 +610,6 @@ const createFee = feeInfo.createFee; // BTS only
|
|
|
651
610
|
|
|
652
611
|
### 5.4 BUY Side Sizing & Fee Accounting
|
|
653
612
|
|
|
654
|
-
**Problem Fixed**: BUY side fee calculations incorrectly applied fees to base asset instead of quote asset.
|
|
655
|
-
|
|
656
|
-
**Solution**: Corrected fee accounting with proper asset assignment.
|
|
657
|
-
|
|
658
613
|
#### Fee Application by Side
|
|
659
614
|
|
|
660
615
|
| Side | Asset | Calculation | Notes |
|
|
@@ -702,13 +657,11 @@ For BUY orders that are makers:
|
|
|
702
657
|
|
|
703
658
|
### 5.5 Precision & Quantization
|
|
704
659
|
|
|
705
|
-
|
|
706
|
-
|
|
707
|
-
**Solution**: Centralized quantization utilities that eliminate float accumulation by round-tripping through blockchain integer representation.
|
|
660
|
+
Floating-point arithmetic drifts from true blockchain integer representations over many order-size calculations, price derivations, and fund allocations. Quantization eliminates this accumulation by round-tripping every value through its blockchain integer form.
|
|
708
661
|
|
|
709
662
|
#### 5.5.1 Core Quantization Functions
|
|
710
663
|
|
|
711
|
-
**Location**: `modules/order/utils/math.ts` (line
|
|
664
|
+
**Location**: `modules/order/utils/math.ts` (line 265)
|
|
712
665
|
|
|
713
666
|
##### `quantizeFloat(value, precision)` - Eliminate Accumulation Errors
|
|
714
667
|
|
|
@@ -771,21 +724,13 @@ const normalized = normalizeInt(currentSizeInt, 8);
|
|
|
771
724
|
- Normalizing fund totals before invariant checks
|
|
772
725
|
- Preparing sizes for blockchain transaction encoding
|
|
773
726
|
|
|
774
|
-
#### 5.5.2
|
|
727
|
+
#### 5.5.2 Consolidated Quantization
|
|
775
728
|
|
|
776
|
-
|
|
777
|
-
- `dexbot_class.ts` - Manual rounding logic
|
|
778
|
-
- `order.ts` - Custom precision handling
|
|
779
|
-
- `strategy.ts` - Divergent rounding approach
|
|
780
|
-
- `chain_orders.ts` - Different quantization pattern
|
|
781
|
-
- `export.ts` - Isolated float conversions
|
|
729
|
+
Quantization has a single source of truth: `quantizeFloat()` in `modules/order/utils/math.ts`.
|
|
782
730
|
|
|
783
|
-
**After Consolidation:**
|
|
784
|
-
✅ Single source of truth (`math.ts`)
|
|
785
731
|
✅ Consistent precision handling across all modules
|
|
786
732
|
✅ Reduced regression risk (tested once, used everywhere)
|
|
787
|
-
✅
|
|
788
|
-
✅ All 34+ test suites pass with zero regressions
|
|
733
|
+
✅ No subtle float accumulation bugs from divergent rounding paths
|
|
789
734
|
|
|
790
735
|
#### 5.5.3 Precision Best Practices
|
|
791
736
|
|
|
@@ -818,7 +763,7 @@ This prevents the bug where `available = chainFree + required` created a tautolo
|
|
|
818
763
|
|
|
819
764
|
## 6. Safety & Invariants
|
|
820
765
|
|
|
821
|
-
The `Accountant` enforces strict mathematical invariants to detect bugs or manual interference. Invariants are checked by `_verifyFundInvariants()` (`modules/order/accounting.ts` line
|
|
766
|
+
The `Accountant` enforces strict mathematical invariants to detect bugs or manual interference. Invariants are checked by `_verifyFundInvariants()` (`modules/order/accounting.ts` line 501) after every blockchain sync cycle. The verification reads from a snapshot captured under `_fundLock` — `actualBuy`/`actualSell` are captured at snapshot time, not read live outside the lock, closing a TOCTOU window. When a violation is detected, the system logs a `CRITICAL` error and attempts automatic recovery via `manager.accountant.recalculateFunds()` (`modules/order/accounting.ts` line 346, delegated from `modules/order/manager.ts` lines 981–990) — resetting internal state to match on-chain reality. If the grid lock is held (mid-rebalance), recovery is deferred until the lock is released. The bot continues operating throughout; it does **not** halt on invariant violations.
|
|
822
767
|
|
|
823
768
|
### 6.1 The Equality Invariant
|
|
824
769
|
Total funds on chain must equal free plus committed.
|
|
@@ -834,18 +779,17 @@ A violation here means the grid has allocated more capital than actually exists
|
|
|
834
779
|
|
|
835
780
|
### 6.3 Race Condition Protection (TOCTOU)
|
|
836
781
|
To prevent "Time-of-Check to Time-of-Use" errors:
|
|
837
|
-
1. **Locking:** `AsyncLock` (re-entrant) prevents concurrent updates to the same order. Nested `acquire()` from the same execution context runs the callback directly instead of queueing
|
|
782
|
+
1. **Locking:** `AsyncLock` (re-entrant) prevents concurrent updates to the same order. Nested `acquire()` from the same execution context runs the callback directly instead of queueing.
|
|
838
783
|
2. **Atomic Deduct:** `tryDeductFromChainFree` checks *and* subtracts in a single synchronous step.
|
|
839
784
|
3. **Bootstrapping:** Fills arriving during startup (`isBootstrapping=true`) are queued until the grid is fully reconciled ([GRID_RECONCILE.md](GRID_RECONCILE.md)).
|
|
840
785
|
|
|
841
|
-
### 6.4 Stale Accounting & Fee Over-Credit Guards
|
|
842
|
-
Two additional accounting hardening measures added in v1.2.1:
|
|
786
|
+
### 6.4 Stale Accounting & Fee Over-Credit Guards
|
|
843
787
|
|
|
844
|
-
**Stale `accountTotals`
|
|
788
|
+
**Stale `accountTotals` does not abort COW commit.** Transient staleness (e.g., the periodic balance fetch overlaps with a COW commit) logs a `WARN` and schedules recovery instead of throwing `ACCOUNTING_COMMITMENT_FAILED`. Totals are also refreshed after bootstrap to prevent a spurious full recovery on the first maintenance cycle.
|
|
845
789
|
|
|
846
|
-
**Fee-deduction failure
|
|
790
|
+
**Fee-deduction failure logs at `error`.** When `getAssetFees` throws during fill processing (e.g., network blip), `_deductFeesFromProceeds` skips the deduction and logs at `error` with explicit "fund tracking will over-credit" language so operators can detect the drift source in production logs.
|
|
847
791
|
|
|
848
|
-
**TOCTOU in `processFillAccounting`.** `_buildBtsDeferredRefundAdjustment` reads `btsFeeState` from `mgr.orders
|
|
792
|
+
**TOCTOU protection in `processFillAccounting`.** `_buildBtsDeferredRefundAdjustment` reads `btsFeeState` from `mgr.orders` while the order lock is held — the lock is acquired before accounting runs, and the POST-RESET and BOOTSTRAP tracked-fill accounting paths follow the same locking pattern.
|
|
849
793
|
|
|
850
794
|
---
|
|
851
|
-
*Technical Reference for DEXBot2 v1.4.
|
|
795
|
+
*Technical Reference for DEXBot2 v1.4.21 release*
|
|
@@ -397,7 +397,7 @@ and dedupes while one resync is already pending or running.
|
|
|
397
397
|
- Fill/rotation operations are causing noticeable size errors
|
|
398
398
|
|
|
399
399
|
**Disable (set to 0) if:**
|
|
400
|
-
- You want to rely ONLY on AMA triggers
|
|
400
|
+
- You want to rely ONLY on AMA triggers
|
|
401
401
|
- You want to prevent automatic full resync from divergence alone
|
|
402
402
|
- You manually trigger full grid resyncs through other mechanisms
|
|
403
403
|
|
|
@@ -596,18 +596,6 @@ Removed trigger file.
|
|
|
596
596
|
|
|
597
597
|
---
|
|
598
598
|
|
|
599
|
-
## Related Issues
|
|
600
|
-
|
|
601
|
-
> Issue numbers below refer to the project's historical internal tracker and
|
|
602
|
-
> are kept here for archival context. They are not always resolvable against
|
|
603
|
-
> external issue trackers.
|
|
604
|
-
|
|
605
|
-
- **Issue #5:** RMS Divergence Check Disabling — Ability to set `RMS_PERCENTAGE: 0` to disable checks
|
|
606
|
-
- **Feature:** AMA Integration — AMA-derived center snapshots are already used for market-adapter-triggered grid recentering
|
|
607
|
-
- **Issue #1:** Fund Validation Bug — Fixed validation logic for order batch placement
|
|
608
|
-
|
|
609
|
-
---
|
|
610
|
-
|
|
611
599
|
## References
|
|
612
600
|
|
|
613
601
|
- `modules/constants.ts` — Default configuration values
|
package/docs/GRID_RECONCILE.md
CHANGED
|
@@ -78,16 +78,16 @@ Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations a
|
|
|
78
78
|
|
|
79
79
|
### Phase 1 — Pure Planning Under `_gridLock`
|
|
80
80
|
|
|
81
|
-
**`grid_reconcile.ts:
|
|
81
|
+
**`grid_reconcile.ts:208-375`**
|
|
82
82
|
|
|
83
|
-
1. **Phantom order sanitization** (lines
|
|
84
|
-
- **Freshly-assigned deferral** (lines
|
|
85
|
-
- **Ghost heuristic** (line
|
|
83
|
+
1. **Phantom order sanitization** (lines 216-245): For each `isOrderPlaced()` order whose `orderId` is not in the chain snapshot, reset it to VIRTUAL with `skipAccounting` to prevent fund inflation. Two absence-decision guards make this safe:
|
|
84
|
+
- **Freshly-assigned deferral** (lines 226-235): an `orderId` assigned within `TIMING.SYNC_LOCK_TIMEOUT_MS` (via `manager._orderIdAssignedAt`) may be an in-flight create/adopt whose broadcast has not landed or is not yet visible to a lagging/truncated read — virtualizing it and re-creating would duplicate a real live order (the reconcile-timeout death-spiral root cause). It is skipped (deferred) rather than virtualized.
|
|
85
|
+
- **Ghost heuristic** (line 229): an order with `size <= 0` && `PARTIAL` (a known filled ghost) still passes through so known fills get cleaned up.
|
|
86
86
|
- Virtualization always uses `{ skipAccounting: true }` so startup cleanup never inflates `ChainFree`.
|
|
87
87
|
|
|
88
|
-
2. **Duplicate detection** (lines
|
|
88
|
+
2. **Duplicate detection** (lines 258-331): For each unmatched chain order, find the nearest active same-side grid order. If `priceDiff ≤ tolerance × 5`, flag it as a suspected duplicate and queue for a Phase 2 cancel (never cancelled under lock). Tolerance is computed from price impact via `calculatePriceTolerance`: capped at `PRICE_TOLERANCE_MAX_PERCENT` (1%) with a `PRICE_TOLERANCE_MIN_ABSOLUTE` (0.0001) floor. Duplicate IDs are removed from the unmatched set so they aren't also paired for updates/creates.
|
|
89
89
|
|
|
90
|
-
3. **Per-side reconciliation** via `_reconcileStartupSide(planOnly=true)` (lines
|
|
90
|
+
3. **Per-side reconciliation** via `_reconcileStartupSide(planOnly=true)` (lines 343-372):
|
|
91
91
|
- Count `matchedOnGrid` (active grid orders with `orderId`)
|
|
92
92
|
- `neededSlots = targetCount - matchedOnGrid`; pick virtual slots to activate
|
|
93
93
|
- Match sorted unmatched chain orders to virtual slots → `plannedUpdates`
|
|
@@ -99,29 +99,29 @@ Returns `{ plannedCreates, plannedUpdates, plannedCancels, chainSellCount, chain
|
|
|
99
99
|
|
|
100
100
|
### Phase 2 — Blockchain Execution Outside Lock
|
|
101
101
|
|
|
102
|
-
**`grid_reconcile.ts:
|
|
102
|
+
**`grid_reconcile.ts:376-500`**
|
|
103
103
|
|
|
104
104
|
Each sub-phase releases `_gridLock` before starting and re-acquires it per operation (through `synchronizeWithChain` in individual helpers). No single long-held lock blocks fills, sync, or divergence checks — but each operation still runs under the lock for consistency.
|
|
105
105
|
|
|
106
|
-
**Cancellations** (lines
|
|
106
|
+
**Cancellations** (lines 384-411): Execute `plannedCancels`. Each `_cancelChainOrder` acquires `_gridLock` internally. This covers duplicate cancels, edge-release cancels, and excess-order cancels.
|
|
107
107
|
|
|
108
|
-
**Updates** (lines
|
|
108
|
+
**Updates** (lines 413-485):
|
|
109
109
|
- Batch via `_executeStartupUpdateBatch` when `supportsBatchUpdate` is available
|
|
110
110
|
- Retry up to 3× (`maxBatchAttempts = 3`)
|
|
111
111
|
- On each failure: `_recoverStartupSyncFailure()` re-fetches open orders from chain (guarded read) and re-syncs `manager` state via `manager.syncFromOpenOrders()`, then `_refreshStartupUpdatePlans()` rebuilds plans against the fresh chain state
|
|
112
112
|
- If retries exhausted or batch helpers are unavailable → `_executeStartupSequentialUpdateFallback()` one-by-one with per-failure recovery
|
|
113
113
|
|
|
114
|
-
**Creates** (lines
|
|
114
|
+
**Creates** (lines 487-500): `_executePlannedStartupCreates` runs with the outside-in pair grouping — grouped from the outermost grid slots toward the center, BUY descending / SELL ascending, so the most price-critical orders are placed first. BitShares DEX batch-create operations are used where supported. Every created chain ID is captured into `phase2CreatedOrderIds` so Phase 3 cannot later cancel the freshly-created orders.
|
|
115
115
|
|
|
116
116
|
### Phase 3 — Fresh Re-read, Adoption, Stale Surplus Cleanup
|
|
117
117
|
|
|
118
|
-
**`grid_reconcile.ts:502-
|
|
118
|
+
**`grid_reconcile.ts:502-641`** (guarded by `if (!dryRun)` at line 504)
|
|
119
119
|
|
|
120
120
|
1. **Guarded fresh re-read** (lines 512-516): `readOpenOrdersGuarded` re-fetches all open orders. On a truncated/empty read it returns early (defers), keeping the pre-Phase-2 counts for the summary log — a capped window omits exactly the freshest Phase-2 creates.
|
|
121
121
|
|
|
122
122
|
2. **Adopt uncertain-landed creates** (lines 533-578): For any fresh chain order not matching a grid `orderId` and not created by a slot, it attempts targeted slot adoption — matching a VIRTUAL slot by type+price+size (within tolerance) and registering it via `_applySync(..., 'createOrder')` with the create-fee deduction. Full `syncFromOpenOrders` is deliberately **not** used here (its pass-1 virtualizes ACTIVE slots missing from the snapshot, and a lagging read right after the Phase-2 broadcast would destroy the confirmed grid). If adoption fails, the ID is still protected from surplus-cancel; the next sync loop's orphan adoption registers it.
|
|
123
123
|
|
|
124
|
-
3. **Stale surplus cancellation** (lines 579-
|
|
124
|
+
3. **Stale surplus cancellation** (lines 579-625): Per side, count orders exceeding `targetCount` that no grid slot holds via `orderId` (including the phase-2 created IDs). Cancel only these untracked surplus orders, sorted by chain ID for determinism. This catches orphans lost during grid reinitialization — on-chain orders with no corresponding grid slot.
|
|
125
125
|
|
|
126
126
|
### Partial Failure State
|
|
127
127
|
|
|
@@ -130,8 +130,8 @@ If Phase 2 partially succeeds (some cancels, some creates fail), there is no rol
|
|
|
130
130
|
### Timeouts and Read Coverage
|
|
131
131
|
|
|
132
132
|
- **No per-attempt race** around the reconcile itself — the 1.4.8 change removed it to avoid orphaning mid-batch broadcasts (see the [`recalculateGrid`](../modules/order/grid.ts) call site in `modules/order/grid.ts`).
|
|
133
|
-
- The whole resync is bounded by a **10-minute total timeout** (`PIPELINE_TIMING.TIMEOUT_MS * 2` at `grid.ts:
|
|
134
|
-
- Every internal chain read goes through `readOpenOrdersGuarded` (`chain_orders.ts:
|
|
133
|
+
- The whole resync is bounded by a **10-minute total timeout** (`PIPELINE_TIMING.TIMEOUT_MS * 2` at `grid.ts:1184`), applied via `Promise.race` at `grid.ts:1279`.
|
|
134
|
+
- Every internal chain read goes through `readOpenOrdersGuarded` (`chain_orders.ts:610`) with the 30s / 3-retry / node-failover standard, and empty/truncated reads are treated as **ambiguous** — never as authoritative absence.
|
|
135
135
|
|
|
136
136
|
---
|
|
137
137
|
|
|
@@ -139,7 +139,7 @@ If Phase 2 partially succeeds (some cancels, some creates fail), there is no rol
|
|
|
139
139
|
|
|
140
140
|
### Fresh Grid Guard (`matchedOnGrid > 0`)
|
|
141
141
|
|
|
142
|
-
**`grid_reconcile_internal.ts:
|
|
142
|
+
**`grid_reconcile_internal.ts:1600`**
|
|
143
143
|
|
|
144
144
|
When a brand-new grid is generated, every slot is VIRTUAL — `matchedOnGrid = 0`. Without a guard, every on-chain order appears "unmatched" and would be cancelled as excess:
|
|
145
145
|
|
|
@@ -153,11 +153,11 @@ When `matchedOnGrid === 0` AND scaling up (`neededSlots > 0`), excess cancellati
|
|
|
153
153
|
|
|
154
154
|
### Grid-Edge Lock
|
|
155
155
|
|
|
156
|
-
**`grid_reconcile_internal.ts:
|
|
156
|
+
**`grid_reconcile_internal.ts:244`** — `_isGridEdgeFullyActive` detects when the grid boundary is fully active (all slots on-chain) before cancelling excess orders.
|
|
157
157
|
|
|
158
|
-
When all outermost orders of a side are ACTIVE with `orderId`, all balance is committed to the edges. Cancel the **largest** order among the update candidates (`_cancelLargestOrder`, line
|
|
158
|
+
When all outermost orders of a side are ACTIVE with `orderId`, all balance is committed to the edges. Cancel the **largest** order among the update candidates (`_cancelLargestOrder`, line 314) to free maximum funds with minimum operations, since the DEX does not expose partial-reduce in one operation. The cancelled slot gets a replacement create.
|
|
159
159
|
|
|
160
|
-
Detection (`_isGridEdgeFullyActive`, line
|
|
160
|
+
Detection (`_isGridEdgeFullyActive`, line 244): sort orders by price (BUY descending, SELL ascending), and check the outermost ones are all `isOrderPlaced()`.
|
|
161
161
|
|
|
162
162
|
### Duplicate Tolerance (5× Multiplier)
|
|
163
163
|
|
|
@@ -181,10 +181,10 @@ Reconcile Phase 1 runs under `_gridLock` with no side effects on the frozen mast
|
|
|
181
181
|
|
|
182
182
|
### Truncated-Read Ambiguity (since 1.4.8)
|
|
183
183
|
|
|
184
|
-
Every chain read feeding an absence/surplus decision goes through `readOpenOrdersGuarded` (`chain_orders.ts:
|
|
184
|
+
Every chain read feeding an absence/surplus decision goes through `readOpenOrdersGuarded` (`chain_orders.ts:610`) and treats an empty or truncated snapshot as **unreadable** — never as "nothing landed" or "nothing to cancel":
|
|
185
185
|
|
|
186
|
-
- `_recoverSyncFromChain` (`grid_reconcile_internal.ts:
|
|
187
|
-
- `_adoptPossiblyLandedCreate` (`grid_reconcile_internal.ts:
|
|
186
|
+
- `_recoverSyncFromChain` (`grid_reconcile_internal.ts:592`) — plus its three recovery sites in `_createOrderFromGrid` / `_cancelChainOrder` — defers on empty/truncated reads (`deferEmpty: true`). A pass-1 phantom cleanup would otherwise virtualize live slots from a partial window.
|
|
187
|
+
- `_adoptPossiblyLandedCreate` (`grid_reconcile_internal.ts:932`) defers to an uncertain outcome on truncated reads, and the startup group batch uncertain verification follows the same rule.
|
|
188
188
|
- Phase 3 final refresh (`grid_reconcile.ts:512`) skips adoption/surplus-cancel on a truncated read, keeping the pre-phase-2 counts for the summary log.
|
|
189
189
|
- Adoption paths (`_adoptPossiblyLandedCreate`, grouping path, reconcile adoption loop) apply the create-fee deduction via `_applySync` for accounting parity.
|
|
190
190
|
|
|
@@ -194,7 +194,7 @@ The underlying rule is `INV-BROADCAST-004`: a capped `get_full_accounts` window
|
|
|
194
194
|
|
|
195
195
|
## Lock Hierarchy
|
|
196
196
|
|
|
197
|
-
**`manager.ts:
|
|
197
|
+
**`manager.ts:474-489`** — canonical reference in [`developer_guide.md`](developer_guide.md#lock-ordering-for-deadlock-prevention).
|
|
198
198
|
|
|
199
199
|
```
|
|
200
200
|
Level 0: _fillProcessingLock Level 1: _divergenceLock
|
|
@@ -224,7 +224,7 @@ Commit `e64db685` replaced 6 single-value boolean state fields with refcounts/st
|
|
|
224
224
|
| `maxBatchAttempts` | `3` | `grid_reconcile.ts:415` | Update-batch retry limit |
|
|
225
225
|
| `PRICE_TOLERANCE_MAX_PERCENT` | `0.01` (1%) | `constants.ts:452` | Cap on price tolerance |
|
|
226
226
|
| `PRICE_TOLERANCE_MIN_ABSOLUTE` | `0.0001` | `constants.ts:456` | Floor for price tolerance |
|
|
227
|
-
| `PIPELINE_TIMING.TIMEOUT_MS` | `300000` (5min) | `constants.ts:
|
|
227
|
+
| `PIPELINE_TIMING.TIMEOUT_MS` | `300000` (5min) | `constants.ts:800` | Base pipeline timing; resync uses 2× (10 min) |
|
|
228
228
|
|
|
229
229
|
---
|
|
230
230
|
|
|
@@ -250,7 +250,7 @@ Commit `e64db685` replaced 6 single-value boolean state fields with refcounts/st
|
|
|
250
250
|
| File | Role |
|
|
251
251
|
|------|------|
|
|
252
252
|
| `modules/order/grid_reconcile.ts` | Public API + 3-phase orchestrator (642 lines) |
|
|
253
|
-
| `modules/order/grid_reconcile_internal.ts` | Internal helpers — `_reconcileStartupSide`, grid detection, recovery, uncertainty (
|
|
253
|
+
| `modules/order/grid_reconcile_internal.ts` | Internal helpers — `_reconcileStartupSide`, grid detection, recovery, uncertainty (1675 lines) |
|
|
254
254
|
| `modules/order/manager.ts` | Lock hierarchy definition, `_applyOrderUpdate`, phantom guard, `reconcileGrid` entry, COW integration |
|
|
255
255
|
| `modules/order/async_lock.ts` | AsyncLock engine with ALS re-entrancy (424 lines) |
|
|
256
256
|
| `modules/order/sync_engine.ts` | Blockchain sync pipeline |
|
package/docs/LIFECYCLE.md
CHANGED
|
@@ -134,7 +134,7 @@ sequenceDiagram
|
|
|
134
134
|
participant STORE as Storage
|
|
135
135
|
|
|
136
136
|
MA->>CL: AMA center snapshot updated
|
|
137
|
-
CL->>CL: _performPeriodicGridChecks() (dexbot_class.ts:
|
|
137
|
+
CL->>CL: _performPeriodicGridChecks() (dexbot_class.ts:1694)
|
|
138
138
|
CL->>RT: performPeriodicGridChecks() (dexbot_maintenance_runtime.ts:1043)
|
|
139
139
|
RT->>RT: runGridMaintenance(bot,'periodic') (dexbot_maintenance_runtime.ts:1845)
|
|
140
140
|
RT->>RT: executeMaintenanceLogic() (dexbot_maintenance_runtime.ts:1452)
|
|
@@ -158,10 +158,10 @@ the parameters.
|
|
|
158
158
|
|
|
159
159
|
> Note: `runMaintenance()` is a **different** subsystem — the credit/MPA debt
|
|
160
160
|
> runtime (`modules/credit_runtime.ts:3041`, reached via
|
|
161
|
-
> `_runCreditRuntimeMaintenance` at `dexbot_class.ts:
|
|
161
|
+
> `_runCreditRuntimeMaintenance` at `dexbot_class.ts:1794`). The grid maintenance
|
|
162
162
|
> chain above is the one that matters for order/price upkeep.
|
|
163
163
|
|
|
164
|
-
References: `modules/dexbot_class.ts:
|
|
164
|
+
References: `modules/dexbot_class.ts:1694` (`_performPeriodicGridChecks`) →
|
|
165
165
|
`modules/dexbot_maintenance_runtime.ts:1043` (`performPeriodicGridChecks`) →
|
|
166
166
|
`:1845` (`runGridMaintenance`) → `:1452` (`executeMaintenanceLogic`),
|
|
167
167
|
`docs/GRID_RECALCULATION.md`, `docs/GRID_RECONCILE.md`.
|
|
@@ -191,7 +191,7 @@ enforced*, not compiler-enforced — learn them or you will introduce fund bugs.
|
|
|
191
191
|
| How a fill becomes orders | `modules/dexbot_fill_runtime.ts` → `modules/order/manager.ts` |
|
|
192
192
|
| Grid math / recalculation | `modules/order/grid.ts`, `docs/GRID_RECALCULATION.md` |
|
|
193
193
|
| Funds & accounting | `modules/order/accounting.ts`, `docs/FUND_MOVEMENT_AND_ACCOUNTING.md` |
|
|
194
|
-
| Periodic loop & AMA hook | `modules/dexbot_maintenance_runtime.ts`, `modules/dexbot_class.ts:
|
|
194
|
+
| Periodic loop & AMA hook | `modules/dexbot_maintenance_runtime.ts`, `modules/dexbot_class.ts:1694` |
|
|
195
195
|
| Market signal source | `market_adapter/market_adapter.ts`, `analysis/README.md` |
|
|
196
196
|
| Startup & orchestration | `modules/dexbot_class.ts`, `docs/developer_guide.md` §"Startup Sequence" |
|
|
197
197
|
| Why COW exists | `docs/architecture.md` §"Copy-on-Write (COW) Grid Pattern", `docs/COW_INVARIANTS.md` |
|