dexbot 1.7.0 → 1.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +42 -0
- package/README.md +3 -3
- package/analysis/README.md +33 -13
- package/claw/package.json +1 -1
- package/claw/runtimes/openclaw-plugin/openclaw.plugin.json +1 -1
- package/claw/runtimes/openclaw-plugin/package.json +1 -1
- package/dist/analysis/account_resolver.js +5 -6
- package/dist/analysis/analyze_liquidity.js +15 -4
- package/dist/analysis/bot_key_utils.d.ts +44 -1
- package/dist/analysis/bot_key_utils.js +76 -7
- package/dist/analysis/fills_cache.js +2 -1
- package/dist/analysis/grid_correction_check.d.ts +1 -19
- package/dist/analysis/grid_correction_check.js +12 -30
- package/dist/analysis/liquidity/liquidity_chart_generator.js +190 -20
- package/dist/analysis/liquidity/liquidity_depth_model.d.ts +41 -16
- package/dist/analysis/liquidity/liquidity_depth_model.js +49 -21
- package/dist/analysis/liquidity/liquidity_view.d.ts +15 -0
- package/dist/analysis/liquidity/liquidity_view.js +11 -0
- package/dist/analysis/resolve_bot_accounts.js +7 -6
- package/dist/analysis/trade_profitability.js +2 -1
- package/dist/credential-daemon.js +13 -2
- package/dist/dexbot.js +4 -4
- package/dist/modules/account_bots.d.ts +11 -1
- package/dist/modules/account_orders.d.ts +12 -0
- package/dist/modules/account_orders.js +30 -0
- package/dist/modules/bitshares-native/transport.js +24 -6
- package/dist/modules/bitshares_client.d.ts +24 -6
- package/dist/modules/bitshares_client.js +44 -7
- package/dist/modules/bot_settings.d.ts +2 -0
- package/dist/modules/constants.d.ts +26 -3
- package/dist/modules/constants.js +63 -8
- package/dist/modules/dexbot_class.d.ts +1 -6
- package/dist/modules/dexbot_class.js +14 -0
- package/dist/modules/dexbot_cow_runtime.d.ts +97 -7
- package/dist/modules/dexbot_cow_runtime.js +620 -132
- package/dist/modules/dexbot_fill_runtime.js +124 -35
- package/dist/modules/dexbot_maintenance_runtime.js +226 -73
- package/dist/modules/dexbot_startup_runtime.js +32 -20
- package/dist/modules/dexbot_state_recovery.d.ts +33 -1
- package/dist/modules/dexbot_state_recovery.js +167 -5
- package/dist/modules/node_connect_policy.d.ts +8 -1
- package/dist/modules/node_connect_policy.js +18 -2
- package/dist/modules/node_flap_score.d.ts +87 -0
- package/dist/modules/node_flap_score.js +124 -0
- package/dist/modules/node_health_cache.d.ts +40 -3
- package/dist/modules/node_health_cache.js +66 -20
- package/dist/modules/node_manager.d.ts +101 -19
- package/dist/modules/node_manager.js +255 -65
- package/dist/modules/order/accounting.js +56 -7
- package/dist/modules/order/format.d.ts +24 -10
- package/dist/modules/order/format.js +30 -10
- package/dist/modules/order/grid.js +82 -11
- package/dist/modules/order/grid_reconcile.d.ts +5 -1
- package/dist/modules/order/grid_reconcile.js +333 -210
- package/dist/modules/order/grid_reconcile_internal.d.ts +38 -2
- package/dist/modules/order/grid_reconcile_internal.js +207 -87
- package/dist/modules/order/index.d.ts +4 -0
- package/dist/modules/order/lifecycle.d.ts +353 -0
- package/dist/modules/order/lifecycle.js +1109 -0
- package/dist/modules/order/manager.d.ts +271 -5
- package/dist/modules/order/manager.js +651 -48
- package/dist/modules/order/placement_lock.d.ts +102 -0
- package/dist/modules/order/placement_lock.js +222 -0
- package/dist/modules/order/strategy.js +13 -1
- package/dist/modules/order/sync_engine.d.ts +13 -1
- package/dist/modules/order/sync_engine.js +153 -53
- package/dist/modules/order/utils/math.d.ts +22 -1
- package/dist/modules/order/utils/math.js +54 -1
- package/dist/modules/order/utils/order.d.ts +75 -5
- package/dist/modules/order/utils/order.js +158 -9
- package/dist/modules/order/utils/system.d.ts +18 -0
- package/dist/modules/order/utils/system.js +84 -8
- package/dist/modules/order/utils/validate.d.ts +7 -1
- package/dist/modules/order/utils/validate.js +48 -2
- package/dist/modules/outcome.d.ts +54 -0
- package/dist/modules/outcome.js +51 -0
- package/dist/modules/settings_merge.d.ts +4 -0
- package/dist/modules/settings_merge.js +9 -0
- package/dist/modules/timing_policy.d.ts +82 -0
- package/dist/modules/timing_policy.js +244 -0
- package/dist/modules/types.d.ts +132 -0
- package/dist/scripts/analyze-credit.js +54 -17
- package/dist/scripts/analyze-orders.d.ts +4 -2
- package/dist/scripts/analyze-orders.js +113 -34
- package/dist/scripts/chart_targets.js +11 -15
- package/dist/scripts/test-credit-renewal.js +12 -5
- package/docs/COW_INVARIANTS.md +67 -8
- package/docs/DEXBOT_COMPARISON.md +6 -6
- package/docs/EVOLUTION.md +9 -8
- package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md +1 -1
- package/docs/GRID_PRICE_INVARIANT.md +1 -1
- package/docs/GRID_RECONCILE.md +17 -12
- package/docs/LOGGING.md +25 -1
- package/docs/ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md +175 -26
- package/docs/README.md +23 -2
- package/docs/WORKFLOW.md +9 -3
- package/docs/architecture.md +42 -5
- package/docs/developer_guide.md +25 -8
- package/modules/README.md +3 -0
- package/package.json +1 -1
- package/scripts/README.md +16 -0
- package/scripts/pdev.sh +14 -1
- package/scripts/pmain.sh +23 -8
- package/scripts/ptest.sh +7 -0
- package/tests/README.md +30 -1
package/docs/GRID_RECONCILE.md
CHANGED
|
@@ -22,7 +22,9 @@ Reconciliation aligns the bot's model with on-chain reality. It runs at startup
|
|
|
22
22
|
|
|
23
23
|
The reconcile runs from [`recalculateGrid`](../modules/order/grid.ts) during startup full-resync. **There is no per-attempt wall-clock race around the reconcile itself** — an outer timeout would fire mid-batch and orphan in-flight broadcasts (the duplicate-accumulation death spiral). The reconcile is bounded only by the 10-minute total resync safety net, and every internal chain read follows the shared guarded-read standard.
|
|
24
24
|
|
|
25
|
-
Phase 1 does all reasoning in memory under `_gridLock` (fast); Phases 2 and 3 execute outside that planning lock — holding it across RPC calls would block fills, sync, and divergence checks. Phase 2 holds the broadcasting single-flight marker, while the sync/apply paths it invokes follow the canonical lock hierarchy. No planning-phase grid lock spans a network operation.
|
|
25
|
+
Phase 1 does all reasoning in memory under `_gridLock` (fast); Phases 2 and 3 execute outside that planning lock — holding it across RPC calls would block fills, sync, and divergence checks. **Both phases run inside the exclusive placement region** (`manager.acquirePlacement('startup-reconcile')`, acquired before the chain snapshot is trusted and released after `stopBroadcasting`), so no other chain-mutating path can broadcast between Phase 1's plan and Phase 2's execution. Phase 2 additionally holds the broadcasting single-flight marker (which only defers fills), while the sync/apply paths it invokes follow the canonical lock hierarchy. No planning-phase grid lock spans a network operation.
|
|
26
|
+
|
|
27
|
+
The reconcile is not purely time-unbounded: besides the 10-minute total resync safety net it also waits up to 120 s for the placement region and re-reads the open-orders snapshot after acquiring it (planning from a snapshot older than the region is exactly the duplicate class). A reconcile that cannot get the region returns `null` and defers to the next sync cycle.
|
|
26
28
|
|
|
27
29
|
Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations are attempted — plans are logged but not executed.
|
|
28
30
|
|
|
@@ -80,8 +82,8 @@ Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations a
|
|
|
80
82
|
|
|
81
83
|
**Source:** `reconcileGridOrders()` Phase 1 in `grid_reconcile.ts`.
|
|
82
84
|
|
|
83
|
-
1. **Phantom order sanitization** (the `isOrderPlaced()` absence pass in `reconcileGridOrders()`): For each `isOrderPlaced()` order whose `orderId` is not in the chain snapshot, reset it to VIRTUAL with `skipAccounting` to prevent fund inflation.
|
|
84
|
-
- **
|
|
85
|
+
1. **Phantom order sanitization** (the `isOrderPlaced()` absence pass in `reconcileGridOrders()`): For each `isOrderPlaced()` order whose `orderId` is not in the chain snapshot, reset it to VIRTUAL with `skipAccounting` to prevent fund inflation. One absence-decision guard makes this safe:
|
|
86
|
+
- **In-flight deferral** (`LifecycleView.isInFlight()`, read through the slot with `readSlotLifecycle`): an order whose chain id was assigned/committed within the guard window, or whose lifecycle phase is still unresolved (`unknown`/`broadcasting` — a lost CREATE receipt, an ambiguous or unparseable read), may be live and merely missing from a lagging/truncated read — virtualizing it and re-creating would duplicate a real live order (the reconcile-timeout death-spiral root cause). It is skipped (deferred) rather than virtualized. The same predicate is the only absence guard in the sync pass-1 phantom cleanup and the fund recalibration sweep; asking a per-site subset is the bug class [ORDER_LIFECYCLE.md](ORDER_LIFECYCLE.md) exists to remove.
|
|
85
87
|
- **Ghost heuristic** (same absence pass): an order with `size <= 0` && `PARTIAL` (a known filled ghost) still passes through so known fills get cleaned up.
|
|
86
88
|
- Virtualization always uses `{ skipAccounting: true }` so startup cleanup never inflates `ChainFree`.
|
|
87
89
|
|
|
@@ -93,7 +95,7 @@ Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations a
|
|
|
93
95
|
- Match sorted unmatched chain orders to virtual slots → `plannedUpdates`
|
|
94
96
|
- Detect grid-edge lock and plan a largest-order cancel
|
|
95
97
|
- Plan creates for remaining slots
|
|
96
|
-
- Plan excess cancellations (guarded by `matchedOnGrid > 0`): orphans first, then **matched surplus** (`chainCount - targetCount`, edge slots last) —
|
|
98
|
+
- Plan excess cancellations (guarded by `matchedOnGrid > 0`): orphans first, then **matched surplus** (`chainCount - targetCount`, edge slots last) — ONE selection feeds both the planOnly and the inline sink (the `excessPlans` list), so planning can never drift from execution; matched slots omit `releaseUntrackedFunds` (their funds are tracked on the grid slot). Every plan stamps the gap geometry from `resolveGapSlots` (never the raw `_gapSlots` field), which is also what the Phase-2 plan validator compares against
|
|
97
99
|
- Fork-kept **shelf orders** (live non-slot-N ids below the rail, e.g. `deep-*`) are never cancel candidates or reserve members — see [Shelf Orders](#shelf-orders-fork-kept-manual-orders)
|
|
98
100
|
- **Vacated-rail refill**: each PROCEEDING update whose vacated price exactly matches (`priceSlotEqual`) an empty, sized, in-rail slot of the same side queues a refill CREATE in the same plan (`source startupVacatedRailRefill`) — skipped updates, ghost prices (lattice moved), in-band slots, and already-desired slots never refill; refill targets require VIRTUAL state with no `orderId`
|
|
99
101
|
|
|
@@ -115,7 +117,9 @@ snapshots. For every plan, `_startupCancelPlanStillCurrent()` requires:
|
|
|
115
117
|
|
|
116
118
|
- the target chain order still exists with its exact Phase-1 signature;
|
|
117
119
|
- a matched plan still owns the same chain order in the same live grid slot;
|
|
118
|
-
- planned type, slot price, boundary, and gap geometry still match
|
|
120
|
+
- planned type, slot price, boundary, and gap geometry still match (both sides
|
|
121
|
+
read the geometry through `resolveGapSlots`, so an unpublished `_gapSlots`
|
|
122
|
+
field cannot make every plan look stale); and
|
|
119
123
|
- an originally unmatched surplus is still not owned by any live slot.
|
|
120
124
|
|
|
121
125
|
Validation is per-plan: an earlier cancellation or unrelated order change does
|
|
@@ -127,7 +131,8 @@ remain unchanged.
|
|
|
127
131
|
|
|
128
132
|
**Fresh-placement grace (`TIMING.SURPLUS_CANCEL_GRACE_MS`, 15 min):** surplus
|
|
129
133
|
and cancel-only cancellations also skip any order whose `orderId` was placed
|
|
130
|
-
inside the grace window. The timestamp is written by
|
|
134
|
+
inside the grace window. The timestamp is written by the lifecycle write funnel
|
|
135
|
+
(and by `recordOrderPlacement` for manager-shaped callers) into
|
|
131
136
|
`manager._placedAt` and tested with `isFreshlyPlacedOrder`; the reconciler's CANCEL
|
|
132
137
|
actions are filtered by `COWRebalanceEngine.execute`, which drops the whole slot's
|
|
133
138
|
action group (a paired CANCEL+CREATE must not lose only its CANCEL, or the CREATE
|
|
@@ -207,7 +212,7 @@ Up to 3 batch attempts (`maxBatchAttempts = 3`). Each failure triggers a recover
|
|
|
207
212
|
|
|
208
213
|
### Phantom Orders via Reconcile
|
|
209
214
|
|
|
210
|
-
Reconcile's role in the defense-in-depth: during Phase 1, any ACTIVE/PARTIAL order whose `orderId` is not found on-chain is reset to VIRTUAL with `skipAccounting`. The
|
|
215
|
+
Reconcile's role in the defense-in-depth: during Phase 1, any ACTIVE/PARTIAL order whose `orderId` is not found on-chain is reset to VIRTUAL with `skipAccounting`. The in-flight deferral (`LifecycleView.isInFlight()`) protects in-flight broadcasts and unresolved receipts, and the ghost heuristic lets known fills pass. See [`developer_guide.md`](developer_guide.md#phantom-orders-prevention-defense-in-depth) for the full 3-layer defense.
|
|
211
216
|
|
|
212
217
|
### Shelf Orders (Fork-Kept Manual Orders)
|
|
213
218
|
|
|
@@ -232,7 +237,7 @@ Reconcile derives its window from the **master grid**, which is only as fresh as
|
|
|
232
237
|
- **Unsettled-fill gate** (`hasUnsettledFills`, `utils/system.ts`): while fills are queued (`_incomingFillQueue`), draining (`_deferredFillsPending`), in flight (`_fillBatchInFlight`) or owed a crawl (`_pendingFillCrawls`), `maybeRunTargetedDriftReconciliation` defers the targeted reconcile entirely and `_reconcileStartupSide` defers only the **window top-up** creates (1:1 repairs — cancelled-slot replacement, vacated-rail refill — still run). The master still counts filled slots as ACTIVE in that window, so a top-up would restore the expensive end of the rail.
|
|
233
238
|
- **Last-fill placement guard** (`isLastFillPlacementBlocked`, `utils/system.ts`): every reconcile placement (create in `_createOrderFromGrid` and the batch create path, plus the re-pricing relocation in `_prepareStartupUpdatePlan`) is refused when it sits on the wrong side of the last fill **or on the same grid level** (within half an increment — a slot level and its execution price differ by rounding). A same-price relocation is an adoption, not a placement, and is exempt. `_pickVirtualSlotsToActivate` applies the same predicate so the pick budget rolls to the next level down/up instead of being spent on a level the create would refuse — the filled level is never re-stamped same-side (the "rotate to bottom BUY / counter SELL" behaviour).
|
|
234
239
|
|
|
235
|
-
A fill whose order id no longer sits on an ACTIVE/PARTIAL slot is no longer dropped: `attributeUnmatchedFillToGrid` (shared by `SyncEngine._absorbUnmatchedFill` and the orphan-credit sweep in `dexbot_fill_runtime.ts`) records the **pivot + owed crawl from the real execution price**. It resolves the slot two ways — the chain id when a virtualised slot still owns it, otherwise the **genesis ladder level of the execution price** (the common case, because `virtualizeOrder` NULLS `orderId`; this is what the incident actually hit). A crawl is only owed by a HOLE, and a price outside the frozen rail is refused so a shelf order cannot arm the pivot. Without this the pivot stays on a stale, higher fill and the direction rule then permits re-buys above the market.
|
|
240
|
+
A fill whose order id no longer sits on an ACTIVE/PARTIAL slot is no longer dropped: `attributeUnmatchedFillToGrid` (shared by `SyncEngine._absorbUnmatchedFill` and the orphan-credit sweep in `dexbot_fill_runtime.ts`) records the **pivot + owed crawl from the real execution price**. It resolves the slot two ways — the chain id when a virtualised slot still owns it, otherwise the **genesis ladder level of the execution price** (the common case, because `virtualizeOrder` NULLS `orderId`; this is what the incident actually hit). A crawl is only owed by a HOLE, and a price outside the frozen rail is refused so a shelf order cannot arm the pivot. Without this the pivot stays on a stale, higher fill and the direction rule then permits re-buys above the market. The sweep is **replay-safe**: the processed-fill ledger is the authority, so attribution runs only when the accounting is not a `duplicate`. A replay already armed the pivot/crawl on its first delivery; re-attributing it re-records an old execution price (live: a 12-minute-old replay re-recorded four crawls and moved the boundary 165 → 169).
|
|
236
241
|
|
|
237
242
|
**The dark level it leaves behind is repaired too.** A slot virtualised while its order stayed live on chain reads as *not active* (`_countActiveOnGrid` needs ACTIVE/PARTIAL) while its order still counts in `chainCount`, so the side reports a shortfall it can never repair — and the planner sees a free level that already holds an order. Three pieces close that:
|
|
238
243
|
|
|
@@ -260,12 +265,12 @@ The underlying rule is `INV-BROADCAST-004`: a capped `get_full_accounts` window
|
|
|
260
265
|
**`manager.ts` lock declaration** — canonical reference in [`developer_guide.md`](developer_guide.md#lock-ordering-for-deadlock-prevention).
|
|
261
266
|
|
|
262
267
|
```
|
|
263
|
-
Level 0:
|
|
264
|
-
Level
|
|
265
|
-
Level 4:
|
|
268
|
+
Level 0: _fillProcessingLock Level 0.5: placement region
|
|
269
|
+
Level 1: _divergenceLock Level 2: _syncLock
|
|
270
|
+
Level 3: _gridLock Level 4: _fundLock
|
|
266
271
|
```
|
|
267
272
|
|
|
268
|
-
Acquire in ascending level order only. AsyncLock is re-entrant (nested `acquire()` run directly, not queued).
|
|
273
|
+
Acquire in ascending level order only. AsyncLock is re-entrant (nested `acquire()` run directly, not queued); the placement region (`modules/order/placement_lock.ts`) is NOT re-entrant — it is a FIFO lease, and nested work must reuse the holder's lease (verified with `manager.isPlacementLeaseCurrent()`).
|
|
269
274
|
|
|
270
275
|
### Historical Correction (1.4.6)
|
|
271
276
|
|
package/docs/LOGGING.md
CHANGED
|
@@ -252,7 +252,7 @@ Prefix tags used in log messages to help operators identify event types. To find
|
|
|
252
252
|
| `[STRATEGY]` | `order/strategy.ts` | Fee event cache and strategy decisions |
|
|
253
253
|
| `[RECONCILE]` | `order/utils/validate.ts` | Grid reconciliation ([GRID_RECONCILE.md](GRID_RECONCILE.md)) |
|
|
254
254
|
| `[RMS]` | `order/grid.ts`, `dexbot_maintenance_runtime.ts` | RMS structural-divergence checks: per-side `metric vs threshold → TRIGGER-RESYNC/no trigger` at `debug` (plus a `checks disabled` variant when the threshold is 0), reset line with threshold + breaching sides at `info` |
|
|
255
|
-
| `[GAP-EVAC]` | `order/utils/validate.ts`, `order/manager.ts`, `dexbot_startup_runtime.ts`, `dexbot_state_recovery.ts` | Stuck in-band (gap-band) order streak warnings, cancel-only evacuation teeth, persisted-streak restore counts |
|
|
255
|
+
| `[GAP-EVAC]` | `order/utils/validate.ts`, `order/manager.ts`, `dexbot_startup_runtime.ts`, `dexbot_state_recovery.ts`, `dexbot_maintenance_runtime.ts` | Stuck in-band (gap-band) order streak warnings, cancel-only evacuation teeth, persisted-streak restore counts, and the committed-geometry stranding sweep (post-commit + maintenance tick) |
|
|
256
256
|
| `[GRID-TYPE-CORRECT]` | `order/grid.ts` | One-time backfill retype of legacy empty slots to rail-typed holes on load |
|
|
257
257
|
| `[LAST-FILL-GUARD]` | `dexbot_cow_runtime.ts` | Last-fill-guard blocks plus gap-evacuation bypass allows/stale-stamp downgrades |
|
|
258
258
|
| `[RESTAMP-BUY]` | `dexbot_cow_runtime.ts` | BUY emitted at/above the last filled BUY pivot (guard bypass / unjudgeable pivot), counted not blocked (P2) |
|
|
@@ -313,11 +313,35 @@ re-plan retries). Kept separate from the generic `refusedCommitCount`.
|
|
|
313
313
|
| `[SPREAD] No fundable side, but a rotation-only repair is available on <side>; using resting inventory (no new commitment).` | Both sides read zero free (or side selection returned no fundable side), but a promoted gap slot can be filled by rotating a resting order. Cross-asset side selection was not overridden when the market price itself was unavailable |
|
|
314
314
|
| `[LAST-FILL-GUARD] Skipping <type> UPDATE for <src> -> <dst>: stale gap-evacuation (...) — next plan re-evaluates` | A plan-build gap-evacuation stamp no longer matches the LIVE boundary/gap (or the live probe is not violation-reducing). The op is dropped instead of falling through to the direction-only last-fill guard; the next plan/reconcile owns the move |
|
|
315
315
|
| `[LAST-FILL-GUARD] Final gate dropping stale gap-evacuation ...` | The final pre-broadcast pivot gate also refuses a stamp whose live geometry went stale after op-building, even when the destination price itself is direction-legal |
|
|
316
|
+
| `[LAST-FILL-GUARD] Stamped evacuation for <src> -> <dst> is stale under live boundary B/gap G, but the live rails already conflict (maxBuy=X > minSell-gap-1=Y) — the retreat this batch commits is forced, so the evacuation proceeds` | Forced-retreat allowance (`liveRailsConflictWithGap`). The stale-stamp refusal assumes the live boundary survives the commit; when the live rails already overlap, NO boundary can host both rails with the gap band, so the batch's retreat is the only consistent geometry and refusing the evacuation is what permanently strands the order in the band (spread collapse). The same allowance is applied by the build loop and by the final pivot gate |
|
|
317
|
+
| `[LAST-FILL-GUARD] Final gate keeping gap-evacuation ... despite live boundary B/gap G: the live rails already conflict ...` | The final-gate mirror of the forced-retreat allowance above (keeping the op is violation-reducing, not a guard bypass) |
|
|
318
|
+
| `[GAP-EVAC] N live order(s) stranded inside the spread band (boundary B, gap G): <slot(side), ...> — scheduling a fill-less re-plan to evacuate (<context>)` | Post-commit / maintenance sweep: live rail orders are resting inside the committed band, so the real top-of-book spread is narrower than the configured one. A fill-less COW re-plan runs against the committed geometry (where the evacuation stamp validates) — plans are fill-driven, so a quiet market would otherwise never heal. Rate-limited (60s) per bot; repeats inside the window log at `debug` |
|
|
319
|
+
| `[GAP-EVAC] ... repair cooldown active (Ns left); teeth queued=N` | The sweep found the same stranding again inside the repair cooldown window; the cancel-only teeth remain the last resort |
|
|
320
|
+
| `[COW] Holding N gap-evacuation cancel(s) back from the pre-batch drain: this batch rotates the same chain order (the rotation IS the evacuation)` | A cancel-only evacuation queued for a chain order the current batch rotates. Draining it first would cancel the batch's own rotation source and the UPDATE batch would be rejected pre-broadcast (`target(s) missing on chain` → structural resync). The entry is restored after the drain and dropped as stale once the slot no longer owns the order |
|
|
321
|
+
| `[GAP-EVAC] Spread band clean: cleared N resolved in-band streak(s)` | The sweep found no live order inside the committed band and dropped N resolved ledger entries, so a future return to the band re-arms instead of inheriting a stale cycle count. Logged only when something was actually cleared |
|
|
322
|
+
| `[GAP-EVAC] ... unknown geometry` (silent) | With no committed boundary (or no known band width) the sweep returns without touching the ledger or the pipeline: absent geometry is not health, it is unknown |
|
|
323
|
+
| `[GAP-EVAC] ... no fill-less re-plan available on this path; teeth queued=N` | The stranding is real but this bot shape cannot run a fill-less rebalance (no `performSafeRebalance`/`updateOrdersOnChainBatch`); the cancel-only teeth own the repair |
|
|
324
|
+
| `[BOUNDARY] restore boundary=B sellStart=S gapSlots=G spreadCount=N\|? activeBuy=... activeSell=...` | Boundary restore telemetry. `gapSlots` is the resolved geometry (`_gapSlots ?? config-derived`) — never `0` for an unpublished band — and `spreadCount=?` means the grid has not been loaded/created yet, so no spread metric exists. A literal `gapSlots=0 sellStart=B+1` now always means a genuinely 0-wide band, not an initialization window |
|
|
316
325
|
| `[SPREAD-CORRECTION] <side> rail is OVERSUBSCRIBED: virtual reservation X exceeds chain free Y by Z — no correction can be funded until the rail is re-sized ...` | `availableFund` is `free − virtual`, so an oversubscribed rail reads 0 while the side still holds inventory. The watchdog escalates this on its short fund window |
|
|
317
326
|
| `[SPREAD-STALE] Spread out of tolerance for Nmin ... (outOfSpread=..., stall=...)` | Persistence warn with the last stall cause; the clock survives single in-tolerance readings and the terminal cases escalate early: `stall=last-fill-blocked` (sell window behind the last fill) on the 10-minute geometry window, a rail oversubscription on the 5-minute fund window |
|
|
318
327
|
| `[SPREAD-STALE] ... (sell window behind the last fill — re-centering)` | The `last-fill-blocked` geometry bind is escalating; placement cannot repair it, a re-center can |
|
|
319
328
|
| `[SPREAD-STALE] Still out of tolerance Nmin after re-centering on fill pivot P: the pivot is outside this generation's placeable side, so a re-center cannot clear it. A fill ... or pivot expiry restores the side; no further re-center will be requested for this pivot.` | Churn cap: a re-center does not move the pivot, so the same fill gets ONE re-center (`_lastGeometryResyncPivotKey`). Logged once per pivot; the same-side rail stays blocked until a fill or the read-time 24h pivot TTL, by design |
|
|
320
329
|
| `[BOUNDARY] Re-anchored from the last fill: SELL filled at slot 153 (0.447640); boundary 146 -> 149 so the fill sits outside its own window` | The window was behind the fill and a ±1 crawl cannot catch up; the fill now anchors the geometry. The anchor is the current fill batch's extremum for that side (max SELL / min BUY), not the last-processed pivot. Refusals stay silent (cold/off-ladder/ceiling-clamped/unsafe preconditions); the fill's own PARTIAL residual at the ANCHOR slot is allow-listed, so a partial fill does not deadlock the repair |
|
|
330
|
+
| `[BOUNDARY] Re-anchor accepted as a violation-reducing repair: boundary X -> Y strands fewer placed orders than the committed geometry (placed_order_in_band)` | The strict gate refused the candidate only because older GAP-EVAC strands are still in band; the move strands STRICTLY FEWER than the committed boundary, so it is accepted. Equal counts and structural refusals (crossed book/ceiling) stay closed. The rule lives in ONE helper (`boundaryMoveReducesStrands`) shared with the rail clamp — see `[COW] Boundary rail-derived` below. The promotion-side counterpart is the in-band absorption logged as `[SPREAD-CORRECTION] N gap slot(s) available for boundary promotion` |
|
|
331
|
+
| `[PLACEMENT] Force-releasing stale placement region held by <kind> for Ns (watchdog Ms); fills/rebalances must not stay parked` | The exclusive placement region outlived `TIMING.PLACEMENT_STALE_CLEAR_MS` (generous: above the longest observed reconcile), so the lease is force-released; the holder's later `release()` becomes a no-op (token guard). This watchdog is the ONE authority that tears a hung region down — when the region also still holds a stale broadcast refcount, it clears that too (`[PLACEMENT] Also clearing the stale broadcast region ...; one teardown for one hung region`) |
|
|
332
|
+
| `[COW] Placement lease was force-released while this batch was in flight (stale-region watchdog); aborting this batch instead of broadcasting without the region.` | The owner re-verifies its own lease with `isPlacementLeaseCurrent()` before broadcasting — the mirror of the nested drain's check. Aborting is the fail-closed answer: another path may own the region by now, and broadcasting anyway re-opens the duplicate window the teardown existed to avoid |
|
|
333
|
+
| `Startup reconcile: placement lease was force-released after planning (stale-region watchdog); discarding the plan instead of broadcasting without the region.` | The reconcile's own lease-liveness gate before Phase 2 (the mirror of `[COW] Placement lease was force-released ...`). Phase 1 touched only the master, so the plan is dropped with no chain effect; the next cycle re-reads the chain after re-acquiring the region |
|
|
334
|
+
| `[PLACEMENT] Watchdog skipped: region held by <kind> for Ns, but the pipeline is still making progress (in flight Ms)` | A bot-level batch/recovery counter younger than the placement bound proves the region is live (e.g. a long create-group reconcile), so nothing is torn down. Only a counter past the same bound is treated as part of the hung region |
|
|
335
|
+
| `[COW] <stage>: exclusive placement region unavailable (timeout/shutdown); aborting this batch instead of overlapping another placement path.` | Another chain-mutating path owns the region and the bounded wait expired. The batch is aborted (never broadcast unprotected) and retried on the next cycle |
|
|
336
|
+
| `Startup reconcile: exclusive placement region unavailable (timeout/shutdown); deferring to the next sync cycle.` | The reconcile could not serialize with a live placement path; it returns without mutating anything |
|
|
337
|
+
| `[DUST] Deferring dust cancels: exclusive placement region busy or shutting down` | Dust cancels mutate the chain, so they need the region. Deferral (not a bypass) is the contract; the next fill batch / maintenance tick retries |
|
|
338
|
+
| `[RESIDUAL] Deferring N residual cancel(s): exclusive placement region busy or shutting down` | Sub-dust residual cancels after a full fill are deferred to the next sync/reconcile instead of racing a live batch |
|
|
339
|
+
| `[MAINT] Deferring price-correction drain: exclusive placement region busy or shutting down` / `[FILL-QUEUE] Deferring price-correction drain: ...` / `[COW] Draining N pending correction(s) before batch` | The correction drain runs inside the region (nested callers pass the holder's lease, verified with `isPlacementLeaseCurrent`). A deferred drain keeps its queued entries for the next tick; the fill consumer and the maintenance tick report the same deferral under their own tags |
|
|
340
|
+
| `[BROADCAST] Stale-flag watchdog skipped: placement region still held by <kind> for Ns` | The refcount watchdog refuses to clear a broadcast region a live placement lease still owns — the mirror rule of the placement watchdog's pipeline-progress check |
|
|
341
|
+
| `[REBUILD] Structural rebuild started (<reason>); fill-driven rebalances deferred` | A trigger-file/startup rebuild owns placement and update-first repair; fill-driven COW rebalances defer instead of re-planning against the order-id-less master |
|
|
342
|
+
| `[SAFE-REBALANCE] Deferred: placement region held by <kind>; rebalance retries after it ends` | A fill-driven rebalance found the region held (reconcile/drain/dust/cancel/swap). The owed fill crawls stay recorded and the region-end hook reschedules the no-fill rebalance |
|
|
343
|
+
| `[FILL-DEDUP] Fill <key>: N/M base units already absorbed by sync pass 1; the transition will not shrink them again` | The open-orders pass resized the slot from the same post-fill snapshot. The explicit applied amount (never the legacy boolean) drives the remainder, so the trade's pays/receives are still booked while the size is released exactly once |
|
|
344
|
+
| `[COW] Pending-broadcast entry <fp> is Ns old but no sync has completed since it was recorded; keeping it (fail-closed) until a caught-up read` / `[COW] Dropped expired pending-broadcast entry ...` | The single pending-broadcast liveness authority (`getLivePendingBroadcasts`): TTL, or TTL + a successful sync after the record, or the 2× hard cap. Every reader (COW guard, reconcile CREATE, reconcile surplus sweep, crossing candidates) obeys it |
|
|
321
345
|
| `[FILL-PIVOT] pivot=... buy=... sell=... n=N src=full/direct` (+ `edge-skipped=K` when present) | One line per fill batch: the LAST-FILL-GUARD pivot after the batch. `edge-skipped=K` means K edge-pinned insurance fills were ignored — they never arm the pivot, so they can never drive the re-anchor |
|
|
322
346
|
| `[FILL-PIVOT] K edge-pinned fill(s) ignored — the guard pivot and the boundary anchor stay on the last window trade` | Edge-only batch: nothing armed, which is the point |
|
|
323
347
|
| `[LAST-FILL-GUARD] Re-armed on the new generation: 0.447640(SELL) from the last real fill (age Ns) — the guard survives a rebuild by design` | A grid rebuild dropped the generation-bound pivot and re-armed it explicitly from the bot-scoped last-market-fill row (`rearmLastFillPivotFromMarketFill`). Provenance stays `fill`; the timestamp is the original fill's, taken from the SAME row that supplied the price (a book-seeded live pivot never lends its clock) |
|
|
@@ -4,23 +4,27 @@
|
|
|
4
4
|
> misbehaving (Part I) and the preserved **incident & fix ledger** (Part II), formerly
|
|
5
5
|
> `CONSOLIDATED_ORPHAN_FIX_SUMMARY.md`.
|
|
6
6
|
>
|
|
7
|
-
> **Sources:** `CHANGELOG.md` v1.0.0 → v1.
|
|
8
|
-
> `docs/COW_INVARIANTS.md
|
|
7
|
+
> **Sources:** `CHANGELOG.md` v1.0.0 → v1.7.0 (plus the unreleased post-1.7.0 arc);
|
|
8
|
+
> `docs/GRID_PRICE_INVARIANT.md`; `docs/COW_INVARIANTS.md`; `docs/ORDER_LIFECYCLE.md`.
|
|
9
|
+
> Generic terms only — no live markets, accounts, or order ids.
|
|
9
10
|
>
|
|
10
|
-
> **Scope / snapshot:** v1.0.0 → v1.
|
|
11
|
-
>
|
|
11
|
+
> **Scope / snapshot:** v1.0.0 → v1.7.0 (`5644aa3a`, 2026-11-02), extended through the
|
|
12
|
+
> unreleased post-1.7.0 work at `84b935d9` (2026-11-04). §9 is the v1.6.4 → v1.6.17
|
|
13
|
+
> addendum; §9.10 closes the v1.6.18 review pass and §9.11 carries v1.6.18 → unreleased.
|
|
12
14
|
>
|
|
13
15
|
> **Section numbering:** Part II deliberately keeps its original `§0`–`§8` numbers because
|
|
14
16
|
> source-code comments and tests cite them (e.g. `… §2 fix #1`, `… §3 lineage`); §9 is a
|
|
15
|
-
> later addendum (v1.6.4 → v1.6.
|
|
16
|
-
> uses `R1`–`R5` so the two never collide. The closing appendix
|
|
17
|
+
> later addendum (v1.6.4 → v1.6.17, with §9.10/§9.11 extending it) that adds to, and does not
|
|
18
|
+
> renumber, that range. Part I uses `R1`–`R5` so the two never collide. The closing appendix
|
|
19
|
+
> is unnumbered (`A`).
|
|
17
20
|
>
|
|
18
21
|
> **How to read:** start with **Part I** for the explanation; drop into **Part II** when you need
|
|
19
22
|
> the per-incident evidence, commit hashes, and verification targets.
|
|
20
23
|
>
|
|
21
24
|
> **See also:** [GRID_PRICE_INVARIANT.md](GRID_PRICE_INVARIANT.md) ·
|
|
22
25
|
> [COW_INVARIANTS.md](COW_INVARIANTS.md) · [GRID_RECONCILE.md](GRID_RECONCILE.md) ·
|
|
23
|
-
> [
|
|
26
|
+
> [ORDER_LIFECYCLE.md](ORDER_LIFECYCLE.md) · [EVOLUTION.md](EVOLUTION.md) ·
|
|
27
|
+
> [CHANGELOG.md](../CHANGELOG.md).
|
|
24
28
|
|
|
25
29
|
---
|
|
26
30
|
|
|
@@ -44,7 +48,8 @@
|
|
|
44
48
|
- 7. Consolidated verification — grid-correction check as regression gate
|
|
45
49
|
- 8. Provenance & supersession record
|
|
46
50
|
- 9. COW Broadcast, Resync & Geometry Safety (v1.6.4 → v1.6.17) — addendum
|
|
47
|
-
- 9.10 Review pass over v1.6.12 → v1.6.17 (findings P1–P5)
|
|
51
|
+
- 9.10 Review pass over v1.6.12 → v1.6.17 (findings P1–P5), released in v1.6.18
|
|
52
|
+
- 9.11 v1.6.18 → unreleased: fill identity, absence and the geometry budget
|
|
48
53
|
- A. Grid-Price Invariant — former failure trace (appendix)
|
|
49
54
|
|
|
50
55
|
---
|
|
@@ -77,6 +82,9 @@ local state.
|
|
|
77
82
|
| **Silent hangs** | Retries that depended on a future event that never came | Level-triggered scheduling with time-based watchdogs |
|
|
78
83
|
| **Stale queues** | Queued price corrections replayed onto a resynced grid, reverting good placements | Pre-broadcast validation against the live slot |
|
|
79
84
|
| **Shelf / edge orders** | Non-grid orders poisoning every "orders on grid" counter (issue #27) | Slot-N gating across all counters |
|
|
85
|
+
| **Stale window after a fill** | A fill is queued but not yet absorbed; the master still counts the filled slot ACTIVE, so a window top-up re-creates the expensive end of the rail and re-buys into a stale pivot | `hasUnsettledFills` defers the targeted reconcile and the window top-up (1:1 repairs still run) + attribution of the real execution price (§9.11.1) |
|
|
86
|
+
| **Fill identity by chain id** | `virtualizeOrder` nulls `orderId`, so a real fill matched no ACTIVE slot and was dropped *after* its accounting was booked; pivot and owed crawl stayed on a stale, higher price | `attributeUnmatchedFillToGrid` falls back to the genesis level of the execution price, refuses prices outside the frozen rail, and owes a crawl only for a hole the grid owns (§9.11.1) |
|
|
87
|
+
| **Chain-existence bookkeeping** | Seven structures with seven clocks; four read sites consulted four subsets and could give four answers for one order | `modules/order/lifecycle.ts`: one record per slot, `unknown` is a real phase, every absence decision asks the same time-bounded `isInFlight()` (§9.11.3) |
|
|
80
88
|
|
|
81
89
|
Families **not** detailed in Part II: silent hangs and stale correction queues are covered in
|
|
82
90
|
`CHANGELOG.md` (v1.6.1 and v1.6.3 respectively); shelf/edge orders in issue #27 and the v1.6.x
|
|
@@ -100,13 +108,16 @@ notes; the grid-price invariant has its own doc — [GRID_PRICE_INVARIANT.md](GR
|
|
|
100
108
|
the runtime guard, the spread-correction planner, the rail clamp and the offline checker.
|
|
101
109
|
Every copy drifted on its own schedule: the planner used the crossing floor, the checker
|
|
102
110
|
kept the old band, the rail clamp kept the old slot ids (§9.7, §9.10). The fix that
|
|
103
|
-
sticks is one exported predicate and callers that re-implement nothing.
|
|
111
|
+
sticks is one exported predicate and callers that re-implement nothing. The same shape
|
|
112
|
+
produced the existence bugs: seven local structures were seven policies for "does this
|
|
113
|
+
order exist?", and the first disagreement was the incident (§9.11.3).
|
|
104
114
|
|
|
105
115
|
### R4. What actually fixed it
|
|
106
116
|
|
|
107
117
|
- **Genesis-frozen price-slot determinism** (v1.4.25+): `slot-N` → exact ladder price
|
|
108
|
-
becomes the single source of truth
|
|
109
|
-
|
|
118
|
+
becomes the single source of truth. Grid-vs-grid identity is now exact; a CHAIN price is
|
|
119
|
+
compared by which ladder level it maps to (`pricesOnSameLevel`, bounded by asset precision),
|
|
120
|
+
never by a tuned band. Killed a whole family of orphan/duplicate bugs.
|
|
110
121
|
- **Single ownership:** one writer for the boundary, one owner for the adapter/market
|
|
111
122
|
config, one authoritative set of counters.
|
|
112
123
|
- **Global last-fill guard + final pre-broadcast gate:** one pivot rule, enforced at the
|
|
@@ -124,6 +135,16 @@ notes; the grid-price invariant has its own doc — [GRID_PRICE_INVARIANT.md](GR
|
|
|
124
135
|
each path carrying its own idea of "stranded".
|
|
125
136
|
- **Live identity over local keys (§9.10, `531edeb4`):** every rail/geometry decision maps the
|
|
126
137
|
order's LIVE chain price onto the ladder; a slot id is a hint, never the fact.
|
|
138
|
+
- **An id is not an identity (§9.11.1, `8705381b`):** a fill whose chain id was nulled by
|
|
139
|
+
virtualisation is resolved to the genesis level of its execution price, so attribution
|
|
140
|
+
cannot go blind exactly when the order is most interesting.
|
|
141
|
+
- **An unabsorbed fill blocks the geometry (§9.11.1, `a93f7eef`):** while fills are queued,
|
|
142
|
+
draining, in flight or owed a crawl, the targeted reconcile and the window top-up defer.
|
|
143
|
+
- **Repairs must reduce the violation (§9.11.3, `c493a604`):** a boundary move is accepted
|
|
144
|
+
only when it strands *strictly fewer* placed orders than the committed value, measured by
|
|
145
|
+
the validator's own count — never a second predicate.
|
|
146
|
+
- **One existence record (§9.11.3, `826d84a3`/`84b935d9`):** seven per-site structures
|
|
147
|
+
collapsed into `lifecycle.ts`; `unknown` is a phase, not an absence.
|
|
127
148
|
- **Observability instead of a veto (§9.10/P4):** where a hard guard would re-create a
|
|
128
149
|
deadlock (a counter order crossing the floor by construction), log it once and let the
|
|
129
150
|
chain price it as a taker fill.
|
|
@@ -157,6 +178,15 @@ The winning strategy was **removing writers and sources of truth**, not adding h
|
|
|
157
178
|
repair that can cite one stray order is not a repair.
|
|
158
179
|
11. **When a veto would deadlock, ship the observation.** A taker-fee risk earns a log line,
|
|
159
180
|
not a rule that re-creates the deadlock the previous fix removed (§9.10/P4).
|
|
181
|
+
12. **An id is not an identity.** Virtualising an order clears its chain id, so anything that
|
|
182
|
+
matches fills, occupancy or rails by id alone goes blind exactly when the order is most
|
|
183
|
+
interesting. Match the ladder level of the live price; keep the id as a hint (§9.11.1).
|
|
184
|
+
13. **A fill the geometry has not absorbed must block the geometry.** Deferring a repair for
|
|
185
|
+
one cycle is cheaper than a top-up built from a stale master, because a repair path that
|
|
186
|
+
runs on un-absorbed state is a way to re-create that state (§9.11.1).
|
|
187
|
+
14. **Absence needs one definition, one clock and one owner.** Seven local counters are seven
|
|
188
|
+
policies; if two sites can disagree about whether an order exists, the disagreement is
|
|
189
|
+
the bug, not either answer (§9.11.3).
|
|
160
190
|
|
|
161
191
|
---
|
|
162
192
|
|
|
@@ -183,7 +213,7 @@ plans, `LANDED`/`REVERTED`/`SUPERSEDED` statuses, commit hashes, and verificatio
|
|
|
183
213
|
|
|
184
214
|
**Grid terms:** `boundary` = discrete sell/buy split index; `gap band` = `SPREAD/virtual` slots around boundary; `sellStartIdx = splitIdx - floor(gap/2) - 1`; center-priced ladder `createOrderGrid` = `startPrice × step^(k+½)` (`modules/order/grid.ts`); `MarketAnchor` = fill-derived `lastFillPrice/maxFilledSellPrice/minFilledBuyPrice/lastFillSide`; tolerance helpers `calculatePriceTolerance = (1/satsA+1/satsB)×price` (`modules/order/utils/math.ts`), `GRID_LIMITS.PRICE_TOLERANCE_MAX_PERCENT 0.01` (`modules/constants.ts`).
|
|
185
215
|
|
|
186
|
-
**Validation tool:** `analysis/grid_correction_check.ts` (
|
|
216
|
+
**Validation tool:** `analysis/grid_correction_check.ts` (offline Node tool — it imports the runtime's own `isLastFillGuardBlocked`, so the verdict cannot drift; its Kibana query mirrors `analysis/trade_profitability.ts` (`https://kibana.bitshares.dev`, `bitshares-*`, `operation_type:4` `fill_order` only; `buildFillQuery`, paginated `search_after`, `_source` `block_data` fields). **Invariants:** `sell→sell` with no `buy` between must rise (`curr > prev`, **equal is a violation** — a re-place at the just-filled level), `buy→buy` must fall (`curr < prev`); `per-order` aggregated default (weighted avg collapses multi-fill `1.7.x`), `--per-fill` raw. Resolution via `bot_key_utils.ts` `loadBotMeta/computeBotKey`. Reports violations, `%`, daily histogram, time/range, table with price/Δ%/times/orderIds; `--json/--csv` export; exit `0` pass / `2` fail. Strict sat-level is ground truth — the verdict is **direction only** (equal price is a violation); `--increment` is **informational only** (the band the removed `--tolerance` flag used to express is gone, §9.10/P5). Used as regression gate below; pre-fix per-account baselines are intentionally omitted here — §7 states the targets (`0` violations).
|
|
187
217
|
|
|
188
218
|
---
|
|
189
219
|
|
|
@@ -538,15 +568,17 @@ Original doc regression excerpts preserved: GAP P6 A/B/C; LADDER §2.8 checks (1
|
|
|
538
568
|
- [x] No other doc/index lists the four (verified 2026-08-31); `analysis/README.md` documents the `grid_correction_check` tool.
|
|
539
569
|
- [x] Stale `*.md` filename references in code comments/tests updated to the consolidated doc (9 module files, 6 test files), then re-pointed to this hub when it absorbed the summary.
|
|
540
570
|
|
|
541
|
-
*Generated 2026-08-31 from the 4 test-only docs at their HEAD contents; statuses corrected against `test` HEAD `a54863ca` the same day, then refreshed against `1382f267` (2026-09-13), extended through v1.6.14 at `f6c0df32` (2026-10-12), through v1.6.17 at `78ccbb8e`,
|
|
571
|
+
*Generated 2026-08-31 from the 4 test-only docs at their HEAD contents; statuses corrected against `test` HEAD `a54863ca` the same day, then refreshed against `1382f267` (2026-09-13), extended through v1.6.14 at `f6c0df32` (2026-10-12), through v1.6.17 at `78ccbb8e`, through the v1.6.18 review pass (`§9.10`: `f4d6da8c`, `a2bf6ba5`, `56481b16`, `3d16f9e4`), and through v1.7.0 + the unreleased post-1.7.0 arc (`§9.11`, HEAD `84b935d9`). Landed/reverted/superseded annotations carry commit hashes where known; **line references in §§1-4 were re-verified at `a54863ca` and are subject to drift** — treat them as pointers, not assertions. The GRID-PRICE-INVARIANT guard is **BLOCKING** at all six emission sites (CREATE / UPDATE / CREATE-FALLBACK in `modules/dexbot_cow_runtime.ts`, RECONCILE-CREATE / RECONCILE-UPDATE / STARTUP-CREATE in `modules/order/grid_reconcile_internal.ts`): an off-grid emission is skipped, not merely counted, so a `violated>0` line means an emission was *prevented* rather than a writer merely *pinned*. Counters are still reported (`site=`, `checked=`, `violated=`, `unchecked=`) for observability — see `docs/GRID_PRICE_INVARIANT.md`.*
|
|
542
572
|
|
|
543
573
|
---
|
|
544
574
|
|
|
545
575
|
### 9. COW Broadcast, Resync & Geometry Safety (v1.6.4 → v1.6.17)
|
|
546
576
|
|
|
547
|
-
> **Source:** `CHANGELOG.md` v1.6.4 → v1.6.17 · **Snapshot:** `78ccbb8e` (v1.6.17 tag)
|
|
577
|
+
> **Source:** `CHANGELOG.md` v1.6.4 → v1.6.17 · **Snapshot:** `78ccbb8e` (v1.6.17 tag);
|
|
578
|
+
> §9.10 and §9.11 extend past it — §9.10 released in v1.6.18, §9.11 unreleased to HEAD
|
|
579
|
+
> `84b935d9`.
|
|
548
580
|
> Extends the §§1–8 ledger; the `§0`–`§8` numbers are unchanged (code and tests cite them).
|
|
549
|
-
>
|
|
581
|
+
> The ledger labels each row with its release; `unreleased` means "in no tag yet".
|
|
550
582
|
|
|
551
583
|
Sections 1–4 end with the prevention sweeps reverted: a cancel sweep on ambiguous evidence
|
|
552
584
|
proved worse than the orphan it targets. The post-1.6.3 arc is the constructive inverse —
|
|
@@ -636,7 +668,7 @@ fail-closed behind `_assertGenesisInvariant`. This is R4's "removing writers and
|
|
|
636
668
|
truth" carried to its end: the no-ladder state that Appendix A's `slot.price` corruption
|
|
637
669
|
required is no longer reachable at runtime.
|
|
638
670
|
|
|
639
|
-
#### 9.5 Ledger (order-engine commits, v1.6.4 → v1.
|
|
671
|
+
#### 9.5 Ledger (order-engine commits, v1.6.4 → v1.7.0 + unreleased)
|
|
640
672
|
|
|
641
673
|
| Commit | Release | Change |
|
|
642
674
|
|---|---|---|
|
|
@@ -672,11 +704,27 @@ required is no longer reachable at runtime.
|
|
|
672
704
|
| `62312e60` | 1.6.17 | Plan only placeable tightening slots; name the stall causes |
|
|
673
705
|
| `276cd504` | 1.6.17 | Fund from chain truth and re-anchor the boundary from the fill (§9.9) |
|
|
674
706
|
| `2e618197` | 1.6.17 | Consume only the owed fill crawls a derivation applied (§9.8) |
|
|
675
|
-
| `d99b8685` |
|
|
676
|
-
| `531edeb4` |
|
|
677
|
-
| `f4d6da8c` |
|
|
678
|
-
| `a2bf6ba5` |
|
|
679
|
-
| `56481b16` |
|
|
707
|
+
| `d99b8685` | 1.6.18 | Harden partial-fill re-anchor and recovery (residual allow-list, tolerant load) |
|
|
708
|
+
| `531edeb4` | 1.6.18 | Keep the boundary on the placed rails; use live fill prices for geometry |
|
|
709
|
+
| `f4d6da8c` | 1.6.18 | Edge-pinned fills never arm the pivot or the anchor (§9.10/P1) |
|
|
710
|
+
| `a2bf6ba5` | 1.6.18 | Guard arity, explicit pivot re-arm, stray-aware rail clamp, crossing observation (§9.10/P2-P4) |
|
|
711
|
+
| `56481b16` | 1.6.18 | Offline checker judges by the same direction-only predicate (§9.10/P5) |
|
|
712
|
+
| `ddc4b4a3` | 1.6.18 | Count last-fill-excluded candidates per slot, not per pool visit |
|
|
713
|
+
| `a93f7eef` | 1.7.0 | Stale-grid re-buys: unsettled-fill gate, attribution, same-level refusal (§9.11.1) |
|
|
714
|
+
| `8705381b` | 1.7.0 | Orphan-fill attribution by price; dark-level revive (§9.11.1) |
|
|
715
|
+
| `8aff50bb` | 1.7.0 | Anchor-aware residual allow-list, pivot clock provenance, read-time TTL (§9.11.2) |
|
|
716
|
+
| `c493a604` | unreleased | Stranded-boundary repair without re-arming on replays (§9.11.3) |
|
|
717
|
+
| `ed52f9ef` | unreleased | Exclusive placement region; no create-only re-plan after a rebuild (§9.11.3) |
|
|
718
|
+
| `970fedfd` | unreleased | Remaining placement races; fill-accounting double-apply |
|
|
719
|
+
| `ad081e3d` | unreleased | Placement lifecycle + fill-marker idempotency |
|
|
720
|
+
| `7a3571a4` | unreleased | Follow-up gaps: one teardown, one lease check, one pending-broadcast policy |
|
|
721
|
+
| `94223d19` | unreleased | Re-verify the placement lease before a region holder broadcasts |
|
|
722
|
+
| `8bab9881` | unreleased | Unknown gap geometry is not gap 0 |
|
|
723
|
+
| `4d5e3d81` | unreleased | Forced retreats evacuate, then re-plan the committed band |
|
|
724
|
+
| `7618f299` | unreleased | Cancel teeth may not destroy the rotation that evacuates |
|
|
725
|
+
| `826d84a3` | unreleased | Chain existence unified behind the lifecycle registry (§9.11.3) |
|
|
726
|
+
| `7252d7d3` | unreleased | Broadcast uncertainty as a typed three-valued outcome (§9.11.3) |
|
|
727
|
+
| `84b935d9` | unreleased | One writer per life-cycle clock, one guard per absence decision (§9.11.3) |
|
|
680
728
|
|
|
681
729
|
#### 9.6 Invariants added
|
|
682
730
|
|
|
@@ -698,8 +746,18 @@ required is no longer reachable at runtime.
|
|
|
698
746
|
bounded by the same 24h TTL (enforced at restore and at read time, so a stale row can neither
|
|
699
747
|
arm the guard nor be re-armed), and a re-center is requested at most once per pivot because a
|
|
700
748
|
re-center cannot move it (§9.10/P2).
|
|
701
|
-
|
|
702
|
-
|
|
749
|
+
- **An unabsorbed fill blocks the geometry:** while fills are queued, draining, in flight or
|
|
750
|
+
owed a crawl, the targeted reconcile and the window top-up defer (1:1 repairs still run),
|
|
751
|
+
and a fill is attributed by the ladder level of its execution price, not by a chain id that
|
|
752
|
+
virtualisation has already nulled (§9.11.1).
|
|
753
|
+
- **A repair must reduce the violation:** a boundary move is accepted only when it strands
|
|
754
|
+
strictly fewer placed orders than the committed value, counted by `validatePersistedBoundary`
|
|
755
|
+
itself; an equal count is refused (§9.11.3).
|
|
756
|
+
- **Existence is a record, not a lookup:** one per-slot lifecycle record owns the phase, the
|
|
757
|
+
evidence rank and every clock; `unknown` is a phase that never reads as "gone", and every
|
|
758
|
+
absence decision asks the same time-bounded guard (§9.11.3).
|
|
759
|
+
|
|
760
|
+
#### 9.7 The guard was a band, not an economic rule (`8e5e1a3c`)
|
|
703
761
|
|
|
704
762
|
**Observed (2026-10-24/25, a live market-pair bot).** A dump swept the grid and the
|
|
705
763
|
market then held. Spread correction planned, the broadcast guard dropped the creates,
|
|
@@ -820,7 +878,7 @@ must stay fund-driven from the side pool: a SELL fill returns quote while the
|
|
|
820
878
|
same-side repair needs base, so no fill can ever fund its own side, and pairing
|
|
821
879
|
would hide that asymmetry instead of resolving it.
|
|
822
880
|
|
|
823
|
-
#### 9.10 Review pass over v1.6.12 → v1.6.17 (
|
|
881
|
+
#### 9.10 Review pass over v1.6.12 → v1.6.17 (released in v1.6.18)
|
|
824
882
|
|
|
825
883
|
A code review of the whole 1.6.12→1.6.17 order-engine arc (90 commits) against
|
|
826
884
|
the three stated goals — buy-low/sell-high counter orders, never trading through
|
|
@@ -844,8 +902,99 @@ extremum extracted as `resolveFillAnchorIndex` (pure, batch-only).
|
|
|
844
902
|
|
|
845
903
|
**Commits:** `f4d6da8c` (P1) · `a2bf6ba5` (P2–P4 plus the arity and the anchor
|
|
846
904
|
extraction) · `56481b16` (P5) · `3d16f9e4` (this document, the doc-level invariants
|
|
847
|
-
and the log rows it cites). **
|
|
848
|
-
|
|
905
|
+
and the log rows it cites). **Released in v1.6.18:** the review pass itself carried no
|
|
906
|
+
version bump, but every commit above is an ancestor of the `v1.6.18` tag (2026-11-01).
|
|
907
|
+
|
|
908
|
+
#### 9.11 v1.6.18 → unreleased: fill identity, absence and the geometry budget
|
|
909
|
+
|
|
910
|
+
The post-1.6.17 work stopped asking "which guard is missing?" and started asking
|
|
911
|
+
"which *identity* is the code using?". Three answers landed: a fill is identified
|
|
912
|
+
by its ladder level, existence by one lifecycle record, and a repair by the
|
|
913
|
+
amount of violation it removes.
|
|
914
|
+
|
|
915
|
+
##### 9.11.1 A fill the grid could not name (v1.7.0, `a93f7eef`/`8705381b`)
|
|
916
|
+
|
|
917
|
+
**Observed (2026-11-02, a live market-pair bot).** A multi-minute stale broadcast
|
|
918
|
+
region deferred the fill queue; the post-fill targeted reconcile then derived its
|
|
919
|
+
window from a master that still held the filled slots as ACTIVE and re-created
|
|
920
|
+
same-side orders at the top of the un-crawled rail. They filled within seconds:
|
|
921
|
+
9 buy-lower violations, 23.5% of the window's BTS.
|
|
922
|
+
|
|
923
|
+
**The root cause sat upstream of the reconcile.** A fill whose order id no longer
|
|
924
|
+
sat on an ACTIVE/PARTIAL slot was dropped ("order not found in active grid")
|
|
925
|
+
*after* its accounting was booked, so neither the pivot nor the owed crawl
|
|
926
|
+
learned the real execution price and every downstream gate compared against a
|
|
927
|
+
stale, higher price. `virtualizeOrder` nulls `orderId`, so the very fills the
|
|
928
|
+
attribution was written for could never match by id.
|
|
929
|
+
|
|
930
|
+
**Four gates, one direction:**
|
|
931
|
+
|
|
932
|
+
- `hasUnsettledFills` defers the targeted reconcile and the window top-up while
|
|
933
|
+
fills are queued, draining, in flight or owed a crawl. 1:1 repairs
|
|
934
|
+
(cancelled-slot replacement, vacated-rail refill) still run, so a rail is never
|
|
935
|
+
left un-repaired.
|
|
936
|
+
- `attributeUnmatchedFillToGrid` resolves the slot by chain id when a virtualised
|
|
937
|
+
slot still owns it, else by the **genesis level of the execution price**,
|
|
938
|
+
records pivot + owed crawl, and refuses a price outside the frozen rail so a
|
|
939
|
+
shelf/manual order can never arm the geometry.
|
|
940
|
+
- `isLastFillPlacementBlocked` (direction **plus** same grid level) gates every
|
|
941
|
+
reconcile placement, and `_pickVirtualSlotsToActivate` uses the same predicate,
|
|
942
|
+
so the pick budget rolls to the next level instead of being spent on a level
|
|
943
|
+
the create would refuse.
|
|
944
|
+
- A slot virtualised while its order stayed live on chain is revived in place
|
|
945
|
+
(`reviveVirtualSlotFromChain`, level-gated, master-only), a genuinely stale id
|
|
946
|
+
on a non-placed slot is cleared without booking a fill, and CREATE occupancy is
|
|
947
|
+
keyed on the id alone (any state) rather than on `isOrderOnChain()`.
|
|
948
|
+
|
|
949
|
+
##### 9.11.2 The pivot is a row, not a price (v1.6.18/v1.7.0, `d99b8685`/`8aff50bb`)
|
|
950
|
+
|
|
951
|
+
Three defects of the same shape:
|
|
952
|
+
|
|
953
|
+
- The in-band residual allow-list that lets a fill's own PARTIAL survive a re-anchor was
|
|
954
|
+
keyed on the last-processed pivot instead of the batch **extremum**, so for every chunked
|
|
955
|
+
sweep whose furthest fill was not the pivot the re-anchor was refused as
|
|
956
|
+
`unsafe: placed_order_in_band` — the exact shape the extremum resolution exists for.
|
|
957
|
+
- A market-row price could inherit a book-seed timestamp and then re-stamp it, refreshing the
|
|
958
|
+
24h lifetime on every rebuild until the row never expired.
|
|
959
|
+
- The TTL existed only at restore, so a long-lived process kept an expired pivot vetoing one
|
|
960
|
+
side forever while the geometry watchdog claimed a fill or the TTL would restore it.
|
|
961
|
+
|
|
962
|
+
Price, side and clock now travel as one row, and the TTL is enforced where the reference
|
|
963
|
+
is **read**; an unknown age never expires.
|
|
964
|
+
|
|
965
|
+
##### 9.11.3 Absence gets one record; repairs must reduce the violation (unreleased)
|
|
966
|
+
|
|
967
|
+
Chain existence used to be answered by seven structures with seven clocks
|
|
968
|
+
(`_pendingBroadcasts`, `createUncertain`, `_committedOrderIds`,
|
|
969
|
+
`_orderIdAssignedAt`, `_placedAt`, `_chainSizeAppliedFills`,
|
|
970
|
+
`_lastUnmatchedChainOrders`). Four read sites consulted four subsets and could
|
|
971
|
+
return four different answers for the same order, so each fix closed one site and
|
|
972
|
+
the bug reappeared at the next.
|
|
973
|
+
|
|
974
|
+
`modules/order/lifecycle.ts` (`826d84a3`, `7252d7d3`, `84b935d9`) now owns one
|
|
975
|
+
record per slot: a six-phase vocabulary in which `unknown` is a real state (a
|
|
976
|
+
lost receipt is *not* an absence), a monotonic evidence rank, and every clock
|
|
977
|
+
recorded once. The legacy maps survive as live views, so no call site had to be
|
|
978
|
+
rewritten to get one truth. Three rules fall out of it: one absence guard
|
|
979
|
+
(`isInFlight()`, time-bounded), one teardown authority for a hung region
|
|
980
|
+
(`_clearStalePlacementRegion`, `7a3571a4`), and one pending-broadcast liveness
|
|
981
|
+
policy (`getLivePendingBroadcasts`).
|
|
982
|
+
|
|
983
|
+
Geometry gained the matching rule. A boundary writer may move the boundary only
|
|
984
|
+
when the move strands **strictly fewer** placed orders than the committed value
|
|
985
|
+
(`boundaryMoveReducesStrands`, counted by `validatePersistedBoundary` itself); an
|
|
986
|
+
equal count stays refused, and a replayed fill no longer re-arms the pivot
|
|
987
|
+
(`c493a604`). The same arc put every chain-mutating path inside one exclusive
|
|
988
|
+
placement region (`modules/order/placement_lock.ts`, `ed52f9ef`/`970fedfd`) after
|
|
989
|
+
a trigger-file rebuild and a fill-driven rebalance interleaved into 25
|
|
990
|
+
duplicate-price orders; made a forced gap-evacuation retreat evacuate before
|
|
991
|
+
re-planning instead of letting its cancel teeth destroy the rotation (`7618f299`,
|
|
992
|
+
`4d5e3d81`); and stopped an absent gap band from reading as a valid `0`
|
|
993
|
+
(`8bab9881`).
|
|
994
|
+
|
|
995
|
+
**Why it belongs in this document:** R3.5's "one rule, several implementations"
|
|
996
|
+
is not a style complaint. Every family above is that pattern applied to a
|
|
997
|
+
different noun — price, fill, existence.
|
|
849
998
|
|
|
850
999
|
---
|
|
851
1000
|
|
package/docs/README.md
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
|
|
3
3
|
This directory contains the comprehensive technical documentation for the DEXBot2 trading bot. It is designed to guide developers from high-level architecture down to the nuances of fund accounting and state management.
|
|
4
4
|
|
|
5
|
-
**Version context:** v1.7.
|
|
5
|
+
**Version context:** v1.7.1 (released).
|
|
6
6
|
|
|
7
7
|
---
|
|
8
8
|
|
|
@@ -131,9 +131,22 @@ This directory contains the comprehensive technical documentation for the DEXBot
|
|
|
131
131
|
### 🔒 [COW Invariants](COW_INVARIANTS.md)
|
|
132
132
|
*Stable theory contract for COW pipeline.*
|
|
133
133
|
- **Non-negotiable invariants**: Master immutability, commit atomicity, projection rules, accounting separation
|
|
134
|
-
- **Subsystem Scope**:
|
|
134
|
+
- **Subsystem Scope**: 15 `INV-*` prefix groups mapping each invariant to its runtime subsystem
|
|
135
135
|
- **Change Policy**: Required steps for an intentional invariant change — same-PR doc update, rationale, and regression tests
|
|
136
136
|
|
|
137
|
+
### 🧭 [Order Lifecycle Registry](ORDER_LIFECYCLE.md)
|
|
138
|
+
*How the engine answers one question from one record: does this order exist on chain, and how sure are we?*
|
|
139
|
+
- **The problem**: seven structures with seven clocks, four read sites consulting four different subsets of them
|
|
140
|
+
- **The model**: one record per slot, six phases, `unknown` as a real state that never reads as "gone"
|
|
141
|
+
- **The wiring**: legacy maps are live views onto the record; every mutation funnels through `_applyOrderUpdate`
|
|
142
|
+
- **Invariants**: `INV-LIFE-001…007` in [COW_INVARIANTS.md](COW_INVARIANTS.md#order-lifecycle-order_lifecyclemd)
|
|
143
|
+
|
|
144
|
+
### ⏱ [Timing Window Inventory](TIMING_INVENTORY.md)
|
|
145
|
+
*Every timing knob classified, with the self-healing ones marked as removal targets.*
|
|
146
|
+
- **The audit**: `external` / `bound` / `policy` / `self-healing`, each with a rationale
|
|
147
|
+
- **The gate**: `tests/test_timing_policy.ts` fails on an unclassified window, so the inventory can only shrink by decision
|
|
148
|
+
- **Removal targets**: the six windows that compensate for our own lost state, and the evidence that replaces each
|
|
149
|
+
|
|
137
150
|
### 📐 [Grid-Price Invariant](GRID_PRICE_INVARIANT.md)
|
|
138
151
|
*Why a slot's emitted price must equal its genesis level — and how that failed.*
|
|
139
152
|
- **The invariant**: `order.price === priceForSlot(idx, genesis)`, and why range guards cannot substitute for it
|
|
@@ -169,6 +182,14 @@ This directory contains the comprehensive technical documentation for the DEXBot
|
|
|
169
182
|
- **Practical How-Tos**: Adding features step by step, common pitfalls to avoid, and useful debugging commands.
|
|
170
183
|
- **Glossary**: Definitions of project-specific terminology (e.g., "VIRTUAL order state", "Rotation", "Pipeline Safety", "WorkingGrid", "Atomic Commit", "Dynamic Weight", "Regime Detection").
|
|
171
184
|
|
|
185
|
+
### 🌉 [Liquidity Mirror Design](LIQUIDITY_MIRROR_DESIGN.md)
|
|
186
|
+
*Design paper for a cross-venue liquidity mirror running alongside DEXBot2.*
|
|
187
|
+
- **Roles**: Chain host (TS, DEXBot2) owns identity, pool reads, and hedging; a separate venue daemon owns CEX credentials, the order book, and the rate-limit budget
|
|
188
|
+
- **Contract**: Intent-level, HMAC-authenticated NDJSON over a unix socket — desired book in, fills and positions out; no message whose retry can duplicate an order
|
|
189
|
+
- **Change surface**: Four new modules under `modules/venues/`, a `cex_mirror/` tree, and five mechanical edits; the grid, sync engine, accounting, and strategy engine are untouched
|
|
190
|
+
- **Guardrails**: Spread floor from pool fee plus impact, hedge batching trade-off, deadman cancellation, divergence breaker, and a split inventory-cap model
|
|
191
|
+
- **Rollout**: Milestones M0–M5 with acceptance gates, from protocol conformance vectors to capped live hedging
|
|
192
|
+
|
|
172
193
|
### 🔄 [Workflow](WORKFLOW.md)
|
|
173
194
|
*How we build and release.*
|
|
174
195
|
- **Branching Strategy**: Explanation of the `test` → `dev` → `main` lifecycle.
|
package/docs/WORKFLOW.md
CHANGED
|
@@ -250,8 +250,8 @@ the canonical name is preferred in scripts and docs.
|
|
|
250
250
|
| `dexbot update` | — | Update DEXBot2 from the repository and restart active bots |
|
|
251
251
|
| `dexbot export <bot>` | — | Export bot trades/settings to CSV/JSON for local analysis/ |
|
|
252
252
|
| `dexbot order` | `orders` | Analyze persisted order grids (spread, increment, funds); `--export` for HTML |
|
|
253
|
-
| `dexbot order [<bot>]` | — | Analyze only the specified bot's order grid |
|
|
254
|
-
| `dexbot credit` | — | Live summed MPA + borrowed-credit positions per asset per bot (`[<bot>]`) |
|
|
253
|
+
| `dexbot order [<bot|account>]` | — | Analyze only the specified bot's order grid — or every grid on that account (`<bot|account>`) |
|
|
254
|
+
| `dexbot credit` | — | Live summed MPA + borrowed-credit positions per asset per bot (`[<bot|account>]` — a bot name/key, or the account the bot trades from, name or `1.2.x` id) |
|
|
255
255
|
| `dexbot rich <asset...>` | — | Top holder(s) per asset via the chain asset-holders API (`--top N`, `--json`/`--csv`) |
|
|
256
256
|
| `dexbot tv <target>` | — | TradingView 1h chart for `<bot|pool|AssetA/AssetB>` over `--month N` (default 3); pool = id (133 / 1.19.133) or pool token (LP share-asset symbol) |
|
|
257
257
|
| `dexbot dw <target>` | — | Dynamic-weight research chart: same targets/flags as `tv` |
|
|
@@ -313,7 +313,13 @@ npm run pmain
|
|
|
313
313
|
|--------|---------|-------------|-----------|
|
|
314
314
|
| `npm run ptest` | Push test to origin | Checks out `test` (if not on it) and pushes local commits to `origin/test`. Does **not** touch `dev`/`main`. | Daily development; ensures `origin/test` is up-to-date |
|
|
315
315
|
| `npm run pdev` | Mirror test onto dev | Pushes `origin/test`, then `git push origin test:dev --force` and updates the local `dev` pointer. No merge commit. | When test is stable and you want `dev` to exactly equal `test` |
|
|
316
|
-
| `npm run pmain` | Mirror test onto dev and main | Pushes `origin/test`, then force-pushes `test` to both `dev` and `main` and updates local pointers. **No tagging.** | Full release where `test == dev == main` is intended. Tag manually afterward (see step 5) |
|
|
316
|
+
| `npm run pmain` | Mirror test onto dev and main | Pushes `origin/test`, then force-pushes `test` to both `dev` and `main` in one `--atomic` push and updates local pointers. **No tagging.** | Full release where `test == dev == main` is intended. Tag manually afterward (see step 5) |
|
|
317
|
+
|
|
318
|
+
All three scripts run a shared preflight before touching anything
|
|
319
|
+
(`scripts/lib/git_preflight.sh`): a read-only `git ls-remote` against `origin`.
|
|
320
|
+
An unreachable remote or a rejected SSH key now fails up front instead of
|
|
321
|
+
mid-script, and `pmain`'s `--atomic` push keeps `dev`/`main` together — a
|
|
322
|
+
refusal moves neither. Bypass the check with `DEXBOT_SKIP_GIT_PREFLIGHT=1`.
|
|
317
323
|
|
|
318
324
|
Note: `pmain` does **not** create a release tag. Run `git tag -a v0.X.Y -m "..."` manually after `pmain` if you need a tagged release point.
|
|
319
325
|
|