dexbot 1.7.0 → 1.7.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (105) hide show
  1. package/CHANGELOG.md +42 -0
  2. package/README.md +3 -3
  3. package/analysis/README.md +33 -13
  4. package/claw/package.json +1 -1
  5. package/claw/runtimes/openclaw-plugin/openclaw.plugin.json +1 -1
  6. package/claw/runtimes/openclaw-plugin/package.json +1 -1
  7. package/dist/analysis/account_resolver.js +5 -6
  8. package/dist/analysis/analyze_liquidity.js +15 -4
  9. package/dist/analysis/bot_key_utils.d.ts +44 -1
  10. package/dist/analysis/bot_key_utils.js +76 -7
  11. package/dist/analysis/fills_cache.js +2 -1
  12. package/dist/analysis/grid_correction_check.d.ts +1 -19
  13. package/dist/analysis/grid_correction_check.js +12 -30
  14. package/dist/analysis/liquidity/liquidity_chart_generator.js +190 -20
  15. package/dist/analysis/liquidity/liquidity_depth_model.d.ts +41 -16
  16. package/dist/analysis/liquidity/liquidity_depth_model.js +49 -21
  17. package/dist/analysis/liquidity/liquidity_view.d.ts +15 -0
  18. package/dist/analysis/liquidity/liquidity_view.js +11 -0
  19. package/dist/analysis/resolve_bot_accounts.js +7 -6
  20. package/dist/analysis/trade_profitability.js +2 -1
  21. package/dist/credential-daemon.js +13 -2
  22. package/dist/dexbot.js +4 -4
  23. package/dist/modules/account_bots.d.ts +11 -1
  24. package/dist/modules/account_orders.d.ts +12 -0
  25. package/dist/modules/account_orders.js +30 -0
  26. package/dist/modules/bitshares-native/transport.js +24 -6
  27. package/dist/modules/bitshares_client.d.ts +24 -6
  28. package/dist/modules/bitshares_client.js +44 -7
  29. package/dist/modules/bot_settings.d.ts +2 -0
  30. package/dist/modules/constants.d.ts +26 -3
  31. package/dist/modules/constants.js +63 -8
  32. package/dist/modules/dexbot_class.d.ts +1 -6
  33. package/dist/modules/dexbot_class.js +14 -0
  34. package/dist/modules/dexbot_cow_runtime.d.ts +97 -7
  35. package/dist/modules/dexbot_cow_runtime.js +620 -132
  36. package/dist/modules/dexbot_fill_runtime.js +124 -35
  37. package/dist/modules/dexbot_maintenance_runtime.js +226 -73
  38. package/dist/modules/dexbot_startup_runtime.js +32 -20
  39. package/dist/modules/dexbot_state_recovery.d.ts +33 -1
  40. package/dist/modules/dexbot_state_recovery.js +167 -5
  41. package/dist/modules/node_connect_policy.d.ts +8 -1
  42. package/dist/modules/node_connect_policy.js +18 -2
  43. package/dist/modules/node_flap_score.d.ts +87 -0
  44. package/dist/modules/node_flap_score.js +124 -0
  45. package/dist/modules/node_health_cache.d.ts +40 -3
  46. package/dist/modules/node_health_cache.js +66 -20
  47. package/dist/modules/node_manager.d.ts +101 -19
  48. package/dist/modules/node_manager.js +255 -65
  49. package/dist/modules/order/accounting.js +56 -7
  50. package/dist/modules/order/format.d.ts +24 -10
  51. package/dist/modules/order/format.js +30 -10
  52. package/dist/modules/order/grid.js +82 -11
  53. package/dist/modules/order/grid_reconcile.d.ts +5 -1
  54. package/dist/modules/order/grid_reconcile.js +333 -210
  55. package/dist/modules/order/grid_reconcile_internal.d.ts +38 -2
  56. package/dist/modules/order/grid_reconcile_internal.js +207 -87
  57. package/dist/modules/order/index.d.ts +4 -0
  58. package/dist/modules/order/lifecycle.d.ts +353 -0
  59. package/dist/modules/order/lifecycle.js +1109 -0
  60. package/dist/modules/order/manager.d.ts +271 -5
  61. package/dist/modules/order/manager.js +651 -48
  62. package/dist/modules/order/placement_lock.d.ts +102 -0
  63. package/dist/modules/order/placement_lock.js +222 -0
  64. package/dist/modules/order/strategy.js +13 -1
  65. package/dist/modules/order/sync_engine.d.ts +13 -1
  66. package/dist/modules/order/sync_engine.js +153 -53
  67. package/dist/modules/order/utils/math.d.ts +22 -1
  68. package/dist/modules/order/utils/math.js +54 -1
  69. package/dist/modules/order/utils/order.d.ts +75 -5
  70. package/dist/modules/order/utils/order.js +158 -9
  71. package/dist/modules/order/utils/system.d.ts +18 -0
  72. package/dist/modules/order/utils/system.js +84 -8
  73. package/dist/modules/order/utils/validate.d.ts +7 -1
  74. package/dist/modules/order/utils/validate.js +48 -2
  75. package/dist/modules/outcome.d.ts +54 -0
  76. package/dist/modules/outcome.js +51 -0
  77. package/dist/modules/settings_merge.d.ts +4 -0
  78. package/dist/modules/settings_merge.js +9 -0
  79. package/dist/modules/timing_policy.d.ts +82 -0
  80. package/dist/modules/timing_policy.js +244 -0
  81. package/dist/modules/types.d.ts +132 -0
  82. package/dist/scripts/analyze-credit.js +54 -17
  83. package/dist/scripts/analyze-orders.d.ts +4 -2
  84. package/dist/scripts/analyze-orders.js +113 -34
  85. package/dist/scripts/chart_targets.js +11 -15
  86. package/dist/scripts/test-credit-renewal.js +12 -5
  87. package/docs/COW_INVARIANTS.md +67 -8
  88. package/docs/DEXBOT_COMPARISON.md +6 -6
  89. package/docs/EVOLUTION.md +9 -8
  90. package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md +1 -1
  91. package/docs/GRID_PRICE_INVARIANT.md +1 -1
  92. package/docs/GRID_RECONCILE.md +17 -12
  93. package/docs/LOGGING.md +25 -1
  94. package/docs/ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md +175 -26
  95. package/docs/README.md +23 -2
  96. package/docs/WORKFLOW.md +9 -3
  97. package/docs/architecture.md +42 -5
  98. package/docs/developer_guide.md +25 -8
  99. package/modules/README.md +3 -0
  100. package/package.json +1 -1
  101. package/scripts/README.md +16 -0
  102. package/scripts/pdev.sh +14 -1
  103. package/scripts/pmain.sh +23 -8
  104. package/scripts/ptest.sh +7 -0
  105. package/tests/README.md +30 -1
@@ -22,7 +22,9 @@ Reconciliation aligns the bot's model with on-chain reality. It runs at startup
22
22
 
23
23
  The reconcile runs from [`recalculateGrid`](../modules/order/grid.ts) during startup full-resync. **There is no per-attempt wall-clock race around the reconcile itself** — an outer timeout would fire mid-batch and orphan in-flight broadcasts (the duplicate-accumulation death spiral). The reconcile is bounded only by the 10-minute total resync safety net, and every internal chain read follows the shared guarded-read standard.
24
24
 
25
- Phase 1 does all reasoning in memory under `_gridLock` (fast); Phases 2 and 3 execute outside that planning lock — holding it across RPC calls would block fills, sync, and divergence checks. Phase 2 holds the broadcasting single-flight marker, while the sync/apply paths it invokes follow the canonical lock hierarchy. No planning-phase grid lock spans a network operation.
25
+ Phase 1 does all reasoning in memory under `_gridLock` (fast); Phases 2 and 3 execute outside that planning lock — holding it across RPC calls would block fills, sync, and divergence checks. **Both phases run inside the exclusive placement region** (`manager.acquirePlacement('startup-reconcile')`, acquired before the chain snapshot is trusted and released after `stopBroadcasting`), so no other chain-mutating path can broadcast between Phase 1's plan and Phase 2's execution. Phase 2 additionally holds the broadcasting single-flight marker (which only defers fills), while the sync/apply paths it invokes follow the canonical lock hierarchy. No planning-phase grid lock spans a network operation.
26
+
27
+ The reconcile is not purely time-unbounded: besides the 10-minute total resync safety net it also waits up to 120 s for the placement region and re-reads the open-orders snapshot after acquiring it (planning from a snapshot older than the region is exactly the duplicate class). A reconcile that cannot get the region returns `null` and defers to the next sync cycle.
26
28
 
27
29
  Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations are attempted — plans are logged but not executed.
28
30
 
@@ -80,8 +82,8 @@ Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations a
80
82
 
81
83
  **Source:** `reconcileGridOrders()` Phase 1 in `grid_reconcile.ts`.
82
84
 
83
- 1. **Phantom order sanitization** (the `isOrderPlaced()` absence pass in `reconcileGridOrders()`): For each `isOrderPlaced()` order whose `orderId` is not in the chain snapshot, reset it to VIRTUAL with `skipAccounting` to prevent fund inflation. Two absence-decision guards make this safe:
84
- - **Freshly-assigned deferral** (the `_orderIdAssignedAt` freshness guard): an `orderId` assigned within `TIMING.SYNC_LOCK_TIMEOUT_MS` may be an in-flight create/adopt whose broadcast has not landed or is not yet visible to a lagging/truncated read — virtualizing it and re-creating would duplicate a real live order (the reconcile-timeout death-spiral root cause). It is skipped (deferred) rather than virtualized.
85
+ 1. **Phantom order sanitization** (the `isOrderPlaced()` absence pass in `reconcileGridOrders()`): For each `isOrderPlaced()` order whose `orderId` is not in the chain snapshot, reset it to VIRTUAL with `skipAccounting` to prevent fund inflation. One absence-decision guard makes this safe:
86
+ - **In-flight deferral** (`LifecycleView.isInFlight()`, read through the slot with `readSlotLifecycle`): an order whose chain id was assigned/committed within the guard window, or whose lifecycle phase is still unresolved (`unknown`/`broadcasting` — a lost CREATE receipt, an ambiguous or unparseable read), may be live and merely missing from a lagging/truncated read — virtualizing it and re-creating would duplicate a real live order (the reconcile-timeout death-spiral root cause). It is skipped (deferred) rather than virtualized. The same predicate is the only absence guard in the sync pass-1 phantom cleanup and the fund recalibration sweep; asking a per-site subset is the bug class [ORDER_LIFECYCLE.md](ORDER_LIFECYCLE.md) exists to remove.
85
87
  - **Ghost heuristic** (same absence pass): an order with `size <= 0` && `PARTIAL` (a known filled ghost) still passes through so known fills get cleaned up.
86
88
  - Virtualization always uses `{ skipAccounting: true }` so startup cleanup never inflates `ChainFree`.
87
89
 
@@ -93,7 +95,7 @@ Phase 2 and 3 both respect the `dryRun` flag: when true, no on-chain mutations a
93
95
  - Match sorted unmatched chain orders to virtual slots → `plannedUpdates`
94
96
  - Detect grid-edge lock and plan a largest-order cancel
95
97
  - Plan creates for remaining slots
96
- - Plan excess cancellations (guarded by `matchedOnGrid > 0`): orphans first, then **matched surplus** (`chainCount - targetCount`, edge slots last) — the matched-excess selection is shared by the planOnly and execute branches so planning can never drift from execution; planOnly omits `releaseUntrackedFunds` for matched slots (their funds are tracked on the grid slot)
98
+ - Plan excess cancellations (guarded by `matchedOnGrid > 0`): orphans first, then **matched surplus** (`chainCount - targetCount`, edge slots last) — ONE selection feeds both the planOnly and the inline sink (the `excessPlans` list), so planning can never drift from execution; matched slots omit `releaseUntrackedFunds` (their funds are tracked on the grid slot). Every plan stamps the gap geometry from `resolveGapSlots` (never the raw `_gapSlots` field), which is also what the Phase-2 plan validator compares against
97
99
  - Fork-kept **shelf orders** (live non-slot-N ids below the rail, e.g. `deep-*`) are never cancel candidates or reserve members — see [Shelf Orders](#shelf-orders-fork-kept-manual-orders)
98
100
  - **Vacated-rail refill**: each PROCEEDING update whose vacated price exactly matches (`priceSlotEqual`) an empty, sized, in-rail slot of the same side queues a refill CREATE in the same plan (`source startupVacatedRailRefill`) — skipped updates, ghost prices (lattice moved), in-band slots, and already-desired slots never refill; refill targets require VIRTUAL state with no `orderId`
99
101
 
@@ -115,7 +117,9 @@ snapshots. For every plan, `_startupCancelPlanStillCurrent()` requires:
115
117
 
116
118
  - the target chain order still exists with its exact Phase-1 signature;
117
119
  - a matched plan still owns the same chain order in the same live grid slot;
118
- - planned type, slot price, boundary, and gap geometry still match; and
120
+ - planned type, slot price, boundary, and gap geometry still match (both sides
121
+ read the geometry through `resolveGapSlots`, so an unpublished `_gapSlots`
122
+ field cannot make every plan look stale); and
119
123
  - an originally unmatched surplus is still not owned by any live slot.
120
124
 
121
125
  Validation is per-plan: an earlier cancellation or unrelated order change does
@@ -127,7 +131,8 @@ remain unchanged.
127
131
 
128
132
  **Fresh-placement grace (`TIMING.SURPLUS_CANCEL_GRACE_MS`, 15 min):** surplus
129
133
  and cancel-only cancellations also skip any order whose `orderId` was placed
130
- inside the grace window. The timestamp is written by `recordOrderPlacement` into
134
+ inside the grace window. The timestamp is written by the lifecycle write funnel
135
+ (and by `recordOrderPlacement` for manager-shaped callers) into
131
136
  `manager._placedAt` and tested with `isFreshlyPlacedOrder`; the reconciler's CANCEL
132
137
  actions are filtered by `COWRebalanceEngine.execute`, which drops the whole slot's
133
138
  action group (a paired CANCEL+CREATE must not lose only its CANCEL, or the CREATE
@@ -207,7 +212,7 @@ Up to 3 batch attempts (`maxBatchAttempts = 3`). Each failure triggers a recover
207
212
 
208
213
  ### Phantom Orders via Reconcile
209
214
 
210
- Reconcile's role in the defense-in-depth: during Phase 1, any ACTIVE/PARTIAL order whose `orderId` is not found on-chain is reset to VIRTUAL with `skipAccounting`. The freshly-assigned deferral protects in-flight broadcasts, and the ghost heuristic lets known fills pass. See [`developer_guide.md`](developer_guide.md#phantom-orders-prevention-defense-in-depth) for the full 3-layer defense.
215
+ Reconcile's role in the defense-in-depth: during Phase 1, any ACTIVE/PARTIAL order whose `orderId` is not found on-chain is reset to VIRTUAL with `skipAccounting`. The in-flight deferral (`LifecycleView.isInFlight()`) protects in-flight broadcasts and unresolved receipts, and the ghost heuristic lets known fills pass. See [`developer_guide.md`](developer_guide.md#phantom-orders-prevention-defense-in-depth) for the full 3-layer defense.
211
216
 
212
217
  ### Shelf Orders (Fork-Kept Manual Orders)
213
218
 
@@ -232,7 +237,7 @@ Reconcile derives its window from the **master grid**, which is only as fresh as
232
237
  - **Unsettled-fill gate** (`hasUnsettledFills`, `utils/system.ts`): while fills are queued (`_incomingFillQueue`), draining (`_deferredFillsPending`), in flight (`_fillBatchInFlight`) or owed a crawl (`_pendingFillCrawls`), `maybeRunTargetedDriftReconciliation` defers the targeted reconcile entirely and `_reconcileStartupSide` defers only the **window top-up** creates (1:1 repairs — cancelled-slot replacement, vacated-rail refill — still run). The master still counts filled slots as ACTIVE in that window, so a top-up would restore the expensive end of the rail.
233
238
  - **Last-fill placement guard** (`isLastFillPlacementBlocked`, `utils/system.ts`): every reconcile placement (create in `_createOrderFromGrid` and the batch create path, plus the re-pricing relocation in `_prepareStartupUpdatePlan`) is refused when it sits on the wrong side of the last fill **or on the same grid level** (within half an increment — a slot level and its execution price differ by rounding). A same-price relocation is an adoption, not a placement, and is exempt. `_pickVirtualSlotsToActivate` applies the same predicate so the pick budget rolls to the next level down/up instead of being spent on a level the create would refuse — the filled level is never re-stamped same-side (the "rotate to bottom BUY / counter SELL" behaviour).
234
239
 
235
- A fill whose order id no longer sits on an ACTIVE/PARTIAL slot is no longer dropped: `attributeUnmatchedFillToGrid` (shared by `SyncEngine._absorbUnmatchedFill` and the orphan-credit sweep in `dexbot_fill_runtime.ts`) records the **pivot + owed crawl from the real execution price**. It resolves the slot two ways — the chain id when a virtualised slot still owns it, otherwise the **genesis ladder level of the execution price** (the common case, because `virtualizeOrder` NULLS `orderId`; this is what the incident actually hit). A crawl is only owed by a HOLE, and a price outside the frozen rail is refused so a shelf order cannot arm the pivot. Without this the pivot stays on a stale, higher fill and the direction rule then permits re-buys above the market.
240
+ A fill whose order id no longer sits on an ACTIVE/PARTIAL slot is no longer dropped: `attributeUnmatchedFillToGrid` (shared by `SyncEngine._absorbUnmatchedFill` and the orphan-credit sweep in `dexbot_fill_runtime.ts`) records the **pivot + owed crawl from the real execution price**. It resolves the slot two ways — the chain id when a virtualised slot still owns it, otherwise the **genesis ladder level of the execution price** (the common case, because `virtualizeOrder` NULLS `orderId`; this is what the incident actually hit). A crawl is only owed by a HOLE, and a price outside the frozen rail is refused so a shelf order cannot arm the pivot. Without this the pivot stays on a stale, higher fill and the direction rule then permits re-buys above the market. The sweep is **replay-safe**: the processed-fill ledger is the authority, so attribution runs only when the accounting is not a `duplicate`. A replay already armed the pivot/crawl on its first delivery; re-attributing it re-records an old execution price (live: a 12-minute-old replay re-recorded four crawls and moved the boundary 165 → 169).
236
241
 
237
242
  **The dark level it leaves behind is repaired too.** A slot virtualised while its order stayed live on chain reads as *not active* (`_countActiveOnGrid` needs ACTIVE/PARTIAL) while its order still counts in `chainCount`, so the side reports a shortfall it can never repair — and the planner sees a free level that already holds an order. Three pieces close that:
238
243
 
@@ -260,12 +265,12 @@ The underlying rule is `INV-BROADCAST-004`: a capped `get_full_accounts` window
260
265
  **`manager.ts` lock declaration** — canonical reference in [`developer_guide.md`](developer_guide.md#lock-ordering-for-deadlock-prevention).
261
266
 
262
267
  ```
263
- Level 0: _fillProcessingLock Level 1: _divergenceLock
264
- Level 2: _syncLock Level 3: _gridLock
265
- Level 4: _fundLock
268
+ Level 0: _fillProcessingLock Level 0.5: placement region
269
+ Level 1: _divergenceLock Level 2: _syncLock
270
+ Level 3: _gridLock Level 4: _fundLock
266
271
  ```
267
272
 
268
- Acquire in ascending level order only. AsyncLock is re-entrant (nested `acquire()` run directly, not queued).
273
+ Acquire in ascending level order only. AsyncLock is re-entrant (nested `acquire()` run directly, not queued); the placement region (`modules/order/placement_lock.ts`) is NOT re-entrant — it is a FIFO lease, and nested work must reuse the holder's lease (verified with `manager.isPlacementLeaseCurrent()`).
269
274
 
270
275
  ### Historical Correction (1.4.6)
271
276
 
package/docs/LOGGING.md CHANGED
@@ -252,7 +252,7 @@ Prefix tags used in log messages to help operators identify event types. To find
252
252
  | `[STRATEGY]` | `order/strategy.ts` | Fee event cache and strategy decisions |
253
253
  | `[RECONCILE]` | `order/utils/validate.ts` | Grid reconciliation ([GRID_RECONCILE.md](GRID_RECONCILE.md)) |
254
254
  | `[RMS]` | `order/grid.ts`, `dexbot_maintenance_runtime.ts` | RMS structural-divergence checks: per-side `metric vs threshold → TRIGGER-RESYNC/no trigger` at `debug` (plus a `checks disabled` variant when the threshold is 0), reset line with threshold + breaching sides at `info` |
255
- | `[GAP-EVAC]` | `order/utils/validate.ts`, `order/manager.ts`, `dexbot_startup_runtime.ts`, `dexbot_state_recovery.ts` | Stuck in-band (gap-band) order streak warnings, cancel-only evacuation teeth, persisted-streak restore counts |
255
+ | `[GAP-EVAC]` | `order/utils/validate.ts`, `order/manager.ts`, `dexbot_startup_runtime.ts`, `dexbot_state_recovery.ts`, `dexbot_maintenance_runtime.ts` | Stuck in-band (gap-band) order streak warnings, cancel-only evacuation teeth, persisted-streak restore counts, and the committed-geometry stranding sweep (post-commit + maintenance tick) |
256
256
  | `[GRID-TYPE-CORRECT]` | `order/grid.ts` | One-time backfill retype of legacy empty slots to rail-typed holes on load |
257
257
  | `[LAST-FILL-GUARD]` | `dexbot_cow_runtime.ts` | Last-fill-guard blocks plus gap-evacuation bypass allows/stale-stamp downgrades |
258
258
  | `[RESTAMP-BUY]` | `dexbot_cow_runtime.ts` | BUY emitted at/above the last filled BUY pivot (guard bypass / unjudgeable pivot), counted not blocked (P2) |
@@ -313,11 +313,35 @@ re-plan retries). Kept separate from the generic `refusedCommitCount`.
313
313
  | `[SPREAD] No fundable side, but a rotation-only repair is available on <side>; using resting inventory (no new commitment).` | Both sides read zero free (or side selection returned no fundable side), but a promoted gap slot can be filled by rotating a resting order. Cross-asset side selection was not overridden when the market price itself was unavailable |
314
314
  | `[LAST-FILL-GUARD] Skipping <type> UPDATE for <src> -> <dst>: stale gap-evacuation (...) — next plan re-evaluates` | A plan-build gap-evacuation stamp no longer matches the LIVE boundary/gap (or the live probe is not violation-reducing). The op is dropped instead of falling through to the direction-only last-fill guard; the next plan/reconcile owns the move |
315
315
  | `[LAST-FILL-GUARD] Final gate dropping stale gap-evacuation ...` | The final pre-broadcast pivot gate also refuses a stamp whose live geometry went stale after op-building, even when the destination price itself is direction-legal |
316
+ | `[LAST-FILL-GUARD] Stamped evacuation for <src> -> <dst> is stale under live boundary B/gap G, but the live rails already conflict (maxBuy=X > minSell-gap-1=Y) — the retreat this batch commits is forced, so the evacuation proceeds` | Forced-retreat allowance (`liveRailsConflictWithGap`). The stale-stamp refusal assumes the live boundary survives the commit; when the live rails already overlap, NO boundary can host both rails with the gap band, so the batch's retreat is the only consistent geometry and refusing the evacuation is what permanently strands the order in the band (spread collapse). The same allowance is applied by the build loop and by the final pivot gate |
317
+ | `[LAST-FILL-GUARD] Final gate keeping gap-evacuation ... despite live boundary B/gap G: the live rails already conflict ...` | The final-gate mirror of the forced-retreat allowance above (keeping the op is violation-reducing, not a guard bypass) |
318
+ | `[GAP-EVAC] N live order(s) stranded inside the spread band (boundary B, gap G): <slot(side), ...> — scheduling a fill-less re-plan to evacuate (<context>)` | Post-commit / maintenance sweep: live rail orders are resting inside the committed band, so the real top-of-book spread is narrower than the configured one. A fill-less COW re-plan runs against the committed geometry (where the evacuation stamp validates) — plans are fill-driven, so a quiet market would otherwise never heal. Rate-limited (60s) per bot; repeats inside the window log at `debug` |
319
+ | `[GAP-EVAC] ... repair cooldown active (Ns left); teeth queued=N` | The sweep found the same stranding again inside the repair cooldown window; the cancel-only teeth remain the last resort |
320
+ | `[COW] Holding N gap-evacuation cancel(s) back from the pre-batch drain: this batch rotates the same chain order (the rotation IS the evacuation)` | A cancel-only evacuation queued for a chain order the current batch rotates. Draining it first would cancel the batch's own rotation source and the UPDATE batch would be rejected pre-broadcast (`target(s) missing on chain` → structural resync). The entry is restored after the drain and dropped as stale once the slot no longer owns the order |
321
+ | `[GAP-EVAC] Spread band clean: cleared N resolved in-band streak(s)` | The sweep found no live order inside the committed band and dropped N resolved ledger entries, so a future return to the band re-arms instead of inheriting a stale cycle count. Logged only when something was actually cleared |
322
+ | `[GAP-EVAC] ... unknown geometry` (silent) | With no committed boundary (or no known band width) the sweep returns without touching the ledger or the pipeline: absent geometry is not health, it is unknown |
323
+ | `[GAP-EVAC] ... no fill-less re-plan available on this path; teeth queued=N` | The stranding is real but this bot shape cannot run a fill-less rebalance (no `performSafeRebalance`/`updateOrdersOnChainBatch`); the cancel-only teeth own the repair |
324
+ | `[BOUNDARY] restore boundary=B sellStart=S gapSlots=G spreadCount=N\|? activeBuy=... activeSell=...` | Boundary restore telemetry. `gapSlots` is the resolved geometry (`_gapSlots ?? config-derived`) — never `0` for an unpublished band — and `spreadCount=?` means the grid has not been loaded/created yet, so no spread metric exists. A literal `gapSlots=0 sellStart=B+1` now always means a genuinely 0-wide band, not an initialization window |
316
325
  | `[SPREAD-CORRECTION] <side> rail is OVERSUBSCRIBED: virtual reservation X exceeds chain free Y by Z — no correction can be funded until the rail is re-sized ...` | `availableFund` is `free − virtual`, so an oversubscribed rail reads 0 while the side still holds inventory. The watchdog escalates this on its short fund window |
317
326
  | `[SPREAD-STALE] Spread out of tolerance for Nmin ... (outOfSpread=..., stall=...)` | Persistence warn with the last stall cause; the clock survives single in-tolerance readings and the terminal cases escalate early: `stall=last-fill-blocked` (sell window behind the last fill) on the 10-minute geometry window, a rail oversubscription on the 5-minute fund window |
318
327
  | `[SPREAD-STALE] ... (sell window behind the last fill — re-centering)` | The `last-fill-blocked` geometry bind is escalating; placement cannot repair it, a re-center can |
319
328
  | `[SPREAD-STALE] Still out of tolerance Nmin after re-centering on fill pivot P: the pivot is outside this generation's placeable side, so a re-center cannot clear it. A fill ... or pivot expiry restores the side; no further re-center will be requested for this pivot.` | Churn cap: a re-center does not move the pivot, so the same fill gets ONE re-center (`_lastGeometryResyncPivotKey`). Logged once per pivot; the same-side rail stays blocked until a fill or the read-time 24h pivot TTL, by design |
320
329
  | `[BOUNDARY] Re-anchored from the last fill: SELL filled at slot 153 (0.447640); boundary 146 -> 149 so the fill sits outside its own window` | The window was behind the fill and a ±1 crawl cannot catch up; the fill now anchors the geometry. The anchor is the current fill batch's extremum for that side (max SELL / min BUY), not the last-processed pivot. Refusals stay silent (cold/off-ladder/ceiling-clamped/unsafe preconditions); the fill's own PARTIAL residual at the ANCHOR slot is allow-listed, so a partial fill does not deadlock the repair |
330
+ | `[BOUNDARY] Re-anchor accepted as a violation-reducing repair: boundary X -> Y strands fewer placed orders than the committed geometry (placed_order_in_band)` | The strict gate refused the candidate only because older GAP-EVAC strands are still in band; the move strands STRICTLY FEWER than the committed boundary, so it is accepted. Equal counts and structural refusals (crossed book/ceiling) stay closed. The rule lives in ONE helper (`boundaryMoveReducesStrands`) shared with the rail clamp — see `[COW] Boundary rail-derived` below. The promotion-side counterpart is the in-band absorption logged as `[SPREAD-CORRECTION] N gap slot(s) available for boundary promotion` |
331
+ | `[PLACEMENT] Force-releasing stale placement region held by <kind> for Ns (watchdog Ms); fills/rebalances must not stay parked` | The exclusive placement region outlived `TIMING.PLACEMENT_STALE_CLEAR_MS` (generous: above the longest observed reconcile), so the lease is force-released; the holder's later `release()` becomes a no-op (token guard). This watchdog is the ONE authority that tears a hung region down — when the region also still holds a stale broadcast refcount, it clears that too (`[PLACEMENT] Also clearing the stale broadcast region ...; one teardown for one hung region`) |
332
+ | `[COW] Placement lease was force-released while this batch was in flight (stale-region watchdog); aborting this batch instead of broadcasting without the region.` | The owner re-verifies its own lease with `isPlacementLeaseCurrent()` before broadcasting — the mirror of the nested drain's check. Aborting is the fail-closed answer: another path may own the region by now, and broadcasting anyway re-opens the duplicate window the teardown existed to avoid |
333
+ | `Startup reconcile: placement lease was force-released after planning (stale-region watchdog); discarding the plan instead of broadcasting without the region.` | The reconcile's own lease-liveness gate before Phase 2 (the mirror of `[COW] Placement lease was force-released ...`). Phase 1 touched only the master, so the plan is dropped with no chain effect; the next cycle re-reads the chain after re-acquiring the region |
334
+ | `[PLACEMENT] Watchdog skipped: region held by <kind> for Ns, but the pipeline is still making progress (in flight Ms)` | A bot-level batch/recovery counter younger than the placement bound proves the region is live (e.g. a long create-group reconcile), so nothing is torn down. Only a counter past the same bound is treated as part of the hung region |
335
+ | `[COW] <stage>: exclusive placement region unavailable (timeout/shutdown); aborting this batch instead of overlapping another placement path.` | Another chain-mutating path owns the region and the bounded wait expired. The batch is aborted (never broadcast unprotected) and retried on the next cycle |
336
+ | `Startup reconcile: exclusive placement region unavailable (timeout/shutdown); deferring to the next sync cycle.` | The reconcile could not serialize with a live placement path; it returns without mutating anything |
337
+ | `[DUST] Deferring dust cancels: exclusive placement region busy or shutting down` | Dust cancels mutate the chain, so they need the region. Deferral (not a bypass) is the contract; the next fill batch / maintenance tick retries |
338
+ | `[RESIDUAL] Deferring N residual cancel(s): exclusive placement region busy or shutting down` | Sub-dust residual cancels after a full fill are deferred to the next sync/reconcile instead of racing a live batch |
339
+ | `[MAINT] Deferring price-correction drain: exclusive placement region busy or shutting down` / `[FILL-QUEUE] Deferring price-correction drain: ...` / `[COW] Draining N pending correction(s) before batch` | The correction drain runs inside the region (nested callers pass the holder's lease, verified with `isPlacementLeaseCurrent`). A deferred drain keeps its queued entries for the next tick; the fill consumer and the maintenance tick report the same deferral under their own tags |
340
+ | `[BROADCAST] Stale-flag watchdog skipped: placement region still held by <kind> for Ns` | The refcount watchdog refuses to clear a broadcast region a live placement lease still owns — the mirror rule of the placement watchdog's pipeline-progress check |
341
+ | `[REBUILD] Structural rebuild started (<reason>); fill-driven rebalances deferred` | A trigger-file/startup rebuild owns placement and update-first repair; fill-driven COW rebalances defer instead of re-planning against the order-id-less master |
342
+ | `[SAFE-REBALANCE] Deferred: placement region held by <kind>; rebalance retries after it ends` | A fill-driven rebalance found the region held (reconcile/drain/dust/cancel/swap). The owed fill crawls stay recorded and the region-end hook reschedules the no-fill rebalance |
343
+ | `[FILL-DEDUP] Fill <key>: N/M base units already absorbed by sync pass 1; the transition will not shrink them again` | The open-orders pass resized the slot from the same post-fill snapshot. The explicit applied amount (never the legacy boolean) drives the remainder, so the trade's pays/receives are still booked while the size is released exactly once |
344
+ | `[COW] Pending-broadcast entry <fp> is Ns old but no sync has completed since it was recorded; keeping it (fail-closed) until a caught-up read` / `[COW] Dropped expired pending-broadcast entry ...` | The single pending-broadcast liveness authority (`getLivePendingBroadcasts`): TTL, or TTL + a successful sync after the record, or the 2× hard cap. Every reader (COW guard, reconcile CREATE, reconcile surplus sweep, crossing candidates) obeys it |
321
345
  | `[FILL-PIVOT] pivot=... buy=... sell=... n=N src=full/direct` (+ `edge-skipped=K` when present) | One line per fill batch: the LAST-FILL-GUARD pivot after the batch. `edge-skipped=K` means K edge-pinned insurance fills were ignored — they never arm the pivot, so they can never drive the re-anchor |
322
346
  | `[FILL-PIVOT] K edge-pinned fill(s) ignored — the guard pivot and the boundary anchor stay on the last window trade` | Edge-only batch: nothing armed, which is the point |
323
347
  | `[LAST-FILL-GUARD] Re-armed on the new generation: 0.447640(SELL) from the last real fill (age Ns) — the guard survives a rebuild by design` | A grid rebuild dropped the generation-bound pivot and re-armed it explicitly from the bot-scoped last-market-fill row (`rearmLastFillPivotFromMarketFill`). Provenance stays `fill`; the timestamp is the original fill's, taken from the SAME row that supplied the price (a book-seeded live pivot never lends its clock) |
@@ -4,23 +4,27 @@
4
4
  > misbehaving (Part I) and the preserved **incident & fix ledger** (Part II), formerly
5
5
  > `CONSOLIDATED_ORPHAN_FIX_SUMMARY.md`.
6
6
  >
7
- > **Sources:** `CHANGELOG.md` v1.0.0 → v1.6.14; `docs/GRID_PRICE_INVARIANT.md`;
8
- > `docs/COW_INVARIANTS.md`. Generic terms only — no live markets, accounts, or order ids.
7
+ > **Sources:** `CHANGELOG.md` v1.0.0 → v1.7.0 (plus the unreleased post-1.7.0 arc);
8
+ > `docs/GRID_PRICE_INVARIANT.md`; `docs/COW_INVARIANTS.md`; `docs/ORDER_LIFECYCLE.md`.
9
+ > Generic terms only — no live markets, accounts, or order ids.
9
10
  >
10
- > **Scope / snapshot:** v1.0.0 → v1.6.14 (`f6c0df32`, 2026-10-12); the v1.6.4 → v1.6.14 arc
11
- > is the §9 addendum below.
11
+ > **Scope / snapshot:** v1.0.0 → v1.7.0 (`5644aa3a`, 2026-11-02), extended through the
12
+ > unreleased post-1.7.0 work at `84b935d9` (2026-11-04). §9 is the v1.6.4 → v1.6.17
13
+ > addendum; §9.10 closes the v1.6.18 review pass and §9.11 carries v1.6.18 → unreleased.
12
14
  >
13
15
  > **Section numbering:** Part II deliberately keeps its original `§0`–`§8` numbers because
14
16
  > source-code comments and tests cite them (e.g. `… §2 fix #1`, `… §3 lineage`); §9 is a
15
- > later addendum (v1.6.4 → v1.6.14) that extends, and does not renumber, that range. Part I
16
- > uses `R1`–`R5` so the two never collide. The closing appendix is unnumbered (`A`).
17
+ > later addendum (v1.6.4 → v1.6.17, with §9.10/§9.11 extending it) that adds to, and does not
18
+ > renumber, that range. Part I uses `R1`–`R5` so the two never collide. The closing appendix
19
+ > is unnumbered (`A`).
17
20
  >
18
21
  > **How to read:** start with **Part I** for the explanation; drop into **Part II** when you need
19
22
  > the per-incident evidence, commit hashes, and verification targets.
20
23
  >
21
24
  > **See also:** [GRID_PRICE_INVARIANT.md](GRID_PRICE_INVARIANT.md) ·
22
25
  > [COW_INVARIANTS.md](COW_INVARIANTS.md) · [GRID_RECONCILE.md](GRID_RECONCILE.md) ·
23
- > [EVOLUTION.md](EVOLUTION.md) · [CHANGELOG.md](../CHANGELOG.md).
26
+ > [ORDER_LIFECYCLE.md](ORDER_LIFECYCLE.md) · [EVOLUTION.md](EVOLUTION.md) ·
27
+ > [CHANGELOG.md](../CHANGELOG.md).
24
28
 
25
29
  ---
26
30
 
@@ -44,7 +48,8 @@
44
48
  - 7. Consolidated verification — grid-correction check as regression gate
45
49
  - 8. Provenance & supersession record
46
50
  - 9. COW Broadcast, Resync & Geometry Safety (v1.6.4 → v1.6.17) — addendum
47
- - 9.10 Review pass over v1.6.12 → v1.6.17 (findings P1–P5)
51
+ - 9.10 Review pass over v1.6.12 → v1.6.17 (findings P1–P5), released in v1.6.18
52
+ - 9.11 v1.6.18 → unreleased: fill identity, absence and the geometry budget
48
53
  - A. Grid-Price Invariant — former failure trace (appendix)
49
54
 
50
55
  ---
@@ -77,6 +82,9 @@ local state.
77
82
  | **Silent hangs** | Retries that depended on a future event that never came | Level-triggered scheduling with time-based watchdogs |
78
83
  | **Stale queues** | Queued price corrections replayed onto a resynced grid, reverting good placements | Pre-broadcast validation against the live slot |
79
84
  | **Shelf / edge orders** | Non-grid orders poisoning every "orders on grid" counter (issue #27) | Slot-N gating across all counters |
85
+ | **Stale window after a fill** | A fill is queued but not yet absorbed; the master still counts the filled slot ACTIVE, so a window top-up re-creates the expensive end of the rail and re-buys into a stale pivot | `hasUnsettledFills` defers the targeted reconcile and the window top-up (1:1 repairs still run) + attribution of the real execution price (§9.11.1) |
86
+ | **Fill identity by chain id** | `virtualizeOrder` nulls `orderId`, so a real fill matched no ACTIVE slot and was dropped *after* its accounting was booked; pivot and owed crawl stayed on a stale, higher price | `attributeUnmatchedFillToGrid` falls back to the genesis level of the execution price, refuses prices outside the frozen rail, and owes a crawl only for a hole the grid owns (§9.11.1) |
87
+ | **Chain-existence bookkeeping** | Seven structures with seven clocks; four read sites consulted four subsets and could give four answers for one order | `modules/order/lifecycle.ts`: one record per slot, `unknown` is a real phase, every absence decision asks the same time-bounded `isInFlight()` (§9.11.3) |
80
88
 
81
89
  Families **not** detailed in Part II: silent hangs and stale correction queues are covered in
82
90
  `CHANGELOG.md` (v1.6.1 and v1.6.3 respectively); shelf/edge orders in issue #27 and the v1.6.x
@@ -100,13 +108,16 @@ notes; the grid-price invariant has its own doc — [GRID_PRICE_INVARIANT.md](GR
100
108
  the runtime guard, the spread-correction planner, the rail clamp and the offline checker.
101
109
  Every copy drifted on its own schedule: the planner used the crossing floor, the checker
102
110
  kept the old band, the rail clamp kept the old slot ids (§9.7, §9.10). The fix that
103
- sticks is one exported predicate and callers that re-implement nothing.
111
+ sticks is one exported predicate and callers that re-implement nothing. The same shape
112
+ produced the existence bugs: seven local structures were seven policies for "does this
113
+ order exist?", and the first disagreement was the incident (§9.11.3).
104
114
 
105
115
  ### R4. What actually fixed it
106
116
 
107
117
  - **Genesis-frozen price-slot determinism** (v1.4.25+): `slot-N` → exact ladder price
108
- becomes the single source of truth, replacing tolerance-based price matching. Killed a
109
- whole family of orphan/duplicate bugs.
118
+ becomes the single source of truth. Grid-vs-grid identity is now exact; a CHAIN price is
119
+ compared by which ladder level it maps to (`pricesOnSameLevel`, bounded by asset precision),
120
+ never by a tuned band. Killed a whole family of orphan/duplicate bugs.
110
121
  - **Single ownership:** one writer for the boundary, one owner for the adapter/market
111
122
  config, one authoritative set of counters.
112
123
  - **Global last-fill guard + final pre-broadcast gate:** one pivot rule, enforced at the
@@ -124,6 +135,16 @@ notes; the grid-price invariant has its own doc — [GRID_PRICE_INVARIANT.md](GR
124
135
  each path carrying its own idea of "stranded".
125
136
  - **Live identity over local keys (§9.10, `531edeb4`):** every rail/geometry decision maps the
126
137
  order's LIVE chain price onto the ladder; a slot id is a hint, never the fact.
138
+ - **An id is not an identity (§9.11.1, `8705381b`):** a fill whose chain id was nulled by
139
+ virtualisation is resolved to the genesis level of its execution price, so attribution
140
+ cannot go blind exactly when the order is most interesting.
141
+ - **An unabsorbed fill blocks the geometry (§9.11.1, `a93f7eef`):** while fills are queued,
142
+ draining, in flight or owed a crawl, the targeted reconcile and the window top-up defer.
143
+ - **Repairs must reduce the violation (§9.11.3, `c493a604`):** a boundary move is accepted
144
+ only when it strands *strictly fewer* placed orders than the committed value, measured by
145
+ the validator's own count — never a second predicate.
146
+ - **One existence record (§9.11.3, `826d84a3`/`84b935d9`):** seven per-site structures
147
+ collapsed into `lifecycle.ts`; `unknown` is a phase, not an absence.
127
148
  - **Observability instead of a veto (§9.10/P4):** where a hard guard would re-create a
128
149
  deadlock (a counter order crossing the floor by construction), log it once and let the
129
150
  chain price it as a taker fill.
@@ -157,6 +178,15 @@ The winning strategy was **removing writers and sources of truth**, not adding h
157
178
  repair that can cite one stray order is not a repair.
158
179
  11. **When a veto would deadlock, ship the observation.** A taker-fee risk earns a log line,
159
180
  not a rule that re-creates the deadlock the previous fix removed (§9.10/P4).
181
+ 12. **An id is not an identity.** Virtualising an order clears its chain id, so anything that
182
+ matches fills, occupancy or rails by id alone goes blind exactly when the order is most
183
+ interesting. Match the ladder level of the live price; keep the id as a hint (§9.11.1).
184
+ 13. **A fill the geometry has not absorbed must block the geometry.** Deferring a repair for
185
+ one cycle is cheaper than a top-up built from a stale master, because a repair path that
186
+ runs on un-absorbed state is a way to re-create that state (§9.11.1).
187
+ 14. **Absence needs one definition, one clock and one owner.** Seven local counters are seven
188
+ policies; if two sites can disagree about whether an order exists, the disagreement is
189
+ the bug, not either answer (§9.11.3).
160
190
 
161
191
  ---
162
192
 
@@ -183,7 +213,7 @@ plans, `LANDED`/`REVERTED`/`SUPERSEDED` statuses, commit hashes, and verificatio
183
213
 
184
214
  **Grid terms:** `boundary` = discrete sell/buy split index; `gap band` = `SPREAD/virtual` slots around boundary; `sellStartIdx = splitIdx - floor(gap/2) - 1`; center-priced ladder `createOrderGrid` = `startPrice × step^(k+½)` (`modules/order/grid.ts`); `MarketAnchor` = fill-derived `lastFillPrice/maxFilledSellPrice/minFilledBuyPrice/lastFillSide`; tolerance helpers `calculatePriceTolerance = (1/satsA+1/satsB)×price` (`modules/order/utils/math.ts`), `GRID_LIMITS.PRICE_TOLERANCE_MAX_PERCENT 0.01` (`modules/constants.ts`).
185
215
 
186
- **Validation tool:** `analysis/grid_correction_check.ts` (test-only, mirrors `analysis/trade_profitability.ts` Kibana path `https://kibana.bitshares.dev`, `bitshares-*`, `operation_type:4` `fill_order` only; `buildFillQuery`, paginated `search_after`, `_source` `block_data` fields). **Invariants:** `sell→sell` with no `buy` between must rise (`curr > prev`, **equal is a violation** — a re-place at the just-filled level), `buy→buy` must fall (`curr < prev`); `per-order` aggregated default (weighted avg collapses multi-fill `1.7.x`), `--per-fill` raw. Resolution via `bot_key_utils.ts` `loadBotMeta/computeBotKey`. Reports violations, `%`, daily histogram, time/range, table with price/Δ%/times/orderIds; `--json/--csv` export; exit `0` pass / `2` fail. Strict sat-level is ground truth — the verdict is **direction only** (equal price is a violation); `--increment` is **informational only** (the band the removed `--tolerance` flag used to express is gone, §9.10/P5). Used as regression gate below; pre-fix per-account baselines are intentionally omitted here — §7 states the targets (`0` violations).
216
+ **Validation tool:** `analysis/grid_correction_check.ts` (offline Node tool — it imports the runtime's own `isLastFillGuardBlocked`, so the verdict cannot drift; its Kibana query mirrors `analysis/trade_profitability.ts` (`https://kibana.bitshares.dev`, `bitshares-*`, `operation_type:4` `fill_order` only; `buildFillQuery`, paginated `search_after`, `_source` `block_data` fields). **Invariants:** `sell→sell` with no `buy` between must rise (`curr > prev`, **equal is a violation** — a re-place at the just-filled level), `buy→buy` must fall (`curr < prev`); `per-order` aggregated default (weighted avg collapses multi-fill `1.7.x`), `--per-fill` raw. Resolution via `bot_key_utils.ts` `loadBotMeta/computeBotKey`. Reports violations, `%`, daily histogram, time/range, table with price/Δ%/times/orderIds; `--json/--csv` export; exit `0` pass / `2` fail. Strict sat-level is ground truth — the verdict is **direction only** (equal price is a violation); `--increment` is **informational only** (the band the removed `--tolerance` flag used to express is gone, §9.10/P5). Used as regression gate below; pre-fix per-account baselines are intentionally omitted here — §7 states the targets (`0` violations).
187
217
 
188
218
  ---
189
219
 
@@ -538,15 +568,17 @@ Original doc regression excerpts preserved: GAP P6 A/B/C; LADDER §2.8 checks (1
538
568
  - [x] No other doc/index lists the four (verified 2026-08-31); `analysis/README.md` documents the `grid_correction_check` tool.
539
569
  - [x] Stale `*.md` filename references in code comments/tests updated to the consolidated doc (9 module files, 6 test files), then re-pointed to this hub when it absorbed the summary.
540
570
 
541
- *Generated 2026-08-31 from the 4 test-only docs at their HEAD contents; statuses corrected against `test` HEAD `a54863ca` the same day, then refreshed against `1382f267` (2026-09-13), extended through v1.6.14 at `f6c0df32` (2026-10-12), through v1.6.17 at `78ccbb8e`, and then through the post-1.6.17 review pass (`§9.10`: `f4d6da8c`, `a2bf6ba5`, `56481b16`, `3d16f9e4`). Landed/reverted/superseded annotations carry commit hashes where known; **line references in §§1-4 were re-verified at `a54863ca` and are subject to drift** — treat them as pointers, not assertions. The GRID-PRICE-INVARIANT guard is **BLOCKING** at all six emission sites (CREATE / UPDATE / CREATE-FALLBACK in `modules/dexbot_cow_runtime.ts`, RECONCILE-CREATE / RECONCILE-UPDATE / STARTUP-CREATE in `modules/order/grid_reconcile_internal.ts`): an off-grid emission is skipped, not merely counted, so a `violated>0` line means an emission was *prevented* rather than a writer merely *pinned*. Counters are still reported (`site=`, `checked=`, `violated=`, `unchecked=`) for observability — see `docs/GRID_PRICE_INVARIANT.md`.*
571
+ *Generated 2026-08-31 from the 4 test-only docs at their HEAD contents; statuses corrected against `test` HEAD `a54863ca` the same day, then refreshed against `1382f267` (2026-09-13), extended through v1.6.14 at `f6c0df32` (2026-10-12), through v1.6.17 at `78ccbb8e`, through the v1.6.18 review pass (`§9.10`: `f4d6da8c`, `a2bf6ba5`, `56481b16`, `3d16f9e4`), and through v1.7.0 + the unreleased post-1.7.0 arc (`§9.11`, HEAD `84b935d9`). Landed/reverted/superseded annotations carry commit hashes where known; **line references in §§1-4 were re-verified at `a54863ca` and are subject to drift** — treat them as pointers, not assertions. The GRID-PRICE-INVARIANT guard is **BLOCKING** at all six emission sites (CREATE / UPDATE / CREATE-FALLBACK in `modules/dexbot_cow_runtime.ts`, RECONCILE-CREATE / RECONCILE-UPDATE / STARTUP-CREATE in `modules/order/grid_reconcile_internal.ts`): an off-grid emission is skipped, not merely counted, so a `violated>0` line means an emission was *prevented* rather than a writer merely *pinned*. Counters are still reported (`site=`, `checked=`, `violated=`, `unchecked=`) for observability — see `docs/GRID_PRICE_INVARIANT.md`.*
542
572
 
543
573
  ---
544
574
 
545
575
  ### 9. COW Broadcast, Resync & Geometry Safety (v1.6.4 → v1.6.17)
546
576
 
547
- > **Source:** `CHANGELOG.md` v1.6.4 → v1.6.17 · **Snapshot:** `78ccbb8e` (v1.6.17 tag).
577
+ > **Source:** `CHANGELOG.md` v1.6.4 → v1.6.17 · **Snapshot:** `78ccbb8e` (v1.6.17 tag);
578
+ > §9.10 and §9.11 extend past it — §9.10 released in v1.6.18, §9.11 unreleased to HEAD
579
+ > `84b935d9`.
548
580
  > Extends the §§1–8 ledger; the `§0`–`§8` numbers are unchanged (code and tests cite them).
549
- > `§9.10` and the last two ledger rows cover the **post-1.6.17** work, which is unreleased.
581
+ > The ledger labels each row with its release; `unreleased` means "in no tag yet".
550
582
 
551
583
  Sections 1–4 end with the prevention sweeps reverted: a cancel sweep on ambiguous evidence
552
584
  proved worse than the orphan it targets. The post-1.6.3 arc is the constructive inverse —
@@ -636,7 +668,7 @@ fail-closed behind `_assertGenesisInvariant`. This is R4's "removing writers and
636
668
  truth" carried to its end: the no-ladder state that Appendix A's `slot.price` corruption
637
669
  required is no longer reachable at runtime.
638
670
 
639
- #### 9.5 Ledger (order-engine commits, v1.6.4 → v1.6.17 + unreleased)
671
+ #### 9.5 Ledger (order-engine commits, v1.6.4 → v1.7.0 + unreleased)
640
672
 
641
673
  | Commit | Release | Change |
642
674
  |---|---|---|
@@ -672,11 +704,27 @@ required is no longer reachable at runtime.
672
704
  | `62312e60` | 1.6.17 | Plan only placeable tightening slots; name the stall causes |
673
705
  | `276cd504` | 1.6.17 | Fund from chain truth and re-anchor the boundary from the fill (§9.9) |
674
706
  | `2e618197` | 1.6.17 | Consume only the owed fill crawls a derivation applied (§9.8) |
675
- | `d99b8685` | post-1.6.17 | Harden partial-fill re-anchor and recovery (residual allow-list, tolerant load) |
676
- | `531edeb4` | post-1.6.17 | Keep the boundary on the placed rails; use live fill prices for geometry |
677
- | `f4d6da8c` | post-1.6.17 | Edge-pinned fills never arm the pivot or the anchor (§9.10/P1) |
678
- | `a2bf6ba5` | post-1.6.17 | Guard arity, explicit pivot re-arm, stray-aware rail clamp, crossing observation (§9.10/P2-P4) |
679
- | `56481b16` | post-1.6.17 | Offline checker judges by the same direction-only predicate (§9.10/P5) |
707
+ | `d99b8685` | 1.6.18 | Harden partial-fill re-anchor and recovery (residual allow-list, tolerant load) |
708
+ | `531edeb4` | 1.6.18 | Keep the boundary on the placed rails; use live fill prices for geometry |
709
+ | `f4d6da8c` | 1.6.18 | Edge-pinned fills never arm the pivot or the anchor (§9.10/P1) |
710
+ | `a2bf6ba5` | 1.6.18 | Guard arity, explicit pivot re-arm, stray-aware rail clamp, crossing observation (§9.10/P2-P4) |
711
+ | `56481b16` | 1.6.18 | Offline checker judges by the same direction-only predicate (§9.10/P5) |
712
+ | `ddc4b4a3` | 1.6.18 | Count last-fill-excluded candidates per slot, not per pool visit |
713
+ | `a93f7eef` | 1.7.0 | Stale-grid re-buys: unsettled-fill gate, attribution, same-level refusal (§9.11.1) |
714
+ | `8705381b` | 1.7.0 | Orphan-fill attribution by price; dark-level revive (§9.11.1) |
715
+ | `8aff50bb` | 1.7.0 | Anchor-aware residual allow-list, pivot clock provenance, read-time TTL (§9.11.2) |
716
+ | `c493a604` | unreleased | Stranded-boundary repair without re-arming on replays (§9.11.3) |
717
+ | `ed52f9ef` | unreleased | Exclusive placement region; no create-only re-plan after a rebuild (§9.11.3) |
718
+ | `970fedfd` | unreleased | Remaining placement races; fill-accounting double-apply |
719
+ | `ad081e3d` | unreleased | Placement lifecycle + fill-marker idempotency |
720
+ | `7a3571a4` | unreleased | Follow-up gaps: one teardown, one lease check, one pending-broadcast policy |
721
+ | `94223d19` | unreleased | Re-verify the placement lease before a region holder broadcasts |
722
+ | `8bab9881` | unreleased | Unknown gap geometry is not gap 0 |
723
+ | `4d5e3d81` | unreleased | Forced retreats evacuate, then re-plan the committed band |
724
+ | `7618f299` | unreleased | Cancel teeth may not destroy the rotation that evacuates |
725
+ | `826d84a3` | unreleased | Chain existence unified behind the lifecycle registry (§9.11.3) |
726
+ | `7252d7d3` | unreleased | Broadcast uncertainty as a typed three-valued outcome (§9.11.3) |
727
+ | `84b935d9` | unreleased | One writer per life-cycle clock, one guard per absence decision (§9.11.3) |
680
728
 
681
729
  #### 9.6 Invariants added
682
730
 
@@ -698,8 +746,18 @@ required is no longer reachable at runtime.
698
746
  bounded by the same 24h TTL (enforced at restore and at read time, so a stale row can neither
699
747
  arm the guard nor be re-armed), and a re-center is requested at most once per pivot because a
700
748
  re-center cannot move it (§9.10/P2).
701
-
702
- #### 9.7 The guard was a band, not an economic rule (`c86b4b0e`)
749
+ - **An unabsorbed fill blocks the geometry:** while fills are queued, draining, in flight or
750
+ owed a crawl, the targeted reconcile and the window top-up defer (1:1 repairs still run),
751
+ and a fill is attributed by the ladder level of its execution price, not by a chain id that
752
+ virtualisation has already nulled (§9.11.1).
753
+ - **A repair must reduce the violation:** a boundary move is accepted only when it strands
754
+ strictly fewer placed orders than the committed value, counted by `validatePersistedBoundary`
755
+ itself; an equal count is refused (§9.11.3).
756
+ - **Existence is a record, not a lookup:** one per-slot lifecycle record owns the phase, the
757
+ evidence rank and every clock; `unknown` is a phase that never reads as "gone", and every
758
+ absence decision asks the same time-bounded guard (§9.11.3).
759
+
760
+ #### 9.7 The guard was a band, not an economic rule (`8e5e1a3c`)
703
761
 
704
762
  **Observed (2026-10-24/25, a live market-pair bot).** A dump swept the grid and the
705
763
  market then held. Spread correction planned, the broadcast guard dropped the creates,
@@ -820,7 +878,7 @@ must stay fund-driven from the side pool: a SELL fill returns quote while the
820
878
  same-side repair needs base, so no fill can ever fund its own side, and pairing
821
879
  would hide that asymmetry instead of resolving it.
822
880
 
823
- #### 9.10 Review pass over v1.6.12 → v1.6.17 (unreleased)
881
+ #### 9.10 Review pass over v1.6.12 → v1.6.17 (released in v1.6.18)
824
882
 
825
883
  A code review of the whole 1.6.12→1.6.17 order-engine arc (90 commits) against
826
884
  the three stated goals — buy-low/sell-high counter orders, never trading through
@@ -844,8 +902,99 @@ extremum extracted as `resolveFillAnchorIndex` (pure, batch-only).
844
902
 
845
903
  **Commits:** `f4d6da8c` (P1) · `a2bf6ba5` (P2–P4 plus the arity and the anchor
846
904
  extraction) · `56481b16` (P5) · `3d16f9e4` (this document, the doc-level invariants
847
- and the log rows it cites). **Unreleased:** no version bump and no `CHANGELOG.md`
848
- entry — both belong to the release step, not to this pass.
905
+ and the log rows it cites). **Released in v1.6.18:** the review pass itself carried no
906
+ version bump, but every commit above is an ancestor of the `v1.6.18` tag (2026-11-01).
907
+
908
+ #### 9.11 v1.6.18 → unreleased: fill identity, absence and the geometry budget
909
+
910
+ The post-1.6.17 work stopped asking "which guard is missing?" and started asking
911
+ "which *identity* is the code using?". Three answers landed: a fill is identified
912
+ by its ladder level, existence by one lifecycle record, and a repair by the
913
+ amount of violation it removes.
914
+
915
+ ##### 9.11.1 A fill the grid could not name (v1.7.0, `a93f7eef`/`8705381b`)
916
+
917
+ **Observed (2026-11-02, a live market-pair bot).** A multi-minute stale broadcast
918
+ region deferred the fill queue; the post-fill targeted reconcile then derived its
919
+ window from a master that still held the filled slots as ACTIVE and re-created
920
+ same-side orders at the top of the un-crawled rail. They filled within seconds:
921
+ 9 buy-lower violations, 23.5% of the window's BTS.
922
+
923
+ **The root cause sat upstream of the reconcile.** A fill whose order id no longer
924
+ sat on an ACTIVE/PARTIAL slot was dropped ("order not found in active grid")
925
+ *after* its accounting was booked, so neither the pivot nor the owed crawl
926
+ learned the real execution price and every downstream gate compared against a
927
+ stale, higher price. `virtualizeOrder` nulls `orderId`, so the very fills the
928
+ attribution was written for could never match by id.
929
+
930
+ **Four gates, one direction:**
931
+
932
+ - `hasUnsettledFills` defers the targeted reconcile and the window top-up while
933
+ fills are queued, draining, in flight or owed a crawl. 1:1 repairs
934
+ (cancelled-slot replacement, vacated-rail refill) still run, so a rail is never
935
+ left un-repaired.
936
+ - `attributeUnmatchedFillToGrid` resolves the slot by chain id when a virtualised
937
+ slot still owns it, else by the **genesis level of the execution price**,
938
+ records pivot + owed crawl, and refuses a price outside the frozen rail so a
939
+ shelf/manual order can never arm the geometry.
940
+ - `isLastFillPlacementBlocked` (direction **plus** same grid level) gates every
941
+ reconcile placement, and `_pickVirtualSlotsToActivate` uses the same predicate,
942
+ so the pick budget rolls to the next level instead of being spent on a level
943
+ the create would refuse.
944
+ - A slot virtualised while its order stayed live on chain is revived in place
945
+ (`reviveVirtualSlotFromChain`, level-gated, master-only), a genuinely stale id
946
+ on a non-placed slot is cleared without booking a fill, and CREATE occupancy is
947
+ keyed on the id alone (any state) rather than on `isOrderOnChain()`.
948
+
949
+ ##### 9.11.2 The pivot is a row, not a price (v1.6.18/v1.7.0, `d99b8685`/`8aff50bb`)
950
+
951
+ Three defects of the same shape:
952
+
953
+ - The in-band residual allow-list that lets a fill's own PARTIAL survive a re-anchor was
954
+ keyed on the last-processed pivot instead of the batch **extremum**, so for every chunked
955
+ sweep whose furthest fill was not the pivot the re-anchor was refused as
956
+ `unsafe: placed_order_in_band` — the exact shape the extremum resolution exists for.
957
+ - A market-row price could inherit a book-seed timestamp and then re-stamp it, refreshing the
958
+ 24h lifetime on every rebuild until the row never expired.
959
+ - The TTL existed only at restore, so a long-lived process kept an expired pivot vetoing one
960
+ side forever while the geometry watchdog claimed a fill or the TTL would restore it.
961
+
962
+ Price, side and clock now travel as one row, and the TTL is enforced where the reference
963
+ is **read**; an unknown age never expires.
964
+
965
+ ##### 9.11.3 Absence gets one record; repairs must reduce the violation (unreleased)
966
+
967
+ Chain existence used to be answered by seven structures with seven clocks
968
+ (`_pendingBroadcasts`, `createUncertain`, `_committedOrderIds`,
969
+ `_orderIdAssignedAt`, `_placedAt`, `_chainSizeAppliedFills`,
970
+ `_lastUnmatchedChainOrders`). Four read sites consulted four subsets and could
971
+ return four different answers for the same order, so each fix closed one site and
972
+ the bug reappeared at the next.
973
+
974
+ `modules/order/lifecycle.ts` (`826d84a3`, `7252d7d3`, `84b935d9`) now owns one
975
+ record per slot: a six-phase vocabulary in which `unknown` is a real state (a
976
+ lost receipt is *not* an absence), a monotonic evidence rank, and every clock
977
+ recorded once. The legacy maps survive as live views, so no call site had to be
978
+ rewritten to get one truth. Three rules fall out of it: one absence guard
979
+ (`isInFlight()`, time-bounded), one teardown authority for a hung region
980
+ (`_clearStalePlacementRegion`, `7a3571a4`), and one pending-broadcast liveness
981
+ policy (`getLivePendingBroadcasts`).
982
+
983
+ Geometry gained the matching rule. A boundary writer may move the boundary only
984
+ when the move strands **strictly fewer** placed orders than the committed value
985
+ (`boundaryMoveReducesStrands`, counted by `validatePersistedBoundary` itself); an
986
+ equal count stays refused, and a replayed fill no longer re-arms the pivot
987
+ (`c493a604`). The same arc put every chain-mutating path inside one exclusive
988
+ placement region (`modules/order/placement_lock.ts`, `ed52f9ef`/`970fedfd`) after
989
+ a trigger-file rebuild and a fill-driven rebalance interleaved into 25
990
+ duplicate-price orders; made a forced gap-evacuation retreat evacuate before
991
+ re-planning instead of letting its cancel teeth destroy the rotation (`7618f299`,
992
+ `4d5e3d81`); and stopped an absent gap band from reading as a valid `0`
993
+ (`8bab9881`).
994
+
995
+ **Why it belongs in this document:** R3.5's "one rule, several implementations"
996
+ is not a style complaint. Every family above is that pattern applied to a
997
+ different noun — price, fill, existence.
849
998
 
850
999
  ---
851
1000
 
package/docs/README.md CHANGED
@@ -2,7 +2,7 @@
2
2
 
3
3
  This directory contains the comprehensive technical documentation for the DEXBot2 trading bot. It is designed to guide developers from high-level architecture down to the nuances of fund accounting and state management.
4
4
 
5
- **Version context:** v1.7.0 (released).
5
+ **Version context:** v1.7.1 (released).
6
6
 
7
7
  ---
8
8
 
@@ -131,9 +131,22 @@ This directory contains the comprehensive technical documentation for the DEXBot
131
131
  ### 🔒 [COW Invariants](COW_INVARIANTS.md)
132
132
  *Stable theory contract for COW pipeline.*
133
133
  - **Non-negotiable invariants**: Master immutability, commit atomicity, projection rules, accounting separation
134
- - **Subsystem Scope**: 13 `INV-*` prefix groups mapping each invariant to its runtime subsystem
134
+ - **Subsystem Scope**: 15 `INV-*` prefix groups mapping each invariant to its runtime subsystem
135
135
  - **Change Policy**: Required steps for an intentional invariant change — same-PR doc update, rationale, and regression tests
136
136
 
137
+ ### 🧭 [Order Lifecycle Registry](ORDER_LIFECYCLE.md)
138
+ *How the engine answers one question from one record: does this order exist on chain, and how sure are we?*
139
+ - **The problem**: seven structures with seven clocks, four read sites consulting four different subsets of them
140
+ - **The model**: one record per slot, six phases, `unknown` as a real state that never reads as "gone"
141
+ - **The wiring**: legacy maps are live views onto the record; every mutation funnels through `_applyOrderUpdate`
142
+ - **Invariants**: `INV-LIFE-001…007` in [COW_INVARIANTS.md](COW_INVARIANTS.md#order-lifecycle-order_lifecyclemd)
143
+
144
+ ### ⏱ [Timing Window Inventory](TIMING_INVENTORY.md)
145
+ *Every timing knob classified, with the self-healing ones marked as removal targets.*
146
+ - **The audit**: `external` / `bound` / `policy` / `self-healing`, each with a rationale
147
+ - **The gate**: `tests/test_timing_policy.ts` fails on an unclassified window, so the inventory can only shrink by decision
148
+ - **Removal targets**: the six windows that compensate for our own lost state, and the evidence that replaces each
149
+
137
150
  ### 📐 [Grid-Price Invariant](GRID_PRICE_INVARIANT.md)
138
151
  *Why a slot's emitted price must equal its genesis level — and how that failed.*
139
152
  - **The invariant**: `order.price === priceForSlot(idx, genesis)`, and why range guards cannot substitute for it
@@ -169,6 +182,14 @@ This directory contains the comprehensive technical documentation for the DEXBot
169
182
  - **Practical How-Tos**: Adding features step by step, common pitfalls to avoid, and useful debugging commands.
170
183
  - **Glossary**: Definitions of project-specific terminology (e.g., "VIRTUAL order state", "Rotation", "Pipeline Safety", "WorkingGrid", "Atomic Commit", "Dynamic Weight", "Regime Detection").
171
184
 
185
+ ### 🌉 [Liquidity Mirror Design](LIQUIDITY_MIRROR_DESIGN.md)
186
+ *Design paper for a cross-venue liquidity mirror running alongside DEXBot2.*
187
+ - **Roles**: Chain host (TS, DEXBot2) owns identity, pool reads, and hedging; a separate venue daemon owns CEX credentials, the order book, and the rate-limit budget
188
+ - **Contract**: Intent-level, HMAC-authenticated NDJSON over a unix socket — desired book in, fills and positions out; no message whose retry can duplicate an order
189
+ - **Change surface**: Four new modules under `modules/venues/`, a `cex_mirror/` tree, and five mechanical edits; the grid, sync engine, accounting, and strategy engine are untouched
190
+ - **Guardrails**: Spread floor from pool fee plus impact, hedge batching trade-off, deadman cancellation, divergence breaker, and a split inventory-cap model
191
+ - **Rollout**: Milestones M0–M5 with acceptance gates, from protocol conformance vectors to capped live hedging
192
+
172
193
  ### 🔄 [Workflow](WORKFLOW.md)
173
194
  *How we build and release.*
174
195
  - **Branching Strategy**: Explanation of the `test` → `dev` → `main` lifecycle.
package/docs/WORKFLOW.md CHANGED
@@ -250,8 +250,8 @@ the canonical name is preferred in scripts and docs.
250
250
  | `dexbot update` | — | Update DEXBot2 from the repository and restart active bots |
251
251
  | `dexbot export <bot>` | — | Export bot trades/settings to CSV/JSON for local analysis/ |
252
252
  | `dexbot order` | `orders` | Analyze persisted order grids (spread, increment, funds); `--export` for HTML |
253
- | `dexbot order [<bot>]` | — | Analyze only the specified bot's order grid |
254
- | `dexbot credit` | — | Live summed MPA + borrowed-credit positions per asset per bot (`[<bot>]`) |
253
+ | `dexbot order [<bot|account>]` | — | Analyze only the specified bot's order grid — or every grid on that account (`<bot|account>`) |
254
+ | `dexbot credit` | — | Live summed MPA + borrowed-credit positions per asset per bot (`[<bot|account>]` — a bot name/key, or the account the bot trades from, name or `1.2.x` id) |
255
255
  | `dexbot rich <asset...>` | — | Top holder(s) per asset via the chain asset-holders API (`--top N`, `--json`/`--csv`) |
256
256
  | `dexbot tv <target>` | — | TradingView 1h chart for `<bot|pool|AssetA/AssetB>` over `--month N` (default 3); pool = id (133 / 1.19.133) or pool token (LP share-asset symbol) |
257
257
  | `dexbot dw <target>` | — | Dynamic-weight research chart: same targets/flags as `tv` |
@@ -313,7 +313,13 @@ npm run pmain
313
313
  |--------|---------|-------------|-----------|
314
314
  | `npm run ptest` | Push test to origin | Checks out `test` (if not on it) and pushes local commits to `origin/test`. Does **not** touch `dev`/`main`. | Daily development; ensures `origin/test` is up-to-date |
315
315
  | `npm run pdev` | Mirror test onto dev | Pushes `origin/test`, then `git push origin test:dev --force` and updates the local `dev` pointer. No merge commit. | When test is stable and you want `dev` to exactly equal `test` |
316
- | `npm run pmain` | Mirror test onto dev and main | Pushes `origin/test`, then force-pushes `test` to both `dev` and `main` and updates local pointers. **No tagging.** | Full release where `test == dev == main` is intended. Tag manually afterward (see step 5) |
316
+ | `npm run pmain` | Mirror test onto dev and main | Pushes `origin/test`, then force-pushes `test` to both `dev` and `main` in one `--atomic` push and updates local pointers. **No tagging.** | Full release where `test == dev == main` is intended. Tag manually afterward (see step 5) |
317
+
318
+ All three scripts run a shared preflight before touching anything
319
+ (`scripts/lib/git_preflight.sh`): a read-only `git ls-remote` against `origin`.
320
+ An unreachable remote or a rejected SSH key now fails up front instead of
321
+ mid-script, and `pmain`'s `--atomic` push keeps `dev`/`main` together — a
322
+ refusal moves neither. Bypass the check with `DEXBOT_SKIP_GIT_PREFLIGHT=1`.
317
323
 
318
324
  Note: `pmain` does **not** create a release tag. Run `git tag -a v0.X.Y -m "..."` manually after `pmain` if you need a tagged release point.
319
325