dexbot 1.6.3 → 1.6.5

package/docs/COPY_ON_WRITE_MASTER_PLAN.md

@@ -78,6 +78,13 @@ NORMAL → REBALANCING → BROADCASTING → _commitWorkingGrid() → NORMAL
 - Any master mutation marks the working grid stale.
 - Commit succeeds only when stale/version/delta guards all pass.
 
+### Atomic Boundary Shifts
+
+Boundary index changes during divergence correction are atomic with slot-type reassignment:
+`pendingBoundaryIdx` carries boundary changes through the COW pipeline and `manager.boundaryIdx`
+is untouched until `_commitWorkingGrid` completes. This prevents transient mismatches between
+boundary position and slot BUY/SELL roles during blockchain execution.
+
 ## Data Flow
 
 ### Normal Rebalance Flow
@@ -215,145 +222,6 @@ syncFromMaster(masterGrid, orderId, masterVersion?) {
 - Prevents stale data from being committed.
 - Avoids unnecessary aborts for individual fills.
 
-## Historical Context: Immutable Master Grid Evolution
-
-The COW architecture evolved from earlier attempts to achieve grid immutability. The three
-eras below describe design evolution over time; they are **distinct from the Build Steps**
-numbered 0–9 in the *Implementation Status* section (Build Steps track the construction
-work that produced the current code).
-
-### Era 0: Original Optimistic State (Pre-v1.0) — ❌ Removed
-- **Approach:** Direct in-memory mutation of master grid during planning.
-- **Pattern:** Modify master directly → Broadcast to blockchain → No recovery mechanism.
-- **Vulnerability:** State corruption during any failure, no isolation between planning and committed state, no rollback capability.
-- **Incident:** This approach caused the Price Jump incident — a sudden market move corrupted in-flight grid state because planning mutations were applied directly to the master grid, with no isolation or rollback.
-
-### Era 1: Frozen Master State (v1.0) — ✅ Retained as defense-in-depth
-- **Approach:** `Object.freeze()` on Maps and order objects.
-- **Implementation:** Each `_applyOrderUpdate` creates a new frozen Map via immutable-swap pattern.
-- **Advantage:** Runtime enforcement prevents accidental mutations; catches bugs that read `manager.orders` and mutate in-place.
-- **Original concern:** Performance overhead, complexity in deep-freezing nested structures.
-
-### Era 2: Copy-on-Write (v2.0 — Current) — ✅ Production-ready
-- **Approach:** Working copy during planning, atomic swap on blockchain confirmation.
-- **Pattern:** Clone → Modify working copy → Broadcast → Commit on success / Discard on failure.
-- **Advantage:** True transactional semantics; master never in intermediate state; cleaner than snapshot/rollback.
-
-The production implementation uses both Era 1 (`Object.freeze`) and Era 2 (COW) as
-complementary layers; see **Freeze + COW Hybrid** under *Architecture* above.
-
-## Implementation Status
-
-Build Steps track the construction work that produced the current code; they are distinct
-from the *Era* numbering used in *Historical Context*.
-
-### Build Step 0: Dependencies ✅
-Dependency utilities merged into `modules/order/utils/order.ts` during v0.6.0-patch.19 consolidation.
-
-### Build Step 1: Infrastructure ✅
-- Created `modules/order/working_grid.ts` — WorkingGrid class.
-- Added `COW_PERFORMANCE` thresholds to `modules/constants.ts`.
-
-### Build Step 2: Core Integration ✅
-- `performSafeRebalance()` → delegates to `_applySafeRebalanceCOW()`.
-- `_applySafeRebalanceCOW()` — creates working grid, runs planning, returns result without modifying master.
-- `WorkingGrid.buildDelta()` — delta reconciliation against working copy (`modules/order/working_grid.ts:204`, delegating to `utils/order.ts`).
-- `_commitWorkingGrid()` — atomic swap from working to master.
-
-### Build Step 3: Broadcast Integration ✅
-- `updateOrdersOnChainBatch()` — routes to COW path when `workingGrid` present.
-- `_updateOrdersOnChainBatchCOW()` — full COW broadcast with commit on success.
-- Removed legacy rollback code.
-
-### Build Step 4: Fill Handling Strategy ✅
-**Decision:** "Selective abort — continue individual fills, block full-side updates."  
-Full design is documented in the **Fill Handling Strategy** section above.
-
-### Build Step 5: Tests ✅
-- `tests/test_cow_master_plan.ts` — 11 COW core tests.
-- `tests/test_cow_commit_guards.ts` — commit guard regression tests.
-- `tests/test_cow_concurrent_fills.ts` — concurrent fill integration tests.
-- `tests/test_cow_divergence_correction.ts` — divergence correction COW tests.
-- `tests/test_cow_orchestration_fixes.ts` — COW orchestration fixes.
-- `tests/test_cow_structural_resync.ts` — structural grid resync tests.
-- `tests/test_cow_static_analysis.ts` — static analysis and invariant checks.
-- `tests/test_cow_index_mutation_detection.ts` — index mutation detection.
-- `tests/test_cow_fund_validation_precision.ts` — fund validation precision tests.
-- `tests/test_cow_set_mutation_report.ts` — set mutation report tests.
-- `tests/test_sync_lock_routing.ts` — lock routing verification tests.
-- `tests/test_working_grid.ts` — WorkingGrid unit tests.
-
-### Build Step 6: Divergence & Cache Updates ✅
-Divergence checks and cache function updates only execute when no fills are pending. See
-**Fill Handling Strategy → Divergence & Cache Checks Blocked During Rebalance**.
-
-### Build Step 7: Divergence Correction COW Migration ✅
-Migrated `applyGridDivergenceCorrections` from queue-based cancellations to full COW pattern.
-
-**Atomic Boundary Shifts (Patch 20):** Boundary index changes during divergence correction are
-now atomic with slot-type reassignment. The `pendingBoundaryIdx` variable carries boundary
-changes through the COW pipeline without touching `manager.boundaryIdx` until
-`_commitWorkingGrid` completes. This prevents temporary mismatches between boundary position
-and slot BUY/SELL roles during blockchain execution.
-
-```javascript
-// Boundary changes flow through COW pipeline atomically
-const boundarySync = syncBoundaryToFunds(manager);  // Returns { changed, newIdx }
-if (boundarySync.changed) {
-    pendingBoundaryIdx = boundarySync.newIdx;  // NOT manager.boundaryIdx!
-    // updateGridFromBlockchainSnapshot reassigns slot types in WorkingGrid
-    // manager.boundaryIdx updated atomically in _commitWorkingGrid
-}
-```
-
-**Before (Queue-Based):**
-```javascript
-// Detect divergence → Queue corrections → Execute batch → Clear queue
-// Master grid stays ACTIVE during entire process (race condition)
-ordersNeedingPriceCorrection.push({ gridOrder, chainOrderId, isSurplus: true });
-// ...later...
-await updateOrdersOnChainBatchFn({ ordersToCancel, ordersToPlace, ordersToRotate });
-```
-
-**After (COW-Based):**
-```javascript
-// Detect divergence → Create WorkingGrid → Update sizes in working copy
-// → Execute UPDATE/CANCEL/CREATE ops on chain → Commit working grid on success
-const workingGrid = new WorkingGrid(manager.orders);
-workingGrid.set(orderId, convertToSpreadPlaceholder(order)); // Surplus → virtual slot
-const actions = [{ type: COW_ACTIONS.CANCEL, id, orderId }, ...];
-const cowResult = { actions, workingGrid, ... };
-await updateOrdersOnChainBatch(cowResult); // Commit only on success
-```
-
-**Key Changes:**
-1. **Surplus orders**: `CANCEL` on-chain and virtualize in working grid.
-2. **State preservation**: `ACTIVE`/`PARTIAL` orders keep their state in working grid.
-3. **No race conditions**: master unchanged until blockchain confirms.
-4. **Unified flow**: same COW pattern as fill rebalancing.
-
-**Grid Resizing Also Migrated:** `updateGridFromBlockchainSnapshot` now returns a COW result:
-
-```javascript
-// Before: Modified master grid directly
-await Grid.updateGridFromBlockchainSnapshot(manager, 'buy'); // Direct update!
-
-// After: Returns COW result for batch execution
-const cowResult = await Grid.updateGridFromBlockchainSnapshot(manager, 'buy');
-await updateOrdersOnChainBatch(cowResult); // Execute via COW
-```
-
-### Build Step 8: Benchmarks ✅
-- 100 orders: ~0.03ms clone
-- 500 orders: ~0.05ms clone
-- 1000 orders: ~0.08ms clone
-- 5000 orders: ~0.5ms clone
-
-### Build Step 9: Cleanup ✅
-- Removed snapshot/rollback pattern; `performSafeRebalance()` now delegates to `_applySafeRebalanceCOW()`.
-- Removed duplicate `_updateOrdersOnChainBatchCOW`.
-- Removed legacy rollback references in `dexbot_class.ts`.
-
 ## Key Methods
 
 ### OrderManager (`modules/order/manager.ts`)
@@ -411,78 +279,13 @@ This architecture makes the "Metadata Reinterpretation" bug impossible by ensuri
 
 ---
 
-## Appendix A: Files
-
-### Files Created
-- `modules/order/working_grid.ts` — WorkingGrid class (COW wrapper with clone/delta/stale tracking).
-- Test files listed under **Build Step 5**.
-- *Dependency utilities consolidated into `modules/order/utils/order.ts` during v0.6.0-patch.19.*
-
-### Files Modified
-- `modules/constants.ts` — added `COW_PERFORMANCE` thresholds.
-- `modules/order/manager.ts` — added COW methods, immutable master (`Object.freeze`), version tracking.
-- `modules/dexbot_class.ts` — wired COW broadcast, removed legacy rollback.
-- `modules/order/sync_engine.ts` — uses `_applyOrderUpdate` (lock-free) for all sync paths.
-- `modules/order/grid_reconcile.ts` — uses `_applySync` (lock-free) when inside `_gridLock`.
-- `modules/order/utils/system.ts` — migrated `applyGridDivergenceCorrections` to full COW pattern.
-- `modules/order/grid.ts` — migrated `updateGridFromBlockchainSnapshot` to return COW result instead of modifying master directly.
-
-## Appendix B: Test Results
-
-```
-Core COW Tests (test_cow_master_plan.ts):
-  ✓ COW-001: Master unchanged on failure
-  ✓ COW-002: Master updated only on success
-  ✓ COW-003: Index transfer
-  ✓ COW-004: Fund recalculation
-  ✓ COW-005: Order comparison
-  ✓ COW-006: Delta building
-  ✓ COW-007: Index validation
-  ✓ COW-008: Working grid independence
-  ✓ COW-009: Empty grid handling
-  ✓ COW-010: Memory stats
-  ✓ COW-011: No spurious updates on unchanged grid
-
-Commit Guard Tests (test_cow_commit_guards.ts):
-  ✓ COW-COMMIT-001: Version mismatch rejection
-  ✓ COW-COMMIT-002: Empty delta rejection
-
-Concurrent Fill Tests (test_cow_concurrent_fills.ts):
-  ✓ COW-FILL-001: Fill during REBALANCING syncs to working grid
-  ✓ COW-FILL-002: Fill during BROADCASTING syncs to working grid
-  ✓ COW-FILL-003: Commit rejected after fill during broadcast
-  ✓ COW-FILL-004: No working grid sync during NORMAL state
-  ✓ COW-FILL-005: _cloneOrder deep-clones rawOnChain
-  ✓ COW-FILL-006: _cloneOrder handles missing rawOnChain
-  ✓ COW-FILL-007: Staleness reason includes phase context
-
-Divergence Correction Tests (test_cow_divergence_correction.ts):
-  ✓ Surplus orders are CANCELLED (not UPDATE to size=0)
-  ✓ Working grid preserves order states (ACTIVE, PARTIAL)
-  ✓ Orders within target window get size updates
-  ✓ No duplicate UPDATE/CANCEL overlap for same order
-
-Stale-Plan & Stack Discipline Tests (v1.4.8):
-  ✓ test_cow_guard_replan.ts — bounded re-plan from fresh master, boundary-budget
-    restore, push-marker contract, no double-pop on re-plan failure
-  ✓ test_cow_stale_slot_guard.ts — slot-id based stale-placement veto,
-    boundary-only semantics, rotation-UPDATE coverage
-  ✓ test_cow_commit_guards.ts — empty-action / never-pushed marker contract
-  ✓ test_uncertain_broadcast.ts — verify-before-retry per op kind, truncated-read
-    deferral, pending-broadcast protection kept on ambiguous reads
-```
-
-**Additional Checks:**
-- Unchanged grids do not emit global COW `update` actions.
-- Missing on-chain `ACTIVE` order with `orderId` appears in `filledOrders` from open-order sync.
-
-## Appendix C: Constants Added (`modules/constants.ts`)
+## Appendix: COW Constants (`modules/constants.ts`)
 
 ### COW Performance Thresholds
 - `COW_PERFORMANCE.MAX_REBALANCE_PLANNING_MS` — planning-phase duration above which a slow-plan warning is logged (100ms).
 - `COW_PERFORMANCE.GRID_MEMORY_WARNING` — working grid size (bytes) that triggers a memory warning (5,000).
 - `COW_PERFORMANCE.WORKING_GRID_BYTES_PER_ORDER` — estimated memory per order (500 bytes).
-- `COW_PERFORMANCE.MAX_OPS_PER_BROADCAST` — *removed*; the per-broadcast operation cap is now derived from the grid gap-slot count (`DEXBot._getGapSlotBatchSize`); larger batches are split into sequential broadcasts.
+- `COW_PERFORMANCE.MAX_OPS_PER_BROADCAST` — *removed*; the per-broadcast operation cap is now derived from the grid gap-slot count + 1 (`DEXBot._getGapSlotBatchSize`); larger batches are split into sequential broadcasts.
 
 ### Pipeline Timing
 - `PIPELINE_TIMING.RECOVERY_DECAY_FALLBACK_MS` — recovery decay fallback (180 seconds).
@@ -491,7 +294,7 @@ Stale-Plan & Stack Discipline Tests (v1.4.8):
 - `GRID_LIMITS.RELATIVE_ORDER_UPDATE_THRESHOLD_PERCENT` — relative threshold (%) for in-memory COW order equality checks.
 - `TIMING.LOCK_REFRESH_MIN_MS` — minimum lock refresh interval (250ms).
 
-## Appendix D: Validation Gates
+## Appendix: Validation Gates
 
 Run these tests before promotion:
 - `node dist/tests/test_engine_integration.js`

package/docs/COW_INVARIANTS.md

@@ -79,11 +79,11 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
 - `INV-PROJ-002` Preserve on-chain PARTIAL size in projection
   - If identity is retained (`keepOrderId=true`) and current state is `PARTIAL`, projected size must preserve current on-chain remaining size.
   - It must not be overwritten by ideal geometric `targetSize`.
-  - Exception: a `PARTIAL` with a rotation/size-update action targeting its `orderId` does use `targetSize` (the explicit-UPDATE path at `modules/order/utils/validate.ts:959`).
+  - Exception: a `PARTIAL` with a rotation/size-update action targeting its `orderId` does use `targetSize` (the explicit-UPDATE path at `modules/order/utils/validate.ts:1129`).
   - Preserve-path size must be normalized to finite, non-negative value.
 
 - `INV-PROJ-003` ACTIVE on-chain projection preserves current size (same as PARTIAL)
-  - If identity is retained and state is `ACTIVE`, projection preserves current on-chain size via the same `shouldPreserveSize` path as `PARTIAL` (`validate.ts:958-972`).
+  - If identity is retained and state is `ACTIVE`, projection preserves current on-chain size via the same `shouldPreserveSize` path as `PARTIAL` (`validate.ts:1129`).
   - An explicit UPDATE action targeting the `orderId` is required to apply `targetSize`.
 
 - `INV-ID-001` Order identity retention rule
@@ -102,7 +102,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
 - `INV-ACC-003` Cross-bot fund registry invariant (INVARIANT 3)
   - Shared-account per-bot commitment must not exceed the bot's proportional share of chain balance.
   - Checked with widened tolerance `max(PERCENT_TOLERANCE * 3, 0.15)`.
-  - Registry failure logs an error (`accounting.ts:554-563`, with a "CRITICAL FIX: Log as ERROR instead of WARN" comment), not a silent skip.
+  - Registry failure logs an error (`order/accounting.ts:574-590`, with a "CRITICAL FIX: Log as ERROR instead of WARN" comment), not a silent skip.
 
 ---
 
@@ -195,6 +195,13 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
   - Top-of-window partials remain always eligible.
   - Two PARTIALs sharing a price with no active sibling do not qualify (left to rebalancer).
 
+- `INV-GRID-004` Slot price equals its genesis level ([GRID_PRICE_INVARIANT.md](GRID_PRICE_INVARIANT.md))
+  - `order.price` for a slot-`idx` order must equal `priceForSlot(idx, genesis)`; the genesis ladder is the only authoritative price for a slot.
+  - Enforced at all six emission sites (CREATE / UPDATE / CREATE-FALLBACK, RECONCILE-CREATE / RECONCILE-UPDATE, STARTUP-CREATE): an off-grid emission is blocked, never broadcast.
+  - Range guards (`isChainPriceOutOfGrid`) are bounds checks, not membership checks — they cannot substitute for this invariant.
+  - Adoption keeps the slot's own level (a fill/chain price is metadata, not the slot's price); `loadGrid` repairs a pre-existing off-grid slot price at load.
+  - Tests: GPI-001..015 (`tests/test_grid_price_invariant_guard.ts`), GPI-WIRE-001..009 (`tests/test_grid_price_invariant_wiring.ts`), LEGACY-ADOPT/MATERIALIZE/ADOPT-NAME (`tests/test_sync_out_of_grid_defer.ts`).
+
 ---
 
 ## Reconcile ([GRID_RECONCILE.md](GRID_RECONCILE.md))
@@ -208,18 +215,17 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
   - Dust health thresholding applies consistently to both CREATE and rotation destination holes.
 
 - `INV-RECON-003` Reconcile cancels duplicate chain orders unconditionally
-  - When an unmatched order is within `looseTolerance` of an active grid order, it must be cancelled on chain via `_cancelChainOrder` with `releaseUntrackedFunds: true`.
+  - An unmatched chain order whose price equals an active same-type grid slot's price (exact slot-price equality via `priceSlotEqual` at the asset precision) is a suspected duplicate and must be cancelled on chain via `_cancelChainOrder` with `releaseUntrackedFunds: true`.
   - Cancelled IDs are filtered out of `unmatchedParsed` to prevent reprocessing.
   - No size guard — any duplicate at the same price is a violation.
-  - `SUSPECTED_DUPLICATE_TOLERANCE_FLOOR` (absolute price floor) is removed — only `tolerance * SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER` is used.
-  - `SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER` is a file-local constant (`modules/order/grid_reconcile.ts`, value `5`), not a centralized `constants.ts` entry.
+  - The earlier fuzzy `SUSPECTED_DUPLICATE_TOLERANCE_MULTIPLIER` (5× `calculatePriceTolerance`) and `SUSPECTED_DUPLICATE_TOLERANCE_FLOOR` are removed; only exact price-level equality triggers a reconcile cancel.
 
 - `INV-RECON-004` Rebalance must not convert on-chain slots to SPREAD via CREATE
   - `performSafeRebalance` must not emit `CREATE` actions that convert existing on-chain slots into SPREAD orders.
   - On-chain mid-slot must keep its BUY/SELL type before commit.
 
 - `INV-RECON-005` Extreme placement ordering
-  - BUY placements must use nearest available free slots first (descending price, so the nearest-to-center slots fill first — `validate.ts:465-468`).
+  - BUY placements must use nearest available free slots first (descending price, so the nearest-to-center slots fill first — `order/utils/order.ts` `buildOutsideInPairGroups`).
   - SELL placements must use nearest available free slots first (ascending price).
 
 ---
@@ -228,7 +234,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
 
 - `INV-BATCH-001` Illegal state batch abort
   - `executeBatch` throws `ILLEGAL_SPREAD_STATE` on an illegal grid layout (emitted at `modules/order/utils/validate.ts`, propagated via `modules/order/manager.ts` `_throwOnIllegalState`).
-  - The `_handleBatchHardAbort` catch for `ILLEGAL_ORDER_STATE` (`dexbot_state_recovery.ts:133`) is a test-only dead branch — production never emits that code; only the test stub at `tests/test_patch17_invariants.ts:396` uses it.
+  - The `_handleBatchHardAbort` catch for `ILLEGAL_ORDER_STATE` (`dexbot_class.ts:455`) is a test-only dead branch — production never emits that code; only a test stub uses it.
   - In production, recovery + cooldown are armed on the next maintenance tick via `_abortFlowIfIllegalState` (the `INV-MAINT-002` path), returning `abortedForIllegalState: true` to the caller. The caller does not need to return immediately; the maintenance tick handles recovery.
   - Hard abort triggers one immediate recovery sync (`_triggerStateRecoverySync`) plus arms one maintenance cooldown cycle (`_maintenanceCooldownCycles = Math.max(current, 1)`).
 
@@ -247,7 +253,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
     - NOT virtualize the slot.
     - Preserve `orderId` until sync reconciles it.
     - NOT mark the order as stale-cleaned.
-  - Fast path: if the batch result indicates `ORDER_SIZE_DRIFT_TARGETED` (`dexbot_state_recovery.ts:263`), a targeted repair applies the correction directly and skips `_triggerStateRecoverySync`.
+  - Fast path: if the batch result indicates `ORDER_SIZE_DRIFT_TARGETED` (`dexbot_state_recovery.ts:269`), a targeted repair applies the correction directly and skips `_triggerStateRecoverySync`.
 
 ---
 
@@ -271,7 +277,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
 - `INV-REG-001` Cross-bot allocation ≤ proportional share
   - Per-bot committed amounts (sum of on-chain orders) must not exceed `totalChainBalance × allocatedPercent`.
   - Violation triggers an error-level log entry (not silent), with tolerance `max(PERCENT_TOLERANCE * 3, 0.15)`.
-  - Registry registration is pre-flight + atomic; only shared-account bots register (`dexbot.ts:535` filters `accountGroups[a].length > 1`), and registration completes before any shared-account bot starts.
+  - Registry registration is pre-flight + atomic; only shared-account bots register (`dexbot.ts:611` filters `accountGroups[a].length > 1`), and registration completes before any shared-account bot starts.
   - Release happens in `DEXBot.shutdown`.
 
 - `INV-REG-002` Async-locked registry writes

package/docs/DEXBOT_COMPARISON.md

@@ -1,7 +1,7 @@
 # DEXBot vs DEXBot2 — Detailed Comparison Report
 
 > **Date:** 2026-09-02 *(metrics refreshed against local source trees)*
-> **Scope:** Full architectural, functional, and operational comparison between the original [DEXBot](https://github.com/Codaone/DEXBot) (Python, v1.0.0) and DEXBot2 (TypeScript, v1.6.3).
+> **Scope:** Full architectural, functional, and operational comparison between the original [DEXBot](https://github.com/Codaone/DEXBot) (Python, v1.0.0) and DEXBot2 (TypeScript, v1.6.5).
 > **Audience:** Developers, contributors, and operators evaluating or migrating between the two projects.
 
 ---
@@ -36,7 +36,7 @@
 
 | Attribute | DEXBot (original) | DEXBot2 |
 |---|---|---|
-| **Release Track** | 1.0.0 | v1.6.3 |
+| **Release Track** | 1.0.0 | v1.6.5 |
 | **Language** | Python 3.6+ | TypeScript 5.x |
 | **Status** | Released 1.0.0, unmaintained | Active development |
 | **Last Repo Activity** | May 23, 2020 | 2026-09-02 |
@@ -776,7 +776,7 @@ Where:
 
 | Metric | DEXBot | DEXBot2 |
 |---|---|---|
-| **Release Track** | 1.0.0 | v1.6.3 |
+| **Release Track** | 1.0.0 | v1.6.5 |
 | **Active Since** | ~2018 | December 2025 |
 | **Last Commit** | May 23, 2020 | 2026-09-02 |
 | **Total Commits** | 2281 | 2,125 (v1.4.25) |

package/docs/EVOLUTION.md

@@ -2,14 +2,19 @@
 
 ## Executive Summary
 
-DEXBot2 is a sophisticated decentralized exchange trading bot for the BitShares blockchain. This report documents the complete evolution of the project from its inception in December 2025 through the current 1.6.3 stable release.
+DEXBot2 is a sophisticated decentralized exchange trading bot for the BitShares blockchain. This report documents the complete evolution of the project from its inception in December 2025 through the current 1.6.5 stable release.
 
 ### Key Milestones
 - **Project Inception**: December 2, 2025
- - **Growth Phase**: 2,227 commits over ~9 active months
+- **Growth Phase**: 2,263 commits over ~9 active months
 - **Code Maturity**: Evolution from basic utilities to a ~100,000+ LoC intelligent TypeScript system
-- **Stability**: Progression from manual testing to a suite of 294 automated test files
-- **Releases**: 107 release entries (v0.1.0 to v1.6.3)
+- **Stability**: Progression from manual testing to a suite of 309 automated test files
+- **Releases**: 109 release entries (v0.1.0 to v1.6.5)
+
+> **Post-1.0.0 "why":** the thematic story behind the hardening releases — root cause, recurring
+> bug families, and lessons — lives in
+> [ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md](ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md).
+> This report stays chronological.
 
 ---
 
@@ -61,10 +66,62 @@ v1.4.12 completed the module transition to native ES modules (root + claw `"type
 
 ### Phase 9: Post-ESM Cleanup, Consolidation & Hardening (August 2026)
 
-The post-ESM releases consolidated state, code, and tooling while hardening the grid engine. **State & packaging** (v1.4.14–v1.4.17): all user/runtime state centralized on a resolver-derived profiles dir (`~/.config/dexbot2/profiles`) safe from re-clones, read-only prefixes, and npm wipes; divergence surplus/hole pairs became in-place order rotations; npm auto-update shipped; duplicated EC-crypto/settings/asset-resolution code collapsed; dead exports purged; analysis tooling moved under strict TypeScript. **Grid hardening** (v1.4.19–v1.4.21): COW broadcasts capped at `MAX_OPS_PER_BROADCAST` (4) with chunked retry-on-uncertain; boundary promotion and persisted-boundary restore gated against gap-floor overrun poison behind a shared sell-rail ceiling enforced at commit and restore time; spread-collapse fixed via the shared `isSlotInRail` filter; silent-failure runtime defects from a modules-wide audit fixed (NaN fund-invariant tolerance, always-flush fill store, double-decremented fill guard). **Tooling & UX** (v1.4.20–v1.4.22): live/research clip parity with centralized chart sliders and analysis outputs on the central path resolver; claw logic deduplicated with hardened error paths; browser storage adapter persists deletions; editor green/red input feedback extended to funds and prices; compile-first runtime completed — tsx removed entirely, every entry point and the test suite running compiled dist under plain node through frozen-ESM-safe seams, plus exact AMA cold-start bootstrap sizing and research tools unified onto production slope/bounds math. Post-v1.4.22 work followed the same themes: bot-fitting backtests re-modeled on the production grid lifecycle, analysis tooling consolidated onto market_adapter sources, Kibana proxy-reset hardening, and TradingView price-axis interaction. **v1.4.24** fixed native fill-gap recovery and LP pricing; **v1.4.25** freezes genesis price-slots and hardens grid orphan/self-trade/fill-guard/shutdown paths, plus whitelist scoped `--bot` overwrite and Range quality legend.
+The post-ESM releases consolidated state, packaging, and tooling while hardening the grid engine — profile state on a resolver-derived `~/.config/dexbot2` dir, in-place order rotations, npm auto-update, dead-code purge, and a compile-first runtime (tsx removed; every entry point and the test suite run compiled `dist` under plain node). Grid work capped COW broadcasts and chunked retry-on-uncertain, gated boundary promotion and persisted-restore against gap-floor poison, fixed spread-collapse via the shared `isSlotInRail` filter, and cleared silent-failure defects from a modules-wide audit. **v1.4.24** fixed native fill-gap recovery and LP pricing; **v1.4.25** froze genesis price-slots and hardened orphan/self-trade/fill-guard/shutdown paths.
 
 ---
 
+## Architecture Deep-Dive: COW & Memory Tracking
+
+Two mechanisms shaped the order engine after the browser/TypeScript era: the Copy-on-Write grid
+and the memory-only integer tracking model. The construction detail that used to live in
+`COPY_ON_WRITE_MASTER_PLAN.md` and `architecture.md` is recorded here; those docs now describe
+only the current design.
+
+### Copy-on-Write: three eras
+
+- **Era 0 — original optimistic state (pre-v1.0):** the master grid was mutated directly during
+  planning, with no isolation or rollback. A sudden market move corrupted in-flight state (the
+  "Price Jump" incident — planning mutations applied straight to the master grid).
+- **Era 1 — frozen master state (v1.0):** `Object.freeze()` on the master Map and `deepFreeze()`
+  on order objects; every `_applyOrderUpdate` creates a new frozen Map via the immutable-swap
+  pattern. Retained as defense-in-depth — it catches accidental in-place mutation of
+  `manager.orders`.
+- **Era 2 — Copy-on-Write (v2.0, current):** clone the master into a `WorkingGrid`, plan and
+  broadcast on the clone, commit atomically on blockchain success (discard on failure). True
+  transactional semantics; the master is never in an intermediate state.
+
+The production code layers Era 1 and Era 2: freeze provides runtime mutation enforcement, COW
+provides the plan → broadcast → commit/discard lifecycle.
+
+### COW construction milestones (February–April 2026)
+
+- `modules/order/working_grid.ts` (`WorkingGrid`: clone, delta, stale tracking) and
+  `COW_PERFORMANCE` thresholds added.
+- `performSafeRebalance` → `_applySafeRebalanceCOW`; `buildDelta`; `_commitWorkingGrid` atomic
+  swap.
+- COW broadcast path (`_updateOrdersOnChainBatchCOW`); legacy rollback code removed.
+- Selective-abort fill strategy: individual fills continue, full-side updates block.
+- Divergence corrections and `updateGridFromBlockchainSnapshot` migrated to the COW pattern.
+- Atomic boundary shifts (patch 20): `pendingBoundaryIdx` carries boundary changes through the
+  pipeline and applies them only at `_commitWorkingGrid`, so boundary position and slot BUY/SELL
+  roles never transiently disagree during blockchain execution.
+- Validation suites added: COW core, commit-guard, concurrent-fill, divergence-correction, and
+  stale-plan/stack-discipline tests.
+
+### Memory-only integer tracking
+
+- **Raw order cache (`rawOnChain`):** grid slots store the exact blockchain order integers
+  (satoshis); seeded from broadcast arguments on placement, updated in place on partial fills,
+  refreshed on updates/rotations.
+- **Chain-free planning:** redundant `readOpenOrders()` calls were removed from the size-update
+  and rotation builders (`_buildSizeUpdateOps()`, `_buildRotationOps()`), and the
+  `computeVirtualOpenOrders()` virtual-order computation was dropped; `buildUpdateOrderOp`
+  gained an optional `cachedOrder` and returns `finalInts`.
+- **Result:** batch updates and rotations run without blockchain fetches; only placements and
+  recovery syncs query the chain (~10–20× faster high-frequency operations).
+- **Self-healing:** a failed memory-driven transaction triggers a full state-recovery sync so the
+  internal ledger stays consistent with the chain.
+
 ## Technical Challenges & Solutions
 
 | Challenge | Solution | Impact |
@@ -86,42 +143,31 @@ Evolved from a basic README to a comprehensive framework (50+ docs entries, 80%+
 
 ## Post-1.0.0 Status
 
-**Completed**: browser-safe core (140+ files, portable abstractions, pure-JS crypto); credit/MPA runtime (multi-asset collateral, oversize deal splitting); storage-adapter I/O centralization; self-healing (structural resync, subscription watchdog, bot/daemon broadcast-deadlock recovery); Kibana-driven trade PnL analytics; multi-round AMA refits; documented subsystem invariants; bot identity enforcement; credit-only mode; Docker support; npm package with lockfile sync; hardened grid order engine (COW pipeline, zero-amount prevention, fill batching, dust detection, spread correction).
+**Completed**: browser-safe core; credit/MPA runtime; storage-adapter I/O centralization; self-healing recovery; Kibana PnL analytics; credit-only mode; Docker support; npm package. For the grid order engine arc specifically (COW pipeline, orphan/self-trade/fill-guard hardening, invariants) see [ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md](ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md) §R4.
 
 **Planned**: backtesting engine (historical candle replay via exchange abstraction); injectable interfaces at call boundaries; SQLite persistence + Zod validation at the blockchain boundary; Telegram bot (**not yet implemented**) — owner-gated monitoring (`/status`, `/orders`, `/grid`, `/balance`) and opt-in+confirm gated control (`/start`, `/stop`, `/pause`); DEXBot is the only writer, private keys never reach the module (`TELEGRAM` block + `DEXBOT_TELEGRAM_TOKEN` env).
 
 ## Version History
 
-Compact, era-level view; per-release commit detail lives in [CHANGELOG.md](../CHANGELOG.md).
+Compact, era-level view. Per-release detail lives in [CHANGELOG.md](../CHANGELOG.md); the thematic post-1.0.0 story in [ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md](ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md).
 
 | Era | Commits | Theme |
 |-----|--------:|-------|
 | v0.1.0 → v0.6.0 | 1,217 | Foundation → COW architecture, strategy/sync engine, credential daemon, AMA prototype, credit/MPA runtime |
-| v0.6.0 → v1.0.0 | 309 | Zero-dependency & TS migration, native BitShares, fill detection overhaul, first stable release |
-| v1.0.0 → v1.1.0 | 85 | Post-stable hardening, PnL analytics, auto-update, broadcast deadlock fixes |
-| v1.1.0 → v1.3.3 | 114 | AMA refits, credit-only mode, COW recovery hardening, runtime extraction |
-| v1.3.3 → v1.4.8 | 74 | CJS→ESM completion, concurrency correction, uncertain-broadcast safety, truncated-read ambiguity |
-| v1.4.8 → v1.4.13 | 45 | Native ESM runtime, broadcast serialization, onboarding |
-| v1.4.13 → v1.4.19 | 36 | Profile-state centralization, code consolidation, per-broadcast op cap |
-| v1.4.19 → v1.4.20 | 5 | Grid boundary promotion hardening, recovery poison gate, analysis output centralization |
-| v1.4.20 → v1.4.21 | 15 | Runtime audit fixes, claw dedup hardening, boundary ceiling alignment, editor color feedback |
-| v1.4.21 → v1.4.22 | 4 | tsx removal completion (dist-only runtime + tests), exact AMA bootstrap sizing, research-tool production parity |
-| v1.4.22 → v1.4.23 | 12 | Even geometric AMA ladder, BTS fee-carve fix, sub-1x price-bound rejection, tradingview axis restore, doc realignment |
-| v1.4.23 → v1.4.24 | 3 | Native fill gap recovery with eager coalesced retry, LP collateral offer-first pricing |
-| v1.4.24 → v1.4.25 | 26 | Genesis-frozen price-slots, self-trade & fill-guard hardening, orphan & gap-band fixes, trigger/shutdown hardening, bot poll, grid monotonicity gate, whitelist scoped overwrite, Range legend |
-| v1.4.25 → v1.5.0 | 13 | Credit overview CLI + whitelist-scoped CR on shared pricing math, one-step TradingView chart, daemon-safe reload, offline account-ID cache, case-insensitive bot identity, stale-pivot guard fix, partial-surplus rotation clamp, adapter ownership centralization, op-77 bot discovery split |
-| v1.5.0 → v1.5.1 | 11 | Gap-evacuation guard allowance + rail-typed holes, persisted streaks with cancel-only teeth, vacated-rail refill, adoption/accounting/duplicate-guard hardening |
-| v1.5.1 → v1.5.2 | 10 | Sync rejection handling across pass-1/pass-2 adoption, crossing-guard candidate sharing, empty-read confirmation, broadcast-price CREATE validation, stamped gap-evacuation re-proof, credit whole-account display + expiry, offline export fill-block derivation |
-| v1.5.2 → v1.5.3 | 6 | Boundary ownership hardening (fund-driven sync removal, guard-skipped refill hold), COW broadcast/reconcile dedup, TradingView bot-grid range highlight, createOrder unknown-id materialize-or-error, curve-comparison docs |
-| v1.5.3 → v1.6.0 | 37 | Node-failure strike ledger and broadcast-deferred fill rebalancing, trust-chain free-balance heal with deferred-drain tolerance, bidirectional grid-regeneration trigger, TradingView order overlay and chart pref namespacing, credit short-offer id display, live-config pickup (issue #27), reserve ladder anchored at resolved bounds + live-grid rail edges with single-source ordering and exact-size activation, owed-crawl persistence across refused broadcasts/restarts + hold-aware reload-safe lifecycle, fill-anchored boundary recovery + poisoned persisted-boundary erase, startup rail gate + static-center crawl fold, all `*-deferred` holds non-blocking + hold metrics surfaced, opt-in MPA price-feed charts + range-aware shared candle cache, range-band span parity on grid-less charts, orange range zone widened to 1.40x, docs reserve-ladder sweep, reserve-deficit targeted-sync trigger with window-exclusion counting, matched-surplus startup excess planning, shelf-order guards across reserve classification/placement/startup cancels/size recalc (issue #27 follow-ups), unified Kibana candle cache on runCachedWindows with fetch retry budgets, genuine-coverage LP window reuse, feed volume/AMA timeframe alignment, TradingView monthly candles/stat badges/rigid pan/volume toggle/feed affordance, credit full offer id + empty-pair Curr. CR hiding, live-config onboarding note |
-| v1.6.0 → v1.6.1 | 3 | Never-run-stale hardening (level-triggered deferred-fill retry, stale-totals fill parking, out-of-spread watchdog, region-end fan-out, one-sided spread honesty), whitelist range-scaling opt-in defaults, live-save vs reset vs reload docs + power-law paper restructure |
-| v1.6.1 → v1.6.2 | 6 | Grid-price invariant + resync escalation, gap-slot batch sizing, shard candle cache + chart fixes, recovery stranding tolerance, fill-counter hygiene, dead-node prune + trace single-sourcing, obsolete-doc removal |
-| v1.6.2 → v1.6.3 | 3 | Correction-queue staleness guard (validate queued price corrections against live slot), grid-checker price epochs across repriced order lifetimes, final pre-broadcast pivot gate (re-check built ops when a fill queued after the freeze moved the pivot) |
+| v0.6.0 → v1.0.0 | 309 | Zero-dependency & TS migration, native BitShares, fill-detection overhaul, first stable release |
+| v1.0.0 → v1.3.3 | 199 | Post-stable hardening, PnL analytics, auto-update, broadcast-deadlock fixes, AMA refits, credit-only mode, COW recovery hardening, runtime extraction |
+| v1.3.3 → v1.4.13 | 119 | CJS→ESM completion, concurrency correction, uncertain-broadcast & truncated-read safety, native ESM runtime, broadcast serialization, onboarding |
+| v1.4.13 → v1.4.25 | 101 | Profile-state centralization, consolidation, per-broadcast op cap, grid boundary/recovery hardening, tsx removal, genesis-frozen price-slots, self-trade & orphan fixes |
+| v1.4.25 → v1.5.3 | 40 | Credit overview + whitelist-scoped CR, TradingView tooling, daemon-safe reload, gap-evacuation/rail-hole hardening, sync adoption hardening, boundary ownership |
+| v1.5.3 → v1.6.0 | 37 | Node-failure ledger, grid regeneration, reserve ladder, live-config pickup, owed-crawl persistence, fill-anchored boundary recovery, TradingView overlay |
+| v1.6.0 → v1.6.3 | 12 | Never-run-stale hardening, whitelist range-scaling opt-in, grid-price invariant, shard candle cache, correction-queue staleness, final pre-broadcast pivot gate |
+| v1.6.3 → v1.6.4 | 18 | Fund-driven spread correction, gapSlots+1 batch cap, VIRTUAL RMS divergence, sync-lock log fix, invariant-doc contract, analysis shared modules, window-aware profitability annualisation, portable chart exports, range-threshold restore, dead-code purge, doc consolidation |
+| v1.6.4 → v1.6.5 | 15 | Editor-managed whitelist flags + legacy generator removal, centralized bot defaults/settings docs, log-symmetric range-scaling tilt, AMA gridPrice default + unset → startPrice normalization, Pool default/warn-color cues, Grid Health AMA-slope Δ knob, dynamic-weight chart CLI, update dist-freshness self-heal, launcher worker rename, RMS log tagging |
 
 ---
 
 **Report Originally Generated**: February 19, 2026
-**Last Updated**: September 15, 2026
-**Total Commits**: 2,228
-**Date Range**: December 2, 2025 – September 15, 2026
+**Last Updated**: September 23, 2026
+**Total Commits**: 2,263
+**Date Range**: December 2, 2025 – September 23, 2026
 **Repository**: DEXBot2 (BitShares DEX Trading Bot)

package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md

@@ -67,7 +67,7 @@ Mixed BUY/SELL batches are validated per asset using a signed-delta **peak** run
 
 #### Implementation Location
 
-File: `modules/dexbot_cow_runtime.ts` — `validateOperationFunds()` (line 1488), called from the COW batch broadcast path at line 2908. `modules/dexbot_class.ts` exposes a thin wrapper `_validateOperationFunds()` (line 1122).
+File: `modules/dexbot_cow_runtime.ts` — `validateOperationFunds()` (line 1655), called from the COW batch broadcast path at line 4457.
 
 ```javascript
 // Per-asset peak requirement vs. quantized chain-free snapshot.
@@ -114,9 +114,9 @@ See [developer_guide.md#order-state-helper-functions](developer_guide.md#order-s
 
 **Mechanism**: Fill events arrive via `modules/dexbot_fill_runtime.ts` (the fill-runtime module), which pushes them into `bot._incomingFillQueue` (declared in `modules/dexbot_class.ts`). The drain loop in `dexbot_fill_runtime.ts` then chunks the queue into capped batches and calls `modules/order/manager.ts::processFilledOrders` (line 1438) once per chunk to run the full rebalance pipeline.
 
-**Batch Sizing Algorithm**: Batch size is derived from the grid gap-slot count (`DEXBot._getGapSlotBatchSize`): a queue depth at or below gapSlots is processed as one unified batch; deeper queues are chunked into repeated batches of gapSlots (the last chunk may be smaller). The same gap-slot size caps order operations per broadcast transaction (oversized op batches are split into sequential broadcasts).
+**Batch Sizing Algorithm**: Batch size is derived from the grid gap-slot count + 1 (`DEXBot._getGapSlotBatchSize`): a queue depth at or below gapSlots+1 is processed as one unified batch; deeper queues are chunked into repeated batches of gapSlots+1 (the last chunk may be smaller). The same gapSlots+1 size caps order operations per broadcast transaction (oversized op batches are split into sequential broadcasts).
 
-**Configuration**: no fixed constant — both `FILL_PROCESSING.MAX_FILL_BATCH_SIZE` and `COW_PERFORMANCE.MAX_OPS_PER_BROADCAST` were removed; batch sizing follows the grid gap-slot count.
+**Configuration**: no fixed constant — both `FILL_PROCESSING.MAX_FILL_BATCH_SIZE` and `COW_PERFORMANCE.MAX_OPS_PER_BROADCAST` were removed; batch sizing follows the grid gap-slot count + 1.
 
 #### Fill Batch Processing Timeline
 
@@ -299,7 +299,7 @@ When a fill occurs, the boundary shifts to "follow" the price.
 
 ### 3.2 Global Side Capping
 
-Budgets are dynamic. The bot calculates `TotalSideBudget` from `funds.allocated.{buy,sell}` (the `botFunds`-capped capital per side — see §1.3). This ensures the bot never attempts to deploy more than its configured share of account capital, even when the account holds additional free balance for other bots or manual trading.
+Budgets are dynamic. The bot calculates the per-side budget via `getSideBudget` from `funds.allocated.{buy,sell}` (the `botFunds`-capped capital per side — see §1.3). This ensures the bot never attempts to deploy more than its configured share of account capital, even when the account holds additional free balance for other bots or manual trading.
 
 **Safety Check:**
 If the calculated ideal grid requires more capital than available in the allocation, the *increase* is capped.
@@ -478,15 +478,15 @@ When a grid is regenerated or resized, existing partial orders (partially filled
 A partial order is classified as **Dust** if:
 $$Size_{current} < Size_{ideal} \times 0.05$$
 
-Dust orders are too small to be efficient on-chain and are marked for consolidation into the grid rebuild cycle.
+Dust orders are too small to be efficient on-chain and are **cancelled immediately on detection** — no delay, no timer (`cancelDustOrders()`, `[DUST]` tag). Detection runs on every fill/sync tick plus a 5-minute health check (`DUST_HEALTH_CHECK_INTERVAL_MS`) as a crash/restart safety net; the cancel flows through the synthetic-fill pipeline so funds return to `ChainFree`.
 
 ### 4.2 Consolidation Strategy
 
 When the strategy engine encounters partial orders during rebalancing:
 
 **Direct Approach** (Simplified):
-1. **Identify unhealthy partials**: Detect any partial orders below the 5% dust threshold on each side
-2. **Mark for consolidation**: Flag partials as needing attention in the next rebalance cycle
+1. **Cancel dust first**: partials below the 5% dust threshold are cancelled on detection (see §4.1) and never reach the rebuild
+2. **Consolidate the remainder**: surviving partials are absorbed when the grid is rebuilt in the next rebalance cycle
 3. **Fund-driven grid rebuild**: Rather than complex slot-by-slot merge/split logic, the entire grid is regenerated based on current total funds (including proceeds from fills)
 4. **Natural redistribution**: The rebuilt grid automatically sizes all orders (including those replacing consolidation candidates) using the Ideal Grid sizing formula
 5. **Spread maintenance**: The target spread gap remains constant at `targetSpreadPercent`—no dynamically inflated corrections
@@ -751,7 +751,7 @@ Quantization has a single source of truth: `quantizeFloat()` in `modules/order/u
 
 #### 5.5.4 Relationship to Fund Validation
 
-The corrected fund validation in `_validateOperationFunds()` uses quantized values:
+The corrected fund validation in `validateOperationFunds()` uses quantized values:
 
 ```javascript
 // Check: Does required amount fit in available balance?
@@ -801,4 +801,4 @@ To prevent "Time-of-Check to Time-of-Use" errors:
 **TOCTOU protection in `processFillAccounting`.** `_buildBtsDeferredRefundAdjustment` reads `btsFeeState` from `mgr.orders` while the order lock is held — the lock is acquired before accounting runs, and the POST-RESET and BOOTSTRAP tracked-fill accounting paths follow the same locking pattern.
 
 ---
-*Technical Reference for DEXBot2 v1.6.3 release*
+*Technical Reference for DEXBot2 v1.6.5 release*