dexbot 1.6.2 → 1.6.4

package/docs/README.md

@@ -2,7 +2,7 @@
 
 This directory contains the comprehensive technical documentation for the DEXBot2 trading bot. It is designed to guide developers from high-level architecture down to the nuances of fund accounting and state management.
 
-**Version context:** v1.6.2 (released).
+**Version context:** v1.6.4 (released).
 
 ---
 
@@ -116,6 +116,38 @@ This directory contains the comprehensive technical documentation for the DEXBot
 - **Lifecycle B (Maintenance / AMA-Driven)**: Periodic path from `_performPeriodicGridChecks` → `executeMaintenanceLogic`.
 - **Cross-Cutting Invariants**: COW boundary, fund SSOT, replay-safe fills, lock ordering.
 
+### 🧩 [Copy-on-Write Master Plan](COPY_ON_WRITE_MASTER_PLAN.md)
+*COW design, phases, and state machine details.*
+- **Architecture**: Master-grid projection model and rebalance flow
+- **Lifecycle**: Implementation phases, commit boundaries, and test coverage
+- **Safety**: Invariants and guardrails for concurrent updates
+
+### 🔒 [COW Invariants](COW_INVARIANTS.md)
+*Stable theory contract for COW pipeline.*
+- **Non-negotiable invariants**: Master immutability, commit atomicity, projection rules, accounting separation
+- **Test mapping**: Links each invariant to regression tests
+- **Review checklist**: Quick-use verification for COW/accounting changes
+
+### 📐 [Grid-Price Invariant](GRID_PRICE_INVARIANT.md)
+*Why a slot's emitted price must equal its genesis level — and how that failed.*
+- **The invariant**: `order.price === priceForSlot(idx, genesis)`, and why range guards cannot substitute for it
+- **Failure mechanism**: Chain price overwriting slot identity, pre-broadcast substitution, untrusted fill-guard pivot
+- **Enforcement**: The six emission sites, the blocking rejection of off-grid emissions, the final pre-broadcast pivot gate, and the fail-open policy on unjudgeable inputs
+- **Out-of-bounds policy**: Hold and surface; refill in-grid slots at their genesis price
+- **Status**: Landed enforcement map, key constants, and why the removed 5% sanity gate must not be naively re-landed
+
+### 💰 [Fund Movement & Accounting](FUND_MOVEMENT_AND_ACCOUNTING.md)
+*The most critical part of the bot: safe capital management.*
+- **Single Source of Truth**: How the bot avoids double-spending and out-of-sync balances.
+- **Optimistic ChainFree**: The mechanism that allows the bot to trade with fill proceeds before they are finalized on-chain.
+- **Fill Batch Processing**: Fixed-cap batching for efficient fill processing (`<=4` unified, `>4` chunked)
+- **Partial Order Consolidation**: Simplified, direct consolidation through grid rebuilding (no merge/split mechanics)
+- **Dust Detection & Management**: Partials below the dust threshold are cancelled on-chain immediately on detection (no delay, no timer)
+- **BTS Fee Object Structure**: `netProceeds` field for accounting precision
+- **BUY Side Sizing & Fee Accounting**: Correct fee application by order side
+- **Mixed Order Fund Validation**: Separate validation for BUY vs SELL order fund checks
+- **Fee Management**: Detailed logic for BTS fee reservations and market fee deductions.
+
 ### 📖 [Developer Guide](developer_guide.md)
 *Your daily companion for coding.*
 - **Quick Start**: How to get the development environment running.
@@ -136,57 +168,31 @@ This directory contains the comprehensive technical documentation for the DEXBot
 - **Branching Strategy**: Explanation of the `test` → `dev` → `main` lifecycle.
 - **CI/CD Patterns**: Standards for merging and ensuring code quality across branches.
 
-### 🧮 [DEXBot vs DEXBot2 Comparison](DEXBOT_COMPARISON.md)
-*Architectural, functional, and operational comparison with the original Python DEXBot.*
-- **Scope**: Full side-by-side of technology stack, architecture, trading strategies, order management, configuration, blockchain integration, fund accounting, and concurrency safety
-- **Audience**: Developers and operators evaluating or migrating between the two projects
+### 🧪 [Test Suite](../tests/README.md)
+*Test organization, categories, and key architectural patterns tested.*
+- **Test Layout**: Directory structure, helpers, and quick-start commands
+- **Categories**: Core infrastructure, order management, COW rebalancing, fees/accounting, integration, edge cases, and more
+- **Architectural Patterns**: COW rebalancing, RMS divergence, and fund invariants with doc cross-references
 
 ### 🧭 [Evolution Report](EVOLUTION.md)
 *Project timeline and major architecture phases.*
 - **Coverage**: Historical milestones from bootstrap through the current stable release; per-release detail lives in [CHANGELOG.md](../CHANGELOG.md)
 - **Focus**: Architecture evolution, release history, and test growth
 
+### ⏪ [Order Engine Retrospective](ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md)
+*Why the post-1.0.0 order engine kept misbehaving — synthesis plus the incident/fix ledger.*
+- **Part I — Synthesis**: root cause (uncertain broadcast), recurring bug families, meta-patterns, what actually fixed it, lessons
+- **Part II — Incident & Fix Ledger**: preserved gap-band / ladder-recenter / orphan-fill / price-first plans with `LANDED`/`REVERTED`/`SUPERSEDED` status and commit hashes
+- **Regression gate**: `npm run analysis:grid-check` (see [analysis/README.md](../analysis/README.md))
+
 ### 🗒️ [Changelog](../CHANGELOG.md)
 *Release notes and documentation history.*
 - **Scope**: Versioned notes per release
 
-### 🧩 [Copy-on-Write Master Plan](COPY_ON_WRITE_MASTER_PLAN.md)
-*COW design, phases, and state machine details.*
-- **Architecture**: Master-grid projection model and rebalance flow
-- **Lifecycle**: Implementation phases, commit boundaries, and test coverage
-- **Safety**: Invariants and guardrails for concurrent updates
-
-### 🔒 [COW Invariants](COW_INVARIANTS.md)
-*Stable theory contract for COW pipeline.*
-- **Non-negotiable invariants**: Master immutability, commit atomicity, projection rules, accounting separation
-- **Test mapping**: Links each invariant to regression tests
-- **Review checklist**: Quick-use verification for COW/accounting changes
-
-### 📐 [Grid-Price Invariant](GRID_PRICE_INVARIANT.md)
-*Why a slot's emitted price must equal its genesis level — and how that failed.*
-- **The invariant**: `order.price === priceForSlot(idx, genesis)`, and why range guards cannot substitute for it
-- **Failure mechanism**: Chain price overwriting slot identity, pre-broadcast substitution, untrusted fill-guard pivot
-- **Enforcement**: The six emission sites, the blocking rejection of off-grid emissions, and the fail-open policy on unjudgeable inputs
-- **Out-of-bounds policy**: Hold and surface; refill in-grid slots at their genesis price
-- **Status**: What is landed, what remains open, and why the removed 5% sanity gate must not be naively re-landed
-
-### 🧪 [Test Suite](../tests/README.md)
-*Test organization, categories, and key architectural patterns tested.*
-- **Test Layout**: Directory structure, helpers, and quick-start commands
-- **Categories**: Core infrastructure, order management, COW rebalancing, fees/accounting, integration, edge cases, and more
-- **Architectural Patterns**: COW rebalancing, RMS divergence, and fund invariants with doc cross-references
-
-### 💰 [Fund Movement & Accounting](FUND_MOVEMENT_AND_ACCOUNTING.md)
-*The most critical part of the bot: safe capital management.*
-- **Single Source of Truth**: How the bot avoids double-spending and out-of-sync balances.
-- **Optimistic ChainFree**: The mechanism that allows the bot to trade with fill proceeds before they are finalized on-chain.
-- **Fill Batch Processing**: Fixed-cap batching for efficient fill processing (`<=4` unified, `>4` chunked)
-- **Partial Order Consolidation**: Simplified, direct consolidation through grid rebuilding (no merge/split mechanics)
-- **Dust Detection & Management**: Partials below the dust threshold are cancelled on-chain immediately on detection (no delay, no timer)
-- **BTS Fee Object Structure**: `netProceeds` field for accounting precision
-- **BUY Side Sizing & Fee Accounting**: Correct fee application by order side
-- **Mixed Order Fund Validation**: Separate validation for BUY vs SELL order fund checks
-- **Fee Management**: Detailed logic for BTS fee reservations and market fee deductions.
+### 🧮 [DEXBot vs DEXBot2 Comparison](DEXBOT_COMPARISON.md)
+*Architectural, functional, and operational comparison with the original Python DEXBot.*
+- **Scope**: Full side-by-side of technology stack, architecture, trading strategies, order management, configuration, blockchain integration, fund accounting, and concurrency safety
+- **Audience**: Developers and operators evaluating or migrating between the two projects
 
 ---
 

package/docs/architecture.md

@@ -377,13 +377,13 @@ The fill pipeline handles incoming filled orders efficiently through fixed-cap b
                       ↓
 ┌─────────────────────────────────────────────────────────────┐
 │             processFilledOrders() - Entry Point             │
-│    Use gap-slot batch size for deterministic batching   │
-│      Rules: <=gapSlots unified, >gapSlots chunked      │
+│    Use gap-slot batch size (gapSlots+1) for deterministic batching   │
+│      Rules: <=gapSlots+1 unified, >gapSlots+1 chunked      │
 └─────────────────────┬───────────────────────────────────────┘
                       ↓
 ┌─────────────────────────────────────────────────────────────┐
-│            Pop Batch (up to gapSlots)                     │
-│        Takes N fills from queue head (N = 1..gapSlots)     │
+│            Pop Batch (up to gapSlots+1)                     │
+│        Takes N fills from queue head (N = 1..gapSlots+1)     │
 │   Example: pops [fill1, fill2, fill3] for batch processing  │
 └─────────────────────┬───────────────────────────────────────┘
                       ↓
@@ -418,9 +418,9 @@ The fill pipeline handles incoming filled orders efficiently through fixed-cap b
 
 ### Key Properties
 
-- **Gap-Slot Batch Sizing**: Batch size is deterministic, derived from the grid gap-slot count (`DEXBot._getGapSlotBatchSize`)
-  - 1..gapSlots awaiting: single unified batch (one rebalance/broadcast cycle)
-  - more than gapSlots awaiting: repeated chunks of gapSlots (last chunk may be smaller)
+- **Gap-Slot Batch Sizing**: Batch size is deterministic, derived from the grid gap-slot count + 1 (`DEXBot._getGapSlotBatchSize`)
+  - 1..gapSlots+1 awaiting: single unified batch (one rebalance/broadcast cycle)
+  - more than gapSlots+1 awaiting: repeated chunks of gapSlots+1 (last chunk may be smaller)
 
 - **Single Rebalance Cycle**: All fills in batch processed in ONE rebalance
   - No "split across cycles" delays
@@ -916,7 +916,7 @@ graph TB
     START[Grid Update Triggered] --> CALC[Calculate Ideal Grid<br/>Based on current funds]
     CALC --> RELOAD[Force Reload Persisted Grid<br/>Ensure fresh blockchain state]
     RELOAD --> COMPARE[Compare to Persisted Grid]
-    COMPARE --> RMS[Calculate RMS Divergence<br/>For PARTIAL orders only]
+    COMPARE --> RMS[Calculate RMS Divergence<br/>ACTIVE + VIRTUAL orders per side]
 
     RMS --> CHECK{RMS > Threshold?}
     CHECK -->|Yes| UPDATE[Update Grid Sizes<br/>Trigger rebalance]
@@ -1043,28 +1043,17 @@ graph LR
 
 The system has been optimized to use a "memory-driven" model for order updates, eliminating redundant blockchain API calls during normal operation.
 
-### Key Changes
-
-**1. Raw Order Cache (`rawOnChain`)**
-- Grid slots now store exact blockchain order representations (integers/satoshis) in a `rawOnChain` cache
-- **Birth**: Cache populated immediately after successful order placement using broadcasted arguments
-- **Partial Fills**: Cache updated in-place via integer subtraction (subtracting filled satoshis from `for_sale`)
-- **Updates/Rotations**: Cache refreshed with adjusted integers returned by build process
-
-**2. Eliminated Redundant API Calls**
-- Removed all `readOpenOrders()` calls from `_buildSizeUpdateOps()` and `_buildRotationOps()`
-- Removed `computeVirtualOpenOrders()` logic that was redundantly fetching entire account state
-- The bot now trusts its internal state, backed by real-time fill listener, to build transactions
-
-**3. Refactored `buildUpdateOrderOp()`**
-- Updated to support optional `cachedOrder` parameter
-- Allows callers to bypass blockchain queries if they have raw state in memory
-- Returns `finalInts` along with operation data for local tracking
-
-**4. Self-Healing Resilience**
-- Maintains "State Recovery Sync" fallback
-- If a memory-driven transaction fails, bot catches error and performs a full refresh
-- Ensures internal ledger stays synchronized with BitShares blockchain
+### How it works
+
+- **Raw order cache (`rawOnChain`):** each grid slot stores the exact blockchain order integers
+  (satoshis) — seeded from broadcast arguments on placement, updated in place on partial fills,
+  and refreshed on updates/rotations.
+- **Chain-free planning:** size updates and rotations build their operations from the cache;
+  only placements and recovery syncs query the blockchain.
+- **`buildUpdateOrderOp(cachedOrder?)`:** accepts an optional cached order and returns
+  `finalInts` alongside the operation for local tracking.
+- **Self-healing:** a failed memory-driven transaction triggers a full state-recovery sync so the
+  internal ledger stays consistent with the chain.
 
 ### Benefits
 - **Faster reaction time**: No waiting for blockchain queries during order updates

package/docs/developer_guide.md

@@ -79,7 +79,7 @@ A **phantom order** is an order in ACTIVE/PARTIAL state WITHOUT a valid `orderId
 
 | Term | Meaning |
 |------|---------|
-| **Gap-Slot Batch Fill Processing** | Groups fills using the grid gap-slot count as batch size (`DEXBot._getGapSlotBatchSize`): `<= gapSlots` uses one unified batch; `> gapSlots` chunks at gapSlots. In the documented 29-fill Feb 7 crash scenario, this reduces the estimated divergence window from ~90s to ~24s; see [`FUND_MOVEMENT_AND_ACCOUNTING.md`](FUND_MOVEMENT_AND_ACCOUNTING.md#15-fill-batch-processing--timeline). |
+| **Gap-Slot Batch Fill Processing** | Groups fills using the grid gap-slot count + 1 as batch size (`DEXBot._getGapSlotBatchSize`): `<= gapSlots + 1` uses one unified batch; `> gapSlots + 1` chunks at `gapSlots + 1`. In the documented 29-fill Feb 7 crash scenario, this reduces the estimated divergence window from ~90s to ~24s; see [`FUND_MOVEMENT_AND_ACCOUNTING.md`](FUND_MOVEMENT_AND_ACCOUNTING.md#15-fill-batch-processing--timeline). |
 | **Recovery Retry System** | Count+time-based retry mechanism with periodic reset. Replaces one-shot `_recoveryAttempted` flag. Max 5 attempts per episode with 60s minimum interval between retries. |
 | **Orphan-Fill Deduplication** | Map+TTL-based tracking of stale-cleaned order IDs to prevent double-crediting. Delayed orphan fill events are still blocked by checking `_staleCleanedOrderIds`. |
 
@@ -171,8 +171,8 @@ A **phantom order** is an order in ACTIVE/PARTIAL state WITHOUT a valid `orderId
 | **Atomic Check-and-Deduct** | Verify funds + deduct in single operation |
 | **Divergence Detection** | Comparing ideal grid vs. persisted grid |
 | **Invariant Verification** | Checking fund accounting consistency |
-| **Batch Processing** | Grouping multiple fills into a single rebalance cycle instead of one-at-a-time. Gap-slot sizing: `<= gapSlots` unified, otherwise chunked at gapSlots. |
-| **Gap-Slot Batch Sizing** | Deterministic chunking model with the grid gap-slot count as the per-broadcast bound. Keeps throughput high while avoiding tier-lookup complexity. |
+| **Batch Processing** | Grouping multiple fills into a single rebalance cycle instead of one-at-a-time. Gap-slot sizing: `<= gapSlots + 1` unified, otherwise chunked at `gapSlots + 1`. |
+| **Gap-Slot Batch Sizing** | Deterministic chunking model with the grid gap-slot count + 1 as the per-broadcast bound. Keeps throughput high while avoiding tier-lookup complexity. |
 | **Stale-Order Recovery** | Fast-path recovery for single-operation batches that encounter stale orders on-chain. Executes cleanup without full state sync. |
 | **Orphan-Fill Prevention** | Deduplication mechanism that prevents double-crediting fills from stale-cleaned orders using timestamp-based ID tracking (TTL pruning). |
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dexbot",
-  "version": "1.6.2",
+  "version": "1.6.4",
   "description": "The first open source trading bot with zero runtime dependencies and a fully adaptive market making strategy.",
   "main": "dist/modules/dexbot_class.js",
   "exports": {

package/scripts/git-viewer.sh

@@ -44,7 +44,7 @@ check_dependencies() {
         elif command -v dnf &> /dev/null; then
             sudo dnf install -y "${missing_packages[@]}"
         elif command -v pacman &> /dev/null; then
-            sudo pacman -S --noconfirm "${missing_packages[@]}"
+            sudo pacman -Syu --noconfirm "${missing_packages[@]}"
         else
             echo "Warning: Could not detect package manager. Please install manually: ${missing_packages[@]}"
             echo "  apt-get: sudo apt-get install ${missing_packages[*]}"

package/tests/README.md

@@ -88,7 +88,7 @@ Interactive tools and performance benchmarks (not part of CI).
 
 ### Edge Cases & Regression
 Tests targeting specific bugs, race conditions, and failure modes.
-*Examples:* `test_critical_bug_fixes.ts`, `test_race_condition_fixes_batch1.ts`, `test_patch17_invariants.ts`, `test_shutdown_reentrancy.ts`, `test_multifill_opposite_partial.ts`
+*Examples:* `test_critical_bug_fixes.ts`, `test_race_condition_fixes_batch1.ts`, `test_patch17_invariants.ts`, `test_shutdown_reentrancy.ts`, `test_multifill_opposite_partial.ts`, `test_correction_queue_staleness.ts`, `test_spread_pure_fund_driven.ts`, `test_sync_lock_id_verification.ts`
 
 ### Utilities & Helpers
 Shared utility functions, precision handling, chain helpers.
@@ -120,6 +120,15 @@ Shared utility functions, precision handling, chain helpers.
 
 **Reference:** [docs/FUND_MOVEMENT_AND_ACCOUNTING.md](../docs/FUND_MOVEMENT_AND_ACCOUNTING.md)
 
+### Grid-Price Invariant & Hold Guards
+- Every emitted order's price must equal its slot's genesis level (`priceForSlot(idx, genesis)`); off-grid emissions are blocked, not counted
+- The guard itself plus the live batch wiring, escalation thresholds, and per-order stranded-hold clocks are each pinned and mutation-tested
+- Adoption keeps the slot's own level; `loadGrid` repairs a pre-existing off-grid slot price
+
+**Reference:** [docs/GRID_PRICE_INVARIANT.md](../docs/GRID_PRICE_INVARIANT.md)
+
+**Examples:** `test_grid_price_invariant_guard.ts`, `test_grid_price_invariant_wiring.ts`, `test_grid_price_slot_invariant.ts`, `test_final_pivot_gate.ts`, `test_hold_and_center_guards.ts`, `test_sync_out_of_grid_defer.ts`
+
 ---
 
 ## Documentation References