dexbot 1.6.5 → 1.6.7

package/docs/WORKFLOW.md

@@ -240,25 +240,31 @@ the canonical name is preferred in scripts and docs.
 |---------|---------|---------|
 | `dexbot test <bot>` | — | Run a single bot in live mode (one-shot; not a dry run) |
 | `dexbot drystart <bot>` | — | Same as `test` but forces dry-run execution |
-| `dexbot reset <bot>` | — | Trigger a grid reset (applies live or on next start) |
+| `dexbot reset <bot>` | — | Trigger a grid reset (applies live or on next start; `reset all` for all) |
 | `dexbot default` | `defaults` | Reset settings to defaults (deletes generated settings files) |
 | `dexbot disable <bot>` | — | Mark a bot inactive in config (`disable all` for all) |
 | `dexbot enable <bot>` | — | Mark a bot active in config (`enable all` for all) |
-| `dexbot key` | `key` | Launch the chain key helper (`modules/chain_keys.ts`) |
-| `dexbot bot` | `bot` | Launch the interactive bot configurator |
+| `dexbot key` | `keys` | Launch the chain key helper (`modules/chain_keys.ts`) |
+| `dexbot bot` | `bots` | Launch the interactive bot configurator |
 | `dexbot pm2` | — | Start all active bots via PM2 |
 | `dexbot update` | — | Update DEXBot2 from the repository and restart active bots |
 | `dexbot export <bot>` | — | Export bot trades/settings to CSV/JSON for local analysis/ |
-| `dexbot order` | `orders` | Analyze persisted order grids (spread, increment, funds) |
+| `dexbot order` | `orders` | Analyze persisted order grids (spread, increment, funds); `--export` for HTML |
 | `dexbot order [<bot>]` | — | Analyze only the specified bot's order grid |
 | `dexbot credit` | — | Live summed MPA + borrowed-credit positions per asset per bot (`[<bot>]`) |
-| `dexbot status` | `stat` | Unified runtime health — daemon, adapter, bots |
+| `dexbot tv <target>` | — | TradingView 1h chart for `<bot|pool-id|AssetA/AssetB>` over `--month N` (default 3) |
+| `dexbot dw <target>` | — | Dynamic-weight research chart: same targets/flags as `tv` |
+| `dexbot status` | `stat`, `stats` | Unified runtime health — daemon, adapter, bots |
 | `dexbot start` | `unlock` | Run credential daemon + bot (equivalent to running the `unlock` runtime, `dist/unlock.js`) |
-| `dexbot stop` / `dexbot start` | `stp`, `stopall` | Stop/start the monolithic runtime (unlock mode) |
+| `dexbot stop` | `stp`, `stopall` | Stop the monolithic runtime (unlock mode) |
 | `dexbot reload` | `reloadall` | Reload the monolithic runtime without touching the credential daemon (unlock mode) |
 | `dexbot restart` | `restartall` | Restart the monolithic runtime (unlock mode, re-unlocks credential daemon) |
 | `dexbot delete` | — | Shut down and clean up the monolithic runtime (unlock mode) |
 | `dexbot clear` | — | Remove all log files from the logs directory (`<profiles>/logs`) |
+| `dexbot clear-orders` | — | Remove all persisted order files (`<profiles>/orders`) |
+| `dexbot clear-market-adapter` | — | Remove market adapter data, state, and logs |
+| `dexbot clear-all` | — | Remove orders, logs, market adapter, and claw files (all of the above) |
+| `dexbot help` | — | Print the full CLI reference (grouped by Runtime/Trading/Config/Analysis/Files) |
 
 ## NPM Scripts for Branch Synchronization
 

package/docs/architecture.md

@@ -22,7 +22,7 @@ DEXBot2 is a grid trading bot for the BitShares blockchain. It maintains a geome
 DEXBot2 prioritizes **simplicity and operational efficiency** over complex partial-handling mechanics:
 
 1. **Constant Spread**: The spread zone width remains fixed at `targetSpreadPercent`, eliminating dynamic inflation triggers.
-2. **Direct Consolidation**: Dust partials are absorbed into the next grid rebuild cycle, not handled by complex merge/split logic.
+2. **Immediate Dust Cancellation**: Sub-threshold partials are cancelled on-chain as soon as detection runs; surviving non-dust partials are handled by ordinary fund-driven rebalancing without complex merge/split logic.
 3. **Minimal Blockchain Interaction**: Fund-driven rebalancing occurs once per fill batch, not per-partial. Grid generation uses only available funds—no forced allocations.
 4. **Closed-Loop Market Dynamics**: The boundary-crawl mechanism naturally handles price movement and fill flows without special-case logic.
 5. **Powerful Maintenance Tools**: Periodic grid regeneration, recovery retries, and fund invariant verification keep the system healthy over long operations.
@@ -56,7 +56,7 @@ graph TB
         MASTERGRID[Master Grid - immutable/frozen<br/>slot-id, price, size, state<br/>orderId, blockchain, grid, proceeds]
         TWOPASS[SyncEngine<br/>2-pass: grid-to-chain then chain-to-grid<br/>match orderId, detect partials, flag stale]
         FUNDS["Accounting - SSOT for funds<br/>available, virtual, committed<br/>btsFeesOwed<br/>Avail = max 0 ChainFree minus Virtual minus Fees"]
-        TARGET[Strategy Engine<br/>calculateTargetGrid<br/>boundary-crawl pivot<br/>partial-fill consolidation, rotation]
+        TARGET[Strategy Engine<br/>calculateTargetGrid<br/>boundary-crawl pivot<br/>partial correction, rotation]
         WORKGRID[WorkingGrid - COW copy<br/>all mutations here only<br/>commit to Master on confirmation]
         FILLQUEUE[Fill Queue<br/>AsyncLock + dedup 5-60 min]
         BATCHER[Fixed-Cap Batcher<br/>queue within cap: unified batch<br/>queue above cap: chunk at cap size<br/>cap = gapSlots + 1]
@@ -443,7 +443,7 @@ The fill pipeline handles incoming filled orders efficiently through fixed-cap b
 
 Scenario source: 29-fill burst during the Feb 7 market crash, modeled at
 roughly 3 seconds per broadcast; see
-[`FUND_MOVEMENT_AND_ACCOUNTING.md`](FUND_MOVEMENT_AND_ACCOUNTING.md#14-fill-batch-processing--cache-fund-timeline).
+[`FUND_MOVEMENT_AND_ACCOUNTING.md`](FUND_MOVEMENT_AND_ACCOUNTING.md#15-fill-batch-processing--timeline).
 
 | Metric | Legacy (1-at-a-time) | Fixed-Cap Batching | Improvement |
 |--------|---------------------|-------------------|-------------|
@@ -464,46 +464,53 @@ Instead of complex partial handling, spread corrections are **conservative and f
 - Target spread width stays **constant** at `targetSpreadPercent`
 - Corrections scale with **actual available funds**, not arbitrary slot budgets
 - No dynamic spread inflation based on partial consolidation flags
-- Edge-first surplus selection ensures stable rotation candidates
+- Safe window-contiguous candidate ordering (with an edge-first fallback) prevents cross-side placement
 
 ### Algorithm
 
-**Location**: `modules/order/strategy.ts::calculateTargetGrid()`
+**Location**: `modules/order/grid.ts::prepareSpreadCorrectionOrders()`, selected by
+`modules/order/grid.ts::determineOrderSideByFunds()`.
 
-The simplified approach prioritizes fund availability over aggressive corrections:
+The correction path is fund-only and never manufactures budget by shrinking a resting order:
 
 ```
-1. Detect that spread is wider than targetSpreadPercent
-2. Calculate how many edge slots are missing
-3. Attempt to place new edge orders with available funds
-4. If insufficient funds for all edges:
-   - Create what's affordable with available funds
-   - Log shortfall (smooth over next rebalance cycle)
-5. If a dust partial exists in the correction window:
-   - Mark for consolidation in next grid rebuild
-   - Don't create complex merge/split side effects
-6. Maintain constant target spread—no inflation based on partial flags
+1. Detect that the live window is wider than targetSpreadPercent.
+2. Select up to `outOfSpread` eligible holes in safe window/gap order.
+3. If the funded edge holds a non-dust PARTIAL, optionally top it up, but only
+   when `ideal - current` is positive and free funds can cover the increase.
+4. Create replacements from free available/chainFree funds; contiguous gap-slot
+   promotion may extend the window while preserving MIN_SPREAD_ORDERS and the
+   opposite placed rail.
+5. If free funds cannot cover every target, place/top up the affordable prefix,
+   stop without shrinking inventory, and log the fund-constrained remainder.
+6. If neither side has free funds, skip correction and let the maintenance
+   runtime refresh balances and open orders rather than recycle stale inventory.
 ```
 
+Detected dust partials never enter this path: `cancelDustOrders()` cancels them
+on-chain immediately, and the five-minute health check is only a restart safety net.
+
 ### Fund-Safe Constraints
 
 Spread corrections respect these hard limits:
 
 ```javascript
 // In modules/constants.ts (GRID_LIMITS)
-MIN_SPREAD_ORDERS: 2,           // Always maintain minimum gap
+MIN_SPREAD_ORDERS: 2,           // Preserve the minimum empty gap reserve
 
-// Spread width itself stays user-configured (`targetSpreadPercent`) — fixed, no inflation.
-// Correction count has no separate slot cap; it is bounded by the funds check below.
+// `outOfSpread` supplies the requested slot count. Execution is bounded by
+// safe candidate selection, healthy minimum sizes, and actual free funds.
+const availableFund = Math.max(0, Math.min(
+    manager.funds.available[side],
+    manager.accountTotals[side === 'buy' ? 'buyFree' : 'sellFree']
+));
 
-// Each correction order must be healthy
-const minHealthySize = calculateMinOrderSize(side);
-const affordableOrderCount = Math.floor(availableFunds / minHealthySize);
-const correctionOrders = Math.min(missingSlots, affordableOrderCount);
+// Each target is created or topped up only while remainingBudget covers it.
+// No resting ACTIVE/PARTIAL order is shrunk to fund the correction.
 ```
 
 **Benefits**:
-- ✅ No "double-dust" fragmentation
+- ✅ No inventory recycling from stale-size snapshots
 - ✅ Constant, predictable spread width
 - ✅ Funds always respected (no forced allocation)
 - ✅ Natural smoothing over multiple rebalance cycles
@@ -591,12 +598,11 @@ if (mgr.outOfSpread) {
 ### After (Numeric Distance)
 
 ```javascript
-// New approach: distance in steps
-mgr.outOfSpread = 3;  // 3 steps beyond target spread
+// New approach: requested missing slots derived from the live geometry
+mgr.outOfSpread = 3;  // three slots beyond the target spread
 
-// Use distance in correction logic
-const spreadDistance = mgr.outOfSpread;
-const replacementSlots = Math.min(spreadDistance, MAX_CORRECTION_SLOTS);
+// The grid engine bounds actual work by eligible candidates, MIN_SPREAD_ORDERS,
+// healthy sizes, and available free funds. There is no MAX_CORRECTION_SLOTS cap.
 ```
 
 **Benefit**: Enables scaled corrections based on actual severity.
@@ -706,7 +712,7 @@ stateDiagram-v2
 
     note right of PARTIAL
         Partially filled on-chain
-        Waiting for consolidation
+        Dust is cancelled immediately; non-dust is corrected in rebalance
     end note
 ```
 
@@ -881,7 +887,7 @@ sequenceDiagram
     Strat-->>Mgr: target grid
     Mgr->>Mgr: Apply rotations via WorkingGrid
     Mgr->>Acct: Deduct BTS fees during recalculateFunds
-    Mgr->>Mgr: Consolidate dust partials
+    Mgr->>Mgr: Cancel detected dust partials immediately
 ```
 
 ### 2. Order Rotation (Crawl Mechanism)
@@ -1059,7 +1065,7 @@ The system has been optimized to use a "memory-driven" model for order updates,
 - **Faster reaction time**: No waiting for blockchain queries during order updates
 - **Reduced API load**: Fewer fetches, less network congestion
 - **Mathematical precision**: Integer-based tracking prevents float precision errors
-  - *See [FUND_MOVEMENT_AND_ACCOUNTING.md § 5.5](FUND_MOVEMENT_AND_ACCOUNTING.md#55-precision--quantization-patch-14) for quantization utilities and best practices*
+  - *See [FUND_MOVEMENT_AND_ACCOUNTING.md § 5.5](FUND_MOVEMENT_AND_ACCOUNTING.md#55-precision--quantization) for quantization utilities and best practices*
 - **Fallback safety**: Automatic recovery if memory state becomes inconsistent
 
 ### Performance Impact
@@ -1155,8 +1161,6 @@ graph LR
     B -->|Real-world Scenarios| B3["Fills/FEE Tests"]
 
     C -->|Dynamic Weight| C1["dynamic_weight"]
-    C -->|Derivative Trap| C2["derivative_signal"]
-    C -->|Momentum Gate| C3["momentum_gate"]
 
     D -->|CR Planner| D1["cr_planner"]
     D -->|Credit Runtime| D2["credit_runtime"]
@@ -1189,13 +1193,12 @@ node dist/tests/test_dexbot_credit_wiring.js
 - ✅ Edge cases covered (zero funds, max orders, etc.)
 - ✅ Concurrent operations tested with locks
 - ✅ State transitions validated end-to-end
-- ✅ Signal pipelines tested (dynamic weight, derivative traps)
+- ✅ Signal pipelines tested (dynamic weight)
 - ✅ Credit/debt runtime tested (CR planner, MPA wiring)
 
 **Test Suite Evolution:**
 - 50+ test cases for signal intelligence and credit runtime
 - Dynamic weight override and market adapter signal gate tests
-- Derivative momentum gate and signal trap regression tests
 - Credit/debt tests with CR planner and MPA wiring validation
 
 ### Testing Best Practices
@@ -1323,8 +1326,7 @@ The strategy engine has been significantly strengthened with improvements to fun
 **2. Dust Partial Handling**
 - Improved dust detection algorithm prevents false positives
 - Detects dust as `< 5% of ideal order size`
-- Dust partials are absorbed into the next grid rebuild cycle (no merge/split mechanics)
-- **Auto-Cancellation**: `_cancelDustOrders()` cancels dust partials on-chain immediately on detection — no delay, no timer maps, no retry budgets. Cancel is attempted post-fill (inside fill lock) and every 5-min health check (safety net).
+- **Immediate Cancellation**: `_cancelDustOrders()` cancels dust partials on-chain when detection runs — no delay, timer map, or consolidation cycle. Detection runs after fills/sync, and a 5-minute health check is the restart safety net.
 
 **3. Strict Order Size Constraints**
 - Orders validated to not exceed available funds

package/docs/developer_guide.md

@@ -129,8 +129,6 @@ A **phantom order** is an order in ACTIVE/PARTIAL state WITHOUT a valid `orderId
 | **Kalman Confirmation** | Kalman-filtered trend signal blended with AMA slope for smoother weight transitions |
 | **Symmetric Shift** | Volatility-driven downward weight penalty applied equally to both sides (ATR-based) |
 | **Asymmetric Offset** | Directional weight shift (buy-heavy or sell-heavy) driven by AMA/Kalman trend |
-| **Derivative Signal** | SMA/MACD/RSI-based entry bias and momentum gate for optional strategy filtering |
-| **Momentum Gate** | N-bar commitment tracking that confirms derivative signals before acting |
 | **GridPrice** | Price anchor for grid math; can be numeric, `"pool"`, `"book"`, or AMA keyword (`"ama"`, `"ama1"`–`"ama4"`) |
 
 ### Grid Concepts
@@ -165,7 +163,7 @@ A **phantom order** is an order in ACTIVE/PARTIAL state WITHOUT a valid `orderId
 | Term | Meaning |
 |------|---------|
 | **Rotation** | Moving an order from one price level to another |
-| **Consolidation** | Absorbing dust partials into the next grid rebuild cycle. All slots are treated uniformly—no side-specific flags or bonuses. |
+| **Consolidation** | Absorbing surviving non-dust partials into ordinary fund-driven rebalancing. Detected dust is cancelled immediately and never waits for a rebuild. |
 | **Rebalancing** | Adjusting order sizes based on current funds |
 | **Global Side Capping** | Scaling order sizes when insufficient funds |
 | **Atomic Check-and-Deduct** | Verify funds + deduct in single operation |
@@ -277,7 +275,7 @@ const correctedPrice = quantizeFloat(derivedPrice, 8);
 const normalized = normalizeInt(currentSizeInt, assetPrecision);
 ```
 
-**See [FUND_MOVEMENT_AND_ACCOUNTING.md § 5.5](FUND_MOVEMENT_AND_ACCOUNTING.md#55-precision--quantization-patch-14) for complete quantization guide and edge case handling.**
+**See [FUND_MOVEMENT_AND_ACCOUNTING.md § 5.5](FUND_MOVEMENT_AND_ACCOUNTING.md#55-precision--quantization) for complete quantization guide and edge case handling.**
 
 ---
 
@@ -620,7 +618,7 @@ const minHealthySize = getMinOrderSize(ORDER_TYPES.BUY, assets, 1.0);
 if (isOrderHealthy(order, minHealthySize)) {
     // Order is valid for placement/rotation
 } else {
-    // Order is dust - consolidate or skip
+    // Order is dust - skip it here; cancelDustOrders() handles it on-chain
 }
 ```
 
@@ -737,7 +735,7 @@ _verifyFundInvariants(...)
 - Process filled orders
 - Identify shortages and surpluses
 - Execute order rotations
-- Handle partial order consolidation
+- Handle surviving non-dust partials during rebalance
 
 **Critical Methods**:
 ```javascript
@@ -1211,8 +1209,6 @@ Located in `tests/` (with `helpers/` subdirectory):
 - `test_bts_fee_logic.ts` - BTS fee deduction and settlement
 - `test_market_adapter_signal_gates.ts` - Market adapter signal validation
 - `test_dynamic_weight_override_wiring.ts` - Dynamic weight config wiring
-- `test_derivative_signal_trap_regression.ts` - Derivative signal trap tests
-- `test_derivative_momentum_gate.ts` - Momentum gate tests
 - `test_cr_planner.ts` - Collateral ratio planner tests
 - `test_dexbot_credit_wiring.ts` - Credit runtime integration tests
 - `test_credential_daemon.ts` / `test_credential_session_cache.ts` - Credential security tests

package/market_adapter/README.md

@@ -21,13 +21,13 @@ The live signal layer for AMA-priced bots. It reads candles, computes the AMA ce
 
 | If you want to… | Read this | Key command |
 |-----------------|-----------|-------------|
-| Enable AMA pricing for a bot | [Quick Start](#quick-start) | `dexbot bot` → `6) Adapter` |
+| Enable AMA pricing for a bot | [Quick Start](#quick-start) | `dexbot bot` → `2) Modify bot` → `6) Adapter` |
 | Change how often the grid rebuilds | [Trigger Threshold](#trigger-threshold) | edit `AMA_DELTA_THRESHOLD_PERCENT` |
-| Tune buy/sell weight bias | [Asymmetric Weight Shift](#asymmetric-weight-shift) | `6) Adapter` → Weight |
-| Widen/tighten grid bounds by trend | [Grid Range Scaling](#grid-range-scaling) | `6) Adapter` → Range |
+| Tune buy/sell weight bias | [Asymmetric Weight Shift](#asymmetric-weight-shift) | `dexbot bot` → `2) Modify bot` → `6) Adapter` → Weight |
+| Widen/tighten grid bounds by trend | [Grid Range Scaling](#grid-range-scaling) | `dexbot bot` → `2) Modify bot` → `6) Adapter` → Range |
 | Override settings for one pair or bot | [Settings and Overrides](#settings-and-overrides) | edit `profiles/market_adapter_settings.json` |
 | Run the adapter standalone or test dry-run | [Live Writes and Dry-Run](#live-writes-and-dry-run) | `node dist/market_adapter/market_adapter.js --dryRun` |
-| Debug a bot not being processed | [Troubleshooting](#troubleshooting) | `6) Adapter` flags |
+| Debug a bot not being processed | [Troubleshooting](#troubleshooting) | `dexbot bot` → `2) Modify bot` → `6) Adapter` flags |
 | Understand the signal pipeline or module layout | [Technical Reference](#technical-reference) | — |
 
 ## Big Picture
@@ -50,9 +50,10 @@ applies the symmetric penalty.
 
 ### 1. Enable AMA
 
-Set `gridPrice` to `ama`, `ama1`, `ama2`, `ama3`, or `ama4` in
-`profiles/bots.json` or through `dexbot bot`. Use `ama` for the pair's
-default preset.
+New bots default to `gridPrice: "ama3"`. Change it in `profiles/bots.json`
+or through `dexbot bot` to `ama`, `ama1`, `ama2`, `ama3`, or `ama4`. `ama1`
+reacts fastest, `ama4` is the slowest and steadiest (higher numbers smooth
+more), and `ama` uses the pair's default preset.
 
 `startPrice` selects the candle source:
 
@@ -309,7 +310,7 @@ Dry-run log lines include `[DRY RUN]` or `[suppressed, dry-run]`.
 | Task | Command |
 |------|---------|
 | Enable/inspect Price, Weight, Range for a bot | `dexbot bot` → `2) Modify bot` → `6) Adapter` |
-| Remove a deleted bot's entry | Automatic — `dexbot bot` → `3) Delete` also drops its whitelist key |
+| Remove a deleted bot's entry | Automatic — `dexbot bot` → `3) Delete bot` also drops its whitelist key |
 | Probe public CEX availability | `node dist/market_adapter/inputs/fetch_cex_synthetic_data.js --exchange auto --check-only` |
 | Seed synthetic cross candles | `node dist/market_adapter/inputs/fetch_cex_synthetic_data.js --exchange auto --bot-key <bot-key>` |
 | Run one adapter cycle | `node dist/market_adapter/market_adapter.js --once` |
@@ -348,7 +349,7 @@ match the bot's eventual `botKey`.
 - Check `lastDeltaPercent` vs `thresholdPercent`.
 - Check `staleData` and `staleAgeHours`.
 - **Confirm the bot is whitelisted.** Non-whitelisted bots only log and do not write triggers.
-- Confirm the bot's whitelist entry has `"ama": true` (`dexbot bot` → `6) Adapter` shows it as **Price**).
+- Confirm the bot's whitelist entry has `"ama": true` (`dexbot bot` → `2) Modify bot` → `6) Adapter` shows it as **Price**).
 - Run `node dist/market_adapter/market_adapter.js --once --deltaPercent <lower-value>` for a one-cycle threshold test.
 
 ### Trigger fires too often
@@ -586,6 +587,9 @@ market_adapter/
 |-- inputs/
 |   |-- kibana_source.ts           Elasticsearch LP data source
 |   |-- fetch_lp_data.ts           historical LP candle exporter
+|   |-- fetch_book_data.ts         order-book candle exporter
+|   |-- kibana_feed_source.ts      MPA feed candle source
+|   |-- window_cache.ts            shared month-shard candle cache
 |   `-- fetch_cex_synthetic_data.ts  public CEX synthetic-candle seed importer
 |-- utils/
 |   |-- chain.ts                   blockchain query helpers
@@ -604,7 +608,7 @@ market_adapter/
 ### Whitelist Semantics
 
 `profiles/market_adapter_whitelist.json` controls live writes. It is edited
-per bot in the bot editor (`dexbot bot` → `6) Adapter`), which reads and
+per bot in the bot editor (`dexbot bot` → `2) Modify bot` → `6) Adapter`), which reads and
 writes this same file:
 
 ```json
@@ -828,9 +832,9 @@ suppress writes via `unresolved_candle_gaps` until repaired on a future cycle.
 The adapter prunes old candles to the required AMA window and acts only on
 closed 1h candles.
 
-#### Shared Chunk Cache and Fetch Robustness
+#### Shared Month-Shard Cache and Fetch Robustness
 
-Pool, book, and feed candle fetches share one cache entry point (`runCachedWindows` in `market_adapter/inputs/window_cache.ts`): candles live in fixed calendar-month shards (`<base>.shard_YYYY-MM.json`, UTC) whose names never shift, so a run loads only the shards overlapping its requested range, queries only genuinely missing buckets plus a bounded 48h tail refresh, and rewrites only shards that gained buckets or query coverage — pure-reuse runs perform zero writes and zero deletes. Shard metas record the ranges actually queried (`meta.queriedRanges`, monotonically unioned). A missing range is pruned only when recorded query coverage genuinely covers it — the absence of local buckets alone never certifies history as empty. Partial windows merge into the run output but are never persisted. Legacy run-relative `*.chunk_*` files are still read: overlapping ones are absorbed into the shards (buckets + coverage) and retired once every bucket provably lives in a shard, while disjoint ones are never loaded and never deleted — narrow runs cannot wipe older history by construction. Every range fetch runs through `fetchRangeWithRetry` (per-range attempts + linear backoff + abort-signal timeout; the LP path keeps a 4-attempt budget), one-shot Kibana queries retry transient errors (3 attempts), paged fetchers cap at `kibanaMaxPages` (500), and bidirectional fetches tolerate a one-direction failure.
+Pool, book, and feed candle fetches share one cache entry point (`runCachedWindows` in `market_adapter/inputs/window_cache.ts`): candles live in fixed calendar-month shards (`<base>.shard_YYYY-MM.json`, UTC) whose names never shift, so a run loads only the shards overlapping its requested range, queries only genuinely missing buckets plus a bounded 48h tail refresh, and rewrites only shards that gained buckets or query coverage — pure-reuse runs perform zero writes and zero deletes. Shard metas record the ranges actually queried (`meta.queriedRanges`, monotonically unioned). A missing range is pruned only when recorded query coverage genuinely covers it — the absence of local buckets alone never certifies history as empty. Partial windows merge into the run output but are never persisted. Stable month shards are the only supported cache format; obsolete run-relative cache files are ignored. Every range fetch runs through `fetchRangeWithRetry` (per-range attempts + linear backoff + abort-signal timeout; the LP path keeps a 4-attempt budget), one-shot Kibana queries retry transient errors (3 attempts), paged fetchers cap at `kibanaMaxPages` (500), and bidirectional fetches tolerate a one-direction failure.
 
 #### AMA Warmup Window — Why Candle Length Matters
 

package/modules/README.md

@@ -163,7 +163,7 @@ External consumers — other parts of DEXBot2 interact with modules through thes
 - **Fund-driven sizing, fill-driven boundary** — available funds set order sizing and budget allocation, not the boundary position; no forced allocations. The boundary moves only through boundary crawl on fills or spread promotion onto orders placed in the same atomic batch.
 - **Replay-safe accounting** — fill processing in `dexbot_fill_runtime.ts` uses `processed_fill_store.ts` to prevent double-counting. If the bot restarts mid-fill, it can safely replay without creating duplicate orders.
 - **Daemon-backed signing** — the credential daemon holds decrypted keys; modules never handle raw private keys. If the main bot crashes, keys stay encrypted on disk — only the small daemon process sees them.
-- **Fixed-cap batch processing** — fill batches are capped (default 4) to keep blockchain broadcasts predictable. Even if 20 fills arrive at once, they're processed in small chunks to avoid overwhelming the chain.
+- **Gap-slot batch processing** — each fill batch and broadcast is capped at the current grid's gap-slot count + 1. A queue at or below that depth is processed as one unified batch; deeper queues are chunked at the same deterministic cap.
 
 ## Related
 

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "dexbot",
-  "version": "1.6.5",
+  "version": "1.6.7",
   "description": "The first open source trading bot with zero runtime dependencies and a fully adaptive market making strategy.",
   "main": "dist/modules/dexbot_class.js",
   "exports": {
@@ -14,6 +14,7 @@
   },
   "files": [
     "dist",
+    "!dist/analysis/legacy",
     "ecosystem.config.cjs",
     "scripts/bots",
     "scripts/clean-dist.js",
@@ -40,8 +41,10 @@
     "docs/CREDENTIAL_SECURITY.md",
     "docs/GRID_RECALCULATION.md",
     "docs/GRID_RECONCILE.md",
+    "docs/GRID_PRICE_INVARIANT.md",
     "docs/LOGGING.md",
     "docs/MPA_CREDIT_USAGE.md",
+    "docs/ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md",
     "docs/COPY_ON_WRITE_MASTER_PLAN.md",
     "docs/COW_INVARIANTS.md",
     "docs/FUND_MOVEMENT_AND_ACCOUNTING.md",
@@ -56,6 +59,7 @@
     "modules/README.md",
     "analysis",
     "!analysis/charts",
+    "!analysis/legacy",
     "tests/README.md",
     "CHANGELOG.md",
     "README.md",
@@ -86,7 +90,6 @@
     "market-adapter:fetch-cex-synthetic": "npm run build && node dist/market_adapter/inputs/fetch_cex_synthetic_data.js",
     "analysis:tradingview": "npm run build && node dist/analysis/tradingview/analyze_tradingview.js",
     "ama:chart:lp-local": "npm run build && node dist/analysis/ama_fitting/generate_unified_comparison_chart.js",
-    "analysis:derivatives": "npm run build && node dist/analysis/analyze_derivatives.js",
     "analysis:trade-pnl": "npm run build && node dist/analysis/trade_profitability.js",
     "analysis:grid-check": "npm run build && node dist/analysis/grid_correction_check.js",
     "analysis:resolve-accounts": "npm run build && node dist/analysis/resolve_bot_accounts.js",
@@ -95,15 +98,16 @@
     "native:ecc-invariants": "npm run build && npm run build:tests && node dist/tests/test_native_ecc.js",
     "native:corpus": "npm run build && node dist/scripts/generate_mainnet_corpus_report.js",
     "test:credit-renewal": "npm run build && node dist/scripts/test-credit-renewal.js",
+    "test:legacy": "npm run build && npm run build:tests && node dist/analysis/legacy/tests/test_derivative_chart.js && node dist/analysis/legacy/tests/test_derivative_entry_bias.js && node dist/analysis/legacy/tests/test_derivative_momentum_gate.js && node dist/analysis/legacy/tests/test_derivative_signal_trap_regression.js",
     "version:sync": "npm run build && node dist/scripts/sync-version.js",
     "version:check": "npm run build && node dist/scripts/sync-version.js --check",
     "verify:browser-bundle": "npm run build && node dist/scripts/verify-browser-bundle.js",
     "test": "npm run build && npm run build:tests && npm run build --prefix claw && node dist/scripts/run-tests.js",
     "test:live": "RUN_LIVE_BITSHARES_TESTS=1 npm run build && npm run build:tests && npm run build --prefix claw && node dist/scripts/run-tests.js",
     "clean": "node scripts/clean-dist.js",
-    "build": "tsc && chmod +x dist/dexbot.js dist/bot.js dist/unlock.js dist/pm2.js dist/credential-daemon.js",
+    "build": "echo \"tsc is compiling ...\" && tsc && chmod +x dist/dexbot.js dist/bot.js dist/unlock.js dist/pm2.js dist/credential-daemon.js",
     "build:clean": "npm run clean && npm run build",
-    "build:tests": "tsc -p tsconfig.tests.json && cp tests/package.json dist/tests/package.json && cp tests/helpers/esm_mock_hooks.mjs tests/helpers/esm_mock_loader.mjs dist/tests/helpers/",
+    "build:tests": "tsc -p tsconfig.tests.json && cp tests/package.json dist/tests/package.json && cp tests/helpers/esm_mock_hooks.mjs tests/helpers/esm_mock_loader.mjs dist/tests/helpers/ && mkdir -p dist/analysis/legacy/tests && cp analysis/legacy/tests/package.json dist/analysis/legacy/tests/package.json",
     "typecheck": "tsc --noEmit",
     "typecheck:tests": "tsc -p tsconfig.tests.json --noEmit",
     "build:watch": "tsc --watch",

package/scripts/README.md

@@ -341,7 +341,6 @@ The following scripts allow you to call `dexbot` commands directly from the `scr
 | Command | Purpose |
 |:---|:---|
 | `npm run market-adapter:fetch-cex-synthetic` | Fetch CEX synthetic data for market adapter |
-| `npm run analysis:derivatives` | Derivative analysis report |
 | `npm run analysis:tradingview` | TradingView-style chart export |
 | `npm run analysis:trade-pnl` | Trade PnL analysis from fill data |
 | `npm run ama:chart:lp-local` | Generate local LP comparison chart |
@@ -410,14 +409,6 @@ npm run lp:chart -- --data <lp-export.json>
 npm run ama:chart:lp-local -- --data <lp-export.json>
 ```
 
-### Derivative Trend Analysis
-**File:** `analysis/analyze_derivatives.ts`
-**Purpose:** Generate the derivative analysis report.
-```bash
-# Generate the derivative analysis report
-npm run analysis:derivatives -- --source json --file <file.json>
-```
-
 ---
 
 ## 📚 DOCUMENTATION REFERENCES

package/scripts/reset-settings.sh

@@ -108,7 +108,7 @@ if [ "$REMAINING" -eq 0 ]; then
     log_info ""
     log_info "Next steps:"
     log_info "- Re-run \`dexbot bot\` to recreate general settings if needed"
-    log_info "- Re-open \`dexbot bot\` → 6) Adapter if your AMA bot set changed"
+    log_info "- Re-open \`dexbot bot\` → 2) Modify bot → 6) Adapter if your AMA bot set changed"
     log_info "- Re-fit market profiles only if you want custom AMA presets again"
 else
     log_warning "Cleanup incomplete. Remaining settings files: $REMAINING"

package/analysis/ama_fitting/package.json

@@ -1,12 +0,0 @@
-{
-  "name": "ama_fitting",
-  "version": "1.6.5",
-  "description": "Tools for fitting AMA parameters to market data",
-  "main": "../../dist/analysis/ama_fitting/optimizer_high_resolution.js",
-  "scripts": {
-    "optimize": "npm --prefix ../.. run build && node ../../dist/analysis/ama_fitting/optimizer_high_resolution.js",
-    "chart:lp-local": "npm --prefix ../.. run build && node ../../dist/analysis/ama_fitting/generate_unified_comparison_chart.js"
-  },
-  "dependencies": {},
-  "type": "module"
-}