dexbot 1.6.18 → 1.7.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +68 -0
- package/README.md +24 -18
- package/analysis/README.md +117 -5
- package/claw/package.json +1 -1
- package/claw/runtimes/openclaw-plugin/openclaw.plugin.json +1 -1
- package/claw/runtimes/openclaw-plugin/package.json +1 -1
- package/dist/analysis/account_resolver.js +5 -6
- package/dist/analysis/analyze_dynamic_weight.js +2 -1
- package/dist/analysis/analyze_kalman.js +3 -2
- package/dist/analysis/analyze_liquidity.d.ts +17 -0
- package/dist/analysis/analyze_liquidity.js +336 -0
- package/dist/analysis/analyze_regime.js +3 -2
- package/dist/analysis/analyze_regime_windows.js +3 -2
- package/dist/analysis/analyze_volatility.js +3 -2
- package/dist/analysis/bot_key_utils.d.ts +44 -1
- package/dist/analysis/bot_key_utils.js +76 -7
- package/dist/analysis/fills_cache.js +2 -1
- package/dist/analysis/grid_correction_check.d.ts +1 -19
- package/dist/analysis/grid_correction_check.js +12 -30
- package/dist/analysis/liquidity/liquidity_chain.d.ts +66 -0
- package/dist/analysis/liquidity/liquidity_chain.js +178 -0
- package/dist/analysis/liquidity/liquidity_chart_generator.d.ts +19 -0
- package/dist/analysis/liquidity/liquidity_chart_generator.js +693 -0
- package/dist/analysis/liquidity/liquidity_depth_model.d.ts +166 -0
- package/dist/analysis/liquidity/liquidity_depth_model.js +258 -0
- package/dist/analysis/liquidity/liquidity_view.d.ts +53 -0
- package/dist/analysis/liquidity/liquidity_view.js +66 -0
- package/dist/analysis/resolve_bot_accounts.js +7 -6
- package/dist/analysis/trade_profitability.js +4 -2
- package/dist/analysis/tradingview/analyze_tradingview.js +2 -1
- package/dist/credential-daemon.js +13 -2
- package/dist/dexbot.js +77 -172
- package/dist/modules/account_bots.d.ts +11 -1
- package/dist/modules/account_orders.d.ts +12 -0
- package/dist/modules/account_orders.js +30 -0
- package/dist/modules/asset_lookup.d.ts +25 -0
- package/dist/modules/asset_lookup.js +75 -0
- package/dist/modules/bitshares-native/transport.js +24 -6
- package/dist/modules/bitshares_client.d.ts +24 -6
- package/dist/modules/bitshares_client.js +44 -7
- package/dist/modules/bot_settings.d.ts +2 -0
- package/dist/modules/chain_scan.d.ts +64 -0
- package/dist/modules/chain_scan.js +80 -0
- package/dist/modules/cli_colors.d.ts +5 -0
- package/dist/modules/cli_colors.js +7 -0
- package/dist/modules/cli_progress.d.ts +51 -0
- package/dist/modules/cli_progress.js +113 -0
- package/dist/modules/constants.d.ts +26 -3
- package/dist/modules/constants.js +63 -8
- package/dist/modules/dexbot_class.d.ts +1 -6
- package/dist/modules/dexbot_class.js +14 -0
- package/dist/modules/dexbot_cow_runtime.d.ts +97 -7
- package/dist/modules/dexbot_cow_runtime.js +634 -136
- package/dist/modules/dexbot_fill_runtime.js +132 -23
- package/dist/modules/dexbot_maintenance_runtime.js +242 -74
- package/dist/modules/dexbot_startup_runtime.js +32 -20
- package/dist/modules/dexbot_state_recovery.d.ts +33 -1
- package/dist/modules/dexbot_state_recovery.js +167 -5
- package/dist/modules/launcher/runtime_entry.d.ts +21 -1
- package/dist/modules/launcher/runtime_entry.js +36 -1
- package/dist/modules/liquidity_pool_ref.d.ts +32 -0
- package/dist/modules/liquidity_pool_ref.js +116 -0
- package/dist/modules/node_connect_policy.d.ts +8 -1
- package/dist/modules/node_connect_policy.js +18 -2
- package/dist/modules/node_flap_score.d.ts +87 -0
- package/dist/modules/node_flap_score.js +124 -0
- package/dist/modules/node_health_cache.d.ts +40 -3
- package/dist/modules/node_health_cache.js +66 -20
- package/dist/modules/node_manager.d.ts +101 -19
- package/dist/modules/node_manager.js +255 -65
- package/dist/modules/order/accounting.js +56 -7
- package/dist/modules/order/format.d.ts +24 -10
- package/dist/modules/order/format.js +30 -10
- package/dist/modules/order/grid.js +91 -13
- package/dist/modules/order/grid_reconcile.d.ts +5 -1
- package/dist/modules/order/grid_reconcile.js +333 -189
- package/dist/modules/order/grid_reconcile_internal.d.ts +38 -2
- package/dist/modules/order/grid_reconcile_internal.js +298 -100
- package/dist/modules/order/index.d.ts +19 -1
- package/dist/modules/order/lifecycle.d.ts +353 -0
- package/dist/modules/order/lifecycle.js +1109 -0
- package/dist/modules/order/manager.d.ts +273 -5
- package/dist/modules/order/manager.js +654 -48
- package/dist/modules/order/placement_lock.d.ts +102 -0
- package/dist/modules/order/placement_lock.js +222 -0
- package/dist/modules/order/strategy.js +13 -1
- package/dist/modules/order/sync_engine.d.ts +33 -1
- package/dist/modules/order/sync_engine.js +241 -73
- package/dist/modules/order/utils/math.d.ts +39 -1
- package/dist/modules/order/utils/math.js +83 -1
- package/dist/modules/order/utils/order.d.ts +181 -5
- package/dist/modules/order/utils/order.js +455 -9
- package/dist/modules/order/utils/system.d.ts +81 -1
- package/dist/modules/order/utils/system.js +331 -44
- package/dist/modules/order/utils/validate.d.ts +7 -1
- package/dist/modules/order/utils/validate.js +80 -3
- package/dist/modules/outcome.d.ts +54 -0
- package/dist/modules/outcome.js +51 -0
- package/dist/modules/settings_merge.d.ts +4 -0
- package/dist/modules/settings_merge.js +9 -0
- package/dist/modules/timing_policy.d.ts +82 -0
- package/dist/modules/timing_policy.js +244 -0
- package/dist/modules/types.d.ts +137 -0
- package/dist/modules/utils/asset_symbols.d.ts +13 -2
- package/dist/modules/utils/asset_symbols.js +21 -2
- package/dist/scripts/analysis_runner.d.ts +10 -0
- package/dist/scripts/analysis_runner.js +33 -0
- package/dist/scripts/analyze-credit.js +54 -17
- package/dist/scripts/analyze-orders.d.ts +4 -2
- package/dist/scripts/analyze-orders.js +115 -35
- package/dist/scripts/asset.d.ts +26 -0
- package/dist/scripts/asset.js +165 -0
- package/dist/scripts/bts_distribution.d.ts +1 -0
- package/dist/scripts/bts_distribution.js +194 -142
- package/dist/scripts/chart_command.d.ts +1 -1
- package/dist/scripts/chart_command.js +60 -86
- package/dist/scripts/chart_targets.d.ts +18 -0
- package/dist/scripts/chart_targets.js +48 -0
- package/dist/scripts/dw.js +1 -1
- package/dist/scripts/liquid.d.ts +24 -0
- package/dist/scripts/liquid.js +134 -0
- package/dist/scripts/rich_list.js +12 -16
- package/dist/scripts/test-credit-renewal.js +12 -5
- package/dist/scripts/token_distribution.d.ts +2 -0
- package/dist/scripts/token_distribution.js +132 -108
- package/dist/scripts/tv.js +1 -1
- package/dist/scripts/verify-browser-bundle.js +18 -0
- package/docs/COW_INVARIANTS.md +69 -10
- package/docs/DEXBOT_COMPARISON.md +6 -6
- package/docs/EVOLUTION.md +10 -8
- package/docs/FUND_MOVEMENT_AND_ACCOUNTING.md +1 -1
- package/docs/GRID_PRICE_INVARIANT.md +5 -2
- package/docs/GRID_RECONCILE.md +36 -11
- package/docs/LOGGING.md +50 -4
- package/docs/ORDER_ENGINE_POST_1.0_RETROSPECTIVE.md +177 -27
- package/docs/README.md +23 -2
- package/docs/WORKFLOW.md +12 -4
- package/docs/architecture.md +42 -5
- package/docs/developer_guide.md +25 -8
- package/modules/README.md +8 -1
- package/package.json +3 -1
- package/scripts/README.md +39 -7
- package/scripts/pdev.sh +14 -1
- package/scripts/pmain.sh +23 -8
- package/scripts/ptest.sh +7 -0
- package/tests/README.md +30 -1
|
@@ -35,6 +35,8 @@
|
|
|
35
35
|
* no keys, no transactions. Chain source of truth: bitshares-core 7.0.x.
|
|
36
36
|
*
|
|
37
37
|
* Usage:
|
|
38
|
+
* dexbot asset <symbol|1.3.x> [options] # CLI shortcut
|
|
39
|
+
* dexbot a <symbol|1.3.x> [options] # shorthand for the line above
|
|
38
40
|
* node dist/scripts/token_distribution.js <symbol|1.3.x>
|
|
39
41
|
* node dist/scripts/token_distribution.js <token> --json
|
|
40
42
|
* node dist/scripts/token_distribution.js <asset-id> --no-html
|
|
@@ -47,6 +49,10 @@ import { pathToFileURL } from 'node:url';
|
|
|
47
49
|
import { NODE_MANAGEMENT } from '../modules/constants.js';
|
|
48
50
|
import { PATHS } from '../modules/paths.js';
|
|
49
51
|
import { CLI_COLORS as colors } from '../modules/cli_colors.js';
|
|
52
|
+
import { CLI_TAGS, clearProgressLine, loadingLine, savedLine, section, summaryLine, taggedProgress, warningsBlock, } from '../modules/cli_progress.js';
|
|
53
|
+
import { scanIdRange, DENSE_SPACE_GAP } from '../modules/chain_scan.js';
|
|
54
|
+
import { resolveAssetRef } from '../modules/asset_lookup.js';
|
|
55
|
+
import { toFileUrl } from '../analysis/chart_utils.js';
|
|
50
56
|
// ---------------------------------------------------------------------------
|
|
51
57
|
// Constants
|
|
52
58
|
// ---------------------------------------------------------------------------
|
|
@@ -58,8 +64,19 @@ const NAME_BATCH = 50;
|
|
|
58
64
|
const DEFAULT_DETAIL = 15;
|
|
59
65
|
const DEFAULT_VESTING_MAX = 200_000;
|
|
60
66
|
const DEFAULT_SCAN_GAP = 20_000;
|
|
61
|
-
|
|
67
|
+
// One get_limit_orders call per asset (core exposes no per-asset order index);
|
|
68
|
+
// 32 concurrent reads measured ~2x faster than 12 with an identical report.
|
|
69
|
+
const DEFAULT_CONCURRENCY = 32;
|
|
70
|
+
// Holder pages are offset-addressable (asset_api walks by_asset_balance by
|
|
71
|
+
// position), so the rich list is fetched in parallel pages. The asset_api nodes
|
|
72
|
+
// tolerate this well; 8 keeps ~800 holders in flight.
|
|
73
|
+
const HOLDER_PAGE_CONCURRENCY = 8;
|
|
62
74
|
const DEFAULT_HTML = path.join(PATHS.ANALYSIS.CHARTS_DIR, 'token-distribution.html');
|
|
75
|
+
// Console tag for status lines, matching the `[Liquidity]` style used by the
|
|
76
|
+
// other analysis entries.
|
|
77
|
+
const LOG_TAG = CLI_TAGS.assets;
|
|
78
|
+
const progress = taggedProgress(LOG_TAG);
|
|
79
|
+
const clearProgress = clearProgressLine;
|
|
63
80
|
// asset_api (get_asset_holders) is allowlisted on only a few public nodes; this
|
|
64
81
|
// mirrors scripts/rich_list.ts and scripts/bts_distribution.ts.
|
|
65
82
|
const HOLDER_NODES = [
|
|
@@ -78,13 +95,14 @@ class UsageError extends Error {
|
|
|
78
95
|
const HELP = `BitShares token distribution analyzer (read-only)
|
|
79
96
|
|
|
80
97
|
Usage:
|
|
98
|
+
dexbot asset <symbol|1.3.x> [options] (shorthand: dexbot a <symbol|1.3.x>)
|
|
81
99
|
node dist/scripts/token_distribution.js <symbol|1.3.x> [options]
|
|
82
100
|
|
|
83
101
|
Options:
|
|
84
102
|
--node URL BitShares node (repeatable; defaults to NODE_MANAGEMENT.DEFAULT_NODES)
|
|
85
103
|
--detail N Rows in top-holder / P2P / vesting tables (default ${DEFAULT_DETAIL})
|
|
86
|
-
--no-orders Skip the per-asset limit-order scan (saves ~
|
|
87
|
-
--concurrency N Concurrent get_limit_orders calls (default ${DEFAULT_CONCURRENCY})
|
|
104
|
+
--no-orders Skip the per-asset limit-order scan (~6.5k calls; saves ~15s)
|
|
105
|
+
--concurrency N Concurrent get_limit_orders calls (default ${DEFAULT_CONCURRENCY}, max 64)
|
|
88
106
|
--vesting-max N Highest 1.13.x object id to scan (default ${DEFAULT_VESTING_MAX})
|
|
89
107
|
--gap N Stop the 1.13.x scan after N consecutive missing ids (default ${DEFAULT_SCAN_GAP})
|
|
90
108
|
--html PATH HTML output path (default ${DEFAULT_HTML})
|
|
@@ -345,57 +363,6 @@ async function listPages(fetchPage, limit, maxPages = 10000) {
|
|
|
345
363
|
}
|
|
346
364
|
return out;
|
|
347
365
|
}
|
|
348
|
-
/** Enumerate every asset via list_assets (symbol-ordered, inclusive lower bound). */
|
|
349
|
-
async function listAllAssets(session) {
|
|
350
|
-
const limit = Math.max(1, Math.min(session.config.api_limit_get_assets ?? 100, 100));
|
|
351
|
-
const seen = new Set();
|
|
352
|
-
const out = [];
|
|
353
|
-
let lower = '';
|
|
354
|
-
for (let page = 0; page < 10000; page++) {
|
|
355
|
-
const rows = await callDb(session, 'list_assets', [lower, limit]);
|
|
356
|
-
if (!Array.isArray(rows) || rows.length === 0)
|
|
357
|
-
break;
|
|
358
|
-
let fresh = 0;
|
|
359
|
-
for (const row of rows) {
|
|
360
|
-
if (!row || seen.has(row.id))
|
|
361
|
-
continue;
|
|
362
|
-
seen.add(row.id);
|
|
363
|
-
out.push(row);
|
|
364
|
-
fresh++;
|
|
365
|
-
}
|
|
366
|
-
if (rows.length < limit || fresh === 0)
|
|
367
|
-
break;
|
|
368
|
-
const lastSymbol = rows[rows.length - 1]?.symbol;
|
|
369
|
-
if (typeof lastSymbol !== 'string')
|
|
370
|
-
break;
|
|
371
|
-
lower = lastSymbol;
|
|
372
|
-
}
|
|
373
|
-
return out;
|
|
374
|
-
}
|
|
375
|
-
/**
|
|
376
|
-
* Scan a dense-ish object id range with a gap tolerance. Used for 1.13.x
|
|
377
|
-
* vesting balances, which have no global by-asset index.
|
|
378
|
-
*/
|
|
379
|
-
async function scanIdRange(session, prefix, maxId, gap, onProgress) {
|
|
380
|
-
const objects = [];
|
|
381
|
-
let index = 0;
|
|
382
|
-
let lastSeen = -1;
|
|
383
|
-
while (index <= maxId && index - lastSeen <= gap) {
|
|
384
|
-
const count = Math.min(OBJECT_BATCH, maxId - index + 1);
|
|
385
|
-
const ids = Array.from({ length: count }, (_, i) => `${prefix}.${index + i}`);
|
|
386
|
-
const page = await callDb(session, 'get_objects', [ids]);
|
|
387
|
-
for (let j = 0; j < count; j++) {
|
|
388
|
-
const obj = page?.[j];
|
|
389
|
-
if (obj) {
|
|
390
|
-
lastSeen = index + j;
|
|
391
|
-
objects.push(obj);
|
|
392
|
-
}
|
|
393
|
-
}
|
|
394
|
-
index += count;
|
|
395
|
-
onProgress?.(index);
|
|
396
|
-
}
|
|
397
|
-
return { objects, scannedTo: index, hitLimit: index > maxId && index - lastSeen <= gap };
|
|
398
|
-
}
|
|
399
366
|
async function resolveAccountNames(session, ids) {
|
|
400
367
|
const names = new Map();
|
|
401
368
|
const unique = [...new Set(ids.filter(Boolean))];
|
|
@@ -414,31 +381,94 @@ async function resolveAccountNames(session, ids) {
|
|
|
414
381
|
return names;
|
|
415
382
|
}
|
|
416
383
|
async function resolveAsset(session, input) {
|
|
417
|
-
|
|
418
|
-
|
|
419
|
-
|
|
420
|
-
|
|
421
|
-
|
|
422
|
-
|
|
423
|
-
|
|
424
|
-
stub =
|
|
425
|
-
if (!stub && input.toLowerCase().startsWith('bit')) {
|
|
426
|
-
const retry = await callDb(session, 'lookup_asset_symbols', [[input.slice(3)]]);
|
|
427
|
-
stub = retry?.[0] ?? null;
|
|
428
|
-
}
|
|
384
|
+
// Shared case-insensitive lookup (modules/asset_lookup.ts), so a lowercase
|
|
385
|
+
// symbol such as `token.sym` resolves to the on-chain TOKEN.SYM. The legacy
|
|
386
|
+
// `bit*` MPA aliases were renamed on chain, so those are retried with the
|
|
387
|
+
// prefix stripped.
|
|
388
|
+
const read = (method, args) => callDb(session, method, args);
|
|
389
|
+
let stub = await resolveAssetRef(read, input);
|
|
390
|
+
if (!stub && input.toLowerCase().startsWith('bit')) {
|
|
391
|
+
stub = await resolveAssetRef(read, input.slice(3));
|
|
429
392
|
}
|
|
430
393
|
if (!stub)
|
|
431
394
|
throw new Error(`asset not found: ${input}`);
|
|
432
|
-
|
|
433
|
-
|
|
395
|
+
// The shared resolver fetches through get_assets, so this is already the
|
|
396
|
+
// extended object (float/dynamic ids included): no second round trip.
|
|
397
|
+
return stub;
|
|
434
398
|
}
|
|
435
399
|
async function fetchAssetHolders(session, assetId, onProgress) {
|
|
436
400
|
if (session.assetApi === null)
|
|
437
401
|
throw new Error('asset_api unavailable on this node');
|
|
438
402
|
const api = session.assetApi;
|
|
403
|
+
// The node caps holders per call (api_limit_get_asset_holders, commonly 100).
|
|
439
404
|
const pageSize = Math.max(1, Math.min(session.config.api_limit_get_asset_holders ?? 100, 100));
|
|
405
|
+
let totalCount = null;
|
|
406
|
+
try {
|
|
407
|
+
const count = await session.client.call(api, 'get_asset_holders_count', [assetId]);
|
|
408
|
+
if (typeof count === 'number' && count >= 0)
|
|
409
|
+
totalCount = count;
|
|
410
|
+
}
|
|
411
|
+
catch {
|
|
412
|
+
// optional: only used to plan the pages
|
|
413
|
+
}
|
|
440
414
|
const rows = [];
|
|
441
415
|
let complete = false;
|
|
416
|
+
// Reported while the pages are still in flight (rows is empty until the
|
|
417
|
+
// parallel fetch is reassembled), so count the rows themselves.
|
|
418
|
+
let fetchedRows = 0;
|
|
419
|
+
let fetchedPages = 0;
|
|
420
|
+
const report = (rowsInPage) => {
|
|
421
|
+
fetchedRows += rowsInPage;
|
|
422
|
+
fetchedPages += 1;
|
|
423
|
+
if (fetchedPages % 20 === 0)
|
|
424
|
+
onProgress?.(fetchedRows);
|
|
425
|
+
};
|
|
426
|
+
// `get_asset_holders(asset, start, limit)` walks the by_asset_balance index by
|
|
427
|
+
// *position*, so offsets are independent. With the holder count known the
|
|
428
|
+
// whole rich list is fetched as parallel pages instead of one blocking chain
|
|
429
|
+
// (15k holders = 153 sequential calls today).
|
|
430
|
+
if (totalCount !== null && totalCount > pageSize) {
|
|
431
|
+
const offsets = [];
|
|
432
|
+
for (let start = 0; start < totalCount; start += pageSize)
|
|
433
|
+
offsets.push(start);
|
|
434
|
+
// One sentinel page past the count: it absorbs a balance that appeared
|
|
435
|
+
// (or a rank that shifted) mid-scan, so a full last page cannot be
|
|
436
|
+
// mistaken for the whole list.
|
|
437
|
+
offsets.push(offsets[offsets.length - 1] + pageSize);
|
|
438
|
+
const pages = new Array(offsets.length);
|
|
439
|
+
let cursor = 0;
|
|
440
|
+
const worker = async () => {
|
|
441
|
+
for (;;) {
|
|
442
|
+
const index = cursor++;
|
|
443
|
+
if (index >= offsets.length)
|
|
444
|
+
return;
|
|
445
|
+
const page = await session.client.call(api, 'get_asset_holders', [assetId, offsets[index], pageSize]);
|
|
446
|
+
pages[index] = Array.isArray(page) ? page : [];
|
|
447
|
+
report(pages[index].length);
|
|
448
|
+
}
|
|
449
|
+
};
|
|
450
|
+
const workers = Math.min(HOLDER_PAGE_CONCURRENCY, offsets.length);
|
|
451
|
+
await Promise.all(Array.from({ length: workers }, () => worker()));
|
|
452
|
+
// Reassemble in rank order. A balance change mid-scan can shift ranks and
|
|
453
|
+
// repeat an account, so keep its highest-ranked occurrence only.
|
|
454
|
+
const seen = new Set();
|
|
455
|
+
let shortPage = false;
|
|
456
|
+
for (const page of pages) {
|
|
457
|
+
for (const holder of page) {
|
|
458
|
+
if (!holder?.account_id || seen.has(holder.account_id))
|
|
459
|
+
continue;
|
|
460
|
+
seen.add(holder.account_id);
|
|
461
|
+
rows.push(holder);
|
|
462
|
+
}
|
|
463
|
+
if (page.length < pageSize) {
|
|
464
|
+
shortPage = true;
|
|
465
|
+
break;
|
|
466
|
+
}
|
|
467
|
+
}
|
|
468
|
+
complete = shortPage ? true : rows.length >= totalCount;
|
|
469
|
+
onProgress?.(rows.length);
|
|
470
|
+
return { rows, totalCount, complete };
|
|
471
|
+
}
|
|
442
472
|
let nextProgress = 10000;
|
|
443
473
|
for (let start = 0; start < 2_000_000; start += pageSize) {
|
|
444
474
|
const page = await session.client.call(api, 'get_asset_holders', [assetId, start, pageSize]);
|
|
@@ -456,15 +486,6 @@ async function fetchAssetHolders(session, assetId, onProgress) {
|
|
|
456
486
|
break;
|
|
457
487
|
}
|
|
458
488
|
}
|
|
459
|
-
let totalCount = null;
|
|
460
|
-
try {
|
|
461
|
-
const count = await session.client.call(api, 'get_asset_holders_count', [assetId]);
|
|
462
|
-
if (typeof count === 'number')
|
|
463
|
-
totalCount = count;
|
|
464
|
-
}
|
|
465
|
-
catch {
|
|
466
|
-
// optional
|
|
467
|
-
}
|
|
468
489
|
return { rows, totalCount, complete };
|
|
469
490
|
}
|
|
470
491
|
// ---------------------------------------------------------------------------
|
|
@@ -581,19 +602,6 @@ function esc(text) {
|
|
|
581
602
|
}
|
|
582
603
|
});
|
|
583
604
|
}
|
|
584
|
-
// ---------------------------------------------------------------------------
|
|
585
|
-
// Collection
|
|
586
|
-
// ---------------------------------------------------------------------------
|
|
587
|
-
function progress(enabled, message) {
|
|
588
|
-
if (!enabled || !process.stderr.isTTY)
|
|
589
|
-
return;
|
|
590
|
-
process.stderr.write(`\r\x1b[2K ${message}`);
|
|
591
|
-
}
|
|
592
|
-
function clearProgress(enabled) {
|
|
593
|
-
if (!enabled || !process.stderr.isTTY)
|
|
594
|
-
return;
|
|
595
|
-
process.stderr.write('\r\x1b[2K');
|
|
596
|
-
}
|
|
597
605
|
async function scanLimitOrders(session, tokenId, assets, limit, concurrency, onProgress) {
|
|
598
606
|
const byCounterpart = new Map();
|
|
599
607
|
const depthByCounterpart = new Map();
|
|
@@ -737,8 +745,22 @@ async function collect(session, opts) {
|
|
|
737
745
|
const precision = asset.precision;
|
|
738
746
|
const money = (value) => formatUnits(value, precision);
|
|
739
747
|
// ---- Assets (symbols + order scan universe) -----------------------------
|
|
740
|
-
|
|
741
|
-
|
|
748
|
+
// Assets are enumerated by id instead of `list_assets` (100 assets per page):
|
|
749
|
+
// the 1.3.x space is dense and append-only, so a batched get_objects walk is
|
|
750
|
+
// the same payload with ~7 round trips instead of ~66. The object list also
|
|
751
|
+
// feeds the limit-order sweep below.
|
|
752
|
+
progress(showProgress, 'listing assets (1.3.x) ...');
|
|
753
|
+
const assetScan = await scanIdRange({
|
|
754
|
+
read: (method, args) => callDb(session, method, args),
|
|
755
|
+
prefix: '1.3',
|
|
756
|
+
maxId: 200_000,
|
|
757
|
+
gap: DENSE_SPACE_GAP,
|
|
758
|
+
onProgress: (n) => progress(showProgress, `listing assets (1.3.x) ... ${n} ids`),
|
|
759
|
+
});
|
|
760
|
+
const assets = assetScan.objects;
|
|
761
|
+
if (assetScan.hitLimit) {
|
|
762
|
+
warnings.push('Asset enumeration hit its 200,000-id limit; the limit-order scan may miss newer assets.');
|
|
763
|
+
}
|
|
742
764
|
const assetById = new Map(assets.map((row) => [row.id, row]));
|
|
743
765
|
assetById.set(asset.id, asset);
|
|
744
766
|
const symbolOf = (id) => assetById.get(id)?.symbol ?? id;
|
|
@@ -877,7 +899,13 @@ async function collect(session, opts) {
|
|
|
877
899
|
}
|
|
878
900
|
// ---- Vesting balances (1.13.x) ------------------------------------------
|
|
879
901
|
progress(showProgress, 'scanning vesting balances (1.13.x) ...');
|
|
880
|
-
const vestingScan = await scanIdRange(
|
|
902
|
+
const vestingScan = await scanIdRange({
|
|
903
|
+
read: (method, args) => callDb(session, method, args),
|
|
904
|
+
prefix: '1.13',
|
|
905
|
+
maxId: opts.vestingMax,
|
|
906
|
+
gap: opts.gap,
|
|
907
|
+
onProgress: (n) => progress(showProgress, `scanning vesting balances (1.13.x) ... ${n} ids`),
|
|
908
|
+
});
|
|
881
909
|
let vestingTotal = 0n;
|
|
882
910
|
let vestingCount = 0;
|
|
883
911
|
const vestingByType = new Map();
|
|
@@ -1299,11 +1327,8 @@ function renderTerminal(report, opts) {
|
|
|
1299
1327
|
if (report.locked > 0n && report.float > 0n) {
|
|
1300
1328
|
console.log(c(colors.gray, ` locked / float ${(Number(report.locked * 10000n / report.float) / 10_000).toFixed(2)}×`));
|
|
1301
1329
|
}
|
|
1302
|
-
const
|
|
1303
|
-
|
|
1304
|
-
console.log(c(colors.bold + colors.yellow, ` ── ${title} ${'─'.repeat(Math.max(0, 70 - title.length))}`));
|
|
1305
|
-
};
|
|
1306
|
-
section('Where the token sits');
|
|
1330
|
+
const heading = (title) => { console.log(section(title)); };
|
|
1331
|
+
heading('Where the token sits');
|
|
1307
1332
|
console.log(c(colors.gray, ` ${padEnd('Category', 26)} ${padStart(symbol, 18)} ${padStart('% supply', 10)} ${padStart('% float', 10)}`));
|
|
1308
1333
|
for (const bucket of report.buckets) {
|
|
1309
1334
|
console.log(` ${padEnd(bucket.label.slice(0, 26), 26)} ${padStart(money(bucket.amount), 18)} `
|
|
@@ -1408,13 +1433,10 @@ function renderTerminal(report, opts) {
|
|
|
1408
1433
|
console.log(` HTLC escrow ${money(report.htlcs.total)} in ${report.htlcs.count} HTLC(s)`);
|
|
1409
1434
|
}
|
|
1410
1435
|
}
|
|
1411
|
-
if (report.warnings.length > 0)
|
|
1412
|
-
|
|
1413
|
-
for (const warning of report.warnings)
|
|
1414
|
-
console.log(c(colors.sell, ` • ${warning}`));
|
|
1415
|
-
}
|
|
1436
|
+
if (report.warnings.length > 0)
|
|
1437
|
+
console.log(warningsBlock(report.warnings));
|
|
1416
1438
|
console.log();
|
|
1417
|
-
console.log(
|
|
1439
|
+
console.log(summaryLine(`reconcile: free ${money(report.free)} + locked ${money(report.locked)} `
|
|
1418
1440
|
+ `+ blinded ${money(report.supply.confidential)} + other ${money(report.residual)} = ${money(supply)}`));
|
|
1419
1441
|
}
|
|
1420
1442
|
function groupSlices(slices, total) {
|
|
@@ -1802,7 +1824,7 @@ ${warnings}
|
|
|
1802
1824
|
</div>
|
|
1803
1825
|
|
|
1804
1826
|
<footer>
|
|
1805
|
-
<p>Sources: bitshares-core 7.0.x database_api — <code>
|
|
1827
|
+
<p>Sources: bitshares-core 7.0.x database_api — <code>get_objects</code> (1.3.x assets, 1.13.x vesting), <code>get_limit_orders</code>, <code>get_liquidity_pools_by_one_asset</code>, <code>get_credit_offers_by_asset</code>, <code>get_credit_deals_by_collateral_asset</code>, <code>get_credit_deals_by_debt_asset</code>, <code>get_samet_funds_by_asset</code>, <code>list_htlcs</code>${report.holders.sourceNode ? ', <code>asset_api::get_asset_holders</code>' : ''}.</p>
|
|
1806
1828
|
<p>Read-only; no keys, no transactions. Limit-order totals are per market and capped by the node's per-side limit.</p>
|
|
1807
1829
|
</footer>
|
|
1808
1830
|
</body>
|
|
@@ -1826,7 +1848,7 @@ async function main() {
|
|
|
1826
1848
|
}
|
|
1827
1849
|
catch (e) {
|
|
1828
1850
|
if (e instanceof UsageError) {
|
|
1829
|
-
console.error(
|
|
1851
|
+
console.error(`${LOG_TAG} Error: ${e.message}\n`);
|
|
1830
1852
|
console.error(HELP);
|
|
1831
1853
|
process.exit(2);
|
|
1832
1854
|
}
|
|
@@ -1836,6 +1858,8 @@ async function main() {
|
|
|
1836
1858
|
console.log(HELP);
|
|
1837
1859
|
return;
|
|
1838
1860
|
}
|
|
1861
|
+
if (!opts.json)
|
|
1862
|
+
console.log(loadingLine(LOG_TAG, `Loading token distribution for ${opts.asset} from chain...`));
|
|
1839
1863
|
const nodes = opts.nodes.length > 0 ? opts.nodes : [...NODE_MANAGEMENT.DEFAULT_NODES];
|
|
1840
1864
|
const session = await openSession(nodes);
|
|
1841
1865
|
try {
|
|
@@ -1854,7 +1878,7 @@ async function main() {
|
|
|
1854
1878
|
else {
|
|
1855
1879
|
renderTerminal(report, opts);
|
|
1856
1880
|
if (htmlPath)
|
|
1857
|
-
console.log(
|
|
1881
|
+
console.log(savedLine(LOG_TAG, 'Report', toFileUrl(htmlPath)));
|
|
1858
1882
|
}
|
|
1859
1883
|
}
|
|
1860
1884
|
finally {
|
|
@@ -1863,7 +1887,7 @@ async function main() {
|
|
|
1863
1887
|
}
|
|
1864
1888
|
if (process.argv[1] && import.meta.url === pathToFileURL(process.argv[1]).href) {
|
|
1865
1889
|
main().catch((e) => {
|
|
1866
|
-
console.error(`\
|
|
1890
|
+
console.error(`\n${LOG_TAG} Error: ${e.message}`);
|
|
1867
1891
|
process.exit(1);
|
|
1868
1892
|
});
|
|
1869
1893
|
}
|
package/dist/scripts/tv.js
CHANGED
|
@@ -2,7 +2,7 @@
|
|
|
2
2
|
'use strict';
|
|
3
3
|
/**
|
|
4
4
|
* Usage:
|
|
5
|
-
* dexbot tv <bot|pool
|
|
5
|
+
* dexbot tv <bot|pool|AssetA/AssetB> [--month N] [--chart <path>] [--feed|--pool|--book]
|
|
6
6
|
*
|
|
7
7
|
* Thin entry for the shared chart-command pipeline in scripts/chart_command.ts
|
|
8
8
|
* (identical fetch/target logic as `dexbot dw`). This command renders through
|
|
@@ -38,6 +38,24 @@ const TESTS = [
|
|
|
38
38
|
expectFail: false,
|
|
39
39
|
externalizeNodeBuiltins: true,
|
|
40
40
|
},
|
|
41
|
+
{
|
|
42
|
+
label: 'modules/liquidity_pool_ref.ts (browser-safe)',
|
|
43
|
+
entry: 'modules/liquidity_pool_ref.ts',
|
|
44
|
+
expectFail: false,
|
|
45
|
+
externalizeNodeBuiltins: true,
|
|
46
|
+
},
|
|
47
|
+
{
|
|
48
|
+
label: 'modules/chain_scan.ts (browser-safe)',
|
|
49
|
+
entry: 'modules/chain_scan.ts',
|
|
50
|
+
expectFail: false,
|
|
51
|
+
externalizeNodeBuiltins: true,
|
|
52
|
+
},
|
|
53
|
+
{
|
|
54
|
+
label: 'modules/asset_lookup.ts (browser-safe)',
|
|
55
|
+
entry: 'modules/asset_lookup.ts',
|
|
56
|
+
expectFail: false,
|
|
57
|
+
externalizeNodeBuiltins: true,
|
|
58
|
+
},
|
|
41
59
|
{
|
|
42
60
|
label: 'modules/crypto/index.ts (browser-safe)',
|
|
43
61
|
entry: 'modules/crypto/index.ts',
|
package/docs/COW_INVARIANTS.md
CHANGED
|
@@ -20,6 +20,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
20
20
|
| `INV-COW` | COW pipeline |
|
|
21
21
|
| `INV-PROJ` | Projection |
|
|
22
22
|
| `INV-ID` | Order identity |
|
|
23
|
+
| `INV-LIFE` | Order lifecycle (chain existence) |
|
|
23
24
|
| `INV-ACC` | Accounting / fund tracking |
|
|
24
25
|
| `INV-SYNC` | Sync engine |
|
|
25
26
|
| `INV-MAINT` | Maintenance runtime |
|
|
@@ -30,6 +31,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
30
31
|
| `INV-REG` | Fund registry |
|
|
31
32
|
| `INV-SUB` | Subscriptions |
|
|
32
33
|
| `INV-BROADCAST` | Broadcast |
|
|
34
|
+
| `INV-TIME` | Timing windows |
|
|
33
35
|
|
|
34
36
|
---
|
|
35
37
|
|
|
@@ -57,12 +59,12 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
57
59
|
- `_clearOrderCachesLogic` must bump `_gridVersion` so an in-flight COW plan (baseVersion from the pre-swap grid) is refused at commit instead of committing over a regenerated zero-slot grid.
|
|
58
60
|
|
|
59
61
|
- `INV-COW-006` Boundary moves only on fills and same-batch spread promotion
|
|
60
|
-
- Fund changes never move the boundary. Only boundary crawl (`deriveTargetBoundary`, `order/utils/order.ts`), spread promotion, and the fill-driven rail clamp move it
|
|
62
|
+
- Fund changes never move the boundary. Only boundary crawl (`deriveTargetBoundary`, `order/utils/order.ts`), spread promotion, and the fill-driven rail clamp move it. Promotion shifts it onto orders placed in the same atomic batch **and may absorb a same-side order already placed inside the band** (that order is rail-typed by construction — a placed SPREAD slot is `ILLEGAL_SPREAD_STATE` in `validateOrder`, so "same-side" needs no extra resolution); it becomes a legal rail member once the boundary slides past it. Only the empty band slot is a CREATE target, and the absorption still counts against the reserve/stranding depth caps.
|
|
61
63
|
- Funds drive order sizing/budget allocation only; `syncBoundaryToFunds`/`calculateFundDrivenBoundary` are deleted (1.5.3).
|
|
62
|
-
- `deriveRailBoundary` (`dexbot_cow_runtime.ts`, pre-broadcast) clamps the committed boundary to the batch's resulting rails: `maxBuyIdx <= boundary <= minSellIdx - gapSlots - 1`. A fill's counter order is the rail fact, so it decides the boundary; an empty refill hole never outvotes it. Rail membership = the batch's own op destinations + master rail orders on their own side of the planned boundary (BUY at/below, SELL at/above the implied sellStart); a master order parked on the wrong side is a STRAY (refused-commit adoption, SPREAD-GUARD straggler, stranded hold) and is skipped, because clamping onto it would self-legalize the corrupt state (live: one stray sell clamped 138 → 115). Rail indices are the orders' LIVE chain price (`resolveLiveOrderLadderPosition`), not stale slot ids. A crossed/over-ceiling rail fails closed (`rail-crossed` / `rail-exceeds-grid`) and keeps the previous value instead of legalizing it; the result then passes the shared `validatePersistedBoundary` gate (with cancel/rotation sources allow-listed), so the rail clamp and the fill re-anchor end in one validator.
|
|
64
|
+
- `deriveRailBoundary` (`dexbot_cow_runtime.ts`, pre-broadcast) clamps the committed boundary to the batch's resulting rails: `maxBuyIdx <= boundary <= minSellIdx - gapSlots - 1`. A fill's counter order is the rail fact, so it decides the boundary; an empty refill hole never outvotes it. Rail membership = the batch's own op destinations + master rail orders on their own side of the planned boundary (BUY at/below, SELL at/above the implied sellStart); a master order parked on the wrong side is a STRAY (refused-commit adoption, SPREAD-GUARD straggler, stranded hold) and is skipped, because clamping onto it would self-legalize the corrupt state (live: one stray sell clamped 138 → 115). Rail indices are the orders' LIVE chain price (`resolveLiveOrderLadderPosition`), not stale slot ids. A crossed/over-ceiling rail fails closed (`rail-crossed` / `rail-exceeds-grid`) and keeps the previous value instead of legalizing it; the result then passes the shared `validatePersistedBoundary` gate (with cancel/rotation sources allow-listed), so the rail clamp and the fill re-anchor end in one validator. The gate accepts a **violation-reducing repair**: a candidate that strands strictly fewer placed orders than the committed boundary is allowed (the fill-driven retreat case), while an equal count stays refused. The comparison uses the validator's own in-band count (`countInBandPlacements` / `boundaryMoveReducesStrands`), never a second stranding predicate.
|
|
63
65
|
|
|
64
66
|
- `INV-COW-007` Boundary hold on guard-skipped refills
|
|
65
|
-
- When a slot on the plan's refill wire (`collectRefillSlotIds`: the plan's CREATE ids minus edge-ladder edge ids) was guard-skipped at broadcast (`skippedUpdateSlotIds` ∪ `clampedUpdateSlotIds` ∪ `skippedCreateSlotIds`), `resolveRefillBoundaryHold` keeps the COMMITTED boundary instead of committing the plan's shifted one over the stranded rail hole; the grid still commits.
|
|
67
|
+
- When a slot on the plan's refill wire (`collectRefillSlotIds`: the plan's CREATE ids minus edge-ladder edge ids) was guard-skipped at broadcast (`skippedUpdateSlotIds` ∪ `clampedUpdateSlotIds` ∪ `skippedCreateSlotIds`), `resolveRefillBoundaryHold` keeps the COMMITTED boundary instead of committing the plan's shifted one over the stranded rail hole; the grid still commits. The hold is scoped to the **promotion direction** (the move advances toward the band holes): a fill-driven retreat moves away from them, so its skipped refills must not pin the old boundary. `resolveRefillSide` supplies the plan's rail side; an unknown side keeps the legacy hold (fail safe).
|
|
66
68
|
- Edge-ladder CREATEs are excluded because reserves are static edge insurance whose fills never crawl — a skipped reserve must not pin geometry.
|
|
67
69
|
- The hold is intersected with `deriveRailBoundary`: when the batch actually placed a counter order above the committed boundary, the rail clamp advances the boundary (the hold becomes a no-op and the counter stays live). The hold still applies when nothing was placed on the promoted rail. `boundaryHeld` is `effectiveBoundary !== workingBoundary`, so crawl consumption follows the value actually committed.
|
|
68
70
|
- The hold rides to the commit as `options.boundaryHeld`, which suppresses the pending-crawl clear.
|
|
@@ -73,7 +75,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
73
75
|
- `INV-COW-008` Owed fill crawls survive unapplied commits
|
|
74
76
|
- `order/strategy.ts` records a crawl per shift-eligible fill at intake (slot-level dedupe, capped at 500); `deriveTargetBoundary` folds still-owed `_pendingFillCrawls` into the next derivation, excluding the current batch's slots, edge slots, and window members — a window that reaches the grid edge is not an edge order, using the same window exclusion every placement picker and `countLiveEdgeOrders` apply.
|
|
75
77
|
- `_commitWorkingGrid` clears them only when the plan's boundary was actually applied — a held boundary (`boundaryHeld`), a gate-rejected boundary, and a null boundary all leave them owed.
|
|
76
|
-
- A window that lands BEHIND the newest real fill is repaired by `reanchorBoundaryFromLastFill` (called from `calculateTargetGrid` right after the derivation): the fill anchors the geometry — `SELL fill at f => boundary ≥ f − gapSlots`, `BUY fill at f => boundary ≤ f − 1` — because a ±1 crawl can never catch up. The anchor index is the current fill batch's EXTREMUM for the fill side (max slot for SELL, min slot for BUY), not the mutable last-processed pivot: a sweep processed in chunks overwrites the pivot per chunk (live: 1366 → 1349 → 1361), and anchoring on a non-extremum leaves the furthest fill's counter order in-band. Prices come from the fill record (the real execution price, `INV-COW-011`) with the slot id as a legacy fallback. Fail-closed preconditions: a pivot whose `lastFillPivotSource` is `fill` (a book seed never anchors; the bot-scoped last-market-fill row is the fallback), on-ladder price within one increment, a post-clamp target that actually puts the fill outside the window, and `validatePersistedBoundary`
|
|
78
|
+
- A window that lands BEHIND the newest real fill is repaired by `reanchorBoundaryFromLastFill` (called from `calculateTargetGrid` right after the derivation): the fill anchors the geometry — `SELL fill at f => boundary ≥ f − gapSlots`, `BUY fill at f => boundary ≤ f − 1` — because a ±1 crawl can never catch up. The anchor index is the current fill batch's EXTREMUM for the fill side (max slot for SELL, min slot for BUY), not the mutable last-processed pivot: a sweep processed in chunks overwrites the pivot per chunk (live: 1366 → 1349 → 1361), and anchoring on a non-extremum leaves the furthest fill's counter order in-band. Prices come from the fill record (the real execution price, `INV-COW-011`) with the slot id as a legacy fallback. Fail-closed preconditions: a pivot whose `lastFillPivotSource` is `fill` (a book seed never anchors; the bot-scoped last-market-fill row is the fallback), on-ladder price within one increment, a post-clamp target that actually puts the fill outside the window, and `validatePersistedBoundary` — a strand refuses the re-anchor unless the candidate strands **strictly fewer** placed orders than the committed boundary (the violation-reducing repair, measured with the validator's own count; an equal count still refuses). ELIGIBILITY matches the crawl: EDGE-pinned insurance fills never arm the pivot (`recordLastFilledPrices` skips them, shared classifier `collectEdgePinnedFillIds`) and so can never anchor — an edge order is static insurance, not market movement, and an absolute anchor at a rail-edge price would teleport the window there; PARTIAL fills still anchor (real trade) because the fill's OWN PARTIAL residual at the anchor slot is explicitly allow-listed (it is inside the band by construction). Startup/recovery load paths tolerate that pure in-band state (`tolerateTransientStranding`) so a restart does not rebuild the boundary before GAP-EVAC clears the residual. A successful re-anchor is absolute: it consumes the whole pending-crawl snapshot taken at plan time, so no older delta can be applied on top.
|
|
77
79
|
- Consumption is SCOPED to what the derivation applied: `deriveTargetBoundary` returns `consumedCrawlIds` (the owed records whose shift it folded, plus the records it deliberately never counts — current-batch slots, edge-classified slots, malformed records), the plan carries it (`buildSuccessResult` → `CowResult` → `_commitWorkingGrid` options), and `_consumePendingFillCrawls` drops exactly those, logging `Dropped N … kept=M`. A capped fold consumes only the entries it actually applied (same-side prefix up to the applied magnitude), so the remainder stays owed exactly once. `_clearPendingFillCrawls` remains the explicit allowed-loss path (rebuild, rejected snapshot, restart replay) and drops everything.
|
|
78
80
|
- A commit **without** the wire consumes nothing (a create-only plan never derived a boundary): clearing it would be the old silent-loss bug (2026-10-25: 7 slots 147–153 swept, every record dropped, boundary ending 7 levels behind).
|
|
79
81
|
- `consumePendingFillCrawls` applies them onto a restored finite boundary; `applyPersistedPendingCrawls` is the shared startup/recovery wrapper used before sync/reconcile; `_clearPendingFillCrawls` drops them when the boundary is re-anchored (grid rebuild via `initializeGrid`, rejected snapshot via `rejectCorruptedGridSnapshot`, persisted snapshot wipe via `AccountOrders.clearGrid`).
|
|
@@ -83,9 +85,9 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
83
85
|
- Provenance gate: only a pivot written through the shared `setLastFillPivot(type, price, 'fill')` writer is persist-eligible. A book-seeded pivot (`seedLastFilledPricesFromBook`, provenance `'book'`) is a heuristic — max resting buy / min resting sell is not market truth — and is never fossilized into the snapshot. The fill-batch and queued-fill refresh sites funnel through one implementation (free function in `utils/system.ts`; `OrderManager._setLastFillPivot` delegates to it), so value, timestamp, and provenance cannot drift apart; the queued-fill refresh and the restore path call the same free function directly, with the legacy-stub fallback built into the shared writer rather than copy-pasted. A single-sided book arms through the same writer with provenance `'book'`; the two-sided midpoint and per-side mirror seeds stay direct writes by design, because they carry no pivot provenance and must not be persist-eligible.
|
|
84
86
|
- The row carries `{price, type, fillsAt, genesisHash}` validated by one shared gate (`normalizeLastFillPivot`) used by BOTH the `storeMasterGrid` sanitizer and the loader — the two shape checks cannot drift. `null` from the payload builder always clears the stored row (consumed pivot / cold manager / no live genesis), `undefined` stays a legacy no-op for old callers, and a malformed shape also clears.
|
|
85
87
|
- Restore ordering: `grid.loadGrid` restores the pivot right after `_restoreBoundary` — with the persisted genesis applied and the grid re-typed, BEFORE the first reconcile/broadcast — so a restart cannot re-open the cold window where evacuation stamps get applied against a boundary still being rebuilt. All `loadGrid` callers (startup resume, price-match resume, recovery reload) inherit it; there is no second restore call site to drift.
|
|
86
|
-
- Validation chain (`restoreLastFillPivot`): TTL expiry (`GRID_LIMITS.LAST_FILL_PIVOT_TTL_MS`, 24h; the pivot is a "latest fill" fact, not a permanent ratchet; expired pivots hand over to the startup book seed) and genesis binding (mismatched hash = dead generation) erase the row through one shared drop path (`clearPersistedLastFillPivot` under the bot's persistence lock, best-effort). Off-grid refusal (`resolveOnGridPivot` — the runtime's own ladder validator, imported lazily from the restore site so ONE implementation serves both the per-probe live guard and the restore; the restorer can never accept a value the runtime would refuse per probe) is a no-op that falls back to the book seed and leaves the row for the next snapshot flush to clear (the cold in-memory pivot makes the payload builder return null). The snapped ladder level is restored, never the raw float, and the ORIGINAL `fillsAt` is preserved through `setLastFillPivot`'s `atMs` param so the TTL keeps meaning "age of the last fill", not "time since this restart".
|
|
88
|
+
- Validation chain (`restoreLastFillPivot`): TTL expiry (`GRID_LIMITS.LAST_FILL_PIVOT_TTL_MS`, 24h; the pivot is a "latest fill" fact, not a permanent ratchet; expired pivots hand over to the startup book seed) and genesis binding (mismatched hash = dead generation) erase the row through one shared drop path (`clearPersistedLastFillPivot` under the bot's persistence lock, best-effort). The SAME TTL bounds the in-memory reference at READ time (`resolveLastFillGuardReference` / `resolveFillProvenancedRow`), because the geometry-bind watchdog re-centers at most once per pivot and reports that a fill or the TTL restores the side — that promise must not depend on a restart. Off-grid refusal (`resolveOnGridPivot` — the runtime's own ladder validator, imported lazily from the restore site so ONE implementation serves both the per-probe live guard and the restore; the restorer can never accept a value the runtime would refuse per probe) is a no-op that falls back to the book seed and leaves the row for the next snapshot flush to clear (the cold in-memory pivot makes the payload builder return null). The snapped ladder level is restored, never the raw float, and the ORIGINAL `fillsAt` is preserved through `setLastFillPivot`'s `atMs` param so the TTL keeps meaning "age of the last fill", not "time since this restart".
|
|
87
89
|
- Generation invalidation mirrors the pending-crawl ledger: `initializeGrid` and `rejectCorruptedGridSnapshot` clear the in-memory pivot via `resetLastFillPivot` (exposed as `OrderManager._resetLastFillPivot`), which clears the FULL scalar family — per-side mirrors included — so `seedLastFilledPricesFromBook`'s cold gate cannot be silently suppressed by stale per-side values after a grid rebuild, and `AccountOrders.clearGrid` wipes the persisted row with the snapshot.
|
|
88
|
-
- A rebuild then RE-ARMS explicitly (`rearmLastFillPivotFromMarketFill`) from the bot-scoped last-market-fill row, through the same writer, at the original timestamp and with provenance `'fill'`. The guard must not go cold while the account's last trade price is known (pre-1.6.14 behaviour: a rebuild let a fresh grid sell below the last sale), and the re-arm must be a logged, persisted write rather than an accident of the read-time fallback — the rebuild site's comment and the runtime behaviour were allowed to differ before this. Consequence, pinned by test: the pivot SURVIVES a rebuild
|
|
90
|
+
- A rebuild then RE-ARMS explicitly (`rearmLastFillPivotFromMarketFill`) from the bot-scoped last-market-fill row, through the same writer, at the original timestamp (taken from the same row that supplied the price, so a book-seeded live pivot never lends its clock) and with provenance `'fill'`. The guard must not go cold while the account's last trade price is known (pre-1.6.14 behaviour: a rebuild let a fresh grid sell below the last sale), and the re-arm must be a logged, persisted write rather than an accident of the read-time fallback — the rebuild site's comment and the runtime behaviour were allowed to differ before this. Consequence, pinned by test: the pivot SURVIVES a rebuild, bounded by the same 24h TTL (enforced at restore AND at read time; an expired row is neither used as a fallback nor re-armed). If the market has moved away from that price, the same-side rail reads `last-fill-blocked` and only a fill (or TTL expiry) restores it; the geometry-bind watchdog therefore re-centers at most ONCE per pivot (`_lastGeometryResyncPivotKey`) — a re-center cannot move the pivot, so a second attempt would only churn the new generation — and then reports that fact instead of escalating.
|
|
89
91
|
- EDGE-PINNED fills never arm the pivot: `recordLastFilledPrices` skips the ids returned by `collectEdgePinnedFillIds` (the same window/edge classifier the placement pickers use). An edge order is static fat-finger insurance, not market movement; letting a wick on one arm the pivot at a rail-edge price would veto that whole side of the book AND, because the pivot doubles as the re-anchor's absolute geometry anchor, teleport the window to the edge.
|
|
90
92
|
|
|
91
93
|
- `INV-COW-010` Grid-layer spread-correction cycle is atomic and bounded
|
|
@@ -101,6 +103,8 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
101
103
|
- A fill record carries the chain execution price (`sync_engine._computeFillTransitionResult` / pass-1 full-fill / pass-2 adoption), computed from paid/received amounts when only the history is available; the slot's generation price is a fallback, never the first choice. The queued-fill pivot refresh (`refreshLastFillPivotFromQueue`) likewise prefers fill economics over the slot price. The slot's identity price is NOT rewritten (`GRID-PRICE-INVARIANT`); only the transient fill/pivot fact uses the chain price.
|
|
102
104
|
- Gap-evacuation geometry maps an on-chain order's LIVE price onto the genesis ladder (`resolveLiveOrderLadderPosition`) before falling back to `parseSlotIndex(slot.id)`. Detection (`detectGapEvacuationCandidates`), the drain-time validation (`_validatePriceCorrectionEntry`), and the teeth (`_processGapEvacuationTeeth`) all use it, so a refused-commit adoption that leaves an order keyed at an in-rail slot cannot hide an in-band order.
|
|
103
105
|
- A price mismatch whose live chain price maps inside the gap band queues a cancel-only `gapEvacuation` correction and drops any opposite-kind (plain re-price) entry for the same chain order. Evacuation takes precedence over correcting the order back to a stale slot price.
|
|
106
|
+
- **Stale-stamp refusal vs. forced retreat.** A gap-evacuation rotation is B-stamped with its plan-build boundary/gap and re-proven against the LIVE geometry at execution (`isEvacuationStampStillValid`): a boundary commit landing mid-batch must not grant a bypass for a move that no longer makes grid sense (the 2026-10-30 bid-edge incident). That refusal assumes the live boundary survives the commit. When the live rails ALREADY conflict — `maxBuy > minSell − gap − 1`, i.e. no boundary can host both rails with the band (`liveRailsConflictWithGap`, reached through the single decision point `forcedRetreatJustifiesEvacuation`, which also refuses to override a stamp whose source/dest ids are unparseable — an unprovable stamp is never overridden) — the batch's retreat is forced, the plan's boundary is the only consistent geometry, and refusing the evacuation is what permanently strands the order inside the band (spread collapse: 4-slot band, real spread 1.21% vs 2.0% target). Both the build loop and the final pivot gate therefore keep the evacuation in that case.
|
|
107
|
+
- **Committed-geometry sweep.** Because the in-batch refusal is fail-closed by design, the writer of the stranding must repair it: after a commit, and on every maintenance tick, `scheduleGapEvacRepair` re-derives the in-band set against the COMMITTED geometry (pruning resolved ledger entries WITHOUT counting a new observation — `reconcileGrid` is the cycle's single tick, and counting twice armed the cancel-only teeth in the very cycle whose rotation evacuated the order), queues the cancel-only teeth past the threshold, and schedules a fill-less COW re-plan (rate-limited 60s per bot). A queued cancel-only evacuation whose chain order the SAME batch rotates is held out of the pre-batch correction drain (`splitEvacuationCancelsSupersededByRotations`): the rotation IS the evacuation, and cancelling first destroyed the batch's own source (live: `2 target(s) missing on chain` → structural resync + full grid reload instead of two in-place price updates). Plans are fill-driven, so without this a quiet market never heals a collapsed spread.
|
|
104
108
|
- The crossing guard's candidate set (`buildCrossingCheckCandidates`) uses the live chain price for master and pending orders, so an emission cannot print onto a live opposite-side order that was adopted at a stale slot price.
|
|
105
109
|
|
|
106
110
|
- `INV-PROJ-001` New projected orders remain virtual
|
|
@@ -136,6 +140,46 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
136
140
|
|
|
137
141
|
---
|
|
138
142
|
|
|
143
|
+
## Order Lifecycle ([ORDER_LIFECYCLE.md](ORDER_LIFECYCLE.md))
|
|
144
|
+
|
|
145
|
+
Chain existence is answered by ONE record per slot (`modules/order/lifecycle.ts`),
|
|
146
|
+
not by the seven per-site structures that used to disagree
|
|
147
|
+
(`_pendingBroadcasts`, `createUncertain`, `_committedOrderIds`,
|
|
148
|
+
`_orderIdAssignedAt`, `_placedAt`, `_chainSizeAppliedFills`,
|
|
149
|
+
`_lastUnmatchedChainOrders`). Those now exist only as live views onto the record.
|
|
150
|
+
|
|
151
|
+
- `INV-LIFE-001` Unknown is never gone
|
|
152
|
+
- `phase === 'unknown'` (lost receipt, ambiguous read, unparseable chain entry) implies `mayExistOnChain === true` and `isConfirmedGone === false`.
|
|
153
|
+
- Only a chain-observed absence sets `phase === 'gone'`. A consumer must never coerce `unknown` into "absent" via a default.
|
|
154
|
+
|
|
155
|
+
- `INV-LIFE-002` One record per slot
|
|
156
|
+
- A slot is the identity across its order incarnations. A chain-id change (rotation, re-place, adoption) updates the existing record; it must not start a parallel one.
|
|
157
|
+
- Writers funnel through `_applyOrderUpdate` → `_syncLifecycleFromMutation`; no call site maintains its own chain-existence bookkeeping (that includes the stamps: `assignedAt`/`placedAt` are written by the funnel only).
|
|
158
|
+
- The one mutation that does not call `_applyOrderUpdate` is the COW whole-map commit (`_commitWorkingGrid`): it projects every slot of the committed map through the same funnel before swapping, so the primary placement path stamps `placedAt`/`assignedAt` too. Without it the surplus-cancel grace is inert for every COW-created order.
|
|
159
|
+
|
|
160
|
+
- `INV-LIFE-003` One definition per guard
|
|
161
|
+
- Absence decisions ("is this order's absence trustworthy yet?") use `isInFlight()`, read through the slot (`readSlotLifecycle`) so a persisted `createUncertain` marker participates.
|
|
162
|
+
- The committed-but-missing guard uses `isRecentlyCommitted()`.
|
|
163
|
+
- The fresh-id guard uses `isRecentlyAssigned()`.
|
|
164
|
+
- A read site must not re-derive a window from a raw map, and must not consult a subset of `isInFlight()`: that is how four sites ended up with four answers.
|
|
165
|
+
|
|
166
|
+
- `INV-LIFE-004` A LIVE unresolved record is never evicted or downgraded
|
|
167
|
+
- Registry pruning skips `unknown` and `broadcasting` while their uncertainty is inside the bound; past it no decision consults them, so they become evictable (the registry keeps a ceiling).
|
|
168
|
+
- `markIntentIfResolved` likewise refuses to downgrade a live uncertainty, and resolves one whose bound has expired instead of leaving an un-honoured suppression behind.
|
|
169
|
+
- A pending CREATE has two distinct exits: *resolved* (`deletePending` → `intent`) and *abandoned* (`_expirePendingBroadcast` → `unknown`). Collapsing them turns a lost receipt into a clean hole that gets re-CREATEd.
|
|
170
|
+
|
|
171
|
+
- `INV-LIFE-005` Unparseable chain entries make absence unknown, not absent
|
|
172
|
+
- A chain order that was returned but could not be decoded puts its slot's absence in the UNKNOWN bucket (`sync_engine` pass-1 guard 0, mirroring the accounting recovery sweep's `malformedOrderIds`); the slot is not virtualized.
|
|
173
|
+
|
|
174
|
+
- `INV-LIFE-006` Evidence is monotonic; clocks cannot be refreshed for free
|
|
175
|
+
- Within a phase the evidence rank may only rise (`local` < `broadcast-receipt` < `adoption` < `chain-snapshot` < `fill-history`), so a local write cannot overwrite what the chain said.
|
|
176
|
+
- `observedAt` advances only for chain-backed evidence.
|
|
177
|
+
- `recordedAt` (the anchor of the unresolved bound) advances only on a phase change or chain-backed evidence: re-asserting `unknown` is not new information.
|
|
178
|
+
|
|
179
|
+
- `INV-LIFE-007` The unresolved bound is measured from when the uncertainty was RECORDED
|
|
180
|
+
- `isInFlight()` is bounded (2× `PENDING_BROADCAST_TTL_MS`) from `recordedAt`, so a lagging node cannot park the grid and a bot that re-asserts `unknown` every cycle cannot extend its own suppression.
|
|
181
|
+
- An explicit decision (`markIntent`) or chain evidence resolves the phase sooner; age alone never extends it.
|
|
182
|
+
|
|
139
183
|
## Sync Engine
|
|
140
184
|
|
|
141
185
|
- `INV-SYNC-001` Drift-refetch for partial fill correctness
|
|
@@ -153,8 +197,9 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
153
197
|
- The sync engine must treat this as a chain-confirmed empty, not a connection failure.
|
|
154
198
|
|
|
155
199
|
- `INV-SYNC-004` Orphan adoption rejects duplicate price levels
|
|
156
|
-
- Pass-2
|
|
157
|
-
-
|
|
200
|
+
- Pass-2 adoption must check whether an active grid order of the same type already maps to the same ladder level (`chainPriceOnSameLevel` → `pricesOnSameLevel`: the chain price and the grid level resolve to one slot, so equality is by level, not by a tuned band).
|
|
201
|
+
- `calculatePriceTolerance` is NOT the duplicate rule; it survives only inside helpers that need a representable-price tolerance, and the old `findMatchingGridOrderByOpenOrder` matcher is gone (see [GRID_PRICE_INVARIANT.md](GRID_PRICE_INVARIANT.md#the-tolerance-matcher-is-gone)).
|
|
202
|
+
- If a duplicate exists, the orphan is NOT adopted; it is pushed to `unmatchedChainOrders` as `duplicate-price-level` for chain cancel.
|
|
158
203
|
- Size is irrelevant — any duplicate violates the one-order-per-price-level invariant.
|
|
159
204
|
|
|
160
205
|
- `INV-SYNC-005` Fill queue back-pressure
|
|
@@ -183,7 +228,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
183
228
|
|
|
184
229
|
- `INV-MAINT-001` Pipeline in-flight defers maintenance
|
|
185
230
|
- When `isPipelineEmpty` returns `isEmpty=false` (batch in-flight, recovery-in-flight, or broadcasting active), `checkSpreadCondition` and divergence corrections must be skipped.
|
|
186
|
-
- Dust cancels
|
|
231
|
+
- Dust cancels mutate the chain, so they take the **exclusive placement region** (`withPlacement('dust-cancel')`) and defer to the next cycle when it is busy or the bot is shutting down — they are NOT "safe regardless of pipeline state". The rotation batch that follows the cancels runs after the region is released and acquires its own lease.
|
|
187
232
|
- Pipeline signals (`batchInFlight`, `recoveryInFlight`, `broadcasting`) must be passed to `isPipelineEmpty`.
|
|
188
233
|
|
|
189
234
|
- `INV-MAINT-002` Illegal state abort
|
|
@@ -262,7 +307,7 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
262
307
|
- SELL placements must use nearest available free slots first (ascending price).
|
|
263
308
|
|
|
264
309
|
- `INV-RECON-006` Fresh-placement cancels are deferred for a grace window
|
|
265
|
-
- A CANCEL action targeting an `orderId` placed within `TIMING.SURPLUS_CANCEL_GRACE_MS` (15 min) is deferred. The timestamp is written by `recordOrderPlacement` into `manager._placedAt
|
|
310
|
+
- A CANCEL action targeting an `orderId` placed within `TIMING.SURPLUS_CANCEL_GRACE_MS` (15 min) is deferred. The timestamp is written by the lifecycle write funnel (and by `recordOrderPlacement` for manager-shaped callers) into `manager._placedAt`, and tested by `isFreshlyPlacedOrder`.
|
|
266
311
|
- `COWRebalanceEngine.execute` drops the whole slot's action group for a deferred cancel (dropping only the CANCEL would let a paired CREATE fold back into an UPDATE and re-materialize the placement), then re-syncs the deferred slots from master via `workingGrid.syncFromMaster` so the commit does not virtualize a still-live order.
|
|
267
312
|
- Divergence correction (`applyGridDivergenceCorrections`) applies the same guard after dropping any queued committed-size UPDATE for the not-desired/size-0 slot.
|
|
268
313
|
- `grid-load`/`grid-init` load contexts are excluded so restarts do not suppress the sweep.
|
|
@@ -363,6 +408,20 @@ This document defines the non-negotiable behavioral invariants for the DEXBot2 s
|
|
|
363
408
|
- A `get_full_accounts` window capped at the order limit omits the freshest orders (by_account index = seller,id ascending — fresh creates sort last), so absence on a truncated/empty read can never free slots/capital, confirm a cancel, discard a batch, or clear pending-broadcast protection.
|
|
364
409
|
- All absence decisions use `readOpenOrdersWithMeta`; truncated/empty snapshots defer (keep protection + request structural resync) instead of acting.
|
|
365
410
|
|
|
411
|
+
- `INV-BROADCAST-005` One exclusive placement region; one lease-liveness predicate
|
|
412
|
+
- Every chain-mutating path (COW broadcast, reconcile Phase 1+2, correction drain, dust/residual cancel, pool swap) runs inside the exclusive placement region (`modules/order/placement_lock.ts`). The broadcast refcount only defers fills; it is not mutual exclusion.
|
|
413
|
+
- The region is NOT reentrant. Nested work receives the holder's lease and verifies it with `manager.isPlacementLeaseCurrent(lease)` — `isPlacementActive()` is true for ANY holder and must never stand in for ownership.
|
|
414
|
+
- Non-token holders must use `manager.withPlacement()` (single acquire/release contract, bounded wait, shutdown probe). `correctAllPriceMismatches` is the only nested consumer and takes the inherited lease explicitly.
|
|
415
|
+
- Pending-broadcast liveness has ONE policy (`getLivePendingBroadcasts`): TTL, or TTL + a sync completed after the record, or 2× TTL hard cap. Every reader (COW guard, reconcile CREATE, reconcile surplus sweep, crossing candidates, maintenance lightweight-sync gate) uses it.
|
|
416
|
+
- A PLACEMENT-BACKED hung region is torn down by ONE authority: `_clearStalePlacementRegion()` after `PLACEMENT_STALE_CLEAR_MS`, and it also clears that region's stale broadcast refcount atomically. The teardown's refcount clear is sound only because a refcount raised by a placement holder belongs to that same region; a refcount raised OUTSIDE any placement region (the spread-correction pre-broadcast hold in `order/grid.ts`) is not touched by it and is still cleared by its own watchdog. The broadcast watchdog refuses to act while a placement lease is held (so it only clears a refcount leaked without a lease), and the placement watchdog refuses a teardown while a bot-level pipeline counter younger than the bound proves progress.
|
|
417
|
+
- The owner of a region must re-verify its own lease (`isPlacementLeaseCurrent`) before it BROADCASTS a CREATE/UPDATE batch: a watchdog teardown turns the lease into a stale token, and a resumed holder must abort instead of broadcasting unprotected (COW body: `PLACEMENT_LEASE_LOST`; reconcile Phase 1+2: the plan is discarded and the cycle defers). Pure cancel paths are exempt — a cancel carries its own order id and cannot duplicate a CREATE.
|
|
418
|
+
|
|
419
|
+
|
|
420
|
+
- `INV-TIME-001` Every timing window is classified, and self-healing windows are removal targets
|
|
421
|
+
- `modules/timing_policy.ts` classifies every `TIMING` window as `external`, `bound`, `policy` or `self-healing`, each with a rationale; a `self-healing` window must also name the evidence that replaces it.
|
|
422
|
+
- `tests/test_timing_policy.ts` fails on an unclassified window, so the inventory can only shrink by decision.
|
|
423
|
+
- A wait that compensates for our own lost state is a bug with a timer attached; the fix is a lifecycle phase, not patience. See [TIMING_INVENTORY.md](TIMING_INVENTORY.md).
|
|
424
|
+
|
|
366
425
|
## Change Policy
|
|
367
426
|
|
|
368
427
|
- Any intentional invariant change must:
|