@dusk-network/app-core 0.3.1 → 0.3.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- package/CHANGELOG.md +7 -0
- package/README.md +1 -1
- package/package.json +1 -1
- package/src/bridge.d.ts +12 -3
- package/src/bridge.js +302 -44
package/CHANGELOG.md
CHANGED
|
@@ -2,6 +2,13 @@
|
|
|
2
2
|
|
|
3
3
|
All notable changes to this package are listed here. It follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic Versioning](https://semver.org/).
|
|
4
4
|
|
|
5
|
+
## [0.3.2]
|
|
6
|
+
|
|
7
|
+
### Fixed
|
|
8
|
+
|
|
9
|
+
- `/bridge`: an RPC search reads each block more than once while its transfers wait, from the first transfer's block on, rather than once
|
|
10
|
+
- `/bridge`: on an RPC, an earlier one of equal transfers takes a credit a later one could have had once the blocks before it are read again, rather than at once
|
|
11
|
+
|
|
5
12
|
## [0.3.1]
|
|
6
13
|
|
|
7
14
|
### Added
|
package/README.md
CHANGED
|
@@ -206,7 +206,7 @@ arrivalsOf(arrivals); // Map { hash => { hash: "0x029e…", at: 1793080333000 }
|
|
|
206
206
|
- A transfer out is read from the chain: what the route's account received, the address its memo gives as the route's bridge reads it (0x and 40 hex digits, any case, nothing around them), and its block. It arrives with the DUSK token's `Transfer` from its route's payout address to that address, of what the account received less the fee. The account's history lists every transfer that named the same address, in any case: equal ones in the hours before (a given one's by its own fee) are paid first, and one that can't be read leaves the order unclear. One that pays nothing out is `no-payout`, with its `problem`. `serviceTransferOf` gives what left the account (`sent`), and what the account received (`amount`) only where no gas was left to refund: an entry's value holds any refund paid to another account.
|
|
207
207
|
- With the archive, each transfer reads only its own hours of its account's history (its sender's for a deposit, the route's account's for a transfer out), further back while equal ones before it run on, and only once the archive lists the transfer there. A call reads 4 pages of it (`HISTORY_PAGES`) for each: a transfer whose history is longer waits, and the next call reads on from its `scan`.
|
|
208
208
|
- A route without its payout account or address in the network's file pays out nothing that can be told apart: `no-route`.
|
|
209
|
-
- An RPC search starts at a block from before the transfer, estimated from the chain's pace and checked by its time, and reads 8 windows of 5,000 blocks at most, narrower where the RPC asks for it. A result that isn't arrived says where the next search starts, with what it searched for and the credits found so far (`scan`): the app keeps it with the arrivals it was made with, and passes both back. A search for another sender or amount starts afresh.
|
|
209
|
+
- An RPC search starts at a block from before the transfer, estimated from the chain's pace and checked by its time, and reads 8 windows of 5,000 blocks at most, narrower where the RPC asks for it. It reads each block more than once: its window at the head reaches back 1,000 blocks, so the newest blocks are read again on each call, and once it has reached the head with a transfer not arrived, it reads its blocks again from the first, a pass at a time with the windows a call has left (one at least while a pass is under way, so 9 where the search ahead takes all 8), in windows no wider than the RPC took: the first a minute later, each one after as long again as it has been since it reached the head. Where a later equal transfer, still without its credit, could have had the credit an earlier one takes, the earlier one waits until the blocks are read again 8 times since the credit was found, one pass after the other. A result that isn't arrived says where the next search starts, with what it searched for and the credits found so far (`scan`): the app keeps it with the arrivals it was made with, and passes both back. A search for another sender or amount starts afresh.
|
|
210
210
|
|
|
211
211
|
A result is facts: `arrived`, with the other side's transaction, its block's time, the lux it paid and its block; `waiting`; or `unknown`, with why: `unsure` (a credit could be another's), `late` (none came in its window), `beyond` (the explorer's pages don't reach it), `searching` (carry on from `scan`), `expired`, `no-route`, `no-payout` or `error`. The words are the app's.
|
|
212
212
|
|
package/package.json
CHANGED
package/src/bridge.d.ts
CHANGED
|
@@ -76,7 +76,7 @@ export declare const EXPLORER_PAGES: 5;
|
|
|
76
76
|
/** Blocks one log search asks for at once; narrower where the RPC asks for it. */
|
|
77
77
|
export declare const LOG_WINDOW: 5000;
|
|
78
78
|
|
|
79
|
-
/** Log searches one call makes for a group of equal transfers, by default. */
|
|
79
|
+
/** Log searches one call makes for a group of equal transfers, by default: one more where the search ahead takes them all while its blocks are read again. */
|
|
80
80
|
export declare const LOG_WINDOWS: 8;
|
|
81
81
|
|
|
82
82
|
/** Equal deposits sent within this of each other are paid in their order: 2 hours, in milliseconds. */
|
|
@@ -113,6 +113,12 @@ export type HistoryScan = {
|
|
|
113
113
|
* transfer it began from, the logs it searched for (`search`), and the
|
|
114
114
|
* credits it found before (`seen`), which the next one counts again. A
|
|
115
115
|
* search for other logs, as for another sender or amount, starts afresh.
|
|
116
|
+
* The block it began at (`start`), and its passes over the blocks it read
|
|
117
|
+
* before (`recheck`): where the one under way is (`from`, null between
|
|
118
|
+
* two), when the next one begins (`at`), how many are done, as each credit
|
|
119
|
+
* seen has the passes done before it was found (`pass`), when the search
|
|
120
|
+
* first reached the head (`since`), in milliseconds, and the widest window
|
|
121
|
+
* the blocks are read again in, where the RPC refused a wider one (`size`).
|
|
116
122
|
* With the archive, how far the transfer's own history is read
|
|
117
123
|
* (`history`): a history longer than one call reads is read on from there,
|
|
118
124
|
* before any search. Kept by the app with the credits it was made with,
|
|
@@ -122,7 +128,9 @@ export type Scan = {
|
|
|
122
128
|
from?: string;
|
|
123
129
|
basis?: number;
|
|
124
130
|
search?: string;
|
|
125
|
-
seen?: ReadonlyArray<{ hash: string; block: string; at: number; sender: string | null }>;
|
|
131
|
+
seen?: ReadonlyArray<{ hash: string; block: string; at: number; sender: string | null; pass?: number }>;
|
|
132
|
+
start?: string;
|
|
133
|
+
recheck?: { from: string | null; at: number; passes: number; since: number; size?: number };
|
|
126
134
|
history?: HistoryScan;
|
|
127
135
|
};
|
|
128
136
|
|
|
@@ -250,7 +258,8 @@ export type DepositArrivalOptions = {
|
|
|
250
258
|
* deposits, newest first, back past the oldest one waited for, at most
|
|
251
259
|
* `maxPages` pages; none is asked while every deposit's window is still
|
|
252
260
|
* open. On the RPC, a group of equal deposits from one sender is searched
|
|
253
|
-
* from about when the first was sent, `maxWindows` windows at most
|
|
261
|
+
* from about when the first was sent, `maxWindows` windows at most (one more
|
|
262
|
+
* where the search ahead takes them all while its blocks are read again).
|
|
254
263
|
* A network without DuskEVM asks nothing. Throws a TypeError for a deposit
|
|
255
264
|
* that isn't one, and for a network there's no file for.
|
|
256
265
|
*/
|
package/src/bridge.js
CHANGED
|
@@ -47,9 +47,10 @@ import { DETAILS_BATCH, TRANSFER_CONTRACT, fetchTxDetails, historyRange } from "
|
|
|
47
47
|
/**
|
|
48
48
|
* A credit on the other chain: its transaction, its block and time, its
|
|
49
49
|
* amount in wei, whether it went through, and the address it came from as
|
|
50
|
-
* its log names it (lower case; null where it isn't named).
|
|
50
|
+
* its log names it (lower case; null where it isn't named). Found by an RPC
|
|
51
|
+
* search, the passes over its blocks again done before it was (`pass`).
|
|
51
52
|
*
|
|
52
|
-
* @typedef {{ hash: string, block: bigint | null, at: number, wei: bigint, success: boolean, from: string | null }} Credit
|
|
53
|
+
* @typedef {{ hash: string, block: bigint | null, at: number, wei: bigint, success: boolean, from: string | null, pass?: number }} Credit
|
|
53
54
|
*/
|
|
54
55
|
|
|
55
56
|
/**
|
|
@@ -83,6 +84,17 @@ const PACE_MARGIN = 1.2;
|
|
|
83
84
|
const START_MARGIN = 50n;
|
|
84
85
|
const START_CHECKS = 4;
|
|
85
86
|
const MIN_WINDOW = 50n;
|
|
87
|
+
// How far back from the head a search's last window reaches: those blocks
|
|
88
|
+
// are read again on each call.
|
|
89
|
+
const OVERLAP = 1_000n;
|
|
90
|
+
// How long after a search first reaches the head it reads its blocks again
|
|
91
|
+
// from its first one; each pass after waits as long again as it has been
|
|
92
|
+
// since then, so the waits about double each time.
|
|
93
|
+
const RECHECK_MS = 60_000;
|
|
94
|
+
// How many passes over the blocks before a credit, since it was found, a
|
|
95
|
+
// transfer waits for before it takes the credit where a later equal one,
|
|
96
|
+
// still without its own, could have had it instead.
|
|
97
|
+
const CONFIRM_PASSES = 8;
|
|
86
98
|
// What an RPC says when it won't search that many blocks at once.
|
|
87
99
|
const RANGE_REFUSED = /limit|range|too many|exceed/i;
|
|
88
100
|
const DUSK_HASH = /^[0-9a-f]{64}$/;
|
|
@@ -253,11 +265,19 @@ function scanOf(value) {
|
|
|
253
265
|
const block = quantityOf(credit?.block);
|
|
254
266
|
const at = msOf(credit?.at);
|
|
255
267
|
const sender = credit?.sender === null ? null : addressOf(credit?.sender);
|
|
268
|
+
// The passes done before it was found: none where it isn't said.
|
|
269
|
+
const pass = Number.isSafeInteger(credit?.pass) && credit.pass >= 0 ? credit.pass : 0;
|
|
256
270
|
|
|
257
|
-
return hash && block !== null && at !== null && sender !== "" ? [{ at, block: block.toString(), hash, sender }] : [];
|
|
271
|
+
return hash && block !== null && at !== null && sender !== "" ? [{ at, block: block.toString(), hash, pass, sender }] : [];
|
|
258
272
|
});
|
|
273
|
+
// Where it began, and its passes over the blocks again: without them,
|
|
274
|
+
// as a search kept by an earlier version.
|
|
275
|
+
const start = quantityOf(scan.start);
|
|
276
|
+
const recheck = start === null ? null : recheckOf(scan.recheck);
|
|
259
277
|
|
|
260
278
|
Object.assign(out, { basis: scan.basis, from: from.toString(), search: scan.search, seen });
|
|
279
|
+
if (start !== null) out.start = start.toString();
|
|
280
|
+
if (recheck) out.recheck = recheckIn(recheck);
|
|
261
281
|
} else if (!history) {
|
|
262
282
|
return null;
|
|
263
283
|
}
|
|
@@ -450,6 +470,36 @@ function clustersOf(queue, windowMs) {
|
|
|
450
470
|
*/
|
|
451
471
|
const allOpen = (queue, windowMs, now) => clustersOf(queue, windowMs).every((cluster) => now < cluster[cluster.length - 1].at + windowMs);
|
|
452
472
|
|
|
473
|
+
/**
|
|
474
|
+
* How a group's credits are matched: see `settle`.
|
|
475
|
+
*
|
|
476
|
+
* @typedef {{ guarded: boolean, now: number, slackMs: number, used: ReadonlySet<string>, windowMs: number, rivals?: ReadonlySet<string> }} Rules
|
|
477
|
+
*/
|
|
478
|
+
|
|
479
|
+
/**
|
|
480
|
+
* Whether a credit could pay a transfer, whether or not another one took it:
|
|
481
|
+
* it went through, isn't one the app gave another of its transfers when
|
|
482
|
+
* this one is the app's, comes from none of `rivals`, and is in the
|
|
483
|
+
* transfer's time and at or after the block it can't be before.
|
|
484
|
+
*
|
|
485
|
+
* @param {Queued} item
|
|
486
|
+
* @param {Credit} credit
|
|
487
|
+
* @param {Rules} rules
|
|
488
|
+
*/
|
|
489
|
+
function pays(item, credit, { guarded, rivals = new Set(), slackMs, used, windowMs }) {
|
|
490
|
+
const lo = item.at - (guarded ? windowMs : slackMs);
|
|
491
|
+
const hi = guarded ? item.at + windowMs : Infinity;
|
|
492
|
+
|
|
493
|
+
return (
|
|
494
|
+
credit.success &&
|
|
495
|
+
(!item.given || !used.has(credit.hash)) &&
|
|
496
|
+
(credit.from === null || !rivals.has(credit.from)) &&
|
|
497
|
+
credit.at >= lo &&
|
|
498
|
+
credit.at <= hi &&
|
|
499
|
+
(item.notBefore === null || credit.block === null || credit.block >= item.notBefore)
|
|
500
|
+
);
|
|
501
|
+
}
|
|
502
|
+
|
|
453
503
|
/**
|
|
454
504
|
* Each transfer of a queue of equal ones, oldest first, takes the first
|
|
455
505
|
* credit that went through after it that no earlier one took: from
|
|
@@ -470,10 +520,11 @@ const allOpen = (queue, windowMs, now) => clustersOf(queue, windowMs).every((clu
|
|
|
470
520
|
*
|
|
471
521
|
* @param {ReadonlyArray<Queued>} queue oldest first
|
|
472
522
|
* @param {ReadonlyArray<Credit>} credits
|
|
473
|
-
* @param {
|
|
523
|
+
* @param {Rules} rules
|
|
474
524
|
* @returns {{ assigned: Map<string, Credit>, verdicts: Map<string, Verdict> }}
|
|
475
525
|
*/
|
|
476
|
-
function settle(queue, credits,
|
|
526
|
+
function settle(queue, credits, rules) {
|
|
527
|
+
const { guarded, now, used, windowMs } = rules;
|
|
477
528
|
const sorted = [...credits].sort(byTime);
|
|
478
529
|
/** @type {Map<string, Credit>} */
|
|
479
530
|
const assigned = new Map();
|
|
@@ -482,18 +533,7 @@ function settle(queue, credits, { guarded, now, rivals = new Set(), slackMs, use
|
|
|
482
533
|
const verdicts = new Map();
|
|
483
534
|
|
|
484
535
|
for (const item of queue) {
|
|
485
|
-
const
|
|
486
|
-
const hi = guarded ? item.at + windowMs : Infinity;
|
|
487
|
-
const credit = sorted.find(
|
|
488
|
-
(c) =>
|
|
489
|
-
c.success &&
|
|
490
|
-
!taken.has(c.hash) &&
|
|
491
|
-
(!item.given || !used.has(c.hash)) &&
|
|
492
|
-
(c.from === null || !rivals.has(c.from)) &&
|
|
493
|
-
c.at >= lo &&
|
|
494
|
-
c.at <= hi &&
|
|
495
|
-
(item.notBefore === null || c.block === null || c.block >= item.notBefore)
|
|
496
|
-
);
|
|
536
|
+
const credit = sorted.find((c) => !taken.has(c.hash) && pays(item, c, rules));
|
|
497
537
|
|
|
498
538
|
if (credit) {
|
|
499
539
|
assigned.set(item.hash, credit);
|
|
@@ -734,32 +774,48 @@ async function startBefore(read, url, atMs, cache) {
|
|
|
734
774
|
|
|
735
775
|
/**
|
|
736
776
|
* The credits asked for from block `start`, a window of blocks at a time, at
|
|
737
|
-
* most `maxWindows`, until
|
|
777
|
+
* most `maxWindows`, until `head`, `enough`, or a window ending after
|
|
738
778
|
* `until` (`covered`): each checked against what was asked, with its block's
|
|
739
779
|
* time. A window the RPC won't search at once is asked again a quarter as
|
|
740
|
-
* wide
|
|
780
|
+
* wide (`size`, the width it ends with; `refused`, the narrowest it
|
|
781
|
+
* refused, and `took`, the widest it answered). The window ending at `head`
|
|
782
|
+
* reaches back up to `overlap` blocks from it, no wider than `size`, though
|
|
783
|
+
* not under `floor`: those blocks are read again (from `low`). `asked`: the
|
|
784
|
+
* windows asked.
|
|
741
785
|
*
|
|
742
786
|
* @param {Reader} read
|
|
743
|
-
* @param {{ url: string, address: string, topics: ReadonlyArray<string | null>, wei: bigint, amountOf: (data: string) => bigint | null, start: bigint, head: bigint, maxWindows: number, blockTimes: BlockTimes | undefined, enough: (credits: Credit[]) => boolean, until: number | null, known: Credit[] }} search
|
|
744
|
-
* @returns {Promise<{ complete: boolean, covered: boolean, credits: Credit[], next: bigint }>}
|
|
787
|
+
* @param {{ url: string, address: string, topics: ReadonlyArray<string | null>, wei: bigint, amountOf: (data: string) => bigint | null, start: bigint, floor: bigint, head: bigint, size?: bigint, overlap?: bigint, maxWindows: number, blockTimes: BlockTimes | undefined, enough: (credits: Credit[]) => boolean, until: number | null, known: Credit[] }} search
|
|
788
|
+
* @returns {Promise<{ asked: number, complete: boolean, covered: boolean, credits: Credit[], low: bigint | null, next: bigint, refused: bigint | null, size: bigint, took: bigint | null }>}
|
|
745
789
|
*/
|
|
746
|
-
async function searchLogs(read, { address, amountOf, blockTimes, enough, head, known, maxWindows, start, topics, until, url, wei }) {
|
|
790
|
+
async function searchLogs(read, { address, amountOf, blockTimes, enough, floor, head, known, maxWindows, overlap = OVERLAP, size: width = BigInt(LOG_WINDOW), start, topics, until, url, wei }) {
|
|
747
791
|
// What an earlier search found before `start`.
|
|
748
792
|
/** @type {Credit[]} */
|
|
749
793
|
const credits = [...known];
|
|
750
794
|
let from = start;
|
|
751
|
-
let size =
|
|
795
|
+
let size = width;
|
|
752
796
|
let covered = false;
|
|
753
|
-
|
|
754
|
-
|
|
797
|
+
let asked = 0;
|
|
798
|
+
/** @type {bigint | null} */
|
|
799
|
+
let low = null;
|
|
800
|
+
/** @type {bigint | null} */
|
|
801
|
+
let refused = null;
|
|
802
|
+
/** @type {bigint | null} */
|
|
803
|
+
let took = null;
|
|
804
|
+
|
|
805
|
+
while (asked < maxWindows && from <= head) {
|
|
755
806
|
const to = from + size - 1n < head ? from + size - 1n : head;
|
|
807
|
+
const back = head + 1n - (size < overlap ? size : overlap);
|
|
808
|
+
const lo = to === head && back < from ? (back > floor ? back : floor) : from;
|
|
756
809
|
/** @type {any} */
|
|
757
810
|
let logs;
|
|
758
811
|
|
|
812
|
+
asked += 1;
|
|
813
|
+
|
|
759
814
|
try {
|
|
760
|
-
logs = await read.rpc(url, "eth_getLogs", [{ address, fromBlock: hex(
|
|
815
|
+
logs = await read.rpc(url, "eth_getLogs", [{ address, fromBlock: hex(lo), toBlock: hex(to), topics }]);
|
|
761
816
|
} catch (error) {
|
|
762
817
|
if (size > MIN_WINDOW && RANGE_REFUSED.test(messageOf(error))) {
|
|
818
|
+
if (refused === null || to - lo + 1n < refused) refused = to - lo + 1n;
|
|
763
819
|
size = size / 4n > MIN_WINDOW ? size / 4n : MIN_WINDOW;
|
|
764
820
|
continue;
|
|
765
821
|
}
|
|
@@ -767,8 +823,11 @@ async function searchLogs(read, { address, amountOf, blockTimes, enough, head, k
|
|
|
767
823
|
throw error;
|
|
768
824
|
}
|
|
769
825
|
|
|
826
|
+
if (to === head) low = lo;
|
|
827
|
+
if (took === null || to - lo + 1n > took) took = to - lo + 1n;
|
|
828
|
+
|
|
770
829
|
for (const log of Array.isArray(logs) ? logs : []) {
|
|
771
|
-
const found = creditLog(log, { address, amountOf, from, to, topics, wei });
|
|
830
|
+
const found = creditLog(log, { address, amountOf, from: lo, to, topics, wei });
|
|
772
831
|
|
|
773
832
|
if (!found || credits.some((credit) => credit.hash === found.hash)) continue;
|
|
774
833
|
if (found.at !== null) blockTimes?.set(Number(found.block), found.at);
|
|
@@ -785,7 +844,39 @@ async function searchLogs(read, { address, amountOf, blockTimes, enough, head, k
|
|
|
785
844
|
}
|
|
786
845
|
}
|
|
787
846
|
|
|
788
|
-
return { complete: from > head, covered, credits, next: from };
|
|
847
|
+
return { asked, complete: from > head, covered, credits, low, next: from, refused, size, took };
|
|
848
|
+
}
|
|
849
|
+
|
|
850
|
+
/**
|
|
851
|
+
* A search's passes over its blocks again, as a result gave them: where the
|
|
852
|
+
* one under way is (null between two), when the next begins, how many are
|
|
853
|
+
* done, when the search first reached the head, and the widest window it
|
|
854
|
+
* reads them in where the RPC refused a wider one; null for anything else.
|
|
855
|
+
*
|
|
856
|
+
* @param {unknown} value
|
|
857
|
+
* @returns {{ at: number, from: bigint | null, passes: number, since: number, size: number | null } | null}
|
|
858
|
+
*/
|
|
859
|
+
function recheckOf(value) {
|
|
860
|
+
const kept = /** @type {any} */ (value);
|
|
861
|
+
const from = kept?.from === null ? null : quantityOf(kept?.from);
|
|
862
|
+
/** @param {unknown} ms */
|
|
863
|
+
const time = (ms) => typeof ms === "number" && Number.isFinite(ms);
|
|
864
|
+
const size = Number.isSafeInteger(kept?.size) && kept.size >= Number(MIN_WINDOW) && kept.size <= LOG_WINDOW ? kept.size : null;
|
|
865
|
+
|
|
866
|
+
if (!kept || !time(kept.at) || !time(kept.since) || !Number.isSafeInteger(kept.passes) || kept.passes < 0) return null;
|
|
867
|
+
if (kept.from !== null && from === null) return null;
|
|
868
|
+
|
|
869
|
+
return { at: kept.at, from, passes: kept.passes, since: kept.since, size };
|
|
870
|
+
}
|
|
871
|
+
|
|
872
|
+
/**
|
|
873
|
+
* A search's passes over its blocks again, as its scan keeps them.
|
|
874
|
+
*
|
|
875
|
+
* @param {NonNullable<ReturnType<typeof recheckOf>>} recheck
|
|
876
|
+
* @returns {NonNullable<Scan["recheck"]>}
|
|
877
|
+
*/
|
|
878
|
+
function recheckIn({ at, from, passes, since, size }) {
|
|
879
|
+
return { at, from: from === null ? null : from.toString(), passes, since, ...(size === null ? {} : { size }) };
|
|
789
880
|
}
|
|
790
881
|
|
|
791
882
|
/**
|
|
@@ -795,6 +886,16 @@ async function searchLogs(read, { address, amountOf, blockTimes, enough, head, k
|
|
|
795
886
|
* sent at (as the app saw the chain's head then), else from about when it
|
|
796
887
|
* was sent.
|
|
797
888
|
*
|
|
889
|
+
* Each block is read more than once: on each call, those up to `OVERLAP`
|
|
890
|
+
* under the head, and once the search has reached the head with a transfer
|
|
891
|
+
* not arrived, all of them again from where the search began, a pass at a
|
|
892
|
+
* time: the first a minute later, each after as long again as it has been
|
|
893
|
+
* since then, and one after the other, as far as its credit, while an
|
|
894
|
+
* arrival is held back. A pass under way takes what the search ahead leaves
|
|
895
|
+
* of a call's windows, one at least, in windows no wider than the RPC took
|
|
896
|
+
* before. A pass counts once it read the blocks again, as does a call whose
|
|
897
|
+
* window at the head reached back to where the search began.
|
|
898
|
+
*
|
|
798
899
|
* @param {Reader} read
|
|
799
900
|
* @param {{ given: Queued[], queue: Queued[], guarded: boolean, unsure: ReadonlySet<string>, rivals?: ReadonlySet<string>, filter: { address: string, topics: ReadonlyArray<string | null>, wei: bigint, amountOf: (data: string) => bigint | null } }} group
|
|
800
901
|
* @param {{ url: string, maxWindows: number, blockTimes: BlockTimes | undefined, now: number, slackMs: number, windowMs: number, used: Set<string> }} options
|
|
@@ -821,9 +922,11 @@ async function followLogs(read, { filter, given, guarded, queue, rivals, unsure
|
|
|
821
922
|
// The logs searched for: a search for others, as for another sender or
|
|
822
923
|
// amount, isn't carried on.
|
|
823
924
|
const search = [filter.address, ...filter.topics.map((topic) => topic ?? "*"), filter.wei.toString()].join(":");
|
|
824
|
-
/** @type {{ basis: number, from: string, seen?: Scan["seen"] }[]} */
|
|
925
|
+
/** @type {{ basis: number, from: string, seen?: Scan["seen"], start?: string, recheck?: Scan["recheck"] }[]} */
|
|
825
926
|
const resumed = live.flatMap(({ scan }) =>
|
|
826
|
-
scan?.from !== undefined && scan.search === search && scan.basis !== undefined && scan.basis <= first.at
|
|
927
|
+
scan?.from !== undefined && scan.search === search && scan.basis !== undefined && scan.basis <= first.at
|
|
928
|
+
? [{ basis: scan.basis, from: scan.from, recheck: scan.recheck, seen: scan.seen, start: scan.start }]
|
|
929
|
+
: []
|
|
827
930
|
);
|
|
828
931
|
/** @type {Map<string, Credit>} */
|
|
829
932
|
const carried = new Map();
|
|
@@ -832,7 +935,7 @@ async function followLogs(read, { filter, given, guarded, queue, rivals, unsure
|
|
|
832
935
|
// counts again, as the transfer it paid may be among the queue's.
|
|
833
936
|
for (const scan of resumed) {
|
|
834
937
|
for (const credit of scan.seen ?? []) {
|
|
835
|
-
if (credit.at >= reach) carried.set(credit.hash, { at: credit.at, block: BigInt(credit.block), from: credit.sender, hash: credit.hash, success: true, wei: filter.wei });
|
|
938
|
+
if (credit.at >= reach) carried.set(credit.hash, { at: credit.at, block: BigInt(credit.block), from: credit.sender, hash: credit.hash, pass: credit.pass ?? 0, success: true, wei: filter.wei });
|
|
836
939
|
}
|
|
837
940
|
}
|
|
838
941
|
|
|
@@ -840,10 +943,29 @@ async function followLogs(read, { filter, given, guarded, queue, rivals, unsure
|
|
|
840
943
|
let head;
|
|
841
944
|
/** @type {bigint} */
|
|
842
945
|
let start;
|
|
946
|
+
// The block the search began at, and its passes over the blocks before.
|
|
947
|
+
/** @type {bigint} */
|
|
948
|
+
let floor;
|
|
949
|
+
/** @type {ReturnType<typeof recheckOf>} */
|
|
950
|
+
let recheck = null;
|
|
843
951
|
|
|
844
952
|
if (resumed.length && resumed.every((scan) => scan.basis === first.at)) {
|
|
845
|
-
|
|
846
|
-
|
|
953
|
+
const kept = resumed.reduce((a, b) => (BigInt(b.from) < BigInt(a.from) ? b : a));
|
|
954
|
+
|
|
955
|
+
start = BigInt(kept.from);
|
|
956
|
+
recheck = recheckOf(kept.recheck);
|
|
957
|
+
|
|
958
|
+
if (resumed.every((scan) => scan.start !== undefined)) {
|
|
959
|
+
floor = minOf(resumed.map((scan) => BigInt(/** @type {string} */ (scan.start))));
|
|
960
|
+
head = await headOf(read, url);
|
|
961
|
+
} else {
|
|
962
|
+
// Kept without where it began: from where a search from the first
|
|
963
|
+
// begins.
|
|
964
|
+
const fresh = first.notBefore !== null ? { head: await headOf(read, url), start: first.notBefore } : await startBefore(read, url, reach, blockTimes);
|
|
965
|
+
|
|
966
|
+
floor = fresh.start;
|
|
967
|
+
head = fresh.head;
|
|
968
|
+
}
|
|
847
969
|
} else if (resumed.length) {
|
|
848
970
|
// Begun from a transfer before the first now, as one that since
|
|
849
971
|
// arrived: none of the queue's credits is before where a search from
|
|
@@ -853,39 +975,161 @@ async function followLogs(read, { filter, given, guarded, queue, rivals, unsure
|
|
|
853
975
|
|
|
854
976
|
head = fresh.head;
|
|
855
977
|
start = from > fresh.start ? from : fresh.start;
|
|
978
|
+
floor = fresh.start;
|
|
856
979
|
} else if (first.notBefore !== null) {
|
|
857
980
|
start = first.notBefore;
|
|
981
|
+
floor = start;
|
|
858
982
|
head = await headOf(read, url);
|
|
859
983
|
} else {
|
|
860
984
|
({ head, start } = await startBefore(read, url, reach, blockTimes));
|
|
985
|
+
floor = start;
|
|
861
986
|
}
|
|
862
987
|
|
|
988
|
+
if (floor > start) floor = start;
|
|
989
|
+
|
|
990
|
+
// No later than a credit found before: the blocks are read again from it.
|
|
991
|
+
for (const credit of carried.values()) if (credit.block !== null && credit.block < floor) floor = credit.block;
|
|
992
|
+
|
|
993
|
+
// The widest window the blocks are read again in, where the RPC refused a
|
|
994
|
+
// wider one.
|
|
995
|
+
const width = recheck?.size ? BigInt(recheck.size) : BigInt(LOG_WINDOW);
|
|
996
|
+
|
|
997
|
+
/**
|
|
998
|
+
* The given transfers' credits with `found`: those that arrived, by their
|
|
999
|
+
* credits, and those held back (`held`), with the last of their credits'
|
|
1000
|
+
* blocks (`top`). One a later transfer of the queue, still without a
|
|
1001
|
+
* credit, could have had the credit of instead is held back until the
|
|
1002
|
+
* blocks before it are read again `CONFIRM_PASSES` times since it was
|
|
1003
|
+
* found: its own may yet be found there.
|
|
1004
|
+
*
|
|
1005
|
+
* @param {Credit[]} found
|
|
1006
|
+
* @param {boolean} seen
|
|
1007
|
+
*/
|
|
1008
|
+
const settled = (found, seen) => {
|
|
1009
|
+
const { assigned, verdicts } = settle(queue, found, rules);
|
|
1010
|
+
const passes = recheck?.passes ?? 0;
|
|
1011
|
+
const mine = new Set(live.map((item) => item.hash));
|
|
1012
|
+
/** @type {Set<string>} */
|
|
1013
|
+
const arrived = new Set();
|
|
1014
|
+
/** @type {Set<string>} */
|
|
1015
|
+
const held = new Set();
|
|
1016
|
+
/** @type {bigint | null} */
|
|
1017
|
+
let top = null;
|
|
1018
|
+
|
|
1019
|
+
queue.forEach((item, i) => {
|
|
1020
|
+
const credit = assigned.get(item.hash);
|
|
1021
|
+
|
|
1022
|
+
if (!credit || !mine.has(item.hash) || verdicts.get(item.hash) !== "arrived" || !(seen || !guarded)) return;
|
|
1023
|
+
|
|
1024
|
+
// A later one without a credit could have had this one instead.
|
|
1025
|
+
const doubt = !guarded && passes - (credit.pass ?? 0) < CONFIRM_PASSES && queue.slice(i + 1).some((later) => !assigned.has(later.hash) && pays(later, credit, rules));
|
|
1026
|
+
|
|
1027
|
+
if (!doubt) {
|
|
1028
|
+
arrived.add(credit.hash);
|
|
1029
|
+
} else {
|
|
1030
|
+
held.add(item.hash);
|
|
1031
|
+
if (credit.block !== null && (top === null || credit.block > top)) top = credit.block;
|
|
1032
|
+
}
|
|
1033
|
+
});
|
|
1034
|
+
|
|
1035
|
+
return { arrived, assigned, held, top, verdicts };
|
|
1036
|
+
};
|
|
1037
|
+
/**
|
|
1038
|
+
* The credits found in this call, with the passes done before the next
|
|
1039
|
+
* one to begin.
|
|
1040
|
+
*
|
|
1041
|
+
* @param {Credit[]} found
|
|
1042
|
+
* @returns {Credit[]}
|
|
1043
|
+
*/
|
|
1044
|
+
const mark = (found) => {
|
|
1045
|
+
const pass = recheck === null ? 0 : recheck.passes + (recheck.from !== null ? 1 : 0);
|
|
1046
|
+
|
|
1047
|
+
return found.map((credit) => (credit.pass === undefined ? { ...credit, pass } : credit));
|
|
1048
|
+
};
|
|
863
1049
|
// Where the credits can't be told apart, only those in the transfers'
|
|
864
1050
|
// windows count: once they're all over, read no further than the last.
|
|
865
1051
|
const last = queue[queue.length - 1].at + windowMs;
|
|
866
|
-
const
|
|
1052
|
+
const until = guarded && now >= last ? last : null;
|
|
1053
|
+
const due = recheck !== null && (recheck.from !== null || now >= recheck.at);
|
|
1054
|
+
const ahead = await searchLogs(read, {
|
|
867
1055
|
...filter,
|
|
868
1056
|
blockTimes,
|
|
869
1057
|
// Once every transfer of the queue has its credit: read no further.
|
|
870
1058
|
enough: (found) => !guarded && settle(queue, found, rules).assigned.size === queue.length,
|
|
1059
|
+
floor,
|
|
871
1060
|
head,
|
|
872
1061
|
known: [...carried.values()],
|
|
873
1062
|
maxWindows,
|
|
1063
|
+
overlap: width < OVERLAP ? width : OVERLAP,
|
|
874
1064
|
start,
|
|
875
|
-
until
|
|
1065
|
+
until,
|
|
876
1066
|
url,
|
|
877
1067
|
});
|
|
1068
|
+
const { complete, covered, next } = ahead;
|
|
878
1069
|
const seen = complete || covered;
|
|
879
|
-
|
|
880
|
-
|
|
881
|
-
const
|
|
1070
|
+
// Its window at the head reached back to where the search began: every
|
|
1071
|
+
// block read before this call is read again, as by a pass.
|
|
1072
|
+
const again = recheck !== null && ahead.low !== null && ahead.low <= floor && start > floor;
|
|
1073
|
+
|
|
1074
|
+
if (again && recheck !== null) recheck = { ...recheck, passes: recheck.passes + 1 };
|
|
1075
|
+
|
|
1076
|
+
let credits = mark(ahead.credits);
|
|
1077
|
+
const before = settled(credits, seen);
|
|
1078
|
+
let { refused, took } = ahead;
|
|
1079
|
+
|
|
1080
|
+
if ((seen || before.held.size) && recheck === null) recheck = { at: before.held.size ? now : now + RECHECK_MS, from: null, passes: 0, since: now, size: null };
|
|
1081
|
+
|
|
1082
|
+
// A transfer not arrived yet: the blocks read before this call, again, as
|
|
1083
|
+
// far as the search ahead reads; while an arrival is held back, as far as
|
|
1084
|
+
// its credit. With what the search ahead leaves of a call's windows, one
|
|
1085
|
+
// at least.
|
|
1086
|
+
if (recheck !== null && (due || before.held.size) && before.arrived.size < live.length) {
|
|
1087
|
+
const from = recheck.from !== null && recheck.from > floor ? recheck.from : floor;
|
|
1088
|
+
const below = ahead.low !== null && ahead.low < start ? ahead.low : start;
|
|
1089
|
+
const pass = await searchLogs(read, {
|
|
1090
|
+
...filter,
|
|
1091
|
+
blockTimes,
|
|
1092
|
+
// Read through: a credit further on, still before those found ahead,
|
|
1093
|
+
// could be a transfer's first.
|
|
1094
|
+
enough: () => false,
|
|
1095
|
+
floor: from,
|
|
1096
|
+
head: before.top !== null && before.top < below ? before.top : below - 1n,
|
|
1097
|
+
known: credits,
|
|
1098
|
+
maxWindows: maxWindows < 1 ? 0 : Math.max(1, maxWindows - ahead.asked),
|
|
1099
|
+
size: ahead.size < width ? ahead.size : width,
|
|
1100
|
+
start: from,
|
|
1101
|
+
until,
|
|
1102
|
+
url,
|
|
1103
|
+
});
|
|
1104
|
+
const done = pass.complete || pass.covered;
|
|
1105
|
+
// A pass that read no block is none: only one that read the blocks again
|
|
1106
|
+
// counts, and not twice in a call.
|
|
1107
|
+
const counted = done && !again && pass.next > floor;
|
|
1108
|
+
|
|
1109
|
+
recheck = done ? { ...recheck, at: now + Math.max(RECHECK_MS, now - recheck.since), from: null, passes: recheck.passes + (counted ? 1 : 0) } : { ...recheck, from: pass.next };
|
|
1110
|
+
credits = mark(pass.credits);
|
|
1111
|
+
if (pass.refused !== null && (refused === null || pass.refused < refused)) refused = pass.refused;
|
|
1112
|
+
if (pass.took !== null && (took === null || pass.took > took)) took = pass.took;
|
|
1113
|
+
}
|
|
882
1114
|
|
|
883
|
-
|
|
884
|
-
|
|
1115
|
+
// A window refused: from the next call on, the blocks are read again in
|
|
1116
|
+
// ones as wide as the widest it took narrower than that, else a quarter
|
|
1117
|
+
// as wide.
|
|
1118
|
+
if (recheck !== null && refused !== null) {
|
|
1119
|
+
const fits = took !== null && took < refused ? took : refused / 4n;
|
|
1120
|
+
const narrower = fits > MIN_WINDOW ? fits : MIN_WINDOW;
|
|
885
1121
|
|
|
886
|
-
if (
|
|
1122
|
+
if (narrower < width) recheck = { ...recheck, size: Number(narrower) };
|
|
887
1123
|
}
|
|
888
1124
|
|
|
1125
|
+
const { arrived, assigned, held, verdicts } = settled(credits, seen);
|
|
1126
|
+
|
|
1127
|
+
// Passes go on one after the other while an arrival is held back; once one
|
|
1128
|
+
// arrives with others still waiting, theirs begin again a minute later,
|
|
1129
|
+
// the waits growing from there.
|
|
1130
|
+
if (recheck !== null && held.size) recheck = { ...recheck, at: Math.min(recheck.at, now) };
|
|
1131
|
+
else if (recheck !== null && arrived.size && arrived.size < live.length) recheck = { ...recheck, at: Math.min(recheck.at, now + RECHECK_MS), since: now };
|
|
1132
|
+
|
|
889
1133
|
// The next search carries on from where this one stopped, with the
|
|
890
1134
|
// credits found so far: a transfer that arrived may come back from the
|
|
891
1135
|
// chain among those before the next one, and takes its own again.
|
|
@@ -893,15 +1137,19 @@ async function followLogs(read, { filter, given, guarded, queue, rivals, unsure
|
|
|
893
1137
|
const scan = {
|
|
894
1138
|
basis: first.at,
|
|
895
1139
|
from: next.toString(),
|
|
1140
|
+
...(recheck ? { recheck: recheckIn(recheck) } : {}),
|
|
896
1141
|
search,
|
|
897
|
-
seen: credits.flatMap((credit) => (credit.at >= reach && credit.block !== null ? [{ at: credit.at, block: credit.block.toString(), hash: credit.hash, sender: credit.from }] : [])),
|
|
1142
|
+
seen: credits.flatMap((credit) => (credit.at >= reach && credit.block !== null ? [{ at: credit.at, block: credit.block.toString(), hash: credit.hash, pass: credit.pass ?? 0, sender: credit.from }] : [])),
|
|
1143
|
+
start: floor.toString(),
|
|
898
1144
|
};
|
|
899
1145
|
|
|
900
1146
|
for (const item of live) {
|
|
901
1147
|
const credit = assigned.get(item.hash);
|
|
902
1148
|
const verdict = verdicts.get(item.hash);
|
|
903
1149
|
|
|
904
|
-
if (
|
|
1150
|
+
if (held.has(item.hash)) {
|
|
1151
|
+
out.set(item.hash, { scan, state: "waiting" });
|
|
1152
|
+
} else if (credit && arrived.has(credit.hash)) {
|
|
905
1153
|
used.add(credit.hash);
|
|
906
1154
|
out.set(item.hash, {
|
|
907
1155
|
amount: credit.wei / WEI_PER_LUX,
|
|
@@ -1243,7 +1491,17 @@ async function depositPeers(archive, net, found, { amount, bounds, calls, suppli
|
|
|
1243
1491
|
* @param {Scan | null | undefined} scan
|
|
1244
1492
|
* @returns {Scan}
|
|
1245
1493
|
*/
|
|
1246
|
-
|
|
1494
|
+
function searchOf(scan) {
|
|
1495
|
+
if (scan?.from === undefined) return {};
|
|
1496
|
+
|
|
1497
|
+
/** @type {Scan} */
|
|
1498
|
+
const out = { basis: scan.basis, from: scan.from, search: scan.search, seen: scan.seen };
|
|
1499
|
+
|
|
1500
|
+
if (scan.start !== undefined) out.start = scan.start;
|
|
1501
|
+
if (scan.recheck !== undefined) out.recheck = scan.recheck;
|
|
1502
|
+
|
|
1503
|
+
return out;
|
|
1504
|
+
}
|
|
1247
1505
|
|
|
1248
1506
|
/**
|
|
1249
1507
|
* The transfers whose history isn't all read yet wait, with where their
|