@dusk-network/app-core 0.3.1 → 0.4.0

package/CHANGELOG.md

@@ -2,6 +2,39 @@
 
 All notable changes to this package are listed here. It follows [Keep a Changelog](https://keepachangelog.com/en/1.1.0/) and [Semantic Versioning](https://semver.org/).
 
+## [0.4.0]
+
+### Added
+
+- `/history`: `memoBinary` on an entry whose memo is bytes given as 0x hex, rather than typed 0x text
+- `/history`: a `check` for the store's `patch` and `clear` to make in the same transaction, so a run after a reset leaves another run's history of the new chain alone
+- `/shielded`: an optional `update` on the store, which keeps the decoys in one transaction, so two reads at once ask among the same ones
+- `/bridge`: `serviceArrivals` takes `blockTimes` as a cache for each chain, by its chain ID
+
+### Changed
+
+- `/history`: a details request that fails in `sync()` ends the run's details and resolves it with `detailsError`, rather than rejecting the run
+- `/bridge`: an RPC search's windows grow up to 100,000 blocks while the RPC answers them whole, and a call reads a chain's head and pace once for all its searches
+
+### Fixed
+
+- `/shielded`: a conversion is a shield or an unshield as its event says, also for a profile that only got its change
+- `/shielded`: two notes a block made for one shielded address go to a transaction each, rather than both to the first
+- `/activity`: a gas refund from another account's call to DuskEVM's bridge is a received transfer, rather than a bridge withdrawal
+- `/bridge`: an RPC search starts 10 minutes before the transfer as the chain's pace says, at a block whose time is checked to be before it, rather than a fifth further back than the pace says
+- `/bridge`: an RPC search keeps the width of its windows in its `scan`, so one window a call moves on past an RPC that takes fewer blocks than it asked
+- `/bridge`: `serviceArrivals` uses one `blockTimes` cache only for a network whose service routes are all on one chain
+- `/bridge`: a deposit without its sender's history reads no further once its windows are over and read
+- `/duskevm-withdrawals`: `syncWithdrawals` reads the final blocks again, the newest 50,000 on each run and all of them a pass at a time once covered, rather than once
+- `/duskevm-withdrawals`: `syncWithdrawals` counts both its head's requests against `maxRequests` where the RPC has no finalized block
+
+## [0.3.2]
+
+### Fixed
+
+- `/bridge`: an RPC search reads each block more than once while its transfers wait, from the first transfer's block on, rather than once
+- `/bridge`: on an RPC, an earlier one of equal transfers takes a credit a later one could have had once the blocks before it are read again, rather than at once
+
 ## [0.3.1]
 
 ### Added

package/README.md

@@ -53,13 +53,14 @@ const { entries, complete } = await history.read({ limit: 50 });
 ```
 
 - A run reads the newest heights first, in windows under the last finalized block: a day, then wider while they hold few entries, narrower where a page of 50 fills. It makes at most 8 history requests and 2 details requests (`maxRequests`, 2 at least, and `maxDetailRequests`), and keeps each step as it lands, each page of a block with more than a page too, so the next run carries on where it stopped. A later run asks only for what is above the heights kept.
+- A details request that fails ends the run's details, not the run: it reads on, and resolves with `detailsError`. The next run asks for the details again. A history request that fails rejects the run, as does the store.
 - `read({ limit, before })` gives the entries above the first gap, newest first: below it, newer ones may be missing. `complete` says every height from the head down is covered. When `limit` cuts through a block, the rest of that block comes too, so `before` the last entry's height reads on without skipping any.
 - A node whose GraphQL has no archive is asked nothing for an hour. Without `refusals`, the archive object alone remembers it.
-- A node on another chain than the history kept (a reset devnet) makes the run ask the store to `clear()` and start over, within its budget. A run another one overtook that way, or whose history was cleared under it, keeps nothing more and rejects with a `StaleHistoryError`.
+- A node on another chain than the history kept (a reset devnet) makes the run ask the store to `clear()` and start over, within its budget; where another run started it over on that chain already, it carries on from what that run kept. A run another one overtook that way, or whose history was cleared under it, keeps nothing more and rejects with a `StaleHistoryError`.
 - `network` is the node's network or its chain ID, for the DuskEVM bridge's calls, the bridge's routes and Sozu. A node of a network there's no file for passes none.
 - `fillDetails(entries)` reads the details of entries shown before a run gave them any.
 
-An entry is facts, not text: `direction` ("in", "out", "self"), `kind` (`transfer`, `shield`, `unshield`, `contract_call`, `stake`, `unstake`, `withdraw_reward`), `counterparty`, `value` and `fee` in lux (a gas refund paid to another account counts in the sender's `value` and in that account's, told apart as `refund`), `memo`, and with its details its time, outcome, gas and call. A call to DuskEVM's bridge or Portal has `bridge`, what it did (`decodeBridgeCall`): a deposit's amount and DuskEVM address, or a withdrawal's proof or claim, its hash, whom it pays and its nonce. A transfer with one of the bridge's accounts has its `route` (`routeForAccount`), and a stake or an unstake with Sozu `sozu: true` (`isSozuCall`, with the pools the hub named in `read({ pools })`).
+An entry is facts, not text: `direction` ("in", "out", "self"), `kind` (`transfer`, `shield`, `unshield`, `contract_call`, `stake`, `unstake`, `withdraw_reward`), `counterparty`, `value` and `fee` in lux (a gas refund paid to another account counts in the sender's `value` and in that account's, told apart as `refund`), `memo` (as text, or as 0x hex with `memoBinary: true` for bytes that aren't text), and with its details its time, outcome, gas and call. A call to DuskEVM's bridge or Portal has `bridge`, what it did (`decodeBridgeCall`): a deposit's amount and DuskEVM address, or a withdrawal's proof or claim, its hash, whom it pays and its nonce. A transfer with one of the bridge's accounts has its `route` (`routeForAccount`), and a stake or an unstake with Sozu `sozu: true` (`isSozuCall`, with the pools the hub named in `read({ pools })`).
 
 ### The store
 
@@ -69,9 +70,11 @@ The app keeps each account's history where it wants, under its own keys, and pas
 |---|---|
 | `getState()` | The sync state, or null before the first run. |
 | `commit(entries, update)` | Keeps one step in one transaction: `entries` by hash, an entry kept already keeping the fields these don't carry (its details), and the state `update(kept)` makes from the state kept now. When `update` throws, nothing of the step is kept. Gives the state kept. |
-| `patch(details)` | Adds details to kept entries, by hash (a `Map`); entries that aren't kept are left alone. |
+| `patch(details, check)` | Adds details to kept entries, by hash (a `Map`); entries that aren't kept are left alone. |
 | `list({ limit, before })` | Kept entries, newest first: at most `limit`, below the height `before` when given. |
-| `clear()` | Forgets what is kept of the old chain. |
+| `clear(check)` | Forgets what is kept of the old chain. |
+
+A store that can calls `check(kept)` first, with the state kept now, in the same transaction, and writes nothing when it throws. Then no details go on a history another run started over since, and a run that finds a reset doesn't forget what another one already read of the new chain. A store that doesn't call it works as before.
 
 The state is `{ account, covered, crowded, finalized, genesis, headAt, target, updatedAt, window }`; fields the store keeps beside these stay. An entry is kept with its amounts as decimal strings, so it stores as JSON. A store that rejects a write, as one bound to a wallet that has been reset since, makes the run reject, and nothing more is kept.
 
@@ -130,7 +133,7 @@ const { entries, pending } = await shieldedHistory({
 - `isCurrent` is asked between every two steps, before each request and write, and once more after the history is kept: once the wallet, the profile or the node changed, the read asks and keeps nothing more and rejects. A node that answered without the archive is asked nothing for an hour, its tree included.
 - The blocks come 15 heights a request, through the archive of `/history`.
 
-An entry is facts, not text: `kind` (the kinds of `/history`), `direction` for the profile's notes ("in", "out", or "self" when what they spent came back but the fee), `value` and `fee` in lux as decimal strings (a fee only where the profile's notes paid it), `success` and `error`, and its block's `timestamp`. It stores as JSON as it is.
+An entry is facts, not text: `kind` (the kinds of `/history`; a conversion's by where its value went, as its event says), `direction` for the profile's notes ("in", "out", or "self" when what they spent came back but the fee), `value` and `fee` in lux as decimal strings (a fee only where the profile's notes paid it), `success` and `error`, and its block's `timestamp`. It stores as JSON as it is.
 
 ### The stores and the tree
 
@@ -140,6 +143,7 @@ The app keeps the history where it wants, under its own keys, and reads its note
 |---|---|
 | `store.get()` | The profile's kept history, `{ fingerprint, entries, decoys }`, or null. Reject when it can't be read: the read then asks nothing rather than draw new decoys. |
 | `store.put(record)` | Keeps it in place of the one kept. A put that rejects, as one bound to a wallet that has been reset since, rejects the read. A store that can't keep a history for want of room may pass over it: the read gives it anyway, and the next one works it out again among the same decoys. |
+| `store.update(change)` | Optional: keeps `change(kept)` in place of the record kept now, in one transaction, and gives what it kept. The decoys are then drawn from those kept and kept at once, so two reads of a profile at once ask among the same ones. Without it, two such reads can each draw their own. It rejects and passes over a history as `put` does. |
 | `treeHeights.get()`, `treeHeights.put({ heights, notes })` | The note tree's distinct heights from its first `notes` leaves, shared by the network's profiles. A put that fails is passed over. |
 | `tree.size()` | How many leaves the node's note tree has (the transfer contract's `num_notes`). |
 | `tree.leaves(from)` | Its leaves from position `from` on (`leaves_from_pos`): a stream of their bytes, or an answer whose body is one. |
@@ -167,7 +171,7 @@ const rows = activityRows({
 - A send the chain's histories don't have yet is `pending`: for 15 minutes at most (`PENDING_MS`), unless the node has it in its queue, then `unknown`. One the node dropped is `unknown` and `dropped`. A send marked settled shows with the chain's row alone.
 - The shielded side of a shield or an unshield is the public row's, with the fee the shielded side paid.
 - `category` is the filter a row is under: "staking" (native and Sozu's), "bridge" (DuskEVM's bridge and the routes to other chains), "sent", "received", or "other" (a shield, an unshield, a transfer to the account itself).
-- A call to DuskEVM's bridge or Portal has `bridge`: what it did (`step`: "deposit", "withdrawal", "proof", "claim" or "call"), the amount its call names, a deposit's DuskEVM address and where it arrived there (`arrivals`), a withdrawal's hash, recipient and nonce, and whether a proof's withdrawal is claimed: by a claim of the same activity that went through, is on its way or is on chain, or as the app read it (`claims`).
+- A call to DuskEVM's bridge or Portal has `bridge`: what it did (`step`: "deposit", "withdrawal", "proof", "claim" or "call"), the amount its call names, a deposit's DuskEVM address and where it arrived there (`arrivals`), a withdrawal's hash, recipient and nonce, and whether a proof's withdrawal is claimed: by a claim of the same activity that went through, is on its way or is on chain, or as the app read it (`claims`). What came in from an account has none, whatever its transaction called: a gas refund from another account's call to the bridge is a transfer from that account.
 - A transfer with one of the bridge's accounts has its `route`, and a call of Sozu's pool `sozu`, "stake" or "unstake" (with the pools the hub named in `pools`).
 - `nameCall` is the app's hook for the name service, as the wallets read .dusk names with `@duskdomains/sdk/call-facts`: given a call's contract and function, and the entry or the send it is from, it says what the call does, and the name and the reservation it is for. A reservation has the name of a registration after it with the same reservation.
 
@@ -206,7 +210,7 @@ arrivalsOf(arrivals); // Map { hash => { hash: "0x029e…", at: 1793080333000 }
 - A transfer out is read from the chain: what the route's account received, the address its memo gives as the route's bridge reads it (0x and 40 hex digits, any case, nothing around them), and its block. It arrives with the DUSK token's `Transfer` from its route's payout address to that address, of what the account received less the fee. The account's history lists every transfer that named the same address, in any case: equal ones in the hours before (a given one's by its own fee) are paid first, and one that can't be read leaves the order unclear. One that pays nothing out is `no-payout`, with its `problem`. `serviceTransferOf` gives what left the account (`sent`), and what the account received (`amount`) only where no gas was left to refund: an entry's value holds any refund paid to another account.
 - With the archive, each transfer reads only its own hours of its account's history (its sender's for a deposit, the route's account's for a transfer out), further back while equal ones before it run on, and only once the archive lists the transfer there. A call reads 4 pages of it (`HISTORY_PAGES`) for each: a transfer whose history is longer waits, and the next call reads on from its `scan`.
 - A route without its payout account or address in the network's file pays out nothing that can be told apart: `no-route`.
-- An RPC search starts at a block from before the transfer, estimated from the chain's pace and checked by its time, and reads 8 windows of 5,000 blocks at most, narrower where the RPC asks for it. A result that isn't arrived says where the next search starts, with what it searched for and the credits found so far (`scan`): the app keeps it with the arrivals it was made with, and passes both back. A search for another sender or amount starts afresh.
+- An RPC search starts at a block from before the transfer, 10 minutes before it as the chain's pace says, checked by its time and stepped back until it is, and reads 8 windows at most: of 5,000 blocks, twice as many after each one the RPC answers whole, up to 100,000, and fewer where it asks for it, the width kept in the search's `scan` from call to call. A call reads a chain's head and pace once for all its searches. Once a deposit's windows are over and read, a search without its sender's history reads no further. It reads each block more than once: its window at the head reaches back 1,000 blocks, so the newest blocks are read again on each call, and once it has reached the head with a transfer not arrived, it reads its blocks again from the first, a pass at a time with the windows a call has left (one at least while a pass is under way, so 9 where the search ahead takes all 8): the first a minute later, each one after as long again as it has been since it reached the head. Where a later equal transfer, still without its credit, could have had the credit an earlier one takes, the earlier one waits until the blocks are read again 8 times since the credit was found, one pass after the other. A result that isn't arrived says where the next search starts, with what it searched for and the credits found so far (`scan`): the app keeps it with the arrivals it was made with, and passes both back. A search for another sender or amount starts afresh. The block times the app keeps (`blockTimes`) are a chain's: for transfers out, a cache a chain by its chain ID, or one cache where the network's service routes are all on one chain.
 
 A result is facts: `arrived`, with the other side's transaction, its block's time, the lux it paid and its block; `waiting`; or `unknown`, with why: `unsure` (a credit could be another's), `late` (none came in its window), `beyond` (the explorer's pages don't reach it), `searching` (carry on from `scan`), `expired`, `no-route`, `no-payout` or `error`. The words are the app's.
 
@@ -248,7 +252,7 @@ const claimCall = await withdrawals.claimCall(txHash, { submitter: account }); /
 
 ### Finding withdrawals
 
-- `syncWithdrawals({ address, duskEvm, network, state })` reads an address's withdrawals from the bridge's own `ETHBridgeInitiated` events, the newest final blocks first, a window of 100,000 blocks at a time (narrower where the RPC asks for it), 8 requests a run. It gives the app the state to keep, and the next run carries on below and reads only what is new above. Under a final head lower than the last run's, what it covered and found above it is dropped, and read again.
+- `syncWithdrawals({ address, duskEvm, network, state })` reads an address's withdrawals from the bridge's own `ETHBridgeInitiated` events, the newest final blocks first, a window of 100,000 blocks at a time (narrower where the RPC asks for it), 8 requests a run, the head's among them. It gives the app the state to keep, and the next run carries on below. Each run reads the newest 50,000 final blocks, the new ones first, in as many windows as they take, though in half its requests at most (while older blocks are still to be read, only once the final head has moved). Under a final head lower than the last run's, what it covered and found above it is dropped, and read again. It reads each final block more than once: once it has covered them all, it reads them again from the newest down, a pass at a time with the requests a run has left, the first an hour later and each one after as long again as it has been since. A withdrawal is found once.
 - `explorerWithdrawals({ address, network, fetch })` lists the address's own calls of the bridge on DuskEVM's explorer, each with the explorer's word for it (`explorerStage`), and `withdrawalsByNonce({ nonces, network, fetch })` finds the DuskEVM transactions of withdrawals known by the nonces their proofs and claims on Dusk carry.
 
 ### Reads and caches

package/package.json

@@ -1,6 +1,6 @@
 {
   "name": "@dusk-network/app-core",
-  "version": "0.3.1",
+  "version": "0.4.0",
   "description": "Shared logic for the Dusk apps: public and shielded history, activity rows, bridge arrivals and DuskEVM withdrawals.",
   "license": "MPL-2.0",
   "type": "module",

package/src/activity.d.ts

@@ -248,7 +248,11 @@ export type ActivityRow = {
   route: Route | null;
   /** A call of Sozu's pool (`isSozuCall`): whether it staked or unstaked. */
   sozu: "stake" | "unstake" | null;
-  /** A call to DuskEVM's bridge or Portal. */
+  /**
+   * A call to DuskEVM's bridge or Portal. What came in from an account has
+   * none, whatever its transaction called: a gas refund from another
+   * account's call to the bridge is a transfer from that account.
+   */
   bridge: ActivityBridge | null;
   /**
    * A call of the name service, as the app's hook reads it; a reservation

package/src/activity.js

@@ -8,6 +8,7 @@
  */
 
 import { networkFor, routeForAccount } from "@dusk-network/networks";
+import { isAccount } from "@dusk-network/networks/account";
 import { BRIDGE_FUNCTIONS, isBridgeContract } from "@dusk-network/networks/duskevm";
 import { SOZU_FN, isSozuCall } from "@dusk-network/networks/sozu";
 
@@ -570,7 +571,12 @@ export function activityRows({
             : null;
     const sozuCall = Boolean(c?.sozu) || Boolean(net && isSozuCall(net, { contractId, fnName }, { pools: poolList }));
     const call = c?.bridge ?? s?.bridge ?? null;
-    const touchesBridge = Boolean(call) || Boolean(net && [contractId, counterparty].some((id) => isBridgeContract(/** @type {Network} */ (net), id)));
+    const touchesBridge =
+      (Boolean(call) || Boolean(net && [contractId, counterparty].some((id) => isBridgeContract(/** @type {Network} */ (net), id)))) &&
+      // What came in from an account isn't the bridge's, whatever its
+      // transaction called: a gas refund from another account's call to the
+      // bridge is a transfer from that account.
+      !(direction === "in" && isAccount(counterparty));
     const bridge = touchesBridge ? bridgeOf({ call, direction, fnName, value }) : null;
 
     /** @type {ActivityRow["nameCall"]} */

package/src/bridge.d.ts

@@ -73,10 +73,14 @@ export declare const WAIT_MS: 604800000;
 /** Pages of an address's incoming calls on DuskEVM's explorer read at most, by default. */
 export declare const EXPLORER_PAGES: 5;
 
-/** Blocks one log search asks for at once; narrower where the RPC asks for it. */
+/**
+ * Blocks a search's first log search asks for at once: each one after asks
+ * for twice as many as one the RPC answered whole, up to 100,000, and fewer
+ * where the RPC asks for it. The width is kept in the search's `scan`.
+ */
 export declare const LOG_WINDOW: 5000;
 
-/** Log searches one call makes for a group of equal transfers, by default. */
+/** Log searches one call makes for a group of equal transfers, by default: one more where the search ahead takes them all while its blocks are read again. */
 export declare const LOG_WINDOWS: 8;
 
 /** Equal deposits sent within this of each other are paid in their order: 2 hours, in milliseconds. */
@@ -113,6 +117,14 @@ export type HistoryScan = {
  * transfer it began from, the logs it searched for (`search`), and the
  * credits it found before (`seen`), which the next one counts again. A
  * search for other logs, as for another sender or amount, starts afresh.
+ * The block it began at (`start`), and its passes over the blocks it read
+ * before (`recheck`): where the one under way is (`from`, null between
+ * two), when the next one begins (`at`), how many are done, as each credit
+ * seen has the passes done before it was found (`pass`), when the search
+ * first reached the head (`since`), in milliseconds (and the width of the
+ * windows they were read in, as an earlier version kept it: `size`). The
+ * width of the next search's windows (`size`), and the narrowest the RPC
+ * refused (`refused`): it stays narrower than that.
  * With the archive, how far the transfer's own history is read
  * (`history`): a history longer than one call reads is read on from there,
  * before any search. Kept by the app with the credits it was made with,
@@ -122,7 +134,11 @@ export type Scan = {
   from?: string;
   basis?: number;
   search?: string;
-  seen?: ReadonlyArray<{ hash: string; block: string; at: number; sender: string | null }>;
+  seen?: ReadonlyArray<{ hash: string; block: string; at: number; sender: string | null; pass?: number }>;
+  start?: string;
+  recheck?: { from: string | null; at: number; passes: number; since: number; size?: number };
+  size?: number;
+  refused?: number;
   history?: HistoryScan;
 };
 
@@ -250,7 +266,8 @@ export type DepositArrivalOptions = {
  * deposits, newest first, back past the oldest one waited for, at most
  * `maxPages` pages; none is asked while every deposit's window is still
  * open. On the RPC, a group of equal deposits from one sender is searched
- * from about when the first was sent, `maxWindows` windows at most.
+ * from about when the first was sent, `maxWindows` windows at most (one more
+ * where the search ahead takes them all while its blocks are read again).
  * A network without DuskEVM asks nothing. Throws a TypeError for a deposit
  * that isn't one, and for a network there's no file for.
  */
@@ -285,8 +302,12 @@ export type ServiceArrivalOptions = {
   rpcUrl?: string;
   /** Transactions on the other chain that paid the app's other transfers, which aren't among `transfers`. */
   taken?: Iterable<string>;
-  /** That chain's block times the app keeps. */
-  blockTimes?: BlockTimes;
+  /**
+   * The block times the app keeps for each chain, by its chain ID: or one
+   * cache, for a network whose service routes are all on one chain (it
+   * isn't used for any other).
+   */
+  blockTimes?: BlockTimes | ((chainId: number) => BlockTimes | undefined);
   now?: number;
   waitMs?: number;
   maxWindows?: number;