@bsv/wallet-toolbox 1.1.0

package/docs/monitor.md

@@ -0,0 +1,648 @@
+# API
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+## Interfaces
+
+| |
+| --- |
+| [MonitorDaemonSetup](#interface-monitordaemonsetup) |
+| [MonitorOptions](#interface-monitoroptions) |
+| [TaskPurgeParams](#interface-taskpurgeparams) |
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+
+### Interface: MonitorDaemonSetup
+
+```ts
+export interface MonitorDaemonSetup {
+    chain?: sdk.Chain;
+    sqliteFilename?: string;
+    mySQLConnection?: string;
+    knexConfig?: Knex.Config;
+    knex?: Knex<any, any[]>;
+    storageKnexOptions?: StorageKnexOptions;
+    storageProvider?: StorageProvider;
+    storageManager?: WalletStorageManager;
+    servicesOptions?: sdk.WalletServicesOptions;
+    services?: Services;
+    monitor?: Monitor;
+}
+```
+
+See also: [Chain](#type-chain), [Monitor](#class-monitor), [Services](#class-services), [StorageKnexOptions](#interface-storageknexoptions), [StorageProvider](#class-storageprovider), [WalletServicesOptions](#interface-walletservicesoptions), [WalletStorageManager](#class-walletstoragemanager)
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Interface: MonitorOptions
+
+```ts
+export interface MonitorOptions {
+    chain: sdk.Chain;
+    services: Services;
+    storage: MonitorStorage;
+    chaintracks: ChaintracksServiceClient;
+    msecsWaitPerMerkleProofServiceReq: number;
+    taskRunWaitMsecs: number;
+    abandonedMsecs: number;
+    unprovenAttemptsLimitTest: number;
+    unprovenAttemptsLimitMain: number;
+}
+```
+
+See also: [Chain](#type-chain), [MonitorStorage](#type-monitorstorage), [Services](#class-services)
+
+<details>
+
+<summary>Interface MonitorOptions Details</summary>
+
+#### Property msecsWaitPerMerkleProofServiceReq
+
+How many msecs to wait after each getMerkleProof service request.
+
+```ts
+msecsWaitPerMerkleProofServiceReq: number
+```
+
+</details>
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Interface: TaskPurgeParams
+
+The database stores a variety of data that may be considered transient.
+
+At one extreme, the data that must be preserved:
+  - unspent outputs (UTXOs)
+  - in-use metadata (labels, baskets, tags...)
+
+At the other extreme, everything can be preserved to fully log all transaction creation and processing actions.
+
+The following purge actions are available to support sustained operation:
+  - Failed transactions, delete all associated data including:
+      + Delete tag and label mapping records
+      + Delete output records
+      + Delete transaction records
+      + Delete mapi_responses records
+      + Delete proven_tx_reqs records
+      + Delete commissions records
+      + Update output records marked spentBy failed transactions
+  - Completed transactions, delete transient data including:
+      + transactions table set truncatedExternalInputs = null
+      + transactions table set beef = null
+      + transactions table set rawTx = null
+      + Delete mapi_responses records
+      + proven_tx_reqs table delete records
+
+```ts
+export interface TaskPurgeParams extends sdk.PurgeParams {
+}
+```
+
+See also: [PurgeParams](#interface-purgeparams)
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+## Classes
+
+| |
+| --- |
+| [Monitor](#class-monitor) |
+| [MonitorDaemon](#class-monitordaemon) |
+| [TaskCheckForProofs](#class-taskcheckforproofs) |
+| [TaskClock](#class-taskclock) |
+| [TaskFailAbandoned](#class-taskfailabandoned) |
+| [TaskNewHeader](#class-tasknewheader) |
+| [TaskPurge](#class-taskpurge) |
+| [TaskReviewStatus](#class-taskreviewstatus) |
+| [TaskSendWaiting](#class-tasksendwaiting) |
+| [TaskSyncWhenIdle](#class-tasksyncwhenidle) |
+| [WalletMonitorTask](#class-walletmonitortask) |
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+
+### Class: Monitor
+
+Background task to make sure transactions are processed, transaction proofs are received and propagated,
+and potentially that reorgs update proofs that were already received.
+
+```ts
+export class Monitor {
+    static createDefaultWalletMonitorOptions(chain: sdk.Chain, storage: MonitorStorage, services?: Services): MonitorOptions 
+    options: MonitorOptions;
+    services: Services;
+    chain: sdk.Chain;
+    storage: MonitorStorage;
+    chaintracks: ChaintracksServiceClient;
+    constructor(options: MonitorOptions) 
+    oneSecond = 1000;
+    oneMinute = 60 * this.oneSecond;
+    oneHour = 60 * this.oneMinute;
+    oneDay = 24 * this.oneHour;
+    oneWeek = 7 * this.oneDay;
+    _tasks: WalletMonitorTask[] = [];
+    _otherTasks: WalletMonitorTask[] = [];
+    _tasksRunning = false;
+    defaultPurgeParams: TaskPurgeParams = {
+        purgeSpent: false,
+        purgeCompleted: false,
+        purgeFailed: true,
+        purgeSpentAge: 2 * this.oneWeek,
+        purgeCompletedAge: 2 * this.oneWeek,
+        purgeFailedAge: 5 * this.oneDay
+    };
+    addAllTasksToOther(): void 
+    addDefaultTasks(): void 
+    addMultiUserTasks(): void 
+    addTask(task: WalletMonitorTask): void 
+    removeTask(name: string): void 
+    async setupChaintracksListeners(): Promise<void> 
+    async runTask(name: string): Promise<string> 
+    async runOnce(): Promise<void> 
+    _runAsyncSetup: boolean = true;
+    async startTasks(): Promise<void> 
+    async logEvent(event: string, details?: string): Promise<void> 
+    stopTasks(): void 
+    lastNewHeader: BlockHeader | undefined;
+    lastNewHeaderWhen: Date | undefined;
+    processNewBlockHeader(header: BlockHeader): void 
+    processReorg(depth: number, oldTip: BlockHeader, newTip: BlockHeader): void 
+}
+```
+
+See also: [BlockHeader](#interface-blockheader), [Chain](#type-chain), [MonitorOptions](#interface-monitoroptions), [MonitorStorage](#type-monitorstorage), [Services](#class-services), [TaskPurgeParams](#interface-taskpurgeparams), [WalletMonitorTask](#class-walletmonitortask)
+
+<details>
+
+<summary>Class Monitor Details</summary>
+
+#### Property _otherTasks
+
+_otherTasks can be run by runTask but not by scheduler.
+
+```ts
+_otherTasks: WalletMonitorTask[] = []
+```
+See also: [WalletMonitorTask](#class-walletmonitortask)
+
+#### Property _tasks
+
+_tasks are typically run by the scheduler but may also be run by runTask.
+
+```ts
+_tasks: WalletMonitorTask[] = []
+```
+See also: [WalletMonitorTask](#class-walletmonitortask)
+
+#### Method addDefaultTasks
+
+Default tasks with settings appropriate for a single user storage
+possibly with sync'ing enabled
+
+```ts
+addDefaultTasks(): void 
+```
+
+#### Method addMultiUserTasks
+
+Tasks appropriate for multi-user storage
+without sync'ing enabled.
+
+```ts
+addMultiUserTasks(): void 
+```
+
+#### Method processNewBlockHeader
+
+Process new chain header event received from Chaintracks
+
+Kicks processing 'unconfirmed' and 'unmined' request processing.
+
+```ts
+processNewBlockHeader(header: BlockHeader): void 
+```
+See also: [BlockHeader](#interface-blockheader)
+
+#### Method processReorg
+
+Process reorg event received from Chaintracks
+
+Reorgs can move recent transactions to new blocks at new index positions.
+Affected transaction proofs become invalid and must be updated.
+
+It is possible for a transaction to become invalid.
+
+Coinbase transactions always become invalid.
+
+```ts
+processReorg(depth: number, oldTip: BlockHeader, newTip: BlockHeader): void 
+```
+See also: [BlockHeader](#interface-blockheader)
+
+</details>
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: MonitorDaemon
+
+```ts
+export class MonitorDaemon {
+    setup?: MonitorDaemonSetup;
+    doneListening?: Promise<void>;
+    doneTasks?: Promise<void>;
+    stopDaemon: boolean = false;
+    constructor(public args: MonitorDaemonSetup, public noRunTasks?: boolean) 
+    async createSetup(): Promise<void> 
+    async start(): Promise<void> 
+    async stop(): Promise<void> 
+    async destroy(): Promise<void> 
+    async runDaemon(): Promise<void> 
+}
+```
+
+See also: [MonitorDaemonSetup](#interface-monitordaemonsetup)
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: TaskCheckForProofs
+
+`TaskCheckForProofs` is a WalletMonitor task that retreives merkle proofs for
+transactions.
+
+It is normally triggered by the Chaintracks new block header event.
+
+When a new block is found, cwi-external-services are used to obtain proofs for
+any transactions that are currently in the 'unmined' or 'unknown' state.
+
+If a proof is obtained and validated, a new ProvenTx record is created and
+the original ProvenTxReq status is advanced to 'notifying'.
+
+```ts
+export class TaskCheckForProofs extends WalletMonitorTask {
+    static taskName = "CheckForProofs";
+    static checkNow = false;
+    constructor(monitor: Monitor, public triggerMsecs = 0) 
+    trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    } 
+    async runTask(): Promise<string> 
+    async getProofs(reqs: table.ProvenTxReq[], indent = 0, countsAsAttempt = false, ignoreStatus = false): Promise<{
+        proven: table.ProvenTxReq[];
+        invalid: table.ProvenTxReq[];
+        log: string;
+    }> 
+}
+```
+
+See also: [Monitor](#class-monitor), [WalletMonitorTask](#class-walletmonitortask), [log](#variable-log)
+
+<details>
+
+<summary>Class TaskCheckForProofs Details</summary>
+
+#### Property checkNow
+
+An external service such as the chaintracks new block header
+listener can set this true to cause
+
+```ts
+static checkNow = false
+```
+
+#### Method getProofs
+
+Process an array of table.ProvenTxReq (typically with status 'unmined' or 'unknown')
+
+If req is invalid, set status 'invalid'
+
+Verify the requests are valid, lookup proofs or updated transaction status using the array of getProofServices,
+
+When proofs are found, create new ProvenTxApi records and transition the requests' status to 'unconfirmed' or 'notifying',
+depending on chaintracks succeeding on proof verification.
+
+Increments attempts if proofs where requested.
+
+```ts
+async getProofs(reqs: table.ProvenTxReq[], indent = 0, countsAsAttempt = false, ignoreStatus = false): Promise<{
+    proven: table.ProvenTxReq[];
+    invalid: table.ProvenTxReq[];
+    log: string;
+}> 
+```
+See also: [log](#variable-log)
+
+Returns
+
+reqs partitioned by status
+
+#### Method trigger
+
+Normally triggered by checkNow getting set by new block header found event from chaintracks
+
+```ts
+trigger(nowMsecsSinceEpoch: number): {
+    run: boolean;
+} 
+```
+
+</details>
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: TaskClock
+
+```ts
+export class TaskClock extends WalletMonitorTask {
+    static taskName = "Clock";
+    nextMinute: number;
+    constructor(monitor: Monitor, public triggerMsecs = 1 * monitor.oneSecond) 
+    trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    } 
+    async runTask(): Promise<string> 
+    getNextMinute(): number 
+}
+```
+
+See also: [Monitor](#class-monitor), [WalletMonitorTask](#class-walletmonitortask)
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: TaskFailAbandoned
+
+Handles transactions which do not have terminal status and have not been
+updated for an extended time period.
+
+Calls `updateTransactionStatus` to set `status` to `failed`.
+This returns inputs to spendable status and verifies that any
+outputs are not spendable.
+
+```ts
+export class TaskFailAbandoned extends WalletMonitorTask {
+    static taskName = "FailAbandoned";
+    constructor(monitor: Monitor, public triggerMsecs = 1000 * 60 * 5) 
+    trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    } 
+    async runTask(): Promise<string> 
+}
+```
+
+See also: [Monitor](#class-monitor), [WalletMonitorTask](#class-walletmonitortask)
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: TaskNewHeader
+
+```ts
+export class TaskNewHeader extends WalletMonitorTask {
+    static taskName = "NewHeader";
+    header?: sdk.BlockHeader;
+    constructor(monitor: Monitor, public triggerMsecs = 1 * monitor.oneMinute) 
+    async getHeader(): Promise<sdk.BlockHeader> 
+    trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    } 
+    async runTask(): Promise<string> 
+}
+```
+
+See also: [BlockHeader](#interface-blockheader), [Monitor](#class-monitor), [WalletMonitorTask](#class-walletmonitortask)
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: TaskPurge
+
+```ts
+export class TaskPurge extends WalletMonitorTask {
+    static taskName = "Purge";
+    static checkNow = false;
+    constructor(monitor: Monitor, public params: TaskPurgeParams, public triggerMsecs = 0) 
+    trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    } 
+    async runTask(): Promise<string> 
+}
+```
+
+See also: [Monitor](#class-monitor), [TaskPurgeParams](#interface-taskpurgeparams), [WalletMonitorTask](#class-walletmonitortask)
+
+<details>
+
+<summary>Class TaskPurge Details</summary>
+
+#### Property checkNow
+
+Set to true to trigger running this task
+
+```ts
+static checkNow = false
+```
+
+</details>
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: TaskReviewStatus
+
+Notify Transaction records of changes in ProvenTxReq records they may have missed.
+
+The `notified` property flags reqs that do not need to be checked.
+
+Looks for aged Transactions with provenTxId with status != 'completed', sets status to 'completed'.
+
+Looks for reqs with 'invalid' status that
+
+```ts
+export class TaskReviewStatus extends WalletMonitorTask {
+    static taskName = "ReviewStatus";
+    static checkNow = false;
+    constructor(monitor: Monitor, public triggerMsecs = 1000 * 60 * 15, public agedMsecs = 1000 * 60 * 5) 
+    trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    } 
+    async runTask(): Promise<string> 
+}
+```
+
+See also: [Monitor](#class-monitor), [WalletMonitorTask](#class-walletmonitortask)
+
+<details>
+
+<summary>Class TaskReviewStatus Details</summary>
+
+#### Property checkNow
+
+Set to true to trigger running this task
+
+```ts
+static checkNow = false
+```
+
+</details>
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: TaskSendWaiting
+
+```ts
+export class TaskSendWaiting extends WalletMonitorTask {
+    static taskName = "SendWaiting";
+    constructor(monitor: Monitor, public triggerMsecs = 1000 * 60 * 5, public agedMsecs = 0) 
+    trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    } 
+    async runTask(): Promise<string> 
+    async processUnsent(reqApis: table.ProvenTxReq[], indent = 0): Promise<string> 
+}
+```
+
+See also: [Monitor](#class-monitor), [WalletMonitorTask](#class-walletmonitortask)
+
+<details>
+
+<summary>Class TaskSendWaiting Details</summary>
+
+#### Method processUnsent
+
+Process an array of 'unsent' status table.ProvenTxReq
+
+Send rawTx to transaction processor(s), requesting proof callbacks when possible.
+
+Set status 'invalid' if req is invalid.
+
+Set status to 'callback' on successful network submission with callback service.
+
+Set status to 'unmined' on successful network submission without callback service.
+
+Add mapi responses to database table if received.
+
+Increments attempts if sending was attempted.
+
+```ts
+async processUnsent(reqApis: table.ProvenTxReq[], indent = 0): Promise<string> 
+```
+
+</details>
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: TaskSyncWhenIdle
+
+```ts
+export class TaskSyncWhenIdle extends WalletMonitorTask {
+    static taskName = "SyncWhenIdle";
+    constructor(monitor: Monitor, public triggerMsecs = 1000 * 60 * 1) 
+    trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    } 
+    async runTask(): Promise<string> 
+}
+```
+
+See also: [Monitor](#class-monitor), [WalletMonitorTask](#class-walletmonitortask)
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+### Class: WalletMonitorTask
+
+A monitor task performs some periodic or state triggered maintenance function
+on the data managed by a wallet (Bitcoin UTXO manager, aka wallet)
+
+The monitor maintains a collection of tasks.
+
+It runs each task's non-asynchronous trigger to determine if the runTask method needs to run.
+
+Tasks that need to be run are run consecutively by awaiting their async runTask override method.
+
+The monitor then waits a fixed interval before repeating...
+
+Tasks may use the monitor_events table to persist their execution history.
+This is done by accessing the wathman.storage object.
+
+```ts
+export abstract class WalletMonitorTask {
+    lastRunMsecsSinceEpoch = 0;
+    storage: MonitorStorage;
+    constructor(public monitor: Monitor, public name: string) 
+    async asyncSetup(): Promise<void> 
+    abstract trigger(nowMsecsSinceEpoch: number): {
+        run: boolean;
+    };
+    abstract runTask(): Promise<string>;
+}
+```
+
+See also: [Monitor](#class-monitor), [MonitorStorage](#type-monitorstorage)
+
+<details>
+
+<summary>Class WalletMonitorTask Details</summary>
+
+#### Property lastRunMsecsSinceEpoch
+
+Set by monitor each time runTask completes
+
+```ts
+lastRunMsecsSinceEpoch = 0
+```
+
+#### Method asyncSetup
+
+Override to handle async task setup configuration.
+
+Called before first call to `trigger`
+
+```ts
+async asyncSetup(): Promise<void> 
+```
+
+#### Method trigger
+
+Return true if `runTask` needs to be called now.
+
+```ts
+abstract trigger(nowMsecsSinceEpoch: number): {
+    run: boolean;
+}
+```
+
+</details>
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+## Functions
+
+## Types
+
+### Type: MonitorStorage
+
+```ts
+export type MonitorStorage = WalletStorageManager
+```
+
+See also: [WalletStorageManager](#class-walletstoragemanager)
+
+Links: [API](#api), [Interfaces](#interfaces), [Classes](#classes), [Functions](#functions), [Types](#types), [Variables](#variables)
+
+---
+## Variables
+