@bsv/wallet-toolbox 2.12.0 → 2.13.2

package/CHANGELOG.md

@@ -6,6 +6,298 @@ attention to changes that materially alter behavior or extend functionality.
 
 ## wallet-toolbox (unreleased)
 
+- Keep cold raw-transaction reads on the caller's transaction, including SQLite
+  with one connection; do not cache uncommitted settings or start background
+  work. Accept missing optional inputBEEF when returning stored raw bytes.
+- Reject unresolved output-basket mappings before sync writes. Apply valid newer
+  basket assignments/removals, preserving equal/older local relinquishment and
+  transaction rollback/retry. Included in unpublished 2.13.2; no schema migration.
+- Restore transactional SQLite migrations in the unpublished 2.13.2 candidate.
+  DDL, migration journal and lock changes roll back after an interrupted attempt;
+  foreign-key enforcement is restored after success or failure. Existing stores
+  with unjournaled partial schema need operator-reviewed recovery; this change
+  does not delete or automatically reconcile historical wallet data.
+
+- Implement `BHServiceClient.findChainTipHash()` by delegating to its existing
+  `findChainTipHeader()` call against `/api/v1/chain/tip/longest`, instead of
+  throwing `Not implemented`. `ChaintracksChainTracker.getVerificationContextToken()`
+  calls `findChainTipHash()` on every verification attempt, so any wallet
+  configured with a `BHServiceClient` as `options.chaintracks`, or as a
+  `LocalChainTracker` participating source, previously failed on every
+  attempt. No `ChaintracksClientApi` contract change; no migration required.
+
+- Preserve valid compound proofs with multiple marked transactions. Rotate
+  unresolved proof repairs behind waiting heights across monitor restarts, and
+  retain retries that become temporarily ineligible after the chain tip
+  retreats. Existing checkpoints remain compatible; no migration is required.
+
+- Validate every newly acquired and replacement Merkle proof against the active
+  ChainTracks root before persistence, continue to later providers when an
+  earlier provider returns an orphan proof, and retain unresolved reorg heights
+  for bounded per-run retries without stopping the forward review cursor. No
+  schema or consumer migration is required.
+
+- Require mutually authenticated responses from remote wallet storage, pin the
+  authenticated server identity for each client instance, optionally accept an
+  independently validated `serverIdentityKey`, and bind the distinct advertised
+  storage identity to that channel, with an optional independent
+  `storageIdentityKey` pin. Strictly correlate JSON-RPC version and request ID,
+  require one unambiguous result or error, keep remote response details out of
+  transport diagnostics, and preserve an explicit null result for void server
+  methods.
+
+- Complete WAB faucet funding through one exact wallet-owned BRC-29 output.
+  Fee-adjusted output amounts are bounded without weakening exact script
+  binding, real Wallet Toolbox symbol metadata remains accessor-safe, and the
+  redemption is staged and labeled before broadcast. Interrupted retries
+  recover the same transaction from its dedicated basket or recognize the
+  already-internalized action instead of signing the faucet input again.
+
+- Make ChainTracks lifecycle and event delivery fail closed. Startup errors are
+  awaited and retryable, failed or cold instances can be fully destroyed,
+  long-running source promises are drained, subscriptions are bounded and
+  deleted rather than tombstoned, and every listener receives an isolated
+  header snapshot. Provider heights, bulk results, type guards, direct source
+  options, and diagnostics are now bounded and accessor-safe. WhatsOnChain
+  options and shared chain-info results cannot be mutated after validation.
+  Filesystem cache mutation is serialized per digest across processes, caller
+  metadata/bytes are snapshotted before awaits, and bulk revalidation takes the
+  writer lock. Bulk-file reconciliation and multi-record replacement are atomic
+  in Knex and IndexedDB, ordinary mutations reach durable storage before memory,
+  failed writes restore the prior manager state, and remote manifests cannot
+  supply local storage identities. Custom storage implementations must add the
+  optional atomic replacement method before performing multi-file changes.
+  Remote clients require complete live metadata and bounded event topology,
+  subscription state, timers, and single-line diagnostics. Destructive Knex
+  rollback failures now propagate. The wallet monitor independently binds its
+  event source to the configured network, authenticates and copies every
+  reorganization header, bounds and deduplicates deactivation retries,
+  coalesces prepared-proof invalidation, cleans up partial subscriptions, and
+  contains application callback failures without implicit console output.
+
+- Make Arcade transaction-status SSE a bounded, ordered durable stream. Client
+  options are snapshotted without accessors; credentials, cursors, payloads,
+  fields, queued count, and queued bytes have explicit ceilings; and event data
+  is validated and copied. Only one event reaches storage at a time, its cursor
+  advances only after processing and persistence succeed, and a malformed,
+  excessive, or failed event closes the stream without dispatching later queued
+  events. Logging is silent by default and transport errors never forward the
+  credential-bearing EventSource object. Monitor task setup retains one pending
+  event, retries storage/checkpoint failures, and closes on removal or teardown.
+
+- Repair monitor and daemon lifecycle ordering. Every scheduled or standalone
+  run now awaits retryable ChainTracks subscriptions, standalone runs initialize
+  every task, transient setup failures retry, and loop failures reset running
+  state. The daemon starts subscriptions before ChainTracks and tasks, can stop
+  after a successful start, and destroys monitor resources before storage even
+  when cleanup fails. Monitor event/task names and diagnostic text are bounded;
+  library logging is opt-in and dotenv loading is quiet.
+
+- Authenticate every ordinary Merkle-proof result before it can create a
+  `ProvenTx` or restore a failed transaction. Provider responses are copied and
+  structurally bounded, must contain a leaf matching the requested txid
+  (independently of the optional `txid` marker),
+  must compute the canonical header root at the same height, and must resolve
+  through a proof-of-work-valid wallet header. Monitor consumers additionally
+  require the local ChainTracks root verdict and bind the request's raw
+  transaction to its txid. WhatsOnChain, Bitails, and Arcade enforce the same
+  invariant when called directly; TSC conversion rejects malformed, sparse,
+  oversized, and unsafe-integer proofs. `TaskUnFail` uses a one-shot internal
+  authorization guard and re-reads its first mutating page so batches over 100
+  cannot be skipped. Existing wire formats and public method signatures remain
+  compatible; the optional root-validator argument is additive.
+
+- Authenticate external transaction-status results before they can influence
+  wallet state. Results must be uniquely bound to requested txids and have
+  consistent mined/known/unknown depth, terminal, and input-conflict fields;
+  provider attribution is local, while descriptions and competing txids are
+  copied and bounded. Malformed providers fail over instead of supplying
+  failure, quarantine, abort-protection, or double-spend evidence. Arcade also
+  validates and binds its complete transaction-data response before status
+  classification or proof parsing, and WhatsOnChain rejects impossible depth.
+  Monitor proof, no-send, and abandonment scans now preserve stable pagination
+  while rows leave their query, and a rejected proof counts as one attempt.
+
+- Bound raw-transaction and proof-provider envelopes before inspection.
+  Raw results are txid-bound, accessor-free owned byte arrays capped at 32 MiB;
+  malformed providers fail over and cannot forge local attribution or mutate
+  accepted bytes. Merkle provider notes are copied as bounded scalar data and
+  legacy custom services may supply at most eight dense proof candidates.
+  Monitor options, task work controls, persisted checkpoints, and polled block
+  headers are also validated and copied before they control background work.
+
+- Contain UTXO-oracle evidence and invalid-change review work. Provider results
+  are accessor-free, bounded, outpoint-consistent owned data with locally
+  configured attribution; malformed responses fail over and never become spent
+  evidence. Manual and admin review require canonical identities and exact
+  modes, booleans, pages, offsets, tags, and safe monetary totals. Whole-wallet
+  compatibility scans have a 10,000-candidate ceiling, and a release rebinds the
+  exact classified outpoint, value, basket, ownership, and allocation state
+  under the write lock before changing spendability.
+
+- Authenticate and bound script-history and transaction-broadcast provider
+  boundaries. History is hash/endian-bound, accessor-free, deduplicated, and
+  capped; all WhatsOnChain calls carry validated whole-request deadlines and
+  chain responses are copied before use. BEEF and direct raw submissions are
+  size- and txid-bound, providers receive isolated snapshots, result identities
+  and state invariants are enforced with local attribution, and sensitive or
+  excessive diagnostics are discarded. Malformed, throwing, or hung providers
+  fail over without forging the wallet's internal timeout marker. A broadcast
+  double-spend claim is terminal only after at least one exact input is
+  conclusively observed spent; unknown oracle evidence remains retryable.
+
+- Enforce one atomic `sendWith` resource invariant from SDK validation and
+  binary decoding through storage, action-batch commit, provider submission,
+  and delayed monitor retry. Broadcast sets contain at most 1,000 unique,
+  canonical transaction IDs, including the newly created or signed
+  transaction. Malformed storage flags cannot diverge from the actual list,
+  and an oversized legacy batch is reported and skipped as a whole rather than
+  partially broadcast or expanded without bound. If any member cannot be
+  resolved or assembled, no ready subset is scheduled or broadcast; members
+  already known by the network remain compatible with retrying the rest.
+
+- Make user provisioning and schema lifecycle failure-atomic. A newly inserted
+  user and its mandatory managed-change basket now share one transaction, and
+  creation-race retry restarts the whole transaction. Knex migration and
+  destructive rollback errors propagate, explicit empty-schema state is the
+  only rollback terminator, and SQLite foreign-key enforcement is restored in
+  `finally` after both successful and failed migration work.
+
+- Keep secrets and untrusted wallet material out of implicit process logs.
+  `Setup.makeEnv()` returns generated development keys without printing them;
+  invalid BEEF, identity/certificate parsing, permission batching, provisioning
+  races, migration failures, and IndexedDB orphan handling no longer emit raw
+  objects, transaction/proof data, database details, or row identifiers. The
+  existing explicit wallet logger receives only a generic BEEF-validation
+  message when a caller opts into it.
+
+- Preserve wallet write atomicity across metadata revival, entity insertion,
+  and abort-time chain checks. Soft-deleted baskets, labels, label mappings,
+  output tags, and tag mappings now revive in the caller's exact transaction;
+  new proof-request and sync-state entities retain that transaction as well.
+  `abortAction` queries external chain status before opening an IndexedDB write
+  scope, then rebinds the observation to the exact transaction ID and txid
+  under the write lock before committing its decision.
+
+- Keep shared-storage schema and process lifecycle under operator authority.
+  Authenticated remote migration requests are compatibility no-ops, matching
+  remote destroy requests, so a wallet tenant cannot contend on global DDL or
+  initialize shared storage metadata.
+
+- Bound authenticated proof-request reads before database and memory work.
+  Knex and IndexedDB apply wallet ownership in the same query/cursor scan as
+  status, partial, offset, and limit filtering. Custom providers retain a
+  bounded compatibility path and must supply exact txids or a provider-native
+  scoped query for wallets with more than 10,000 proof requests.
+
+- Bound caller-selected Wallet Storage RPC offsets as well as page sizes.
+  Top-level list offsets, nested find paging, and synchronization checkpoints
+  are rejected before database/cursor work when they exceed the selected
+  resource profile. Operators can raise the explicit ceiling for larger
+  histories without changing the wire format.
+
+- Redact internal Wallet Storage failures at every authenticated HTTP response
+  boundary. JSON-RPC and action-batch upload routes preserve explicit public
+  wallet errors, but generic exceptions and custom `WERR_INTERNAL` diagnostics
+  now return only the stable internal-error name and default message. Detailed
+  provider, database, and filesystem failures remain available through
+  explicitly configured operator logging and telemetry.
+
+- Clarify local-contact authority throughout SDK and Wallet Toolbox identity
+  documentation. A saved contact is an authoritative wallet-local trust anchor
+  based on the user's independent validation, analogous to accepting a
+  self-signed certificate. Its infinite trust is intentionally local—not a
+  third-party certification or transferable trust claim—and authenticated
+  contact storage proves retention of that decision rather than the underlying
+  real-world identity.
+
+- Re-bind cryptographically authenticated identity-overlay certificates to the
+  exact lookup they answer before applying trust thresholds. A dishonest lookup
+  host can no longer answer an identity-key or public-attribute query with a
+  different valid trusted certificate. Local contacts remain authoritative;
+  contact lookup matching continues to follow the caller-installed source's
+  explicit local trust contract.
+
+- Harden local ChainTracks persistence and legacy bulk tooling. Memory-backed
+  trackers no longer share state, IndexedDB and Knex require unique active
+  headers and consecutive parent linkage, reorganization walks are bounded and
+  fail atomically, direct admissions are authenticated, and Knex serializes tip
+  mutation through an internal state row. MySQL now stores 64-character hex
+  values as `VARCHAR(64)` and multi-megabyte header objects as `LONGBLOB`; the
+  additive migration clears the rebuildable legacy live-header cache before
+  widening fields that permissive MySQL could already have truncated. Bulk
+  manifests, local readers, exporters, filesystem operations, and lock queues
+  now have canonical path, byte, count, continuity, genesis, digest,
+  proof-of-work, and progress bounds. Filesystem cache reads are bounded before
+  allocation and authenticate writes, while the durable download budget now
+  serializes reservations across processes and fails closed on an abandoned
+  lock instead of silently resetting or losing allowance.
+
+- Fix the legacy `BHServiceClient` Merkle-root cache so rejected, stale, or
+  externally written roots can never become authoritative. Every verdict now
+  resolves a bounded canonical header, validates its computed hash and proof
+  of work, and binds it to the requested height or hash. Its HTTP reads reject
+  redirects and malformed endpoints, retain whole-body deadlines, and enforce
+  response and batch ceilings. ChainTracks core options are validated before
+  startup, and all library/factory logging is now silent unless the embedding
+  application supplies the existing optional logger.
+
+- Treat every remote ChainTracks HTTP/SSE and WhatsOnChain HTTP/CDN/WebSocket
+  response as untrusted input: retain the
+  deadline through bounded body consumption, bound and idle-timeout streams,
+  prohibit redirects and ambiguous endpoint URLs, validate canonical header
+  bytes and proof of work, and bind network, height, hash, and reorganization
+  results to each request before returning or queueing them. Submitted and live
+  header queues are copy-isolated, deduplicated where applicable, and bounded.
+  Service-discovered CDN links must remain credential-free public HTTPS with
+  DNS-pinned Node connections, legacy history streams are range-ordered and
+  chunked, and no declared target contributes to bulk chain selection before
+  its complete header and proof of work have been authenticated.
+
+- Authenticate every discovered permission token against its exact BEEF
+  outpoint, canonical signed PushDrop fields, locally derived locking key, and
+  signed grant content. Basket membership and tags remain selectors only. Bind
+  public partial actions and all token mint, renewal, coalescing, and revocation
+  flows to the exact authorized final transaction and actual input indexes.
+  Include signer-authenticated storage service charges in the exact spend sent
+  for permission approval, without changing the BRC-100 API or wire format, and
+  fail closed on inconsistent sources or unsafe monetary arithmetic. The exact
+  packed platform measurements are 2,184,519 / 520,753 / 397,289 bytes for Vite
+  and 1,704,383 / 471,571 / 370,439 for esbuild (raw/gzip/Brotli), with reviewed
+  ceilings of 2,230,000 / 535,000 / 410,000 and
+  1,740,000 / 485,000 / 380,000 respectively. Mobile measures
+  2,344,815 / 597,625 / 454,020 for Metro and
+  4,579,110 / 1,919,816 / 1,489,229 for optimized Hermes bytecode, with reviewed
+  ceilings of 2,400,000 / 610,000 / 465,000 and
+  4,675,000 / 1,960,000 / 1,520,000 respectively.
+
+- Bind UMP renewal to an exact canonical, field-signed predecessor owned by the
+  local wallet and require one exact replacement token in the completed action.
+  Bind WAB faucet funding to its exact Atomic BEEF target, output-zero R-puzzle
+  commitment, actual wallet input index, and returned final transaction ID.
+
+- Bind legacy P2PKH funding imports to the exact atomic signing target and
+  final wallet transaction. Reject result-only transaction IDs, ambiguous
+  outpoints, oversized or redirected provider responses, and unbounded BEEF
+  recursion/source graphs before signing or allocation.
+
+- Atomically consume BRC-103 signed-message nonces across StorageServer replicas.
+  The additive `2026-09-16-001 add auth message replay claims` migration creates
+  `auth_message_nonces` with a composite session-or-identity scope/nonce
+  uniqueness constraint and expiry index. Signed follow-up and unsigned
+  initial-request replays are both rejected. Run the migration before deploying
+  the new runtime; otherwise authentication processing fails closed.
+
+- Integrate upstream security corrections without dropping the sync recovery contracts.
+  Record combined browser/mobile artifact costs and limits in the sync transfer guide.
+
+- Validate replacement proofs across providers before reconciling stale RPC sync metadata; preserve raw transactions and checkpoints on failure.
+
+- Start sync copies conservatively and adapt page work after committed responses.
+  Validate at most eight remote proofs concurrently, draining failure work before
+  rejecting the page. Preserve all proof and checkpoint checks. Record-transfer
+  candidates now use 2.13.0 to distinguish them from published 2.12.0.
+
 - Require an affirmative certifier-signature verification result before
   storing directly acquired or issuer-returned certificates. Identity overlay
   results are verified before decryption and trust scoring, so forged
@@ -30,6 +322,44 @@ attention to changes that materially alter behavior or extend functionality.
   a clean missing-token result only for a server-declared pending registration,
   finalizes after successful publication, repairs a lost acknowledgement on the
   next login, and preserves fail-closed continuity for active and legacy WABs.
+- Add optional bounded sync transfers for oversized records, authenticated pieces,
+  durable Knex staging, full-frame SHA-256 verification, and stale-checkpoint
+  rejection before merging. Add the two staging tables through migration
+  `2026-09-09-001`; reverse only that migration before restarting older code. Frames are limited
+  to 64 MiB and ordinary pages keep their existing transport. See
+  [the transport and migration guide](./docs/sync-transfer.md).
+
+- Upgrade IndexedDB to schema version 6 with a non-unique transaction-ID/user
+  index. Use existing reference, reclaim, commission, and relation keys for
+  sync lookups, avoiding repeated scans as a local wallet grows. Preserve
+  user filtering, pagination, existing bytes, and duplicate transaction IDs.
+  Proof-batch preflight checks resolve only requested transaction IDs through
+  the existing index, preserving proof validation and primary-key ordering.
+  Older clients requesting schema version 5 cannot open the upgraded database.
+
+- Add negotiated compact sync checkpoints, reusing committed page progress while
+  retaining complete writer-side ID mappings and the legacy provider fallback.
+  Avoid per-scalar reviver callbacks when decoding binary JSON pages without
+  changing marker escaping or byte semantics. Use the already negotiated binary
+  codec for large schema-defined sync response byte arrays, reducing wire and
+  authentication payload size while preserving legacy arrays and unrelated data.
+  Retain authenticated full-copy,
+  resume, malformed-input, and transport CPU/payload regression coverage.
+  Local packed measurements are 1,701,711 raw / 401,038 gzip / 314,272 Brotli
+  bytes (Vite), 1,327,368 / 364,550 / 293,224 (esbuild),
+  1,753,648 / 445,136 / 345,369 (Metro), and
+  3,557,663 / 1,426,740 / 1,121,821 (Hermes). The raw ceilings become
+  1,702,000, 1,327,500, 1,755,000, and 3,558,500 respectively.
+  Vite and esbuild gzip ceilings increase to 402,000 and 366,000 bytes.
+  Hosted Linux measures 365,818 esbuild gzip bytes and 1,445,624 Hermes gzip
+  bytes; the Hermes gzip ceiling becomes 1,446,500.
+  The esbuild Brotli ceiling is 293,750 bytes. Asynchronous platform hashing
+  for large SDK signature payloads brings the Vite Brotli ceiling to 314,500
+  bytes, the Hermes raw ceiling to 3,560,000 bytes, and its Brotli ceiling to
+  1,125,000 bytes (measured 1,124,446); other compressed ceilings remain unchanged.
+  The added portable sync/progress and decoding paths account for
+  the increase; no Node-only backend is introduced into portable bundles.
+
 - Keep Argon2id-backed UMP v3 wallets available in React Native and other
   runtimes without WebAssembly by falling back to an asynchronously yielding,
   standards-compatible JavaScript implementation. The same KDF parameters and
@@ -136,6 +466,25 @@ attention to changes that materially alter behavior or extend functionality.
   transaction startup/commit cycles, failed pages roll back without advancing
   the checkpoint, and abort cleanup preserves the original storage error.
 
+- Fill wallet-storage sync pages with adaptive, size-aware source queries and
+  add composite SQL indexes for user-scoped proof lookups. Sync clients may
+  request optional source record totals for exact progress and ETA displays;
+  older clients and providers remain wire-compatible and do not incur count
+  queries unless totals are requested. Remote clients also recover from a
+  provider's HTTP 413 response ceiling by retrying the read-only sync request
+  with a smaller chunk budget and reusing the working limit for later pages.
+  Runtime validation rejects malformed remote totals, and MySQL rollback
+  restores the foreign-key support index before removing the new composites.
+  The retained authenticated candidate-provider benchmark fills a 250-record
+  proof page with three source reads (`10, 80, 160`) on SQLite and MySQL.
+  Clean macOS fixtures measure Vite at 1,609,916 raw, 379,552 gzip, and 297,188
+  Brotli bytes; esbuild at 1,254,827 raw, 344,883 gzip, and 277,729 Brotli
+  bytes; Metro at 1,662,714 raw, 419,740 gzip, and 326,785 Brotli bytes; and
+  Hermes at 3,371,236 raw, 1,347,950 gzip, and 1,061,946 Brotli bytes. The
+  reviewed ceilings advance to 1,611,000/380,000/297,500 for Vite,
+  1,256,000/346,000/278,200 for esbuild, and 3,373,000/1,368,000 raw/gzip for
+  Hermes. Metro and the Hermes Brotli ceiling remain unchanged.
+
 - Make verified phone changes interruption-safe by staging the replacement key
   in WAB, publishing the UMP rotation, and then finalizing WAB. Authentication
   can recover an interrupted transition from the current or pending key and
@@ -468,7 +817,7 @@ attention to changes that materially alter behavior or extend functionality.
 
 ## wallet-toolbox 2.1.27
 
-- Add optional `contactSource` (and exported `ContactSource` / `ContactRecord` interfaces) on `WalletArgs` and the `Wallet` class. When provided, `Wallet.discoverByIdentityKey` consults the local contacts source **before** the in-process `_overlayCache` and before any network call; on a hit, the overlay is not queried at all. `Wallet.discoverByAttributes` consults the contact source's optional `findByAttributes` when present. Contact-source failures are swallowed and fall through to the existing network path so the network is never gated on a flaky contact store.
+- Add optional `contactSource` (and exported `ContactSource` / `ContactRecord` interfaces) on `WalletArgs` and the `Wallet` class. When provided, `Wallet.discoverByIdentityKey` consults the local contacts source **before** the in-process `_overlayCache` and before any network call; on a hit, the overlay is not queried at all. `Wallet.discoverByAttributes` consults the contact source's optional `findByAttributes` when present. Contact-source failures are swallowed and fall through to the existing network path so the network is never gated on a flaky contact store. A caller-installed contact source is an explicit personal trust anchor: its records are locally authoritative, not third-party certifier attestations. Synthetic contact results default to type `contact`, use empty certificate-proof fields, and receive infinite trust to express that local policy decision.
 - `Wallet.discoverByIdentityKey` and `Wallet.discoverByAttributes` accept `forceRefresh?: boolean`. When `true`, both the contacts short-circuit and the 2-minute `_overlayCache` are bypassed so the network is consulted fresh — useful for a manual refresh action.
 - `identityUtils.parseResults` now yields to the host runtime between certificates on UI runtimes (browser / React Native, detected at call time). On Node the yield is skipped to avoid timer overhead. Same total work; the JS thread no longer owns the frame for the duration of the loop, so menu taps and scroll keep working while a large identity result is parsed.
 - `identityUtils.parseResults$` — new async-iterable form that emits each successfully parsed `VerifiableCertificate` as soon as it's ready, for callers that want progressive rendering.