bsv-sdk 0.8.0 → 0.8.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: efdb265c4e97396d1769a3652cbe04c57ad154035d2d990f118e343bbe7fafa8
-  data.tar.gz: c955e289d5b4316db586980bc02287d29d2e973ac071b5be5ca33298f6e647a5
+  metadata.gz: d6ca8a4c5f04ec956d46b46d18286a7dad889496c1ae2683ff79210c01e10529
+  data.tar.gz: d2c327e4b61a979909dd31f4755f852c3d0c6153f00c135c446ec64386f5709a
 SHA512:
-  metadata.gz: a2ca1b5d8d35586daa3da201ebf137e5aa01034452b931185b7804add1537c35db0f1d95463676dbb060b1475a41b236fa178dee2bdcd748051b27a6ce9c40ea
-  data.tar.gz: fa8c3e8f7b4a1ad2b5b9870fbccc65c0fbbc85c9b55607f00068dfcf008c4bfee403df05777106ac54c3c7ee0384841bc258f0f8e918a852578fb65309415502
+  metadata.gz: 015f6727214704a6a972f1fd6f8cc7c9597f21df21ab794698d9862f7c9a4df702f33be01cec402dc5726240137ae22137f74e08cfc820dd25204ffa92b32768
+  data.tar.gz: 53517f95c64d850fa7b164f4b785020d81866525d9bda7623e22eeb89ba7a55d61f9d8c17abadf8cadcee9a5a4469516bbe9f87102f50bbc2f32559963ff5fd0

data/CHANGELOG.md

@@ -21,6 +21,67 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and each gem adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html)
 independently.
 
+## sdk-0.8.1 — 2026-04-08
+
+### Fixed
+
+- [sdk] **`Transaction#to_beef` strips phantom `txid: true` leaves** —
+  when a proof loaded from a shared `LocalProofStore` carries txid flags
+  for transactions that are not part of the bundle being constructed,
+  `to_beef` now rebuilds each per-block BUMP from only the bundle's own
+  txids instead of propagating the phantoms into the serialised output.
+  ARC previously rejected such BEEFs with misleading parser errors,
+  blocking any wallet workflow that received a BEEF via
+  `internalize_action` and then spent the internalised UTXOs.
+  Closes #302.
+
+### Added
+
+- [sdk] **`MerklePath#extract(txid_hashes)`** — returns a new trimmed
+  compound path covering only the requested txids, reconstructing the
+  minimum set of sibling hashes at each tree level. Raises
+  `ArgumentError` on empty input, unknown txid, or root mismatch.
+  Ported from the TypeScript SDK. Used internally by
+  `Transaction#to_beef` and available for direct use.
+- [sdk] **`MerklePath#trim`** — removes internal nodes not required by
+  level-zero txid leaves. Called implicitly by `#combine` and `#extract`
+  and rarely needs to be invoked directly. Ported from the TypeScript
+  SDK.
+- [sdk] **`MerklePath#initialize_copy`** — `.dup` now produces a new
+  MerklePath whose outer and level arrays are independent of the
+  source, so the copy can be freely mutated via `#combine`, `#trim`,
+  or `#extract` without affecting the original. `PathElement`s
+  remain immutable and are shared between source and copy.
+
+### Changed
+
+- [sdk] **`MerklePath#combine`** now calls `#trim` at the end so merged
+  paths stay minimal across repeated merges, matching the TypeScript
+  SDK. Combined paths are strictly smaller than before — external
+  callers that inspected `mp.path` after `#combine` may see fewer
+  nodes, though every txid leaf's merkle proof is preserved.
+- [sdk] **`MerklePath#combine`** also preserves `txid: true` flags when
+  the incoming leaf is flagged and the existing leaf at the same offset
+  isn't, so merging an ancestor's single-leaf proof into a compound
+  that already contains the same offset as a sibling no longer loses
+  the txid flag.
+- [sdk] **`Transaction#to_beef`** now raises `ArgumentError` if an
+  ancestor's merkle path doesn't actually contain that transaction's
+  txid, or if the rebuilt BUMP's root doesn't match the source root.
+  Previously such corrupt proof data would silently emit a broken BEEF.
+  Callers relying on `to_beef` not raising on valid data are
+  unaffected; the new exception only triggers on corrupt proof stores.
+
+### Internal
+
+- [sdk] **`Beef#merge_transaction`** indirectly benefits from the
+  tighter `#combine` + `#trim` behaviour: compound BUMPs no longer
+  accumulate dead sibling hashes across repeated merges.
+- [sdk] On the real-world `#302` regression fixture, the cleaned BUMP
+  shrinks from 2476 B to 1300 B (47% reduction) as a side effect of
+  `#extract` removing intermediate siblings that are no longer needed
+  once phantom leaves are gone.
+
 ## sdk-0.8.0 — 2026-04-08
 
 ### Added

data/lib/bsv/transaction/merkle_path.rb

@@ -1,5 +1,7 @@
 # frozen_string_literal: true
 
+require 'set'
+
 module BSV
   module Transaction
     # A BRC-74 merkle path (BUMP — Bitcoin Unified Merkle Path).
@@ -50,6 +52,19 @@ module BSV
         @path = path
       end
 
+      # Produce an independent copy: a new MerklePath whose outer +path+
+      # array and each inner level array can be mutated (via {#combine},
+      # {#trim}, {#extract}) without affecting the original. PathElements
+      # themselves are immutable and are shared between the original and
+      # the copy.
+      #
+      # @param source [MerklePath] the MerklePath being copied from
+      # @return [void]
+      def initialize_copy(source)
+        super
+        @path = source.path.map(&:dup)
+      end
+
       # --- Binary serialisation (BRC-74) ---
 
       # Deserialise a merkle path from BRC-74 binary format.
@@ -270,7 +285,11 @@ module BSV
       # Merge another merkle path into this one.
       #
       # Both paths must share the same block height and merkle root.
-      # After combining, this path contains the union of all leaves.
+      # After combining, this path contains the union of all leaves,
+      # trimmed to the minimum set required to prove every txid-flagged
+      # leaf. The trim matches the TS SDK's +combine+ behaviour and
+      # prevents accumulation of unnecessary sibling hashes across
+      # repeated merges.
       #
       # @param other [MerklePath] the path to merge in
       # @return [self] for chaining
@@ -289,16 +308,182 @@ module BSV
 
           existing = @path[h].to_h { |e| [e.offset, e] }
           other.path[h].each do |elem|
-            existing[elem.offset] ||= elem
+            # Preserve txid flag when combining: if the incoming leaf is
+            # flagged, never downgrade an existing entry.
+            if existing.key?(elem.offset)
+              existing_elem = existing[elem.offset]
+              if elem.txid && !existing_elem.txid
+                existing[elem.offset] = PathElement.new(
+                  offset: existing_elem.offset,
+                  hash: existing_elem.hash,
+                  txid: true,
+                  duplicate: existing_elem.duplicate
+                )
+              end
+            else
+              existing[elem.offset] = elem
+            end
           end
           @path[h] = existing.values.sort_by(&:offset)
         end
 
+        trim
         self
       end
 
+      # --- Trim ---
+
+      # Remove all internal nodes that are not required by level zero
+      # txid-flagged leaves. Assumes the path has at least the minimum
+      # set of sibling hashes needed to prove every txid leaf. Leaves
+      # each level sorted by increasing offset.
+      #
+      # This is the Ruby port of the TypeScript SDK's +MerklePath.trim+.
+      # It is called implicitly by {#combine} and {#extract} and rarely
+      # needs to be invoked directly.
+      #
+      # @return [self] for chaining
+      def trim
+        @path.each { |level| level.sort_by!(&:offset) }
+
+        computed_offsets = []
+        drop_offsets = []
+
+        @path[0].each_with_index do |node, i|
+          if node.txid
+            # level 0 must enable computing level 1 for every txid node
+            trim_push_if_new(computed_offsets, node.offset >> 1)
+          else
+            # Array-index peer — works for well-formed compound BUMPs
+            # where level 0 is a sequence of adjacent (txid, sibling) pairs.
+            peer_index = node.offset.odd? ? i - 1 : i + 1
+            peer = @path[0][peer_index] if peer_index.between?(0, @path[0].length - 1)
+            # Drop non-txid level 0 nodes whose peer is also non-txid
+            trim_push_if_new(drop_offsets, peer.offset) if peer && !peer.txid
+          end
+        end
+
+        trim_drop_offsets_from_level(drop_offsets, 0)
+
+        (1...@path.length).each do |h|
+          drop_offsets = computed_offsets
+          computed_offsets = trim_next_computed_offsets(computed_offsets)
+          trim_drop_offsets_from_level(drop_offsets, h)
+        end
+
+        self
+      end
+
+      # --- Extract ---
+
+      # Extract a minimal compound MerklePath covering only the specified
+      # transaction IDs.
+      #
+      # Given a compound path (e.g. one merged from multiple single-leaf
+      # proofs in the same block), this method reconstructs the minimum
+      # set of sibling hashes at each tree level for every requested txid,
+      # assembles them into a new trimmed compound path, and verifies
+      # that the extracted path computes the same merkle root as the
+      # source.
+      #
+      # The primary use case is +Transaction#to_beef+: when a BUMP loaded
+      # from a proof store carries +txid: true+ flags for transactions
+      # that are not part of the current BEEF bundle, extracting only the
+      # bundled txids strips the phantom flags (and the now-unneeded
+      # sibling nodes) from the serialised output. See issue #302 for
+      # background.
+      #
+      # Matches the TS SDK's +MerklePath.extract+ behaviour.
+      #
+      # @param txid_hashes [Array<String>] 32-byte txids in internal byte
+      #   order (reverse of display order). To pass hex strings, use
+      #   +txid_hexes.map { |h| [h].pack('H*').reverse }+.
+      # @return [MerklePath] a new trimmed compound path proving only the
+      #   requested txids
+      # @raise [ArgumentError] if +txid_hashes+ is empty, any requested
+      #   txid is not present in the source path's level 0, or the
+      #   extracted path's root does not match the source root
+      def extract(txid_hashes)
+        raise ArgumentError, 'at least one txid must be provided to extract' if txid_hashes.empty?
+
+        original_root = compute_root
+        indexed = build_indexed_path
+
+        # Build a level-0 hash → offset lookup
+        txid_to_offset = {}
+        @path[0].each do |leaf|
+          txid_to_offset[leaf.hash] = leaf.offset if leaf.hash
+        end
+
+        max_offset = @path[0].map(&:offset).max || 0
+        tree_height = [@path.length, max_offset.bit_length].max
+
+        needed = Array.new(tree_height) { {} }
+
+        txid_hashes.each do |txid|
+          tx_offset = txid_to_offset[txid]
+          if tx_offset.nil?
+            raise ArgumentError,
+                  "transaction ID #{txid.reverse.unpack1('H*')} not found in the Merkle Path"
+          end
+
+          # Level 0: the txid leaf itself + its tree sibling
+          needed[0][tx_offset] = PathElement.new(offset: tx_offset, hash: txid, txid: true)
+          sib0_offset = tx_offset ^ 1
+          unless needed[0].key?(sib0_offset)
+            sib = offset_leaf(indexed, 0, sib0_offset)
+            needed[0][sib0_offset] = sib if sib
+          end
+
+          # Higher levels: just the sibling at each height
+          (1...tree_height).each do |h|
+            sib_offset = (tx_offset >> h) ^ 1
+            next if needed[h].key?(sib_offset)
+
+            sib = offset_leaf(indexed, h, sib_offset)
+            if sib
+              needed[h][sib_offset] = sib
+            elsif (tx_offset >> h) == (max_offset >> h)
+              # Rightmost path in a tree whose last leaf has no real sibling —
+              # BRC-74 represents this as a duplicate marker.
+              needed[h][sib_offset] = PathElement.new(offset: sib_offset, duplicate: true)
+            end
+          end
+        end
+
+        compound_path = needed.map { |level| level.values.sort_by(&:offset) }
+        compound = self.class.new(block_height: @block_height, path: compound_path)
+        compound.trim
+
+        extracted_root = compound.compute_root
+        unless extracted_root == original_root
+          raise ArgumentError,
+                "extracted path root #{extracted_root.reverse.unpack1('H*')} " \
+                "does not match source root #{original_root.reverse.unpack1('H*')}"
+        end
+
+        compound
+      end
+
       private
 
+      def trim_push_if_new(arr, value)
+        arr << value if arr.empty? || arr.last != value
+      end
+
+      def trim_drop_offsets_from_level(drop_offsets, level)
+        return if drop_offsets.empty?
+
+        drop_set = drop_offsets.to_set
+        @path[level].reject! { |node| drop_set.include?(node.offset) }
+      end
+
+      def trim_next_computed_offsets(offsets)
+        next_offsets = []
+        offsets.each { |o| trim_push_if_new(next_offsets, o >> 1) }
+        next_offsets
+      end
+
       def build_indexed_path
         @path.map do |level|
           level.to_h { |elem| [elem.offset, elem] }

data/lib/bsv/transaction/transaction.rb

@@ -299,24 +299,36 @@ module BSV
       # Transactions with a `merkle_path` are treated as proven leaves — their
       # ancestors are not traversed further.
       #
+      # Proven ancestors that share a block are combined into a single BUMP per
+      # block, then trimmed via {MerklePath#extract} so the serialised bundle
+      # carries only the +txid: true+-flagged leaves that correspond to
+      # transactions in this BEEF. This prevents "phantom" txid leaves carried
+      # over from a shared {LocalProofStore} entry (issue #302) and also
+      # shrinks the BEEF by dropping intermediate sibling hashes that are no
+      # longer needed.
+      #
+      # Ancestor +merkle_path+ objects are not mutated: paths are deep-copied
+      # before any combine/trim work.
+      #
       # @return [String] raw BEEF V1 binary
+      # @raise [ArgumentError] if an ancestor's merkle_path does not actually
+      #   contain that transaction's txid, or if the cleaned BUMP's root does
+      #   not match the source root (both indicate corrupt proof data)
       def to_beef
         beef = Beef.new
         ancestors = collect_ancestors
 
+        bump_index_by_height = build_beef_bumps(beef, ancestors)
+
         ancestors.each do |tx|
           entry = if tx.merkle_path
-                    bump_idx = beef.merge_bump(tx.merkle_path)
                     Beef::BeefTx.new(
                       format: Beef::FORMAT_RAW_TX_AND_BUMP,
                       transaction: tx,
-                      bump_index: bump_idx
+                      bump_index: bump_index_by_height.fetch(tx.merkle_path.block_height)
                     )
                   else
-                    Beef::BeefTx.new(
-                      format: Beef::FORMAT_RAW_TX,
-                      transaction: tx
-                    )
+                    Beef::BeefTx.new(format: Beef::FORMAT_RAW_TX, transaction: tx)
                   end
           beef.transactions << entry
         end
@@ -724,6 +736,35 @@ module BSV
         result << tx
       end
 
+      # Group proven ancestors by block height, combine each group into a
+      # single compound merkle path (without mutating the source paths), then
+      # extract just the txids actually in the bundle. The resulting clean
+      # BUMPs are appended to +beef.bumps+, one per block height.
+      #
+      # @return [Hash{Integer => Integer}] block height → bump index mapping
+      def build_beef_bumps(beef, ancestors)
+        proven_by_height = ancestors.each_with_object({}) do |tx, h|
+          next unless tx.merkle_path
+
+          (h[tx.merkle_path.block_height] ||= []) << tx
+        end
+
+        bump_index_by_height = {}
+        proven_by_height.each do |height, txs|
+          # Deep-dup the first source so combine/trim can't mutate caller state
+          merged = txs.first.merkle_path.dup
+          txs.drop(1).each { |t| merged.combine(t.merkle_path) }
+
+          txid_hashes = txs.map { |t| t.txid.reverse }
+          clean = merged.extract(txid_hashes)
+
+          bump_index_by_height[height] = beef.bumps.length
+          beef.bumps << clean
+        end
+
+        bump_index_by_height
+      end
+
       def compute_fee_sats(model_or_fee)
         case model_or_fee
         when nil

data/lib/bsv/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module BSV
-  VERSION = '0.8.0'
+  VERSION = '0.8.1'
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: bsv-sdk
 version: !ruby/object:Gem::Version
-  version: 0.8.0
+  version: 0.8.1
 platform: ruby
 authors:
 - Simon Bettison