opentimestamps 0.1.0 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 6f146094f21ed3a732a3a7cfa6b799366a25a2434953d16dcf3825052f3c967a
-  data.tar.gz: '08b95d3c6e8e7b3853e4753d76171e72eb46b9645c07454ebf6ea2b1407510db'
+  metadata.gz: 0b838c1eea0b1a967eca1038728e5aaa07aea6fc10ea932258543cda2d31c91a
+  data.tar.gz: '009ed17044167540849e8b07dde1f218a5bd4bd2e55dbf593ecfe07126fe1a54'
 SHA512:
-  metadata.gz: 1d063990950ea15459b67adf25fdb8699dec5ff1ce75901cefdeee5c03b34373207b6a7ec988c1764cc19458576254cc0b8a06e796ea9ad98d282776c63d34c5
-  data.tar.gz: 41756a3dd2934d2f236b43c5738524cf121a62f692c026144910aa8150525cad8aa2891585e1bb58ba62ea6b5a39f40c617c7d8d617b31dc98d02208b02bfe1e
+  metadata.gz: 91e9a62492b192bc54dcdc71c63d69c6964921f6a3003e9b5055b83444f60ef537bed2fac99100061e25800706c39fcae1e9cb337e72a39ae3337126e7e02bb3
+  data.tar.gz: 592987fab8b0eb07e7dd9165d2441df8352abb210170d0f19572a303fbc6686169208f8161114c663570f5c3694441982e5aef3e6774a201a1501f6eeb1d54f0

data/CHANGELOG.md

@@ -5,6 +5,78 @@ project uses [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+## [0.3.0]
+
+### Added
+
+- `OpenTimestamps.digest_file` streams a file through a file-hash op, so the `ots`
+  CLI no longer reads a whole file into memory to stamp or verify it.
+
+### Changed
+
+- `verify` tolerates a chain oracle that is unreachable for some blocks: if the
+  reachable anchors already meet the quorum it succeeds; if a quorum is reachable
+  but blocked by the outage it raises `NetworkError` (carrying the underlying
+  cause) rather than `VerificationError`; and if the quorum is unreachable no
+  matter the oracle it still raises `VerificationError`. A transient outage is
+  never reported as a failed proof, and an impossible quorum is never reported as
+  an outage.
+- A chain oracle now signals a nonexistent block with the new `BlockNotFound`
+  (HTTP 404 for the explorer, RPC -8/-5 for a node), which `verify` treats as "not
+  anchored" (a `VerificationError`), not as an outage.
+- **Breaking for direct callers of `Timestamp#verify`:** each result now includes
+  an `error:` field, and `verified: false` can mean either a root mismatch or an
+  oracle that could not be reached (check `error`). The high-level `verify` /
+  `verified?` API is unchanged.
+
+### Hardened
+
+- `BitcoinCore` tags each JSON-RPC call with a unique id and rejects a response
+  that does not echo it (a stale or mixed-up connection), after surfacing any
+  bitcoind error first.
+- More transport failures (`Timeout::Error`, `Net::ProtocolError`) are wrapped as
+  `NetworkError` instead of escaping as a raw error.
+- The CLI rejects unexpected extra arguments and a non-positive `--timeout`.
+
+## [0.2.0]
+
+### Added
+
+- `Chain::BitcoinCore`, a JSON-RPC oracle that verifies against your own Bitcoin
+  Core node (`getblockhash` + `getblockheader`) for a fully trustless check;
+  supports basic-auth credentials via argument, URL, or `.cookie` file.
+- `ots` command-line tool (`stamp | upgrade | verify | info | version`), a thin
+  wrapper over the library. `verify` binds the proof to the file and can target a
+  node (`--node`) and require an m-of-n quorum (`--quorum`).
+- `verify` / `verified?` take a `quorum:` (m-of-n): require the proof to be
+  anchored in at least that many *distinct Bitcoin blocks*. Several tree paths
+  proving the same anchor collapse to one, so a single anchor can never satisfy
+  quorum > 1.
+- `OpenTimestamps.digest_for_kind`, which hashes data with any file-hash op
+  (sha1 / ripemd160 / sha256) to bind a proof to a document.
+
+### Changed
+
+- Timestamp children now serialize in the reference client's canonical order (ops
+  by `[tag, operand]`, Bitcoin attestations by height, pending by uri), so a proof
+  we build or merge is byte-identical to `ots` output and any reference proof
+  still round-trips byte-for-byte.
+
+### Hardened
+
+- Duplicate attestations are dropped on deserialize (the reference keeps a set),
+  and a binary op with an empty operand is rejected, closing two ways a proof
+  could otherwise pad a quorum from a single real anchor.
+- Chain oracles validate the shape of everything they read (merkle-root hex,
+  block hash, JSON-RPC envelope, HTTP status) and raise `NetworkError` rather than
+  a raw Ruby error on malformed responses.
+
+### Internal
+
+- `Calendar`, `Chain::Explorer`, and `Chain::BitcoinCore` take an injectable
+  transport; `stamp_digest` / `upgrade` take a `calendar_factory`, enabling
+  offline unit tests of the network paths and merge logic.
+
 ## [0.1.0]
 
 Initial release: a pure-Ruby OpenTimestamps client with no runtime dependencies.
@@ -31,5 +103,7 @@ caps; only `DeserializationError` on malformed bytes). Tested with byte-exact
 round-trips of twelve reference vectors, an offline check against Bitcoin block
 358391, and Keccak-256 known-answer vectors.
 
-[Unreleased]: https://github.com/sirruf/opentimestamps/compare/v0.1.0...HEAD
+[Unreleased]: https://github.com/sirruf/opentimestamps/compare/v0.3.0...HEAD
+[0.3.0]: https://github.com/sirruf/opentimestamps/compare/v0.2.0...v0.3.0
+[0.2.0]: https://github.com/sirruf/opentimestamps/compare/v0.1.0...v0.2.0
 [0.1.0]: https://github.com/sirruf/opentimestamps/releases/tag/v0.1.0

data/README.md

@@ -75,13 +75,44 @@ byte-for-byte, and checks a proof against Bitcoin block 358391 using a recorded
 merkle root, offline. The reference `ots` client also reads proofs this gem
 creates.
 
+## Command line
+
+The gem installs an `ots` executable, a thin wrapper over the library:
+
+```console
+$ ots stamp report.pdf                 # writes report.pdf.ots (pending)
+$ ots upgrade report.pdf.ots           # an hour later: fold in the Bitcoin path
+$ ots verify report.pdf                # checks report.pdf.ots and binds it to the file
+$ ots info report.pdf.ots              # dump the proof's structure
+```
+
+`verify` recomputes the file's hash and confirms it matches the proof, so a
+success means *this* document is anchored, not just that the proof is valid. It
+can also require the proof to be anchored in several distinct Bitcoin blocks
+(`--quorum 2`) and check against your own node (see below).
+
+## Verifying against your own node
+
+`Chain::Explorer` trusts a public explorer. For a check that trusts nothing
+external, point verification at your own Bitcoin Core:
+
+```ruby
+chain = OpenTimestamps::Chain::BitcoinCore.new(
+  "http://127.0.0.1:8332", user: "rpcuser", password: "secret"
+)
+OpenTimestamps.verify(ots, chain: chain, quorum: 2)
+```
+
+`BitcoinCore.from_cookie(path)` reads the node's `.cookie` file instead of a
+password. From the CLI: `ots verify file --node http://127.0.0.1:8332 --cookie ~/.bitcoin/.cookie`.
+
 ## Roadmap
 
 - [x] Byte-exact interop with reference `.ots` vectors, both directions.
 - [x] Multi-calendar submit with merge (a single calendar being down is not fatal).
-- [ ] `Chain::BitcoinCore` JSON-RPC adapter (verification against your own node).
-- [ ] Calendar quorum on upgrade (m-of-n).
-- [ ] Optional CLI (`ots stamp | upgrade | verify | info`).
+- [x] `Chain::BitcoinCore` JSON-RPC adapter (verification against your own node).
+- [x] Quorum (m-of-n distinct Bitcoin blocks), enforced at verify.
+- [x] CLI (`ots stamp | upgrade | verify | info`).
 
 ## License
 

data/exe/ots

@@ -0,0 +1,188 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+require "optparse"
+require "opentimestamps"
+
+# Command-line front end for the opentimestamps gem: stamp a file against the
+# Bitcoin blockchain, upgrade the pending proof once a calendar anchors it, and
+# verify it. Thin wrapper over the library API.
+module OTSCLI
+  module_function
+
+  def main(argv)
+    command = argv.shift
+    case command
+    when "stamp"   then stamp(argv)
+    when "upgrade" then upgrade(argv)
+    when "verify"  then verify(argv)
+    when "info"    then info(argv)
+    when "version", "--version", "-v" then puts "opentimestamps #{OpenTimestamps::VERSION}"
+    when nil, "help", "--help", "-h" then warn(usage) || (exit(command.nil? ? 1 : 0))
+    else
+      warn "ots: unknown command #{command.inspect}\n\n#{usage}"
+      exit 1
+    end
+  rescue OpenTimestamps::Error, OptionParser::ParseError, SystemCallError, IOError, URI::InvalidURIError => e
+    warn "ots: #{e.message}"
+    exit 1
+  rescue Interrupt
+    warn "ots: interrupted"
+    exit 130
+  end
+
+  def usage
+    <<~TXT
+      Usage: ots <command> [options]
+
+      Commands:
+        stamp <file>            Create <file>.ots (submits the file hash to calendars)
+        upgrade <file.ots>      Fold in the Bitcoin path once a calendar has anchored it
+        verify <file>           Check <file>.ots against the chain and bind it to <file>
+        info <file.ots>         Print the structure of a proof
+        version                 Print the library version
+
+      Run "ots <command> --help" for command-specific options.
+    TXT
+  end
+
+  def stamp(argv)
+    opts = { calendars: [], timeout: 20 }
+    OptionParser.new do |p|
+      p.banner = "Usage: ots stamp <file> [options]"
+      p.on("-c", "--calendar URL", "Calendar URL (repeatable; defaults to the public calendars)") { |u| opts[:calendars] << u }
+      p.on("-o", "--out PATH", "Output path (default: <file>.ots)") { |o| opts[:out] = o }
+      p.on("--timeout SECONDS", Integer, "Per-request timeout (default: 20)") { |t| opts[:timeout] = t }
+    end.parse!(argv)
+
+    file = require_arg(argv, "stamp <file>")
+    no_extra(argv)
+    check_timeout(opts[:timeout])
+    out = opts[:out] || "#{file}.ots"
+
+    calendars = opts[:calendars].empty? ? OpenTimestamps::DEFAULT_CALENDARS : opts[:calendars]
+    # Stream the file through the hash rather than reading it whole into memory.
+    digest = OpenTimestamps.digest_file(file, :sha256)
+    detached = OpenTimestamps.stamp_digest(digest, calendars: calendars, timeout: opts[:timeout])
+    write_new(out, detached.serialize)
+    warn "Stamped #{file} -> #{out} (pending; upgrade in an hour or so)"
+  end
+
+  def upgrade(argv)
+    opts = { timeout: 20 }
+    OptionParser.new do |p|
+      p.banner = "Usage: ots upgrade <file.ots> [options]"
+      p.on("--timeout SECONDS", Integer, "Per-request timeout (default: 20)") { |t| opts[:timeout] = t }
+    end.parse!(argv)
+
+    path = require_arg(argv, "upgrade <file.ots>")
+    no_extra(argv)
+    check_timeout(opts[:timeout])
+    detached = OpenTimestamps::DetachedTimestampFile.deserialize(File.binread(path))
+    if OpenTimestamps.upgrade(detached, timeout: opts[:timeout])
+      write_atomic(path, detached.serialize)
+      warn "Upgraded #{path} (now carries a Bitcoin attestation)"
+    else
+      warn "No change: #{path} is already anchored, or no calendar has anchored it yet."
+    end
+  end
+
+  def verify(argv)
+    opts = { quorum: 1 }
+    OptionParser.new do |p|
+      p.banner = "Usage: ots verify <file> [options]"
+      p.on("-o", "--ots PATH", "Proof path (default: <file>.ots)") { |o| opts[:ots] = o }
+      p.on("--quorum N", Integer, "Require N independent Bitcoin anchors (default: 1)") { |n| opts[:quorum] = n }
+      p.on("--node URL", "Verify against your Bitcoin Core node instead of a public explorer") { |u| opts[:node] = u }
+      p.on("--rpcuser USER", "Bitcoin Core RPC user") { |u| opts[:user] = u }
+      p.on("--rpcpassword PASS", "Bitcoin Core RPC password") { |v| opts[:password] = v }
+      p.on("--cookie PATH", "Bitcoin Core .cookie file for RPC auth") { |c| opts[:cookie] = c }
+    end.parse!(argv)
+
+    file = require_arg(argv, "verify <file>")
+    no_extra(argv)
+    ots_path = opts[:ots] || "#{file}.ots"
+    detached = OpenTimestamps::DetachedTimestampFile.deserialize(File.binread(ots_path))
+
+    # Bind the proof to the document first: it is cheap and needs no network, so a
+    # mismatch fails fast before any chain lookup.
+    unless bound_to_file?(detached, file)
+      warn "ots: #{ots_path} is a valid proof but does NOT match #{file}"
+      exit 1
+    end
+
+    chain = chain_for(opts)
+    results = OpenTimestamps.verify(detached, chain: chain, quorum: opts[:quorum])
+    results.each { |r| puts "OK: #{file} anchored in Bitcoin block ##{r.height} at #{r.time.iso8601}" }
+  end
+
+  def info(argv)
+    OptionParser.new { |p| p.banner = "Usage: ots info <file.ots>" }.parse!(argv)
+    path = require_arg(argv, "info <file.ots>")
+    no_extra(argv)
+    detached = OpenTimestamps::DetachedTimestampFile.deserialize(File.binread(path))
+    puts OpenTimestamps.info(detached)
+  end
+
+  # --- helpers ---
+
+  def chain_for(opts)
+    return OpenTimestamps::Chain::Explorer.new unless opts[:node]
+
+    if opts[:cookie]
+      OpenTimestamps::Chain::BitcoinCore.from_cookie(opts[:cookie], opts[:node])
+    else
+      OpenTimestamps::Chain::BitcoinCore.new(opts[:node], user: opts[:user], password: opts[:password])
+    end
+  end
+
+  # Recompute the file's hash with the proof's file-hash op and compare it to the
+  # digest the proof commits to; this is what binds a proof to a document.
+  def bound_to_file?(detached, file)
+    OpenTimestamps.digest_file(file, detached.file_hash_op.kind) == detached.file_digest
+  end
+
+  def require_arg(argv, shape)
+    arg = argv.shift
+    return arg if arg
+
+    warn "ots: missing argument\nUsage: ots #{shape}"
+    exit 1
+  end
+
+  # Reject leftover positional arguments rather than silently ignoring them.
+  def no_extra(argv)
+    return if argv.empty?
+
+    warn "ots: unexpected argument(s): #{argv.join(' ')}"
+    exit 1
+  end
+
+  def check_timeout(seconds)
+    return if seconds.positive?
+
+    warn "ots: --timeout must be a positive integer (seconds)"
+    exit 1
+  end
+
+  # Create a new file, refusing to clobber an existing one. O_EXCL makes the
+  # check-and-create atomic (no TOCTOU window).
+  def write_new(path, bytes)
+    File.open(path, File::WRONLY | File::CREAT | File::EXCL | File::BINARY) { |f| f.write(bytes) }
+  rescue Errno::EEXIST
+    warn "ots: #{path} already exists; refusing to overwrite"
+    exit 1
+  end
+
+  # Replace an existing file atomically: write a sibling temp file, then rename it
+  # over the target, so a crash mid-write cannot destroy the only copy of a proof.
+  def write_atomic(path, bytes)
+    tmp = "#{path}.tmp.#{Process.pid}"
+    File.binwrite(tmp, bytes)
+    File.rename(tmp, path)
+  ensure
+    File.unlink(tmp) if tmp && File.exist?(tmp)
+  end
+end
+
+OTSCLI.main(ARGV)

data/lib/opentimestamps/attestation.rb

@@ -46,6 +46,7 @@ module OpenTimestamps
 
       def pending? = true
       def components = [@uri]
+      def sort_key = [PENDING_TAG, @uri.b]
       def to_s = "PENDING #{@uri}"
 
       def serialize(writer)
@@ -64,6 +65,7 @@ module OpenTimestamps
 
       def bitcoin? = true
       def components = [@height]
+      def sort_key = [BITCOIN_TAG, @height]
       def to_s = "BITCOIN block ##{@height}"
 
       def serialize(writer)
@@ -83,6 +85,7 @@ module OpenTimestamps
       end
 
       def components = [@tag, @payload]
+      def sort_key = [@tag, @payload]
       def to_s = "UNKNOWN(#{@tag.unpack1('H*')})"
 
       def serialize(writer)

data/lib/opentimestamps/calendar.rb

@@ -13,8 +13,12 @@ module OpenTimestamps
 
     attr_reader :url
 
-    def initialize(url)
+    # +transport+ is an injection seam for tests and custom clients: a callable
+    # taking (request, timeout) and returning [status_string, body_bytes]. It
+    # defaults to a capped Net::HTTP round-trip.
+    def initialize(url, transport: nil)
       @url = url.to_s.chomp("/")
+      @transport = transport
     end
 
     # Submit a digest; returns a pending Timestamp rooted at that digest.
@@ -44,6 +48,8 @@ module OpenTimestamps
     def host = URI(@url).host
 
     def request(req, timeout)
+      return @transport.call(req, timeout) if @transport
+
       uri = URI(@url)
       req["Accept"] = MIME
       req["User-Agent"] = "opentimestamps-ruby/#{VERSION}"

data/lib/opentimestamps/chain.rb

@@ -4,6 +4,7 @@ require "net/http"
 require "uri"
 require "json"
 require "time"
+require "securerandom"
 
 module OpenTimestamps
   # A chain oracle resolves a block height to its merkle root (in internal byte
@@ -11,27 +12,51 @@ module OpenTimestamps
   # depends only on this interface, so a Bitcoin node can be dropped in for a
   # fully trustless check.
   module Chain
+    MAX_RESPONSE_BYTES = 1 << 20
+
+    HEX64 = /\A[0-9a-fA-F]{64}\z/
+
+    # Turn a block's display-order merkle-root hex into internal byte order, the
+    # form an OTS commitment is compared against. Rejects anything that is not a
+    # 32-byte hex string so junk from an oracle fails loudly, not as a silent
+    # mismatch.
+    def self.root_display_to_internal(hex)
+      raise NetworkError, "chain oracle returned an invalid merkle root" unless hex.is_a?(String) && hex.match?(HEX64)
+
+      [hex].pack("H*").reverse
+    end
+
     # Public block explorer (Esplora API). Convenient, but trusts the explorer.
     class Explorer
-      MAX_RESPONSE_BYTES = 1 << 20
+      # Preserved for backward compatibility; the cap now lives on the module.
+      MAX_RESPONSE_BYTES = Chain::MAX_RESPONSE_BYTES
 
-      def initialize(base = "https://blockstream.info/api", timeout: 20)
+      # +transport+ is an injection seam for tests and custom clients: a callable
+      # taking a URL and returning the response body. It defaults to a capped
+      # Net::HTTP GET.
+      def initialize(base = "https://blockstream.info/api", timeout: 20, transport: nil)
         @base = base.chomp("/")
         @timeout = timeout
+        @transport = transport
       end
 
       def block_merkle_root_and_time(height)
-        hash = get("#{@base}/block-height/#{height}").strip
+        hash = get("#{@base}/block-height/#{Integer(height)}").strip
+        raise NetworkError, "explorer returned an invalid block hash" unless hash.match?(Chain::HEX64)
+
         blk = JSON.parse(get("#{@base}/block/#{hash}"))
-        root_internal = [blk.fetch("merkle_root")].pack("H*").reverse # display -> internal
-        [root_internal, Time.at(blk.fetch("timestamp")).utc]
-      rescue JSON::ParserError, KeyError => e
+        raise NetworkError, "explorer returned unexpected data" unless blk.is_a?(Hash)
+
+        [Chain.root_display_to_internal(blk.fetch("merkle_root")), Time.at(Integer(blk.fetch("timestamp"))).utc]
+      rescue JSON::ParserError, KeyError, TypeError, ArgumentError => e
         raise NetworkError, "explorer returned unexpected data: #{e.class}"
       end
 
       private
 
       def get(url)
+        return @transport.call(url) if @transport
+
         uri = URI(url)
         http = Net::HTTP.new(uri.host, uri.port)
         http.use_ssl = (uri.scheme == "https")
@@ -40,24 +65,136 @@ module OpenTimestamps
 
         http.start do |conn|
           conn.request(Net::HTTP::Get.new(uri)) do |res|
+            # A 404 means the block does not exist (height past the tip, or a bogus
+            # proof), which is a verification fact, not an outage.
+            raise BlockNotFound, "explorer: block not found (HTTP 404)" if res.code == "404"
             raise NetworkError, "explorer: HTTP #{res.code}" unless res.code == "200"
 
-            return read_capped(res)
+            return Chain.read_capped(res) { |n| "explorer: response exceeds #{n} bytes" }
           end
         end
-      rescue SocketError, SystemCallError, Net::OpenTimeout, Net::ReadTimeout,
+      rescue SocketError, SystemCallError, Timeout::Error, Net::ProtocolError,
              OpenSSL::SSL::SSLError, IOError => e
         raise NetworkError, "explorer: #{e.class}: #{e.message}"
       end
+    end
+
+    # Your own Bitcoin Core node over JSON-RPC. Trusts nothing external: the
+    # merkle root and time come straight from the block header your node holds.
+    #
+    #   Chain::BitcoinCore.new("http://127.0.0.1:8332", user: "rpcuser", password: "secret")
+    #
+    # Credentials may also be embedded in the URL (http://user:pass@host:port) or
+    # read from the node's cookie file (Chain::BitcoinCore.from_cookie(path)).
+    class BitcoinCore
+      # +transport+ is an injection seam for tests: a callable taking the request
+      # body (a JSON string) and returning the response body. It defaults to a
+      # capped Net::HTTP POST with basic auth.
+      def initialize(url = "http://127.0.0.1:8332", user: nil, password: nil, timeout: 20, transport: nil)
+        @uri = URI(url)
+        # userinfo is percent-encoded per RFC 3986, not form-encoded, so "+" is a
+        # literal plus, not a space.
+        @user = user || unescape(@uri.user)
+        @password = password || unescape(@uri.password)
+        @timeout = timeout
+        @transport = transport
+      rescue URI::InvalidURIError => e
+        raise Error, "invalid node URL: #{e.message}"
+      end
+
+      # Build a client from a node's .cookie file (contents are "__cookie__:hex").
+      def self.from_cookie(cookie_path, url = "http://127.0.0.1:8332", **opts)
+        user, password = File.read(cookie_path).strip.split(":", 2)
+        raise Error, "malformed cookie file #{cookie_path}" unless user && password
+
+        new(url, user: user, password: password, **opts)
+      end
+
+      def block_merkle_root_and_time(height)
+        hash = rpc("getblockhash", [Integer(height)])
+        raise NetworkError, "bitcoind returned an invalid block hash" unless hash.is_a?(String)
+
+        header = rpc("getblockheader", [hash])
+        raise NetworkError, "bitcoind returned unexpected data" unless header.is_a?(Hash)
+
+        [Chain.root_display_to_internal(header.fetch("merkleroot")), Time.at(Integer(header.fetch("time"))).utc]
+      rescue KeyError, TypeError, ArgumentError => e
+        raise NetworkError, "bitcoind returned unexpected data: #{e.class}"
+      end
+
+      private
+
+      def unescape(str)
+        str && URI::DEFAULT_PARSER.unescape(str)
+      end
+
+      # RPC error codes that mean "this block does not exist" rather than a fault.
+      BLOCK_NOT_FOUND_CODES = [-8, -5].freeze # height out of range / block not found
+
+      def rpc(method, params)
+        id = SecureRandom.hex(8)
+        body = JSON.generate(jsonrpc: "1.0", id: id, method: method, params: params)
+        parsed = JSON.parse(post(body))
+        raise NetworkError, "bitcoind returned a non-object JSON-RPC response" unless parsed.is_a?(Hash)
+
+        # Handle bitcoind's own error before the id check: a request-level error
+        # (parse error, bad params) comes back with "id": null, and we want its
+        # message, not an id-mismatch complaint.
+        if (error = parsed["error"])
+          code = error.is_a?(Hash) ? error["code"] : nil
+          message = error.is_a?(Hash) ? error["message"] : error
+          raise BlockNotFound, "bitcoind: #{message}" if BLOCK_NOT_FOUND_CODES.include?(code)
+
+          raise NetworkError, "bitcoind error: #{message}"
+        end
+
+        # A per-call random id echoed back mismatching means the response was not
+        # for this call (a stale or mixed-up connection).
+        raise NetworkError, "bitcoind response id mismatch" if parsed.key?("id") && parsed["id"] != id
+
+        parsed.fetch("result")
+      rescue JSON::ParserError => e
+        raise NetworkError, "bitcoind returned non-JSON: #{e.class}"
+      end
+
+      def post(body)
+        return @transport.call(body) if @transport
 
-      def read_capped(res)
-        buffer = +"".b
-        res.read_body do |chunk|
-          buffer << chunk
-          raise NetworkError, "explorer: response exceeds #{MAX_RESPONSE_BYTES} bytes" if buffer.bytesize > MAX_RESPONSE_BYTES
+        req = Net::HTTP::Post.new(@uri)
+        req["Content-Type"] = "application/json"
+        req.basic_auth(@user, @password) if @user
+        req.body = body
+
+        http = Net::HTTP.new(@uri.host, @uri.port)
+        http.use_ssl = (@uri.scheme == "https")
+        http.open_timeout = @timeout
+        http.read_timeout = @timeout
+
+        http.start do |conn|
+          conn.request(req) do |res|
+            # bitcoind answers 200 on success and 500 with a JSON error body for
+            # RPC-level errors (parsed in #rpc). Any other status is a transport or
+            # auth failure, not a usable response.
+            raise NetworkError, "bitcoind: HTTP #{res.code}" unless %w[200 500].include?(res.code)
+
+            return Chain.read_capped(res) { |n| "bitcoind: response exceeds #{n} bytes" }
+          end
         end
-        buffer
+      rescue SocketError, SystemCallError, Timeout::Error, Net::ProtocolError,
+             OpenSSL::SSL::SSLError, IOError => e
+        raise NetworkError, "bitcoind: #{e.class}: #{e.message}"
+      end
+    end
+
+    # Stream a response body, capping its size. The block builds the error message
+    # so each caller names itself.
+    def self.read_capped(res)
+      buffer = +"".b
+      res.read_body do |chunk|
+        buffer << chunk
+        raise NetworkError, yield(MAX_RESPONSE_BYTES) if buffer.bytesize > MAX_RESPONSE_BYTES
       end
+      buffer
     end
   end
 end

data/lib/opentimestamps/error.rb

@@ -14,4 +14,10 @@ module OpenTimestamps
 
   # Raised when a calendar or chain oracle cannot be reached or answers badly.
   class NetworkError < Error; end
+
+  # Raised by a chain oracle when the block a proof references does not exist (a
+  # height past the chain tip, or a bogus attestation). This is a fact about the
+  # proof, not an outage, so verification treats it as "not anchored" rather than
+  # as an unreachable oracle.
+  class BlockNotFound < Error; end
 end

data/lib/opentimestamps/operation.rb

@@ -18,7 +18,10 @@ module OpenTimestamps
     def initialize(kind, arg = nil)
       raise Error, "unknown op #{kind}" unless TAG.key?(kind)
       raise Error, "#{kind} takes no argument" if arg && UNARY.value?(kind)
-      raise Error, "#{kind} requires an argument" if arg.nil? && BINARY.value?(kind)
+      # A binary op must carry a non-empty operand: the reference requires
+      # min_len 1, and an empty operand would be an identity step that could pad a
+      # proof with extra tree paths to the same commitment.
+      raise Error, "#{kind} requires a non-empty argument" if BINARY.value?(kind) && (arg.nil? || arg.empty?)
 
       @kind = kind
       @arg = arg&.b
@@ -26,6 +29,10 @@ module OpenTimestamps
 
     def binary? = BINARY.value?(@kind)
 
+    # Canonical sort key matching the reference client: order by tag, then by the
+    # raw operand bytes (empty for a unary op).
+    def sort_key = [TAG.fetch(@kind), @arg || "".b]
+
     # Apply the operation to a message, returning the new bytes.
     def apply(msg)
       case @kind
@@ -50,7 +57,10 @@ module OpenTimestamps
 
     def self.deserialize(reader, tag)
       if BINARY.key?(tag)
-        new(BINARY[tag], reader.varbytes)
+        arg = reader.varbytes
+        raise DeserializationError, "#{BINARY[tag]} op with empty argument" if arg.empty?
+
+        new(BINARY[tag], arg)
       elsif UNARY.key?(tag)
         new(UNARY[tag])
       else

data/lib/opentimestamps/timestamp.rb

@@ -34,7 +34,10 @@ module OpenTimestamps
 
     def self.read_child(reader, node, tag, depth)
       if tag == 0x00
-        node.attestations << Attestation.deserialize(reader)
+        att = Attestation.deserialize(reader)
+        # The reference client keeps attestations in a set; mirror that so a proof
+        # cannot carry the same attestation twice (which would inflate a quorum).
+        node.attestations << att unless node.attestations.include?(att)
       else
         op = Op.deserialize(reader, tag)
         child_msg = op.apply(node.msg)
@@ -45,17 +48,22 @@ module OpenTimestamps
     end
     private_class_method :read_child
 
+    # Children are written in a canonical order (attestations then ops), each
+    # sorted by the same key the reference client uses: ops by [tag, raw operand],
+    # attestations by [tag, type-specific field] (Bitcoin by height, Pending by
+    # uri). A proof `ots` produced is already in this order, so re-serializing it
+    # is byte-exact; a proof we built or merged is normalized to match `ots`.
     def serialize(writer)
       total = @attestations.size + @ops.size
       raise Error, "timestamp node has no children" if total.zero?
 
       written = 0
-      @attestations.each do |att|
+      @attestations.sort_by(&:sort_key).each do |att|
         writer.u8(0xff) if (written += 1) < total
         writer.u8(0x00)
         att.serialize(writer)
       end
-      @ops.each do |op, child|
+      @ops.sort_by { |op, _| op.sort_key }.each do |op, child|
         writer.u8(0xff) if (written += 1) < total
         op.serialize(writer)
         child.serialize(writer)
@@ -98,13 +106,25 @@ module OpenTimestamps
     end
 
     # Checks every Bitcoin attestation against the chain oracle. Returns one
-    # { height:, time:, verified: } entry per Bitcoin attestation.
+    # { height:, time:, verified:, commitment:, error: } entry per Bitcoin
+    # attestation. The commitment lets a caller collapse several tree paths that
+    # prove the same anchor (same height and root) into one; +error+ is set when
+    # the oracle could not be reached for that block, so the caller can tell "not
+    # anchored" apart from "could not check".
     def verify(chain)
       each_attestation.filter_map do |commitment, att|
         next unless att.bitcoin?
 
-        root, time = chain.block_merkle_root_and_time(att.height)
-        { height: att.height, time: time, verified: commitment == root }
+        begin
+          root, time = chain.block_merkle_root_and_time(att.height)
+          { height: att.height, time: time, verified: commitment == root, commitment: commitment, error: nil }
+        rescue BlockNotFound
+          # The block does not exist: this attestation simply does not verify, the
+          # same as a root mismatch, so it carries no outage error.
+          { height: att.height, time: nil, verified: false, commitment: commitment, error: nil }
+        rescue NetworkError => e
+          { height: att.height, time: nil, verified: false, commitment: commitment, error: e }
+        end
       end
     end
   end

data/lib/opentimestamps/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module OpenTimestamps
-  VERSION = "0.1.0"
+  VERSION = "0.3.0"
 end

data/lib/opentimestamps.rb

@@ -28,6 +28,10 @@ module OpenTimestamps
   # One confirmed Bitcoin attestation: the block that anchors +digest+, and when.
   Verification = Struct.new(:height, :time, :digest, keyword_init: true)
 
+  # Builds a Calendar from a URL. Overridable in stamp_digest / upgrade as an
+  # injection seam for tests and custom transports.
+  DEFAULT_CALENDAR_FACTORY = ->(url) { Calendar.new(url) }
+
   module_function
 
   # Stamp raw data: hash it, then stamp the digest.
@@ -38,13 +42,14 @@ module OpenTimestamps
   # Stamp an already-computed digest. The content itself never leaves the caller
   # (privacy / "sealed" mode). The digest is submitted to every calendar and the
   # replies are merged, so a single calendar being down is not fatal.
-  def stamp_digest(digest, calendars: DEFAULT_CALENDARS, hash: :sha256, timeout: 20)
+  def stamp_digest(digest, calendars: DEFAULT_CALENDARS, hash: :sha256, timeout: 20,
+                   calendar_factory: DEFAULT_CALENDAR_FACTORY)
     DetachedTimestampFile.from_hash(digest, hash: hash) # validate digest length before any network
 
     merged = nil
     failures = []
     Array(calendars).each do |url|
-      timestamp = Calendar.new(url).submit(digest, timeout: timeout)
+      timestamp = calendar_factory.call(url).submit(digest, timeout: timeout)
       merged ? merged.merge(timestamp) : merged = timestamp
     rescue NetworkError => e
       failures << e.message
@@ -57,38 +62,74 @@ module OpenTimestamps
   # Ask each pending calendar to upgrade to its Bitcoin path, folding the result
   # in. Returns true if anything changed; persist the file afterwards. A calendar
   # that is unreachable is skipped, not fatal.
-  def upgrade(detached, timeout: 20)
-    return false if detached.timestamp.each_attestation.any? { |_, att| att.bitcoin? }
+  def upgrade(detached, timeout: 20, calendar_factory: DEFAULT_CALENDAR_FACTORY)
+    before = detached.serialize
 
-    changed = false
     detached.timestamp.each_attestation.select { |_, att| att.pending? }.each do |commitment, att|
+      node = detached.timestamp.find(commitment)
+      # Skip a calendar whose commitment is already anchored at this leaf, but keep
+      # going for the others: with two calendars, one may anchor days before the
+      # other, and a later upgrade must still fold the second one in (so quorum > 1
+      # can eventually be met).
+      next if node&.attestations&.any?(&:bitcoin?)
+
       upgraded = begin
-        Calendar.new(att.uri).upgrade(commitment, timeout: timeout)
+        calendar_factory.call(att.uri).upgrade(commitment, timeout: timeout)
       rescue NetworkError
         nil
       end
-      next unless upgraded
-
-      detached.timestamp.find(commitment)&.merge(upgraded)
-      changed = true
+      node&.merge(upgraded) if upgraded
     end
-    changed
+
+    detached.serialize != before
   end
 
-  # Verify against the chain, failing closed: raises VerificationError unless at
-  # least one Bitcoin attestation matches its block. Returns the confirmed
-  # attestations, each carrying the proven digest to compare with your document.
-  def verify(detached, chain: Chain::Explorer.new)
+  # Verify against the chain, failing closed: raises VerificationError unless the
+  # proof is anchored in at least +quorum+ distinct Bitcoin blocks. Returns one
+  # confirmed attestation per distinct block, each carrying the proven digest to
+  # compare with your document.
+  #
+  # +quorum+ counts distinct block heights, not raw attestations: several tree
+  # paths that prove the same anchor collapse to one, so a single anchor (however
+  # a calendar dresses it up) can never satisfy quorum > 1. Requiring quorum > 1
+  # therefore means the document is provably in that many separate blocks.
+  def verify(detached, chain: Chain::Explorer.new, quorum: 1)
+    raise Error, "quorum must be a positive integer" unless quorum.is_a?(Integer) && quorum >= 1
+
     digest = detached.file_digest
-    confirmed = detached.timestamp.verify(chain).select { |result| result[:verified] }
-    raise VerificationError, "not anchored in Bitcoin (still pending, or the proof does not match)" if confirmed.empty?
+    results = detached.timestamp.verify(chain)
+    confirmed = results.select { |result| result[:verified] }
+                       .uniq { |result| [result[:height], result[:commitment]] }
+    confirmed_heights = confirmed.map { |result| result[:height] }.uniq
+
+    if confirmed_heights.size < quorum
+      outage_heights = results.select { |result| result[:error] }.map { |result| result[:height] }.uniq
+      # The best case even if every unreachable block eventually confirmed. If that
+      # still falls short, the quorum is unreachable for this proof no matter the
+      # oracle, so it is a verification failure, not an outage.
+      if (confirmed_heights | outage_heights).size < quorum
+        raise VerificationError,
+              "not anchored in #{quorum} distinct Bitcoin block(s) " \
+              "(confirmed #{confirmed_heights.size}; still pending, or the proof does not match)"
+      end
+
+      # A reachable quorum is only blocked by the oracle being down for some blocks:
+      # a transient outage, carried with its cause so the real reason is not lost.
+      cause = results.filter_map { |result| result[:error] }.first
+      raise NetworkError,
+            "chain oracle unreachable for #{(outage_heights - confirmed_heights).size} block(s); " \
+            "confirmed #{confirmed_heights.size} of #{quorum} required (#{cause&.message})",
+            cause: cause
+    end
 
     confirmed.map { |result| Verification.new(height: result[:height], time: result[:time], digest: digest) }
   end
 
-  # Boolean form of verify that never raises.
-  def verified?(detached, chain: Chain::Explorer.new)
-    !verify(detached, chain: chain).empty?
+  # Boolean form of verify: returns false instead of raising when a proof does
+  # not verify. A misused +quorum+ (Error) or an unreachable chain (NetworkError)
+  # still propagates, since those are not verification outcomes.
+  def verified?(detached, chain: Chain::Explorer.new, quorum: 1)
+    !verify(detached, chain: chain, quorum: quorum).empty?
   rescue VerificationError
     false
   end
@@ -110,4 +151,36 @@ module OpenTimestamps
     end
   end
   private_class_method :digest_for
+
+  # Hash +data+ with a given file-hash op kind (:sha1, :ripemd160, :sha256), used
+  # to bind a proof to a document. Unlike digest_for, this covers every op a
+  # `.ots` file can carry as its file hash.
+  def digest_for_kind(data, kind)
+    raise Error, "#{kind.inspect} is not a valid file-hash op" unless DetachedTimestampFile::HASH_OPS.key?(kind)
+
+    Op.new(kind).apply(data)
+  end
+
+  # Stream a file through a file-hash op, so a large file is never read whole into
+  # memory. Same result as digest_for_kind(File.binread(path), kind).
+  def digest_file(path, kind)
+    raise Error, "#{kind.inspect} is not a valid file-hash op" unless DetachedTimestampFile::HASH_OPS.key?(kind)
+
+    digest = case kind
+             when :sha256 then Digest::SHA256.new
+             when :sha1   then Digest::SHA1.new
+             when :ripemd160
+               begin
+                 OpenSSL::Digest.new("RIPEMD160")
+               rescue OpenSSL::Digest::DigestError => e
+                 raise Error, "ripemd160 unavailable (enable the OpenSSL legacy provider): #{e.message}"
+               end
+             end
+    File.open(path, "rb") do |file|
+      while (chunk = file.read(64 * 1024))
+        digest.update(chunk)
+      end
+    end
+    digest.digest
+  end
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: opentimestamps
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.0
 platform: ruby
 authors:
 - Artem Kolesnikov
 autorequire:
-bindir: bin
+bindir: exe
 cert_chain: []
-date: 2026-09-02 00:00:00.000000000 Z
+date: 2026-09-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: minitest
@@ -45,13 +45,15 @@ description: |
   Proofs are self-verifying and outlive both this gem and any single server.
 email:
 - sirruf@me.com
-executables: []
+executables:
+- ots
 extensions: []
 extra_rdoc_files: []
 files:
 - CHANGELOG.md
 - LICENSE
 - README.md
+- exe/ots
 - lib/opentimestamps.rb
 - lib/opentimestamps/attestation.rb
 - lib/opentimestamps/calendar.rb