ankusa-sdk 0.3.0 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 99e44d64d5de0bc0b009a85ef6108009f93c42c90a4b28d39c8d39426421d588
-  data.tar.gz: 5084fce56a8a86cc4e1259c1b96a25c2fb1fa599b42e21cbc0248e7f179bd0b8
+  metadata.gz: 817a62bfe2b9d6f6f4dc513d32cb6a4e4091de4e79bd37ef9b5432a44eca6a60
+  data.tar.gz: 74ac90d9dccf5b681519f6870221bcddc11a96d015d945cffc68200606e2c808
 SHA512:
-  metadata.gz: 25f553755bd7f33506aba3992f103c69a82e3e1fa11ffd81909f4a4e4df8fb5a649d0eaa658ec285dffac01dfb6f14b3d057257d8f55ec7b6d667c54b51856b6
-  data.tar.gz: bf270c6e0dcec34696106eaf96cb320004094126f8390e42ff84cd065d874c26b01fc653462aae47eb3960b4ab5a174f49d0e2bbae9fb7f339e4acf1f0956302
+  metadata.gz: ca0aa04bed9460cc20cc1db4b262ff711c65762b284b60486bdcf49825800e03e4173ca319553eaa3ab21f4151aa68fce7a5350e861bb3626d29e48028c02b91
+  data.tar.gz: 290a4cd6343778e9236772c3394877302ce97180c751c80a80420127698bfede89a859c413ea919280870bf57f355f6019628740cf4427c76ad2a39e70e1457b

data/CHANGELOG.md

@@ -6,6 +6,43 @@ this project follows [Semantic Versioning](https://semver.org/).
 
 ## [Unreleased]
 
+## [0.5.0] - 2026-10-14
+
+### Added
+
+- `Ankusa.verify_signature(headers, body, secrets, tolerance_seconds: 300, now: nil)`
+  verifies the Standard Webhooks signature an HTTP sink with a `secret` adds,
+  comparing with `OpenSSL.fixed_length_secure_compare`; failures raise
+  `Ankusa::InvalidSignatureError` (`code`, `field`, never retryable).
+- `Ankusa::Message` and `Ankusa.decode_message(data)`, plus
+  `Ankusa::InvalidMessageError` (with `code`, `field` and `retryable?` false):
+  decode and validate the v1 queue message every sink delivers — the inline
+  (`body_base64`) and claim-check (`claim`) body forms, `sha256`,
+  `dedupe_key`, `replay_id`, and the forwarded `headers`, ignoring unknown
+  keys.
+- `Message#idempotency_key` / `HookHeaders#idempotency_key`, the key to store
+  in a processed-ids table: the key Ankusa shipped (the message's
+  `idempotency_key` field, or the `x-ankusa-idempotency-key` header), and for a
+  message or delivery that predates it `tenant:source_id:dedupe_key` (tenant
+  `default` when there is none) when a `dedupe_key` is set, else `id`, with an
+  optional `#replay:<replay_id>` suffix (`include_replay: true`). The decoded
+  field stays in `to_h[:idempotency_key]`.
+- `HookHeaders#dedupe_key` / `#replay_id` / `#idempotency_key`'s field, parsed
+  from `x-ankusa-dedupe-key` / `x-ankusa-replay-id` / `x-ankusa-idempotency-key`.
+- `AdminClient#create_replay`, `#get_replay`, `#list_replays` and
+  `#update_replay` for the `POST/GET/PATCH /v1/replays` replay-job API (kinds
+  `dlq`, `archive`, `quarantine`).
+
+### Changed
+
+- `ClaimCheckClient#redeem` raises `ClaimCheckUnavailableError` (retryable)
+  for a gateway `408` or `429` instead of `ClaimRejectedError`.
+
+### Removed
+
+- `AdminClient#replay_dead_letters`, replaced by the replay-job API above.
+  `POST /v1/dlq/replay` no longer exists.
+
 ## [0.3.0] - 2026-10-01
 
 ### Added
@@ -37,5 +74,6 @@ this project follows [Semantic Versioning](https://semver.org/).
   family, which carries `.status` and `.body` instead), so a consumer needs
   exactly one bit to route a failure to dead-letter or retry.
 
-[Unreleased]: https://github.com/jamescarr/ankusa/compare/sdk-ruby-v0.3.0...HEAD
+[Unreleased]: https://github.com/jamescarr/ankusa/compare/sdk-ruby-v0.5.0...HEAD
+[0.5.0]: https://github.com/jamescarr/ankusa/compare/sdk-ruby-v0.3.0...sdk-ruby-v0.5.0
 [0.3.0]: https://github.com/jamescarr/ankusa/releases/tag/sdk-ruby-v0.3.0

data/README.md

@@ -5,8 +5,8 @@ deployments: one gem meant to bundle everything a non-Elixir consumer needs to
 talk to an Ankusa deployment. Today that's the
 [claim-check gateway](https://github.com/jamescarr/ankusa/blob/main/docs/claim-check.md)
 client, the route-management and operator (admin) clients, the
-source-management client, and a webhook-receiving header helper; more clients
-(ingest) land here as they're built.
+source-management client, a webhook-receiving header helper, and the queue
+message decoder; more clients (ingest) land here as they're built.
 
 ## Install
 
@@ -81,9 +81,9 @@ consumer needs exactly one bit to decide dead-letter vs. retry:
 | --- | --- | --- |
 | `InvalidClaimRefError` | `false` | `ref` isn't a well-formed claim-check URN, or `sha256` isn't 64-char lowercase hex |
 | `ClaimNotFoundError` | `false` | gateway `404`: expired by retention, or never written |
-| `ClaimRejectedError` | `false` | gateway `4xx` other than `404` (`.status`, `.body`) |
+| `ClaimRejectedError` | `false` | gateway `4xx` other than `404`, `408`, `429` (`.status`, `.body`) |
 | `ClaimIntegrityError` | `false` | sha256 of the returned bytes doesn't match |
-| `ClaimCheckUnavailableError` | `true` | gateway `5xx`/`503`, or unreachable |
+| `ClaimCheckUnavailableError` | `true` | gateway `5xx`, `408`, `429`, or unreachable |
 
 `health` hits `GET /health` for a liveness probe. There's no `close`: the
 default transport opens one connection per request.
@@ -124,11 +124,103 @@ def call(env)
 end
 ```
 
-`HookHeaders#id` is what a receiver dedupes on: delivery is at-least-once (see
+`HookHeaders#idempotency_key` is what a receiver dedupes on, not the hook id
+(`x-ankusa-id` stays required): delivery is at-least-once (see
 "HTTP handoff" in
 [`docs/integrations.md`](https://github.com/jamescarr/ankusa/blob/main/docs/integrations.md)),
-so the same hook can arrive twice after a retry. Header lookup is always
-case-insensitive, regardless of whether the mapping passed in already is.
+so the same hook can arrive twice after a retry. When the source extracted the
+provider's own event key, `HookHeaders#dedupe_key` carries it (and `replay_id`
+marks a replay); `HookHeaders#idempotency_key` returns the tenant-scoped key
+Ankusa computed and shipped in `x-ankusa-idempotency-key` — see "Consuming
+queue messages". Header lookup is always case-insensitive, regardless of
+whether the mapping passed in already is.
+
+### Verifying signed deliveries
+
+An HTTP sink with a `secret` signs every delivery the
+[Standard Webhooks](https://www.standardwebhooks.com/) way:
+
+```ruby
+# Inside a Rack app's `call(env)`, before anything parses the body:
+begin
+  body = env["rack.input"].read # the raw request body, exactly as received
+  Ankusa.verify_signature(header_map(env), body, [ENV.fetch("ANKUSA_WHSEC")])
+rescue Ankusa::InvalidSignatureError => e
+  return [401, {"content-type" => "application/json"},
+    [JSON.generate(error: "invalid_signature", code: e.code)]]
+end
+```
+
+Secrets are `whsec_` + base64, or any other string used as its own bytes;
+pass several during a rotation. `tolerance_seconds:` (default 300) bounds the
+`webhook-timestamp` window. Failures are `Ankusa::InvalidSignatureError` with
+`code` and `field`, never retryable.
+
+## Consuming queue messages
+
+Every message sink — RabbitMQ, Kafka, NATS, Redis — delivers one JSON message
+per hook (the HTTP sink sends the original body plus `x-ankusa-*` headers
+instead):
+the identity fields, the body (inline `body_base64` or a claim-check `claim`),
+`sha256`, and, when present, `dedupe_key`, `replay_id`, `idempotency_key` and
+the forwarded provider `headers`. `Ankusa.decode_message` validates all of it
+and `Message#idempotency_key` gives the value to store in a processed-ids
+table:
+
+```ruby
+require "ankusa/sdk"
+
+CLAIM_CHECK = Ankusa::ClaimCheckClient.new(ENV.fetch("CLAIM_CHECK_URL", "http://localhost:4001"))
+DB = PG.connect(ENV.fetch("DATABASE_URL"))
+DB.exec("CREATE TABLE IF NOT EXISTS processed (key text primary key)")
+
+def consume(raw)
+  begin
+    message = Ankusa.decode_message(raw)
+  rescue Ankusa::InvalidMessageError => e
+    # Poison message: dead-letter, never requeue. `e.retryable?` is false;
+    # `e.code` is e.g. "integrity"; `e.field` names the offending key.
+    raise
+  end
+
+  key = message.idempotency_key   # the key Ankusa shipped (tenant:source_id:dedupe_key, else id)
+  return if DB.exec_params("SELECT 1 FROM processed WHERE key = $1", [key]).any?
+
+  body = message.claim ? CLAIM_CHECK.redeem(message.claim, message.sha256) : message.body
+  handle(body)
+  DB.exec_params("INSERT INTO processed (key) VALUES ($1)", [key])
+end
+```
+
+- `idempotency_key` is the key Ankusa computed once for the hook:
+  `tenant:source_id:dedupe_key` when the source extracted the provider's own
+  event key, else `id` — so a provider retry that arrives with a fresh Ankusa
+  `id` still collapses onto the same row, and two tenants that share a provider
+  event id do not. For a message from a node that predates the field the
+  helper computes the same key itself (tenant `default` when there is none).
+  The decoded field alone is `message.to_h[:idempotency_key]`.
+- A replayed delivery is dropped by default. To reprocess replays instead, pass
+  `include_replay: true`: the key then ends in `#replay:<replay_id>`.
+- `message.body` is the decoded inline bytes (nil for the claim form);
+  `message.headers` holds the forwarded provider request headers (lowercased).
+  `message.sha256` is checked against the inline bytes by `decode_message`
+  itself, and against the claim's bytes by `CLAIM_CHECK.redeem`.
+- `decode_message` raises `InvalidMessageError` before you act on a message:
+  bad JSON, a non-object, an unsupported `v`, a bad field type, both or neither
+  body form, invalid base64, a length/sha256 mismatch, or a claim whose tenant
+  differs from `tenant_id`.
+
+A webhook receiver can take the same shortcut straight off the HTTP sink's
+headers with `Ankusa.parse_headers(headers).idempotency_key`, which reads
+`x-ankusa-idempotency-key` (falling back to computing it, where `source` and
+`tenant` play `source_id` and `tenant_id`).
+
+When a sink has grown a backlog, or a downstream processor failed after the
+sink accepted a batch, re-drive it with a replay job over the admin client (see
+"Admin client" below): `"kind" => "dlq"` re-sends rows that dead-lettered,
+`"kind" => "archive"` re-sends hooks over a time window, `"kind" => "quarantine"`
+re-verifies held hooks and releases those that pass. Replays keep the
+original `id` and `dedupe_key` and add `replay_id`.
 
 ## Routes client
 
@@ -162,8 +254,8 @@ unfollowed redirect, a non-JSON success body, or unreachable; retryable).
 ## Admin client
 
 The operator API on `admin.port` (default 4002): health, Prometheus metrics, the
-redacted config, the DLQ, and the quarantine list — the `operations`, `dlq`, and
-`quarantine` tags of `admin.v1.yaml`.
+redacted config, the DLQ, replay jobs, and the quarantine list — the
+`operations`, `dlq`, `replays`, and `quarantine` tags of `admin.v1.yaml`.
 
 ```ruby
 require "ankusa/sdk"
@@ -172,14 +264,24 @@ admin = Ankusa::AdminClient.new(ENV.fetch("ADMIN_URL", "http://localhost:4002"))
 
 admin.health                          # {"status" => "ok", "instance" => ..., "roles" => [...]}
 admin.list_dead_letters({"limit" => 10})  # {"total" => ..., "entries" => [...]}
-admin.replay_dead_letters({"source_id" => "demo"})
 admin.list_quarantined
+
+job = admin.create_replay({"kind" => "dlq", "source_id" => "stripe", "rate" => 500})
+admin.get_replay(job["id"])
+admin.list_replays                    # {"replays" => [Replay, ...]}, newest first
+admin.update_replay(job["id"], {"state" => "paused"})   # resume, pause or cancel
 ```
 
+`create_replay` is idempotent for retries: a second POST of the same filter
+while the job is `running`/`paused` returns the existing job (`200`) instead of
+starting another (`202`). `list_replays` returns `{"replays" => [...]}` exactly
+as the API sends it.
+
 Methods: `health`, `metrics` (Prometheus text), `config`, `list_dead_letters`,
-`replay_dead_letters`, `list_quarantined`. Failures are `AdminError` subclasses:
-`RoleNotEnabledError` (409 `role_not_enabled`, carrying `role`),
-`AdminRejectedError` (any other 4xx, carrying `code`), and
+`create_replay`, `get_replay`, `list_replays`, `update_replay`,
+`list_quarantined`. Failures are `AdminError` subclasses: `RoleNotEnabledError`
+(409 `role_not_enabled`, carrying `role`), `AdminRejectedError` (any other 4xx —
+a missing replay is a 404 with `code` `replay_not_found`), and
 `AdminUnavailableError` (5xx, an unfollowed redirect, or unreachable;
 retryable).
 
@@ -236,15 +338,16 @@ lib/ankusa/
   connection.rb         # shared URL/query/body/JSON rules (@api private)
   webhook.rb            # x-ankusa-* header parsing for HTTP-sink receivers
   claim_check.rb        # the claim-check gateway client
+  message.rb            # decode the v1 queue message + the idempotency key
   routes.rb             # the route-management client (routes.admin.port)
   admin.rb              # the operator client (admin.port)
   sources.rb            # the tenant-scoped source-management client (admin.port)
 test/
   test_helper.rb
-  test_conformance.rb     # runs the language-neutral vectors in conformance/
-  test_routes.rb
-  test_admin.rb
-  test_sources.rb
+  conformance_test.rb     # runs the language-neutral vectors in conformance/
+  routes_test.rb
+  admin_test.rb
+  sources_test.rb
 ```
 
 ## Develop

data/lib/ankusa/admin.rb

@@ -81,12 +81,33 @@ module Ankusa
       json(request("GET", "/v1/dlq", query: params))
     end
 
-    # POST /v1/dlq/replay -> {replayed}.
+    # POST /v1/replays -> the created Replay, as a Hash.
     #
-    # `filter` is a filter, not a payload; omitted or empty (`{}`) replays
-    # everything.
-    def replay_dead_letters(filter = nil)
-      json(request("POST", "/v1/dlq/replay", json: filter || {}))
+    # `spec` is the replay spec: `{"kind" => "dlq", ...}`,
+    # `{"kind" => "archive", "from" => ms, "to" => ms, ...}` or
+    # `{"kind" => "quarantine", ...}`. A retried POST
+    # with the same normalized filter returns the running job with 200 instead
+    # of starting a second one.
+    def create_replay(spec)
+      json(request("POST", "/v1/replays", json: spec))
+    end
+
+    # GET /v1/replays/{id} -> the Replay, as a Hash.
+    def get_replay(id)
+      json(request("GET", "/v1/replays/#{replay_path(id)}"))
+    end
+
+    # GET /v1/replays -> {"replays" => [Replay, ...]}, newest first.
+    def list_replays
+      json(request("GET", "/v1/replays"))
+    end
+
+    # PATCH /v1/replays/{id} -> the updated Replay, as a Hash.
+    #
+    # `patch` may carry `state` (`running`/`paused`/`cancelled`), `rate` and
+    # `max_lag_ms`.
+    def update_replay(id, patch)
+      json(request("PATCH", "/v1/replays/#{replay_path(id)}", json: patch))
     end
 
     # GET /v1/quarantine -> recent quarantined hooks, newest first.
@@ -99,6 +120,12 @@ module Ankusa
 
     private
 
+    # Percent-encodes a replay id as one path segment, so a `/`, `?` or `#` in
+    # an id can't reshape the URL.
+    def replay_path(id)
+      id.to_s.b.gsub(/[^A-Za-z0-9_.~-]/) { |char| format("%%%02X", char.ord) }
+    end
+
     def request(http_method, path, query: nil, json: nil)
       response = @connection.request(http_method, path, query: query, json: json)
       raise_for_status(response)

data/lib/ankusa/claim_check.rb

@@ -11,8 +11,9 @@ module Ankusa
   # right.
   #
   # Non-retryable: the ref or expected sha256 is malformed, the gateway said
-  # 404/other 4xx, or the bytes that came back don't match the sha256.
-  # Retryable: the gateway said 5xx/503, or the request never completed
+  # 404 or another 4xx (except 408/429), or the bytes that came back don't
+  # match the sha256.
+  # Retryable: the gateway said 5xx, 408 or 429, or the request never completed
   # (network error, timeout).
   class ClaimCheckError < Error
     def retryable? = false
@@ -28,7 +29,7 @@ module Ankusa
   class ClaimNotFoundError < ClaimCheckError
   end
 
-  # The gateway rejected the request (400 or any other non-404 4xx).
+  # The gateway rejected the request (400 or any other 4xx but 404, 408 and 429).
   class ClaimRejectedError < ClaimCheckError
     attr_reader :status, :body
 
@@ -46,7 +47,7 @@ module Ankusa
   class ClaimIntegrityError < ClaimCheckError
   end
 
-  # The gateway is unreachable, or answered 5xx/503. Safe to retry.
+  # The gateway is unreachable, or answered 5xx, 408 or 429. Safe to retry.
   class ClaimCheckUnavailableError < ClaimCheckError
     def retryable? = true
   end
@@ -133,6 +134,13 @@ module Ankusa
         raise ClaimNotFoundError, "claim not found: #{parsed.tenant_id}/#{parsed.claim_id}"
       end
 
+      # A gateway (or a proxy in front of it) that is throttling or timing out
+      # is telling the caller to come back, not that the claim is gone.
+      if [408, 429].include?(status)
+        raise ClaimCheckUnavailableError,
+          "claim-check gateway busy (#{status}): #{Connection.error_body(response).inspect}"
+      end
+
       if status >= 400 && status < 500
         body = Connection.error_body(response)
         raise ClaimRejectedError.new(

data/lib/ankusa/message.rb

@@ -0,0 +1,217 @@
+# frozen_string_literal: true
+
+require "digest"
+require "json"
+
+module Ankusa
+  # Raised by `Ankusa.decode_message` when a queue message fails one of the v1
+  # decode rules. Always worth dead-lettering, never worth retrying: the bytes
+  # will not change on a redelivery.
+  #
+  # `code` names the rule that failed (`invalid_json`, `unsupported_version`,
+  # `invalid_field`, `ambiguous_body`, `missing_body`, `invalid_body_base64`,
+  # `size_mismatch`, `integrity`, `tenant_mismatch`); `field` names the offending
+  # key when the rule is about one, and is nil otherwise.
+  class InvalidMessageError < Error
+    attr_reader :code, :field
+
+    def initialize(message, code:, field: nil)
+      super(message)
+      @code = code
+      @field = field
+    end
+
+    def retryable? = false
+  end
+
+  # The one idempotency-key rule, shared by `Message` and `HookHeaders` so it
+  # lives in one place.
+  #
+  # The key Ankusa shipped (`shipped`) when it is a non-empty string; otherwise,
+  # for a hook from a node that predates the field, `tenant:source:dedupe_key`
+  # (tenant "default" when nil) for a non-empty `dedupe_key`, else `id`. With
+  # `include_replay` and a `replay_id`, `#replay:<replay_id>` is appended.
+  module IdempotencyKey
+    def self.build(shipped:, tenant:, source:, dedupe_key:, id:, replay_id:, include_replay:)
+      key =
+        if !shipped.nil? && !shipped.empty?
+          shipped
+        elsif !dedupe_key.nil? && !dedupe_key.empty?
+          "#{tenant || "default"}:#{source}:#{dedupe_key}"
+        else
+          id
+        end
+      key += "#replay:#{replay_id}" if include_replay && !replay_id.nil?
+      key
+    end
+  end
+  private_constant :IdempotencyKey
+
+  # One decoded v1 queue message. `body` holds the inline bytes (or nil for the
+  # claim form, where the bytes are fetched with `Ankusa::ClaimCheckClient`);
+  # `claim` holds the ref string in that case. `headers` is the forwarded
+  # provider request headers, `{}` when the message carried none.
+  #
+  # `idempotency_key` is the one value a consumer should dedupe on, as
+  # `Message#idempotency_key` (the field is nil only for a message from a node
+  # that predates it); see `Ankusa.decode_message` for the decode rules.
+  Message = Data.define(
+    :v, :id, :source_id, :tenant_id, :received_at, :content_type, :size,
+    :body, :claim, :sha256, :dedupe_key, :replay_id, :idempotency_key, :headers
+  ) do
+    # `Data.define` made `idempotency_key` the reader of the decoded field; keep
+    # that value reachable here and let the helper below take over the name.
+    # The decoded field itself is still in `to_h[:idempotency_key]`.
+    alias_method :shipped_idempotency_key, :idempotency_key
+    private :shipped_idempotency_key
+
+    # The idempotency key for this message: the key Ankusa shipped in the
+    # `idempotency_key` field when it is non-empty; for a message that predates
+    # the field, `tenant:source_id:dedupe_key` (tenant "default" when there is
+    # none) when a non-empty `dedupe_key` is set, else `id`.
+    #
+    # A replay keeps the original `id` and `dedupe_key` and adds `replay_id`, so
+    # by default a replay produces the same key as the delivery it replays and a
+    # consumer that already processed it drops it. Pass `include_replay: true`
+    # to reprocess replays instead.
+    def idempotency_key(include_replay: false)
+      IdempotencyKey.build(
+        shipped: shipped_idempotency_key, tenant: tenant_id, source: source_id,
+        dedupe_key: dedupe_key, id: id, replay_id: replay_id, include_replay: include_replay
+      )
+    end
+  end
+
+  # The optional string-keyed fields that may be a string, null, or absent.
+  # Checked in this order, so the first offending key is the one reported.
+  MESSAGE_STRING_FIELDS = %w[tenant_id content_type dedupe_key replay_id idempotency_key].freeze
+  private_constant :MESSAGE_STRING_FIELDS
+
+  # Decodes one v1 queue message (the JSON body a sink or broker delivers) into
+  # a `Message`, applying every rule in "Consuming queue messages" of the SDK
+  # README in order; the first failure wins.
+  #
+  # Raises `InvalidMessageError` for anything malformed. The `sha256` is checked
+  # against the inline bytes here, but a claim's bytes are only checked by
+  # `ClaimCheckClient#redeem`, after it fetches them.
+  def self.decode_message(data)
+    parsed =
+      begin
+        JSON.parse(data)
+      rescue JSON::ParserError
+        raise InvalidMessageError.new("message is not JSON", code: "invalid_json")
+      end
+
+    unless parsed.is_a?(Hash)
+      raise InvalidMessageError.new("message is not a JSON object", code: "not_an_object")
+    end
+
+    version = parsed["v"]
+    unless version.is_a?(Integer) && version == 1
+      raise InvalidMessageError.new("unsupported message version: #{version.inspect}", code: "unsupported_version")
+    end
+
+    id = parsed["id"]
+    invalid_field("id") unless id.is_a?(String) && !id.empty?
+
+    source_id = parsed["source_id"]
+    invalid_field("source_id") unless source_id.is_a?(String)
+
+    received_at = parsed["received_at"]
+    invalid_field("received_at") unless received_at.is_a?(Integer)
+
+    size = parsed["size"]
+    invalid_field("size") unless size.is_a?(Integer) && size >= 0
+
+    MESSAGE_STRING_FIELDS.each do |field|
+      value = parsed[field]
+      invalid_field(field) unless value.nil? || value.is_a?(String)
+    end
+
+    headers = parsed.key?("headers") ? parsed["headers"] : {}
+    invalid_field("headers") unless headers.is_a?(Hash) && headers.each_value.all? { |value| value.is_a?(String) }
+
+    sha256 = parsed["sha256"]
+    valid_sha256 = sha256.nil? || (sha256.is_a?(String) && SHA256_PATTERN.match?(sha256))
+    invalid_field("sha256") unless valid_sha256
+
+    body_base64 = parsed["body_base64"]
+    claim = parsed["claim"]
+
+    unless body_base64.nil? || claim.nil?
+      raise InvalidMessageError.new("message has both body_base64 and claim", code: "ambiguous_body")
+    end
+
+    if body_base64.nil? && claim.nil?
+      raise InvalidMessageError.new("message has neither body_base64 nor claim", code: "missing_body")
+    end
+
+    body = nil
+    parsed_claim = nil
+
+    if body_base64.nil?
+      parsed_claim = parse_claim(claim)
+      invalid_field("sha256") if sha256.nil?
+    else
+      body = decode_body(body_base64)
+      if body.bytesize != size
+        raise InvalidMessageError.new("body is #{body.bytesize} bytes, size says #{size}", code: "size_mismatch")
+      end
+
+      if !sha256.nil? && Digest::SHA256.hexdigest(body) != sha256
+        raise InvalidMessageError.new("body sha256 does not match", code: "integrity")
+      end
+    end
+
+    tenant_id = parsed["tenant_id"]
+    if !parsed_claim.nil? && !tenant_id.nil? && parsed_claim.tenant_id != tenant_id
+      raise InvalidMessageError.new(
+        "claim tenant #{parsed_claim.tenant_id} does not match tenant_id #{tenant_id}",
+        code: "tenant_mismatch"
+      )
+    end
+
+    Message.new(
+      v: 1,
+      id: id,
+      source_id: source_id,
+      tenant_id: tenant_id,
+      received_at: received_at,
+      content_type: parsed["content_type"],
+      size: size,
+      body: body,
+      claim: claim,
+      sha256: sha256,
+      dedupe_key: parsed["dedupe_key"],
+      replay_id: parsed["replay_id"],
+      idempotency_key: parsed["idempotency_key"],
+      headers: headers
+    )
+  end
+
+  def self.invalid_field(field)
+    raise InvalidMessageError.new("invalid message field: #{field}", code: "invalid_field", field: field)
+  end
+  private_class_method :invalid_field
+
+  def self.decode_body(body_base64)
+    unless body_base64.is_a?(String)
+      raise InvalidMessageError.new("body_base64 is not a string", code: "invalid_body_base64")
+    end
+
+    body_base64.unpack1("m0")
+  rescue ArgumentError
+    raise InvalidMessageError.new("body_base64 is not valid base64", code: "invalid_body_base64")
+  end
+  private_class_method :decode_body
+
+  # A claim that doesn't parse is an invalid *field*, not an invalid claim-ref
+  # error: the message itself is what a consumer rejects, so it carries the same
+  # `invalid_field` code as every other bad key.
+  def self.parse_claim(claim)
+    Ankusa.parse_claim_ref(claim)
+  rescue InvalidClaimRefError
+    invalid_field("claim")
+  end
+  private_class_method :parse_claim
+end

data/lib/ankusa/sdk/version.rb

@@ -2,6 +2,6 @@
 
 module Ankusa
   module SDK
-    VERSION = "0.3.0"
+    VERSION = "0.5.0"
   end
 end

data/lib/ankusa/sdk.rb

@@ -5,7 +5,9 @@ require_relative "error"
 require_relative "transport"
 require_relative "connection"
 require_relative "webhook"
+require_relative "signature"
 require_relative "claim_check"
+require_relative "message"
 require_relative "routes"
 require_relative "admin"
 require_relative "sources"

data/lib/ankusa/signature.rb

@@ -0,0 +1,101 @@
+# frozen_string_literal: true
+
+require "openssl"
+
+module Ankusa
+  # A delivery whose Standard Webhooks signature does not verify. Never
+  # retryable: answer `401`; the sender retries with the same bytes.
+  #
+  # `code` is one of `invalid_secret`, `missing_header`, `invalid_timestamp`,
+  # `timestamp_out_of_tolerance`, `no_matching_signature`; `field` names the
+  # header at fault (nil for `invalid_secret`).
+  class InvalidSignatureError < Error
+    attr_reader :code, :field
+
+    def initialize(message, code:, field: nil)
+      super(message)
+      @code = code
+      @field = field
+    end
+
+    def retryable? = false
+  end
+
+  # A verified delivery's `webhook-id` and `webhook-timestamp`.
+  VerifiedSignature = Data.define(:id, :timestamp)
+
+  BASE64_PATTERN = %r{\A(?:[A-Za-z0-9+/]{4})*(?:[A-Za-z0-9+/]{2}==|[A-Za-z0-9+/]{3}=)?\z}
+  private_constant :BASE64_PATTERN
+
+  # Verifies the Standard Webhooks signature an HTTP sink with a `secret` adds
+  # (https://www.standardwebhooks.com/).
+  #
+  # `webhook-signature` holds space-separated `v1,<base64>` entries, each an
+  # HMAC-SHA256 over `<webhook-id>.<webhook-timestamp>.<body>`. Any `v1` entry
+  # matching any secret (`whsec_` + base64, or any other string used as its own
+  # bytes) passes, compared in constant time, provided `webhook-timestamp` is
+  # within `tolerance_seconds` of `now` (the clock by default). `body` must be
+  # the raw bytes received. Raises `InvalidSignatureError`.
+  def self.verify_signature(headers, body, secrets, tolerance_seconds: 300, now: nil)
+    keys = signature_keys(Array(secrets))
+    lowered = headers.to_h { |key, value| [key.to_s.downcase, value] }
+
+    required = lambda do |name|
+      value = lowered[name]
+      raise InvalidSignatureError.new("missing #{name} header", code: "missing_header", field: name) if value.nil? || value.empty?
+
+      value
+    end
+
+    hook_id = required.call("webhook-id")
+    raw_timestamp = required.call("webhook-timestamp")
+    signature = required.call("webhook-signature")
+
+    unless raw_timestamp.match?(/\A[0-9]+\z/)
+      raise InvalidSignatureError.new("webhook-timestamp is not a unix time",
+        code: "invalid_timestamp", field: "webhook-timestamp")
+    end
+
+    timestamp = Integer(raw_timestamp, 10)
+    current = now || Time.now.to_i
+    if (current - timestamp).abs > tolerance_seconds
+      raise InvalidSignatureError.new("webhook-timestamp is outside the tolerance window",
+        code: "timestamp_out_of_tolerance", field: "webhook-timestamp")
+    end
+
+    signed = "#{hook_id}.#{raw_timestamp}.".b + body.to_s.b
+    candidates = signature.split(" ").filter_map { |entry| entry.delete_prefix("v1,") if entry.start_with?("v1,") }
+
+    keys.each do |key|
+      expected = [OpenSSL::HMAC.digest("SHA256", key, signed)].pack("m0")
+      candidates.each do |candidate|
+        next unless candidate.bytesize == expected.bytesize
+
+        return VerifiedSignature.new(id: hook_id, timestamp: timestamp) if OpenSSL.fixed_length_secure_compare(candidate, expected)
+      end
+    end
+
+    raise InvalidSignatureError.new("no webhook-signature entry matches",
+      code: "no_matching_signature", field: "webhook-signature")
+  end
+
+  def self.signature_keys(secrets)
+    raise InvalidSignatureError.new("no secret configured", code: "invalid_secret") if secrets.empty?
+
+    secrets.map do |secret|
+      if secret.start_with?("whsec_")
+        encoded = secret.delete_prefix("whsec_")
+        unless !encoded.empty? && BASE64_PATTERN.match?(encoded)
+          raise InvalidSignatureError.new("a whsec_ secret is not valid base64", code: "invalid_secret")
+        end
+
+        encoded.unpack1("m0")
+      elsif secret.empty?
+        raise InvalidSignatureError.new("an empty secret", code: "invalid_secret")
+      else
+        secret.b
+      end
+    end
+  end
+  private_class_method :signature_keys
+end

data/lib/ankusa/sources.rb

@@ -21,7 +21,9 @@ module Ankusa
   class SourceNotFoundError < SourcesError
   end
 
-  # 409 source_exists: a source with that name already exists.
+  # 409 source_exists: a source with that name already exists; or
+  # 409 source_has_deliveries: a delete found the source still has undelivered
+  # hooks. The body's "error" says which.
   class SourceConflictError < SourcesError
   end
 
@@ -169,7 +171,10 @@ module Ankusa
     # Delete a source: DELETE /v1/tenants/<tenant>/sources/<name>.
     #
     # Succeeds with no return value (the server answers 204 with an empty body);
-    # a missing source raises `SourceNotFoundError`.
+    # a missing source raises `SourceNotFoundError`. A source that still has
+    # undelivered hooks on the server's node raises `SourceConflictError` whose
+    # body's "error" is "source_has_deliveries" (with "pending"/"inflight"
+    # counts); this method never sends the admin API's ?deliveries=dead_letter.
     def delete_source(tenant, name)
       validate_tenant(tenant)
       validate_name(name)

data/lib/ankusa/webhook.rb

@@ -4,9 +4,36 @@ module Ankusa
   # The identity of one HTTP-sink delivery: the headers Ankusa's HTTP sink
   # attaches to every delivery.
   #
-  # `content_type` and `tenant` are nil when the delivery had no such header
-  # (a tenant only travels when the source has one).
-  HookHeaders = Data.define(:id, :source, :tenant, :content_type)
+  # `content_type`, `tenant`, `dedupe_key`, `replay_id` and `idempotency_key` are
+  # nil when the delivery had no such header (a tenant only travels when the
+  # source has one; `dedupe_key` only when the source declared a dedupe key;
+  # `replay_id` only when the delivery is a replay; `idempotency_key` is
+  # missing only from a sender that predates the header).
+  HookHeaders = Data.define(
+    :id, :source, :tenant, :content_type, :dedupe_key, :replay_id, :idempotency_key
+  ) do
+    # `Data.define` made `idempotency_key` the reader of the parsed header; keep
+    # that value reachable here and let the helper below take over the name.
+    # The parsed header itself is still in `to_h[:idempotency_key]`.
+    alias_method :shipped_idempotency_key, :idempotency_key
+    private :shipped_idempotency_key
+
+    # The idempotency key for this delivery: the key Ankusa shipped in
+    # `x-ankusa-idempotency-key` when it is non-empty; for a delivery that
+    # predates the header, `tenant:source:dedupe_key` (tenant "default" when
+    # there is none) when a non-empty `dedupe_key` is set, else `id`.
+    #
+    # A replay keeps the original `id` and `dedupe_key` and adds `replay_id`, so
+    # by default a replay produces the same key as the delivery it replays and a
+    # receiver that already processed it drops it. Pass `include_replay: true`
+    # to reprocess replays instead.
+    def idempotency_key(include_replay: false)
+      IdempotencyKey.build(
+        shipped: shipped_idempotency_key, tenant: tenant, source: source,
+        dedupe_key: dedupe_key, id: id, replay_id: replay_id, include_replay: include_replay
+      )
+    end
+  end
 
   # Raised by `Ankusa.parse_headers` when `x-ankusa-id` is absent or empty.
   #
@@ -32,7 +59,17 @@ module Ankusa
       id: hook_id,
       source: lowered.fetch("x-ankusa-source", ""),
       tenant: lowered["x-ankusa-tenant"],
-      content_type: lowered["content-type"]
+      content_type: lowered["content-type"],
+      dedupe_key: presence(lowered["x-ankusa-dedupe-key"]),
+      replay_id: presence(lowered["x-ankusa-replay-id"]),
+      idempotency_key: presence(lowered["x-ankusa-idempotency-key"])
     )
   end
+
+  # An empty header counts as absent: Ankusa only sends these when it has a
+  # value, and a proxy that rewrites a header to "" should not fabricate one.
+  def self.presence(value)
+    (value.nil? || value.empty?) ? nil : value
+  end
+  private_class_method :presence
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: ankusa-sdk
 version: !ruby/object:Gem::Version
-  version: 0.3.0
+  version: 0.5.0
 platform: ruby
 authors:
 - James Carr
@@ -24,9 +24,11 @@ files:
 - lib/ankusa/claim_check.rb
 - lib/ankusa/connection.rb
 - lib/ankusa/error.rb
+- lib/ankusa/message.rb
 - lib/ankusa/routes.rb
 - lib/ankusa/sdk.rb
 - lib/ankusa/sdk/version.rb
+- lib/ankusa/signature.rb
 - lib/ankusa/sources.rb
 - lib/ankusa/transport.rb
 - lib/ankusa/webhook.rb