open-loam 0.3.0 → 0.4.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 48bb3940900ff3d8ee4f6a8dc499779b3d00ed9ff37ceeab4849d36662377a47
4
- data.tar.gz: 65440f973fc99d793ae2439cb1a1b373cd7f79d41c82398fd8d59c6e4dcba784
3
+ metadata.gz: 2617b3b1e2ebbdb4b37164f3f0544cdd8ae2130c40d87bda37f535ae08aa05c7
4
+ data.tar.gz: b04798cd19ac5aa56d73c362d848e65220ac61857abd878889b8f0d9aae1def4
5
5
  SHA512:
6
- metadata.gz: a091472e49805c7fffe3aa95a6ddfd0ffc02fea5c7e8d368443be195854e5bdd87b4b7b7635d4c7024842d4a62015d55e6a63064009d13cce8158fff3b7bb207
7
- data.tar.gz: 10546c0bc5b0e6facacf2ede136cd930aa3851c833ae05aa908f990bc60ea02db3efccf1483b5577c0ba4d5cf7136b369c31956a3d1c989444824a1f87836c08
6
+ metadata.gz: d0663a2e2b2145fdc208e614d0b2c468a910125bcee3dcc9709553f85d1eb56c377829dfba630d2c3cd01545e52bac5c3d1568e074d71c3e461fd2a7568d6844
7
+ data.tar.gz: 998ad613e6df75073714b95d6c237d007176861dd47b493e7b79568bd46354fdba9ba4250516eb06516277e6db473f6d1122f8047a012be6cbe126f2dece8de1
data/CHANGELOG.md CHANGED
@@ -5,6 +5,68 @@ with the caveat that 0.x releases may break the public surface; the
5
5
  [backward-compatibility contract](BACKWARD_COMPATIBILITY.md) names what is
6
6
  frozen and what is not.
7
7
 
8
+ ## 0.4.0 — 2026-11-20
9
+
10
+ ### Fixed
11
+
12
+ - **Outbound webhooks dropped data on any receiver error.** The response code
13
+ was never read, so a 500 or a timeout lost the event. Deliveries are now rows
14
+ (`open_loam_webhook_deliveries`) retried by response code.
15
+ - **SSRF through IPv4-mapped IPv6 literals** (`http://[::ffff:127.0.0.1]/`,
16
+ `[::ffff:169.254.169.254]`). `OutboundUrl` judges mapped and compatible
17
+ addresses by the IPv4 ranges and blocks NAT64 (`64:ff9b::/96`,
18
+ `64:ff9b:1::/48`).
19
+
20
+ ### Added
21
+
22
+ - Webhook delivery ledger, retry schedule, `Retry-After`, auto-disable,
23
+ `open_loam.webhook.endpoint_disabled` event, sweep job, retention prune,
24
+ `send_test!`, `redeliver!` (finished deliveries only; resets `attempts`).
25
+ - Auto-disable needs BOTH `disable_after_failures` (default 50) failures in a row
26
+ AND a streak older than `disable_after_duration` (default 24h; new
27
+ `failing_since` column). 410 and a blocked URL still disable at once. Errors
28
+ raised by a host `body_builder` (other than `SkipDelivery`) are retried without
29
+ counting against the endpoint.
30
+ - The transport bounds the whole request at 3 x `timeout`, ignores `http_proxy`,
31
+ and stores a UTF-8-safe, NUL-free excerpt. A stuck row past the retry schedule
32
+ is failed (`last_error: "stuck"`) by the sweep instead of resent.
33
+ - The admin shows a new endpoint's signing secret once, on the page rendered
34
+ after create (`webhook_endpoints/created.html.erb`, `Cache-Control: no-store`).
35
+ - `<prefix>-Signature-V2` (timestamped) plus `-Event-Id`, `-Delivery`,
36
+ `-Timestamp` headers; configurable prefix and user agent. The transport sends
37
+ `Accept-Encoding: identity` and reads at most 512 bytes of the response.
38
+ - Secret rotation with a grace period; optional sealing of secrets
39
+ (`encrypt_secrets`).
40
+ - `event_patterns` (several per endpoint), `metadata` and `description` on
41
+ endpoints; host `body_builder` and `SkipDelivery`.
42
+ - `OutboundUrl` `ports:` allowlist; `require_https`/`allowed_ports` for webhooks.
43
+ - `OpenLoam::OutboundUrl::UnresolvableError` (a `BlockedError` subclass) for a
44
+ host that resolves to nothing. Webhook deliveries retry it like a network
45
+ error; only a URL that resolves to a non-public address fails and disables the
46
+ endpoint (`blocked_url`).
47
+
48
+ #### Migration
49
+
50
+ Copy `demo/db/migrate/20261120090000_add_open_loam_webhook_reliability.rb`.
51
+ Existing endpoints keep working unchanged (single pattern, plaintext secret).
52
+ Jobs enqueued by 0.3.0 are still accepted. Apps that do not tick
53
+ `OpenLoam::Scheduler` must schedule `OpenLoam::WebhookSweepJob` every minute.
54
+ An error raised by a subscriber to `open_loam.webhook.endpoint_disabled`
55
+ propagates out of the delivery job rather than being retried.
56
+ The admin endpoints index no longer prints the secret; re-copy it if you
57
+ generated it. Generated apps get a new `admin/webhook_endpoints/created.html.erb`
58
+ and an updated controller (create renders it with status 201).
59
+
60
+ The migration adds `open_loam_webhook_endpoints.failing_since` and indexes on
61
+ `open_loam_webhook_deliveries` `(status, next_attempt_at)` and
62
+ `(status, updated_at)` for the per-minute cross-tenant sweep lookup. It is tagged
63
+ `Migration[8.2]` and uses bigint keys: adjust the version for your Rails, and the
64
+ `references ... type:` for uuid or string primary keys.
65
+
66
+ With `require_https` or `allowed_ports` switched on, an existing `http://` or
67
+ non-allowed-port endpoint fails its first delivery and is disabled as
68
+ `blocked_url`. Review existing endpoints before turning these on.
69
+
8
70
  ## 0.3.0 — 2026-09-05
9
71
 
10
72
  ### Security
@@ -2,23 +2,16 @@ require "net/http"
2
2
  require "openssl"
3
3
 
4
4
  module OpenLoam
5
- # Delivers one event to one endpoint. Inherits ActiveJob::Base rather than
6
- # the host app's ApplicationJob: the gem must not depend on a class the app
7
- # owns and may have configured for its own retries.
5
+ # Runs one attempt for one ledger row (OpenLoam::WebhookDelivery). Inherits
6
+ # ActiveJob::Base rather than the host's ApplicationJob, and keeps no retry
7
+ # state: OpenLoam::Webhooks.deliver moves the row and schedules the next try.
8
8
  #
9
- # Signing: the receiver recomputes HMAC-SHA256 of the exact body with the
10
- # endpoint's secret and compares it to X-OpenLoam-Signature. Body building and
11
- # signing are class methods so both sides — and the tests — can call them
12
- # without a network.
9
+ # The 4-argument form is what 0.3.0 enqueued (endpoint id, event, payload); a
10
+ # job still in the queue at upgrade time becomes a ledger row and is delivered.
13
11
  class WebhookDeliveryJob < ActiveJob::Base
14
12
  queue_as :default
15
13
 
16
- # A URL that points inside the perimeter will never become deliverable by
17
- # retrying, and retrying is itself the SSRF attempt repeated.
18
- discard_on OpenLoam::OutboundUrl::BlockedError
19
-
20
- TIMEOUT_SECONDS = 5
21
-
14
+ # Frozen (BACKWARD_COMPATIBILITY.md §6): the default body and the v1 signature.
22
15
  def self.body_for(event_name, payload, tenant_id)
23
16
  JSON.generate(event: event_name, payload: payload, tenant_id: tenant_id)
24
17
  end
@@ -27,42 +20,27 @@ module OpenLoam
27
20
  "sha256=#{OpenSSL::HMAC.hexdigest('SHA256', secret.to_s, body)}"
28
21
  end
29
22
 
30
- def perform(tenant_id, endpoint_id, event_name, payload)
23
+ def perform(tenant_id, delivery_id, legacy_event_name = nil, legacy_payload = nil)
31
24
  tenant = OpenLoam::Tenant.find_by(id: tenant_id)
32
25
  return if tenant.nil?
33
26
 
34
27
  OpenLoam.as_tenant(tenant) do
35
- endpoint = OpenLoam::WebhookEndpoint.find_by(id: endpoint_id)
36
- # The endpoint may have been deleted or switched off between enqueue
37
- # and delivery — that is an answer, not an error.
38
- next if endpoint.nil? || !endpoint.active?
39
-
40
- deliver(endpoint, self.class.body_for(event_name, payload, tenant_id), event_name)
28
+ delivery =
29
+ if legacy_event_name
30
+ endpoint = OpenLoam::WebhookEndpoint.find_by(id: delivery_id)
31
+ next if endpoint.nil? || !endpoint.active?
32
+
33
+ endpoint.deliveries.create!(event_id: SecureRandom.uuid, event_name: legacy_event_name.to_s,
34
+ payload: legacy_payload || {})
35
+ else
36
+ OpenLoam::WebhookDelivery.find_by(id: delivery_id)
37
+ end
38
+ # Not visible yet (the creating transaction has not committed) or gone:
39
+ # the sweep covers the first case.
40
+ next if delivery.nil?
41
+
42
+ OpenLoam::Webhooks.deliver(delivery)
41
43
  end
42
44
  end
43
-
44
- private
45
-
46
- def deliver(endpoint, body, event_name)
47
- # Re-checked at delivery, not just at save: the URL was validated when the
48
- # tenant entered it, but DNS can be repointed at an internal address any
49
- # time afterwards. Connecting to the address we just checked closes the
50
- # window between the check and the connection.
51
- uri, address = OpenLoam::OutboundUrl.resolve!(endpoint.url)
52
-
53
- http = Net::HTTP.new(uri.host, uri.port)
54
- http.ipaddr = address
55
- http.use_ssl = uri.scheme == "https"
56
- http.open_timeout = TIMEOUT_SECONDS
57
- http.read_timeout = TIMEOUT_SECONDS
58
-
59
- request = Net::HTTP::Post.new(uri.request_uri)
60
- request["Content-Type"] = "application/json"
61
- request["X-OpenLoam-Event"] = event_name
62
- request["X-OpenLoam-Signature"] = self.class.signature(endpoint.secret, body)
63
- request.body = body
64
-
65
- http.request(request)
66
- end
67
45
  end
68
46
  end
@@ -0,0 +1,22 @@
1
+ module OpenLoam
2
+ # Finishes what a lost job left behind and prunes the ledger.
3
+ #
4
+ # With a tenant (how OpenLoam::Scheduler runs it): sweeps that tenant.
5
+ # Without one (how a host's own cron runs it, e.g. Solid Queue recurring):
6
+ # enqueues one tenant-scoped sweep per tenant that has work.
7
+ class WebhookSweepJob < ActiveJob::Base
8
+ queue_as :default
9
+
10
+ def perform(tenant_id: nil)
11
+ if tenant_id.nil?
12
+ OpenLoam::Webhooks.tenant_ids_needing_sweep.each { |id| self.class.perform_later(tenant_id: id) }
13
+ return
14
+ end
15
+
16
+ tenant = OpenLoam::Tenant.find_by(id: tenant_id)
17
+ return if tenant.nil?
18
+
19
+ OpenLoam.as_tenant(tenant) { OpenLoam::Webhooks.sweep }
20
+ end
21
+ end
22
+ end
@@ -0,0 +1,40 @@
1
+ module OpenLoam
2
+ # One event on its way to one endpoint. The ROW is the record of intent: it is
3
+ # written in the publishing transaction, the job only accelerates it, and the
4
+ # sweep finishes whatever a lost job left behind.
5
+ #
6
+ # pending -> sending -> succeeded | failed | (pending again, with next_attempt_at)
7
+ # pending -> cancelled (endpoint switched off, or the host's body builder skipped it)
8
+ class WebhookDelivery < OpenLoam::TenantRecord
9
+ self.table_name = "open_loam_webhook_deliveries"
10
+
11
+ STATUSES = %w[pending sending succeeded failed cancelled].freeze
12
+ FINAL_STATUSES = %w[succeeded failed cancelled].freeze
13
+
14
+ belongs_to :endpoint, class_name: "OpenLoam::WebhookEndpoint", inverse_of: :deliveries
15
+
16
+ validates :event_id, :event_name, :delivery_id, presence: true
17
+ validates :status, inclusion: { in: STATUSES }
18
+
19
+ before_validation on: :create do
20
+ self.delivery_id ||= "whd_#{SecureRandom.hex(12)}"
21
+ end
22
+
23
+ scope :due, ->(now = Time.current) { where(status: "pending").where("next_attempt_at IS NULL OR next_attempt_at <= ?", now) }
24
+ # Pending rows the job should already have taken: never attempted for a
25
+ # minute, or past their backoff. Fresh rows are left to their own job.
26
+ scope :overdue, ->(now = Time.current) {
27
+ where(status: "pending").where("(next_attempt_at IS NULL AND created_at <= ?) OR next_attempt_at <= ?", now - 60, now)
28
+ }
29
+ scope :stuck, ->(now = Time.current) { where(status: "sending").where("sending_at < ?", now - OpenLoam::Webhooks.stuck_after) }
30
+ scope :prunable, ->(now, retention) { where(status: FINAL_STATUSES).where("updated_at < ?", now - retention) }
31
+ scope :recent_first, -> { order(created_at: :desc, id: :desc) }
32
+
33
+ def payload_hash
34
+ value = self[:payload]
35
+ value.is_a?(String) ? JSON.parse(value) : (value || {})
36
+ end
37
+
38
+ def final? = FINAL_STATUSES.include?(status)
39
+ end
40
+ end
@@ -1,39 +1,145 @@
1
1
  module OpenLoam
2
- # Where one tenant wants its domain events delivered. `event_pattern` uses
3
- # the same rule as OpenLoam::Events.subscribe — a trailing dot is a domain prefix
4
- # ("rental."), anything else is an exact event name.
2
+ # Where one tenant wants its domain events delivered. A pattern uses the rule
3
+ # shared with OpenLoam::Events.subscribe: a trailing dot is a domain prefix
4
+ # ("rental."), anything else is an exact event name. `event_pattern` is the
5
+ # 0.3.0 single pattern; `event_patterns` holds several.
5
6
  #
6
- # The secret signs every delivery (X-OpenLoam-Signature), so a receiver can tell
7
- # a real call from a forged one.
7
+ # The secret signs every delivery. With OpenLoam::Webhooks.encrypt_secrets on,
8
+ # it is sealed with the tenant's key (OpenLoam::Encryption, "v2:..."); rows
9
+ # written before that stay readable as plaintext.
10
+ #
11
+ # `metadata` belongs to the host app (e.g. which fields a receiver may see);
12
+ # OpenLoam stores it and never reads it.
8
13
  class WebhookEndpoint < OpenLoam::TenantRecord
9
14
  self.table_name = "open_loam_webhook_endpoints"
10
15
 
11
- validates :url, presence: true
12
- validates :event_pattern, presence: true
16
+ SEALED_PREFIXES = %w[v1: v2:].freeze
17
+
18
+ has_many :deliveries, class_name: "OpenLoam::WebhookDelivery", foreign_key: :endpoint_id,
19
+ inverse_of: :endpoint, dependent: :delete_all
13
20
 
14
- # The server fetches this URL, so a tenant must not be able to aim it at an
15
- # address only the server can reach. Rejected at save for a clear error;
16
- # OpenLoam::WebhookDeliveryJob re-checks at delivery, because DNS moves.
21
+ validates :url, presence: true
22
+ validate :at_least_one_pattern
23
+ # The server fetches this URL, so a tenant must not aim it at an address only
24
+ # the server can reach. Shape is checked at save; the delivery re-checks with
25
+ # DNS, because DNS moves.
17
26
  validate :url_is_reachable_from_outside, if: -> { url.present? }
18
27
 
19
28
  scope :active, -> { where(active: true) }
20
29
 
21
30
  before_validation on: :create do
22
- self.secret ||= SecureRandom.hex(32)
31
+ self.secret = self.class.generate_secret if read_attribute(:secret).blank?
32
+ end
33
+
34
+ def self.generate_secret = SecureRandom.hex(32)
35
+
36
+ def patterns
37
+ list = Array(self[:event_patterns]).map(&:to_s).reject(&:empty?)
38
+ list.empty? ? [ event_pattern.to_s ].reject(&:empty?) : list
23
39
  end
24
40
 
25
- # One matcher, shared with the event bus, so "rental." means the same thing
26
- # to a subscriber and to a webhook.
27
41
  def matches?(event_name)
28
- OpenLoam::Events.pattern_matches?(event_pattern, event_name)
42
+ patterns.any? { |pattern| OpenLoam::Events.pattern_matches?(pattern, event_name) }
43
+ end
44
+
45
+ def settings
46
+ value = self[:metadata]
47
+ value = JSON.parse(value) if value.is_a?(String)
48
+ (value || {}).to_h.stringify_keys
49
+ end
50
+
51
+ def secret = unseal(read_attribute(:secret), "secret")
52
+ def secret=(value)
53
+ write_attribute(:secret, seal(value, "secret"))
54
+ end
55
+
56
+ def previous_secret = unseal(read_attribute(:previous_secret), "previous_secret")
57
+ def previous_secret=(value)
58
+ write_attribute(:previous_secret, seal(value, "previous_secret"))
59
+ end
60
+
61
+ # The secrets a delivery is signed with right now: the current one, plus the
62
+ # previous one while its grace period runs, so receivers can switch over.
63
+ def signing_secrets(now = Time.current)
64
+ keep_previous = previous_secret_expires_at && previous_secret_expires_at > now
65
+ [ secret, (previous_secret if keep_previous) ].compact
66
+ end
67
+
68
+ def rotate_secret!(now: Time.current)
69
+ new_secret = self.class.generate_secret
70
+ update!(previous_secret: secret, previous_secret_expires_at: now + OpenLoam::Webhooks.secret_rotation_grace,
71
+ secret: new_secret)
72
+ new_secret
73
+ end
74
+
75
+ def record_success!(at = Time.current)
76
+ update_columns(consecutive_failures: 0, failing_since: nil, last_success_at: at, updated_at: at)
77
+ end
78
+
79
+ # SQL increment, so two failing deliveries at once still count twice.
80
+ # Disabled only when the streak is both long (disable_after_failures) and
81
+ # old (failing_since at least disable_after_duration ago): a busy endpoint
82
+ # can rack up many failures during a short outage and should survive it.
83
+ def record_failure!(at = Time.current)
84
+ self.class.where(id: id).update_all([ "consecutive_failures = consecutive_failures + 1, " \
85
+ "failing_since = COALESCE(failing_since, ?), last_failure_at = ?, updated_at = ?",
86
+ at, at, at ])
87
+ reload
88
+ disable!("consecutive_failures") if failing_long_enough?(at)
89
+ end
90
+
91
+ # Idempotent and race-safe: only the caller whose UPDATE flips the row from
92
+ # on to off publishes the event.
93
+ def disable!(reason)
94
+ flipped = self.class.where(id: id, active: true)
95
+ .update_all(active: false, disabled_reason: reason.to_s, updated_at: Time.current) == 1
96
+ return unless flipped
97
+
98
+ reload
99
+ OpenLoam::Events.publish(OpenLoam::Webhooks::DISABLED_EVENT, { endpoint_id: id, reason: reason.to_s })
100
+ end
101
+
102
+ def enable!
103
+ update!(active: true, disabled_reason: nil, consecutive_failures: 0, failing_since: nil)
29
104
  end
30
105
 
31
106
  private
32
107
 
108
+ def failing_long_enough?(at)
109
+ limit = OpenLoam::Webhooks.disable_after_failures
110
+ return false unless limit && consecutive_failures >= limit
111
+
112
+ duration = OpenLoam::Webhooks.disable_after_duration
113
+ duration.nil? || (failing_since && failing_since <= at - duration)
114
+ end
115
+
116
+ def at_least_one_pattern
117
+ errors.add(:event_pattern, :blank) if patterns.empty?
118
+ end
119
+
33
120
  def url_is_reachable_from_outside
34
- OpenLoam::OutboundUrl.validate!(url)
121
+ OpenLoam::OutboundUrl.validate!(url, require_https: OpenLoam::Webhooks.require_https?,
122
+ ports: OpenLoam::Webhooks.allowed_ports)
35
123
  rescue OpenLoam::OutboundUrl::BlockedError => error
36
124
  errors.add(:url, error.message)
37
125
  end
126
+
127
+ def key_scope = "tenant/#{tenant_id || OpenLoam.tenant!.id}"
128
+
129
+ def seal(value, column)
130
+ return value if value.nil? || !OpenLoam::Webhooks.encrypt_secrets?
131
+
132
+ OpenLoam::Encryption.encrypt_scoped(value, key_scope,
133
+ aad: OpenLoam::Encryption.aad(key_scope, self.class.table_name, column))
134
+ end
135
+
136
+ # A hex secret never contains ":", so anything without a version tag is a
137
+ # plaintext row from before encryption was switched on.
138
+ def unseal(stored, column)
139
+ return stored if stored.nil? || SEALED_PREFIXES.none? { |prefix| stored.start_with?(prefix) }
140
+
141
+ OpenLoam::Encryption.decrypt_scoped(stored, key_scope,
142
+ aad: OpenLoam::Encryption.aad(key_scope, self.class.table_name, column))
143
+ end
38
144
  end
39
145
  end
@@ -44,6 +44,7 @@ module OpenLoam
44
44
  migration_template "migrations/create_open_loam_event_deliveries.rb", "db/migrate/create_open_loam_event_deliveries.rb"
45
45
  migration_template "migrations/create_open_loam_event_records.rb", "db/migrate/create_open_loam_event_records.rb"
46
46
  migration_template "migrations/create_open_loam_inbound_webhooks.rb", "db/migrate/create_open_loam_inbound_webhooks.rb"
47
+ migration_template "migrations/create_open_loam_webhook_deliveries.rb", "db/migrate/create_open_loam_webhook_deliveries.rb"
47
48
  end
48
49
 
49
50
  # Attachments (OpenLoam::Attachable, included in every generated entity) are
@@ -135,6 +136,7 @@ module OpenLoam
135
136
  template "admin/notifications_index.html.erb", "app/views/admin/notifications/index.html.erb"
136
137
  template "admin/webhook_endpoints_index.html.erb", "app/views/admin/webhook_endpoints/index.html.erb"
137
138
  template "admin/webhook_endpoints_new.html.erb", "app/views/admin/webhook_endpoints/new.html.erb"
139
+ template "admin/webhook_endpoints_created.html.erb", "app/views/admin/webhook_endpoints/created.html.erb"
138
140
  template "admin/api_tokens_index.html.erb", "app/views/admin/api_tokens/index.html.erb"
139
141
  template "admin/configs_index.html.erb", "app/views/admin/configs/index.html.erb"
140
142
  template "admin/configs_edit.html.erb", "app/views/admin/configs/edit.html.erb"
@@ -54,7 +54,7 @@ the gotchas in the framework itself.
54
54
  | Attachments | ActiveStorage `files` via `OpenLoam::Attachable` | `record.files.attach(...)`, or the file field on the entity's form — uploading counts as an update, so the entity's policy decides |
55
55
  | Sign-in | `app/controllers/admin/sessions_controller.rb` (`has_secure_password` on `User`) | email + password, then a tenant — the picker only ever lists tenants you hold a `OpenLoam::Membership` in |
56
56
  | JSON API | `app/controllers/api/<plural>_controller.rb` | generated with the entity; auth is `Authorization: Bearer <OpenLoam::ApiToken>`, and the same policies apply |
57
- | Webhooks (outbound) | `OpenLoam::WebhookEndpoint` rows, managed at `/admin/webhook_endpoints` | add an endpoint with an event pattern; matching events POST signed JSON via `OpenLoam::WebhookDeliveryJob` |
57
+ | Webhooks (outbound) | `OpenLoam::WebhookEndpoint` rows, managed at `/admin/webhook_endpoints` | add an endpoint with an event pattern; each matching event writes an `OpenLoam::WebhookDelivery` ledger row, POSTed signed JSON via `OpenLoam::WebhookDeliveryJob` and retried; apps not ticking `OpenLoam::Scheduler` must schedule `OpenLoam::WebhookSweepJob` every minute |
58
58
  | Webhooks (inbound) | `OpenLoam::InboundWebhookSource` at `/admin/inbound_webhook_sources`; public `POST /webhooks/:token` | register a source (token+secret generated); external systems POST HMAC-signed bodies; verified, replay-deduped, published on the bus as the source's `event_name` — a durable subscriber reads the body from `OpenLoam::InboundWebhookDelivery` ([details](https://github.com/DeliveristsIO/open-loam/blob/main/docs/_agents/inbound-webhooks.md)) |
59
59
  | Tests | `test/entities/<name>_test.rb` | generated with the entity; extend, never delete |
60
60
  | New-tenant defaults | `OpenLoam.on_tenant_created` blocks in `config/initializers/open_loam.rb` | edit the initializer; backfill with `bin/rails open_loam:sync` |
@@ -17,7 +17,10 @@ module Admin
17
17
  @record = OpenLoam::WebhookEndpoint.new(permitted_params)
18
18
 
19
19
  if @record.save
20
- redirect_to admin_webhook_endpoints_path
20
+ # The only screen that shows the secret: the receiver needs it to verify
21
+ # signatures, and it is never displayed again.
22
+ response.headers["Cache-Control"] = "no-store"
23
+ render :created, status: :created
21
24
  else
22
25
  render :new, status: :unprocessable_entity
23
26
  end
@@ -0,0 +1,9 @@
1
+ <h1>Webhook endpoint created</h1>
2
+
3
+ <p><strong>Copy the signing secret now.</strong> The receiver at
4
+ <code><%%= @record.url %></code> needs it to verify signatures. This is the only
5
+ time it is shown.</p>
6
+
7
+ <p><code><%%= @record.secret %></code></p>
8
+
9
+ <p><%%= link_to "Back to webhook endpoints", admin_webhook_endpoints_path %></p>
@@ -8,25 +8,27 @@ domain ("rental."); anything else is one exact event name.</p>
8
8
  <table>
9
9
  <thead>
10
10
  <tr>
11
- <th>Event pattern</th>
11
+ <th>Event patterns</th>
12
12
  <th>URL</th>
13
13
  <th>Active</th>
14
- <th>Secret</th>
14
+ <th>Status</th>
15
15
  <th></th>
16
16
  </tr>
17
17
  </thead>
18
18
  <tbody>
19
19
  <%% @records.each do |record| %>
20
20
  <tr>
21
- <td><%%= record.event_pattern %></td>
21
+ <td><%%= record.patterns.join(", ") %></td>
22
22
  <td><%%= record.url %></td>
23
23
  <td><%%= record.active? ? "yes" : "no" %></td>
24
- <td><code><%%= record.secret %></code></td>
24
+ <td><%%= record.active? ? "active" : "disabled (#{record.disabled_reason})" %></td>
25
25
  <td><%%= button_to "Delete", admin_webhook_endpoint_path(record), method: :delete, form: { style: "display:inline" } %></td>
26
26
  </tr>
27
27
  <%% end %>
28
28
  </tbody>
29
29
  </table>
30
30
 
31
- <p>Each delivery is signed: <code>X-OpenLoam-Signature: sha256=HMAC-SHA256(secret, body)</code>.
32
- Recompute it on your side and reject anything that does not match.</p>
31
+ <p>Each delivery is signed twice: <code>X-OpenLoam-Signature: sha256=HMAC-SHA256(secret, body)</code> (kept for
32
+ existing receivers) and <code>X-OpenLoam-Signature-V2: t=&lt;unix&gt;,v1=HMAC-SHA256(secret, "&lt;t&gt;.&lt;body&gt;")</code>,
33
+ which new receivers should verify. Read the secret with <code>OpenLoam::WebhookEndpoint#secret</code> in a console;
34
+ it is not shown here.</p>
@@ -1,6 +1,6 @@
1
1
  <h1>New webhook endpoint</h1>
2
2
 
3
- <%%= form_with model: @record, url: admin_webhook_endpoints_path do |f| %>
3
+ <%%= form_with model: @record, scope: :webhook_endpoint, url: admin_webhook_endpoints_path do |f| %>
4
4
  <%% if @record.errors.any? %>
5
5
  <ul>
6
6
  <%% @record.errors.full_messages.each do |message| %>
@@ -25,4 +25,4 @@
25
25
  <%%= f.submit "Create endpoint" %>
26
26
  <%% end %>
27
27
 
28
- <p>A signing secret is generated for you and shown on the list.</p>
28
+ <p>A signing secret is generated for you and shown once after you save.</p>
@@ -0,0 +1,29 @@
1
+ class CreateOpenLoamWebhookDeliveries < ActiveRecord::Migration[<%= ActiveRecord::VERSION::STRING.to_f %>]
2
+ def change
3
+ create_table :open_loam_webhook_deliveries<%= open_loam_id_option %> do |t|
4
+ t.references :tenant, null: false, foreign_key: { to_table: :open_loam_tenants }<%= open_loam_type_option %>
5
+ t.references :endpoint, null: false, foreign_key: { to_table: :open_loam_webhook_endpoints, on_delete: :cascade }<%= open_loam_type_option %>
6
+ t.string :event_id, null: false # one per published event, shared by its endpoints
7
+ t.string :delivery_id, null: false # one per (event, endpoint), sent as a header
8
+ t.string :event_name, null: false
9
+ t.json :payload, null: false # the event payload (ids and scalars)
10
+ t.string :status, null: false, default: "pending" # pending | sending | succeeded | failed | cancelled
11
+ t.integer :attempts, null: false, default: 0
12
+ t.datetime :next_attempt_at
13
+ t.datetime :sending_at
14
+ t.integer :last_response_code
15
+ t.string :last_error
16
+ t.integer :last_duration_ms
17
+ t.string :response_excerpt, limit: 512
18
+ t.datetime :delivered_at
19
+ t.timestamps
20
+ end
21
+ add_index :open_loam_webhook_deliveries, %i[endpoint_id event_id], unique: true
22
+ add_index :open_loam_webhook_deliveries, :delivery_id, unique: true
23
+ add_index :open_loam_webhook_deliveries, %i[tenant_id status next_attempt_at]
24
+ add_index :open_loam_webhook_deliveries, %i[endpoint_id created_at]
25
+ # The per-minute cross-tenant sweep lookup (overdue / stuck / prunable).
26
+ add_index :open_loam_webhook_deliveries, %i[status next_attempt_at]
27
+ add_index :open_loam_webhook_deliveries, %i[status updated_at]
28
+ end
29
+ end
@@ -3,9 +3,19 @@ class CreateOpenLoamWebhookEndpoints < ActiveRecord::Migration[<%= ActiveRecord:
3
3
  create_table :open_loam_webhook_endpoints<%= open_loam_id_option %> do |t|
4
4
  t.references :tenant, null: false, foreign_key: { to_table: :open_loam_tenants }<%= open_loam_type_option %>
5
5
  t.string :url, null: false
6
- t.string :event_pattern, null: false
7
- t.string :secret, null: false
6
+ t.string :event_pattern # single pattern (0.3.0); nil when event_patterns is used
7
+ t.json :event_patterns, null: false, default: []
8
+ t.text :secret, null: false # plaintext hex, or "v2:..." when encrypt_secrets is on
9
+ t.text :previous_secret
10
+ t.datetime :previous_secret_expires_at
8
11
  t.boolean :active, null: false, default: true
12
+ t.string :description
13
+ t.json :metadata, null: false, default: {} # host-owned settings; OpenLoam never reads it
14
+ t.string :disabled_reason
15
+ t.integer :consecutive_failures, null: false, default: 0
16
+ t.datetime :last_success_at
17
+ t.datetime :last_failure_at
18
+ t.datetime :failing_since # start of the current failure streak
9
19
  t.timestamps
10
20
  end
11
21
  add_index :open_loam_webhook_endpoints, %i[tenant_id active]
@@ -27,6 +27,12 @@ module OpenLoam
27
27
  key: OpenLoam::EventLog::PRUNE_KEY, name: "Event log prune",
28
28
  job_class: "OpenLoam::EventLogPruneJob", schedule: "interval:86400", scope: "tenant"
29
29
  )
30
+
31
+ # Webhook ledger sweep: lost/stuck deliveries and retention, every minute.
32
+ OpenLoam::Scheduler.register(
33
+ key: OpenLoam::Webhooks::SWEEP_KEY, name: "Webhook delivery sweep",
34
+ job_class: "OpenLoam::WebhookSweepJob", schedule: "interval:60", scope: "tenant"
35
+ )
30
36
  end
31
37
 
32
38
  # OpenLoam's Active Record layer — TenantRecord, Auditable, Eventful and the
@@ -1,4 +1,5 @@
1
1
  require "ipaddr"
2
+ require "socket"
2
3
  require "resolv"
3
4
  require "uri"
4
5
 
@@ -17,6 +18,10 @@ module OpenLoam
17
18
  module OutboundUrl
18
19
  class BlockedError < OpenLoam::Error; end
19
20
 
21
+ # The host did not resolve (NXDOMAIN, timeout, no records). Not a verdict on
22
+ # the address, so callers that retry may treat it as a network error.
23
+ class UnresolvableError < BlockedError; end
24
+
20
25
  SCHEMES = %w[http https].freeze
21
26
 
22
27
  # Loopback, link-local (incl. the 169.254.169.254 cloud metadata service),
@@ -25,15 +30,16 @@ module OpenLoam
25
30
  "0.0.0.0/8", "10.0.0.0/8", "100.64.0.0/10", "127.0.0.0/8", "169.254.0.0/16",
26
31
  "172.16.0.0/12", "192.0.0.0/24", "192.168.0.0/16", "198.18.0.0/15",
27
32
  "224.0.0.0/4", "240.0.0.0/4",
28
- "::1/128", "::/128", "fc00::/7", "fe80::/10", "ff00::/8"
33
+ "::1/128", "::/128", "fc00::/7", "fe80::/10", "ff00::/8",
34
+ "64:ff9b::/96", "64:ff9b:1::/48" # NAT64 (well-known and local-use) reaches IPv4 space through a gateway
29
35
  ].map { |range| IPAddr.new(range) }.freeze
30
36
 
31
37
  module_function
32
38
 
33
39
  # Shape only — no DNS. Model validations call this: saving must not depend on
34
40
  # the network, and a save-time lookup proves nothing resolve! doesn't re-prove.
35
- def validate!(url, require_https: false)
36
- uri = parse!(url, require_https: require_https)
41
+ def validate!(url, require_https: false, ports: nil)
42
+ uri = parse!(url, require_https: require_https, ports: ports)
37
43
  if literal_address(uri.host) && blocked?(uri.host)
38
44
  raise BlockedError, "#{uri.host} is not a public address"
39
45
  end
@@ -42,12 +48,12 @@ module OpenLoam
42
48
  end
43
49
 
44
50
  # Shape, DNS, and the address to connect to — call this at FETCH time.
45
- def resolve!(url, require_https: false)
46
- uri = validate!(url, require_https: require_https)
51
+ def resolve!(url, require_https: false, ports: nil)
52
+ uri = validate!(url, require_https: require_https, ports: ports)
47
53
  [ uri, address_for!(uri.host) ]
48
54
  end
49
55
 
50
- def parse!(url, require_https: false)
56
+ def parse!(url, require_https: false, ports: nil)
51
57
  uri = begin
52
58
  URI.parse(url.to_s)
53
59
  rescue URI::InvalidURIError => error
@@ -60,6 +66,7 @@ module OpenLoam
60
66
  # Credentials in the URL are a redirect/parsing-confusion trick more often
61
67
  # than a real need, and nothing here has a use for them.
62
68
  raise BlockedError, "credentials in the URL are not allowed" if uri.userinfo.present?
69
+ raise BlockedError, "port #{uri.port} is not allowed" if ports && !Array(ports).include?(uri.port)
63
70
 
64
71
  uri
65
72
  end
@@ -68,7 +75,7 @@ module OpenLoam
68
75
  # public and one private A record is still an internal reach.
69
76
  def address_for!(host)
70
77
  addresses = resolve_all(host)
71
- raise BlockedError, "#{host} does not resolve" if addresses.empty?
78
+ raise UnresolvableError, "#{host} does not resolve" if addresses.empty?
72
79
 
73
80
  addresses.each do |address|
74
81
  raise BlockedError, "#{host} resolves to the non-public address #{address}" if blocked?(address)
@@ -91,10 +98,23 @@ module OpenLoam
91
98
  end
92
99
 
93
100
  def blocked?(address)
94
- ip = IPAddr.new(address.to_s)
95
- BLOCKED.any? { |range| range.include?(ip) }
101
+ ip = normalize(IPAddr.new(address.to_s))
102
+ BLOCKED.any? { |range| range.family == ip.family && range.include?(ip) }
96
103
  rescue IPAddr::Error
97
104
  true # unparseable is not provably public
98
105
  end
106
+
107
+ # ::ffff:a.b.c.d (mapped) and ::a.b.c.d (compatible) reach the IPv4 host,
108
+ # so they are judged by the IPv4 ranges. IPAddr never matches across
109
+ # families, which is exactly how "[::ffff:127.0.0.1]" slipped through.
110
+ def normalize(ip)
111
+ return ip unless ip.ipv6?
112
+
113
+ value = ip.to_i
114
+ high = value >> 32
115
+ return IPAddr.new(value & 0xffff_ffff, Socket::AF_INET) if high == 0xffff || (high.zero? && value > 1)
116
+
117
+ ip
118
+ end
99
119
  end
100
120
  end
@@ -1,3 +1,3 @@
1
1
  module OpenLoam
2
- VERSION = "0.3.0"
2
+ VERSION = "0.4.0"
3
3
  end
@@ -0,0 +1,76 @@
1
+ # lib/open_loam/webhooks/http_transport.rb
2
+ require "net/http"
3
+ require "timeout"
4
+
5
+ module OpenLoam
6
+ module Webhooks
7
+ # The only code that opens a socket for a webhook. Connects to the address
8
+ # OutboundUrl just vetted (no DNS rebinding window), never follows a
9
+ # redirect (a 3xx is the answer), and reads at most EXCERPT_BYTES of the
10
+ # response so a hostile receiver cannot make us buffer a large body.
11
+ #
12
+ # The per-operation timeouts (open/read/write) do not bound a receiver that
13
+ # trickles one byte per read_timeout, so the whole request also runs under
14
+ # an overall deadline of 3 x Webhooks.timeout (30 s by default). Keep that
15
+ # deadline below Webhooks.stuck_after (5 min by default), or the sweep may
16
+ # resend a delivery that is still in flight.
17
+ module HttpTransport
18
+ class ExcerptComplete < StandardError; end
19
+
20
+ module_function
21
+
22
+ def post(url, body:, headers:, timeout:)
23
+ started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
24
+ code = nil
25
+ retry_after = nil
26
+ excerpt = +""
27
+ # Timeout::Error is a StandardError: Webhooks.deliver retries it as a
28
+ # network error, counted against the endpoint like any other.
29
+ Timeout.timeout(timeout * 3) do
30
+ # DNS resolution is inside the deadline too.
31
+ uri, address = OpenLoam::OutboundUrl.resolve!(url, require_https: Webhooks.require_https?,
32
+ ports: Webhooks.allowed_ports)
33
+ # nil proxy: an http_proxy in ENV must not bypass the pinned address.
34
+ http = Net::HTTP.new(uri.host, uri.port, nil)
35
+ http.ipaddr = address
36
+ http.use_ssl = uri.scheme == "https"
37
+ http.verify_mode = OpenSSL::SSL::VERIFY_PEER if http.use_ssl?
38
+ http.open_timeout = timeout
39
+ http.read_timeout = timeout
40
+ http.write_timeout = timeout
41
+
42
+ request = Net::HTTP::Post.new(uri.request_uri)
43
+ headers.each { |name, value| request[name] = value }
44
+ request["Accept-Encoding"] = "identity" # a compressed body could inflate past the excerpt cap
45
+ request.body = body
46
+
47
+ http.start do |connection|
48
+ connection.request(request) do |response|
49
+ code = response.code.to_i
50
+ retry_after = response["retry-after"]
51
+ response.read_body do |chunk|
52
+ excerpt << chunk
53
+ raise ExcerptComplete if excerpt.bytesize >= EXCERPT_BYTES
54
+ end
55
+ end
56
+ end
57
+ rescue ExcerptComplete
58
+ # Enough of the body; the connection is closed by http.start's ensure.
59
+ end
60
+
61
+ Response.new(code: code, headers: { "retry-after" => retry_after }, excerpt: safe_excerpt(excerpt),
62
+ duration_ms: ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000).round)
63
+ end
64
+
65
+ # Always valid UTF-8 without NUL (Postgres rejects both in a text column,
66
+ # and a row that cannot be saved would be resent forever), at most
67
+ # EXCERPT_BYTES. Scrubbing can grow the string (one bad byte becomes a
68
+ # 3-byte U+FFFD), so it is cut again, dropping any split character.
69
+ def safe_excerpt(raw)
70
+ text = raw.byteslice(0, EXCERPT_BYTES).to_s.force_encoding(Encoding::UTF_8).scrub.delete("\u0000")
71
+ text = text.byteslice(0, EXCERPT_BYTES).scrub("") if text.bytesize > EXCERPT_BYTES
72
+ text
73
+ end
74
+ end
75
+ end
76
+ end
@@ -1,3 +1,7 @@
1
+ require "json"
2
+ require "openssl"
3
+ require "securerandom"
4
+
1
5
  module OpenLoam
2
6
  # Domain events, delivered over HTTP to whoever asked for them.
3
7
  #
@@ -6,29 +10,279 @@ module OpenLoam
6
10
  # Matching happens at dispatch time rather than at subscribe time because
7
11
  # endpoints are data: a tenant can add one from the admin without a reboot.
8
12
  module Webhooks
9
- # Wired once from OpenLoam::Engine. Idempotent: subscribing twice would deliver
10
- # every event twice.
11
- def self.subscribe!
12
- @subscription ||= Events.subscribe_all { |event_name, payload| dispatch(event_name, payload) }
13
- end
13
+ DEFAULT_RETRY_SCHEDULE = [ 60, 300, 1_800, 7_200, 21_600, 43_200, 86_400 ].freeze
14
+ DISABLED_EVENT = "open_loam.webhook.endpoint_disabled".freeze
15
+ TEST_EVENT = "open_loam.webhook.test".freeze
16
+ SWEEP_KEY = "open_loam_webhook_sweep".freeze
17
+ EXCERPT_BYTES = 512
18
+ TIMESTAMP_TOLERANCE = 300
19
+
20
+ CONFIG_IVARS = %i[
21
+ @header_prefix @user_agent @retry_schedule @disable_after_failures @delivery_retention @body_builder
22
+ @transport @require_https @allowed_ports @encrypt_secrets @test_event_name @timeout
23
+ @secret_rotation_grace @stuck_after @disable_after_duration
24
+ ].freeze
25
+
26
+ Response = Struct.new(:code, :headers, :excerpt, :duration_ms, keyword_init: true)
27
+
28
+ # Raised by a host body builder to drop one delivery on purpose (the record
29
+ # it describes is gone, the receiver may not see this event). The row is
30
+ # cancelled, not failed, and the endpoint's health is untouched.
31
+ class SkipDelivery < OpenLoam::Error; end
14
32
 
15
- # Runs inline in whatever published the event, so it stays cheap: one
16
- # query for the tenant's active endpoints, then the HTTP call is a job.
17
- def self.dispatch(event_name, payload)
18
- tenant = Tenant.find_by(id: payload[:tenant_id])
19
- return if tenant.nil?
33
+ RETRYABLE_CODES = [ 408, 429 ].freeze
34
+ MAX_RETRY_AFTER = 3_600
20
35
 
21
- # Only JSON-safe primitives cross into the job — event payloads are ids
22
- # and scalars by convention, never records.
23
- deliverable = payload.transform_keys(&:to_s)
36
+ class << self
37
+ attr_writer :header_prefix, :user_agent, :retry_schedule, :disable_after_failures, :delivery_retention,
38
+ :body_builder, :transport, :require_https, :allowed_ports, :encrypt_secrets, :test_event_name,
39
+ :timeout, :secret_rotation_grace, :stuck_after, :disable_after_duration
24
40
 
25
- OpenLoam.as_tenant(tenant) do
26
- WebhookEndpoint.active.each do |endpoint|
27
- next unless endpoint.matches?(event_name)
41
+ def header_prefix = @header_prefix || "X-OpenLoam"
42
+ def user_agent = @user_agent || "OpenLoam-Webhooks/#{OpenLoam::VERSION}"
43
+ def retry_schedule = @retry_schedule || DEFAULT_RETRY_SCHEDULE
44
+ def disable_after_failures = defined?(@disable_after_failures) ? @disable_after_failures : 50
45
+ # An endpoint is disabled for failing only when it has failed
46
+ # disable_after_failures times in a row AND has been failing for at least
47
+ # this long, so a short outage with heavy traffic does not switch it off.
48
+ def disable_after_duration = defined?(@disable_after_duration) ? @disable_after_duration : 24.hours
49
+ def delivery_retention = defined?(@delivery_retention) ? @delivery_retention : 30.days
50
+ def body_builder = @body_builder
51
+ def transport = @transport || HttpTransport
52
+ def require_https? = @require_https ? true : false
53
+ def allowed_ports = @allowed_ports
54
+ def encrypt_secrets? = @encrypt_secrets ? true : false
55
+ def test_event_name = @test_event_name || TEST_EVENT
56
+ def timeout = @timeout || 10
57
+ def secret_rotation_grace = @secret_rotation_grace || 24.hours
58
+ def stuck_after = @stuck_after || 5.minutes
28
59
 
29
- WebhookDeliveryJob.perform_later(tenant.id, endpoint.id, event_name, deliverable)
60
+ # Tests (and only tests) put the module back to its defaults.
61
+ def reset_configuration!
62
+ CONFIG_IVARS.each { |ivar| remove_instance_variable(ivar) if instance_variable_defined?(ivar) }
63
+ end
64
+
65
+ # One attempt for one row, inside the row's tenant. Safe to call twice:
66
+ # only the caller that flips pending -> sending sends.
67
+ def deliver(delivery, now: Time.current)
68
+ return unless claim(delivery, now)
69
+
70
+ delivery.reload
71
+ endpoint = delivery.endpoint
72
+ unless endpoint&.active?
73
+ delivery.update!(status: "cancelled", sending_at: nil, last_error: "endpoint inactive")
74
+ return
75
+ end
76
+
77
+ # Building the request is the host's code: SkipDelivery cancels, any
78
+ # other error is retried without counting against the receiver.
79
+ begin
80
+ body = body_for(endpoint, delivery)
81
+ headers = headers_for(endpoint, delivery, body, now)
82
+ rescue SkipDelivery => error
83
+ delivery.update!(status: "cancelled", sending_at: nil, last_error: short(error.message))
84
+ return
85
+ rescue StandardError => error
86
+ retry_later(delivery, endpoint, now, error: short("#{error.class}: #{error.message}"), count_failure: false)
87
+ return
30
88
  end
89
+
90
+ begin
91
+ response = transport.post(endpoint.url, body: body, headers: headers, timeout: timeout)
92
+ rescue OpenLoam::OutboundUrl::UnresolvableError => error
93
+ # DNS being down is the network's fault, not the receiver's: retry.
94
+ retry_later(delivery, endpoint, now, error: short("#{error.class}: #{error.message}"))
95
+ return
96
+ rescue OpenLoam::OutboundUrl::BlockedError => error
97
+ delivery.update!(status: "failed", sending_at: nil, last_error: short("blocked: #{error.message}"))
98
+ endpoint.disable!("blocked_url")
99
+ return
100
+ rescue StandardError => error
101
+ retry_later(delivery, endpoint, now, error: short("#{error.class}: #{error.message}"))
102
+ return
103
+ end
104
+
105
+ # Outside the rescue: a bookkeeping error must surface, not become a retry.
106
+ record_response(delivery, endpoint, response, now)
107
+ end
108
+
109
+ def body_for(endpoint, delivery)
110
+ if body_builder
111
+ JSON.generate(body_builder.call(endpoint: endpoint, delivery: delivery))
112
+ else
113
+ OpenLoam::WebhookDeliveryJob.body_for(delivery.event_name, delivery.payload_hash, delivery.tenant_id)
114
+ end
115
+ end
116
+
117
+ def headers_for(endpoint, delivery, body, now)
118
+ timestamp = now.to_i
119
+ prefix = header_prefix
120
+ {
121
+ "Content-Type" => "application/json",
122
+ "User-Agent" => user_agent,
123
+ "#{prefix}-Event" => delivery.event_name,
124
+ "#{prefix}-Event-Id" => delivery.event_id,
125
+ "#{prefix}-Delivery" => delivery.delivery_id,
126
+ "#{prefix}-Timestamp" => timestamp.to_s,
127
+ "#{prefix}-Signature" => OpenLoam::WebhookDeliveryJob.signature(endpoint.secret, body),
128
+ "#{prefix}-Signature-V2" => signature_v2(endpoint.signing_secrets(now), body, timestamp)
129
+ }
130
+ end
131
+
132
+ # "t=<unix>,v1=<hex>[,v1=<hex>]" — HMAC-SHA256 of "<t>.<body>" under each
133
+ # signing secret. The timestamp is inside the MAC, so a captured request
134
+ # cannot be replayed after the receiver's tolerance window.
135
+ def signature_v2(secrets, body, timestamp)
136
+ macs = Array(secrets).map { |secret| "v1=#{OpenSSL::HMAC.hexdigest('SHA256', secret.to_s, "#{timestamp}.#{body}")}" }
137
+ ([ "t=#{timestamp}" ] + macs).join(",")
138
+ end
139
+
140
+ # Reference verifier (docs and tests): constant-time compare, bounded clock skew.
141
+ def verify_v2(header, body, secret, now: Time.current, tolerance: TIMESTAMP_TOLERANCE)
142
+ parts = header.to_s.split(",").map { |part| part.split("=", 2) }
143
+ timestamp = parts.find { |key, _| key == "t" }&.last.to_i
144
+ return false if timestamp.zero? || (now.to_i - timestamp).abs > tolerance
145
+
146
+ expected = OpenSSL::HMAC.hexdigest("SHA256", secret.to_s, "#{timestamp}.#{body}")
147
+ parts.any? { |key, value| key == "v1" && value && ActiveSupport::SecurityUtils.secure_compare(value, expected) }
148
+ end
149
+
150
+ # Runs inline in whatever published the event, inside its transaction when
151
+ # there is one: the ledger rows commit or roll back with the change itself,
152
+ # and the jobs are enqueued only after that commit.
153
+ def dispatch(event_name, payload)
154
+ tenant = OpenLoam::Tenant.find_by(id: payload[:tenant_id])
155
+ return if tenant.nil?
156
+
157
+ deliverable = payload.transform_keys(&:to_s)
158
+ event_id = SecureRandom.uuid
159
+
160
+ OpenLoam.as_tenant(tenant) do
161
+ OpenLoam::WebhookEndpoint.active.each do |endpoint|
162
+ next unless endpoint.matches?(event_name)
163
+
164
+ enqueue(endpoint.deliveries.create!(event_id: event_id, event_name: event_name.to_s, payload: deliverable))
165
+ end
166
+ end
167
+ end
168
+
169
+ def send_test!(endpoint)
170
+ delivery = endpoint.deliveries.create!(event_id: SecureRandom.uuid, event_name: test_event_name,
171
+ payload: { "test" => true, "tenant_id" => endpoint.tenant_id })
172
+ enqueue(delivery)
173
+ delivery
31
174
  end
175
+
176
+ # Manual redelivery of a finished row: restarts the full retry schedule
177
+ # (attempts reset). Non-final (pending/sending) rows are returned untouched.
178
+ def redeliver!(delivery)
179
+ return delivery unless delivery.final?
180
+
181
+ delivery.update!(status: "pending", attempts: 0, next_attempt_at: nil, sending_at: nil)
182
+ enqueue(delivery)
183
+ delivery
184
+ end
185
+
186
+ def enqueue(delivery)
187
+ tenant_id = delivery.tenant_id
188
+ delivery_id = delivery.id
189
+ after_commit { OpenLoam::WebhookDeliveryJob.perform_later(tenant_id, delivery_id) }
190
+ end
191
+
192
+ # Rails >= 7.2 defers to the outermost commit (and runs at once when no
193
+ # transaction is open). On 7.1 the job may start before the row is visible;
194
+ # it then finds nothing and the sweep picks the row up.
195
+ def after_commit(&block)
196
+ if ActiveRecord.respond_to?(:after_all_transactions_commit)
197
+ ActiveRecord.after_all_transactions_commit(&block)
198
+ else
199
+ block.call
200
+ end
201
+ end
202
+
203
+ # Inside one tenant: stuck rows go back to pending (at-least-once — the
204
+ # receiver dedupes by event id), every overdue row gets a job, and final
205
+ # rows past the retention are deleted. Returns the number re-enqueued.
206
+ # A stuck row that has used up the retry schedule fails instead, so a
207
+ # persistent error after the send cannot resend it forever.
208
+ def sweep(now: Time.current, limit: 500)
209
+ stuck = OpenLoam::WebhookDelivery.stuck(now)
210
+ stuck.where("attempts > ?", retry_schedule.size)
211
+ .update_all(status: "failed", sending_at: nil, last_error: "stuck", updated_at: now)
212
+ stuck.update_all(status: "pending", sending_at: nil, next_attempt_at: now, updated_at: now)
213
+
214
+ tenant_id = OpenLoam.tenant!.id
215
+ ids = OpenLoam::WebhookDelivery.overdue(now).order(:id).limit(limit).pluck(:id)
216
+ ids.each { |id| OpenLoam::WebhookDeliveryJob.perform_later(tenant_id, id) }
217
+
218
+ OpenLoam::WebhookDelivery.prunable(now, delivery_retention).delete_all if delivery_retention
219
+ ids.size
220
+ end
221
+
222
+ # Blessed cross-tenant lookup (BACKWARD_COMPATIBILITY.md §1), the webhook
223
+ # twin of the scheduler's due-job claim: tenant ids only, never rows.
224
+ def tenant_ids_needing_sweep(now: Time.current)
225
+ all = OpenLoam::WebhookDelivery.unscoped
226
+ work = all.overdue(now).or(all.stuck(now))
227
+ work = work.or(all.prunable(now, delivery_retention)) if delivery_retention
228
+ work.distinct.pluck(:tenant_id)
229
+ end
230
+
231
+ private
232
+
233
+ def claim(delivery, now)
234
+ OpenLoam::WebhookDelivery.where(id: delivery.id, status: "pending")
235
+ .where("next_attempt_at IS NULL OR next_attempt_at <= ?", now)
236
+ .update_all([ "status = 'sending', sending_at = ?, attempts = attempts + 1, updated_at = ?", now, now ]) == 1
237
+ end
238
+
239
+ def record_response(delivery, endpoint, response, now)
240
+ attrs = { last_response_code: response.code, last_duration_ms: response.duration_ms,
241
+ response_excerpt: response.excerpt, sending_at: nil }
242
+ case response.code
243
+ when 200..299
244
+ delivery.update!(attrs.merge(status: "succeeded", delivered_at: now, last_error: nil, next_attempt_at: nil))
245
+ endpoint.record_success!(now)
246
+ when 410
247
+ delivery.update!(attrs.merge(status: "failed", last_error: "HTTP 410"))
248
+ endpoint.disable!("gone")
249
+ when *RETRYABLE_CODES, 500..599
250
+ delivery.assign_attributes(attrs)
251
+ retry_later(delivery, endpoint, now, error: "HTTP #{response.code}", retry_after: retry_after_seconds(response))
252
+ else
253
+ delivery.update!(attrs.merge(status: "failed", last_error: "HTTP #{response.code}"))
254
+ endpoint.record_failure!(now)
255
+ end
256
+ end
257
+
258
+ # count_failure: false for errors that are the host's, not the receiver's.
259
+ def retry_later(delivery, endpoint, now, error:, retry_after: nil, count_failure: true)
260
+ endpoint&.record_failure!(now) if count_failure
261
+ delay = retry_schedule[delivery.attempts - 1]
262
+ if delay.nil?
263
+ delivery.update!(status: "failed", sending_at: nil, last_error: error)
264
+ return
265
+ end
266
+
267
+ wait = retry_after || (delay * (0.9 + (rand * 0.2))).round
268
+ delivery.update!(status: "pending", sending_at: nil, last_error: error, next_attempt_at: now + wait)
269
+ OpenLoam::WebhookDeliveryJob.set(wait: wait).perform_later(delivery.tenant_id, delivery.id)
270
+ end
271
+
272
+ def retry_after_seconds(response)
273
+ value = response.headers.to_h["retry-after"].to_s
274
+ return nil unless value.match?(/\A\d+\z/)
275
+
276
+ value.to_i.clamp(1, MAX_RETRY_AFTER)
277
+ end
278
+
279
+ def short(message) = message.to_s.truncate(255)
280
+ end
281
+
282
+ # Wired once from OpenLoam::Engine. Idempotent: subscribing twice would deliver
283
+ # every event twice.
284
+ def self.subscribe!
285
+ @subscription ||= Events.subscribe_all { |event_name, payload| dispatch(event_name, payload) }
32
286
  end
33
287
  end
34
288
  end
data/lib/open_loam.rb CHANGED
@@ -43,6 +43,7 @@ require "open_loam/notifications"
43
43
  require "open_loam/sso"
44
44
  require "open_loam/business_rules"
45
45
  require "open_loam/webhooks"
46
+ require "open_loam/webhooks/http_transport"
46
47
  require "open_loam/inbound_webhooks"
47
48
  require "open_loam/durable_events"
48
49
  require "open_loam/event_log"
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: open-loam
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.3.0
4
+ version: 0.4.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Grzegorz Smajdor
@@ -67,6 +67,7 @@ files:
67
67
  - app/jobs/open_loam/event_log_prune_job.rb
68
68
  - app/jobs/open_loam/event_redelivery_sweep_job.rb
69
69
  - app/jobs/open_loam/webhook_delivery_job.rb
70
+ - app/jobs/open_loam/webhook_sweep_job.rb
70
71
  - app/models/open_loam/api_token.rb
71
72
  - app/models/open_loam/audit_record.rb
72
73
  - app/models/open_loam/auth_attempt.rb
@@ -96,6 +97,7 @@ files:
96
97
  - app/models/open_loam/sso_provider.rb
97
98
  - app/models/open_loam/tenant.rb
98
99
  - app/models/open_loam/translation.rb
100
+ - app/models/open_loam/webhook_delivery.rb
99
101
  - app/models/open_loam/webhook_endpoint.rb
100
102
  - app/views/open_loam/attachments/_attachments.html.erb
101
103
  - app/views/open_loam/comments/_comments.html.erb
@@ -194,6 +196,7 @@ files:
194
196
  - lib/generators/open_loam/install/templates/admin/translations_controller.rb
195
197
  - lib/generators/open_loam/install/templates/admin/translations_index.html.erb
196
198
  - lib/generators/open_loam/install/templates/admin/webhook_endpoints_controller.rb
199
+ - lib/generators/open_loam/install/templates/admin/webhook_endpoints_created.html.erb
197
200
  - lib/generators/open_loam/install/templates/admin/webhook_endpoints_index.html.erb
198
201
  - lib/generators/open_loam/install/templates/admin/webhook_endpoints_new.html.erb
199
202
  - lib/generators/open_loam/install/templates/api_base_controller.rb
@@ -227,6 +230,7 @@ files:
227
230
  - lib/generators/open_loam/install/templates/migrations/create_open_loam_sso_providers.rb
228
231
  - lib/generators/open_loam/install/templates/migrations/create_open_loam_tenants.rb
229
232
  - lib/generators/open_loam/install/templates/migrations/create_open_loam_translations.rb
233
+ - lib/generators/open_loam/install/templates/migrations/create_open_loam_webhook_deliveries.rb
230
234
  - lib/generators/open_loam/install/templates/migrations/create_open_loam_webhook_endpoints.rb
231
235
  - lib/generators/open_loam/install/templates/migrations/create_users.rb
232
236
  - lib/generators/open_loam/install/templates/user.rb
@@ -302,6 +306,7 @@ files:
302
306
  - lib/open_loam/undo.rb
303
307
  - lib/open_loam/version.rb
304
308
  - lib/open_loam/webhooks.rb
309
+ - lib/open_loam/webhooks/http_transport.rb
305
310
  - lib/open_loam/widgets.rb
306
311
  - lib/open_loam/workflow.rb
307
312
  - lib/tasks/open_loam.rake
@@ -330,7 +335,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
330
335
  - !ruby/object:Gem::Version
331
336
  version: '0'
332
337
  requirements: []
333
- rubygems_version: 4.0.16
338
+ rubygems_version: 4.0.20
334
339
  specification_version: 4
335
340
  summary: 'AI-native Rails business foundation: tenancy, policies, audit, events, admin
336
341
  — decided, not re-litigated.'