angarium 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 3b7cf3c384477b5808150336685e858b28eff1e9590f9ec454569019ec8d10bf
4
- data.tar.gz: 6756a2b028c09b72146f5624e50d08a9ff9f062b4aa7030859548e68d02a5c8d
3
+ metadata.gz: 98d1d438861a4ff090327c932205c0e6345c8f079562a99ca6ab344fff93da10
4
+ data.tar.gz: 6b37af363e8a253db8f9e66feb549a56a2152fb4fc7536c76551f4daf61688de
5
5
  SHA512:
6
- metadata.gz: e291f504be94a517d8276d00f0089aec942d104b134ea7a6d907fffe55c7a0914bb38766fc005ac0c6c1faab0bf05e873ee13fa6fcf20b8b558996224cbc9ec2
7
- data.tar.gz: 676568e4ac7bae762dfc517ba3e3183d2f13a0e8022f8419ed54ded2e0566f2bfb2510c476b4fe3cc84dd150d6f5f0ffdc9d68c3a582272f34e8d0e3e9d4a50b
6
+ metadata.gz: 1675398932024d506e7338faa7b3585df46caf3f6f80090347ee029bb608dd859333eff198259eee769e466d9fbc4bee91be9994e971055659e44dd1d2f87d65
7
+ data.tar.gz: 44481ffed27e96af446720d475124536eb27e384ad2fe28c777bf4b9cadb9033bec49ec345dbb4abc1376deab3d9d7d24917cb283c4aa4d9f6742f34533239e8
data/CHANGELOG.md CHANGED
@@ -5,6 +5,34 @@ All notable changes to this project are documented here. The format is based on
5
5
  follow [Semantic Versioning](https://semver.org). While on 0.x, a minor release
6
6
  may include breaking changes; patch releases stay backward compatible.
7
7
 
8
+ ## [0.2.0] - 2026-07-07
9
+
10
+ ### Bug Fixes
11
+
12
+ - Harden delivery concurrency and endpoint validation
13
+ - Block special-use ranges and bound DNS resolution
14
+ - Robust pagination, JSON error contract, event preload
15
+
16
+ ### Documentation
17
+
18
+ - Add email reporting channel and auto-bump SECURITY.md version
19
+ - Document new config options and SSRF special-use hardening
20
+
21
+ ### Features
22
+
23
+ - Expose endpoint owner in the API behind a policy gate
24
+ - Add dns_timeout, hosts-file toggle, and input limits
25
+ - Add hot-path indexes, persisted forced flag, text url column
26
+
27
+ ### Performance
28
+
29
+ - Memoize the record-less API policy per request
30
+ - Load only the columns subscription matching needs
31
+
32
+ ### Refactoring
33
+
34
+ - Rename ping event and message to plain "ping"
35
+
8
36
  ## [0.1.0] - 2026-07-07
9
37
 
10
38
  ### Bug Fixes
@@ -26,6 +54,7 @@ may include breaking changes; patch releases stay backward compatible.
26
54
  - Close the last README review nits
27
55
  - Include unverified in the ping status lists
28
56
  - Document ActiveSupport::Notifications instrumentation
57
+ - Add RubyGems version badge to README
29
58
 
30
59
  ### Features
31
60
 
data/README.md CHANGED
@@ -351,7 +351,7 @@ redelivery is at-least-once-safe either way. Set it to `nil` to disable reaping.
351
351
 
352
352
  ### Pinging an endpoint
353
353
 
354
- Verify an endpoint end-to-end by delivering a synthetic `angarium.ping` event
354
+ Verify an endpoint end-to-end by delivering a synthetic `ping` event
355
355
  (subscription matching is bypassed, so a ping is always sent). Returns the
356
356
  `Angarium::Delivery`, so you can reload it to inspect the outcome:
357
357
 
@@ -443,6 +443,7 @@ methods:
443
443
  | `create_unverified?` | `false` | Whether endpoints created through the API start `unverified` (no deliveries until a successful ping verifies them) instead of live. |
444
444
  | `permit_allow_private_network?` | `false` | Whether `allow_private_network` (relax the private-IP block) is API-writable. Dangerous; trusted operators only. |
445
445
  | `permit_allowed_networks?` | `false` | Whether `allowed_networks` (a restrictive CIDR allowlist) is API-writable. |
446
+ | `expose_owner?` | `false` | Whether serialized endpoints include `owner_type`/`owner_id`. Off by default (the owner is the tenancy boundary, resolved server-side). Turn on for a cross-owner console that must tell endpoints apart. |
446
447
  | `index?` `show?` `create?` `update?` `destroy?` | `true` | Whether each action is allowed. |
447
448
  | `rotate_secret?` `pause?` `enable?` `verify?` `ping?` `redeliver?` | `update?` | Member actions; default to the `update?` capability. |
448
449
 
@@ -455,6 +456,11 @@ class WebhookEndpointPolicy < Angarium::Api::Policy
455
456
  # Multi-tenant visibility: compose on top of the relation you're given.
456
457
  def scope(relation) = relation.where(owner_id: current_user.account.owner_ids)
457
458
 
459
+ # A console spanning owners needs owner_type/owner_id on each endpoint to tell
460
+ # them apart. Pair this with a scope (above) that actually spans owners —
461
+ # expose_owner? only affects serialization, not what the caller can see.
462
+ def expose_owner? = current_user.admin?
463
+
458
464
  # Admins may create for any owner in their account (via an owner_id param);
459
465
  # everyone else creates for themselves.
460
466
  def owner
@@ -514,6 +520,9 @@ included (see the note below):
514
520
 
515
521
  - **Secrets are never echoed.** `signing_secret` is returned only by `create` and
516
522
  `rotate_secret`; `custom_headers` (which may hold a credential) is write-only.
523
+ - **Owner is opt-in.** The endpoint object omits `owner_type`/`owner_id` unless the
524
+ policy's `expose_owner?` returns true; `owner_id` is a string (a polymorphic
525
+ owner may have any primary-key type).
517
526
  - **Pagination.** List endpoints take `?limit=` (default 50, max 200) and
518
527
  `?offset=`, and each list response carries a `pagination` object (`limit`,
519
528
  `offset`, `count` in this page, `total` overall); there are more when
@@ -582,7 +591,12 @@ at delivery time:
582
591
  - **`config.block_private_ips`** (default `true`) blocks delivery to
583
592
  private, loopback, and link-local addresses (e.g. `127.0.0.1`, `10.0.0.0/8`,
584
593
  `169.254.169.254`), including IPv4-mapped IPv6 forms (e.g. `::ffff:127.0.0.1`)
585
- and the unspecified address (`0.0.0.0` / `::`).
594
+ and the unspecified address (`0.0.0.0` / `::`). It also blocks IANA special-use
595
+ ranges that aren't flagged as "private" but are still unsafe targets —
596
+ `0.0.0.0/8` (routes to localhost on Linux), CGNAT (`100.64.0.0/10`), reserved
597
+ space (`240.0.0.0/4`), and the NAT64 prefix (`64:ff9b::/96`, which can embed the
598
+ metadata IP). Resolution is re-checked at delivery time and the connection is
599
+ pinned to the validated IP, so DNS rebinding can't slip past.
586
600
  - **`endpoint.allow_private_network`** (default `false`) is the per-endpoint opt-in
587
601
  required to deliver to a private address. An allowlist entry alone does **not**
588
602
  unlock a private address.
@@ -687,7 +701,11 @@ which documents every option inline. The delivery and retry settings:
687
701
  | `max_retry_after` | `3600` | Cap (seconds) on a honored `Retry-After`; `nil` is uncapped. |
688
702
  | `auto_disable_endpoint_after` | `nil` | Deactivate an endpoint after N consecutive failures; `nil` never does. |
689
703
  | `signing_secret_grace_period` | `24.hours` | How long a rotated endpoint's previous secret stays valid. |
690
- | `block_private_ips` | `true` | Reject endpoint URLs resolving to private/loopback addresses (SSRF). |
704
+ | `block_private_ips` | `true` | Reject endpoint URLs resolving to private/loopback/special-use addresses (SSRF). |
705
+ | `dns_timeout` | `[1, 3]` | Per-try timeout(s) in seconds for resolving an endpoint host; `nil` uses the resolver default. |
706
+ | `resolve_dns_with_hosts_file` | `true` | Also resolve endpoint hosts via the system hosts file (e.g. `/etc/hosts`), not DNS only. Set `false` to harden to DNS-only. |
707
+ | `max_url_length` | `2048` | Maximum length of an endpoint URL. |
708
+ | `max_subscribed_events` | `100` | Maximum number of event patterns an endpoint may subscribe to. |
691
709
  | `max_response_body_bytes` | `65_536` | Truncate the stored response body; `nil` stores it whole. |
692
710
  | `delivery_attempt_retention` | `nil` | Age past which `angarium:prune` deletes attempts; `nil` keeps all. |
693
711
  | `delivering_timeout` | `15.minutes` | Age after which `angarium:reap` requeues a stuck `delivering` delivery. |
data/SECURITY.md CHANGED
@@ -2,23 +2,27 @@
2
2
 
3
3
  ## Supported versions
4
4
 
5
- Angarium is pre-release. Security fixes are provided for the `0.1.x` series
5
+ Angarium is pre-release. Security fixes are provided for the `0.2.x` series
6
6
  during this phase. Once a stable line is released, this policy will be updated to
7
7
  list the versions that receive security updates.
8
8
 
9
9
  | Version | Supported |
10
10
  | ------- | --------- |
11
- | 0.1.x | ✅ |
11
+ | 0.2.x | ✅ |
12
12
 
13
13
  ## Reporting a vulnerability
14
14
 
15
15
  Please report security vulnerabilities **privately**. Do **not** open a public
16
16
  issue, pull request, or discussion for a suspected vulnerability.
17
17
 
18
- Use GitHub's private vulnerability reporting for the
19
- [`radioactive-labs/angarium`](https://github.com/radioactive-labs/angarium)
20
- repository: go to the **Security** tab and choose **Report a vulnerability**
21
- (this opens a private draft advisory visible only to the maintainers).
18
+ Use one of the following private channels:
19
+
20
+ - **Preferred:** GitHub's private vulnerability reporting for the
21
+ [`radioactive-labs/angarium`](https://github.com/radioactive-labs/angarium)
22
+ repository — go to the **Security** tab and choose **Report a vulnerability**
23
+ (this opens a private draft advisory visible only to the maintainers).
24
+ - **Email:** [sfroelich01@gmail.com](mailto:sfroelich01@gmail.com) with the
25
+ subject line `[SECURITY] Angarium`.
22
26
 
23
27
  We aim to acknowledge new reports within a few business days, and we'll keep you
24
28
  informed as we investigate and prepare a fix. Please give us a reasonable window
@@ -15,6 +15,7 @@ module Angarium
15
15
  rescue_from ActiveRecord::RecordInvalid, with: :angarium_render_invalid
16
16
  rescue_from Angarium::Api::NotAuthorized, with: :angarium_render_forbidden
17
17
  rescue_from Angarium::Api::UnpermittedParameter, with: :angarium_render_unpermitted
18
+ rescue_from ActionController::ParameterMissing, with: :angarium_render_bad_request
18
19
 
19
20
  # Resolved current user (via config.current_user). Public so policies can
20
21
  # read it as `controller.angarium_current_user`.
@@ -31,8 +32,19 @@ module Angarium
31
32
  end
32
33
 
33
34
  # The policy for this request (config.policy_class), holding scope,
34
- # create-owner, and per-action permissions.
35
+ # create-owner, and per-action permissions. The record-less policy is
36
+ # memoized: scope/owner/expose_owner? and friends are request-global and
37
+ # get called repeatedly (expose_owner? runs once per serialized row), so a
38
+ # list response would otherwise allocate a fresh policy per endpoint. A
39
+ # record-bound policy (member authorize!) still constructs fresh, since it
40
+ # varies by record.
35
41
  def angarium_policy(record = nil)
42
+ return build_angarium_policy(record) if record
43
+
44
+ @angarium_policy ||= build_angarium_policy
45
+ end
46
+
47
+ def build_angarium_policy(record = nil)
36
48
  Angarium.config.policy_class.to_s.constantize.new(self, record)
37
49
  end
38
50
 
@@ -41,9 +53,10 @@ module Angarium
41
53
  angarium_policy.scope(Angarium::Endpoint.all)
42
54
  end
43
55
 
44
- # A delivery whose endpoint is within the caller's scope, or 404.
56
+ # A delivery whose endpoint is within the caller's scope, or 404. Eager-load
57
+ # :event since delivery_json serializes event.name (avoids a follow-up query).
45
58
  def scoped_delivery(id)
46
- Angarium::Delivery.where(endpoint_id: endpoint_scope.select(:id)).find(id)
59
+ Angarium::Delivery.includes(:event).where(endpoint_id: endpoint_scope.select(:id)).find(id)
47
60
  end
48
61
 
49
62
  # Guard the current action with the policy's `<action>?` predicate.
@@ -55,8 +68,8 @@ module Angarium
55
68
  # and advertises the window with a `pagination` object so clients can page
56
69
  # (there's more when offset + count < total).
57
70
  def render_collection(key, relation, &serializer)
58
- limit = params.fetch(:limit, 50).to_i.clamp(1, 200)
59
- offset = [params.fetch(:offset, 0).to_i, 0].max
71
+ limit = pagination_param(:limit, 50).clamp(1, 200)
72
+ offset = [pagination_param(:offset, 0), 0].max
60
73
  records = relation.limit(limit).offset(offset).to_a
61
74
  render json: {
62
75
  key => records.map(&serializer),
@@ -64,6 +77,15 @@ module Angarium
64
77
  }
65
78
  end
66
79
 
80
+ # Coerce a pagination query param to an integer, ignoring non-scalar input.
81
+ # A hostile `?limit[]=1` makes params[:limit] an Array/Parameters, which
82
+ # has no #to_i and would otherwise raise an uncaught 500; fall back to the
83
+ # default for anything that isn't a plain string.
84
+ def pagination_param(key, default)
85
+ raw = params[key]
86
+ raw.is_a?(String) ? raw.to_i : default
87
+ end
88
+
67
89
  def render_error(status, message, **extra)
68
90
  render json: {error: message, **extra}, status: status
69
91
  end
@@ -71,6 +93,7 @@ module Angarium
71
93
  def angarium_render_unauthorized = render_error(:unauthorized, "authentication required")
72
94
  def angarium_render_forbidden = render_error(:forbidden, "not authorized")
73
95
  def angarium_render_not_found = render_error(:not_found, "not found")
96
+ def angarium_render_bad_request(error) = render_error(:bad_request, "bad request", details: [error.message])
74
97
 
75
98
  def angarium_render_invalid(error)
76
99
  render_error(:unprocessable_entity, "validation failed", details: error.record.errors.full_messages)
@@ -99,6 +122,12 @@ module Angarium
99
122
  created_at: endpoint.created_at.iso8601,
100
123
  updated_at: endpoint.updated_at.iso8601
101
124
  }
125
+ if angarium_policy.expose_owner?
126
+ # Raw columns off the row (no owner fetch, so no cross-database hit);
127
+ # the caller reassembles the owner via its own type/id convention.
128
+ json[:owner_type] = endpoint.owner_type
129
+ json[:owner_id] = endpoint.owner_id
130
+ end
102
131
  json[:signing_secret] = endpoint.signing_secret if include_secret
103
132
  json
104
133
  end
@@ -1,11 +1,14 @@
1
1
  module Angarium
2
2
  class DeliverJob < ApplicationJob
3
- def perform(delivery_id, force = false)
3
+ # The second positional arg is accepted only for backward compatibility with
4
+ # jobs enqueued by an older version (force is now persisted on the delivery as
5
+ # `forced` and read by #deliver!); new enqueues pass just the id.
6
+ def perform(delivery_id, _legacy_force = nil)
4
7
  delivery = Delivery.find_by(id: delivery_id)
5
8
  return unless delivery
6
9
  return unless delivery.pending?
7
10
 
8
- delivery.deliver!(force: force)
11
+ delivery.deliver!
9
12
  end
10
13
  end
11
14
  end
@@ -13,12 +13,7 @@ module Angarium
13
13
  belongs_to :endpoint, class_name: "Angarium::Endpoint"
14
14
  has_many :delivery_attempts, class_name: "Angarium::DeliveryAttempt", dependent: :destroy
15
15
 
16
- # Transient: set on a delivery built for a manual, forced send (e.g.
17
- # endpoint.ping!(force: true)) so its first attempt bypasses the endpoint
18
- # status guard. Not persisted; only the enqueued job carries it forward.
19
- attr_accessor :force_send
20
-
21
- after_create_commit { force_send ? DeliverJob.perform_later(id, true) : DeliverJob.perform_later(id) }
16
+ after_create_commit { DeliverJob.perform_later(id) }
22
17
 
23
18
  # Recover deliveries stranded in "delivering": a worker set the state to
24
19
  # "delivering" (in #deliver!) but died (crash, deploy, OOM) before
@@ -34,14 +29,30 @@ module Angarium
34
29
  ids = where(state: "delivering").where(last_attempt_at: ..older_than.ago).pluck(:id)
35
30
  return 0 if ids.empty?
36
31
 
37
- where(id: ids).update_all(state: "pending", next_attempt_at: Time.current, updated_at: Time.current)
38
- ids.each { |id| DeliverJob.perform_later(id) }
39
- ids.size
32
+ # Reset each id with a state-scoped compare-and-swap, not a blanket
33
+ # `where(id: ids)`: a delivery can leave `delivering` (succeed, be marked
34
+ # gone, etc.) between the pluck above and this reset. Re-asserting
35
+ # `state: "delivering"` means we never drag a completed delivery back to
36
+ # pending, and — since only the reaper that actually flipped the row
37
+ # enqueues — two reapers racing the same snapshot can't double-enqueue.
38
+ requeued = 0
39
+ ids.each do |id|
40
+ changed = where(id: id, state: "delivering")
41
+ .update_all(state: "pending", next_attempt_at: Time.current, updated_at: Time.current)
42
+ next if changed.zero?
43
+
44
+ DeliverJob.perform_later(id)
45
+ requeued += 1
46
+ end
47
+ requeued
40
48
  end
41
49
 
42
50
  # Performs one attempt. Records a DeliveryAttempt, then transitions to
43
- # succeeded, blocked (SSRF), schedules a retry, or exhausts. Returns the attempt.
44
- def deliver!(client: Client.new, force: false)
51
+ # succeeded, blocked (SSRF), schedules a retry, or exhausts. Returns the
52
+ # attempt. `force` defaults to the persisted `forced` flag so a reaped or
53
+ # redelivered forced attempt keeps bypassing the guard; pass it explicitly
54
+ # only to override in tests.
55
+ def deliver!(client: Client.new, force: forced)
45
56
  payload = {delivery_id: id, endpoint_id: endpoint_id, event: event.name, force: force}
46
57
  ActiveSupport::Notifications.instrument("deliver.angarium", payload) do
47
58
  # An endpoint's status can change after a delivery is queued: auto-disable
@@ -62,7 +73,12 @@ module Angarium
62
73
  end
63
74
  end
64
75
 
65
- update!(state: "delivering", attempt_count: attempt_count + 1, last_attempt_at: Time.current)
76
+ # Atomically claim this delivery (pending -> delivering) so only one
77
+ # worker ever attempts it. Without the compare-and-swap, two jobs for the
78
+ # same id (a reaper requeue racing a stale enqueue, an adapter retry, etc.)
79
+ # would both pass the guard above, both POST, and both read the same stale
80
+ # attempt_count. If the CAS changes no row, another worker owns it: bail.
81
+ return nil unless claim_for_attempt!
66
82
  payload[:attempt] = attempt_count
67
83
 
68
84
  # Re-resolve at delivery time (rather than trusting the save-time check)
@@ -144,15 +160,33 @@ module Angarium
144
160
 
145
161
  # Reset the retry cycle and re-enqueue immediately. Keeps prior
146
162
  # DeliveryAttempt history. `force: true` sends even if the endpoint is no
147
- # longer enabled (for the re-enqueued attempt). Returns self.
163
+ # longer enabled (for the re-enqueued attempt); it is persisted so the reaper
164
+ # honors it too. Returns self.
148
165
  def redeliver!(force: false)
149
- update!(state: "pending", next_attempt_at: nil, attempt_count: 0)
150
- force ? DeliverJob.perform_later(id, true) : DeliverJob.perform_later(id)
166
+ update!(state: "pending", next_attempt_at: nil, attempt_count: 0, forced: force)
167
+ DeliverJob.perform_later(id)
151
168
  self
152
169
  end
153
170
 
154
171
  private
155
172
 
173
+ # Atomic pending -> delivering claim: the single writer gate for an attempt.
174
+ # Returns true if THIS caller won the row (and bumps attempt_count in the same
175
+ # statement so concurrent claimers can't both read a stale count), false if
176
+ # another worker already claimed it. Reload so in-memory state matches the
177
+ # CAS (the rest of #deliver! reads attempt_count, and later update!s need a
178
+ # clean dirty baseline to persist the delivering -> succeeded/pending move).
179
+ def claim_for_attempt!
180
+ now = Time.current
181
+ claimed = self.class.where(id: id, state: "pending").update_all(
182
+ ["state = 'delivering', attempt_count = attempt_count + 1, last_attempt_at = ?, updated_at = ?", now, now]
183
+ )
184
+ return false if claimed.zero?
185
+
186
+ reload
187
+ true
188
+ end
189
+
156
190
  # Endpoint was paused after this delivery was queued. Park it (no attempt
157
191
  # consumed, no failure recorded) by clearing the schedule and leaving it
158
192
  # pending; Endpoint#enable! re-enqueues parked deliveries so a pause/resume
@@ -203,7 +237,9 @@ module Angarium
203
237
  # misconfigured receiver could send a tiny Retry-After to defeat our
204
238
  # backoff and make us hammer it. Retry-After can delay, never expedite.
205
239
  wait = [wait, retry_after].max if retry_after
206
- update!(state: "pending", next_attempt_at: Time.current + wait)
240
+ # Drop force: a recorded failure means the forced first attempt is spent,
241
+ # and per the deliver! contract any scheduled retry follows normal rules.
242
+ update!(state: "pending", next_attempt_at: Time.current + wait, forced: false)
207
243
  DeliverJob.set(wait: wait).perform_later(id)
208
244
  end
209
245
  end
@@ -39,11 +39,13 @@ module Angarium
39
39
 
40
40
  before_validation :ensure_signing_secret, on: :create
41
41
 
42
- validates :name, presence: true
42
+ validates :name, presence: true, length: {maximum: 255}
43
43
  validates :url, presence: true
44
44
  validates :url, "angarium/endpoint_url": true, if: :verify_url_address?
45
+ validate :url_within_length_limit
45
46
  validate :allowed_networks_are_valid_cidrs
46
47
  validate :custom_headers_are_strings
48
+ validate :subscribed_events_are_bounded
47
49
 
48
50
  def self.generate_signing_secret
49
51
  "whsec_#{Base64.strict_encode64(SecureRandom.bytes(32))}"
@@ -120,8 +122,10 @@ module Angarium
120
122
  def enable!
121
123
  return false unless transition_status!(:enabled, consecutive_failures: 0)
122
124
  # Resume deliveries parked while this endpoint was paused (pending with no
123
- # scheduled attempt). Dispatch creates none while paused, so this only
124
- # re-enqueues the held ones.
125
+ # scheduled attempt). This predicate can also match a delivery that already
126
+ # has an in-flight job (a forced ping, a just-redelivered row), but a stray
127
+ # re-enqueue is harmless: the atomic pending->delivering claim in
128
+ # Delivery#deliver! guarantees only one worker ever sends a given delivery.
125
129
  deliveries.where(state: "pending", next_attempt_at: nil).find_each { |d| DeliverJob.perform_later(d.id) }
126
130
  true
127
131
  end
@@ -135,15 +139,15 @@ module Angarium
135
139
  transition_status!(:enabled, from: :unverified) { Angarium.notify(:on_endpoint_verified, self) }
136
140
  end
137
141
 
138
- # Deliver a synthetic `angarium.ping` event to this endpoint, bypassing
142
+ # Deliver a synthetic `ping` event to this endpoint, bypassing
139
143
  # subscription matching (a ping is always sent). By default a ping also
140
144
  # ignores the endpoint's status, so you can test an endpoint that is paused,
141
145
  # disabled, or not yet enabled; pass `force: false` to respect the status
142
146
  # guard instead. Returns the created Angarium::Delivery, whose
143
147
  # after_create_commit enqueues the DeliverJob; reload it to inspect the outcome.
144
- def ping!(payload = {message: "Angarium ping"}, force: true)
145
- event = Angarium::Event.create!(name: "angarium.ping", payload: payload)
146
- deliveries.create!(event: event) { |d| d.force_send = force }
148
+ def ping!(payload = {message: "ping"}, force: true)
149
+ event = Angarium::Event.create!(name: "ping", payload: payload)
150
+ deliveries.create!(event: event, forced: force)
147
151
  end
148
152
 
149
153
  private
@@ -200,6 +204,31 @@ module Angarium
200
204
  end
201
205
  end
202
206
 
207
+ def url_within_length_limit
208
+ max = Angarium.config.max_url_length
209
+ return if url.blank? || max.nil? || url.length <= max
210
+
211
+ errors.add(:url, "is too long (maximum is #{max} characters)")
212
+ end
213
+
214
+ def subscribed_events_are_bounded
215
+ return if subscribed_events.blank?
216
+
217
+ unless subscribed_events.is_a?(Array)
218
+ errors.add(:subscribed_events, "must be an array of event patterns")
219
+ return
220
+ end
221
+
222
+ max = Angarium.config.max_subscribed_events
223
+ if max && subscribed_events.length > max
224
+ errors.add(:subscribed_events, "cannot have more than #{max} subscribed events")
225
+ end
226
+
227
+ unless subscribed_events.all? { |e| e.is_a?(String) && !e.empty? && e.length <= 255 }
228
+ errors.add(:subscribed_events, "must be non-empty strings of at most 255 characters")
229
+ end
230
+ end
231
+
203
232
  def custom_headers_are_strings
204
233
  return if custom_headers.blank?
205
234
 
@@ -212,6 +241,14 @@ module Angarium
212
241
  if custom_headers.keys.any? { |k| RESERVED_HEADERS.include?(k.downcase) }
213
242
  errors.add(:custom_headers, "cannot override reserved or transport headers (#{RESERVED_HEADERS.join(", ")})")
214
243
  end
244
+
245
+ # Reject CR/LF/NUL in any key or value. The outbound HTTP client writes
246
+ # headers verbatim, so a CRLF in a value would inject an extra header line
247
+ # — smuggling e.g. a forged webhook-signature past the RESERVED_HEADERS
248
+ # denylist (which only guards whole keys). Fail closed instead.
249
+ if custom_headers.any? { |k, v| k.match?(/[\r\n\0]/) || v.match?(/[\r\n\0]/) }
250
+ errors.add(:custom_headers, "must not contain control characters (CR, LF, or NUL)")
251
+ end
215
252
  end
216
253
  end
217
254
  end
@@ -61,6 +61,16 @@ module Angarium
61
61
  false
62
62
  end
63
63
 
64
+ # Should serialized endpoints include their owner (owner_type/owner_id)?
65
+ # Default: no. The owner is the tenancy boundary, resolved server-side, so
66
+ # the single-owner default has no reason to echo it back. Override for an
67
+ # admin/multi-tenant console that lists endpoints across owners and needs
68
+ # to tell them apart. The raw columns are exposed (already loaded on the
69
+ # row), not the owner record, so this adds no per-row database fetch.
70
+ def expose_owner?
71
+ false
72
+ end
73
+
64
74
  def index? = true
65
75
  def show? = true
66
76
  def create? = true
@@ -1,9 +1,11 @@
1
1
  class CreateAngariumEndpoints < ActiveRecord::Migration[7.1]
2
2
  def change
3
3
  create_table :angarium_endpoints, id: Angarium.primary_key_type do |t|
4
- t.references :owner, polymorphic: true, null: false, type: :string
4
+ # index: false — the owner scope also orders by created_at, so the
5
+ # composite index below covers both the tenancy filter and the list order.
6
+ t.references :owner, polymorphic: true, null: false, type: :string, index: false
5
7
  t.string :name, null: false
6
- t.string :url, null: false
8
+ t.text :url, null: false
7
9
  t.string :status, null: false, default: "enabled"
8
10
  t.text :signing_secret, null: false
9
11
  t.json :subscribed_events, null: false, default: []
@@ -17,6 +19,9 @@ class CreateAngariumEndpoints < ActiveRecord::Migration[7.1]
17
19
  t.datetime :secret_rotated_at
18
20
  t.datetime :status_changed_at
19
21
  t.timestamps
22
+ # Serves the owner-scoped list endpoint (WHERE owner_type/owner_id ORDER BY
23
+ # created_at DESC) in a single index scan.
24
+ t.index [:owner_type, :owner_id, :created_at], name: "idx_angarium_endpoints_on_owner_created_at"
20
25
  end
21
26
  end
22
27
  end
@@ -2,12 +2,23 @@ class CreateAngariumDeliveries < ActiveRecord::Migration[7.1]
2
2
  def change
3
3
  create_table :angarium_deliveries, id: Angarium.primary_key_type do |t|
4
4
  t.references :event, null: false, type: Angarium.primary_key_type, foreign_key: {to_table: :angarium_events}
5
- t.references :endpoint, null: false, type: Angarium.primary_key_type, foreign_key: {to_table: :angarium_endpoints}
5
+ # index: false — the endpoint-scoped delivery list orders by created_at, so
6
+ # the composite index below covers both the FK lookup and the list order.
7
+ t.references :endpoint, index: false, null: false, type: Angarium.primary_key_type, foreign_key: {to_table: :angarium_endpoints}
6
8
  t.string :state, null: false, default: "pending"
7
9
  t.integer :attempt_count, null: false, default: 0
10
+ # A manual ping!/redeliver! delivery whose next attempt bypasses the
11
+ # endpoint status guard. Persisted (not just a job arg) so the reaper can
12
+ # honor it when re-running an attempt that was stranded before completing;
13
+ # cleared once a recorded failure schedules a retry.
14
+ t.boolean :forced, null: false, default: false
8
15
  t.datetime :last_attempt_at
9
16
  t.datetime :next_attempt_at
10
17
  t.timestamps
18
+ # Endpoint-scoped list endpoint (WHERE endpoint_id ORDER BY created_at DESC).
19
+ t.index [:endpoint_id, :created_at], name: "idx_angarium_deliveries_on_endpoint_created_at"
20
+ # Stalled-delivery reaper: WHERE state = 'delivering' AND last_attempt_at < ?.
21
+ t.index [:state, :last_attempt_at], name: "idx_angarium_deliveries_on_state_last_attempt"
11
22
  end
12
23
  end
13
24
  end
@@ -1,12 +1,18 @@
1
1
  class CreateAngariumDeliveryAttempts < ActiveRecord::Migration[7.1]
2
2
  def change
3
3
  create_table :angarium_delivery_attempts, id: Angarium.primary_key_type do |t|
4
- t.references :delivery, null: false, type: Angarium.primary_key_type, foreign_key: {to_table: :angarium_deliveries}
4
+ # index: false — the delivery-scoped attempt list orders by created_at, so
5
+ # the composite index below covers both the FK lookup and the list order.
6
+ t.references :delivery, index: false, null: false, type: Angarium.primary_key_type, foreign_key: {to_table: :angarium_deliveries}
5
7
  t.integer :response_code
6
8
  t.text :response_body
7
9
  t.string :error
8
10
  t.float :duration
9
11
  t.timestamps
12
+ # Delivery-scoped attempt list (WHERE delivery_id ORDER BY created_at DESC).
13
+ t.index [:delivery_id, :created_at], name: "idx_angarium_attempts_on_delivery_created_at"
14
+ # Retention prune: DELETE WHERE created_at < cutoff over the largest table.
15
+ t.index :created_at, name: "idx_angarium_attempts_on_created_at"
10
16
  end
11
17
  end
12
18
  end
@@ -45,17 +45,50 @@ module Angarium
45
45
  literal = to_ipaddr(host)
46
46
  return [literal] if literal
47
47
 
48
- Resolv.getaddresses(host).filter_map { |a| to_ipaddr(a) }
48
+ # Bound DNS with config.dns_timeout via an explicit Resolv::DNS — an
49
+ # unbounded lookup lets a host with a deliberately slow authoritative
50
+ # nameserver tie up a delivery worker well past our HTTP timeouts. With the
51
+ # hosts file enabled (default), wrap it in a Resolv that consults
52
+ # /etc/hosts first (so an internally-pinned endpoint resolves, and a hosts
53
+ # hit short-circuits DNS); otherwise resolve DNS-only.
54
+ dns = Resolv::DNS.new
55
+ dns.timeouts = Angarium.config.dns_timeout if Angarium.config.dns_timeout
56
+ resolvers = Angarium.config.resolve_dns_with_hosts_file ? [Resolv::Hosts.new, dns] : [dns]
57
+ Resolv.new(resolvers).getaddresses(host).filter_map { |a| to_ipaddr(a.to_s) }
49
58
  rescue
50
59
  []
60
+ ensure
61
+ dns&.close
51
62
  end
52
63
 
64
+ # IANA special-use ranges that are unsafe webhook destinations but that
65
+ # Ruby's IPAddr predicates (loopback?/private?/link_local?/unique_local?) do
66
+ # NOT flag. Without these, e.g. `https://0.0.0.1/` (routes to localhost on
67
+ # Linux) or `https://[64:ff9b::a9fe:a9fe]/` (NAT64 of the cloud metadata IP)
68
+ # would pass validation and be delivered to. Documentation ranges
69
+ # (192.0.2.0/24, 2001:db8::/32, etc.) are intentionally omitted: they aren't
70
+ # an SSRF vector and blocking them would reject the TEST-NET fixtures.
71
+ SPECIAL_USE_RANGES = [
72
+ "0.0.0.0/8", # "this host" — routes to localhost on Linux
73
+ "100.64.0.0/10", # CGNAT (RFC 6598)
74
+ "192.0.0.0/24", # IETF protocol assignments (RFC 6890)
75
+ "198.18.0.0/15", # benchmarking (RFC 2544)
76
+ "240.0.0.0/4", # reserved / future use, incl. 255.255.255.255 broadcast
77
+ "64:ff9b::/96", # NAT64 well-known prefix (RFC 6052)
78
+ "64:ff9b:1::/48" # NAT64 local-use prefix (RFC 8215)
79
+ ].map { |cidr| IPAddr.new(cidr) }.freeze
80
+
53
81
  def private?(ip)
54
82
  ip = normalize(ip)
55
83
  return true if ip.to_i.zero? # 0.0.0.0 / :: (unspecified -> localhost on Linux)
56
84
 
57
85
  ip.loopback? || ip.private? || ip.link_local? ||
58
- (ip.respond_to?(:unique_local?) && ip.unique_local?)
86
+ (ip.respond_to?(:unique_local?) && ip.unique_local?) ||
87
+ special_use?(ip)
88
+ end
89
+
90
+ def special_use?(ip)
91
+ SPECIAL_USE_RANGES.any? { |range| range.family == ip.family && range.include?(ip) }
59
92
  end
60
93
 
61
94
  # Collapse IPv4-mapped IPv6 (::ffff:x.x.x.x) to native IPv4 so the predicates
@@ -5,7 +5,8 @@ module Angarium
5
5
  :primary_key_type, :database, :connects_to, :max_response_body_bytes,
6
6
  :auto_disable_endpoint_after, :respect_retry_after,
7
7
  :max_retry_after, :retry_jitter, :signing_secret_grace_period,
8
- :delivery_attempt_retention, :delivering_timeout,
8
+ :delivery_attempt_retention, :delivering_timeout, :dns_timeout,
9
+ :resolve_dns_with_hosts_file, :max_url_length, :max_subscribed_events,
9
10
  :on_delivery_exhausted, :on_endpoint_deactivated, :on_endpoint_verified,
10
11
  :parent_controller, :current_user, :policy_class
11
12
 
@@ -34,6 +35,10 @@ module Angarium
34
35
  # Takes precedence over @database for the connection.
35
36
  @connects_to = nil
36
37
  @max_response_body_bytes = 65_536
38
+ # Disable an endpoint after this many consecutive *failed deliveries* (a
39
+ # delivery that exhausts its whole retry schedule, or is blocked by the SSRF
40
+ # guard) — NOT individual failed HTTP attempts. A single delivery that
41
+ # retries and eventually gives up counts as one. nil disables auto-disable.
37
42
  @auto_disable_endpoint_after = nil
38
43
  @respect_retry_after = true
39
44
  @max_retry_after = 3600
@@ -41,6 +46,20 @@ module Angarium
41
46
  @signing_secret_grace_period = 24.hours
42
47
  @delivery_attempt_retention = nil
43
48
  @delivering_timeout = 15.minutes
49
+ # Per-try timeout(s), in seconds, for resolving an endpoint host before
50
+ # delivery. Bounds how long a hostile or misconfigured slow-resolving host
51
+ # can stall a delivery worker (Resolv::DNS retries across the array). Set to
52
+ # nil to use the resolver's own defaults.
53
+ @dns_timeout = [1, 3]
54
+ # Whether host resolution also consults the system hosts file (/etc/hosts),
55
+ # not just DNS. Default true, so an internal endpoint pinned via /etc/hosts
56
+ # resolves as expected. Set false to harden a deployment to DNS-only.
57
+ @resolve_dns_with_hosts_file = true
58
+ # Max length of an endpoint URL and the cap on how many event patterns an
59
+ # endpoint may subscribe to (each pattern is replayed by EventMatcher on
60
+ # every dispatch). Both bound user-supplied input.
61
+ @max_url_length = 2048
62
+ @max_subscribed_events = 100
44
63
  @on_delivery_exhausted = nil # ->(delivery) { ... }
45
64
  @on_endpoint_deactivated = nil # ->(endpoint, reason) { ... } reason: :consecutive_failures | :gone
46
65
  @on_endpoint_verified = nil # ->(endpoint) { ... } fired when an unverified endpoint is verified
@@ -5,9 +5,13 @@ module Angarium
5
5
  def call(event_name, payload, owner:)
6
6
  notify_payload = {event: event_name, event_id: nil, deliveries: 0}
7
7
  ActiveSupport::Notifications.instrument("dispatch.angarium", notify_payload) do
8
- endpoints = Endpoint.enabled.where(owner: owner).select do |endpoint|
9
- endpoint.subscribed_to?(event_name)
10
- end
8
+ # Load only the columns subscription matching needs. Endpoints carry
9
+ # large encrypted blobs (signing_secret, previous_signing_secret,
10
+ # custom_headers); pulling them for every candidate — most of which may
11
+ # not even match — is wasted I/O. The matched endpoints' secrets are
12
+ # loaded later, at delivery time, from the delivery's own association.
13
+ candidates = Endpoint.enabled.where(owner: owner).select(:id, :subscribed_events)
14
+ endpoints = candidates.to_a.select { |endpoint| endpoint.subscribed_to?(event_name) }
11
15
  next nil if endpoints.empty?
12
16
 
13
17
  Event.transaction do
@@ -1,3 +1,3 @@
1
1
  module Angarium
2
- VERSION = "0.1.0"
2
+ VERSION = "0.2.0"
3
3
  end
@@ -23,6 +23,10 @@ class <%= class_name %> < Angarium::Api::Policy
23
23
  # def permit_allow_private_network? = current_user.admin?
24
24
  # def permit_allowed_networks? = true
25
25
 
26
+ # Include owner_type/owner_id in serialized endpoints (default off). Turn on
27
+ # for an admin/multi-tenant console that lists endpoints across owners.
28
+ # def expose_owner? = current_user.admin?
29
+
26
30
  # Per-action permissions. rotate_secret?/pause?/enable?/ping?/redeliver? all
27
31
  # default to update?.
28
32
  # def index? = true
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: angarium
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - TheDumbTechGuy