munster 0.3.1 → 0.4.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +13 -11
- data/README.md +11 -6
- data/Rakefile +15 -7
- data/example/app/webhooks/webhook_test_handler.rb +1 -7
- data/example/config/initializers/munster.rb +3 -1
- data/lib/munster/base_handler.rb +43 -21
- data/lib/munster/controllers/receive_webhooks_controller.rb +32 -33
- data/lib/munster/engine.rb +4 -2
- data/lib/munster/install_generator.rb +1 -0
- data/lib/munster/jobs/processing_job.rb +21 -4
- data/lib/munster/models/received_webhook.rb +73 -2
- data/lib/munster/templates/add_headers_to_munster_webhooks.rb.erb +5 -0
- data/lib/munster/templates/create_munster_tables.rb.erb +1 -1
- data/lib/munster/version.rb +1 -1
- data/lib/munster.rb +4 -3
- metadata +8 -8
- data/LICENSE.txt +0 -21
- data/sig/munster.rbs +0 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 86e72b8a849981c0e626054ed78660ba288628389933890d9d662daf2f22daa7
|
4
|
+
data.tar.gz: 62ab2ab22e5805158eb7f9f2d6fd622d03b32db4c985ed20d07d6e21a4b30c89
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5436a8a198fac9c1febfbd351bb44435307d88d7d1f87f70f293359ae88e304aaaa6378c0a0c14f89b534344f088ce14bb3afd8fdc323fd99788559c56362013
|
7
|
+
data.tar.gz: d2f7a667043d29b9bb46f5248963f6e02a6198f012f17ca5f9c9fdbc6c180d6ba0088273942e767b8feece8c1c985fba03183c5eb6a22377bce8853af70102df
|
data/CHANGELOG.md
CHANGED
@@ -3,31 +3,33 @@ All notable changes to this project will be documented in this file.
|
|
3
3
|
|
4
4
|
This format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
5
5
|
|
6
|
-
## 0.3.1
|
7
6
|
|
8
|
-
|
7
|
+
## 0.4.0
|
8
|
+
|
9
|
+
- Limit the size of the request body, since otherwise there can be a large attack vector where random senders can spam the database with data and cause a denial of service. With background validation, this is one of the few cases where we want to reject the payload without persisting it.
|
10
|
+
- Manage the `state` of the `ReceivedWebhook` from the background job itself. This frees up the handler to actually do the work associated with processing only. The job will manage the rest.
|
11
|
+
- Use `valid?` in the background job instead of the controller. Most common configuration issue is an incorrectly specified signing secret, or an incorrectly implemented input validation. When these happen, it is better to allow the webhook to be reprocessed
|
12
|
+
- Use instance methods in handlers instead of class methods, as they are shorter to define. Assume a handler module supports `.new` - with a module using singleton methods it may return `self` from `new`.
|
13
|
+
- In the config, allow the handlers specified as strings. Module resolution in Rails happens after the config gets loaded, because the config may alter the Zeitwerk load paths. To allow the config to get loaded and to allow handlers to be autoloaded using Zeitwerk, the handler modules have to be resolved lazily. This also permits the handlers to be reloadable, like any module under Rails' autoloading control.
|
14
|
+
- Simplify the Rails app used in tests to be small and keep it in a single file
|
15
|
+
- If a handler is willing to expose errors to the caller, let Rails rescue the error and display an error page or do whatever else is configured for Rails globally.
|
16
|
+
- Store request headers with the received webhook to allow for async validation. Run `bin/rails g munster:install` to add the required migration.
|
17
|
+
|
18
|
+
## 0.3.1
|
9
19
|
|
10
20
|
- BaseHandler#expose_errors_to_sender? default to true now.
|
11
21
|
|
12
22
|
## 0.3.0
|
13
23
|
|
14
|
-
### Changed
|
15
24
|
- state_machine_enum library was moved in it's own library/gem.
|
16
|
-
|
17
|
-
### Fixed
|
18
25
|
- Provide handled: true attribute for Rails.error.report method, because it is required in Rails 7.0.
|
19
26
|
|
20
27
|
## 0.2.0
|
21
28
|
|
22
|
-
### Changed
|
23
|
-
|
24
29
|
- Handler methods are now defined as instance methods for simplicity.
|
25
30
|
- Define service_id in initializer with active_handlers, instead of handler class.
|
26
31
|
- Use ruby 3.0 as a base for standard/rubocop, format all code according to it.
|
27
|
-
|
28
|
-
### Added
|
29
|
-
|
30
|
-
- Introduce Rails common error reporter ( https://guides.rubyonrails.org/error_reporting.html )
|
32
|
+
- Use Rails common error reporter ( https://guides.rubyonrails.org/error_reporting.html )
|
31
33
|
|
32
34
|
## 0.1.0
|
33
35
|
|
data/README.md
CHANGED
@@ -2,9 +2,11 @@
|
|
2
2
|
|
3
3
|
Munster is a Rails engine that provides a webhook endpoint for receiving and processing webhooks from various services. Engine stores received webhook first and later processes webhook in a separete async process.
|
4
4
|
|
5
|
-
|
6
|
-
|
7
|
-
|
5
|
+
> [!CAUTION]
|
6
|
+
> At the moment Munster is only used internally at Cheddar. Any support to external parties is on best-effort
|
7
|
+
> basis. While we are happy to see issues and pull requests, we can't guarantee that those will be addressed
|
8
|
+
> quickly. The engine does receive rapid updates which may break your application if you come to depend on
|
9
|
+
> the library. That is to be expected.
|
8
10
|
|
9
11
|
## Installation
|
10
12
|
|
@@ -20,11 +22,13 @@ If bundler is not being used to manage dependencies, install the gem by executin
|
|
20
22
|
|
21
23
|
Generate migrations and initializer file.
|
22
24
|
|
23
|
-
`munster:install`
|
25
|
+
`bin/rails g munster:install`
|
24
26
|
|
25
27
|
Mount munster engine in your routes.
|
26
28
|
|
27
|
-
|
29
|
+
```ruby
|
30
|
+
mount Munster::Engine, at: "/webhooks"
|
31
|
+
```
|
28
32
|
|
29
33
|
## Requirements
|
30
34
|
|
@@ -38,7 +42,8 @@ This project depends on two dependencies:
|
|
38
42
|
This gem uses [Rails common error reporter](https://guides.rubyonrails.org/error_reporting.html) to report any possible error to services like Honeybadger, Appsignal, Sentry and etc. Most of those services already support this common interface, if not - it's not that hard to add this support on your own.
|
39
43
|
|
40
44
|
It's possible to provide additional context for every error. e.g.
|
41
|
-
|
45
|
+
|
46
|
+
```ruby
|
42
47
|
Munster.configure do |config|
|
43
48
|
config.error_context = { appsignal: { namespace: "webhooks" } }
|
44
49
|
end
|
data/Rakefile
CHANGED
@@ -1,12 +1,7 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
3
|
require "bundler/setup"
|
4
|
-
|
5
|
-
APP_RAKEFILE = File.expand_path("test/dummy/Rakefile", __dir__)
|
6
|
-
load "rails/tasks/engine.rake"
|
7
|
-
|
8
|
-
load "rails/tasks/statistics.rake"
|
9
|
-
|
4
|
+
require "rake/testtask"
|
10
5
|
require "bundler/gem_tasks"
|
11
6
|
require "standard/rake"
|
12
7
|
|
@@ -15,4 +10,17 @@ task :format do
|
|
15
10
|
`bundle exec magic_frozen_string_literal .`
|
16
11
|
end
|
17
12
|
|
18
|
-
|
13
|
+
Rake::TestTask.new(:test) do |t|
|
14
|
+
t.libs << "test"
|
15
|
+
t.libs << "lib"
|
16
|
+
|
17
|
+
file_name = ARGV[1]
|
18
|
+
|
19
|
+
t.test_files = if file_name
|
20
|
+
[file_name]
|
21
|
+
else
|
22
|
+
FileList["test/**/*_test.rb"]
|
23
|
+
end
|
24
|
+
end
|
25
|
+
|
26
|
+
task default: [:test, :standard]
|
@@ -2,17 +2,11 @@
|
|
2
2
|
|
3
3
|
# This handler accepts webhooks from our integration tests. This webhook gets dispatched
|
4
4
|
# if a banking provider test fails, indicating that the bank might be having an incident
|
5
|
-
|
6
5
|
class WebhookTestHandler < Munster::BaseHandler
|
7
6
|
def valid?(request) = true
|
8
7
|
|
9
8
|
def process(webhook)
|
10
|
-
|
11
|
-
webhook.update!(status: "processing")
|
12
|
-
webhook.update!(status: "processed")
|
13
|
-
rescue
|
14
|
-
webhook.update!(status: "error")
|
15
|
-
raise
|
9
|
+
Rails.logger.info { webhook.request.params.fetch(:payment_id) }
|
16
10
|
end
|
17
11
|
|
18
12
|
def expose_errors_to_sender? = true
|
data/lib/munster/base_handler.rb
CHANGED
@@ -4,40 +4,58 @@ require_relative "jobs/processing_job"
|
|
4
4
|
|
5
5
|
module Munster
|
6
6
|
class BaseHandler
|
7
|
-
#
|
8
|
-
|
9
|
-
|
10
|
-
|
11
|
-
|
12
|
-
# Reimplement this method, it's being used in WebhooksController to store incoming webhook.
|
13
|
-
# Also que for processing in the end.
|
7
|
+
# `handle` accepts the ActionDispatch HTTP request and saves the webhook for later processing. It then
|
8
|
+
# enqueues an ActiveJob which will perform the processing using `process`.
|
9
|
+
#
|
10
|
+
# @param action_dispatch_request[ActionDispatch::Request] the request from the controller
|
14
11
|
# @return [void]
|
15
12
|
def handle(action_dispatch_request)
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
}
|
22
|
-
webhook = Munster::ReceivedWebhook.create!(**attrs)
|
13
|
+
handler_module_name = is_a?(Munster::BaseHandler) ? self.class.name : to_s
|
14
|
+
handler_event_id = extract_event_id_from_request(action_dispatch_request)
|
15
|
+
|
16
|
+
webhook = Munster::ReceivedWebhook.new(request: action_dispatch_request, handler_event_id: handler_event_id, handler_module_name: handler_module_name)
|
17
|
+
webhook.save!
|
23
18
|
|
24
19
|
Munster.configuration.processing_job_class.perform_later(webhook)
|
25
|
-
rescue ActiveRecord::RecordNotUnique #
|
26
|
-
|
20
|
+
rescue ActiveRecord::RecordNotUnique # Webhook deduplicated
|
21
|
+
Rails.logger.info { "#{inspect} Webhook #{handler_event_id} is a duplicate delivery and will not be stored." }
|
27
22
|
end
|
28
23
|
|
29
|
-
# This
|
24
|
+
# This is the heart of your webhook processing. Override this method and define your processing inside of it.
|
25
|
+
# The `received_webhook` will provide access to the `ReceivedWebhook` model, which contains the received
|
26
|
+
# body of the webhook request, but also the full (as-full-as-possible) clone of the original ActionDispatch::Request
|
27
|
+
# that you can use.
|
28
|
+
#
|
29
|
+
# @param received_webhook[Munster::ReceivedWebhook]
|
30
|
+
# @return [void]
|
30
31
|
def process(received_webhook)
|
31
32
|
end
|
32
33
|
|
33
|
-
# This method verifies that request actually comes from
|
34
|
-
# signature validation, HTTP authentication, IP whitelisting and the like
|
34
|
+
# This method verifies that request is not malformed and actually comes from the webhook sender:
|
35
|
+
# signature validation, HTTP authentication, IP whitelisting and the like. There is a difference depending
|
36
|
+
# on whether you validate sync (in the receiving controller) or async (in the processing job):
|
37
|
+
# Validation is async - it takes place in the background job that gets enqueued to process the webhook.
|
38
|
+
# The `action_dispatch_request` will be reconstructed from the `ReceivedWebhook` data. Background validation
|
39
|
+
# is used because the most common misconfiguration that may occur is usually forgetting or misidentifying the
|
40
|
+
# signature for signed webhooks. If such a misconfiguration has taken place, the background validation
|
41
|
+
# (instead of rejecting the webhook at input) permits you to still process the webhook once the secrets
|
42
|
+
# have been configured correctly.
|
43
|
+
#
|
44
|
+
# If this method returns `false`, the webhook will be marked as `failed_validation` in the database. If this
|
45
|
+
# method returns `true`, the `process` method of the handler is going to be called.
|
46
|
+
#
|
47
|
+
# @see Munster::ReceivedWebhook#request
|
48
|
+
# @param action_dispatch_request[ActionDispatch::Request] the reconstructed request from the controller
|
49
|
+
# @return [Boolean]
|
35
50
|
def valid?(action_dispatch_request)
|
36
51
|
true
|
37
52
|
end
|
38
53
|
|
39
|
-
# Default implementation just generates UUID, but if the webhook sender sends us
|
40
|
-
# an event ID we use it for deduplication.
|
54
|
+
# Default implementation just generates a random UUID, but if the webhook sender sends us
|
55
|
+
# an event ID we use it for deduplication. A duplicate webhook is not going to be
|
56
|
+
# stored in the database if it is already present there.
|
57
|
+
#
|
58
|
+
# @return [String]
|
41
59
|
def extract_event_id_from_request(action_dispatch_request)
|
42
60
|
SecureRandom.uuid
|
43
61
|
end
|
@@ -47,6 +65,8 @@ module Munster
|
|
47
65
|
# data and do not disable your endpoint forcibly. We allow this to be configured
|
48
66
|
# on a per-handler basis - a better webhooks sender will be able to make out
|
49
67
|
# some sense of the errors.
|
68
|
+
#
|
69
|
+
# @return [Boolean]
|
50
70
|
def expose_errors_to_sender?
|
51
71
|
true
|
52
72
|
end
|
@@ -55,6 +75,8 @@ module Munster
|
|
55
75
|
# to deactivate a particular handler via feature flags for example, or use other
|
56
76
|
# logic to determine whether the handler may be used to create new received webhooks
|
57
77
|
# in the system. This is primarily needed for load shedding.
|
78
|
+
#
|
79
|
+
# @return [Boolean]
|
58
80
|
def active?
|
59
81
|
true
|
60
82
|
end
|
@@ -2,55 +2,54 @@
|
|
2
2
|
|
3
3
|
module Munster
|
4
4
|
class ReceiveWebhooksController < ActionController::API
|
5
|
-
class
|
5
|
+
class HandlerInactive < StandardError
|
6
6
|
end
|
7
7
|
|
8
|
-
class
|
8
|
+
class UnknownHandler < StandardError
|
9
9
|
end
|
10
10
|
|
11
11
|
def create
|
12
|
-
|
13
|
-
|
12
|
+
Rails.error.set_context(**Munster.configuration.error_context)
|
13
|
+
handler = lookup_handler(service_id)
|
14
14
|
raise HandlerInactive unless handler.active?
|
15
|
-
raise HandlerRefused unless handler.valid?(request)
|
16
|
-
|
17
15
|
handler.handle(request)
|
18
|
-
|
19
|
-
rescue
|
20
|
-
|
16
|
+
render(json: {ok: true, error: nil})
|
17
|
+
rescue UnknownHandler => e
|
18
|
+
Rails.error.report(e, handled: true, severity: :error)
|
19
|
+
render_error_with_status("No handler found for #{service_id.inspect}", status: :not_found)
|
20
|
+
rescue HandlerInactive => e
|
21
|
+
Rails.error.report(e, handled: true, severity: :error)
|
22
|
+
render_error_with_status("Webhook handler #{service_id.inspect} is inactive", status: :service_unavailable)
|
21
23
|
rescue => e
|
22
|
-
|
23
|
-
|
24
|
-
# We're setting `handled:` and `severity:` attributes to maintain compatibility with all versions of > rails 7.
|
24
|
+
raise e unless handler
|
25
|
+
raise e if handler.expose_errors_to_sender?
|
25
26
|
Rails.error.report(e, handled: true, severity: :error)
|
26
|
-
|
27
|
-
if handler&.expose_errors_to_sender?
|
28
|
-
error_for_sender_from_exception(e)
|
29
|
-
else
|
30
|
-
head :ok
|
31
|
-
end
|
27
|
+
render_error_with_status("Internal error (#{e})")
|
32
28
|
end
|
33
29
|
|
34
|
-
def
|
35
|
-
|
36
|
-
when HandlerRefused
|
37
|
-
render_error("Webhook handler did not validate the request (signature or authentication may be invalid)", :forbidden)
|
38
|
-
when HandlerInactive
|
39
|
-
render_error("Webhook handler is inactive", :service_unavailable)
|
40
|
-
when JSON::ParserError
|
41
|
-
render_error("Request body is not a valid JSON", :bad_request)
|
42
|
-
else
|
43
|
-
render_error("Internal error", :internal_server_error)
|
44
|
-
end
|
30
|
+
def service_id
|
31
|
+
params.require(:service_id)
|
45
32
|
end
|
46
33
|
|
47
|
-
def
|
48
|
-
json = {error: message_str}.to_json
|
49
|
-
render(json: json, status:
|
34
|
+
def render_error_with_status(message_str, status: :ok)
|
35
|
+
json = {ok: false, error: message_str}.to_json
|
36
|
+
render(json: json, status: status)
|
50
37
|
end
|
51
38
|
|
52
39
|
def lookup_handler(service_id_str)
|
53
|
-
Munster.configuration.active_handlers.with_indifferent_access
|
40
|
+
active_handlers = Munster.configuration.active_handlers.with_indifferent_access
|
41
|
+
# The config can specify a mapping of:
|
42
|
+
# {"service-1" => MyHandler }
|
43
|
+
# or
|
44
|
+
# {"service-2" => "MyOtherHandler"}
|
45
|
+
# We need to support both, because `MyHandler` is not loaded yet when Rails initializers run.
|
46
|
+
# Zeitwerk takes over after the initializers. So we can't really use a module in the init cycle just yet.
|
47
|
+
# We can, however, use the module name - and resolve it lazily, later.
|
48
|
+
handler_class_or_class_name = active_handlers.fetch(service_id_str)
|
49
|
+
handler_class = handler_class_or_class_name.respond_to?(:constantize) ? handler_class_or_class_name.constantize : handler_class_or_class_name
|
50
|
+
handler_class.new
|
51
|
+
rescue KeyError
|
52
|
+
raise UnknownHandler
|
54
53
|
end
|
55
54
|
end
|
56
55
|
end
|
data/lib/munster/engine.rb
CHANGED
@@ -1,6 +1,5 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
require_relative "../munster"
|
4
3
|
require_relative "controllers/receive_webhooks_controller"
|
5
4
|
require_relative "jobs/processing_job"
|
6
5
|
require_relative "models/received_webhook"
|
@@ -10,7 +9,6 @@ module Munster
|
|
10
9
|
class Engine < ::Rails::Engine
|
11
10
|
isolate_namespace Munster
|
12
11
|
|
13
|
-
autoload :Munster, "munster"
|
14
12
|
autoload :ReceiveWebhooksController, "munster/controllers/receive_webhooks_controller"
|
15
13
|
autoload :ProcessingJob, "munster/jobs/processing_job"
|
16
14
|
autoload :BaseHandler, "munster/base_handler"
|
@@ -18,5 +16,9 @@ module Munster
|
|
18
16
|
generators do
|
19
17
|
require_relative "install_generator"
|
20
18
|
end
|
19
|
+
|
20
|
+
routes do
|
21
|
+
post "/:service_id" => "received_webhooks#create"
|
22
|
+
end
|
21
23
|
end
|
22
24
|
end
|
@@ -15,6 +15,7 @@ module Munster
|
|
15
15
|
|
16
16
|
def create_migration_file
|
17
17
|
migration_template "create_munster_tables.rb.erb", File.join(db_migrate_path, "create_munster_tables.rb")
|
18
|
+
migration_template "add_headers_to_munster_webhooks.rb.erb", File.join(db_migrate_path, "add_headers_to_munster_webhooks.rb")
|
18
19
|
end
|
19
20
|
|
20
21
|
def copy_files
|
@@ -4,11 +4,28 @@ require "active_job" if defined?(Rails)
|
|
4
4
|
|
5
5
|
module Munster
|
6
6
|
class ProcessingJob < ActiveJob::Base
|
7
|
+
class WebhookPayloadInvalid < StandardError
|
8
|
+
end
|
9
|
+
|
7
10
|
def perform(webhook)
|
8
|
-
|
9
|
-
|
10
|
-
webhook.
|
11
|
-
|
11
|
+
Rails.error.set_context(munster_handler_module_name: webhook.handler_module_name, **Munster.configuration.error_context)
|
12
|
+
|
13
|
+
webhook.with_lock do
|
14
|
+
return unless webhook.received?
|
15
|
+
webhook.processing!
|
16
|
+
end
|
17
|
+
|
18
|
+
if webhook.handler.valid?(webhook.request)
|
19
|
+
webhook.handler.process(webhook)
|
20
|
+
webhook.processed! if webhook.processing?
|
21
|
+
else
|
22
|
+
e = WebhookPayloadInvalid.new("#{webhook.class} #{webhook.id} did not pass validation and was skipped")
|
23
|
+
Rails.error.report(e, handled: true, severity: :error)
|
24
|
+
webhook.failed_validation!
|
25
|
+
end
|
26
|
+
rescue => e
|
27
|
+
webhook.error!
|
28
|
+
raise e
|
12
29
|
end
|
13
30
|
end
|
14
31
|
end
|
@@ -1,6 +1,6 @@
|
|
1
1
|
# frozen_string_literal: true
|
2
2
|
|
3
|
-
require
|
3
|
+
require "state_machine_enum"
|
4
4
|
|
5
5
|
module Munster
|
6
6
|
class ReceivedWebhook < ActiveRecord::Base
|
@@ -11,13 +11,84 @@ module Munster
|
|
11
11
|
|
12
12
|
state_machine_enum :status do |s|
|
13
13
|
s.permit_transition(:received, :processing)
|
14
|
+
s.permit_transition(:processing, :failed_validation)
|
14
15
|
s.permit_transition(:processing, :skipped)
|
15
16
|
s.permit_transition(:processing, :processed)
|
16
17
|
s.permit_transition(:processing, :error)
|
17
18
|
end
|
18
19
|
|
20
|
+
# Store the pertinent data from an ActionDispatch::Request into the webhook.
|
21
|
+
# @param [ActionDispatch::Request]
|
22
|
+
def request=(action_dispatch_request)
|
23
|
+
# Filter out all Rack-specific headers such as "rack.input" and the like. We are
|
24
|
+
# only interested in the actual HTTP headers presented by the webserver. Mostly...
|
25
|
+
headers = action_dispatch_request.env.filter_map do |(header_name, header_value)|
|
26
|
+
if header_name.is_a?(String) && header_name.upcase == header_name && header_value.is_a?(String)
|
27
|
+
[header_name, header_value]
|
28
|
+
end
|
29
|
+
end.to_h
|
30
|
+
|
31
|
+
# ...except the path parameters - they do not get parsed from the headers, but instead get set by Journey - the Rails
|
32
|
+
# router - when the ActionDispatch::Request object gets instantiated. They need to be preserved separately in case the Munster
|
33
|
+
# controller gets mounted under a parametrized path - and the path component actually is a parameter that the webhook
|
34
|
+
# handler either needs for validation or for processing
|
35
|
+
headers["action_dispatch.request.path_parameters"] = action_dispatch_request.env.fetch("action_dispatch.request.path_parameters")
|
36
|
+
|
37
|
+
# ...and the raw request body - because we already save it separately
|
38
|
+
headers.delete("RAW_POST_DATA")
|
39
|
+
|
40
|
+
# Verify the request body is not too large
|
41
|
+
request_body_io = action_dispatch_request.env.fetch("rack.input")
|
42
|
+
if request_body_io.size > Munster.configuration.request_body_size_limit
|
43
|
+
raise "Cannot accept the webhook as the request body is larger than #{limit} bytes"
|
44
|
+
end
|
45
|
+
|
46
|
+
write_attribute("body", request_body_io.read.force_encoding(Encoding::BINARY))
|
47
|
+
write_attribute("request_headers", headers)
|
48
|
+
ensure
|
49
|
+
request_body_io.rewind
|
50
|
+
end
|
51
|
+
|
52
|
+
# A Munster handler is, in a way, a tiny Rails controller which runs in a background job. To allow this,
|
53
|
+
# we need to provide access not only to the webhook payload (the HTTP request body, usually), but also
|
54
|
+
# to the rest of the HTTP request - such as headers and route params. For example, imagine you use
|
55
|
+
# a system where your multiple tenants (users) may receive webhooks from the same sender. However, you
|
56
|
+
# need to dispatch those webhooks to those particular tenants of your application. Instead of mounting
|
57
|
+
# Munster as an engine under a common route, you mount it like so:
|
58
|
+
#
|
59
|
+
# post "/incoming-webhooks/:user_id/:service_id" => "munster/receive_webhooks#create"
|
60
|
+
#
|
61
|
+
# This way, the tenant ID (the `user_id`) parameter is not going to be provided to you inside the webhook
|
62
|
+
# payload, as the sender is not sending it to you at all. However, you do have that parameter in your
|
63
|
+
# route. When processing the webhook, it is important for you to know which tenant has received the
|
64
|
+
# webhook - so that you can manipulate their data, and not the data belonging to another tenant. With
|
65
|
+
# validation, it is important too - in such a multitenant setup every user is likely to have their own,
|
66
|
+
# specific signing secret that they have set up. To find that secret and compare the signature, you
|
67
|
+
# need access to that `user_id` parameter.
|
68
|
+
#
|
69
|
+
# To allow access to these, Munster allows the ActionDispatch::Request object to be persisted. The
|
70
|
+
# persistence is not 1:1 - the Request is a fairly complex object, with lots of things injected into it
|
71
|
+
# by the Rails stack. Not all of those injected properties (Rack headers) are marshalable, some of them
|
72
|
+
# depend on the Rails application configuration, etc. However, we do retain the most important things
|
73
|
+
# for webhooks to be correctly handled.
|
74
|
+
#
|
75
|
+
# * The HTTP request body
|
76
|
+
# * The headers set by the webserver and the downstream proxies
|
77
|
+
# * The request body and query string params, depending on the MIME type
|
78
|
+
# * The route params. These are set by Journey (the Rails router) and cannot be reconstructed from a "bare" request
|
79
|
+
#
|
80
|
+
# While this reconstruction is best-effort it might not be lossless. For example, there might be no access
|
81
|
+
# to Rack hijack, streaming APIs, the cookie jar or other more high-level Rails request access features.
|
82
|
+
# You will, however, have the basics in place - such as the params, the request body, the path params
|
83
|
+
# (as were decoded by your routes) etc. But it should be sufficient to do the basic tasks to process a webhook.
|
84
|
+
#
|
85
|
+
# @return [ActionDispatch::Request]
|
86
|
+
def request
|
87
|
+
ActionDispatch::Request.new(request_headers.merge!("rack.input" => StringIO.new(body.to_s.b)))
|
88
|
+
end
|
89
|
+
|
19
90
|
def handler
|
20
|
-
handler_module_name.constantize
|
91
|
+
handler_module_name.constantize.new
|
21
92
|
end
|
22
93
|
end
|
23
94
|
end
|
@@ -6,7 +6,7 @@ class CreateMunsterTables < ActiveRecord::Migration<%= migration_version %>
|
|
6
6
|
t.string :handler_module_name, null: false
|
7
7
|
t.string :status, default: "received", null: false
|
8
8
|
|
9
|
-
# We don't assume
|
9
|
+
# We don't assume that we can always parse the received body as JSON. Body could be invalid or partly missing,
|
10
10
|
# we can argue how we can handle that for different integrations, but we still should be able to save this data
|
11
11
|
# if it's required. Hence, we don't use :jsonb, but :binary type column here.
|
12
12
|
t.binary :body, null: false
|
data/lib/munster/version.rb
CHANGED
data/lib/munster.rb
CHANGED
@@ -18,7 +18,8 @@ end
|
|
18
18
|
class Munster::Configuration
|
19
19
|
include ActiveSupport::Configurable
|
20
20
|
|
21
|
-
config_accessor(:processing_job_class
|
22
|
-
config_accessor(:active_handlers
|
23
|
-
config_accessor(:error_context
|
21
|
+
config_accessor(:processing_job_class, default: Munster::ProcessingJob)
|
22
|
+
config_accessor(:active_handlers, default: {})
|
23
|
+
config_accessor(:error_context, default: {})
|
24
|
+
config_accessor(:request_body_size_limit, default: 512.kilobytes)
|
24
25
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: munster
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.4.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Stanislav Katkov
|
8
8
|
autorequire:
|
9
|
-
bindir:
|
9
|
+
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2024-
|
11
|
+
date: 2024-07-24 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rails
|
@@ -94,7 +94,7 @@ dependencies:
|
|
94
94
|
- - "~>"
|
95
95
|
- !ruby/object:Gem::Version
|
96
96
|
version: '5.0'
|
97
|
-
description: Webhooks
|
97
|
+
description: Webhooks processing engine for Rails applications
|
98
98
|
email:
|
99
99
|
- skatkov@cheddar.me
|
100
100
|
executables: []
|
@@ -106,7 +106,6 @@ files:
|
|
106
106
|
- ".standard.yml"
|
107
107
|
- CHANGELOG.md
|
108
108
|
- LICENSE
|
109
|
-
- LICENSE.txt
|
110
109
|
- README.md
|
111
110
|
- Rakefile
|
112
111
|
- config/routes.rb
|
@@ -178,17 +177,18 @@ files:
|
|
178
177
|
- lib/munster/install_generator.rb
|
179
178
|
- lib/munster/jobs/processing_job.rb
|
180
179
|
- lib/munster/models/received_webhook.rb
|
180
|
+
- lib/munster/templates/add_headers_to_munster_webhooks.rb.erb
|
181
181
|
- lib/munster/templates/create_munster_tables.rb.erb
|
182
182
|
- lib/munster/templates/munster.rb
|
183
183
|
- lib/munster/version.rb
|
184
184
|
- lib/tasks/munster_tasks.rake
|
185
|
-
- sig/munster.rbs
|
186
185
|
homepage: https://www.cheddar.me/
|
187
186
|
licenses:
|
188
187
|
- MIT
|
189
188
|
metadata:
|
190
189
|
homepage_uri: https://www.cheddar.me/
|
191
190
|
source_code_uri: https://github.com/cheddar-me/munster
|
191
|
+
changelog_uri: https://github.com/cheddar-me/munster/blob/main/CHANGELOG.md
|
192
192
|
post_install_message:
|
193
193
|
rdoc_options: []
|
194
194
|
require_paths:
|
@@ -204,8 +204,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
204
204
|
- !ruby/object:Gem::Version
|
205
205
|
version: '0'
|
206
206
|
requirements: []
|
207
|
-
rubygems_version: 3.5.
|
207
|
+
rubygems_version: 3.5.9
|
208
208
|
signing_key:
|
209
209
|
specification_version: 4
|
210
|
-
summary: Webhooks
|
210
|
+
summary: Webhooks processing engine for Rails applications
|
211
211
|
test_files: []
|
data/LICENSE.txt
DELETED
@@ -1,21 +0,0 @@
|
|
1
|
-
The MIT License (MIT)
|
2
|
-
|
3
|
-
Copyright (c) 2024 Stanislav Katkov
|
4
|
-
|
5
|
-
Permission is hereby granted, free of charge, to any person obtaining a copy
|
6
|
-
of this software and associated documentation files (the "Software"), to deal
|
7
|
-
in the Software without restriction, including without limitation the rights
|
8
|
-
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
|
9
|
-
copies of the Software, and to permit persons to whom the Software is
|
10
|
-
furnished to do so, subject to the following conditions:
|
11
|
-
|
12
|
-
The above copyright notice and this permission notice shall be included in
|
13
|
-
all copies or substantial portions of the Software.
|
14
|
-
|
15
|
-
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
|
16
|
-
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
|
17
|
-
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
|
18
|
-
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
|
19
|
-
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
|
20
|
-
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
|
21
|
-
THE SOFTWARE.
|
data/sig/munster.rbs
DELETED