outboxer 0.1.0 → 0.1.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/README.md +50 -22
- data/generators/outboxer/templates/bin/publisher.rb +8 -1
- data/lib/outboxer/models/message.rb +2 -1
- data/lib/outboxer/publisher.rb +19 -0
- data/lib/outboxer/version.rb +1 -1
- data/lib/outboxer.rb +2 -0
- metadata +1 -1
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: fe0005e2eb4b31e7d9a16694ad68b7f2b3ed349e2153a435547963a714eb6a22
|
4
|
+
data.tar.gz: 703db7ef7cb96cf6ac8cf95aae7bd27d52880c0abf88fa28e9beebf6a0fa328c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: afa1718ef1bcf5d3c375080c53ec347925999023832eb0539bb0ec542f20140e00f8f4795fdf4371ad833a5cd2ced0f8715d692635e66860e4219d38c5b11c99
|
7
|
+
data.tar.gz: 99ea377b056288cc7e5958665d787e765a511f697d26f4a0b4b091b955b776a4ce05fdd5e833e1ba3f38914a25376867f99f24fb18e58dfb6f98cbf1fc4efe98
|
data/README.md
CHANGED
@@ -1,10 +1,23 @@
|
|
1
1
|
# Outboxer
|
2
2
|
|
3
|
-
|
3
|
+
## Background
|
4
4
|
|
5
|
-
|
5
|
+
Typically in event driven Ruby on Rails applications:
|
6
6
|
|
7
|
-
|
7
|
+
1. a domain event model is created in an SQL database table
|
8
|
+
2. a sidekiq worker is queued to handle the domain event asynchronously
|
9
|
+
|
10
|
+
## Problem
|
11
|
+
|
12
|
+
As these two operations span multiple database types (SQL and redis), they can not be combined into a single atomic operation using a transaction. If either step fails, inconsistencies can occur.
|
13
|
+
|
14
|
+
## Solution
|
15
|
+
|
16
|
+
Outboxer is a simple Ruby on Rails implementation of the [transactional outbox pattern](https://microservices.io/patterns/data/transactional-outbox.html): a well established solution to this problem. It ensures both operations succeed _eventually_, or both fail.
|
17
|
+
|
18
|
+
### Getting started
|
19
|
+
|
20
|
+
### Installation
|
8
21
|
|
9
22
|
1. Add the Outboxer gem to your application's Gemfile:
|
10
23
|
|
@@ -24,18 +37,9 @@ bundle install
|
|
24
37
|
bin/rails generate outboxer:install
|
25
38
|
```
|
26
39
|
|
27
|
-
|
28
|
-
|
29
|
-
### 1. Migrate your database
|
30
|
-
|
31
|
-
```bash
|
32
|
-
bin/rake db:migrate
|
33
|
-
```
|
34
|
-
|
35
|
-
|
36
|
-
### 2. Include Outboxer into existing model
|
40
|
+
### Usage
|
37
41
|
|
38
|
-
|
42
|
+
#### 1. Include `Outboxer::Outboxable` into your existing Message model
|
39
43
|
|
40
44
|
```ruby
|
41
45
|
class Message < ApplicationRecord
|
@@ -43,26 +47,50 @@ class Message < ApplicationRecord
|
|
43
47
|
end
|
44
48
|
```
|
45
49
|
|
46
|
-
|
50
|
+
#### 2. Update the generated bin/publish block e.g.
|
47
51
|
|
48
|
-
|
52
|
+
```ruby
|
53
|
+
Outboxer::Publisher.publish do |args|
|
54
|
+
args.logger.info("[#{message.id}] publishing")
|
49
55
|
|
50
|
-
|
56
|
+
Worker.perform_async({ message_id: args.message.id })
|
51
57
|
|
52
|
-
|
53
|
-
Outboxer::Publisher.publish do |message:, logger:|
|
54
|
-
Worker.perform_async({ message_id: message.id })
|
58
|
+
args.logger.info("[#{message.id}] published")
|
55
59
|
end
|
56
60
|
```
|
57
61
|
|
58
|
-
|
62
|
+
#### 3. Migrate the database
|
59
63
|
|
60
|
-
|
64
|
+
```bash
|
65
|
+
bin/rake db:migrate
|
66
|
+
```
|
67
|
+
|
68
|
+
#### 4. Run the publisher
|
61
69
|
|
62
70
|
```bash
|
63
71
|
bin/publisher
|
64
72
|
```
|
65
73
|
|
74
|
+
## Implementation
|
75
|
+
|
76
|
+
1. when an `ActiveRecord` model that includes `Outbox::Outboxable` is created, an `unpublished` `Outboxer::Message` is automatically created in the same transaction, with `Outboxer::Message#message` polymorphically assigned to the original model
|
77
|
+
|
78
|
+
2. When the publisher finds a new `unpublished` `Outboxer::Message`, it yields to a user-supplied block and then:
|
79
|
+
- removes it if the task completes successfully
|
80
|
+
- marks it as failed and records the error if there's a problem
|
81
|
+
|
82
|
+
To see all the parts working together in a single place, check out the [publisher_spec.rb](https://github.com/fast-programmer/outboxer/blob/master/spec/outboxer/publisher_spec.rb)
|
83
|
+
|
84
|
+
|
85
|
+
## Motivation
|
86
|
+
|
87
|
+
Outboxer was created with 4 key benefits in mind:
|
88
|
+
|
89
|
+
1. speed of integration into existing Ruby on Rails applications (< 1 hour)
|
90
|
+
2. comprehensive documentation that is easy to understand
|
91
|
+
3. high reliability in production environments (100% code coverage)
|
92
|
+
4. forever free to use in commerical applications (MIT licence)
|
93
|
+
|
66
94
|
## Contributing
|
67
95
|
|
68
96
|
Bug reports and pull requests are welcome on GitHub at https://github.com/fast-programmer/outboxer. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/fast-programmer/outboxer/blob/main/CODE_OF_CONDUCT.md).
|
@@ -3,5 +3,12 @@
|
|
3
3
|
require_relative "../config/environment"
|
4
4
|
|
5
5
|
Outboxer::Publisher.publish do |args|
|
6
|
-
|
6
|
+
logger = args.logger
|
7
|
+
message = args.message
|
8
|
+
|
9
|
+
logger.info("[#{message.id}] publishing")
|
10
|
+
|
11
|
+
HardJob.perform_async({ "message_id" => message.id })
|
12
|
+
|
13
|
+
logger.info("[#{message.id}] published")
|
7
14
|
end
|
data/lib/outboxer/publisher.rb
CHANGED
@@ -9,6 +9,17 @@ module Outboxer
|
|
9
9
|
|
10
10
|
Args = Struct.new(:message, :logger)
|
11
11
|
|
12
|
+
# This method initiates the publishing process. It dequeues the messages one by one
|
13
|
+
# and yields to a block that should contain the publishing logic.
|
14
|
+
#
|
15
|
+
# @param [Integer] poll
|
16
|
+
# Sleep time in seconds between polling the queue when it's empty.
|
17
|
+
#
|
18
|
+
# @param [Proc] backoff
|
19
|
+
# A Proc that takes the current backoff time and returns the new backoff time.
|
20
|
+
#
|
21
|
+
# @yieldparam [Args] args
|
22
|
+
# An Args object containing the message to publish and a logger.
|
12
23
|
def publish(poll: 1, backoff: ->(current_backoff) { [current_backoff * 2, 5 * 60].min })
|
13
24
|
@publishing = true
|
14
25
|
|
@@ -100,6 +111,12 @@ module Outboxer
|
|
100
111
|
end
|
101
112
|
end
|
102
113
|
|
114
|
+
# Stops the publishing process.
|
115
|
+
#
|
116
|
+
# @note This method will stop the current message publishing process
|
117
|
+
# It is a safe way to interrupt the publishing process at any point.
|
118
|
+
#
|
119
|
+
# @return [void]
|
103
120
|
def stop
|
104
121
|
@publishing = false
|
105
122
|
end
|
@@ -109,5 +126,7 @@ module Outboxer
|
|
109
126
|
|
110
127
|
stop
|
111
128
|
end
|
129
|
+
|
130
|
+
private_class_method :retry_on_error, :dequeue, :published, :failed
|
112
131
|
end
|
113
132
|
end
|
data/lib/outboxer/version.rb
CHANGED
data/lib/outboxer.rb
CHANGED