journaled 2.5.0 → 4.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +115 -24
- data/Rakefile +7 -1
- data/app/controllers/concerns/journaled/actor.rb +8 -5
- data/app/jobs/journaled/application_job.rb +4 -0
- data/app/jobs/journaled/delivery_job.rb +105 -0
- data/app/models/journaled/actor_uri_provider.rb +1 -2
- data/app/models/journaled/change.rb +3 -3
- data/app/models/journaled/change_writer.rb +5 -5
- data/app/models/journaled/event.rb +25 -3
- data/app/models/journaled/writer.rb +17 -14
- data/journaled_schemas/tagged_event.json +14 -0
- data/lib/journaled/current.rb +18 -0
- data/lib/journaled/engine.rb +5 -0
- data/lib/journaled/version.rb +1 -1
- data/lib/journaled.rb +35 -5
- data/spec/dummy/config/application.rb +1 -2
- data/spec/dummy/config/database.yml +4 -19
- data/spec/dummy/config/environments/development.rb +0 -13
- data/spec/dummy/config/environments/test.rb +3 -5
- data/spec/dummy/db/schema.rb +3 -16
- data/spec/{models/journaled/delivery_spec.rb → jobs/journaled/delivery_job_spec.rb} +79 -25
- data/spec/lib/journaled_spec.rb +39 -0
- data/spec/models/concerns/journaled/actor_spec.rb +8 -7
- data/spec/models/database_change_protection_spec.rb +19 -25
- data/spec/models/journaled/actor_uri_provider_spec.rb +6 -5
- data/spec/models/journaled/change_writer_spec.rb +10 -10
- data/spec/models/journaled/event_spec.rb +82 -6
- data/spec/models/journaled/writer_spec.rb +47 -15
- data/spec/rails_helper.rb +1 -2
- data/spec/spec_helper.rb +1 -3
- metadata +29 -84
- data/app/models/journaled/delivery.rb +0 -88
- data/config/routes.rb +0 -2
- data/lib/journaled/enqueue.rb +0 -13
- data/spec/dummy/config/environments/production.rb +0 -78
- data/spec/dummy/config/initializers/assets.rb +0 -8
- data/spec/dummy/db/migrate/20180606205114_create_delayed_jobs.rb +0 -18
- data/spec/support/delayed_job_spec_helper.rb +0 -11
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 88602263644ea51719a18c5ee3e719d2c13a9e69049e1f20a52a44edc38a0835
|
4
|
+
data.tar.gz: c7cafdb9f305e51c3590d09815d5d6f909db496866c774ab442ffaa334be927c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 6bf229efd53882ed50931c964c1564b01603fa22ea2083b8a6039d0a0ed972d2cdc7bfc5913a9e9539d7dd023294c2a226963e2e2a526bf857926afe3e0c76c0
|
7
|
+
data.tar.gz: cf7b28c58c40c9ee8348df60095a7b544c0a8b0ea448ffa20021aceda61ae2feda61110ef9685756d4bac7d3d887e12a181f1840d435b2d2f85aeba5d8a52e45
|
data/README.md
CHANGED
@@ -1,16 +1,16 @@
|
|
1
1
|
# Journaled
|
2
2
|
|
3
|
-
A Rails engine to durably deliver schematized events to Amazon Kinesis via
|
3
|
+
A Rails engine to durably deliver schematized events to Amazon Kinesis via ActiveJob.
|
4
4
|
|
5
5
|
More specifically, `journaled` is composed of three opinionated pieces:
|
6
6
|
schema definition/validation via JSON Schema, transactional enqueueing
|
7
|
-
via
|
7
|
+
via ActiveJob (specifically, via a DB-backed queue adapter), and event
|
8
8
|
transmission via Amazon Kinesis. Our current use-cases include
|
9
9
|
transmitting audit events for durable storage in S3 and/or analytical
|
10
10
|
querying in Amazon Redshift.
|
11
11
|
|
12
12
|
Journaled provides an at-least-once event delivery guarantee assuming
|
13
|
-
|
13
|
+
ActiveJob's queue adapter is not configured to delete jobs on failure.
|
14
14
|
|
15
15
|
Note: Do not use the journaled gem to build an event sourcing solution
|
16
16
|
as it does not guarantee total ordering of events. It's possible we'll
|
@@ -18,11 +18,24 @@ add scoped ordering capability at a future date (and would gladly
|
|
18
18
|
entertain pull requests), but it is presently only designed to provide a
|
19
19
|
durable, eventually consistent record that discrete events happened.
|
20
20
|
|
21
|
+
**See [upgrades](#upgrades) below if you're upgrading from an older `journaled` version!**
|
22
|
+
|
21
23
|
## Installation
|
22
24
|
|
23
|
-
1.
|
24
|
-
|
25
|
+
1. If you haven't already,
|
26
|
+
[configure ActiveJob](https://guides.rubyonrails.org/active_job_basics.html)
|
27
|
+
to use one of the following queue adapters:
|
28
|
+
|
29
|
+
- `:delayed_job` (via `delayed_job_active_record`)
|
30
|
+
- `:que`
|
31
|
+
- `:good_job`
|
32
|
+
- `:delayed`
|
25
33
|
|
34
|
+
Ensure that your queue adapter is not configured to delete jobs on failure.
|
35
|
+
|
36
|
+
**If you launch your application in production mode and the gem detects that
|
37
|
+
`ActiveJob::Base.queue_adapter` is not in the above list, it will raise an exception
|
38
|
+
and prevent your application from performing unsafe journaling.**
|
26
39
|
|
27
40
|
2. To integrate Journaled into your application, simply include the gem in your
|
28
41
|
app's Gemfile.
|
@@ -40,20 +53,21 @@ app's Gemfile.
|
|
40
53
|
require 'journaled/rspec'
|
41
54
|
```
|
42
55
|
|
43
|
-
3. You will
|
56
|
+
3. You will need to set the following config in an initializer to allow Journaled to publish events to your AWS Kinesis event stream:
|
44
57
|
|
45
|
-
|
58
|
+
```ruby
|
59
|
+
Journaled.default_stream_name = "my_app_#{Rails.env}_events"
|
60
|
+
```
|
46
61
|
|
47
|
-
|
48
|
-
`#journaled_app_name` method to a non-nil value e.g. `my_app`, you will
|
49
|
-
instead need to provide a corresponding
|
50
|
-
`[upcased_app_name]_JOURNALED_STREAM_NAME` variable for each distinct
|
51
|
-
value, e.g. `MY_APP_JOURNALED_STREAM_NAME`. You can provide a default value
|
52
|
-
for all `Journaled::Event`s in an initializer like this:
|
62
|
+
You may also define a `#journaled_stream_name` method on `Journaled::Event` instances:
|
53
63
|
|
54
64
|
```ruby
|
55
|
-
|
56
|
-
|
65
|
+
def journaled_stream_name
|
66
|
+
"my_app_#{Rails.env}_alternate_events"
|
67
|
+
end
|
68
|
+
````
|
69
|
+
|
70
|
+
3. You may also need to define environment variables to allow Journaled to publish events to your AWS Kinesis event stream:
|
57
71
|
|
58
72
|
You may optionally define the following ENV vars to specify AWS
|
59
73
|
credentials outside of the locations that the AWS SDK normally looks:
|
@@ -77,17 +91,17 @@ app's Gemfile.
|
|
77
91
|
|
78
92
|
Journaling provides a number of different configuation options that can be set in Ruby using an initializer. Those values are:
|
79
93
|
|
80
|
-
#### `Journaled.
|
94
|
+
#### `Journaled.default_stream_name `
|
81
95
|
|
82
|
-
This is described in the
|
83
|
-
This is the default value for events that do NOT specify their own `#journaled_app_name`. For events that define their own `#journaled_app_name` method, that will take precedence over this default.
|
84
|
-
Ex: `Journaled.default_app_name = 'my_app'`
|
96
|
+
This is described in the "Installation" section above, and is used to specify which stream name to use.
|
85
97
|
|
86
98
|
#### `Journaled.job_priority` (default: 20)
|
87
99
|
|
88
|
-
This can be used to configure what `priority` the
|
100
|
+
This can be used to configure what `priority` the ActiveJobs are enqueued with. This will be applied to all the `Journaled::DeliveryJob`s that are created by this application.
|
89
101
|
Ex: `Journaled.job_priority = 14`
|
90
102
|
|
103
|
+
_Note that job priority is only supported on Rails 6.0+. Prior Rails versions will ignore this parameter and enqueue jobs with the underlying ActiveJob adapter's default priority._
|
104
|
+
|
91
105
|
#### `Journaled.http_idle_timeout` (default: 1 second)
|
92
106
|
|
93
107
|
The number of seconds a persistent connection is allowed to sit idle before it should no longer be used.
|
@@ -100,9 +114,9 @@ Journaling provides a number of different configuation options that can be set i
|
|
100
114
|
|
101
115
|
The number of seconds before the :http_handler should timeout while waiting for a HTTP response.
|
102
116
|
|
103
|
-
####
|
117
|
+
#### ActiveJob `set` options
|
104
118
|
|
105
|
-
Both model-level directives accept additional options to be passed into
|
119
|
+
Both model-level directives accept additional options to be passed into ActiveJob's `set` method:
|
106
120
|
|
107
121
|
```ruby
|
108
122
|
# For change journaling:
|
@@ -136,8 +150,8 @@ class ApplicationController < ActionController::Base
|
|
136
150
|
end
|
137
151
|
```
|
138
152
|
|
139
|
-
Your authenticated entity must respond to `#to_global_id`, which
|
140
|
-
|
153
|
+
Your authenticated entity must respond to `#to_global_id`, which ActiveRecords do by default.
|
154
|
+
This feature relies on `ActiveSupport::CurrentAttributes` under the hood.
|
141
155
|
|
142
156
|
Every time any of the specified attributes is modified, or a `User`
|
143
157
|
record is created or destroyed, an event will be sent to Kinesis with the following attributes:
|
@@ -165,6 +179,40 @@ journaling. Note that the less-frequently-used methods `toggle`,
|
|
165
179
|
`increment*`, `decrement*`, and `update_counters` are not intercepted at
|
166
180
|
this time.
|
167
181
|
|
182
|
+
### Tagged Events
|
183
|
+
|
184
|
+
Events may be optionally marked as "tagged." This will add a `tags` field, intended for tracing and
|
185
|
+
auditing purposes.
|
186
|
+
|
187
|
+
```ruby
|
188
|
+
class MyEvent
|
189
|
+
include Journaled::Event
|
190
|
+
|
191
|
+
journal_attributes :attr_1, :attr_2, tagged: true
|
192
|
+
end
|
193
|
+
```
|
194
|
+
|
195
|
+
You may then use `Journaled.tag!` and `Journaled.tagged` inside of your
|
196
|
+
`ApplicationController` and `ApplicationJob` classes (or anywhere else!) to tag
|
197
|
+
all events with request and job metadata:
|
198
|
+
|
199
|
+
```ruby
|
200
|
+
class ApplicationController < ActionController::Base
|
201
|
+
before_action do
|
202
|
+
Journaled.tag!(request_id: request.request_id, current_user_id: current_user&.id)
|
203
|
+
end
|
204
|
+
end
|
205
|
+
|
206
|
+
class ApplicationJob < ActiveJob::Base
|
207
|
+
around_perform do |job, perform|
|
208
|
+
Journaled.tagged(job_id: job.id) { perform.call }
|
209
|
+
end
|
210
|
+
end
|
211
|
+
```
|
212
|
+
|
213
|
+
This feature relies on `ActiveSupport::CurrentAttributes` under the hood, so these tags are local to
|
214
|
+
the current thread, and will be cleared at the end of each request request/job.
|
215
|
+
|
168
216
|
#### Testing
|
169
217
|
|
170
218
|
If you use RSpec (and have required `journaled/rspec` in your
|
@@ -304,6 +352,49 @@ Returns one of the following in order of preference:
|
|
304
352
|
In order for this to be most useful, you must configure your controller
|
305
353
|
as described in [Change Journaling](#change-journaling) above.
|
306
354
|
|
355
|
+
## Upgrades
|
356
|
+
|
357
|
+
Since this gem relies on background jobs (which can remain in the queue across
|
358
|
+
code releases), this gem generally aims to support jobs enqueued by the prior
|
359
|
+
gem version.
|
360
|
+
|
361
|
+
As such, **we always recommend upgrading only one major version at a time.**
|
362
|
+
|
363
|
+
### Upgrading from 3.1.0
|
364
|
+
|
365
|
+
Versions of Journaled prior to 4.0 relied directly on environment variables for stream names, but now stream names are configured directly.
|
366
|
+
When upgrading, you can use the following configuration to maintain the previous behavior:
|
367
|
+
|
368
|
+
```ruby
|
369
|
+
Journaled.default_stream_name = ENV['JOURNALED_STREAM_NAME']
|
370
|
+
```
|
371
|
+
|
372
|
+
If you previously specified a `Journaled.default_app_name`, you would have required a more precise environment variable name (substitute `{{upcase_app_name}}`):
|
373
|
+
|
374
|
+
```ruby
|
375
|
+
Journaled.default_stream_name = ENV["{{upcase_app_name}}_JOURNALED_STREAM_NAME"]
|
376
|
+
```
|
377
|
+
|
378
|
+
And if you had defined any `journaled_app_name` methods on `Journaled::Event` instances, you can replace them with the following:
|
379
|
+
|
380
|
+
```ruby
|
381
|
+
def journaled_stream_name
|
382
|
+
ENV['{{upcase_app_name}}_JOURNALED_STREAM_NAME']
|
383
|
+
end
|
384
|
+
```
|
385
|
+
|
386
|
+
When upgrading from 3.1 or below, `Journaled::DeliveryJob` will handle any jobs that remain in the queue by accepting an `app_name` argument. **This behavior will be removed in version 5.0**, so it is recommended to upgrade one major version at a time.
|
387
|
+
|
388
|
+
### Upgrading from 2.5.0
|
389
|
+
|
390
|
+
Versions of Journaled prior to 3.0 relied direclty on `delayed_job` and a "performable" class called `Journaled::Delivery`.
|
391
|
+
In 3.0, this was superceded by an ActiveJob class called `Journaled::DeliveryJob`, but the `Journaled::Delivery` class was not removed until 4.0.
|
392
|
+
|
393
|
+
Therefore, when upgrading from 2.5.0 or below, it is recommended to first upgrade to 3.1.0 (to allow any `Journaled::Delivery` jobs to finish working off) before upgrading to 4.0+.
|
394
|
+
|
395
|
+
The upgrade to 3.1.0 will require a working ActiveJob config. ActiveJob can be configured globally by setting `ActiveJob::Base.queue_adapter`, or just for Journaled jobs by setting `Journaled::DeliveryJob.queue_adapter`.
|
396
|
+
The `:delayed_job` queue adapter will allow you to continue relying on `delayed_job`. You may also consider switching your app(s) to [`delayed`](https://github.com/Betterment/delayed) and using the `:delayed` queue adapter.
|
397
|
+
|
307
398
|
## Future improvements & issue tracking
|
308
399
|
Suggestions for enhancements to this engine are currently being tracked via Github Issues. Please feel free to open an
|
309
400
|
issue for a desired feature, as well as for any observed bugs.
|
data/Rakefile
CHANGED
@@ -28,7 +28,13 @@ if %w(development test).include? Rails.env
|
|
28
28
|
RuboCop::RakeTask.new
|
29
29
|
|
30
30
|
task(:default).clear
|
31
|
-
|
31
|
+
if ENV['APPRAISAL_INITIALIZED'] || ENV['CI']
|
32
|
+
task default: %i(rubocop spec)
|
33
|
+
else
|
34
|
+
require 'appraisal'
|
35
|
+
Appraisal::Task.new
|
36
|
+
task default: :appraisal
|
37
|
+
end
|
32
38
|
|
33
39
|
task 'db:test:prepare' => 'db:setup'
|
34
40
|
end
|
@@ -2,11 +2,14 @@ module Journaled::Actor
|
|
2
2
|
extend ActiveSupport::Concern
|
3
3
|
|
4
4
|
included do
|
5
|
-
class_attribute :_journaled_actor_method_name,
|
6
|
-
before_action
|
7
|
-
|
8
|
-
|
9
|
-
|
5
|
+
class_attribute :_journaled_actor_method_name, instance_writer: false
|
6
|
+
before_action :_set_journaled_actor_proc, if: :_journaled_actor_method_name?
|
7
|
+
end
|
8
|
+
|
9
|
+
private
|
10
|
+
|
11
|
+
def _set_journaled_actor_proc
|
12
|
+
Journaled::Current.journaled_actor_proc = -> { send(self.class._journaled_actor_method_name) }
|
10
13
|
end
|
11
14
|
|
12
15
|
class_methods do
|
@@ -0,0 +1,105 @@
|
|
1
|
+
module Journaled
|
2
|
+
class DeliveryJob < ApplicationJob
|
3
|
+
DEFAULT_REGION = 'us-east-1'.freeze
|
4
|
+
|
5
|
+
rescue_from(Aws::Kinesis::Errors::InternalFailure, Aws::Kinesis::Errors::ServiceUnavailable, Aws::Kinesis::Errors::Http503Error) do |e|
|
6
|
+
Rails.logger.error "Kinesis Error - Server Error occurred - #{e.class}"
|
7
|
+
raise KinesisTemporaryFailure
|
8
|
+
end
|
9
|
+
|
10
|
+
rescue_from(Seahorse::Client::NetworkingError) do |e|
|
11
|
+
Rails.logger.error "Kinesis Error - Networking Error occurred - #{e.class}"
|
12
|
+
raise KinesisTemporaryFailure
|
13
|
+
end
|
14
|
+
|
15
|
+
UNSPECIFIED = Object.new
|
16
|
+
private_constant :UNSPECIFIED
|
17
|
+
|
18
|
+
def perform(serialized_event:, partition_key:, stream_name: UNSPECIFIED, app_name: UNSPECIFIED)
|
19
|
+
@serialized_event = serialized_event
|
20
|
+
@partition_key = partition_key
|
21
|
+
if app_name != UNSPECIFIED
|
22
|
+
@stream_name = self.class.legacy_computed_stream_name(app_name: app_name)
|
23
|
+
elsif stream_name != UNSPECIFIED && !stream_name.nil?
|
24
|
+
@stream_name = stream_name
|
25
|
+
else
|
26
|
+
raise(ArgumentError, 'missing keyword: stream_name')
|
27
|
+
end
|
28
|
+
|
29
|
+
journal!
|
30
|
+
end
|
31
|
+
|
32
|
+
def self.legacy_computed_stream_name(app_name:)
|
33
|
+
env_var_name = [app_name&.upcase, 'JOURNALED_STREAM_NAME'].compact.join('_')
|
34
|
+
ENV.fetch(env_var_name)
|
35
|
+
end
|
36
|
+
|
37
|
+
def kinesis_client_config
|
38
|
+
{
|
39
|
+
region: ENV.fetch('AWS_DEFAULT_REGION', DEFAULT_REGION),
|
40
|
+
retry_limit: 0,
|
41
|
+
http_idle_timeout: Journaled.http_idle_timeout,
|
42
|
+
http_open_timeout: Journaled.http_open_timeout,
|
43
|
+
http_read_timeout: Journaled.http_read_timeout,
|
44
|
+
}.merge(credentials)
|
45
|
+
end
|
46
|
+
|
47
|
+
private
|
48
|
+
|
49
|
+
attr_reader :serialized_event, :partition_key, :stream_name
|
50
|
+
|
51
|
+
def journal!
|
52
|
+
kinesis_client.put_record record if Journaled.enabled?
|
53
|
+
end
|
54
|
+
|
55
|
+
def record
|
56
|
+
{
|
57
|
+
stream_name: stream_name,
|
58
|
+
data: serialized_event,
|
59
|
+
partition_key: partition_key,
|
60
|
+
}
|
61
|
+
end
|
62
|
+
|
63
|
+
def kinesis_client
|
64
|
+
Aws::Kinesis::Client.new(kinesis_client_config)
|
65
|
+
end
|
66
|
+
|
67
|
+
def credentials
|
68
|
+
if ENV.key?('JOURNALED_IAM_ROLE_ARN')
|
69
|
+
{
|
70
|
+
credentials: iam_assume_role_credentials,
|
71
|
+
}
|
72
|
+
else
|
73
|
+
legacy_credentials_hash_if_present
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
def legacy_credentials_hash_if_present
|
78
|
+
if ENV.key?('RUBY_AWS_ACCESS_KEY_ID')
|
79
|
+
{
|
80
|
+
access_key_id: ENV.fetch('RUBY_AWS_ACCESS_KEY_ID'),
|
81
|
+
secret_access_key: ENV.fetch('RUBY_AWS_SECRET_ACCESS_KEY'),
|
82
|
+
}
|
83
|
+
else
|
84
|
+
{}
|
85
|
+
end
|
86
|
+
end
|
87
|
+
|
88
|
+
def sts_client
|
89
|
+
Aws::STS::Client.new({
|
90
|
+
region: ENV.fetch('AWS_DEFAULT_REGION', DEFAULT_REGION),
|
91
|
+
}.merge(legacy_credentials_hash_if_present))
|
92
|
+
end
|
93
|
+
|
94
|
+
def iam_assume_role_credentials
|
95
|
+
@iam_assume_role_credentials ||= Aws::AssumeRoleCredentials.new(
|
96
|
+
client: sts_client,
|
97
|
+
role_arn: ENV.fetch('JOURNALED_IAM_ROLE_ARN'),
|
98
|
+
role_session_name: "JournaledAssumeRoleAccess",
|
99
|
+
)
|
100
|
+
end
|
101
|
+
|
102
|
+
class KinesisTemporaryFailure < NotTrulyExceptionalError
|
103
|
+
end
|
104
|
+
end
|
105
|
+
end
|
@@ -6,7 +6,7 @@ class Journaled::Change
|
|
6
6
|
:database_operation,
|
7
7
|
:logical_operation,
|
8
8
|
:changes,
|
9
|
-
:
|
9
|
+
:journaled_stream_name,
|
10
10
|
:journaled_enqueue_opts,
|
11
11
|
:actor
|
12
12
|
|
@@ -22,7 +22,7 @@ class Journaled::Change
|
|
22
22
|
database_operation:,
|
23
23
|
logical_operation:,
|
24
24
|
changes:,
|
25
|
-
|
25
|
+
journaled_stream_name:,
|
26
26
|
journaled_enqueue_opts:,
|
27
27
|
actor:)
|
28
28
|
@table_name = table_name
|
@@ -30,7 +30,7 @@ class Journaled::Change
|
|
30
30
|
@database_operation = database_operation
|
31
31
|
@logical_operation = logical_operation
|
32
32
|
@changes = changes
|
33
|
-
@
|
33
|
+
@journaled_stream_name = journaled_stream_name
|
34
34
|
@journaled_enqueue_opts = journaled_enqueue_opts
|
35
35
|
@actor = actor
|
36
36
|
end
|
@@ -27,7 +27,7 @@ class Journaled::ChangeWriter
|
|
27
27
|
database_operation: database_operation,
|
28
28
|
logical_operation: logical_operation,
|
29
29
|
changes: JSON.dump(changes),
|
30
|
-
|
30
|
+
journaled_stream_name: journaled_stream_name,
|
31
31
|
journaled_enqueue_opts: model.journaled_enqueue_opts,
|
32
32
|
actor: actor_uri,
|
33
33
|
)
|
@@ -57,11 +57,11 @@ class Journaled::ChangeWriter
|
|
57
57
|
end
|
58
58
|
end
|
59
59
|
|
60
|
-
def
|
61
|
-
if model.class.respond_to?(:
|
62
|
-
model.class.
|
60
|
+
def journaled_stream_name
|
61
|
+
if model.class.respond_to?(:journaled_stream_name)
|
62
|
+
model.class.journaled_stream_name
|
63
63
|
else
|
64
|
-
Journaled.
|
64
|
+
Journaled.default_stream_name
|
65
65
|
end
|
66
66
|
end
|
67
67
|
end
|
@@ -35,16 +35,22 @@ module Journaled::Event
|
|
35
35
|
event_type
|
36
36
|
end
|
37
37
|
|
38
|
-
def
|
39
|
-
Journaled.
|
38
|
+
def journaled_stream_name
|
39
|
+
Journaled.default_stream_name
|
40
|
+
end
|
41
|
+
|
42
|
+
def tagged?
|
43
|
+
false
|
40
44
|
end
|
41
45
|
|
42
46
|
private
|
43
47
|
|
44
48
|
class_methods do
|
45
|
-
def journal_attributes(*args, enqueue_with: {})
|
49
|
+
def journal_attributes(*args, enqueue_with: {}, tagged: false)
|
46
50
|
journaled_attributes.concat(args)
|
47
51
|
journaled_enqueue_opts.merge!(enqueue_with)
|
52
|
+
|
53
|
+
include Tagged if tagged
|
48
54
|
end
|
49
55
|
|
50
56
|
def journaled_attributes
|
@@ -61,4 +67,20 @@ module Journaled::Event
|
|
61
67
|
|
62
68
|
journal_attributes :id, :event_type, :created_at
|
63
69
|
end
|
70
|
+
|
71
|
+
module Tagged
|
72
|
+
extend ActiveSupport::Concern
|
73
|
+
|
74
|
+
included do
|
75
|
+
journaled_attributes << :tags
|
76
|
+
end
|
77
|
+
|
78
|
+
def tags
|
79
|
+
Journaled::Current.tags
|
80
|
+
end
|
81
|
+
|
82
|
+
def tagged?
|
83
|
+
true
|
84
|
+
end
|
85
|
+
end
|
64
86
|
end
|
@@ -3,7 +3,7 @@ class Journaled::Writer
|
|
3
3
|
journaled_schema_name
|
4
4
|
journaled_partition_key
|
5
5
|
journaled_attributes
|
6
|
-
|
6
|
+
journaled_stream_name
|
7
7
|
journaled_enqueue_opts
|
8
8
|
).freeze
|
9
9
|
|
@@ -25,9 +25,10 @@ class Journaled::Writer
|
|
25
25
|
end
|
26
26
|
|
27
27
|
def journal!
|
28
|
-
|
29
|
-
|
30
|
-
|
28
|
+
validate!
|
29
|
+
Journaled::DeliveryJob
|
30
|
+
.set(journaled_enqueue_opts.reverse_merge(priority: Journaled.job_priority))
|
31
|
+
.perform_later(delivery_perform_args)
|
31
32
|
end
|
32
33
|
|
33
34
|
private
|
@@ -35,24 +36,26 @@ class Journaled::Writer
|
|
35
36
|
attr_reader :journaled_event
|
36
37
|
delegate(*EVENT_METHOD_NAMES, to: :journaled_event)
|
37
38
|
|
38
|
-
def
|
39
|
-
|
39
|
+
def validate!
|
40
|
+
schema_validator('base_event').validate! serialized_event
|
41
|
+
schema_validator('tagged_event').validate! serialized_event if journaled_event.tagged?
|
42
|
+
schema_validator(journaled_schema_name).validate! serialized_event
|
43
|
+
end
|
44
|
+
|
45
|
+
def delivery_perform_args
|
46
|
+
{
|
40
47
|
serialized_event: serialized_event,
|
41
48
|
partition_key: journaled_partition_key,
|
42
|
-
|
43
|
-
|
49
|
+
stream_name: journaled_stream_name,
|
50
|
+
}
|
44
51
|
end
|
45
52
|
|
46
53
|
def serialized_event
|
47
54
|
@serialized_event ||= journaled_attributes.to_json
|
48
55
|
end
|
49
56
|
|
50
|
-
def
|
51
|
-
|
52
|
-
end
|
53
|
-
|
54
|
-
def base_event_json_schema_validator
|
55
|
-
@base_event_json_schema_validator ||= Journaled::JsonSchemaModel::Validator.new('base_event')
|
57
|
+
def schema_validator(schema_name)
|
58
|
+
Journaled::JsonSchemaModel::Validator.new(schema_name)
|
56
59
|
end
|
57
60
|
|
58
61
|
def respond_to_all?(object, method_names)
|
@@ -0,0 +1,18 @@
|
|
1
|
+
module Journaled
|
2
|
+
class Current < ActiveSupport::CurrentAttributes
|
3
|
+
attribute :tags
|
4
|
+
attribute :journaled_actor_proc
|
5
|
+
|
6
|
+
def tags=(value)
|
7
|
+
super(value.freeze)
|
8
|
+
end
|
9
|
+
|
10
|
+
def tags
|
11
|
+
attributes[:tags] ||= {}.freeze
|
12
|
+
end
|
13
|
+
|
14
|
+
def actor
|
15
|
+
journaled_actor_proc&.call
|
16
|
+
end
|
17
|
+
end
|
18
|
+
end
|
data/lib/journaled/engine.rb
CHANGED
data/lib/journaled/version.rb
CHANGED
data/lib/journaled.rb
CHANGED
@@ -1,17 +1,19 @@
|
|
1
1
|
require "aws-sdk-kinesis"
|
2
|
-
require "
|
2
|
+
require "active_job"
|
3
3
|
require "json-schema"
|
4
|
-
require "request_store"
|
5
4
|
|
6
5
|
require "journaled/engine"
|
7
|
-
require
|
6
|
+
require "journaled/current"
|
8
7
|
|
9
8
|
module Journaled
|
10
|
-
|
9
|
+
SUPPORTED_QUEUE_ADAPTERS = %w(delayed delayed_job good_job que).freeze
|
10
|
+
|
11
|
+
mattr_accessor :default_stream_name
|
11
12
|
mattr_accessor(:job_priority) { 20 }
|
12
13
|
mattr_accessor(:http_idle_timeout) { 5 }
|
13
14
|
mattr_accessor(:http_open_timeout) { 2 }
|
14
15
|
mattr_accessor(:http_read_timeout) { 60 }
|
16
|
+
mattr_accessor(:job_base_class_name) { 'ActiveJob::Base' }
|
15
17
|
|
16
18
|
def development_or_test?
|
17
19
|
%w(development test).include?(Rails.env)
|
@@ -33,5 +35,33 @@ module Journaled
|
|
33
35
|
Journaled::ActorUriProvider.instance.actor_uri
|
34
36
|
end
|
35
37
|
|
36
|
-
|
38
|
+
def detect_queue_adapter!
|
39
|
+
adapter = job_base_class_name.constantize.queue_adapter_name
|
40
|
+
unless SUPPORTED_QUEUE_ADAPTERS.include?(adapter)
|
41
|
+
raise <<~MSG
|
42
|
+
Journaled has detected an unsupported ActiveJob queue adapter: `:#{adapter}`
|
43
|
+
|
44
|
+
Journaled jobs must be enqueued transactionally to your primary database.
|
45
|
+
|
46
|
+
Please install the appropriate gems and set `queue_adapter` to one of the following:
|
47
|
+
#{SUPPORTED_QUEUE_ADAPTERS.map { |a| "- `:#{a}`" }.join("\n")}
|
48
|
+
|
49
|
+
Read more at https://github.com/Betterment/journaled
|
50
|
+
MSG
|
51
|
+
end
|
52
|
+
end
|
53
|
+
|
54
|
+
def self.tagged(**tags)
|
55
|
+
existing_tags = Current.tags
|
56
|
+
tag!(tags)
|
57
|
+
yield
|
58
|
+
ensure
|
59
|
+
Current.tags = existing_tags
|
60
|
+
end
|
61
|
+
|
62
|
+
def self.tag!(**tags)
|
63
|
+
Current.tags = Current.tags.merge(tags)
|
64
|
+
end
|
65
|
+
|
66
|
+
module_function :development_or_test?, :enabled?, :schema_providers, :commit_hash, :actor_uri, :detect_queue_adapter!
|
37
67
|
end
|
@@ -1,10 +1,9 @@
|
|
1
1
|
require File.expand_path('boot', __dir__)
|
2
2
|
|
3
3
|
require "active_record/railtie"
|
4
|
+
require "active_job/railtie"
|
4
5
|
require "active_model/railtie"
|
5
6
|
require "action_controller/railtie"
|
6
|
-
require "action_mailer/railtie"
|
7
|
-
require "sprockets/railtie"
|
8
7
|
|
9
8
|
Bundler.require(*Rails.groups)
|
10
9
|
require "journaled"
|