journaled 4.0.0 → 4.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 4cefb08a9d7ab294f9084f150f89cda921982fffdb0ec7b0a5bc702ede1d23b8
-  data.tar.gz: 21a484a10c58a3f551e95af064b5f718b9496ba7840ac6c1ac84142b859942db
+  metadata.gz: dc6d5adec8f9ee32ba34d1ba8dae8cb3da062e39dd1b67d5fc3e219be116d675
+  data.tar.gz: ae77057a26203e90fbb8770e1b41864edcff69058cd80b65011eb414ddf2365a
 SHA512:
-  metadata.gz: 59d07f836a83dde4e4a24a8a5f326ff5cbc7a751d92fb6df66ee22b5b31ac1aa63326675167d45bcd8c572a0e4afeb831774a468eb45097a71c6940a79ce2681
-  data.tar.gz: 807c603b1085819ed003ed8053f81b6fec59ba895c920bc24ac178b0d3188fac554a2ed9b2dcb80ceceb2a26fc5effe6af5d52819f66d041ad7959cfc30957da
+  metadata.gz: 05a99d3d5d530bca784fb44fa290d0ff661430263fbde8fa882f8c98a9dda2eabe76c82df142b6477e68439066cba52d66d875c035a7af6e9d6a2938c8985b82
+  data.tar.gz: 7726aa5152e545cdba1a50ef475f8df0ce37fb97edc2d4fa146c74f73b41c7096434948afeecbe61bc042138cae9c3b94dda475adbf9c74d42892b1d6286e069

data/README.md

@@ -18,6 +18,8 @@ add scoped ordering capability at a future date (and would gladly
 entertain pull requests), but it is presently only designed to provide a
 durable, eventually consistent record that discrete events happened.
 
+**See [upgrades](#upgrades) below if you're upgrading from an older `journaled` version!**
+
 ## Installation
 
 1. If you haven't already,
@@ -83,52 +85,15 @@ app's Gemfile.
 
     The AWS principal whose credentials are in the environment will need to be allowed to assume this role.
 
-### Upgrading from 3.1.0
-
-Versions of Journaled prior to 4.0 relied directly on environment variables for stream names, but now stream names are configured directly.
-When upgrading, you can use the following configuration to maintain the previous behavior:
-
-```ruby
-Journaled.default_stream_name = ENV['JOURNALED_STREAM_NAME']
-```
-
-If you previously specified a `Journaled.default_app_name`, you would have required a more precise environment variable name (substitute `{{upcase_app_name}}`):
-
-```ruby
-Journaled.default_stream_name = ENV["{{upcase_app_name}}_JOURNALED_STREAM_NAME"]
-```
-
-And if you had defined any `journaled_app_name` methods on `Journaled::Event` instances, you can replace them with the following:
-
-```ruby
-def journaled_stream_name
-  ENV['{{upcase_app_name}}_JOURNALED_STREAM_NAME']
-end
-```
-
-When upgrading from 3.1 or below, `Journaled::DeliveryJob` will handle any jobs that remain in the queue by accepting an `app_name` argument. **This behavior will be removed in version 5.0**, so it is recommended to upgrade one major version at a time.
-
-### Upgrading from 2.5.0
-
-Versions of Journaled prior to 3.0 relied direclty on `delayed_job` and a "performable" class called `Journaled::Delivery`.
-In 3.0, this was superceded by an ActiveJob class called `Journaled::DeliveryJob`, but the `Journaled::Delivery` class was not removed until 4.0.
-
-Therefore, when upgrading from 2.5.0 or below, it is recommended to first upgrade to 3.1.0 (to allow any `Journaled::Delivery` jobs to finish working off) before upgrading to 4.0+.
-
-The upgrade to 3.1.0 will require a working ActiveJob config. ActiveJob can be configured globally by setting `ActiveJob::Base.queue_adapter`, or just for Journaled jobs by setting `Journaled::DeliveryJob.queue_adapter`.
-The `:delayed_job` queue adapter will allow you to continue relying on `delayed_job`. You may also consider switching your app(s) to [`delayed`](https://github.com/Betterment/delayed) and using the `:delayed` queue adapter.
-
 ## Usage
 
 ### Configuration
 
 Journaling provides a number of different configuation options that can be set in Ruby using an initializer. Those values are:
 
-#### `Journaled.default_app_name`
+#### `Journaled.default_stream_name `
 
-  This is described in the proceeding paragraph and is used to specify which app name to use, which corresponds to which Journaled Stream to send events too.
-  This is the default value for events that do NOT specify their own `#journaled_app_name`. For events that define their own `#journaled_app_name` method, that will take precedence over this default.
-  Ex: `Journaled.default_app_name = 'my_app'`
+  This is described in the "Installation" section above, and is used to specify which stream name to use.
 
 #### `Journaled.job_priority` (default: 20)
 
@@ -185,8 +150,8 @@ class ApplicationController < ActionController::Base
 end
 ```
 
-Your authenticated entity must respond to `#to_global_id`, which
-ActiveRecords do by default.
+Your authenticated entity must respond to `#to_global_id`, which ActiveRecords do by default.
+This feature relies on `ActiveSupport::CurrentAttributes` under the hood.
 
 Every time any of the specified attributes is modified, or a `User`
 record is created or destroyed, an event will be sent to Kinesis with the following attributes:
@@ -214,18 +179,6 @@ journaling. Note that the less-frequently-used methods `toggle`,
 `increment*`, `decrement*`, and `update_counters` are not intercepted at
 this time.
 
-#### Testing
-
-If you use RSpec (and have required `journaled/rspec` in your
-`spec/rails_helper.rb`), you can regression-protect important journaling
-config with the `journal_changes_to` matcher:
-
-```ruby
-it "journals exactly these things or there will be heck to pay" do
-  expect(User).to journal_changes_to(:email, :first_name, :last_name, as: :identity_change)
-end
-```
-
 ### Custom Journaling
 
 For every custom implementation of journaling in your application, define the JSON schema for the attributes in your event.
@@ -310,6 +263,40 @@ An event like the following will be journaled to kinesis:
 }
 ```
 
+### Tagged Events
+
+Events may be optionally marked as "tagged." This will add a `tags` field, intended for tracing and
+auditing purposes.
+
+```ruby
+class MyEvent
+  include Journaled::Event
+
+  journal_attributes :attr_1, :attr_2, tagged: true
+end
+```
+
+You may then use `Journaled.tag!` and `Journaled.tagged` inside of your
+`ApplicationController` and `ApplicationJob` classes (or anywhere else!) to tag
+all events with request and job metadata:
+
+```ruby
+class ApplicationController < ActionController::Base
+  before_action do
+    Journaled.tag!(request_id: request.request_id, current_user_id: current_user&.id)
+  end
+end
+
+class ApplicationJob < ActiveJob::Base
+  around_perform do |job, perform|
+    Journaled.tagged(job_id: job.id) { perform.call }
+  end
+end
+```
+
+This feature relies on `ActiveSupport::CurrentAttributes` under the hood, so these tags are local to
+the current thread, and will be cleared at the end of each request request/job.
+
 ### Helper methods for custom events
 
 Journaled provides a couple helper methods that may be useful in your
@@ -353,6 +340,145 @@ Returns one of the following in order of preference:
 In order for this to be most useful, you must configure your controller
 as described in [Change Journaling](#change-journaling) above.
 
+### Testing
+
+If you use RSpec, you can test for journaling behaviors with the
+`journal_event(s)_including` and `journal_changes_to` matchers. First, make
+sure to require `journaled/rspec` in your spec setup (e.g.
+`spec/rails_helper.rb`):
+
+```ruby
+require 'journaled/rspec'
+```
+
+#### Checking for specific events
+
+The `journal_event_including` and `journal_events_including` matchers allow you
+to check for one or more matching event being journaled:
+
+```ruby
+expect { my_code }
+  .to journal_event_including(name: 'foo')
+expect { my_code }
+  .to journal_events_including({ name: 'foo', value: 1 }, { name: 'foo', value: 2 })
+```
+
+This will only perform matches on the specified fields (and will not match one
+way or the other against unspecified fields). These matchers will also ignore
+any extraneous events that are not positively matched (as they may be unrelated
+to behavior under test).
+
+When writing tests, pairing every positive assertion with a negative assertion
+is a good practice, and so negative matching is also supported (via both
+`.not_to` and `.to not_`):
+
+```ruby
+expect { my_code }
+  .not_to journal_events_including({ name: 'foo' }, { name: 'bar' })
+expect { my_code }
+  .to raise_error(SomeError)
+  .and not_journal_event_including(name: 'foo') # the `not_` variant can chain off of `.and`
+```
+
+Several chainable modifiers are also available:
+
+```ruby
+expect { my_code }.to journal_event_including(name: 'foo')
+  .with_schema_name('my_event_schema')
+  .with_partition_key(user.id)
+  .with_stream_name('my_stream_name')
+  .with_enqueue_opts(run_at: future_time)
+  .with_priority(999)
+```
+
+All of this can be chained together to test for multiple sets of events with
+multiple sets of options:
+
+```ruby
+expect { subject.journal! }
+  .to journal_events_including({ name: 'event1', value: 300 }, { name: 'event2', value: 200 })
+    .with_priority(10)
+  .and journal_event_including(name: 'event3', value: 100)
+    .with_priority(20)
+  .and not_journal_event_including(name: 'other_event')
+```
+
+#### Checking for `Journaled::Changes` declarations
+
+The `journal_changes_to` matcher checks against the list of attributes specified
+on the model. It does not actually test that an event is emitted within a given
+codepath, and is instead intended to guard against accidental regressions that
+may impact external consumers of these events:
+
+```ruby
+it "journals exactly these things or there will be heck to pay" do
+  expect(User).to journal_changes_to(:email, :first_name, :last_name, as: :identity_change)
+end
+```
+
+### Instrumentation
+
+When an event is enqueued, an `ActiveSupport::Notification` titled
+`journaled.event.enqueue` is emitted. Its payload will include the `:event` and
+its background job `:priority`.
+
+This can be forwarded along to your preferred monitoring solution via a Rails
+initializer:
+
+```ruby
+ActiveSupport::Notifications.subscribe('journaled.event.enqueue') do |*args|
+  payload = ActiveSupport::Notifications::Event.new(*args).payload
+  journaled_event = payload[:event]
+
+  tags = { priority: payload[:priority], event_type: journaled_event.journaled_attributes[:event_type] }
+
+  Statsd.increment('journaled.event.enqueue', tags: tags.map { |k,v| "#{k.to_s[0..64]}:#{v.to_s[0..255]}" })
+end
+```
+
+## Upgrades
+
+Since this gem relies on background jobs (which can remain in the queue across
+code releases), this gem generally aims to support jobs enqueued by the prior
+gem version.
+
+As such, **we always recommend upgrading only one major version at a time.**
+
+### Upgrading from 3.1.0
+
+Versions of Journaled prior to 4.0 relied directly on environment variables for stream names, but now stream names are configured directly.
+When upgrading, you can use the following configuration to maintain the previous behavior:
+
+```ruby
+Journaled.default_stream_name = ENV['JOURNALED_STREAM_NAME']
+```
+
+If you previously specified a `Journaled.default_app_name`, you would have required a more precise environment variable name (substitute `{{upcase_app_name}}`):
+
+```ruby
+Journaled.default_stream_name = ENV["{{upcase_app_name}}_JOURNALED_STREAM_NAME"]
+```
+
+And if you had defined any `journaled_app_name` methods on `Journaled::Event` instances, you can replace them with the following:
+
+```ruby
+def journaled_stream_name
+  ENV['{{upcase_app_name}}_JOURNALED_STREAM_NAME']
+end
+```
+
+When upgrading from 3.1 or below, `Journaled::DeliveryJob` will handle any jobs that remain in the queue by accepting an `app_name` argument. **This behavior will be removed in version 5.0**, so it is recommended to upgrade one major version at a time.
+
+### Upgrading from 2.5.0
+
+Versions of Journaled prior to 3.0 relied direclty on `delayed_job` and a "performable" class called `Journaled::Delivery`.
+In 3.0, this was superceded by an ActiveJob class called `Journaled::DeliveryJob`, but the `Journaled::Delivery` class was not removed until 4.0.
+
+Therefore, when upgrading from 2.5.0 or below, it is recommended to first upgrade to 3.1.0 (to allow any `Journaled::Delivery` jobs to finish working off) before upgrading to 4.0+.
+
+The upgrade to 3.1.0 will require a working ActiveJob config. ActiveJob can be configured globally by setting `ActiveJob::Base.queue_adapter`, or just for Journaled jobs by setting `Journaled::DeliveryJob.queue_adapter`.
+The `:delayed_job` queue adapter will allow you to continue relying on `delayed_job`. You may also consider switching your app(s) to [`delayed`](https://github.com/Betterment/delayed) and using the `:delayed` queue adapter.
+
 ## Future improvements & issue tracking
 Suggestions for enhancements to this engine are currently being tracked via Github Issues. Please feel free to open an
 issue for a desired feature, as well as for any observed bugs.

data/Rakefile

@@ -4,37 +4,23 @@ rescue LoadError
   puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
 end
 
-require 'rdoc/task'
-
-RDoc::Task.new(:rdoc) do |rdoc|
-  rdoc.rdoc_dir = 'rdoc'
-  rdoc.title    = 'Journaled'
-  rdoc.options << '--line-numbers'
-  rdoc.rdoc_files.include('README.rdoc')
-  rdoc.rdoc_files.include('lib/**/*.rb')
-end
-
-APP_RAKEFILE = File.expand_path('spec/dummy/Rakefile', __dir__)
-load 'rails/tasks/engine.rake'
-
 Bundler::GemHelper.install_tasks
 
-if %w(development test).include? Rails.env
-  require 'rspec/core'
-  require 'rspec/core/rake_task'
-  RSpec::Core::RakeTask.new
+require 'rubocop/rake_task'
+RuboCop::RakeTask.new
 
-  require 'rubocop/rake_task'
-  RuboCop::RakeTask.new
+require 'rspec/core'
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec)
 
-  task(:default).clear
+def default_task
   if ENV['APPRAISAL_INITIALIZED'] || ENV['CI']
-    task default: %i(rubocop spec)
+    %i(rubocop spec)
   else
     require 'appraisal'
     Appraisal::Task.new
-    task default: :appraisal
+    %i(appraisal)
   end
-
-  task 'db:test:prepare' => 'db:setup'
 end
+
+task(:default).clear.enhance(default_task)

data/app/controllers/concerns/journaled/actor.rb

@@ -2,11 +2,14 @@ module Journaled::Actor
   extend ActiveSupport::Concern
 
   included do
-    class_attribute :_journaled_actor_method_name, instance_accessor: false, instance_predicate: false
-    before_action do
-      RequestStore.store[:journaled_actor_proc] = self.class._journaled_actor_method_name &&
-        -> { send(self.class._journaled_actor_method_name) }
-    end
+    class_attribute :_journaled_actor_method_name, instance_writer: false
+    before_action :_set_journaled_actor_proc, if: :_journaled_actor_method_name?
+  end
+
+  private
+
+  def _set_journaled_actor_proc
+    Journaled::Current.journaled_actor_proc = -> { send(self.class._journaled_actor_method_name) }
   end
 
   class_methods do

data/app/models/concerns/journaled/changes.rb

@@ -39,8 +39,8 @@ module Journaled::Changes
       end
     end
 
-    def update_columns(attributes, force: false)
-      unless force || self.class.journaled_attribute_names.empty?
+    def update_columns(attributes, opts = { force: false })
+      unless opts[:force] || self.class.journaled_attribute_names.empty?
         conflicting_journaled_attribute_names = self.class.journaled_attribute_names & attributes.keys.map(&:to_sym)
         raise(<<~ERROR) if conflicting_journaled_attribute_names.present?
           #update_columns aborted by Journaled::Changes due to journaled attributes:
@@ -56,7 +56,7 @@ module Journaled::Changes
   end
 
   class_methods do
-    def journal_changes_to(*attribute_names, as:, enqueue_with: {}) # rubocop:disable Naming/UncommunicativeMethodParamName
+    def journal_changes_to(*attribute_names, as:, enqueue_with: {}) # rubocop:disable Naming/MethodParameterName
       if attribute_names.empty? || attribute_names.any? { |n| !n.is_a?(Symbol) }
         raise "one or more symbol attribute_name arguments is required"
       end
@@ -69,8 +69,8 @@ module Journaled::Changes
     end
 
     if Rails::VERSION::MAJOR > 5 || (Rails::VERSION::MAJOR == 5 && Rails::VERSION::MINOR >= 2)
-      def delete(id_or_array, force: false)
-        if force || journaled_attribute_names.empty?
+      def delete(id_or_array, opts = { force: false })
+        if opts[:force] || journaled_attribute_names.empty?
           where(primary_key => id_or_array).delete_all(force: true)
         else
           raise(<<~ERROR)

data/app/models/journaled/actor_uri_provider.rb

@@ -8,8 +8,7 @@ class Journaled::ActorUriProvider
   private
 
   def actor_global_id_uri
-    actor = RequestStore.store[:journaled_actor_proc]&.call
-    actor.to_global_id.to_s if actor
+    Journaled::Current.actor&.to_global_id&.to_s
   end
 
   def fallback_global_id_uri

data/app/models/journaled/change.rb

@@ -2,20 +2,20 @@ class Journaled::Change
   include Journaled::Event
 
   attr_reader :table_name,
-    :record_id,
-    :database_operation,
-    :logical_operation,
-    :changes,
-    :journaled_stream_name,
-    :journaled_enqueue_opts,
-    :actor
+              :record_id,
+              :database_operation,
+              :logical_operation,
+              :changes,
+              :journaled_stream_name,
+              :journaled_enqueue_opts,
+              :actor
 
   journal_attributes :table_name,
-    :record_id,
-    :database_operation,
-    :logical_operation,
-    :changes,
-    :actor
+                     :record_id,
+                     :database_operation,
+                     :logical_operation,
+                     :changes,
+                     :actor
 
   def initialize(table_name:,
                  record_id:,

data/app/models/journaled/change_writer.rb

@@ -1,5 +1,6 @@
 class Journaled::ChangeWriter
   attr_reader :model, :change_definition
+
   delegate :attribute_names, :logical_operation, to: :change_definition
 
   def initialize(model:, change_definition:)
@@ -52,8 +53,8 @@ class Journaled::ChangeWriter
   private
 
   def pluck_changed_values(change_hash, index:)
-    change_hash.each_with_object({}) do |(k, v), result|
-      result[k] = v[index]
+    change_hash.transform_values do |v|
+      v[index]
     end
   end
 

data/app/models/journaled/event.rb

@@ -39,12 +39,18 @@ module Journaled::Event
     Journaled.default_stream_name
   end
 
+  def tagged?
+    false
+  end
+
   private
 
   class_methods do
-    def journal_attributes(*args, enqueue_with: {})
+    def journal_attributes(*args, enqueue_with: {}, tagged: false)
       journaled_attributes.concat(args)
       journaled_enqueue_opts.merge!(enqueue_with)
+
+      include Tagged if tagged
     end
 
     def journaled_attributes
@@ -57,8 +63,24 @@ module Journaled::Event
   end
 
   included do
-    cattr_accessor(:journaled_enqueue_opts, instance_writer: false) { {} }
+    class_attribute :journaled_enqueue_opts, default: {}
 
     journal_attributes :id, :event_type, :created_at
   end
+
+  module Tagged
+    extend ActiveSupport::Concern
+
+    included do
+      journaled_attributes << :tags
+    end
+
+    def tags
+      Journaled::Current.tags
+    end
+
+    def tagged?
+      true
+    end
+  end
 end

data/app/models/journaled/writer.rb

@@ -25,18 +25,28 @@ class Journaled::Writer
   end
 
   def journal!
-    base_event_json_schema_validator.validate! serialized_event
-    json_schema_validator.validate! serialized_event
-    Journaled::DeliveryJob
-      .set(journaled_enqueue_opts.reverse_merge(priority: Journaled.job_priority))
-      .perform_later(delivery_perform_args)
+    validate!
+    ActiveSupport::Notifications.instrument('journaled.event.enqueue', event: journaled_event, priority: job_opts[:priority]) do
+      Journaled::DeliveryJob.set(job_opts).perform_later(**delivery_perform_args)
+    end
   end
 
   private
 
   attr_reader :journaled_event
+
   delegate(*EVENT_METHOD_NAMES, to: :journaled_event)
 
+  def validate!
+    schema_validator('base_event').validate! serialized_event
+    schema_validator('tagged_event').validate! serialized_event if journaled_event.tagged?
+    schema_validator(journaled_schema_name).validate! serialized_event
+  end
+
+  def job_opts
+    journaled_enqueue_opts.reverse_merge(priority: Journaled.job_priority)
+  end
+
   def delivery_perform_args
     {
       serialized_event: serialized_event,
@@ -49,12 +59,8 @@ class Journaled::Writer
     @serialized_event ||= journaled_attributes.to_json
   end
 
-  def json_schema_validator
-    @json_schema_validator ||= Journaled::JsonSchemaModel::Validator.new(journaled_schema_name)
-  end
-
-  def base_event_json_schema_validator
-    @base_event_json_schema_validator ||= Journaled::JsonSchemaModel::Validator.new('base_event')
+  def schema_validator(schema_name)
+    Journaled::JsonSchemaModel::Validator.new(schema_name)
   end
 
   def respond_to_all?(object, method_names)

data/journaled_schemas/tagged_event.json

@@ -0,0 +1,14 @@
+{
+  "type": "object",
+  "title": "tagged_event",
+  "additionalProperties": true,
+  "required": [
+    "tags"
+  ],
+  "properties": {
+    "tags": {
+      "type": "object",
+      "additionalProperties": true
+    }
+  }
+}

data/lib/journaled/current.rb

@@ -0,0 +1,18 @@
+module Journaled
+  class Current < ActiveSupport::CurrentAttributes
+    attribute :tags
+    attribute :journaled_actor_proc
+
+    def tags=(value)
+      super(value.freeze)
+    end
+
+    def tags
+      attributes[:tags] ||= {}.freeze
+    end
+
+    def actor
+      journaled_actor_proc&.call
+    end
+  end
+end

data/lib/journaled/relation_change_protection.rb

@@ -1,14 +1,15 @@
 module Journaled::RelationChangeProtection
-  def update_all(updates, force: false) # rubocop:disable Metrics/AbcSize, Metrics/PerceivedComplexity
-    unless force || !@klass.respond_to?(:journaled_attribute_names) || @klass.journaled_attribute_names.empty?
-      conflicting_journaled_attribute_names = if updates.is_a?(Hash)
-                                                @klass.journaled_attribute_names & updates.keys.map(&:to_sym)
-                                              elsif updates.is_a?(String)
-                                                @klass.journaled_attribute_names.select do |a|
-                                                  updates.match?(/\b(?<!')#{a}(?!')\b/)
-                                                end
-                                              else
-                                                raise "unsupported type '#{updates.class}' for 'updates'"
+  def update_all(updates, opts = { force: false }) # rubocop:disable Metrics/AbcSize
+    unless opts[:force] || !@klass.respond_to?(:journaled_attribute_names) || @klass.journaled_attribute_names.empty?
+      conflicting_journaled_attribute_names = case updates
+                                                when Hash
+                                                  @klass.journaled_attribute_names & updates.keys.map(&:to_sym)
+                                                when String
+                                                  @klass.journaled_attribute_names.select do |a|
+                                                    updates.match?(/\b(?<!')#{a}(?!')\b/)
+                                                  end
+                                                else
+                                                  raise "unsupported type '#{updates.class}' for 'updates'"
                                               end
       raise(<<~ERROR) if conflicting_journaled_attribute_names.present?
         .update_all aborted by Journaled::Changes due to journaled attributes: