outboxer 0.1.0 → 0.1.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a573b50b1f20cc0966a9c6baef36216240ab20adee17adcf03da8a193436bf78
-  data.tar.gz: 86573195530a94aac112fd0c52169a7564e97f0e677ba03c26d37b87e8eb7c02
+  metadata.gz: fe0005e2eb4b31e7d9a16694ad68b7f2b3ed349e2153a435547963a714eb6a22
+  data.tar.gz: 703db7ef7cb96cf6ac8cf95aae7bd27d52880c0abf88fa28e9beebf6a0fa328c
 SHA512:
-  metadata.gz: edd4bc1e08e57f742985e8382fa1202ee6df3812282f9b0d12054cce63406fb9a7101fa6a53d93f85205faf31ae324b35c3254c9ab586865959cdd1a2ccfe951
-  data.tar.gz: 5d5196f1bdcadb519720c4d80b01852d9463f1d5844a90bd8447d392675b17357adae4deb4088b8210ea65f0ab4555707b0affee69aac20b0832b5ff0505b77a
+  metadata.gz: afa1718ef1bcf5d3c375080c53ec347925999023832eb0539bb0ec542f20140e00f8f4795fdf4371ad833a5cd2ced0f8715d692635e66860e4219d38c5b11c99
+  data.tar.gz: 99ea377b056288cc7e5958665d787e765a511f697d26f4a0b4b091b955b776a4ce05fdd5e833e1ba3f38914a25376867f99f24fb18e58dfb6f98cbf1fc4efe98

data/README.md

@@ -1,10 +1,23 @@
 # Outboxer
 
-Creating a message in an SQL database and publishing it to redis via a sidekiq worker are two operations that cannot be combined into a single atomic operation. If either database fails, inconsistencies can occur.
+## Background
 
-In Outboxer, when a new message is created in your application's SQL database, Outboxer automatically creates a message in an outbox table with a status of 'unpublished', within the same local transaction. This ensures that either both operations succeed or both fail, preserving atomicity.
+Typically in event driven Ruby on Rails applications:
 
-## Installation
+1. a domain event model is created in an SQL database table
+2. a sidekiq worker is queued to handle the domain event asynchronously
+
+## Problem
+
+As these two operations span multiple database types (SQL and redis), they can not be combined into a single atomic operation using a transaction. If either step fails, inconsistencies can occur.
+
+## Solution
+
+Outboxer is a simple Ruby on Rails implementation of the [transactional outbox pattern](https://microservices.io/patterns/data/transactional-outbox.html): a well established solution to this problem. It ensures both operations succeed _eventually_, or both fail.
+
+### Getting started
+
+### Installation
 
 1. Add the Outboxer gem to your application's Gemfile:
 
@@ -24,18 +37,9 @@ bundle install
 bin/rails generate outboxer:install
 ```
 
-## Usage
-
-### 1. Migrate your database
-
-```bash
-bin/rake db:migrate
-```
-
-
-### 2. Include Outboxer into existing model
+### Usage
 
-First, include `Outboxer::Outboxable` into your existing `Message` model:
+#### 1. Include `Outboxer::Outboxable` into your existing Message model
 
 ```ruby
 class Message < ApplicationRecord
@@ -43,26 +47,50 @@ class Message < ApplicationRecord
 end
 ```
 
-### 3. Update the publish block
+#### 2. Update the generated bin/publish block e.g.
 
-By default, the `bin/publisher` script does not do anything with the message in its block.
+```ruby
+Outboxer::Publisher.publish do |args|
+  args.logger.info("[#{message.id}] publishing")
 
-To customize this behavior, you should update the block in the `bin/publisher` file:
+  Worker.perform_async({ message_id: args.message.id })
 
-```ruby
-Outboxer::Publisher.publish do |message:, logger:|
-  Worker.perform_async({ message_id: message.id })
+  args.logger.info("[#{message.id}] published")
 end
 ```
 
-### 3. Run the Publisher
+#### 3. Migrate the database
 
-To start publishing messages, run the `bin/publisher` script:
+```bash
+bin/rake db:migrate
+```
+
+#### 4. Run the publisher
 
 ```bash
 bin/publisher
 ```
 
+## Implementation
+
+1. when an `ActiveRecord` model that includes `Outbox::Outboxable` is created, an `unpublished` `Outboxer::Message` is automatically created in the same transaction, with `Outboxer::Message#message` polymorphically assigned to the original model
+
+2. When the publisher finds a new `unpublished` `Outboxer::Message`, it yields to a user-supplied block and then:
+    - removes it if the task completes successfully
+    - marks it as failed and records the error if there's a problem
+
+To see all the parts working together in a single place, check out the [publisher_spec.rb](https://github.com/fast-programmer/outboxer/blob/master/spec/outboxer/publisher_spec.rb)
+
+
+## Motivation
+
+Outboxer was created with 4 key benefits in mind:
+
+1. speed of integration into existing Ruby on Rails applications (< 1 hour)
+2. comprehensive documentation that is easy to understand
+3. high reliability in production environments (100% code coverage)
+4. forever free to use in commerical applications (MIT licence)
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/fast-programmer/outboxer. This project is intended to be a safe, welcoming space for collaboration, and contributors are expected to adhere to the [code of conduct](https://github.com/fast-programmer/outboxer/blob/main/CODE_OF_CONDUCT.md).

data/generators/outboxer/templates/bin/publisher.rb

@@ -3,5 +3,12 @@
 require_relative "../config/environment"
 
 Outboxer::Publisher.publish do |args|
-  # Worker.perform_async({ message_id: args.message.id })
+  logger = args.logger
+  message = args.message
+
+  logger.info("[#{message.id}] publishing")
+
+  HardJob.perform_async({ "message_id" => message.id })
+
+  logger.info("[#{message.id}] published")
 end

data/lib/outboxer/models/message.rb

@@ -14,7 +14,8 @@ module Outboxer
       belongs_to :message, polymorphic: true
 
       has_many :exceptions, -> { order(created_at: :asc) },
-               class_name: "::Outboxer::Models::Exception"
+               class_name: "::Outboxer::Models::Exception",
+               dependent: :destroy
     end
   end
 end

data/lib/outboxer/publisher.rb

@@ -9,6 +9,17 @@ module Outboxer
 
     Args = Struct.new(:message, :logger)
 
+    # This method initiates the publishing process. It dequeues the messages one by one
+    # and yields to a block that should contain the publishing logic.
+    #
+    # @param [Integer] poll
+    #   Sleep time in seconds between polling the queue when it's empty.
+    #
+    # @param [Proc] backoff
+    #   A Proc that takes the current backoff time and returns the new backoff time.
+    #
+    # @yieldparam [Args] args
+    #   An Args object containing the message to publish and a logger.
     def publish(poll: 1, backoff: ->(current_backoff) { [current_backoff * 2, 5 * 60].min })
       @publishing = true
 
@@ -100,6 +111,12 @@ module Outboxer
       end
     end
 
+    # Stops the publishing process.
+    #
+    # @note This method will stop the current message publishing process
+    #       It is a safe way to interrupt the publishing process at any point.
+    #
+    # @return [void]
     def stop
       @publishing = false
     end
@@ -109,5 +126,7 @@ module Outboxer
 
       stop
     end
+
+    private_class_method :retry_on_error, :dequeue, :published, :failed
   end
 end

data/lib/outboxer/version.rb

@@ -1,3 +1,3 @@
 module Outboxer
-  VERSION = "0.1.0".freeze
+  VERSION = "0.1.1".freeze
 end

data/lib/outboxer.rb

@@ -10,4 +10,6 @@ require_relative "outboxer/publisher"
 
 module Outboxer
   Outboxable = Models::Outboxable
+  Message = Models::Message
+  Exception = Models::Exception
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: outboxer
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.1.1
 platform: ruby
 authors:
 - Adam Mikulasev