one_more_time 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 247918c0b167bd95918b841bf7b35cf1f5d15350dbc834ca7ad3cb91fda99ca6
+  data.tar.gz: 809be36d13054cf346431fd204a2e86702f0cbcee8106f14b958730796c4052a
+SHA512:
+  metadata.gz: 0a9796d7a4c64205aa0e0981043ce1aef31e99e5a0add3f1b5f74c2dd3cd51b36e2182bd388a52d0c515f6583061480ba2ddf11820aa3f02f26974323b531de4
+  data.tar.gz: ad8f0a600eb4e4234867e448c6f80957c46492f8e7e1709dd31fc4c887e4ae29e89fa00ce28e350db7148bd946a180ed91f6948aa208a824eafd936ee3c5a61f

data/LICENSE

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2020 Andrew Cross
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,151 @@
+# One More Time
+
+A simple gem to help make your API idempotent.
+Because it should always be safe to call your API... _one more time_.
+
+* [Overview](#overview)
+* [Installation](#installation)
+* [Usage](#usage)
+* [Contributing](#contributing)
+* [License](#license)
+
+## Overview
+
+As a library for idempotency, One More Time serves two main purposes:
+- Saving and returning the previous response to a repeated request
+- Ensuring side effects that should only happen once (e.g. paying money) in fact only happen once
+
+To accomplish these, this gem provides a single ActiveRecord model called `IdempotentRequest` with some additional methods added. You call those methods at specific points in the lifecycle of your request, and idempotency is guaranteed for you.
+
+Generic solutions (i.e. drop-in Rack plugins) for idempotency do exist, however they can't handle requests that fail during processing very well because they don't know _when_ a request becomes unsafe to retry. If there's nothing you're afraid to retry when your request is left in an undefined state, a generic solution will suffice.
+
+🚨 **Note**: One More Time is intended as middleware and does not know about the network procotol being used; it only provides dumb storage for request and response data. Thus you will need to tell the gem how to store incoming requests, as well as how to convert stored responses into actual responses at the network level. The goal would be to write this glue code once for a given Ruby application framework (e.g. Rails controllers, Gruf, Sinatra) and then re-use it thereafter.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+```ruby
+gem 'one_more_time'
+```
+
+And then execute:
+
+    $ bundle
+
+Or install it yourself as:
+
+    $ gem install one_more_time
+
+Then create a table in your database named `idempotent_requests` with, at minimum, the following columns (a generator for this is in the TODO list):
+
+| Name            | ActiveRecord Type |
+|-----------------|-------------
+| idempotency_key | string / text (with UNIQUE constraint)
+| locked_at       | datetime
+| request_body    | (any)
+| request_path    | (any)
+| response_code   | (any)
+| response_body   | (any)
+
+🚨 **Note**: The idempotency_key column MUST have a unique constraint for the gem to work.
+
+## Usage
+
+In order to guarantee idempotency, One More Time assumes a request can be divided into three phases:
+1. The incoming request is stored in the local database and is validated. Any read-only queries or service calls are made.
+2. State is changed in an external service (e.g. an order is submitted, a payment is made)
+3. The results of step 2 are stored in the local database.
+
+☝️ Note: For more in-depth reading, this is the same pattern enforced by AirBnB's idempotency middleware (https://medium.com/airbnb-engineering/avoiding-double-payments-in-a-distributed-payments-system-2981f6b070bb).
+
+Let's see how this looks in code, for example in a rails controller:
+```ruby
+# We begin by calling OneMoreTime.start_request!, which either creates or finds a record
+# using the given idempotency_key. The record (for now an ActiveRecord model but don't rely
+# on that) has methods to help orchestrate your request.
+# When first created, the record is in a "locked" state so no other server process will be
+# able to work on the same request. If we try to access a locked record here, a
+# RequestInProgressError will be raised.
+idempotent_request = OneMoreTime.start_request!(
+  # This value is supplied by the client, who decides the scope of idempotency
+  idempotency_key: request.headers["Idempotency-Key"],
+  # If supplied, these values will be used to verify that the incoming request data matches
+  # the stored data (when a record with the given idempotency_key already exists).
+  # If there is a mismatch, a RequestMismatchError will be raised.
+  request_path: "#{request.method} #{request.path}",
+  request_body: request.raw_post,
+  )
+
+# Set a callback to specify how to convert a successful result into response data stored
+# on the record
+idempotent_request.success_attributes do |result|
+  {
+    response_code: 200,
+    response_body: result.to_json,
+  }
+end
+
+# Similarly, convert an exception that has been raised into a stored response
+idempotent_request.failure_attributes do |exception|
+  {
+    response_code: 500,
+    response_body: { error: exception.message }.to_json,
+  }
+end
+```
+
+Everything up to this point will likely occur only once, as framework-level code in your app. Individual endpoint implementations should be provided with the `idempotent_request` object and only need to use the following pattern:
+
+```ruby
+# Wrap your request in a block sent to the execute method. If the idempotent_request already
+# has a stored response, the block will be skipped entirely.
+idempotent_request.execute do
+  # Validate the request as needed.
+  # Because we are inside the execute block, raising an error will automatically unlock the
+  # request and NOT store a response, so this block can be retried by the next attempt.
+  raise ActionController::BadRequest unless params[:widget_name].present?
+
+  # Make an external, non-idempotent service call
+  begin
+    widget = ExternalService.purchase_widget(params[:widget_name])
+  rescue ExternalService::ConnectionLostError => exception
+  #   We sent data to the external service but don't know whether it was fully processed.
+  #   If a widget is something we can't afford to accidentally create twice, we need to
+  #   give up and store an error response on this request so it can't be retried.
+  #   This will invoke the failure_attributes callback and raise out of the execute block.
+    idempotent_request.failure!(exception: exception)
+  end
+
+  # Call the success! method with a block. This block is automatically run in a transaction
+  # and should contain any code needed to persist the results of your external service call.
+  # Any failure within this block will internally call failure! on the request - we've
+  # failed to store a record of the widget, but we DID purchase it, so we can't allow a retry.
+  idempotent_request.success do
+    # The return value of this block is what gets passed to the success_attributes callback
+    Widget.create!(widget_id: widget.id)
+  end
+end
+
+# Render the stored response, which we are guaranteed to have if we make it here
+render json: idempotent_request.response_body, status: idempotent_request.response_code
+
+# And that's it!
+```
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at https://github.com/Freshly/one_more_time.
+
+Todos/Enhancements:
+- Add a rails generator to create a migration for the idempotent_requests table.
+- Add example integations for different app frameworks.
+  - Rails
+  - Gruf
+  - Sinatra
+- Possibly add another yielding method that calls `failure!` by default for any exception. When using this method, instead of rescuing errors that are _not_ retryable and calling `failure!` yourself, you would explicitly rescue exceptions that _are_ retryable.
+- Look into supporting recovery points/multiple transactions per request.
+
+## License
+
+The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -0,0 +1,17 @@
+begin
+  require 'bundler/setup'
+rescue LoadError
+  puts 'You must `gem install bundler` and `bundle install` to run rake tasks'
+end
+
+require 'rdoc/task'
+
+RDoc::Task.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'OneMoreTime'
+  rdoc.options << '--line-numbers'
+  rdoc.rdoc_files.include('README.md')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+require 'bundler/gem_tasks'

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "one_more_time"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start(__FILE__)

data/lib/one_more_time.rb

@@ -0,0 +1,50 @@
+# frozen_string_literal: true
+
+require "active_record"
+require "one_more_time/idempotent_request"
+
+module OneMoreTime
+  class Error < StandardError; end
+  class RequestInProgressError < Error; end
+  class RequestMismatchError < Error; end
+
+  class << self
+    def start_request!(idempotency_key:, request_path: nil, request_body: nil)
+      IdempotentRequest.create!(
+        idempotency_key: idempotency_key,
+        locked_at: Time.current,
+        request_path: request_path,
+        request_body: request_body,
+      )
+    rescue ActiveRecord::RecordNotUnique
+      # Our UNIQUE constraint was violated, so a request with the given idempotency
+      # key already exists. Use the highest transaction isolation level to atomically
+      # load that request record and mark it as "locked".
+      # Similar to Rails create_or_find_by, the race condition here is if another
+      # client deleted the request record exactly at this point. For this specific
+      # model there is basically no risk of that happening.
+      serializable_transaction do
+        existing_request = IdempotentRequest.find_by(idempotency_key: idempotency_key, locked_at: nil)
+        validate_incoming_request!(existing_request, request_path, request_body)
+        existing_request.update!(locked_at: Time.current) unless existing_request.finished?
+        existing_request
+      end
+    end
+
+    private
+
+    def validate_incoming_request!(existing_request, request_path, request_body)
+      raise RequestInProgressError if existing_request.blank?
+      raise RequestMismatchError if request_path.present? && request_path != existing_request.request_path
+      raise RequestMismatchError if request_body.present? && request_body != existing_request.request_body
+    end
+
+    def serializable_transaction
+      ActiveRecord::Base.transaction(isolation: :serializable) do
+        yield
+      end
+    rescue ActiveRecord::SerializationFailure
+      raise RequestInProgressError
+    end
+  end
+end

data/lib/one_more_time/idempotent_request.rb

@@ -0,0 +1,58 @@
+# frozen_string_literal: true
+
+module OneMoreTime
+  class IdempotentRequest < ActiveRecord::Base
+    def success_attributes(&block)
+      @success_attributes_block = block
+    end
+
+    def failure_attributes(&block)
+      @failure_attributes_block = block
+    end
+
+    def execute
+      # No-op the request if we already have a saved response.
+      return if finished?
+
+      begin
+        yield if block_given?
+      rescue PermanentError
+        # Something has called .failure!, so we assume there is now a saved response
+        # and can just no-op
+      rescue StandardError
+        update_and_unlock
+        raise
+      end
+      # TODO: raise unless finished?
+    end
+
+    def success
+      ActiveRecord::Base.transaction do
+        result = yield if block_given?
+        attrs = @success_attributes_block&.call(result) || {}
+        update_and_unlock(attrs)
+      end
+    rescue StandardError => exception
+      failure!(exception: exception)
+    end
+
+    def failure!(exception: nil, response_code: nil, response_body: nil)
+      attrs = (exception.present? && @failure_attributes_block&.call(exception)) || {}
+      attrs.merge!({response_code: response_code, response_body: response_body}.compact)
+      update_and_unlock(attrs)
+      raise PermanentError
+    end
+
+    def finished?
+      response_code.present? || response_body.present?
+    end
+
+    private
+
+    class PermanentError < StandardError; end
+
+    def update_and_unlock(attrs={})
+      update!({ locked_at: nil }.merge(attrs.slice(:response_code, :response_body)))
+    end
+  end
+end

data/lib/one_more_time/version.rb

@@ -0,0 +1,3 @@
+module OneMoreTime
+  VERSION = '0.1.0'
+end

data/lib/tasks/one_more_time_tasks.rake

@@ -0,0 +1,4 @@
+# desc "Explaining what the task does"
+# task :one_more_time do
+#   # Task goes here
+# end

metadata

@@ -0,0 +1,113 @@
+--- !ruby/object:Gem::Specification
+name: one_more_time
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: ruby
+authors:
+- Andrew Cross
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2020-06-26 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.0.2
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 6.0.2.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 6.0.2
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: 6.0.2.1
+- !ruby/object:Gem::Dependency
+  name: rspec
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: timecop
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+description: Use your database to store previous responses and guarantee safe retries.
+email:
+- andrew.cross@freshly.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- LICENSE
+- README.md
+- Rakefile
+- bin/console
+- lib/one_more_time.rb
+- lib/one_more_time/idempotent_request.rb
+- lib/one_more_time/version.rb
+- lib/tasks/one_more_time_tasks.rake
+homepage: https://github.com/Freshly/one_more_time
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.0.3
+signing_key: 
+specification_version: 4
+summary: A simple gem to help make your API idempotent
+test_files: []