dj_one 0.1.2

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA1:
+  metadata.gz: a2c77556695fbf526e0b60a7bf999409a03c7ac1
+  data.tar.gz: ff1983338f47733296e7393264e8a85baa94b3e3
+SHA512:
+  metadata.gz: cdcaec857e72e5fbc96e51474eb16ba4e7f7794046e08b62a2cf3f89bd688d1392286d55ffa6b4610a495932e7e5fe53dbec00cea78c02a6ec282cdc4462ef0d
+  data.tar.gz: '0509c74f032e8fe3bf220a6cb86a82fe65f3232f985caf8a7baa41e32d21c47a0a993a6834556b74cf590c6c21fa6528610cab9cfd581636f4a36fdcd5dec684'

data/.gitignore

@@ -0,0 +1,9 @@
+/.bundle/
+/.yardoc
+/Gemfile.lock
+/_yardoc/
+/coverage/
+/doc/
+/pkg/
+/spec/reports/
+/tmp/

data/.travis.yml

@@ -0,0 +1,5 @@
+sudo: false
+language: ruby
+rvm:
+  - 2.3.0
+before_install: gem install bundler -v 1.12.5

data/CODE_OF_CONDUCT.md

@@ -0,0 +1,49 @@
+# Contributor Code of Conduct
+
+As contributors and maintainers of this project, and in the interest of
+fostering an open and welcoming community, we pledge to respect all people who
+contribute through reporting issues, posting feature requests, updating
+documentation, submitting pull requests or patches, and other activities.
+
+We are committed to making participation in this project a harassment-free
+experience for everyone, regardless of level of experience, gender, gender
+identity and expression, sexual orientation, disability, personal appearance,
+body size, race, ethnicity, age, religion, or nationality.
+
+Examples of unacceptable behavior by participants include:
+
+* The use of sexualized language or imagery
+* Personal attacks
+* Trolling or insulting/derogatory comments
+* Public or private harassment
+* Publishing other's private information, such as physical or electronic
+  addresses, without explicit permission
+* Other unethical or unprofessional conduct
+
+Project maintainers have the right and responsibility to remove, edit, or
+reject comments, commits, code, wiki edits, issues, and other contributions
+that are not aligned to this Code of Conduct, or to ban temporarily or
+permanently any contributor for other behaviors that they deem inappropriate,
+threatening, offensive, or harmful.
+
+By adopting this Code of Conduct, project maintainers commit themselves to
+fairly and consistently applying these principles to every aspect of managing
+this project. Project maintainers who do not follow or enforce the Code of
+Conduct may be permanently removed from the project team.
+
+This code of conduct applies both within project spaces and in public spaces
+when an individual is representing the project or its community.
+
+Instances of abusive, harassing, or otherwise unacceptable behavior may be
+reported by contacting a project maintainer at piotrj@gmail.com. All
+complaints will be reviewed and investigated and will result in a response that
+is deemed necessary and appropriate to the circumstances. Maintainers are
+obligated to maintain confidentiality with regard to the reporter of an
+incident.
+
+This Code of Conduct is adapted from the [Contributor Covenant][homepage],
+version 1.3.0, available at
+[http://contributor-covenant.org/version/1/3/0/][version]
+
+[homepage]: http://contributor-covenant.org
+[version]: http://contributor-covenant.org/version/1/3/0/

data/Gemfile

@@ -0,0 +1,7 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in dj_one.gemspec
+gemspec
+
+gem "sqlite3"
+gem 'pry'

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2016 Piotr Jakubowski
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,90 @@
+# DjOne
+
+[![Build Status](https://travis-ci.org/piotrj/dj_one.svg?branch=master)](https://travis-ci.org/piotrj/dj_one)
+
+## What is DjOne
+
+DjOne ensures that you don't have duplicate jobs scheduled or running. When you schedule a job as a reaction to some event you may end up with multiple jobs that will be basically doing the same thing.
+Moreover, if you have more than 1 worker you may end up with more than 1 exactly same job being processed at a time. This means you may need to ensure that those jobs don't corrupt each others computations.
+With DjOne you can ensure that only 1 job is scheduled and processed so you don't need to do synchronization and your workers don't waste time on doing the same thing multiple times.
+
+### Example
+
+Let's say that your application is integrated with Google Calendar. And you listen on push notifications from Google that will let you know that something has changed.
+When you get the notification you schedule a Delayed::Job that will fetch all new calendar events for the calendar you got notification for.
+
+Now let's consider following scenario:
+1. You receive a push notification for calendar calendarA.
+2. You schedule a job to fetch everything new for this calendar. (Let's call it JobA1)
+3. The job does not start yet cause workers are processing other jobs.
+4. Something else changed in calendar calendarA. You get another notification.
+5. At this point you would normally schedule another job. But do you really need to? You already have one job for that calendar that is still pending. So we are using DjOne's functionality and we don't schedule another job.
+6. Worker picks up the job for calendarA
+7. New change in calendarA. You get another push for that calendar.
+8. So now should you schedule a job? In step #5 we didn't want to schedule new one. So probably we don't want to schedule another one now as well, right? All in all we have a job running for that calendar.
+Well, so here is the thing. You don't know at which point of operation JobA1 is. It may have just started, but it may have already finished fetching the events. And if you don't schedule new one you won't have the newest data.
+9. Ok so we schedule new job for that calendar, let's call it JobA2.
+10. Worker is about to pick up JobA2. But, we already have a job for that calendar running. If we start another one maybe they would start interfering on each other's data. So maybe let's not start JobA2 yet. Let's let JobA1 finish.
+11. So using DjOne's functionality we are moving JobA2 start time to be a little bit in the future.
+12. JobA1 gets finished
+13. JobA2 gets picked up by worker again. This time it can run cause there is no other conflicting job.
+14. JobA2 finished. We get the freshest data for calendarA.
+
+## Installation
+
+1. Add `dj_one` to your Gemfile.
+
+```ruby
+  gem 'dj_one'
+```
+
+2. Run `bundle install`.
+3. Generate a migration that will add a column and unique index that will help us ensure uniqueness of jobs.
+4. Run `rake db:migrate`
+5. Add DjOne as a plugin in your DelayedJob initializer
+```ruby
+# config/initializers/delayed_job_config.rb (or other file where you initialize DelayedJob)
+
+Delayed::Worker.plugins << DjOne::Plugin
+
+```
+
+6. You are all set.
+
+
+## Usage
+
+Now that we have the gem installed let's add this functionality to our job.
+
+```ruby
+FetchCalendarEventsJob = Struct.new(:calendar_name)
+  # This is the id that we will try to set when enqueueing the job.
+  def enqueue_id
+    "fetch_calendar_events_enqueued_#{calendar_name}"
+  end
+
+  # This is the id that we will set when we start performing given job
+  def perform_id
+    "fetch_calendar_events_processing_#{calendar_name}"
+  end
+
+  # Used when the worker wants to start the job, but there is already one being performed at the moment.
+  # At such case we need to reschedule the job to some point in the future.
+  # This method defines how far in the future.
+  # If not defined it will use the default 30 seconds.
+  def duplicate_delay
+    # Here if we try to run the job but we are already currently fetching events for given
+    # calendar then we want to try running this job again in 5 minutes.
+    5.minutes
+  end
+end
+```
+
+If you add those 2 methos then you will be sure that only 1 job for given calendar_name would be enqueued and processed at any given time.
+If you don't care how many same jobs are enqueued then you don't need to specify `enqueue_id` method.
+If you don't care how many same jobs are processed at a time then you don't need to specify `perform_id` method.
+If you don't specify any methods it will just work without any restrictions.
+
+## License
+The gem is available as open source under the terms of the MIT License.
+

data/Rakefile

@@ -0,0 +1,17 @@
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList['test/**/*_test.rb']
+end
+
+namespace :test do
+  namespace :db do
+    task :prepare do
+    end
+  end
+end
+
+task :default => :test

data/bin/console

@@ -0,0 +1,14 @@
+#!/usr/bin/env ruby
+
+require "bundler/setup"
+require "dj_one"
+
+# You can add fixtures and/or initialization code here to make experimenting
+# with your gem easier. You can also use a different console, if you like.
+
+# (If you use this, don't forget to add pry to your Gemfile!)
+# require "pry"
+# Pry.start
+
+require "irb"
+IRB.start

data/bin/setup

@@ -0,0 +1,8 @@
+#!/usr/bin/env bash
+set -euo pipefail
+IFS=$'\n\t'
+set -vx
+
+bundle install
+
+# Do any other automated setup that you need to do here

data/dj_one.gemspec

@@ -0,0 +1,27 @@
+# coding: utf-8
+lib = File.expand_path('../lib', __FILE__)
+$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
+require 'dj_one/version'
+
+Gem::Specification.new do |spec|
+  spec.name          = "dj_one"
+  spec.version       = DjOne::VERSION
+  spec.authors       = ["Piotr Jakubowski"]
+  spec.email         = ["piotrj@gmail.com"]
+
+  spec.summary       = "DjOne ensures uniqueness of your DelayedJobs"
+  spec.description   = "With DjOne you can get rid of duplicate schedules of particular DelayedJob so that your workers don't have to do the same work twice and you don't need to ensure synchronization between those jobs"
+  spec.homepage      = "https://github.com/piotrj/dj_one"
+  spec.license       = "MIT"
+
+  spec.files         = `git ls-files -z`.split("\x0").reject { |f| f.match(%r{^(test|spec|features)/}) }
+  spec.bindir        = "exe"
+
+  spec.require_paths = ["lib"]
+
+  spec.add_development_dependency "bundler", "~> 1.12"
+  spec.add_development_dependency "rake", "~> 10.0"
+  spec.add_development_dependency "minitest", "~> 5.0"
+  spec.add_dependency   'delayed_job_active_record',  "~> 4.1"
+  spec.add_dependency   'railties', '>= 4.0'
+end

data/lib/dj_one/generator.rb

@@ -0,0 +1,13 @@
+require "rails/generators"
+require "rails/generators/active_record"
+
+module DelayedJob
+  class AddDjOne < Rails::Generators::Base
+    include ActiveRecord::Generators::Migration
+    source_root File.expand_path("../templates", __FILE__)
+
+    def add_migration
+      migration_template "install_dj_one_migration.rb", "db/migrate/install_dj_one.rb"
+    end
+  end
+end

data/lib/dj_one/templates/install_dj_one_migration.rb

@@ -0,0 +1,18 @@
+superclass = if ActiveRecord.version >= Gem::Version.new(5)
+               ActiveRecord::Migration[5.0]
+             else
+               ActiveRecord::Migration
+             end
+
+
+class InstallDjOne < superclass
+  def self.up
+    add_column :delayed_jobs, :unique_id, :string
+    add_index :delayed_jobs, [:unique_id], name: :unique_delayed_jobs, unique: true
+  end
+
+  def self.down
+    remove_column :delayed_jobs, :unique_id
+    remove_index :delayed_jobs, name: :unique_delayed_jobs
+  end
+end

data/lib/dj_one/version.rb

@@ -0,0 +1,3 @@
+module DjOne
+  VERSION = "0.1.2"
+end

data/lib/dj_one.rb

@@ -0,0 +1,59 @@
+require "dj_one/version"
+
+require "active_record"
+require "delayed_job"
+require "delayed_job_active_record"
+
+require "dj_one/generator"
+
+module DjOne
+  DEFAULT_DUPLICATE_DELAY = 30.seconds
+
+  class Plugin < Delayed::Plugin
+    callbacks do |lifecycle|
+      Handler.new(lifecycle)
+    end
+
+    class Handler
+      def initialize(lifecycle)
+        lifecycle.around(:enqueue, &method(:enqueue))
+        lifecycle.around(:perform, &method(:perform))
+        lifecycle.around(:failure, &method(:failure))
+      end
+
+      def enqueue(job, &proceed)
+        job.unique_id = get_attribute(job, :enqueue_id)
+        proceed.call
+      rescue ActiveRecord::RecordNotUnique
+      end
+
+      def perform(worker, job, &proceed)
+        job.unique_id = get_attribute(job, :perform_id)
+        job.save! if job.changed?
+
+        proceed.call
+      rescue ActiveRecord::RecordNotUnique
+        job.unique_id = job.unique_id_was
+        job.run_at = calculate_run_at(job)
+        job.save! if job.changed?
+      end
+
+      def failure(worker, job, &proceed)
+        job.unique_id = nil
+        proceed.call
+      end
+
+      def get_attribute(job, method_name)
+        object = job.payload_object
+        object.respond_to?(method_name) ? object.public_send(method_name) : nil
+      rescue Delayed::DeserializationError
+        nil
+      end
+
+      def calculate_run_at(job)
+        delay = get_attribute(job, :duplicate_delay) || DEFAULT_DUPLICATE_DELAY
+        Time.now + delay
+      end
+    end
+  end
+end

metadata

@@ -0,0 +1,130 @@
+--- !ruby/object:Gem::Specification
+name: dj_one
+version: !ruby/object:Gem::Version
+  version: 0.1.2
+platform: ruby
+authors:
+- Piotr Jakubowski
+autorequire: 
+bindir: exe
+cert_chain: []
+date: 2017-06-02 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: bundler
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.12'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.12'
+- !ruby/object:Gem::Dependency
+  name: rake
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '10.0'
+- !ruby/object:Gem::Dependency
+  name: minitest
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '5.0'
+- !ruby/object:Gem::Dependency
+  name: delayed_job_active_record
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.1'
+- !ruby/object:Gem::Dependency
+  name: railties
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '4.0'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '4.0'
+description: With DjOne you can get rid of duplicate schedules of particular DelayedJob
+  so that your workers don't have to do the same work twice and you don't need to
+  ensure synchronization between those jobs
+email:
+- piotrj@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- ".gitignore"
+- ".travis.yml"
+- CODE_OF_CONDUCT.md
+- Gemfile
+- LICENSE.txt
+- README.md
+- Rakefile
+- bin/console
+- bin/setup
+- dj_one.gemspec
+- lib/dj_one.rb
+- lib/dj_one/generator.rb
+- lib/dj_one/templates/install_dj_one_migration.rb
+- lib/dj_one/version.rb
+homepage: https://github.com/piotrj/dj_one
+licenses:
+- MIT
+metadata: {}
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 2.6.8
+signing_key: 
+specification_version: 4
+summary: DjOne ensures uniqueness of your DelayedJobs
+test_files: []