sucker_punch 1.6.0 → 2.0.0.beta1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: d6ccd098769b0b630161eb92038c13e662a58786
-  data.tar.gz: 0a99540886529b095b99a745514b54cf49a75cf2
+  metadata.gz: fa1e2aa3c637a849298159655805260a18ce9315
+  data.tar.gz: 2a138d958c61e7c1e014ff7e9d682e8c9a5f78e4
 SHA512:
-  metadata.gz: b9d74d6377d9abc828a43333460966ab7779499c75ece7fe78c68f9b1002572322da70754ed7032ccc963ceac797aa2cb7df0ac77a19b80e8ad13f559bb5001f
-  data.tar.gz: 83a79b2a2ecdb619f781d8dcb2b2e50d6cea08d3a34e134b1c13f29fa129b7e4e92eb671bf4fe43b37788386690bf9273fa85f3ccb3a795536238e17d3a289e7
+  metadata.gz: 03d99e04ebd2fc6aaa07e91242673411727a5cbc50d350d3b83a9a2b28fb81ec90bc26eb7c4d83b3f840da0881928b784b3c07e077205ad1d41333d627dac5ed
+  data.tar.gz: 588214f4b4d233352bf06ae84ad6b2948e575e2b089c7b7fe9b39d9dceecccc4b9f189fd24a6945119b4ffb2cdbd2962529ba43779b213aef65b297f60790fca

data/.travis.yml

@@ -1,12 +1,24 @@
 language: ruby
 rvm:
-  - 1.9.3
-  - jruby-19mode
   - rbx-2
   - 2.0.0
   - 2.1.0
+  - 2.2.0
+  - 2.3.0
   - ruby-head
+  - jruby-9.0.1.0
+  - jruby-9.0.3.0
+  - jruby-9.0.4.0
+  - jruby-head
+
+jdk:
+  - oraclejdk8
+
 matrix:
   allow_failures:
-    - rvm: jruby-19mode
     - rvm: rbx-2
+    - rvm: jruby-head
+
+before_script:
+  - "echo $JAVA_OPTS"
+  - "export JAVA_OPTS=-Xmx1024m"

data/CHANGES.md

@@ -1,3 +1,33 @@
+2.0.0.beta1
+-------
+
+- Refactor internals to use `concurrent-ruby`
+- Yield more exception details to handler. The new syntax allows you to setup a
+    global exception with the following syntax:
+
+    ```ruby
+    # ex    => The caught exception object
+    # klass => The job class
+    # args  => An array of the args passed to the job
+
+    SuckerPunch.exception_handler = -> (ex, klass, args) { ExceptionNotifier.notify_exception(ex) }
+    ```
+
+- Invoke asynchronous job via `perform_async` and `perform_in` class method (*backwards
+    incompatible change*):
+
+    ```ruby
+    LogJob.perform_async("login")
+    LogJob.perform_in(60, "login") # `perform` will be executed 60 sec. later
+    ```
+
+- Drop support for Ruby `< 2.0`
+- Allow shutdown timeout to be set:
+
+    ```ruby
+    SuckerPunch.shutdown_timeout = 15 # time in seconds
+    ```
+
 1.6.0
 --------
 

data/README.md

@@ -1,20 +1,29 @@
+# Notice: This README is for the `master` branch (`v2 beta`), not the latest stable branch
+
 # Sucker Punch
 
 [![Build Status](https://travis-ci.org/brandonhilkert/sucker_punch.png?branch=master)](https://travis-ci.org/brandonhilkert/sucker_punch)
 [![Code Climate](https://codeclimate.com/github/brandonhilkert/sucker_punch.png)](https://codeclimate.com/github/brandonhilkert/sucker_punch)
 
-Sucker Punch is a single-process Ruby asynchronous processing library. It's [girl_friday](https://github.com/mperham/girl_friday)  and DSL sugar on top of [Celluloid](https://github.com/celluloid/celluloid/). With Celluloid's actor pattern, we can do asynchronous processing within a single process. This reduces costs of hosting on a service like Heroku along with the memory footprint of having to maintain additional jobs if hosting on a dedicated server. All queues can run within a single Rails/Sinatra process.
+Sucker Punch is a single-process Ruby asynchronous processing library.
+This reduces costs
+of hosting on a service like Heroku along with the memory footprint of
+having to maintain additional jobs if hosting on a dedicated server.
+All queues can run within a single application (eg. Rails, Sinatra, etc.) process.
 
-Sucker Punch is perfect for asynchronous processes like emailing, data crunching, or social platform manipulation. No reason to hold up a user when you can do these things in the background within the same process as your web application...
+Sucker Punch is perfect for asynchronous processes like emailing, data
+crunching, or social platform manipulation. No reason to hold up a
+user when you can do these things in the background within the same
+process as your web application...
 
-Sucker Punch is built on top of [Celluloid
-Pools](https://github.com/celluloid/celluloid/wiki/Pools). Each job is setup as
+Sucker Punch is built on top of [concurrent-ruby]
+(https://github.com/ruby-concurrency/concurrent-ruby). Each job is setup as
 a pool, which equates to its own queue with individual workers working against
 the jobs. Unlike most other background processing libraries, Sucker
 Punch's jobs are stored in memory. The benefit to this is there is no
-additional infrastructure requirement (ie. database, redis, etc.). The downside
-is that if the web processes is restarted and there are jobs that haven't yet
-been processed, they will be lost. For this reason, Sucker Punch is generally
+additional infrastructure requirement (ie. database, redis, etc.). However,
+if the web processes are restarted with jobs remaining in the queue,
+they will be lost. For this reason, Sucker Punch is generally
 recommended for jobs that are fast and non-mission critical (ie. logs, emails,
 etc.).
 
@@ -22,7 +31,7 @@ etc.).
 
 Add this line to your application's Gemfile:
 
-    gem 'sucker_punch', '~> 1.0'
+    gem 'sucker_punch', '~> 2.0'
 
 And then execute:
 
@@ -37,7 +46,7 @@ Or install it yourself as:
 Each job acts as its own queue and should be a separate Ruby class that:
 
 * includes `SuckerPunch::Job`
-* defines the instance method `perform` that includes the code the job will run when enqueued
+* defines the `perform` instance method that includes the code the job will run when enqueued
 
 
 ```Ruby
@@ -52,90 +61,94 @@ class LogJob
 end
 ```
 
-Synchronous:
+#### Synchronous
 
 ```Ruby
 LogJob.new.perform("login")
 ```
 
-Asynchronous:
+#### Asynchronous
 
 ```Ruby
-LogJob.new.async.perform("login") # => nil
+LogJob.perform_async("login")
 ```
 
-Jobs interacting with `ActiveRecord` should take special precaution not to exhaust connections in the pool. This can be done with `ActiveRecord::Base.connection_pool.with_connection`, which ensures the connection is returned back to the pool when completed.
+#### Configure the # of the Workers
 
-```Ruby
-# app/jobs/awesome_job.rb
+The default number of workers (threads) running against your job is `2`. If
+you'd like to configure this manually, the number of workers can be
+set on the job using the `workers` class method:
 
-class AwesomeJob
+```Ruby
+class LogJob
   include SuckerPunch::Job
+  workers 4
 
-  def perform(user_id)
-    ActiveRecord::Base.connection_pool.with_connection do
-      user = User.find(user_id)
-      user.update_attributes(is_awesome: true)
-    end
+  def perform(event)
+    Log.new(event).track
   end
 end
 ```
 
-We can create a job from within another job:
+#### Executing Jobs in the Future
 
-```Ruby
-class AwesomeJob
+Many background processing libraries have methods to perform operations after a
+certain amount of time and Sucker Punch is no different. Use the `perform_in`
+with an argument of the number of seconds in the future you would like the job
+to job to run.
+
+``` ruby
+class DataJob
   include SuckerPunch::Job
 
-  def perform(user_id)
-    ActiveRecord::Base.connection_pool.with_connection do
-      user = User.find(user_id)
-      user.update_attributes(is_awesome: true)
-      LogJob.new.async.perform("User #{user.id} became awesome!")
-    end
+  def perform(data)
+    puts data
   end
 end
+
+DataJob.perform_async("asdf") # immediately perform asynchronously
+DataJob.perform_in(60, "asdf") # `perform` will be excuted 60 sec. later
 ```
 
-The number of workers can be set from the Job using the `workers` method:
+#### `ActiveRecord` Connection Pool Connections
+
+Jobs interacting with `ActiveRecord` should take special precaution not to
+exhaust connections in the pool. This can be done
+with `ActiveRecord::Base.connection_pool.with_connection`, which ensures
+the connection is returned back to the pool when completed.
 
 ```Ruby
-class LogJob
+# app/jobs/awesome_job.rb
+
+class AwesomeJob
   include SuckerPunch::Job
-  workers 4
 
-  def perform(event)
-    Log.new(event).track
+  def perform(user_id)
+    ActiveRecord::Base.connection_pool.with_connection do
+      user = User.find(user_id)
+      user.update(is_awesome: true)
+    end
   end
 end
 ```
 
-If the `workers` method is not set, the default is `2`.
-
-## Perform In
-
-Many background processing libraries have methods to perform operations after a
-certain amount of time. Fortunately, timers are built-in to Celluloid, so you
-can take advantage of them with the `later` method:
+We can create a job from within another job:
 
-``` ruby
-class Job
+```Ruby
+class AwesomeJob
   include SuckerPunch::Job
 
-  def perform(data)
-    puts data
-  end
-
-  def later(sec, data)
-    after(sec) { perform(data) }
+  def perform(user_id)
+    ActiveRecord::Base.connection_pool.with_connection do
+      user = User.find(user_id)
+      user.update_attributes(is_awesome: true)
+      LogJob.perform_async("User #{user.id} became awesome!")
+    end
   end
 end
-
-Job.new.async.perform("asdf")
-Job.new.async.later(60, "asdf") # `perform` will be excuted 60 sec. later
 ```
 
-## Logger
+#### Logger
 
 ```Ruby
 SuckerPunch.logger = Logger.new('sucker_punch.log')
@@ -145,43 +158,57 @@ SuckerPunch.logger # => #<Logger:0x007fa1f28b83f0>
 _Note: If Sucker Punch is being used within a Rails application, Sucker Punch's logger
 is set to Rails.logger by default._
 
-## Exceptions
+#### Exceptions
 
 You can customize how to handle uncaught exceptions that are raised by your jobs.
 
-For example, using Rails and the ExceptionNotification gem, add a new initializer `config/initializers/sucker_punch.rb`:
+For example, using Rails and the ExceptionNotification gem,
+add a new initializer `config/initializers/sucker_punch.rb`:
 
 ```Ruby
-SuckerPunch.exception_handler { |ex| ExceptionNotifier.notify_exception(ex) }
+# ex    => The caught exception object
+# klass => The job class
+# args  => An array of the args passed to the job
+
+SuckerPunch.exception_handler = -> (ex, klass, args) { ExceptionNotifier.notify_exception(ex) }
 ```
 
 Or, using Airbrake:
 
 ```Ruby
-SuckerPunch.exception_handler { |ex| Airbrake.notify(ex) }
+SuckerPunch.exception_handler = -> (ex, klass, args) { Airbrake.notify(ex) }
 ```
 
-Full job data can be reported like this:
+#### Shutdown Timeout
 
-```Ruby
-def perform(all, my, arguments)
-  ... your code ...
-rescue StandardError
-  Airbrake.error($!, [self.class.name, all, my, arguments].inspect)
-  raise
-end
+Sucker Punch goes through a series of checks to attempt to shut down the queues
+and their threads. A "shutdown" command is issued to the queues, which gives
+them notice but allows them to attempt to finish all remaining jobs.
+Subsequently enqueued jobs are discarded at this time.
+
+The default `shutdown_timeout` (the # of seconds to wait before forcefully
+killing the threads) is 8 sec. This is to allow applications hosted on Heroku
+to attempt to shutdown prior to the 10 sec. they give an application to
+shutdown with some buffer.
+
+To configure something other than the default 8 sec.:
+
+```ruby
+  SuckerPunch.shutdown_timeout = 15 # # of sec. to wait before killing threads
 ```
 
-## Timeouts
+#### Timeouts
 
-Using `Timeout` causes persistent connections to [randomly get corrupted](http://www.mikeperham.com/2015/05/08/timeout-rubys-most-dangerous-api).
-Do not use timeouts as control flow, use builtin connection timeouts.
+Using `Timeout` causes persistent connections to
+[randomly get corrupted](http://www.mikeperham.com/2015/05/08/timeout-rubys-most-dangerous-api).
+Do not use timeouts as control flow, use built-in connection timeouts.
 If you decide to use Timeout, only use it as last resort to know something went very wrong and
 ideally restart the worker process after every timeout.
 
 ## Testing
 
-Requiring this library causes your jobs to run everything inline. So a call to the following will actually be SYNCHRONOUS:
+Requiring this library causes your jobs to run everything inline.
+So a call to the following will actually be SYNCHRONOUS:
 
 ```Ruby
 # spec/spec_helper.rb
@@ -189,7 +216,7 @@ require 'sucker_punch/testing/inline'
 ```
 
 ```Ruby
-Log.new.async.perform("login") # => Will be synchronous and block until job is finished
+LogJob.perform_async("login") # => Will be synchronous and block until job is finished
 ```
 
 ## Rails
@@ -230,28 +257,10 @@ end
 ### Initializers for forking servers (Unicorn, Passenger, etc.)
 
 Previously, Sucker Punch required an initializer and that posed problems for
-Unicorn and Passenger and other servers that fork.  Version 1 was rewritten to
+servers that fork (ie. Unicorn and Passenger). Version 1 was rewritten to
 not require any special code to be executed after forking occurs. Please remove
  if you're using version `>= 1.0.0`
 
-### Class naming
-
-Job classes are ultimately Celluloid Actor classes. As a result, class names
-are susceptible to being clobbered by Celluloid's internal classes.  To ensure
-the intended application class is loaded, preface classes with `::`, or use
-names like `NotificationsMailer` or `UserMailer`. Example:
-
-```Ruby
-class EmailJob
-  include SuckerPunch::Job
-
-  def perform(contact)
-    @contact = contact
-    ::Notifications.contact_form(@contact).deliver # => If you don't use :: in this case, the notifications class from Celluloid will be loaded
-  end
-end
-```
-
 ### Cleaning test data transactions
 
 If you're running tests in transactions (using Database Cleaner or a native solution), Sucker Punch jobs may have trouble finding database records that were created during test setup because the job class is running in a separate thread and the Transaction operates on a different thread so it clears out the data before the job can do its business. The best thing to do is cleanup data created for tests jobs through a truncation strategy by tagging the rspec tests as jobs and then specifying the strategy in `spec_helper` like below. And do not forget to turn off transactional fixtures (delete, comment or set it to `false`).
@@ -298,12 +307,6 @@ describe EmailJob, job: true do
 end
 ```
 
-## Gem Name
-
-...is awesome. But I can't take credit for it. Thanks to
-[@jmazzi](https://twitter.com/jmazzi) for his superior naming skills. If you're
-looking for a name for something, he is the one to go to.
-
 ## Contributing
 
 1. Fork it

data/Rakefile

@@ -1,12 +1,15 @@
 require "bundler/gem_tasks"
-require 'rspec/core/rake_task'
+require "rake/testtask"
 
-RSpec::Core::RakeTask.new('spec')
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList['test/**/*_test.rb']
+end
 
-# If you want to make this the default task
-task :default => :spec
-task :test => :spec
+task :default => :test
 
 task :console do
   exec "irb -r sucker_punch -I ./lib"
-end
+end
+

data/bin/load

@@ -0,0 +1,23 @@
+#!/usr/bin/env ruby
+
+require 'bundler'
+Bundler.require(:default)
+
+require_relative '../lib/sucker_punch'
+
+class NoopJob
+  include SuckerPunch::Job
+
+  def perform(a)
+  end
+end
+
+puts "Enqueuing 1MM jobs..."
+
+1_000_000.times { NoopJob.perform_async(1) }
+
+puts "Executing jobs..."
+
+while SuckerPunch::Queue.all["NoopJob"]["jobs"]["enqueued"] > 0
+  sleep 0.01
+end

data/bin/shutdown

@@ -0,0 +1,20 @@
+#!/usr/bin/env ruby
+
+require 'bundler'
+Bundler.require(:default)
+
+require_relative '../lib/sucker_punch'
+
+class CountdownJob
+  include SuckerPunch::Job
+
+  def perform(i)
+    sleep 0.1
+    print "Executing job #{i}\n"
+  end
+end
+
+puts "Enqueuing 100 jobs..."
+
+100.times { |i| CountdownJob.perform_async(i) }
+sleep 0.5

data/lib/sucker_punch/async_syntax.rb

@@ -0,0 +1,18 @@
+module SuckerPunch
+  module Job
+    def async
+      AsyncProxy.new(self)
+    end
+  end
+
+  class AsyncProxy
+    def initialize(job)
+      @job = job
+    end
+
+    def perform(*args)
+      @job.class.perform_async(*args)
+    end
+  end
+end
+

data/lib/sucker_punch/core_ext.rb

@@ -1,9 +1,51 @@
-class String
-  def underscore
-    self.gsub(/::/, '/').
-    gsub(/([A-Z]+)([A-Z][a-z])/,'\1_\2').
-    gsub(/([a-z\d])([A-Z])/,'\1_\2').
-    tr("-", "_").
-    downcase
-  end if !"".respond_to?(:underscore)
+begin
+  require 'active_support/core_ext/class/attribute'
+rescue LoadError
+
+  # A dumbed down version of ActiveSupport's
+  # Class#class_attribute helper.
+  class Class
+    def class_attribute(*attrs)
+      instance_writer = true
+
+      attrs.each do |name|
+        class_eval <<-RUBY, __FILE__, __LINE__ + 1
+          def self.#{name}() nil end
+          def self.#{name}?() !!#{name} end
+
+          def self.#{name}=(val)
+            singleton_class.class_eval do
+              define_method(:#{name}) { val }
+            end
+
+            if singleton_class?
+              class_eval do
+                def #{name}
+                  defined?(@#{name}) ? @#{name} : singleton_class.#{name}
+                end
+              end
+            end
+            val
+          end
+
+          def #{name}
+            defined?(@#{name}) ? @#{name} : self.class.#{name}
+          end
+
+          def #{name}?
+            !!#{name}
+          end
+        RUBY
+
+        attr_writer name if instance_writer
+      end
+    end
+
+    private
+    def singleton_class?
+      ancestors.first != self
+    end
+  end
 end
+
+

data/lib/sucker_punch/counter.rb

@@ -0,0 +1,72 @@
+module SuckerPunch
+  module Counter
+    module Utilities
+      def value
+        @counter.value
+      end
+
+      def increment
+        @counter.increment
+      end
+
+      def decrement
+        @counter.decrement
+      end
+    end
+
+    class Busy
+      attr_reader :counter
+
+      include Utilities
+
+      COUNTER = Concurrent::Map.new do |hash, name|
+        hash.compute_if_absent(name) { Concurrent::AtomicFixnum.new }
+      end
+
+      def self.clear
+        COUNTER.clear
+      end
+
+      def initialize(queue_name)
+        @counter = COUNTER[queue_name]
+      end
+    end
+
+    class Processed
+      attr_reader :counter
+
+      include Utilities
+
+      COUNTER = Concurrent::Map.new do |hash, name|
+        hash.compute_if_absent(name) { Concurrent::AtomicFixnum.new }
+      end
+
+      def self.clear
+        COUNTER.clear
+      end
+
+      def initialize(queue_name)
+        @counter = COUNTER[queue_name]
+      end
+    end
+
+    class Failed
+      attr_reader :counter
+
+      include Utilities
+
+      COUNTER = Concurrent::Map.new do |hash, name|
+        hash.compute_if_absent(name) { Concurrent::AtomicFixnum.new }
+      end
+
+      def self.clear
+        COUNTER.clear
+      end
+
+      def initialize(queue_name)
+        @counter = COUNTER[queue_name]
+      end
+    end
+  end
+end
+