tennis-jobs 0.3.1 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 49030ba355431fc2193b862fca1b994a08a9b2b3
-  data.tar.gz: 874a6dfb0356b5baa869612716b3e0401c0ea062
+  metadata.gz: 524542c022f441416696197818564f5c424a02fa
+  data.tar.gz: 7c41960d2ca72c6f4a0dee4e3d4af48bdca0c0ee
 SHA512:
-  metadata.gz: e856689513cfbbd2f9c03e2ecc8449037b0183d73edff60f02c939f714e83182f9d657f86700c8765ef5ee000d5f830c8c0dadd53ebe2134b070483734985190
-  data.tar.gz: bd48fcc612bcdc5f327555d0df9d585e7625538a70a0e980230337614daa9fd0836d9e890a6dc22b660c6e9ac43e1deaadcffe573cae4071f57fd56b1995be6f
+  metadata.gz: 1fbd7b880e73ce697cd5b6b001d127e8d2501a60596ea68966ad7739e003d7640c41e54e20a7c277b9059dc66de991b1b24370a3f739be89bef20c932a72abcd
+  data.tar.gz: d6b25ced4f0566bcbb9f44d781e4b6a922dbe5f5606dc6ee104bc6e8a293f6ade4f6044b4ec307c2e0e86fdc1bf2aa55a760cbdd6edc40896de86c709099691a

data/README.markdown

@@ -1,207 +1,76 @@
-This small library is intended to help creating asynchronous jobs
-using Ruby and RabbitMQ via the Sneakers gem.
+This is a celluloid based background jobs library heavily inspired by [Sidekiq][sidekiq].
+The difference is that it allows you to change the backend, Redis, to other ones like RabitMQ.
+You can start with something _à la_ [sucker_punch][sucker_punch]: background jobs threaded within the application process
+and switch to separated processes later using another backend.
 
 <a target="_blank" href="https://travis-ci.org/nicoolas25/tennis"><img src="https://travis-ci.org/nicoolas25/tennis.svg?branch=master" /></a>
 <a target="_blank" href="https://codeclimate.com/github/nicoolas25/tennis"><img src="https://codeclimate.com/github/nicoolas25/tennis/badges/gpa.svg" /></a>
 <a target="_blank" href="https://codeclimate.com/github/nicoolas25/tennis/coverage"><img src="https://codeclimate.com/github/nicoolas25/tennis/badges/coverage.svg" /></a>
 <a target="_blank" href="https://rubygems.org/gems/tennis-jobs"><img src="https://badge.fury.io/rb/tennis-jobs.svg" /></a>
 
-## Features
-
-- Hooks: `.before(symbol, &block)`
-- Serializers: `.serialize(loader:)`
-- Helpers for defering method calls: `object.defer.method(*arguments)`
-
-**Extra**
-
-- A `Tennis::Serializer::Generic` handling classes and ActiveRecord objects
-
 ## Configuration
 
-The background job require a group of processes to handle the tasks you want to
-do asynchronously. Tennis uses YAML configuration file in order to launch thoses
-processes.
-
-``` yaml
-# tennis.conf.yml
-group1:
-  exchange: "default"
-  workers: 1
-  classes:
-    - "Scope::MyClass"
-    - "Scope::Model"
-group2:
-  exchange: "important"
-  workers: 10
-  classes:
-    - "Only::ImportantWorker"
-```
-
-Here we see two groups of worker. Each group can be launch with the `tennis`
-command:
-
-    $ bundle exec tennis group1
-
-The `workers` options is directly given to sneakers, it will determine the
-number of subprocesses that will handle the messages, the level of parallelism.
-
-The classes are the classes that will receive your `send_work` or `defer` calls
-but we'll see that later...
-
-Also it is possible to add options directly in your workers:
+Install the gem in your Gemfile:
 
 ``` ruby
-module WorkerHelpers
-  BeforeFork = -> { ActiveRecord::Base.connection_pool.disconnect! rescue nil }
-  AfterFork = -> { ActiveRecord::Base.establish_connection }
-end
-
-class MyClass
-  include Tennis::Worker::Generic
-
-  set_option :before_fork, WorkerHelpers::BeforeFork
-  set_option :after_fork, WorkerHelpers::AfterFork
-  set_option :handler, Sneakers::Handlers::MaxretryWithoutErrors
-
-  work do |message|
-    MyActiveRecordModel.create(message: message)
-  end
-end
+gem "tennis-jobs"
+gem "tennis-jobs-redis" # Not available at the moment
 ```
 
-In this example I use constants to store the Proc options. This is because
-having different options for the workers of a same group isn't possible. By
-using this constant, I can have another worker with those same options and put
-it in the same group.
-
-## Examples
-
-Those examples are what we wish to achieve.
-
-The name of the queue is the name of the class by default and can be reset
-using `set_option :queue_name, "your-queue-name"`.
-
-### Hooks
+Configure Tennis in your `config/application.rb` (or any other file):
 
 ``` ruby
-class MyClass
-  include Tennis::Worker::Generic
-
-  before do
-    puts "Before processing"
-  end
+Tennis.configure do |config|
+  config.backend Tennis::Backend::Redis.new(redis_url)
 
-  work do |message|
-    puts "Working with #{message}"
-    ack!
-  end
+  # require "logger"
+  # config.logger = Logger.new(STDOUT)
 end
-
-MyClass.send_work("my class")
-# => Before processing
-# => Working with my class
 ```
 
-### Serializers
-
-You can provide a Proc for the loader and/or the dumper keywords.
-The dumper will be used when calling `MyClass.send_work(message)` receiving
-the `message` as argument. It should return a string. The loader will be
-used when the message is poped from the RabbitMQ queue.
+Start tennis from the command line:
 
-``` ruby
-class MyClass
-  include Tennis::Worker::Generic
-
-  serialize loader: ->(message){ JSON.parse(message) },
-            dumper: ->(message){ JSON.generate(message) }
-
-  work do |message|
-    one, two = message
-    puts "Message is serialized and deserialized correctly"
-    puts "one: #{one}, two: #{two}"
-    ack!
-  end
-end
-
-MyClass.send_work([1, "foo"])
-# => Message is serialized and deserialized correctly
-# => one: 1, two: foo
 ```
+bundle exec tennis --concurrency 4 --require ./config/application.rb --jobs "MyJob,MyOtherJob"
 
-``` ruby
-class MyClass
-  include Tennis::Worker::Generic
-
-  serialize Tennis::Serializer::Generic.new
-
-  work do |message|
-    klass, active_record_object = message
-    puts "Classes can be passed: #{klass.name} - #{klass.class}"
-    puts "Active record object can be passed too: #{active_record_object}"
-    ack!
-  end
-end
-
-MyClass.send_work([String, User.find(1)])
-# => Classes can be passed: String - Class
-# => Active record object can be passed too: <User#1 ...>
+# There is also a shorter equivalent:
+# bundle exex tennis -c 4 -r ./config/application.rb -j "MyJob,MyOtherJob"
 ```
 
-### Helpers
-
-Any class method can be defered:
+## Usage
 
 ``` ruby
-class MyClass
-  include Tennis::Worker::Deferable
+MINUTES = 60
 
-  def self.my_method(user)
-    puts "Running my method on #{user}"
+class MyJob
+  include Tennis::Job
+
+  def my_method(*args)
+    puts "=> #{args}.sum = args.inject(0, &:+)"
   end
 end
 
-MyClass.defer.my_method(User.find(1))
-# => Running my method on <User#1 ...>
-```
-
-An ActiveRecord::Base instance can be the receiver if it has an `id`:
+my_job_instance = MyJob.new
+my_job_instance.async.my_method(1, 2, 3)
 
-``` ruby
-class MyModel < ActiveRecord::Base
-  include Tennis::Worker::Deferable
+# Will print in your `tennis` process:
+# => [1, 2, 3].sum = 6
 
-  def my_method
-    puts "Running my method on #{self}"
-  end
-end
+my_job_instance.async_in(2 * MINUTES).my_method(4, 5, 6)
 
-instance = MyModel.find(1)
-instance.defer.my_method
-# => Running my method on <MyModel#1 ...>
+# Will print, in approximatively two minutes, in your `tennis` process:
+# => [4, 5, 6].sum = 15
 ```
 
-Handling errors and results for the deferred methods is via `on_error` and
-`on_success` keyword. You must return `ack!` or `reject!` here as in a
-Sneakers' `work` method.
+The `my_method`'s arguments can be quite complex depending on your backend support.
+The same goes for the `MyJob`'s instance.
 
-``` ruby
-class MyModel < ActiveRecord::Base
-  include Tennis::Worker::Deferable
+With the `Tennis::Backend::Memory` backend, you can use anything and it will be kept as it is.
 
-  on_error do |exception|
-    puts "I just saved the day handling a #{exception}"
-    reject!
-  end
+## Testing
 
-  def my_method
-    puts "Running my method on #{self}"
-    raise StandardError
-  end
-end
+This section is waiting to beeing written.
 
-instance = MyModel.find(1)
-instance.defer.my_method
-# => Running my method on <MyModel#1 ...>
-# => I just saved the day handling a StandardError
-```
+
+[sidekiq]: https://github.com/mperham/sidekiq
+[sucker_punch]: https://github.com/brandonhilkert/sucker_punch

data/examples/async_sum.rb

@@ -0,0 +1,36 @@
+require "tennis"
+require "tennis/backend/memory"
+
+logger = Logger.new(STDOUT)
+logger.level = Logger::DEBUG
+
+Tennis.configure do |config|
+  config.backend = Tennis::Backend::Memory.new(logger: logger)
+  config.logger = logger
+end
+
+class Job
+  include Tennis::Job
+
+  def sum(*numbers)
+    sleep 0.4
+    total = numbers.inject(&:+)
+    puts "Sum #{numbers} => #{total}"
+  end
+end
+
+require "tennis/launcher"
+
+# Instanciate a job and add the sum to the job to do.
+numbers = (1..9).to_a
+10.times do
+  Job.new.async.sum(*numbers.sample(3))
+end
+
+# Start Tennis.
+launcher = Tennis::Launcher.new(concurrency: 2, job_classes: [Job])
+launcher.async.start
+
+# Wait 1 seconds and stop Tennis
+sleep 1
+launcher.async.stop

data/lib/tennis.rb

@@ -1,16 +1,7 @@
-require "sneakers"
+require "tennis/configuration"
 
 module Tennis
-  module Worker
-    autoload :Deferable, "tennis/worker/deferable"
-    autoload :Generic,   "tennis/worker/generic"
-  end
-
-  module Serializer
-    autoload :Generic, "tennis/serializer/generic"
-  end
-
-  autoload :Configuration, "tennis/configuration"
+  autoload :Job, "tennis/job"
 
   def self.configure
     @config = Configuration.new
@@ -22,4 +13,8 @@ module Tennis
     @config or fail "You must run Tennis.configure before accessing the configuration"
   end
 
+  def self.logger
+    @config.logger
+  end
+
 end

data/lib/tennis/action.rb

@@ -0,0 +1,30 @@
+module Tennis
+  class Action
+
+    attr_reader :_receiver
+
+    def initialize(receiver, delay: nil)
+      @_receiver = receiver
+      @delay = delay
+      _create_methods!
+    end
+
+    private
+
+    def _create_methods!
+      _methods.each do |method|
+        self.define_singleton_method(method) do |*arguments|
+          _store(job: @_receiver, method: method, args: arguments)
+        end
+      end
+    end
+
+    def _methods
+      @_receiver.class.instance_methods(false).map(&:to_s)
+    end
+
+    def _store(**kwargs)
+      Tennis.config.backend.enqueue(**kwargs.merge(delay: @delay))
+    end
+  end
+end

data/lib/tennis/actor.rb

@@ -0,0 +1,49 @@
+require "celluloid" unless $TESTING
+
+module Tennis
+  module Actor
+
+    module ClassMethods
+      def new_link(*args)
+        new(*args)
+      end
+
+      def trap_exit(*args)
+      end
+    end
+
+    module InstanceMethods
+      def current_actor
+        self
+      end
+
+      def after(interval)
+        yield
+      end
+
+      def alive?
+        @dead = false unless defined?(@dead)
+        !@dead
+      end
+
+      def terminate
+        @dead = true
+      end
+
+      def async
+        self
+      end
+    end
+
+    # :nocov:
+    def self.included(klass)
+      if $TESTING
+        klass.include InstanceMethods
+        klass.extend ClassMethods
+      else
+        klass.include Celluloid
+      end
+    end
+    # :nocov:
+  end
+end

data/lib/tennis/backend/abstract.rb

@@ -0,0 +1,38 @@
+module Tennis
+  module Backend
+    class Abstract
+
+      attr_reader :logger
+
+      def initialize(logger:)
+        @logger = logger
+      end
+
+      # :nocov:
+
+      # Creates and enqueues a Task.
+      def enqueue(job:, method:, args:, delay: nil)
+        raise NotImplementedError
+      end
+
+      # Returns a Task that have previously been queued. The Task should
+      # contain a Job that is an instance of one of the given classes.
+      def receive(job_classes:, timeout: 1.0)
+        raise NotImplementedError
+      end
+
+      # Acknowledge that a Task has been done.
+      def ack(task)
+        raise NotImplementedError
+      end
+
+      # Requeue a Task that haven't been acked.
+      def requeue(task)
+        raise NotImplementedError
+      end
+
+      # :nocov:
+
+    end
+  end
+end

data/lib/tennis/backend/memory.rb

@@ -0,0 +1,57 @@
+require "thread"
+
+require "tennis/backend/abstract"
+require "tennis/backend/task"
+
+module Tennis
+  module Backend
+    class Memory < Abstract
+
+      attr_reader :queue, :acked_tasks
+
+      def initialize(**kwargs)
+        super
+        @mutex = Mutex.new
+        @task_id = 0
+        @queue = []
+        @acked_tasks = []
+        @acked_history_size = kwargs.fetch(:acked_history_size, 10)
+      end
+
+      def enqueue(job:, method:, args:, delay: nil)
+        @mutex.synchronize do
+          @task_id += 1
+          queue << Task.new(self, @task_id, job, method, args)
+        end
+      end
+
+      def receive(job_classes:, timeout: 1.0)
+        @mutex.synchronize do
+          task = queue.find { |task| job_classes.include?(task.job.class) }
+
+          if task.nil?
+            sleep(timeout)
+            nil
+          else
+            queue.delete(task)
+            task
+          end
+        end
+      end
+
+      def ack(task)
+        @mutex.synchronize do
+          acked_tasks.unshift task
+          acked_tasks.pop if acked_tasks.size > @acked_history_size
+        end
+      end
+
+      def requeue(task)
+        @mutex.synchronize do
+          queue << task
+        end
+      end
+
+    end
+  end
+end