procrastinator 0.2.3 → 0.3.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 48d4ad814bc4f44c3f3af37309040106da854571
-  data.tar.gz: cf4cbc98185f4511a8ea3daaaa4c7aeffa997f4f
+  metadata.gz: 46f0ec23b44a55033b241f8f1eea274a99dcc481
+  data.tar.gz: 84476d93786e0e63f53e11cbf913c8df9f641134
 SHA512:
-  metadata.gz: fb464436151e500f902c3c2ce289d9baefadea247b4ea14197cbc351d84dde58d779f20162b537dcf123b55f4311599e4d21891577d74775d680ec7766b21b84
-  data.tar.gz: 4761c5cdabf41895a16fc5693590a39e030c797e889ebada79327e0c419f6ba39ccd9c32fa1bcf077c14c7a5e06860a6ced3944dfc0bdf5a433f782702656229
+  metadata.gz: 0f54aba2184d65a6dba03a756688565bfc6d45520ea521a0ce488353ea81c02ff1946d072145b176bc5ddc643f01670256bfb9b73819f70a7961d10f63ce1712
+  data.tar.gz: 8181179a9437191dc43b34e61e67c7f846a83633962d7f3f41b1a69696b22769e85b5a15a1f8672c03af7c1f3fb8d0f3290fea6cef908e3bcf8b7da2128589b8

data/README.md

@@ -1,7 +1,9 @@
 # Procrastinator
 
-Procrastinator is a framework-independent job scheduling gem. It creates a subprocess for each queue that performs 
-tasks at the appropriate times. 
+Procrastinator is a framework-independent job scheduling gem to allow your app to put stuff of until later. It creates 
+a subprocess for each queue to performs tasks at the designated times. Or maybe later, depending on how busy it is. 
+
+Don't worry, it'll get done eventually. 
 
 ## Installation
 
@@ -38,12 +40,12 @@ procrastinator.delay(queue: :cleanup, run_at: Time.now + 3600, task: ClearTempDa
 
 Read on for more details on each step. 
 
-###`Procrastinator.setup`
+### Setup Phase
 The setup phase first defines which queues are available and the persistence strategy to use for reading 
 and writing tasks. It then starts up a sub process for working on each queue within that environment.
 
 
-#### Persister Strategy
+#### Declaring a Persistence Strategy
 The persister instance is the last step between Procrastinator and your data storage (eg. database). As core Procrastinator is framework-agnostic, it needs you to provide an object that knows how to read and write task data. 
 
 Your [strategy](https://en.wikipedia.org/wiki/Strategy_pattern) class is required to provide the following methods: 
@@ -71,7 +73,7 @@ If the strategy does not have all of these methods, Procrastinator will explode
 
 Notice that the times are all given as unix epoch timestamps. This is to avoid any confusion with timezones, and it is recommended that you store times in this manner for the same reason. 
 
-####Defining Queues
+#### Defining Queues
 `Procrastinator.setup` requires a block be provided, and that in the block call `#define_queue` be called on the provided environment. Define queue takes a queue name symbol and these properies as a hash
 
  * :timeout - Time, in seconds, after which it should fail tasks in this queue for taking too long to execute.
@@ -89,36 +91,37 @@ Procrastinator.setup(some_persister) do |env|
 end
 ```
   
-#### Queue Sub-Processes
+#### Sub-Processes
 Each queue is worked in a separate process.  
 
 <!-- , and each process multi-threaded to handle more than one task at a time. This should help prevent a single task from clogging up the whole queue, or a single queue clogging up the entire system. -->
 
-The sub-processes check every 5 seconds that the parent process is still alive. If there is no process with the parent's process ID, the sub-process will self-exit. 
+The sub-processes checks that the parent process is still alive every 5 seconds. If there is no process with the parent's PID, the sub-process will self-exit. 
 
-###`Environment#delay`
-When there are multiple queues defined, you are required to provide a queue name:
+###Scheduling Tasks For Later
+Procrastinator will let you be lazy: 
 
 ```ruby
 procrastinator = Procrastinator.setup(task_persister) do |env|
    env.define_queue(:email)
-   env.define_queue(:cleanup)
 end
 
-procrastinator.delay(:email, task: EmailReminder.new('bob@example.com'))
+procrastinator.delay(task: EmailReminder.new('bob@example.com'))
 ```
 
-Otherwise, Procrastinator will let you be lazy: 
+... unless there are multiple queues defined. Thne you must provide a queue name with your task:
 
 ```ruby
 procrastinator = Procrastinator.setup(task_persister) do |env|
    env.define_queue(:email)
+   env.define_queue(:cleanup)
 end
 
-procrastinator.delay(task: EmailReminder.new('bob@example.com'))
+procrastinator.delay(:email, task: EmailReminder.new('bob@example.com'))
 ```
 
-You can set when the particular task is to be run and/or when it should expire. Be aware that neither is guaranteed to be real-time; a task will run as soon as it possible after `run_at` is passed, and will expire once it attempts to run after `expire_at` has passed. 
+You can set when the particular task is to be run and/or when it should expire. Be aware that the task is not guaranteed 
+to run at a precise time; the only promise is that the task will get run some time after `run_at`, unless it's after `expire_at`. 
 
 ```ruby
 procrastinator = Procrastinator.setup(task_persister) do |env|
@@ -145,7 +148,7 @@ procrastinator.delay(expire_at: , task: EmailGreeting.new('bob@example.com'))
 procrastinator.delay(expire_at: nil, task: EmailGreeting.new('bob@example.com'))
 ```
 
-#### Task
+#### Task Definition
 Like the persister provided to `.setup`, your task is a strategy object that fills in the details of what to do. For this, 
 your task **must provide** a `#run` method:
 
@@ -168,6 +171,35 @@ Tasks that fail have their `run_at` rescheduled on an increasing delay according
 
 Both failing and final_failing will cause the error timestamp and reason to be stored in `:last_fail_at` and `:last_error`.
 
+### Testing With Procrastinator
+Procrastinator uses multi-threading and multi-processing internally, which is a nightmare for testing. Fortunately for you, 
+Test Mode will disable all of that, and rely on your tests to tell it when to tick. 
+
+Enable Test Mode by setting `Procrastinator.test_mode` to `true` before setting up, or by calling enable_test_mode on 
+the procrastination environment. 
+
+```ruby
+# all further calls to `Procrastinator.setup` will produce a procrastination environment where Test Mode is enabled
+Procrastinator.test_mode = true
+ 
+# or you can also enable it directly in the setup
+env = Procrastinator.setup do |env|
+   env.enable_test_mode
+    
+   # other settings...
+end
+```
+
+In your tests, tell the procrastinator environment to work off one item from its queues: 
+
+```
+# works one task on all queues
+env.act
+
+# provide queue names to works one task on just those queues
+env.act(:cleanup, :email)
+```
+
 ## Contributing
 Bug reports and pull requests are welcome on GitHub at 
 [https://github.com/TenjinInc/procrastinator](https://github.com/TenjinInc/procrastinator).

data/lib/procrastinator.rb

@@ -5,17 +5,27 @@ require 'procrastinator/environment'
 
 
 module Procrastinator
+   @@test_mode = false
+
    def self.setup(persister, &block)
       raise ArgumentError.new('Procrastinator.setup must be given a block') if block.nil?
 
-      env = Environment.new(persister)
+      env = Environment.new(persister: persister, test_mode: @@test_mode)
 
       yield(env)
 
-      raise RuntimeError.new('setup block did not define any queues') if env.queues.empty?
+      raise RuntimeError.new('setup block did not define any queues') if env.queue_definitions.empty?
 
       env.spawn_workers
 
       env
    end
+
+   def self.test_mode=(value)
+      @@test_mode = value
+   end
+
+   def self.test_mode
+      @@test_mode
+   end
 end

data/lib/procrastinator/environment.rb

@@ -1,50 +1,74 @@
 module Procrastinator
    class Environment
-      attr_reader :persister, :queues, :processes
+      attr_reader :persister, :queue_definitions, :queue_workers, :processes, :test_mode
 
-      def initialize(persister)
+      def initialize(persister:, test_mode: false)
          raise ArgumentError.new('persister cannot be nil') if persister.nil?
 
          [:read_tasks, :create_task, :update_task, :delete_task].each do |method|
             raise MalformedPersisterError.new("persister must repond to ##{method}") unless persister.respond_to? method
          end
 
-         @persister = persister
-         @queues    = {}
-         @processes = []
+         @persister         = persister
+         @test_mode         = test_mode
+         @queue_definitions = {}
+         @queue_workers     = []
+         @processes         = []
       end
 
       def define_queue(name, properties={})
          raise ArgumentError.new('queue name cannot be nil') if name.nil?
 
-         @queues[name] = properties
+         @queue_definitions[name] = properties
       end
 
       def spawn_workers
-         @queues.each do |name, props|
-            pid = fork do
-               Process.setproctitle("#{name}-queue-worker")
+         if @test_mode
+            @queue_definitions.each do |name, props|
+               @queue_workers << QueueWorker.new(props.merge(name: name, persister: @persister))
+            end
+         else
+            @queue_definitions.each do |name, props|
+               pid = fork do
+                  Process.setproctitle("#{name}-queue-worker")
+
+                  worker = QueueWorker.new(props.merge(name: name, persister: @persister))
 
-               worker = QueueWorker.new(props.merge(name: name, persister: @persister))
+                  monitor_parent
 
-               monitor_parent
+                  worker.work
+               end
 
-               worker.work
+               Process.detach(pid) unless pid.nil?
+               @processes << pid
             end
+         end
+      end
+
+      def act(*queue_names)
+         unless @test_mode
+            raise RuntimeError.new('Procrastinator.act called outside Test Mode. Enable test mode by setting Procrastinator.test_mode = true before running setup')
+         end
 
-            Process.detach(pid) unless pid.nil?
-            @processes << pid
+         if queue_names.empty?
+            @queue_workers.each do |worker|
+               worker.act
+            end
+         else
+            queue_names.each do |name|
+               @queue_workers.find { |worker| worker.name == name }.act
+            end
          end
       end
 
       def delay(queue: nil, run_at: Time.now.to_i, expire_at: nil, task:)
          raise ArgumentError.new('task may not be nil') if task.nil?
          raise MalformedTaskError.new('given task does not support #run method') unless task.respond_to? :run
-         if queue.nil? && @queues.size > 1
-            raise ArgumentError.new('queue must be specified when more than one is registered')
+         if queue.nil? && @queue_definitions.size > 1
+            raise ArgumentError.new("queue must be specified when more than one is registered. Defined queues are: #{queue_definitions.keys.map { |k| ':' + k.to_s }.join(', ')}")
          else
-            queue ||= @queues.keys.first
-            raise ArgumentError.new(%Q{there is no "#{queue}" queue registered in this environment}) if @queues[queue].nil?
+            queue ||= @queue_definitions.keys.first
+            raise ArgumentError.new(%Q{there is no "#{queue}" queue registered in this environment}) if @queue_definitions[queue].nil?
          end
 
          @persister.create_task(queue:          queue,
@@ -54,6 +78,10 @@ module Procrastinator
                                 task:           YAML.dump(task))
       end
 
+      def enable_test_mode
+         @test_mode = true
+      end
+
       private
       def monitor_parent
          heartbeat_thread = Thread.new(Process.ppid) do |ppid|

data/lib/procrastinator/queue_worker.rb

@@ -1,6 +1,6 @@
 module Procrastinator
    class QueueWorker
-      DEFAULT_TIMEOUT       = 3600 # seconds = one hour
+      DEFAULT_TIMEOUT       = 3600 # in seconds; one hour total
       DEFAULT_MAX_ATTEMPTS  = 20
       DEFAULT_UPDATE_PERIOD = 10 # seconds
       DEFAULT_MAX_TASKS     = 10
@@ -34,21 +34,25 @@ module Procrastinator
          loop do
             sleep(@update_period)
 
-            # shuffling and re-sorting to avoid worst case O(n^2) on quicksort
-            # when receiving already sorted data. Ideally, we'd use a better algo, but this will do for now
-            tasks = @persister.read_tasks(@name).shuffle.sort_by { |t| t[:run_at] }
+            act
+         end
+      end
+
+      def act
+         # shuffling and re-sorting to avoid worst case O(n^2) on quicksort
+         # when receiving already sorted data. Ideally, we'd use a better algo, but this will do for now
+         tasks = @persister.read_tasks(@name).shuffle.sort_by { |t| t[:run_at] }
 
-            tasks.first(@max_tasks).each do |task_data|
-               if Time.now.to_i >= task_data[:run_at].to_i
-                  tw = TaskWorker.new(task_data)
+         tasks.first(@max_tasks).each do |task_data|
+            if Time.now.to_i >= task_data[:run_at].to_i
+               tw = TaskWorker.new(task_data)
 
-                  tw.work
+               tw.work
 
-                  if tw.successful?
-                     @persister.delete_task(task_data[:id])
-                  else
-                     @persister.update_task(tw.to_hash.merge(queue: @name))
-                  end
+               if tw.successful?
+                  @persister.delete_task(task_data[:id])
+               else
+                  @persister.update_task(tw.to_hash.merge(queue: @name))
                end
             end
          end

data/lib/procrastinator/task_worker.rb

@@ -53,7 +53,7 @@ module Procrastinator
             else
                try_hook(:fail, e)
 
-               @last_error = %Q[Task failed: #{e.message}\n #{e.backtrace.join("\n")}]
+               @last_error = %Q[Task failed: #{e.message}\n#{e.backtrace.join("\n")}]
 
                reschedule
             end

data/lib/procrastinator/version.rb

@@ -1,3 +1,3 @@
 module Procrastinator
-   VERSION = '0.2.3'
+   VERSION = '0.3.0'
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: procrastinator
 version: !ruby/object:Gem::Version
-  version: 0.2.3
+  version: 0.3.0
 platform: ruby
 authors:
 - Robin Miller
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2016-02-25 00:00:00.000000000 Z
+date: 2016-06-12 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: bundler