cloudtasker 0.11.0 → 0.12.rc5

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 8f4e53fe9c99a8c6a74e064d7b7c562cdb671682aaed9bb6112a77c7fd453124
-  data.tar.gz: 3b2e6aeca4fde352acd7b0db6fe469221514bf7fd753c07c3d9540d881bf1c2f
+  metadata.gz: a1f63a9bfefe90d0cfa0d6b567098ec72efe150894fbd878daa72fe934d27a25
+  data.tar.gz: 347d6358120bd83f116b569fafcd0313a96d8c34a4f6222ca20b0dedda866098
 SHA512:
-  metadata.gz: 4be2ce9466e7ffd2cc274d57d79eedad96ae56d699632f88bfdf6384413aa9bfa19dff8a124926bd2b41229ce53f3126b2bd04646cc9125136bc93fdffb0642a
-  data.tar.gz: f6a3f7fa152f5ebedbe25945e86538691ee658c5bc02b840fd3d1f337ba0fcd78db6a84c52bcd6ade01af51335ee81d457152bfa9767d3ac9541b5d31ef48126
+  metadata.gz: a4ecf9ad17d612133653f70e7691fc746742c3ca62eb8f67c09bdc1992776391e755a5c674b97d0339bd74514958299062610d448b089fe5995dd6abe5756021
+  data.tar.gz: 231f0ba89cf89db4bdb138a1d34d10688161d156c9898da7a067783f75f930b9318a08dfdf240188e23dfa3a0e9dd59b38b1b235b5ab5038eb6727a244877017

data/.rubocop.yml

@@ -6,13 +6,15 @@ AllCops:
     - 'vendor/**/*'
 
 Metrics/ClassLength:
-  Max: 150
+  Max: 200
 
 Metrics/ModuleLength:
   Max: 150
 
 Metrics/AbcSize:
   Max: 20
+  Exclude:
+    - 'spec/support/*'
 
 Metrics/LineLength:
   Max: 120
@@ -47,4 +49,9 @@ Metrics/ParameterLists:
   CountKeywordArgs: false
 
 RSpec/MessageSpies:
-  Enabled: false
+  Enabled: false
+
+RSpec/MultipleExpectations:
+  Exclude:
+    - 'examples/**/*'
+    - 'spec/integration/**/*'

data/CHANGELOG.md

@@ -1,6 +1,21 @@
 # Changelog
 
-## [v0.11.0](https://github.com/keypup-io/cloudtasker/tree/v0.11.0) (2020-11-23)
+## Latest RC [v0.12.rc5](https://github.com/keypup-io/cloudtasker/tree/v0.12.rc5) (2021-03-30)
+
+[Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.11.0...v0.12.rc5)
+
+**Improvements:**
+- ActiveJob: do not double log errors (ActiveJob has its own error logging)
+- Error logging: Use worker logger so as to include context (job args etc.)
+- Error logging: Do not log exception and stack trace separately, combine them instead.
+- Batch callbacks: Retry jobs when completion callback fails
+- Redis: Use Redis Sets instead of key pattern matching for listing methods (Cron jobs and Local Server)
+- Batch progress: restrict calculation to direct children by default. Allow depth to be specified. Calculating progress using all tree jobs created significant delays on large batches.
+
+**Fixed bugs:**
+- Retries: Enforce job retry limit on job processing. There was an edge case where jobs could be retried indefinitely on batch callback errors.
+
+## [v0.11.0](https://github.com/keypup-io/cloudtasker/tree/v0.11.0) (2021-03-11)
 
 [Full Changelog](https://github.com/keypup-io/cloudtasker/compare/v0.10.0...v0.11.0)
 

data/README.md

@@ -12,7 +12,7 @@ Cloudtasker also provides optional modules for running [cron jobs](docs/CRON_JOB
 
 A local processing server is also available for development. This local server processes jobs in lieu of Cloud Tasks and allows you to work offline.
 
-**Maturity**: This gem is production-ready. We at Keypup have already processed millions of jobs using Cloudtasker and all related extensions (cron, batch and unique jobs). I'm waiting till the end of 2020 before releasing the official `v1.0.0` in case we've missed any edge-case bug.
+**Maturity**: This gem is production-ready. We at Keypup have already processed millions of jobs using Cloudtasker and all related extensions (cron, batch and unique jobs). We are planning to release the official `v1.0.0` somewhere in 2021, in case we've missed any edge-case bug.
 
 ## Summary
 
@@ -136,7 +136,7 @@ That's it! Your job was picked up by the Cloudtasker local server and sent for p
 Now jump to the next section to configure your app to use Google Cloud Tasks as a backend.
 
 ## Get started with Rails & ActiveJob
-**Note**: ActiveJob is supported since `0.11.0`
+**Note**: ActiveJob is supported since `0.11.0`  
 **Note**: Cloudtasker extensions (cron, batch and unique jobs) are not available when using cloudtasker via ActiveJob.
 
 Cloudtasker is pre-integrated with ActiveJob. Follow the steps below to get started.

data/app/controllers/cloudtasker/worker_controller.rb

@@ -19,32 +19,19 @@ module Cloudtasker
       # Process payload
       WorkerHandler.execute_from_payload!(payload)
       head :no_content
-    rescue DeadWorkerError, MissingWorkerArgumentsError => e
+    rescue DeadWorkerError, MissingWorkerArgumentsError
       # 205: job will NOT be retried
-      log_error(e)
       head :reset_content
-    rescue InvalidWorkerError => e
+    rescue InvalidWorkerError
       # 404: Job will be retried
-      log_error(e)
       head :not_found
-    rescue StandardError => e
-      # 404: Job will be retried
-      log_error(e)
+    rescue StandardError
+      # 422: Job will be retried
       head :unprocessable_entity
     end
 
     private
 
-    #
-    # Log an error via cloudtasker logger.
-    #
-    # @param [Exception] e The error to log
-    #
-    def log_error(error)
-      Cloudtasker.logger.error(error)
-      Cloudtasker.logger.error(error.backtrace.join("\n"))
-    end
-
     #
     # Parse the request body and return the actual job
     # payload.

data/docs/BATCH_JOBS.md

@@ -84,8 +84,29 @@ You can access progression statistics in callback using `batch.progress`. See th
 E.g.
 ```ruby
 def on_batch_node_complete(_child_job)
-  logger.info("Total: #{batch.progress.total}")
-  logger.info("Completed: #{batch.progress.completed}")
-  logger.info("Progress: #{batch.progress.percent.to_i}%")
+  progress = batch.progress
+  logger.info("Total: #{progress.total}")
+  logger.info("Completed: #{progress.completed}")
+  logger.info("Progress: #{progress.percent.to_i}%")
+end
+```
+
+**Since:** `v0.12.rc5`  
+By default the `progress` method only considers the direct child jobs to evaluate the batch progress. You can pass `depth: somenumber` to the `progress` method to calculate the actual batch progress in a more granular way. Be careful however that this method recursively calculates progress on the sub-batches and is therefore expensive.
+
+E.g.
+```ruby
+def on_batch_node_complete(_child_job)
+  # Considers the children for batch progress calculation
+  progress_0 = batch.progress # same as batch.progress(depth: 0)
+
+  # Considers the children and grand-children for batch progress calculation
+  progress_1 = batch.progress(depth: 1)
+
+  # Considers the children, grand-children and grand-grand-children for batch progress calculation
+  progress_2 = batch.progress(depth: 3)
+
+  logger.info("Progress: #{progress_1.percent.to_i}%")
+  logger.info("Progress: #{progress_2.percent.to_i}%")
 end
 ```

data/lib/cloudtasker/backend/redis_task.rb

@@ -23,14 +23,12 @@ module Cloudtasker
       #
       # Return a namespaced key.
       #
-      # @param [String, Symbol] val The key to namespace
+      # @param [String, Symbol, nil] val The key to namespace
       #
       # @return [String] The namespaced key.
       #
-      def self.key(val)
-        return nil if val.nil?
-
-        [to_s.underscore, val.to_s].join('/')
+      def self.key(val = nil)
+        [to_s.underscore, val].compact.map(&:to_s).join('/')
       end
 
       #
@@ -39,9 +37,17 @@ module Cloudtasker
       # @return [Array<Cloudtasker::Backend::RedisTask>] All the tasks.
       #
       def self.all
-        redis.search(key('*')).map do |gid|
-          payload = redis.fetch(gid)
-          new(payload.merge(id: gid.sub(key(''), '')))
+        if redis.exists?(key)
+          # Use Schedule Set if available
+          redis.smembers(key).map { |id| find(id) }
+        else
+          # Fallback to redis key matching and migrate tasks
+          # to use Task Set instead.
+          redis.search(key('*')).map do |gid|
+            task_id = gid.sub(key(''), '')
+            redis.sadd(key, task_id)
+            find(task_id)
+          end
         end
       end
 
@@ -82,6 +88,7 @@ module Cloudtasker
 
         # Save job
         redis.write(key(id), payload)
+        redis.sadd(key, id)
         new(payload.merge(id: id))
       end
 
@@ -105,6 +112,7 @@ module Cloudtasker
       # @param [String] id The task id.
       #
       def self.delete(id)
+        redis.srem(key, id)
         redis.del(key(id))
       end
 
@@ -170,13 +178,14 @@ module Cloudtasker
           schedule_time: (Time.now + interval).to_i,
           queue: queue
         )
+        redis.sadd(self.class.key, id)
       end
 
       #
       # Remove the task from the queue.
       #
       def destroy
-        redis.del(gid)
+        self.class.delete(id)
       end
 
       #

data/lib/cloudtasker/batch/job.rb

@@ -13,6 +13,10 @@ module Cloudtasker
       # List of statuses triggering a completion callback
       COMPLETION_STATUSES = %w[completed dead].freeze
 
+      # These callbacks do not need to raise errors on their own
+      # because the jobs will be either retried or dropped
+      IGNORED_ERRORED_CALLBACKS = %i[on_child_error on_child_dead].freeze
+
       #
       # Return the cloudtasker redis client
       #
@@ -250,8 +254,8 @@ module Cloudtasker
       end
 
       #
-      # Run worker callback in a controlled environment to
-      # avoid interruption of the callback flow.
+      # Run worker callback. The error and dead callbacks get
+      # silenced should they raise an error.
       #
       # @param [String, Symbol] callback The callback to run.
       # @param [Array<any>] *args The callback arguments.
@@ -261,9 +265,14 @@ module Cloudtasker
       def run_worker_callback(callback, *args)
         worker.try(callback, *args)
       rescue StandardError => e
-        Cloudtasker.logger.error("Error running callback #{callback}: #{e}")
-        Cloudtasker.logger.error(e.backtrace.join("\n"))
-        nil
+        # There is no point in retrying jobs due to failure callbacks failing
+        # Only completion callbacks will trigger a re-run of the job because
+        # these do matter for batch completion
+        raise(e) unless IGNORED_ERRORED_CALLBACKS.include?(callback)
+
+        # Log error instead
+        worker.logger.error(e)
+        worker.logger.error("Callback #{callback} failed to run. Skipping to preserve error flow.")
       end
 
       #
@@ -275,7 +284,7 @@ module Cloudtasker
 
         # Propagate event
         parent_batch&.on_child_complete(self, status)
-      ensure
+
         # The batch tree is complete. Cleanup the tree.
         cleanup unless parent_batch
       end
@@ -338,13 +347,20 @@ module Cloudtasker
       #
       # @return [Cloudtasker::Batch::BatchProgress] The batch progress.
       #
-      def progress
+      def progress(depth: 0)
+        depth = depth.to_i
+
         # Capture batch state
         state = batch_state
 
-        # Sum batch progress of current batch and all sub-batches
+        # Return immediately if we do not need to go down the tree
+        return BatchProgress.new(state) if depth <= 0
+
+        # Sum batch progress of current batch and sub-batches up to the specified
+        # depth
         state.to_h.reduce(BatchProgress.new(state)) do |memo, (child_id, child_status)|
-          memo + (self.class.find(child_id)&.progress || BatchProgress.new(child_id => child_status))
+          memo + (self.class.find(child_id)&.progress(depth: depth - 1) ||
+            BatchProgress.new(child_id => child_status))
         end
       end
 

data/lib/cloudtasker/cron/schedule.rb

@@ -21,14 +21,12 @@ module Cloudtasker
       #
       # Return a namespaced key.
       #
-      # @param [String, Symbol] val The key to namespace
+      # @param [String, Symbol, nil] val The key to namespace
       #
       # @return [String] The namespaced key.
       #
-      def self.key(val)
-        return nil if val.nil?
-
-        [to_s.underscore, val.to_s].join('/')
+      def self.key(val = nil)
+        [to_s.underscore, val].compact.map(&:to_s).join('/')
       end
 
       #
@@ -37,8 +35,17 @@ module Cloudtasker
       # @return [Array<Cloudtasker::Batch::Schedule>] The list of stored schedules.
       #
       def self.all
-        redis.search(key('*')).map do |gid|
-          find(gid.sub(key(''), ''))
+        if redis.exists?(key)
+          # Use Schedule Set if available
+          redis.smembers(key).map { |id| find(id) }
+        else
+          # Fallback to redis key matching and migrate schedules
+          # to use Schedule Set instead.
+          redis.search(key('*')).map do |gid|
+            schedule_id = gid.sub(key(''), '')
+            redis.sadd(key, schedule_id)
+            find(schedule_id)
+          end
         end
       end
 
@@ -90,7 +97,7 @@ module Cloudtasker
       end
 
       #
-      # Destroy a schedule by id.
+      # Delete a schedule by id.
       #
       # @param [String] id The schedule id.
       #
@@ -101,6 +108,7 @@ module Cloudtasker
 
           # Delete task and stored schedule
           CloudTask.delete(schedule.task_id) if schedule.task_id
+          redis.srem(key, schedule.id)
           redis.del(schedule.gid)
         end
       end
@@ -270,6 +278,7 @@ module Cloudtasker
 
         # Save schedule
         config_was_changed = config_changed?
+        redis.sadd(self.class.key, id)
         redis.write(gid, to_h)
 
         # Stop there if backend does not need update

data/lib/cloudtasker/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Cloudtasker
-  VERSION = '0.11.0'
+  VERSION = '0.12.rc5'
 end

data/lib/cloudtasker/worker.rb

@@ -311,13 +311,25 @@ module Cloudtasker
     end
 
     #
-    # Return true if the job has excceeded its maximum number
-    # of retries
+    # Return true if the job must declared dead upon raising
+    # an error.
+    #
+    # @return [Boolean] True if the job must die on error.
+    #
+    def job_must_die?
+      job_retries >= job_max_retries
+    end
+
+    #
+    # Return true if the job has strictly excceeded its maximum number
+    # of retries.
+    #
+    # Used a preemptive filter when running the job.
     #
     # @return [Boolean] True if the job is dead
     #
     def job_dead?
-      job_retries >= job_max_retries
+      job_retries > job_max_retries
     end
 
     #
@@ -332,11 +344,35 @@ module Cloudtasker
       (perform_ended_at - perform_started_at).ceil(3)
     end
 
+    #
+    # Run worker callback.
+    #
+    # @param [String, Symbol] callback The callback to run.
+    # @param [Array<any>] *args The callback arguments.
+    #
+    # @return [any] The callback return value
+    #
+    def run_callback(callback, *args)
+      try(callback, *args)
+    end
+
     #=============================
     # Private
     #=============================
     private
 
+    #
+    # Flag the worker as dead by invoking the on_dead hook
+    # and raising a DeadWorkerError
+    #
+    # @param [Exception, nil] error An optional exception to be passed to the DeadWorkerError.
+    #
+    def flag_as_dead(error = nil)
+      run_callback(:on_dead, error || DeadWorkerError.new)
+    ensure
+      raise(DeadWorkerError, error)
+    end
+
     #
     # Execute the worker perform method through the middleware chain.
     #
@@ -346,6 +382,9 @@ module Cloudtasker
       self.perform_started_at = Time.now
 
       Cloudtasker.config.server_middleware.invoke(self) do
+        # Immediately abort the job if it is already dead
+        flag_as_dead if job_dead?
+
         begin
           # Abort if arguments are missing. This may happen with redis arguments storage
           # if Cloud Tasks times out on a job but the job still succeeds
@@ -356,12 +395,11 @@ module Cloudtasker
           # Perform the job
           perform(*job_args)
         rescue StandardError => e
-          try(:on_error, e)
-          return raise(e) unless job_dead?
+          run_callback(:on_error, e)
+          return raise(e) unless job_must_die?
 
           # Flag job as dead
-          try(:on_dead, e)
-          raise(DeadWorkerError, e)
+          flag_as_dead(e)
         end
       end
     ensure

data/lib/cloudtasker/worker_handler.rb

@@ -45,6 +45,28 @@ module Cloudtasker
       end
     end
 
+    #
+    # Log error on execution failure.
+    #
+    # @param [Cloudtasker::Worker, nil] worker The worker.
+    # @param [Exception] error The error to log.
+    #
+    # @void
+    #
+    def self.log_execution_error(worker, error)
+      # ActiveJob has its own error logging. No need to double log the error.
+      # Note: we use string matching instead of class matching as
+      # ActiveJob::QueueAdapters::CloudtaskerAdapter::JobWrapper might not be loaded
+      return if worker.class.to_s =~ /^ActiveJob::/
+
+      # Choose logger to use based on context
+      # Worker will be nil on InvalidWorkerError - in that case we use generic logging
+      logger = worker&.logger || Cloudtasker.logger
+
+      # Log error
+      logger.error(error)
+    end
+
     #
     # Execute a task worker from a task payload
     #
@@ -88,6 +110,10 @@ module Cloudtasker
     rescue DeadWorkerError, MissingWorkerArgumentsError => e
       # Delete stored args payload if job is dead
       redis.expire(args_payload_key, ARGS_PAYLOAD_CLEANUP_TTL) if args_payload_key
+      log_execution_error(worker, e)
+      raise(e)
+    rescue StandardError => e
+      log_execution_error(worker, e)
       raise(e)
     end
 

data/lib/cloudtasker/worker_logger.rb

@@ -51,6 +51,26 @@ module Cloudtasker
       Cloudtasker.logger
     end
 
+    #
+    # Format the log message as string.
+    #
+    # @param [Object] msg The log message or object.
+    #
+    # @return [String] The formatted message
+    #
+    def formatted_message_as_string(msg)
+      # Format message
+      msg_content = if msg.is_a?(Exception)
+                      [msg.inspect, msg.backtrace].flatten(1).join("\n")
+                    elsif msg.is_a?(String)
+                      msg
+                    else
+                      msg.inspect
+                    end
+
+      "[Cloudtasker][#{worker.class}][#{worker.job_id}] #{msg_content}"
+    end
+
     #
     # Format main log message.
     #
@@ -59,7 +79,12 @@ module Cloudtasker
     # @return [String] The formatted log message
     #
     def formatted_message(msg)
-      "[Cloudtasker][#{worker.class}][#{worker.job_id}] #{msg}"
+      if msg.is_a?(String)
+        formatted_message_as_string(msg)
+      else
+        # Delegate object formatting to logger
+        msg
+      end
     end
 
     #
@@ -147,7 +172,9 @@ module Cloudtasker
       # ActiveSupport::Logger does not support passing a payload through a block on top
       # of a message.
       if defined?(ActiveSupport::Logger) && logger.is_a?(ActiveSupport::Logger)
-        logger.send(level) { "#{formatted_message(msg)} -- #{payload_block.call}" }
+        # The logger is fairly basic in terms of formatting. All inputs get converted
+        # as regular strings.
+        logger.send(level) { "#{formatted_message_as_string(msg)} -- #{payload_block.call}" }
       else
         logger.send(level, formatted_message(msg), &payload_block)
       end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: cloudtasker
 version: !ruby/object:Gem::Version
-  version: 0.11.0
+  version: 0.12.rc5
 platform: ruby
 authors:
 - Arnaud Lachaume
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-03-11 00:00:00.000000000 Z
+date: 2021-03-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: activesupport
@@ -427,9 +427,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - ">"
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
 rubygems_version: 3.0.0
 signing_key: