jobly 0.1.3 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 2e477c569a3ff769ded464f2d52e4775b254b348464057de68d3a63f2eec053c
-  data.tar.gz: 4bfdc207e0b541a52eb7e363cce13ed54635ed0b63e15e61ac5f2b17959ec98d
+  metadata.gz: 19d048c26e395618f64751b5d44598f0456782733361ea19f301bb8a3c30f056
+  data.tar.gz: c9407a5d8ac2915115448a6de9d022127fc92d06b81e1952835680c6309f00cc
 SHA512:
-  metadata.gz: 1a427c799068ede5b6a1b97b92f22d14cbc6569cd373d06168058e75f7b91a7cc3bb54e9a91774327ca557bef6c9de54af9e0f28bb7359db514ff7ef1f0bcad3
-  data.tar.gz: deacef64bbfa47d091134103d7450701a5dcdda4e9f5c21175c05969d17b9370552abc7f8e930a00d2271fc79669752e7aec0a5781d1c0e58cb59060b6fd6f82
+  metadata.gz: 9debe7765379021f9fda73fceecbd70b5967fe8b61c1888d628814be0f0e8302d51a7ce49b1c2c610750b8590611768f0e4b1b14ab041d4f0fa12f3a8e3cde8c
+  data.tar.gz: e6cb27d5c6ff080c6518e360d2b4409332548f07b4d26cfcb94ad4040ea08751daaa46df2476b1dc512ddda0e899dc0e8a48ac7b95e31c13dbcccca232b4964a

data/README.md

@@ -22,24 +22,6 @@ Compact job server with API, CLI, Web UI and a Sidekiq heart.
 
 ---
 
-* [Installation](#installation)
-* [What's in the Box](#whats-in-the-box)
-* [Quick Start](#quick-start)
-* [Usage](#usage)
-  * [Server](#server)
-  * [Worker](#worker)
-  * [Running jobs from the command line](#running-jobs-from-the-command-line)
-  * [Running jobs through the API](#running-jobs-through-the-api)
-* [Building Jobs](#building-jobs)
-  * [The Job Class](#the-job-class)
-    * [Job Options](#job-options)
-    * [Before, After, On Failure and On Success](#before-after-on-failure-and-on-success)
-* [Loading Additional Code](#loading-additional-code)
-* [Configuration](#configuration)
-  * [Worker Configuration](#worker-configuration)
-
----
-
 Follow development progress:  
 [![Waffle.io - Columns and their card count](https://badge.waffle.io/DannyBen/jobly.svg?columns=all&style=flat-square)](https://waffle.io/DannyBen/jobly)
 
@@ -48,6 +30,7 @@ Installation
 
     $ gem install jobly
 
+Or with the official [Jobly docker image][2].
 
 
 What's in the Box
@@ -63,205 +46,11 @@ sidekiq backgronud jobs system. It includes the following components:
 
 
 
-Quick Start
---------------------------------------------------
-
-Follow one of these annotated [examples](/examples).
-
-
-
-Usage
+Documentation
 --------------------------------------------------
 
-
-### Server
-
-To start the server run `jobly server` and open <http://localhost:3000/> 
-in your browser.
-
-This will start a webserver with two primary entrypoints:
-
-- `/` (root) - a dashboard for your background job processes.
-- `/do/JobName?param=value` - an API for executing jobs
-
-
-### Worker
-
-To start processing jobs, run `jobly worker`
-
-
-### Running jobs from the command line
-
-There are three ways to run a job from the command line:
-
-Run the job locally, without going through any of the background job 
-processing chain:
-
-    $ jobly run JobName param:value
-
-
-Run the job locally, but wait for a worker to process it.
-
-    $ jobly run --later JobName param:value
-
-
-Send a job through the API (either localhost or remote).
-
-    $ jobly send JobName param:value
-
-
-### Running jobs through the API
-
-The API supports running jobs either by GET or by POST in the following URL
-structure:
-
-<http://localhost:3000/do/JobName>
-
-Using GET:
-
-```
-$ curl localhost:3000/do/Build?deploy=no
-# => {"status":"received","job":"Build","params":{"deploy":"no"}}
-```
-
-Using POST:
-
-```
-$ curl -XPOST localhost:3000/do/Build -d deploy=yes
-{"status":"received","job":"Build","params":{"deploy":"yes"}}
-```
-
-
-Building Jobs
---------------------------------------------------
-
-To build a jobs "workspace", start in an empty folder and create a `./jobs` 
-subfolder inside it. All your job classes go in this folder (configurable).
-
-All job classes will be loaded by any of Jobly's commands.
-
-
-### The Job Class
-
-A job class is a simple Ruby class inheriting from 
-[`Jobly::Job`](/lib/jobly/job.rb).
-
-The only requirement is that your class implements an `execute` method that
-optionally accepts keyword arguments (recommended), or a hash.
-
-Example:
-
-```ruby
-class Hello < Jobly::Job
-  def execute(name: 'bob')
-    puts "Hello #{name}"
-    logger.info "said hello to #{name}"
-  end
-end
-```
-
-Note that these classes are simply Jobly-flavored sidekiq jobs, with these
-key differences:
-
-- You need to implement `execute` instead of `perform`
-- Job arguments are defined as keyword arguments, instead of positional 
-  arguments.
-
-#### Job Options
-
-The `Jobly::Job` class supports these options:
-
-| Key         | Default   | Purpose       
-|-------------|-----------|---------------
-| `queue`     | `default` | set the name of the queue for this job.
-| `retries`   |  `5`      | number of times to retry on failure.
-| `backtrace` |  `5`      | number of backtrace lines to show in case of failures. Can be `true`, `false` or a number of lines to save.
-
-For example:
-
-```ruby
-class Deploy < Jobly::Job
-  queue 'critical'
-  backtrace 10
-  retries 3
-  
-  def execute
-    puts "Deploying"
-  end
-end
-```
-
-
-#### Before, After, On Failure and On Success
-
-The `Jobly::Job` class supports these callback methods:
-
-| Method       | Description 
-|--------------|-------------
-| `before`     | Executes before the job starts
-| `on_success` | Executes after the job finishes, and only if it succeeds
-| `on_failure` | Executes after the job finishes, and only if it fails
-| `after`      | Executes after the job finishes, regardless of success or failure
-
-Each callback method can either be a block or a symbol that points to a local
-method. WHen using a block, you will have the `params` variable available, with
-all the parameteres sent to the job.
-
-For example:
-
-```ruby
-class Greet < Jobly::Job
-  before do
-    logger.info "Starting with #{params[:message]}"
-  end
-
-  after :reboot_computer
-  
-  def execute(message: "Hello")
-    puts message
-  end
-
-  def reboot_computer
-    system "reboot"
-  end
-end
-```
-
-
-Loading Additional Code
---------------------------------------------------
-
-In case your jobs require additional functionality, you may create the 
-`./app` folder as a sibling to the `./jobs` folder (configurable).
-
-Any ruby files in this folder (and subfolders) will be autmatically loaded
-and available to your jobs.
-
-
-Configuration
---------------------------------------------------
-
-Configuring Jobly can be done by one of two methods:
-
-1. Setting environment variables.
-2. Adding a `./config/jobly.rb` file.
-
-See this [example config file](/examples/02-full/config/jobly.rb) for a full
-annotated configuration example and a list of options with their respective
-environment variables.
-
-
-### Worker Configuration
-
-For advanced configuration of the sidekiq worker, beyond what the 
-`jobly worker` command provides, you can place YAML files in the config 
-folder and point the workers to their config file with
-`jobly worker --config name`.
-
-These files are simply [sidekiq configuration files][1].
-
-See the [worker-config](/examples/06-worker-config) example for more 
-information.
+Visit the [Jobly Documentation][1].
 
 
-[1]: https://github.com/mperham/sidekiq/wiki/Advanced-Options
+[1]: https://jobly.dannyb.co
+[2]: https://github.com/dannyben/docker-jobly

data/lib/jobly.rb

@@ -1,6 +1,7 @@
 require 'requires'
 require 'byebug' if ENV['BYEBUG']
 
+requires 'jobly/extensions'
 requires 'jobly/refinements'
 requires 'jobly/job_extensions'
 require 'jobly/exceptions'

data/lib/jobly/extensions/integer.rb

@@ -0,0 +1,21 @@
+class Integer
+  def seconds
+    self
+  end
+  alias_method :second, :seconds
+
+  def minutes
+    60 * seconds
+  end
+  alias_method :minute, :minutes
+
+  def hours
+    60 * minutes
+  end
+  alias_method :hour, :hours
+
+  def days
+    24 * hours
+  end
+  alias_method :day, :days
+end

data/lib/jobly/job.rb

@@ -4,6 +4,7 @@ module Jobly
     include Sidekiq::Status::Worker
     include JobExtensions::OptionAccessors
     include JobExtensions::Actions
+    include JobExtensions::Solo
     using KeywordArgs
 
     sidekiq_options retry: 5, backtrace: 5
@@ -34,20 +35,7 @@ module Jobly
     # implement keyword args.
     def perform(params={})
       @params = params
-      run_actions :before
-
-      begin
-        params.empty? ? execute : execute(params.to_kwargs)
-        run_actions :success
-
-      rescue
-        run_actions :failure
-        raise
-      
-      ensure
-        run_actions :after
-      
-      end
+      run_to_completion if run_before_filter
     end
 
     # Inheriting classes must implement this method only.
@@ -55,5 +43,29 @@ module Jobly
       raise NotImplementedError
     end
 
+  private
+
+    def run_before_filter
+      run_actions :before
+      if skipped?
+        run_actions :skip
+        run_actions :after
+        return false
+      end
+      return true
+    end
+
+    def run_to_completion
+      params.empty? ? execute : execute(params.to_kwargs)
+      run_actions :success
+
+    rescue
+      run_actions :failure
+      raise
+    
+    ensure
+      run_actions :after
+      
+    end
   end
 end

data/lib/jobly/job_extensions/actions.rb

@@ -1,7 +1,6 @@
 module Jobly
   module JobExtensions
     module Actions
-
       def self.included(base)
         base.extend ClassMethods
       end
@@ -27,11 +26,24 @@ module Jobly
           actions[:failure] << (sym || block)
         end
 
+        def on_skip(sym = nil, &block)
+          actions[:skip] ||= []
+          actions[:skip] << (sym || block)
+        end
+
         def actions
           @actions ||= {}
         end
       end
 
+      def skip_job
+        @skipped = true
+      end
+
+      def skipped?
+        @skipped
+      end
+
     protected
 
       def actions

data/lib/jobly/job_extensions/solo.rb

@@ -0,0 +1,54 @@
+require 'jobly/job_extensions/actions'
+
+module Jobly
+  module JobExtensions
+    module Solo
+      include Actions
+
+      def self.included(base)
+        base.extend ClassMethods
+      end
+
+      module ClassMethods
+        attr_reader :solo_key
+
+        def solo(expire: 1.hour, key: nil)
+          @solo_key = key if key
+          before { solo_locked? ? skip_job : solo_lock(expire) }
+          after  { solo_unlock }
+        end
+      end
+
+      def solo_key
+        @solo_key ||= (self.class.solo_key || solo_key!)
+      end
+
+      def solo_key!
+        Digest::MD5.hexdigest "#{self.class.name}:#{params}"
+      end
+
+      def solo_full_key
+        "jobly:solo:#{solo_key}"
+      end
+
+      def solo_locked?
+        Sidekiq.redis do |redis|
+          redis.keys(solo_full_key).count >= 1
+        end
+      end
+
+      def solo_lock(expire = 1.hour)
+        Sidekiq.redis do |redis|
+          redis.setex(solo_full_key, expire, "1")
+        end
+      end
+
+      def solo_unlock
+        Sidekiq.redis do |redis|
+          redis.del solo_full_key
+        end
+      end
+
+    end
+  end
+end

data/lib/jobly/module_functions.rb

@@ -15,8 +15,8 @@ module Jobly
       jobs_path: ENV['JOBLY_JOBS_PATH'] || "jobs",
       config_path: ENV['JOBLY_CONFIG_PATH'] || "config",
       redis_url: ENV['JOBLY_REDIS_URL'] || "redis://localhost:6379/0",
-      status_expiration: 30,
-      jobs_namespace: nil,
+      status_expiration: ENV['JOBLY_STATUS_EXPIRATION']&.to_i || 30,
+      jobs_namespace: ENV['JOBLY_JOBS_NAMESPACE'],
       logger: nil,
     }
   end

data/lib/jobly/version.rb

@@ -1,3 +1,3 @@
 module Jobly
-  VERSION = "0.1.3"
+  VERSION = "0.2.0"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: jobly
 version: !ruby/object:Gem::Version
-  version: 0.1.3
+  version: 0.2.0
 platform: ruby
 authors:
 - Danny Ben Shitrit
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2019-03-12 00:00:00.000000000 Z
+date: 2019-03-15 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: mister_bin
@@ -184,9 +184,11 @@ files:
 - lib/jobly/commands/worker.rb
 - lib/jobly/config.ru
 - lib/jobly/exceptions.rb
+- lib/jobly/extensions/integer.rb
 - lib/jobly/job.rb
 - lib/jobly/job_extensions/actions.rb
 - lib/jobly/job_extensions/option_accessors.rb
+- lib/jobly/job_extensions/solo.rb
 - lib/jobly/jobs.rb
 - lib/jobly/module_functions.rb
 - lib/jobly/refinements/argument_converters.rb