simple_worker 2.0.0.beta.16 → 2.0.0.beta.17

data/README.markdown

@@ -20,19 +20,6 @@ You really just need your token, which you can get [here][2]
         config.token = TOKEN
         config.project_id = MY_PROJECT_ID
     end
-    
-Generate a Worker with Rails
-----------------------------
-If you're rolling on Rails, use the following:    
-
-    rails generate simple_worker worker_class_in_underscore_format
-    
-This will set you up with a worker in ```app/workers```  
-However, this is not autorequired, so a line similar to this will be required in your initializer:  
-
-    Dir.glob("#{Rails.root}/app/workers/*.rb").each{|w| require w}
-    
-This one-liner requires all ```.rb``` files in ```app/workers```  
 
 Write a Worker
 --------------
@@ -41,17 +28,13 @@ Here's an example worker that sends an email:
 
     require 'simple_worker'
 
-    class EmailWorker < SimpleWorker::Base
+    class HelloWorker < SimpleWorker::Base
 
-        attr_accessor :to, :subject, :body
+        attr_accessor :name
 
         # This is the method that will be run
         def run
-            send_email(:to=>to, :subject=>subject, :body=>body)
-        end
-
-        def send_email
-            # Put sending code here
+            puts "Hello #{name}!"
         end
     end
 
@@ -60,10 +43,8 @@ Test It Locally
 
 Let's say someone does something in your app and you want to send an email about it.
 
-    worker = EmailWorker.new
-    worker.to = current_user.email
-    worker.subject = "Here is your mail!"
-    worker.body = "This is the body"
+    worker = HelloWorker.new
+    worker.name = "Travis"
     worker.run_local
 
 Once you've got it working locally, the next step is to run it on the SimpleWorker cloud.
@@ -73,279 +54,19 @@ Queue up your Worker on the SimpleWorker Cloud
 
 Let's say someone does something in your app and you want to send an email about it.
 
-    worker = EmailWorker.new
-    worker.to = current_user.email
-    worker.subject = "Here is your mail!"
-    worker.body = "This is the body"
+    worker = HelloWorker.new
+    worker.name = "Travis"
     worker.queue
 
 This will send it off to the SimpleWorker cloud.
 
-To queue worker without uploading you could try to do following:
-
-    data[:attr_encoded] = Base64.encode64({'@to'=>'example@email.com'}.to_json)
-    data[:sw_config] = SimpleWorker.config.get_atts_to_send
-    SimpleWorker.service.queue('EmailWorker', data)
-
-
-
-Setting Priority
-----------------------------------------------
-
-Simply define the priority in your queue command.
-
-    worker.queue(:priority=>1)
-
-Default priority is 0 and we currently support priority 0, 1, 2. See [pricing page](http://www.simpleworker.com/pricing)
-for more information on priorites.
-
-
-Schedule your Worker
---------------------
-
-There are two scenarios here, one is the scenario where you want something to happen due to a user
-action in your application. This is almost the same as queuing your worker.
-
-    worker = EmailWorker.new
-    worker.to = current_user.email
-    worker.subject = "Here is your mail!"
-    worker.body = "This is the body"
-    worker.schedule(:start_at=>1.hours.since)
-
-By default, if you call `schedule` more than once for the same worker class, the new schedule will replace the old one. If you'd
-like multiple schedules for the same class, provide a `:name` parameter:
-
-    worker.schedule(:name=>"EmailWorkerDaily", :start_at=>......)
-
-You can also set priority for scheduled jobs:
-
-    worker.schedule(:priority=>1, ....)
-
-
-Check Status
-------------
-
-If you still have access to the worker object, just call:
-
-    worker.status
-
-If you only have the job ID, call:
-
-    SimpleWorker.service.status(job_id)
-
-This will return a hash like:
-
-     {"task_id"=>"ece460ce-12d8-11e0-8e15-12313b0440c6",
-     "status"=>"running",
-     "msg"=>nil,
-     "start_time"=>"2010-12-28T23:19:36+00:00",
-     "end_time"=>nil,
-     "duration"=>nil,
-     "progress"=>{"percent"=>25}}
-
-There is also a convenience method `worker.wait_until_complete` that will wait until the status returned is completed or error.
-
-Logging
--------
-
-In your worker, just call the log method with the string you want logged:
-
-    log "Starting to do something..."
-
-The log will be available for viewing via the SimpleWorker UI or via log in the API:
-
-    SimpleWorker.service.log(job_id)
-
-or if you still have a handle to your worker object:
-
-    worker.get_log
-
-Setting Progress
-----------------
-
-This is just a way to let your users know where the job is at if required.
-
-    set_progress(:percent => 25, :message => "We are a quarter of the way there!")
-
-You can actually put anything in this hash and it will be returned with a call to status. We recommend using
-the format above for consistency and to get some additional features where we look for these values.
-
-Schedule a Recurring Job - CRON
-------------------------------
-
-The alternative is when you want to user it like Cron. In this case you'll probably
-want to write a script that will schedule, you don't want to schedule it everytime your
-app starts or anything so best to keep it external.
-
-Create a file called 'schedule_email_worker.rb' and add this:
-
-    require 'simple_worker'
-    require_relative 'email_worker'
-
-    worker = EmailWorker.new
-    worker.to = current_user.email
-    worker.subject = "Here is your mail!"
-    worker.body = "This is the body"
-    worker.schedule(:start_at=>1.hours.since, :run_every=>3600)
-
-Now run it and your worker will be scheduled to run every hour.
-
-SimpleWorker on Rails
----------------------
-
-Rails 2.X:
-
-    config.gem 'simple_worker'
-
-Rails 3.X:
-
-    gem 'simple_worker'
-
-Now you can use your workers like they're part of your app!  We recommend putting your worker classes in
-/app/workers path.  
-
-Configuring a Database Connection
----------------------------------
-
-Although you could easily do this in your worker, this makes it a bit more convenient and more importantly
-it will create the connection for you. If you are using Rails 3, you just need to add one line:
-
-    config.database = Rails.configuration.database_configuration[Rails.env]
-
-For non Rails 3, you would add the following to your SimpleWorker config:
-
-    config.database = {
-      :adapter => "mysql2",
-      :host => "localhost",
-      :database => "appdb",
-      :username => "appuser",
-      :password => "secret"
-    }
-
-Then before you job is run, SimpleWorker will establish the ActiveRecord connection.
-
-Including/Merging other Ruby Classes
-------------------------------------
-
-If you need to inclue other classes in for your worker to use (which is very common), then you'll
-want to use the merge functions. For example: 
-
-    class AvgWorker < SimpleWorker::Base
-
-        attr_accessor :aws_access_key,
-                      :aws_secret_key,
-                      :s3_suffix
-
-        merge File.join(File.dirname(__FILE__), "..", "app", "models", "user.rb")
-        merge File.join(File.dirname(__FILE__), "..", "app", "models", "account")
-
-Or simpler yet, try using relative paths:
-
-    merge "../models/user"
-    merge "../models/account.rb"
-
-Also you could use wildcards and path in merge_folder
-
-    merge_folder "./foldername/"     #will merge all *.rb files from foldername
-    merge_folder "../lib/**/"        # will merge all *.rb files from lib and all subdirectories
-
-The opposite can be done as well with "unmerge" and can be useful when using Rails to exclude classes that are automatically
-merged.
-
-
-Merging other Workers
----------------------
-
-Merging other workers is a bit different than merging other code like above because they will be
-uploaded separately and treated as distinctly separate workers.
-
-    merge_worker "./other_worker.rb", "OtherWorker"
-
-Merging Mailers
----------------------
-
-You could easily merge mailers you're using in your application.
-
-    merge_mailer 'mailer_file' #if your mailer's templates are placed in default path
-    #or
-    merge_mailer 'mailer_file', {:path_to_templates=>"templates_path"}#if you're using mailer outside of rails with custom templates path
-
-### Configuring a Mailer Connection
-
-If you are using Rails 3,your action_mailer connection would be configured automatically from your config
-
-For non Rails 3 or if you want to use different mailer configs, you should add the following to your SimpleWorker config:
-
-    config.mailer = {
-        :address => "smtp.gmail.com",
-        :port => 587,
-        :domain => 'gmail.com',
-        :user_name => GMAIL_USERNAME
-        :password => GMAIL_PASSWORD
-        :authentication => 'plain',
-        :enable_starttls_auto => true}
-
-Then before you job is run, SimpleWorker will establish the ActionMailer connection.
-
-Merging Gems
----------------------
-
-This allows you to use any gem you'd like with SimpleWorker. This uses the same syntax as bundler gem files.
-
-    # merge latest version of the gem
-    merge_gem "some_gem"
-    # or specify specific version
-    merge_gem "some_gem_with_version", "1.2.3"
-    # or if gem has poor naming scheme
-    merge_gem 'mongoid_i18n', :require => 'mongoid/i18n'
-    # of if gem requires other directories outside lib
-    merge_gem 'prawn', :include_dirs=>['data']
-
-
-[Check here for more info on merge_gem](http://support.simpleworker.com/kb/working-with-simpleworker/merging-gems-into-your-worker).
-
-
-Job Timeout
---------------
-
-By default, each job has 60 minutes (3600 seconds to complete). If you know that your job should take less than that, you can specify a timeout explicitly:
-
-    worker.queue(:timeout=>1800)
-
-This will kill your job if it is running more than 1800 seconds, or half an hour.
-
-
-
-Global Merging
---------------
-
-If you want to merge items for all of your workers, you can do merges in your SimpleWorker.configure block:
-
-   config.merge 'my file'
-   config.merge_gem 'httparty'
-
-
-
-Configuration Options
----------------------
-
-### Global Attributes
-
-These are attributes that can be set as part of your config block then will be set on
-all your worker objects automatically. This is particularly good for things like database
-connection info or things that you would need to use across the board.
-
-Eg:
-
-    config.global_attributes[:db_user] = "sa"
-    config.global_attributes[:db_pass] = "pass"
-
-Then in your worker, you would have the attributes defined:
-
-    attr_accessor :db_user, :db_pass
+Full Documentation
+-----------------
 
+Now that you've got your first worker running, be sure to [check out the full documentation](http://docs.simpleworker.com).
+SimpleWorker can do so much more!
 
-Development of the Gem
+Discussion Group
 ----------------------
 
 Join the discussion group at: https://groups.google.com/forum/?hl=en#!forum/simple_worker

data/lib/simple_worker/api.rb

@@ -1,5 +1,5 @@
 require 'rest_client'
-require 'patron'
+require 'typhoeus'
 
 module SimpleWorker
 
@@ -56,14 +56,6 @@ module SimpleWorker
 
         @base_url = "#{@scheme}://#{@host}:#{@port}/#{@version}"
 
-        sess = Patron::Session.new
-        sess.timeout = 10
-        sess.base_url = @base_url
-        sess.headers['User-Agent'] = 'IronMQ Ruby Client'
-        #sess.enable_debug "/tmp/patron.debug"
-        @http_sess = sess
-
-
       end
 
 
@@ -76,13 +68,21 @@ module SimpleWorker
         "/#{command_path}"
       end
 
-      # Used for RestClient ones that haven't been converted to patron yet
-       def url_full(command_path)
+      def url_full(command_path)
         url = "#{base_url}/#{command_path}"
         # @logger.debug "url: " + url.to_s
         url
       end
 
+
+    def common_req_hash
+      {
+          :headers=>{"Content-Type" => 'application/json',
+                     "Authorization"=>"OAuth #{@token}",
+                     "User-Agent"=>"SimpleWorker Ruby Client"}
+      }
+    end
+
       def process_ex(ex)
         logger.error "EX #{ex.class.name}: #{ex.message}"
         body = ex.http_body
@@ -94,9 +94,18 @@ module SimpleWorker
         raise exception
       end
 
+
       def check_response(response, options={})
-        status = response.status
+        # response.code    # http status code
+        #response.time    # time in seconds the request took
+        #response.headers # the http headers
+        #response.headers_hash # http headers put into a hash
+        #response.body    # the response body
+        status = response.code
         body = response.body
+        # todo: check content-type == application/json before parsing
+        logger.debug "response code=" + status.to_s
+        logger.debug "response body=" + body.inspect
         res = nil
         unless options[:parse] == false
           res = JSON.parse(body)
@@ -110,13 +119,16 @@ module SimpleWorker
       end
 
       def get(method, params={}, options={})
-        full_url = url(method)
-        all_params = add_params(method, params)
-        url_plus_params = append_params(full_url, all_params)
-        logger.debug 'get url=' + url_plus_params
-        response = @http_sess.request(:get, url_plus_params,
-                                      {},
-                                      {})
+        full_url = url_full(method)
+        #all_params = add_params(method, params)
+        #url_plus_params = append_params(full_url, all_params)
+        logger.debug 'get url=' + full_url
+        req_hash = common_req_hash
+        req_hash[:params] = params
+        response = Typhoeus::Request.get(full_url, req_hash) # could let typhoeus add params, using :params=>x
+        #response = @http_sess.request(:get, url_plus_params,
+        #                              {},
+        #                              {})
         check_response(response, options)
         body = response.body
         parse_response(body, options)
@@ -131,7 +143,7 @@ module SimpleWorker
           logger.debug "data = " + data
           logger.debug "params = " + params.inspect
           logger.debug "options = " + options.inspect
-          # todo: replace with patron
+          # todo: replace with typhoeus
           parse_response(RestClient.post(append_params(url, add_params(method, params)), {:data => data, :file => file}, :content_type => 'application/json'), options)
         rescue RestClient::Exception => ex
           process_ex(ex)
@@ -144,11 +156,14 @@ module SimpleWorker
         logger.debug "params.payload = " + params[:payload].inspect
         logger.debug "token = "+ token.inspect
         begin
-          url = url(method) + "?oauth=" + token
+          url = url_full(method)
           logger.debug 'post url=' + url
           json = add_params(method, params).to_json
           logger.debug 'body=' + json
-          response = @http_sess.post(url, json, {"Content-Type" => 'application/json'})
+          req_hash = common_req_hash
+          req_hash[:body] = json
+          response = Typhoeus::Request.post(url, req_hash)
+          #response = @http_sess.post(url, json, {"Content-Type" => 'application/json'})
           check_response(response)
           logger.debug 'response: ' + response.inspect
           body = response.body
@@ -156,7 +171,7 @@ module SimpleWorker
         rescue SimpleWorker::RequestError => ex
           # let it throw, came from check_response
           raise ex
-        rescue Exception => ex
+        rescue RestClient::Exception => ex
           logger.warn("Exception in post! #{ex.message}")
           logger.warn(ex.backtrace.join("\n"))
           process_ex(ex)
@@ -166,7 +181,7 @@ module SimpleWorker
 
       def put(method, body, options={})
         begin
-          # todo: replace with patron
+          # todo: replace with typhoeus
           parse_response RestClient.put(url_full(method), add_params(method, body).to_json, headers), options
         rescue RestClient::Exception => ex
           process_ex(ex)
@@ -175,7 +190,7 @@ module SimpleWorker
 
       def delete(method, params={}, options={})
         begin
-          # todo: replace with patron
+          # todo: replace with typhoeus
           parse_response RestClient.delete(append_params(url_full(method), add_params(method, params))), options
         rescue RestClient::Exception => ex
           process_ex(ex)

data/lib/simple_worker/service.rb

@@ -167,6 +167,17 @@ require 'json'
           end
         end
 
+            File.open(File.join(File.dirname(__FILE__), 'server', 'overrides.rb'), 'r') do |fr|
+          while line = fr.gets
+            f.write line
+          end
+        end
+
+        # Now we must disable queuing while loading up classes. This is from the overrides.rb file
+f.write("
+SimpleWorker.disable_queueing()
+")
+
 
         File.open(File.join(File.dirname(__FILE__), "server", 'runner.rb'), 'r') do |fr|
           while line = fr.gets
@@ -210,13 +221,7 @@ end
           f.write "init_mailer(sw_config)\n"
           f.write "ActionMailer::Base.prepend_view_path('templates')\n"
         end
-        f.write "init_database_connection(sw_config)"
-
-        File.open(File.join(File.dirname(__FILE__), 'server', 'overrides.rb'), 'r') do |fr|
-          while line = fr.gets
-            f.write line
-          end
-        end
+        f.write "init_database_connection(sw_config)\n"
 
         merged.each_pair do |k, v|
           if v[:extname] == ".rb"
@@ -228,14 +233,15 @@ end
         end
         #end
         #f.write File.open(filename, 'r') { |mo| mo.read }
-        f.write("require_relative '#{File.basename(filename)}'")
+        f.write("require_relative '#{File.basename(filename)}'\n")
 
         f.write("
-  SimpleWorker.disable_queueing()
   runner_class = get_class_to_run(job_data['class_name'])
   SimpleWorker.running_class = runner_class
   runner = init_runner(runner_class, job_data, dirname, task_id)
   init_worker_service_for_runner(job_data)
+
+  # Now reenable after loading
   SimpleWorker.enable_queueing()
 
 # Let's run it!

metadata

@@ -2,7 +2,7 @@
 name: simple_worker
 version: !ruby/object:Gem::Version 
   prerelease: 6
-  version: 2.0.0.beta.16
+  version: 2.0.0.beta.17
 platform: ruby
 authors: 
 - Travis Reeder
@@ -10,7 +10,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-11-02 00:00:00 Z
+date: 2011-11-03 00:00:00 Z
 dependencies: 
 - !ruby/object:Gem::Dependency 
   name: zip
@@ -35,7 +35,7 @@ dependencies:
   type: :runtime
   version_requirements: *id002
 - !ruby/object:Gem::Dependency 
-  name: patron
+  name: typhoeus
   prerelease: false
   requirement: &id003 !ruby/object:Gem::Requirement 
     none: false