iron_worker 3.0.1 → 3.0.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 8cee19dfeb2c3026c1fe70d47d8b2adf567e4a7e
-  data.tar.gz: c3119b4cadb57d527e7913a05f36f28315e47e00
+  metadata.gz: 810932e73c25acb5793479cc87e1a8a15c4a544a
+  data.tar.gz: a88f5046897d4b62de76c9e8e8621af28fd81230
 SHA512:
-  metadata.gz: 4aa32364026d381b3da802b085f0fc072b97ae30c2ed3946974c168c195a605719cf2821d4f25767084eab61d37e495545997af8cf1ba51c0ea4d704436ae934
-  data.tar.gz: 53fe227506bbfe3df92a1d0f40053f2c12cb1bf4e6ce040acf600e51cf56a60115f2d4f4ae24241bbe89542999a00278cf6e8ac9756a875abd450a7a1e4db1b8
+  metadata.gz: 5ebe5ec38e723ecd40ac2cc51c0f0a5d83e9cb5097117ba74a0e5cb51e894492556dbce799bcae355213450450d5ba51d8ee644d73e39de3f058dea451c50885
+  data.tar.gz: c17994e5fc634ff370a193fff2932d537dc989c37f183a6c1d61fc6f92fc32cd7d95c0cab073ed2a039163c39e413a61f18244e4ac222d910b835ca07a17d1ed

data/README.md

@@ -6,149 +6,41 @@ To run your code in cloud you need to do three things:
 - **Upload code package**
 - **Queue or schedule tasks** for execution
 
-TODO: Link to dockerworker.
+You can read how to create, package and upload your worker here: http://dev.iron.io/worker/getting_started/
 
-While you can use [REST APIs](http://dev.iron.io/worker/reference/api) for that, it's easier to use an
-IronWorker library created specifically for your language of choice, such as this gem, IronWorker.
+This gem has two parts to it, one to access the [IronWorker API[(http://dev.iron.io/worker/reference/api) and
+the other to help you with your Ruby IronWorker's.
 
 # Preparing Your Environment
 
-You'll need to register at http://iron.io/ and get your credentials to use IronWorker. Each account can have an unlimited number of projects, so take advantage of it by creating separate projects for development, testing and production. Each project is identified by a unique project ID and requires your access token before it will perform any action, like uploading or queuing workers.
+You'll need to register at http://iron.io/ and get your credentials to use IronWorker. Each account can have an unlimited number
+ of projects, so take advantage of it by creating separate projects for development, testing and production. 
+ Each project is identified by a unique project ID and requires your access token before it will perform any action, 
+ like uploading or queuing workers.
 
-Also, you'll need a Ruby 1.9 interpreter and the IronWorker gem. Install it using following command.
+Install using the following command.
 
 ```sh
 gem install iron_worker
 ```
 
+# IronWorker Helper Functions
 
-## Queue Up a Task for your Worker
+These functions will help you read in worker payloads and things for when your worker is running. To use these functions
+simple require this gem in your worker and then use the helper functions `IronWorker.payload`, `IronWorker.config` and 
+`IronWorker.id`. For example, this is a complete IronWorker script:
 
-TODO: Drop this, should use new cli tool for queueing.
-
-You can quicky queue up a task for your worker from the command line using:
-
-    iron_worker queue hello
-
-Use the `-p` parameter to pass in a payload:
-
-    iron_worker queue hello -p "{\"hi\": \"world\"}"
-
-Use the `--wait` parameter to queue a task, wait for it to complete and print the log.
-
-    iron_worker queue hello -p "{\"hi\": \"world\"}" --wait
-
-### Queue up a task from code
-
-Most commonly you'll be queuing up tasks from code though, so you can do this:
-
-```ruby
-require "iron_worker_ng"
-client = IronWorker::Client.new
-100.times do
-   client.tasks.create("hello", "foo"=>"bar")
-end
-```
-
-### Setting Task Priority
-
-TODO: Drop this, use new cli tool
-
-You can specify priority of the task using `--priority` parameter:
-
-```ruby
-iron_worker queue hello --priority 0 # default value, lowest priority
-iron_worker queue hello --priority 1 --label 'medium priority task' # medium priority
-```
-
-Value of priority parameter means the priority queue to run the task in. Valid values are 0, 1, and 2. 0 is the default.
-
-From code you can set the priority like it done in snippet below:
-
-```ruby
-client.tasks.create("hello", some_params, priority: 2) # highest priority
-```
-
-### Setting additional Options
-
-
-You can specify not only priority:
-
-  - **priority**: Setting the priority of your job. Valid values are 0, 1, and 2. The default is 0.
-  - **timeout**: The maximum runtime of your task in seconds. No task can exceed 3600 seconds (60 minutes). The default is 3600 but can be set to a shorter duration.
-  - **delay**: The number of seconds to delay before actually queuing the task. Default is 0.
-  - **label**: Optional text label for your task.
-  - **cluster**: cluster name ex: "high-mem" or "dedicated". If not set default is set to "default" which is the public IronWorker cluster.
-
-## Get task status
-
-
-TODO: Drop, use new cli
-
-When you call `iron_worker queue X`, you'll see the task ID in the output which you can use to get the status.
-
-    iron_worker info task 5032f7360a4681382838e082
-
-## Get task log
-
-
-TODO: Drop, use new cli
-
-Similar to getting status, get the task ID in the queue command output, then:
-
-    iron_worker log 5032f7360a4681382838e082 --wait
-
-## Retry a Task
-
-
-TODO: Drop, use new cli
-
-You can retry task by id using same payload and options:
-
-    iron_worker retry 5032f7360a4681382838e082
-
-or
 ```ruby
-client.tasks.retry('5032f7360a4681382838e082', :delay => 10)
-
-## Pause or Resume task processing
-
-
-TODO: Drop, use new cli
+require 'iron_worker'
 
-You can temporarily pause or resume queued and scheduled tasks processing by code name:
-
-    iron_worker pause hello
-
-    iron_worker resume hello
-
-or by code:
-Pause or resume for the code package specified by `code_id`.
-
-```ruby
-response = client.codes.pause_task_queue('1234567890')
-response = client.codes.resume_task_queue('1234567890')
-```
-
-
-### Debugging
-
-To get a bunch of extra output to debug things, turn it on using:
-
-    IronCore::Logger.logger.level = ::Logger::DEBUG
-
-
-# Queue Up Tasks for Your Worker
-
-Now that the code is uploaded, we can create/queue up tasks. You can call this over and over
-for as many tasks as you want.
-
-```ruby
-client.tasks.create('MyWorker', {:client => 'Joe'})
+puts "Here is the payload: #{IronWorker.payload}"
+puts "Here is the config: #{IronWorker.config}"
 ```
 
+# The IronWorker API
 
-# The Rest of the IronWorker API
+This client will enable you to use the IronWorker API in Ruby. 
+Full API documentation is here: http://dev.iron.io/worker/reference/api/
 
 ## IronWorker::Client
 

data/lib/iron_worker.rb

@@ -16,5 +16,6 @@ end
 
 require 'iron_worker/version'
 require 'iron_worker/api_client'
+require 'iron_worker/client'
 require 'iron_worker/worker_helper'
 

data/lib/iron_worker/client.rb

@@ -0,0 +1,461 @@
+require 'ostruct'
+require 'base64'
+require 'tmpdir'
+require 'fileutils'
+
+require 'iron_worker/api_client'
+
+module IronWorker
+  class ClientProxyCaller
+    def initialize(client, prefix)
+      @client = client
+      @prefix = prefix
+    end
+
+    def method_missing(name, *args, &block)
+      full_name = @prefix.to_s + '_' + name.to_s
+      if @client.respond_to?(full_name)
+        @client.send(full_name, *args, &block)
+      else
+        super(name, *args, &block)
+      end
+    end
+  end
+
+  class Client
+    attr_reader :api
+
+    def initialize(options = {}, &block)
+      @api = IronWorker::APIClient.new(options)
+
+      unless block.nil?
+        instance_eval(&block)
+      end
+    end
+
+    def options
+      @api.options
+    end
+
+    def token
+      @api.token
+    end
+
+    def project_id
+      @api.project_id
+    end
+
+    def method_missing(name, *args, &block)
+      if args.length == 0
+        IronWorker::ClientProxyCaller.new(self, name)
+      else
+        super(name, *args, &block)
+      end
+    end
+
+    def stacks_list
+      @api.stacks_list
+    end
+
+
+    def codes_list(options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling codes.list with options='#{options.to_s}'"
+
+      all = options[:all] || options['all']
+
+      if all
+        result = []
+
+        page = options[:page] || options['page'] || 0
+        per_page = options[:per_page] || options['per_page'] || 100
+
+        while true
+          next_codes = codes_list(options.merge({:page => page}).delete_if { |name| name == :all || name == 'all' })
+
+          result += next_codes
+
+          break if next_codes.length != per_page
+          page += 1
+        end
+
+        result
+      else
+        @api.codes_list(options)['codes'].map { |c| OpenStruct.new(c.merge('_id' => c['id'])) }
+      end
+    end
+
+    def codes_get(code_id)
+      IronCore::Logger.debug 'IronWorker', "Calling codes.get with code_id='#{code_id}'"
+
+      c = @api.codes_get(code_id)
+      c['_id'] = c['id']
+      OpenStruct.new(c)
+    end
+
+    def codes_create(code, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling codes.create with code='#{code.to_s}' and options='#{options.to_s}'"
+
+      if options[:config] && options[:config].is_a?(Hash)
+        options = options.dup
+        options[:config] = options[:config].to_json
+      end
+
+      options.merge!(stack:code.stack) if code.stack
+
+      container_file = code.create_container
+
+      if code.zip_package
+        res = nil
+        IronWorker::Fetcher.fetch_to_file(code.zip_package) do |file|
+          res = @api.codes_create(code.name, file, 'sh', '__runner__.sh', options)
+        end
+      elsif code.remote_build_command.nil? && (not code.full_remote_build)
+        res = @api.codes_create(code.name, container_file, 'sh', '__runner__.sh', options)
+      else
+        builder_code_name = code.name + (code.name[0 .. 0].upcase == code.name[0 .. 0] ? '::Builder' : '::builder')
+
+        @api.codes_create(builder_code_name, container_file, 'sh', '__runner__.sh', options)
+
+        builder_task = tasks.create(builder_code_name, :code_name => code.name, :client_options => @api.options.to_json, :codes_create_options => options.to_json)
+
+        builder_task = tasks.wait_for(builder_task._id)
+
+        if builder_task.status != 'complete'
+          log = tasks.log(builder_task._id)
+
+          File.unlink(container_file)
+
+          IronCore::Logger.error 'IronWorker', "Error while remote building worker\n" + log, IronCore::Error
+        end
+
+        res = JSON.parse(builder_task.msg)
+      end
+
+      File.unlink(container_file) if code.zip_package.nil?
+
+      res['_id'] = res['id']
+      OpenStruct.new(res)
+    end
+
+    def codes_create_async(code, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling codes.create_async with code='#{code.to_s}' and options='#{options.to_s}'"
+
+      if options[:config] && options[:config].is_a?(Hash)
+        options = options.dup
+        options[:config] = options[:config].to_json
+      end
+
+      options.merge!(stack:code.stack) if code.stack
+
+      container_file = code.create_container
+
+      if code.remote_build_command.nil? && (not code.full_remote_build)
+        res = @api.codes_create(code.name, container_file, 'sh', '__runner__.sh', options)
+      else
+        builder_code_name = code.name + (code.name[0 .. 0].upcase == code.name[0 .. 0] ? '::Builder' : '::builder')
+
+        @api.codes_create(builder_code_name, container_file, 'sh', '__runner__.sh', options)
+
+        builder_task = tasks.create(builder_code_name, :code_name => code.name, :client_options => @api.options.to_json, :codes_create_options => options.to_json)
+
+        File.unlink(container_file)
+
+        return builder_task._id
+      end
+
+      File.unlink(container_file)
+
+      res['_id'] = res['id']
+      OpenStruct.new(res)
+    end
+
+    def codes_patch(name, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling codes.patch with name='#{name}' and options='#{options.to_s}'"
+
+      code = codes.list(per_page: 100).find { |c| c.name == name }
+
+      if code.nil?
+        IronCore::Logger.error 'IronWorker', "Can't find code with name='#{name}' to patch", IronCore::Error
+      end
+
+      patcher_code_name = name + (name[0 .. 0].upcase == name[0 .. 0] ? '::Patcher' : '::patcher')
+
+      exec_dir = ::Dir.tmpdir + '/' + ::Dir::Tmpname.make_tmpname('iron-worker-ng-', 'exec')
+      exec_file_name = exec_dir + '/patchcer.rb'
+
+      FileUtils.mkdir_p(exec_dir)
+
+      exec_file = File.open(exec_file_name, 'w')
+      exec_file.write <<EXEC_FILE
+# #{IronWorker.full_version}
+
+File.open('.gemrc', 'w') do |gemrc|
+  gemrc.puts('gem: --no-ri --no-rdoc')
+end
+
+`gem install iron_worker_ng`
+
+require 'iron_worker_ng'
+
+client = IronWorker::Client.new(JSON.parse(params[:client_options]))
+
+original_code = client.codes.get(params[:code_id])
+original_code_data = client.codes.download(params[:code_id])
+
+`mkdir code`
+original_code_zip = File.open('code/code.zip', 'w')
+original_code_zip.write(original_code_data)
+original_code_zip.close
+`cd code && unzip code.zip && rm code.zip && cd ..`
+
+patch_params = JSON.parse(params[:patch])
+patch_params.each {|k, v| system("cat patch/\#{k} > code/\#{v}")}
+
+code_container = IronWorker::Code::Container::Zip.new
+
+Dir['code/*'].each do |entry|
+  code_container.add(entry[5 .. -1], entry)
+end
+
+code_container.close
+
+res = client.api.codes_create(original_code.name, code_container.name, 'sh', '__runner__.sh', :config => original_code.config)
+
+res['_id'] = res['id']
+res = OpenStruct.new(res)
+
+client.tasks.set_progress(iron_task_id, :msg => res.marshal_dump.to_json)
+EXEC_FILE
+      exec_file.close
+
+      patcher_code = IronWorker::Code::Base.new
+      patcher_code.runtime = :ruby
+      patcher_code.name = patcher_code_name
+      patcher_code.exec(exec_file_name)
+      options[:patch].keys.each {|v| patcher_code.file(v, 'patch')}
+      patch_params = Hash[options[:patch].map {|k,v| [File.basename(k), v]}]
+
+      patcher_container_file = patcher_code.create_container
+
+      @api.codes_create(patcher_code_name, patcher_container_file, 'sh', '__runner__.sh', {})
+
+      FileUtils.rm_rf(exec_dir)
+      File.unlink(patcher_container_file)
+
+      patcher_task = tasks.create(patcher_code_name, :code_id => code._id, :client_options => @api.options.to_json, patch: patch_params.to_json)
+      patcher_task = tasks.wait_for(patcher_task._id)
+
+      if patcher_task.status != 'complete'
+        log = tasks.log(patcher_task._id)
+        IronCore::Logger.error 'IronWorker', "Error while patching worker\n" + log, IronCore::Error
+      end
+
+      res = JSON.parse(patcher_task.msg)
+      res['_id'] = res['id']
+      OpenStruct.new(res)
+    end
+
+    def codes_delete(code_id)
+      IronCore::Logger.debug 'IronWorker', "Calling codes.delete with code_id='#{code_id}'"
+
+      @api.codes_delete(code_id)
+
+      true
+    end
+
+    def codes_revisions(code_id, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling codes.revisions with code_id='#{code_id}' and options='#{options.to_s}'"
+
+      @api.codes_revisions(code_id, options)['revisions'].map { |c| OpenStruct.new(c.merge('_id' => c['id'])) }
+    end
+
+    def codes_download(code_id, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling codes.download with code_id='#{code_id}' and options='#{options.to_s}'"
+
+      @api.codes_download(code_id, options)
+    end
+
+    def codes_pause_task_queue(code_id, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling codes.pause_task_queue with code_id='#{code_id}' and options='#{options.to_s}'"
+
+      res = @api.codes_pause_task_queue(code_id, options)
+      OpenStruct.new(res)
+    end
+
+    def codes_resume_task_queue(code_id, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling codes.resume_task_queue with code_id='#{code_id}' and options='#{options.to_s}'"
+
+      res = @api.codes_resume_task_queue(code_id, options)
+      OpenStruct.new(res)
+    end
+
+    def tasks_list(options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.list with options='#{options.to_s}'"
+
+      @api.tasks_list(options)['tasks'].map { |t| OpenStruct.new(t.merge('_id' => t['id'])) }
+    end
+
+    def tasks_get(task_id)
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.get with task_id='#{task_id}'"
+
+      t = @api.tasks_get(task_id)
+      t['_id'] = t['id']
+      OpenStruct.new(t)
+    end
+
+    def tasks_create(code_name, params = {}, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.create with code_name='#{code_name}', params='#{params.to_s}' and options='#{options.to_s}'"
+
+      res = @api.tasks_create(code_name, params.is_a?(String) ? params : params.to_json, options)
+
+      t = res['tasks'][0]
+      t['_id'] = t['id']
+      OpenStruct.new(t)
+    end
+
+    def tasks_create_legacy(code_name, params = {}, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.create_legacy with code_name='#{code_name}', params='#{params.to_s}' and options='#{options.to_s}'"
+
+      res = @api.tasks_create(code_name, params_for_legacy(code_name, params), options)
+
+      t = res['tasks'][0]
+      t['_id'] = t['id']
+      OpenStruct.new(t)
+    end
+
+    def tasks_cancel(task_id)
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.cancel with task_id='#{task_id}'"
+
+      @api.tasks_cancel(task_id)
+
+      true
+    end
+
+    def tasks_cancel_all(code_id)
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.cancel_all with code_id='#{code_id}'"
+
+      @api.tasks_cancel_all(code_id)
+
+      true
+    end
+
+    def tasks_log(task_id)
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.log with task_id='#{task_id}'"
+
+      if block_given?
+        @api.tasks_log(task_id) { |chunk| yield(chunk) }
+      else
+        @api.tasks_log(task_id)
+      end
+    end
+
+    def tasks_set_progress(task_id, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.set_progress with task_id='#{task_id}' and options='#{options.to_s}'"
+
+      @api.tasks_set_progress(task_id, options)
+
+      true
+    end
+
+    def tasks_wait_for(task_id, options = {}, &block)
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.wait_for with task_id='#{task_id}' and options='#{options.to_s}'"
+
+      options[:sleep] ||= options['sleep'] || 5
+
+      task = tasks_get(task_id)
+
+      while task.status == 'queued' || task.status == 'running'
+        block.call(task) unless block.nil?
+        sleep options[:sleep]
+        task = tasks_get(task_id)
+      end
+
+      task
+    end
+
+    def tasks_retry(task_id, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling tasks.retry with task_id='#{task_id}' and options='#{options.to_s}'"
+
+      res = @api.tasks_retry(task_id, options)
+
+      t = res['tasks'][0]
+      t['_id'] = t['id']
+      OpenStruct.new(t)
+    end
+
+    def schedules_list(options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling schedules.list with options='#{options.to_s}'"
+
+      @api.schedules_list(options)['schedules'].map { |s| OpenStruct.new(s.merge('_id' => s['id'])) }
+    end
+
+    def schedules_get(schedule_id)
+      IronCore::Logger.debug 'IronWorker', "Calling schedules.get with schedule_id='#{schedule_id}"
+
+      s = @api.schedules_get(schedule_id)
+      s['_id'] = s['id']
+      OpenStruct.new(s)
+    end
+
+    def schedules_create(code_name, params = {}, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling schedules.create with code_name='#{code_name}', params='#{params.to_s}' and options='#{options.to_s}'"
+
+      res = @api.schedules_create(code_name, params.is_a?(String) ? params : params.to_json, options)
+
+      s = res['schedules'][0]
+      s['_id'] = s['id']
+      OpenStruct.new(s)
+    end
+
+    def schedules_update(id, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling schedules.update with id='#{id}', options='#{options.to_s}'"
+
+      res = @api.schedules_update(id, options)
+
+      OpenStruct.new(res)
+    end
+
+    def schedules_create_legacy(code_name, params = {}, options = {})
+      IronCore::Logger.debug 'IronWorker', "Calling schedules.create_legacy with code_name='#{code_name}', params='#{params.to_s}' and options='#{options.to_s}'"
+
+      res = @api.schedules_create(code_name, params_for_legacy(code_name, params), options)
+
+      s = res['schedules'][0]
+      s['_id'] = s['id']
+      OpenStruct.new(s)
+    end
+
+    def schedules_cancel(schedule_id)
+      IronCore::Logger.debug 'IronWorker', "Calling schedules.cancel with schedule_id='#{schedule_id}"
+
+      @api.schedules_cancel(schedule_id)
+
+      true
+    end
+
+    def projects_get
+      IronCore::Logger.debug 'IronWorker', "Calling projects.get"
+
+      res = @api.projects_get
+
+      res['_id'] = res['id']
+      OpenStruct.new(res)
+    end
+
+    def params_for_legacy(code_name, params)
+      if params.is_a?(String)
+        params = JSON.parse(params)
+      end
+
+      attrs = {}
+
+      params.keys.each do |k|
+        attrs['@' + k.to_s] = params[k]
+      end
+
+      attrs = attrs.to_json
+
+      {:class_name => code_name, :attr_encoded => Base64.encode64(attrs), :sw_config => {:project_id => project_id, :token => token}}.to_json
+    end
+  end
+end

data/lib/iron_worker/version.rb

@@ -1,5 +1,5 @@
 module IronWorker
-  VERSION = "3.0.1"
+  VERSION = "3.0.2"
 
   def self.version
     VERSION

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: iron_worker
 version: !ruby/object:Gem::Version
-  version: 3.0.1
+  version: 3.0.2
 platform: ruby
 authors:
 - Travis Reeder
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2015-03-11 00:00:00.000000000 Z
+date: 2015-03-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: iron_core
@@ -52,6 +52,7 @@ files:
 - iron_worker.gemspec
 - lib/iron_worker.rb
 - lib/iron_worker/api_client.rb
+- lib/iron_worker/client.rb
 - lib/iron_worker/version.rb
 - lib/iron_worker/worker_helper.rb
 homepage: https://github.com/iron-io/iron_worker_ruby