ThiagoLelis-backgroundjob 1.0.3 → 1.0.4

data/HISTORY

@@ -1,64 +1,9 @@
-  1.0.1:
-    - fixed name collision with 'record.attributes = hash' ar mass
-      assignment method (thx jon guymon)
-    - added new sponsor: http://igicom.com/
+2009-01-30:
+  - fixed find(:first, :lock => true) issue with oracle databases
+  - converted clob (:text) columns to varchar (:string) when creating Bj tables (a bug in the oracle-adapter prevented the clobs from saving properly)
 
-  0.0.5:
-    - use full path to ruby for plugin mode
-    - plugin correctly installs bin -->> script
-    - plugin install uses --force
-    - properly quote paths in windows (spaces)
-    - switch win signal to ABRT (was INT)
-    - background job regrestration now uses ppid to pin the subprocess to a
-      parent
-    - use ppid to detect parent death and exit in event loop 
-    - don't use gem dependanices in plugin as they are broken when loading from
-      muliple gem repos
-    - added a small amount of drb magic that allows signals to work across
-      processes even on windows (see http://drawohara.com/post/22540307)
-  
-0.0.4:
-  - basic functionality in windows
-  - several small bug fixes
+2008-09-12:
+  - brought the plugin up to 1.0.4 with the new utc option (Brandon Arbini)
 
-0.0.3:
-  - *many* small bug fixes
-
-  - plugin install should now pick up dependancies from plugin dir, last
-    release had LOAD_PATH/gem issues and was picking up globally installed
-    gems
-
-  - automatic management of the background processing can be turned off if you
-    want to manage your own processes 
-
-  - all jobs are automatically restartable unless submitted with 
-
-       :restartable => false
-
-    this means that, should a runner ever die, upon restart any jobs that were
-    mid-process will automatically be restarted
-
-  - signal based parent lifeline move out of thread and into even loop
-
-  - :lock => true added to a few AR finds to support true write serializable
-    transaction isolation when the db supports it
-
-  - all migrations now use :force => true and
-
-  - running 'bj setup' will always generate a new migration, even if you've
-    already run it before.  this allows easy version upgrades.
-
-  - a few command would blow up on windows because they weren't prefixed with
-    'ruby'.  gotta love the lack of #shebang line on windoze...
-
-  - ./script/bj is searched first before system path env var
-
-  - a default PATH is provided for whacky systems without one
-
-  - database.yml is filtered through ERB ala rails
-  
-0.0.2:
-  - path bug fixes
-
-0.0.1:
-  - initial release
+2007-12-18T14:19:22.86-07:00
+  - corrected install libs

data/README

@@ -1,258 +1,100 @@
 NAME
-  BackgroundJob
+  bj
 
 SYNOPSIS
-  BackgroundJob (migration_code|generate_migration|migrate|setup|plugin|run|submit|list|set|config|pid) [options]+
+  bj (migration_code|generate_migration|migrate|setup|run|submit|list|set|config|pid) [options]+
 
 DESCRIPTION
   ________________________________
   Overview
   --------------------------------
   
-  Backgroundjob (Bj) is a brain dead simple zero admin background priority queue
-  for Rails.  Bj is robust, platform independent (including windows), and
-  supports internal or external manangement of the background runner process.
+    Backgroundjob (Bj) is a simple to use background priority queue for rails.
+    Although not yet tested on windows, the design of bj is such that operation
+    should be possible on any operating system, including M$.
   
-  Jobs can be submitted to the queue directly using the api or from the command
-  line using the ./script/bj:
+    Jobs can be submitted to the queue directly using the api or from the
+    commandline using the 'bj' script.  For example
   
-    api:
-      Bj.submit 'cat /etc/password'
+    code:
+        Bj.submit 'cat /etc/password'
   
-    command line:
-      bj submit cat /etc/password
+      cli:
+        bj submit cat /etc/password
   
-  Bj's priority queue lives in the database and is therefore durable - your jobs
-  will live across an app crash or machine reboot.  The job management is
-  comprehensive capturing stdout, stderr, exit_status, and temporal statistics
-  about each job:
+    When used from inside a rails application bj arranges that another process
+    will always be running in the background to process the jobs that you submit.
+    By using a separate process to run jobs bj does not impact the resource
+    utilization of your rails application at all and enables several very cool
+    features:
   
-    jobs = Bj.submit array_of_commands, :priority => 42
+      1) Bj allows you to sumbit jobs to any of your configured databases and,
+      in each case, spawns a separate background process to run jobs from that
+      queue
   
-    ...
-  
-    jobs.each do |job|
-      if job.finished?
-        p job.stdout
-        p job.stderr
-        p job.exit_status
-        p job.started_at
-        p job.finished_at
-      end
-    end
-  
-  In addition the background runner process logs all commands run and their
-  exit_status to a log named using the following convention:
-  
-    rails_root/log/bj.#{ HOSTNAME }.#{ RAILS_ENV }.log
-  
-  Bj allows you to submit jobs to multiple databases; for instance, if your
-  application is running in development mode you may do:
-  
-    Bj.in :production do
-      Bj.submit 'my_job.exe'
-    end
-  
-  Bj manages the ever growing list of jobs ran by automatically archiving them
-  into another table (by default jobs > 24 hrs old are archived) to prevent the
-  jobs table from becoming bloated and huge.
-  
-  All Bj's tables are namespaced and accessible via the Bj module:
-  
-    Bj.table.job.find(:all)         # jobs table
-    Bj.table.job_archive.find(:all) # archived jobs
-    Bj.table.config.find(:all)      # configuration and runner state
-  
-  Bj always arranges for submitted jobs to run with a current working directory
-  of RAILS_ROOT and with the correct RAILS_ENV setting.  For example, if you
-  submit a job in production it will have ENV['RAILS_ENV'] == 'production'.
-  
-  When Bj manages the background runner it will never outlive the rails
-  application - it is started and stopped on demand as the rails app is started
-  and stopped.  This is also true for ./script/console - Bj will automatically
-  fire off the background runner to process jobs submitted using the console.
-  
-  Bj ensures that only one background process is running for your application -
-  firing up three mongrels or fcgi processes will result in only one background
-  runner being started.  Note that the number of background runners does not
-  determine throughput - that is determined primarily by the nature of the jobs
-  themselves and how much work they perform per process.
-  
-  
-  ________________________________
-  Architecture
-  --------------------------------
-  
-  If one ignores platform specific details the design of Bj is quite simple: the
-  main Rails application submits jobs to table, stored in the database.  The act
-  of submitting triggers exactly one of two things to occur:
-  
-    1) a new long running background runner to be started
-  
-    2) an existing background runner to be signaled
-  
-  The background runner refuses to run two copies of itself for a given
-  hostname/rails_env combination.  For example you may only have one background
-  runner processing jobs on localhost in development mode.
-  
-  The background runner, under normal circumstances, is managed by Bj itself -
-  you need do nothing to start, monitor, or stop it - it just works.  However,
-  some people will prefer manage their own background process, see 'External
-  Runner' section below for more on this.
-  
-  The runner simply processes each job in a highest priority oldest-in fashion,
-  capturing stdout, stderr, exit_status, etc. and storing the information back
-  into the database while logging it's actions.  When there are no jobs to run
-  the runner goes to sleep for 42 seconds; however this sleep is interuptable,
-  such as when the runner is signaled that a new job has been submitted so,
-  under normal circumstances there will be zero lag between job submission and
-  job running for an empty queue.
-  
-  
-  ________________________________
-  External Runner / Clustering
-  --------------------------------
-  
-  For the paranoid control freaks out there (myself included) it is quite
-  possible to manage and monitor the runner process manually.  This can be
-  desirable in production setups where monitoring software may kill leaking
-  rails apps periodically.
-  
-  Recalling that Bj will only allow one copy of itself to process jobs per
-  hostname/rails_env pair we can simply do something like this in cron
-  
-    cmd = bj run --forever \
-                 --rails_env=development \
-                 --rails_root=/Users/ahoward/rails_root
-  
-    */15 * * * * $cmd 
-  
-  this will simply attempt the start the background runner every 15 minutes if,
-  and only if, it's not *already* running.
-  
-  In addtion to this you'll want to tell Bj not to manage the runner itself
-  using
-  
-    Bj.config["production.no_tickle"] = true
-  
-  Note that, for clusting setups, it's as simple as adding a crontab and config
-  entry like this for each host.  Because Bj throttles background runners per
-  hostname this will allow one runner per hostname - making it quite simple to
-  cluster three nodes behind a besieged rails application.
-  
-  
-  ________________________________
-  Designing Jobs
-  --------------------------------
-  
-  Bj runs it's jobs as command line applications.  It ensures that all jobs run
-  in RAILS_ROOT so it's quite natural to apply a pattern such as
-  
-    mkdir ./jobs
-    edit ./jobs/background_job_to_run
-  
-    ...
-  
-    Bj.submit "./jobs/background_job_to_run"
-  
-  If you need to run you jobs under an entire rails environment you'll need to
-  do this:
-  
-    Bj.submit "./script/runner ./jobs/background_job_to_run"
-  
-  Obviously "./script/runner" loads the rails environment for you.  It's worth
-  noting that this happens for each job and that this is by design: the reason
-  is that most rails applications leak memory like a sieve so, if one were to
-  spawn a long running process that used the application code base you'd have a
-  lovely doubling of memory usage on you app servers.  Although loading the
-  rails environment for each background job requires a little time, a little
-  cpu, and a lot less memory.  A future version of Bj will provide a way to load
-  the rails environment once and to process background jobs in this environment,
-  but anyone wanting to use this in production will be required to duct tape
-  their entire chest and have a team of oxen rip off the tape without screaming
-  to prove steelyness of spirit and profound understanding of the other side.
-  
-  Don't forget that you can submit jobs with command line arguments:
-  
-    Bj.submit "./jobs/a.rb 1 foobar --force" 
-  
-  and that you can do powerful things by passing stdin to a job that powers
-  through a list of work.  For instance, assume a "./jobs/bulkmail" job
-  resembling
-  
-    STDIN.each do |line|
-      address = line.strip
-      mail_message_to address
-    end
-    
-  then you could
-  
-    stdin = [
-      "foo@bar.com",
-      "bar@foo.com",
-      "ara.t.howard@codeforpeople.com",
-    ]
-  
-    Bj.submit "./script/runner ./jobs/bulkmail", :stdin => stdin
-  
-  and all those emails would be sent in the background.
+        Bj.in :production do
+          Bj.submit 'production_job.exe'
+        end
   
-  Bj's power is putting jobs in the background in a simple and robust fashion.
-  It's your task to build intelligent jobs that leverage batch processing, and
-  other, possibilities.  The upshot of building tasks this way is that they are
-  quite easy to test before submitting them from inside your application.
+        Bj.in :development do
+          Bj.submit 'development_job.exe'
+        end
   
+      2) Although bj ensures that a process is always running to process
+      your jobs, you can start a proces manually.  This means that any machine
+      capable of seeing your RAILS_ROOT can run jobs for your application, allowing
+      one to setup a cluster of machines doing the work of a single front end rails
+      applicaiton.
   
   ________________________________
   Install 
   --------------------------------
     
-    Bj can be installed two ways: as a plugin or via rubygems
-  
-      plugin:
-        1) ./script/plugin install http://codeforpeople.rubyforge.org/svn/rails/plugins/bj
-        2) ./script/bj setup
+    Bj can be installed two ways: as a gem or as a plugin.
   
       gem:
         1) $sudo gem install bj
         2) add "require 'bj'" to config/environment.rb
         3) bj setup
   
+      plugin:
+        1) ./script/plugin install http://codeforpeople.rubyforge.org/svn/rails/plugins/bj
+        2) ./script/bj setup
+  
   ________________________________
   Api
   --------------------------------
   
-  submit jobs for background processing.  'jobs' can be a string or array of
-  strings.  options are applied to each job in the 'jobs', and the list of
-  submitted jobs is always returned.  options (string or symbol) can be
+    submit jobs for background processing.  'jobs' can be a string or array of
+    strings.  options are applied to each job in the 'jobs', and the list of
+    submitted jobs is always returned.  options (string or symbol) can be
     
-    :rails_env => production|development|key_in_database_yml 
-                  when given this keyword causes bj to submit jobs to the
-                  specified database.  default is RAILS_ENV.
-  
-    :priority => any number, including negative ones.  default is zero.
-  
-    :tag => a tag added to the job.  simply makes searching easier.
-  
-    :env => a hash specifying any additional environment vars the background
-            process should have.
-  
-    :stdin => any stdin the background process should have.  must respond_to
-              to_s
+      :rails_env => production|development|key_in_database_yml 
+                    when given this keyword causes bj to submit jobs to the
+                    specified database.  default is RAILS_ENV.
     
-  eg:
-  
-    jobs = Bj.submit 'echo foobar', :tag => 'simple job'
-  
-    jobs = Bj.submit '/bin/cat', :stdin => 'in the hat', :priority => 42
-  
-    jobs = Bj.submit './script/runner ./scripts/a.rb', :rails_env => 'production'
-  
-    jobs = Bj.submit './script/runner /dev/stdin',
-                     :stdin => 'p RAILS_ENV',
-                     :tag => 'dynamic ruby code'
+      :priority => any number, including negative ones.  default is zero.
+    
+      :tag => a tag added to the job.  simply makes searching easier.
+    
+      :env => a hash specifying any additional environment vars the background
+              process should have.
+    
+      :stdin => any stdin the background process should have.
+    
+    eg:
+    
+      jobs = Bj.submit 'echo foobar', :tag => 'simple job'
+    
+      jobs = Bj.submit '/bin/cat', :stdin => 'in the hat', :priority => 42
+    
+      jobs = Bj.submit './script/runner ./scripts/a.rb', :rails_env => 'production'
+     
+      jobs = Bj.submit './script/runner /dev/stdin',
+                       :stdin => 'p RAILS_ENV',
+                       :tag => 'dynamic ruby code'
   
-    jobs Bj.submit array_of_commands, :priority => 451 
+      jobs Bj.submit array_of_commands, :priority => 451 
     
   when jobs are run, they are run in RAILS_ROOT.  various attributes are
   available *only* once the job has finished.  you can check whether or not a
@@ -277,17 +119,9 @@ DESCRIPTION
   ________________________________
   Sponsors 
   --------------------------------
-    http://quintess.com/
     http://www.engineyard.com/
-    http://igicom.com/
+    http://quintess.com/
     http://eparklabs.com/
-  
-    http://your_company.com/      <<-- (targeted marketing aimed at *you*)
-  
-  ________________________________
-  Version
-  --------------------------------
-  1.0.1
 
 PARAMETERS
   --rails_root=rails_root, -R (0 ~> rails_root=) 

data/Rakefile

@@ -0,0 +1,22 @@
+require 'rake'
+require 'rake/testtask'
+require 'rake/rdoctask'
+
+desc 'Default: run unit tests.'
+task :default => :test
+
+desc 'Test the bj plugin.'
+Rake::TestTask.new(:test) do |t|
+  t.libs << 'lib'
+  t.pattern = 'test/**/*_test.rb'
+  t.verbose = true
+end
+
+desc 'Generate documentation for the bj plugin.'
+Rake::RDocTask.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'Bj'
+  rdoc.options << '--line-numbers' << '--inline-source'
+  rdoc.rdoc_files.include('README')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end

data/background.gemspec

@@ -0,0 +1,18 @@
+Gem::Specification.new do |s|
+  s.name = %q{backgroundjob}
+  s.version = "1.0.4"
+ 
+  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.authors = ["Ara T. Howard"]
+  s.date = %q{2008-12-24}
+  s.default_executable = %q{bin/bj}
+  s.email = %q{ara.t.howard@gmail.com}
+  s.executables = ["bj"]
+  s.files = ["background.gemspec", "bin", "bin/bj", "HISTORY", "init.rb", "install.rb", "lib", "lib/arrayfields.rb", "lib/attributes.rb", "lib/bj", "lib/bj/api.rb", "lib/bj/attributes.rb", "lib/bj/bj.rb", "lib/bj/errors.rb", "lib/bj/joblist.rb", "lib/bj/logger.rb", "lib/bj/runner.rb", "lib/bj/stdext.rb", "lib/bj/table.rb", "lib/bj/util.rb", "lib/bj.rb", "lib/fattr.rb", "lib/main", "lib/main/base.rb", "lib/main/cast.rb", "lib/main/factories.rb", "lib/main/getoptlong.rb", "lib/main/logger.rb", "lib/main/mode.rb", "lib/main/parameter.rb", "lib/main/softspoken.rb", "lib/main/stdext.rb", "lib/main/usage.rb", "lib/main/util.rb", "lib/main.rb", "lib/orderedautohash.rb", "lib/orderedhash.rb", "lib/systemu.rb", "Rakefile", "README", "script", "script/bj", "tasks", "tasks/bj_tasks.rake", "test", "test/bj_test.rb", "uninstall.rb"]
+  s.has_rdoc = false
+  s.homepage = %q{http://codeforpeople.com/lib/ruby/bj}
+  s.require_paths = ["lib"]
+  s.rubygems_version = %q{1.3.1}
+  s.summary = %q{Backgroundjob (Bj) is a simple to use background priority queue for rails.}
+  s.rubyforge_project = ""
+end