bricooke-bj 1.0.2

data/HISTORY

@@ -0,0 +1,74 @@
+  1.0.3
+    - env wasn't properly unpacked by runner, added YAML.load(job.env), thanks
+      Chris Wanstrath  
+    - made operations that generate migrations, etc, verbose - they dump
+      stdout and stderr to console when running
+
+  1.0.2:
+    - Bj now (should) auto detect the correct rake command on windows as
+      "rake.bat" not "rake".  see Bj.which_rake and Bj.rake for impl.
+
+  1.0.1:
+    - fixed name collision with 'record.attributes = hash' ar mass
+      assignment method (thx jon guymon)
+    - added new sponsor: http://igicom.com/
+
+  0.0.5:
+    - use full path to ruby for plugin mode
+    - plugin correctly installs bin -->> script
+    - plugin install uses --force
+    - properly quote paths in windows (spaces)
+    - switch win signal to ABRT (was INT)
+    - background job regrestration now uses ppid to pin the subprocess to a
+      parent
+    - use ppid to detect parent death and exit in event loop 
+    - don't use gem dependanices in plugin as they are broken when loading from
+      muliple gem repos
+    - added a small amount of drb magic that allows signals to work across
+      processes even on windows (see http://drawohara.com/post/22540307)
+  
+0.0.4:
+  - basic functionality in windows
+  - several small bug fixes
+
+0.0.3:
+  - *many* small bug fixes
+
+  - plugin install should now pick up dependancies from plugin dir, last
+    release had LOAD_PATH/gem issues and was picking up globally installed
+    gems
+
+  - automatic management of the background processing can be turned off if you
+    want to manage your own processes 
+
+  - all jobs are automatically restartable unless submitted with 
+
+       :restartable => false
+
+    this means that, should a runner ever die, upon restart any jobs that were
+    mid-process will automatically be restarted
+
+  - signal based parent lifeline move out of thread and into even loop
+
+  - :lock => true added to a few AR finds to support true write serializable
+    transaction isolation when the db supports it
+
+  - all migrations now use :force => true and
+
+  - running 'bj setup' will always generate a new migration, even if you've
+    already run it before.  this allows easy version upgrades.
+
+  - a few command would blow up on windows because they weren't prefixed with
+    'ruby'.  gotta love the lack of #shebang line on windoze...
+
+  - ./script/bj is searched first before system path env var
+
+  - a default PATH is provided for whacky systems without one
+
+  - database.yml is filtered through ERB ala rails
+  
+0.0.2:
+  - path bug fixes
+
+0.0.1:
+  - initial release

data/README

@@ -0,0 +1,308 @@
+NAME
+  bj
+
+SYNOPSIS
+  bj (migration_code|generate_migration|migrate|setup|plugin|run|submit|list|set|config|pid) [options]+
+
+DESCRIPTION
+  ________________________________
+  Overview
+  --------------------------------
+  
+  Backgroundjob (Bj) is a brain dead simple zero admin background priority queue
+  for Rails.  Bj is robust, platform independent (including windows), and
+  supports internal or external manangement of the background runner process.
+  
+  Jobs can be submitted to the queue directly using the api or from the command
+  line using the ./script/bj:
+  
+    api:
+      Bj.submit 'cat /etc/password'
+  
+    command line:
+      bj submit cat /etc/password
+  
+  Bj's priority queue lives in the database and is therefore durable - your jobs
+  will live across an app crash or machine reboot.  The job management is
+  comprehensive capturing stdout, stderr, exit_status, and temporal statistics
+  about each job:
+  
+    jobs = Bj.submit array_of_commands, :priority => 42
+  
+    ...
+  
+    jobs.each do |job|
+      if job.finished?
+        p job.stdout
+        p job.stderr
+        p job.exit_status
+        p job.started_at
+        p job.finished_at
+      end
+    end
+  
+  In addition the background runner process logs all commands run and their
+  exit_status to a log named using the following convention:
+  
+    rails_root/log/bj.#{ HOSTNAME }.#{ RAILS_ENV }.log
+  
+  Bj allows you to submit jobs to multiple databases; for instance, if your
+  application is running in development mode you may do:
+  
+    Bj.in :production do
+      Bj.submit 'my_job.exe'
+    end
+  
+  Bj manages the ever growing list of jobs ran by automatically archiving them
+  into another table (by default jobs > 24 hrs old are archived) to prevent the
+  jobs table from becoming bloated and huge.
+  
+  All Bj's tables are namespaced and accessible via the Bj module:
+  
+    Bj.table.job.find(:all)         # jobs table
+    Bj.table.job_archive.find(:all) # archived jobs
+    Bj.table.config.find(:all)      # configuration and runner state
+  
+  Bj always arranges for submitted jobs to run with a current working directory
+  of RAILS_ROOT and with the correct RAILS_ENV setting.  For example, if you
+  submit a job in production it will have ENV['RAILS_ENV'] == 'production'.
+  
+  When Bj manages the background runner it will never outlive the rails
+  application - it is started and stopped on demand as the rails app is started
+  and stopped.  This is also true for ./script/console - Bj will automatically
+  fire off the background runner to process jobs submitted using the console.
+  
+  Bj ensures that only one background process is running for your application -
+  firing up three mongrels or fcgi processes will result in only one background
+  runner being started.  Note that the number of background runners does not
+  determine throughput - that is determined primarily by the nature of the jobs
+  themselves and how much work they perform per process.
+  
+  
+  ________________________________
+  Architecture
+  --------------------------------
+  
+  If one ignores platform specific details the design of Bj is quite simple: the
+  main Rails application submits jobs to table, stored in the database.  The act
+  of submitting triggers exactly one of two things to occur:
+  
+    1) a new long running background runner to be started
+  
+    2) an existing background runner to be signaled
+  
+  The background runner refuses to run two copies of itself for a given
+  hostname/rails_env combination.  For example you may only have one background
+  runner processing jobs on localhost in development mode.
+  
+  The background runner, under normal circumstances, is managed by Bj itself -
+  you need do nothing to start, monitor, or stop it - it just works.  However,
+  some people will prefer manage their own background process, see 'External
+  Runner' section below for more on this.
+  
+  The runner simply processes each job in a highest priority oldest-in fashion,
+  capturing stdout, stderr, exit_status, etc. and storing the information back
+  into the database while logging it's actions.  When there are no jobs to run
+  the runner goes to sleep for 42 seconds; however this sleep is interuptable,
+  such as when the runner is signaled that a new job has been submitted so,
+  under normal circumstances there will be zero lag between job submission and
+  job running for an empty queue.
+  
+  
+  ________________________________
+  External Runner / Clustering
+  --------------------------------
+  
+  For the paranoid control freaks out there (myself included) it is quite
+  possible to manage and monitor the runner process manually.  This can be
+  desirable in production setups where monitoring software may kill leaking
+  rails apps periodically.
+  
+  Recalling that Bj will only allow one copy of itself to process jobs per
+  hostname/rails_env pair we can simply do something like this in cron
+  
+    cmd = bj run --forever \
+                 --rails_env=development \
+                 --rails_root=/Users/ahoward/rails_root
+  
+    */15 * * * * $cmd 
+  
+  this will simply attempt the start the background runner every 15 minutes if,
+  and only if, it's not *already* running.
+  
+  In addtion to this you'll want to tell Bj not to manage the runner itself
+  using
+  
+    Bj.config["production.no_tickle"] = true
+  
+  Note that, for clusting setups, it's as simple as adding a crontab and config
+  entry like this for each host.  Because Bj throttles background runners per
+  hostname this will allow one runner per hostname - making it quite simple to
+  cluster three nodes behind a besieged rails application.
+  
+  
+  ________________________________
+  Designing Jobs
+  --------------------------------
+  
+  Bj runs it's jobs as command line applications.  It ensures that all jobs run
+  in RAILS_ROOT so it's quite natural to apply a pattern such as
+  
+    mkdir ./jobs
+    edit ./jobs/background_job_to_run
+  
+    ...
+  
+    Bj.submit "./jobs/background_job_to_run"
+  
+  If you need to run you jobs under an entire rails environment you'll need to
+  do this:
+  
+    Bj.submit "./script/runner ./jobs/background_job_to_run"
+  
+  Obviously "./script/runner" loads the rails environment for you.  It's worth
+  noting that this happens for each job and that this is by design: the reason
+  is that most rails applications leak memory like a sieve so, if one were to
+  spawn a long running process that used the application code base you'd have a
+  lovely doubling of memory usage on you app servers.  Although loading the
+  rails environment for each background job requires a little time, a little
+  cpu, and a lot less memory.  A future version of Bj will provide a way to load
+  the rails environment once and to process background jobs in this environment,
+  but anyone wanting to use this in production will be required to duct tape
+  their entire chest and have a team of oxen rip off the tape without screaming
+  to prove steelyness of spirit and profound understanding of the other side.
+  
+  Don't forget that you can submit jobs with command line arguments:
+  
+    Bj.submit "./jobs/a.rb 1 foobar --force" 
+  
+  and that you can do powerful things by passing stdin to a job that powers
+  through a list of work.  For instance, assume a "./jobs/bulkmail" job
+  resembling
+  
+    STDIN.each do |line|
+      address = line.strip
+      mail_message_to address
+    end
+    
+  then you could
+  
+    stdin = [
+      "foo@bar.com",
+      "bar@foo.com",
+      "ara.t.howard@codeforpeople.com",
+    ]
+  
+    Bj.submit "./script/runner ./jobs/bulkmail", :stdin => stdin
+  
+  and all those emails would be sent in the background.
+  
+  Bj's power is putting jobs in the background in a simple and robust fashion.
+  It's your task to build intelligent jobs that leverage batch processing, and
+  other, possibilities.  The upshot of building tasks this way is that they are
+  quite easy to test before submitting them from inside your application.
+  
+  
+  ________________________________
+  Install 
+  --------------------------------
+    
+    Bj can be installed two ways: as a plugin or via rubygems
+  
+      plugin:
+        1) ./script/plugin install http://codeforpeople.rubyforge.org/svn/rails/plugins/bj
+        2) ./script/bj setup
+  
+      gem:
+        1) $sudo gem install bj
+        2) add "require 'bj'" to config/environment.rb
+        3) bj setup
+  
+  ________________________________
+  Api
+  --------------------------------
+  
+  submit jobs for background processing.  'jobs' can be a string or array of
+  strings.  options are applied to each job in the 'jobs', and the list of
+  submitted jobs is always returned.  options (string or symbol) can be
+    
+    :rails_env => production|development|key_in_database_yml 
+                  when given this keyword causes bj to submit jobs to the
+                  specified database.  default is RAILS_ENV.
+  
+    :priority => any number, including negative ones.  default is zero.
+  
+    :tag => a tag added to the job.  simply makes searching easier.
+  
+    :env => a hash specifying any additional environment vars the background
+            process should have.
+  
+    :stdin => any stdin the background process should have.  must respond_to
+              to_s
+    
+  eg:
+  
+    jobs = Bj.submit 'echo foobar', :tag => 'simple job'
+  
+    jobs = Bj.submit '/bin/cat', :stdin => 'in the hat', :priority => 42
+  
+    jobs = Bj.submit './script/runner ./scripts/a.rb', :rails_env => 'production'
+  
+    jobs = Bj.submit './script/runner /dev/stdin',
+                     :stdin => 'p RAILS_ENV',
+                     :tag => 'dynamic ruby code'
+  
+    jobs Bj.submit array_of_commands, :priority => 451 
+    
+  when jobs are run, they are run in RAILS_ROOT.  various attributes are
+  available *only* once the job has finished.  you can check whether or not a
+  job is finished by using the #finished method, which simple does a reload and
+  checks to see if the exit_status is non-nil.
+    
+    eg:
+    
+      jobs = Bj.submit list_of_jobs, :tag => 'important'
+      ...
+      
+      jobs.each do |job|
+        if job.finished?
+          p job.exit_status
+          p job.stdout
+          p job.stderr
+        end
+      end
+  
+  See lib/bj/api.rb for more details.
+  
+  ________________________________
+  Sponsors 
+  --------------------------------
+    http://quintess.com/
+    http://www.engineyard.com/
+    http://igicom.com/
+    http://eparklabs.com/
+  
+    http://your_company.com/      <<-- (targeted marketing aimed at *you*)
+  
+  ________________________________
+  Version
+  --------------------------------
+  1.0.1
+
+PARAMETERS
+  --rails_root=rails_root, -R (0 ~> rails_root=) 
+      the rails_root will be guessed unless you set this 
+  --rails_env=rails_env, -E (0 ~> rails_env=development) 
+      set the rails_env 
+  --log=log, -l (0 ~> log=STDERR) 
+      set the logfile 
+  --help, -h 
+
+AUTHOR
+  ara.t.howard@gmail.com
+
+URIS
+  http://codeforpeople.com/lib/ruby/
+  http://rubyforge.org/projects/codeforpeople/
+  http://codeforpeople.rubyforge.org/svn/rails/plugins/
+

data/TODO

@@ -0,0 +1,40 @@
+
+? the whole gem_path thing is still fubar
+
+? commands need quoting, esp for windows, "c:\Documents And..." etc
+
+- signals not operating properly on windows , non critical error tho...
+
+- need to figure out how to cache connections for Bj.in(...)
+
+- ttl will be added. maxing it out will cause auto-resubmission (Steve Midgley)
+
+- is having the runner thread try forever to start the process the best thing?
+
+- allow easy way to run ruby code.  perhaps ./script/runner 'eval STDIN.read'
+  is good enough
+
+- allow easy way to run ruby code that persists
+
+- allow specification of runner on submit (--runner)
+
+- allow specification of tags a runner will consume (--tag)
+
+- flesh out the cli interface - it's a test only at this point
+
+- test in windows
+
+================================================================================
+
+X ./script/console submission hangs on windows
+X default PATH setting
+X install issues for dave? - gem_path...
+X main only loaded for (bin|script)/bj
+X make it possible to declare externally managed runners
+X restartable will be added. true by default (Steve Midgley)
+X do the lifeline inline with the loop
+X need to address the serialzable writer issue (:lock => true ??)
+X migrations use --force
+X i forget to add "#{ Bj.ruby } ... " to the generate command
+X ./script/bj must be found in path before c:/.....bin/bj
+X make sure database.yml is loaded via YAML::load(ERB.new(File.read * "config/database.yml").result)

data/bin/bj

@@ -0,0 +1,687 @@
+#! /usr/bin/env ruby
+
+require "bj"
+require "main"
+
+Main {
+  usage["description"] = <<-txt
+    ________________________________
+    Overview
+    --------------------------------
+
+    Backgroundjob (Bj) is a brain dead simple zero admin background priority queue
+    for Rails.  Bj is robust, platform independent (including windows), and
+    supports internal or external manangement of the background runner process.
+
+    Jobs can be submitted to the queue directly using the api or from the command
+    line using the ./script/bj:
+
+      api:
+        Bj.submit 'cat /etc/password'
+
+      command line:
+        bj submit cat /etc/password
+
+    Bj's priority queue lives in the database and is therefore durable - your jobs
+    will live across an app crash or machine reboot.  The job management is
+    comprehensive capturing stdout, stderr, exit_status, and temporal statistics
+    about each job:
+
+      jobs = Bj.submit array_of_commands, :priority => 42
+
+      ...
+
+      jobs.each do |job|
+        if job.finished?
+          p job.stdout
+          p job.stderr
+          p job.exit_status
+          p job.started_at
+          p job.finished_at
+        end
+      end
+
+    In addition the background runner process logs all commands run and their
+    exit_status to a log named using the following convention:
+
+      rails_root/log/bj.\#{ HOSTNAME }.\#{ RAILS_ENV }.log
+
+    Bj allows you to submit jobs to multiple databases; for instance, if your
+    application is running in development mode you may do:
+
+      Bj.in :production do
+        Bj.submit 'my_job.exe'
+      end
+
+    Bj manages the ever growing list of jobs ran by automatically archiving them
+    into another table (by default jobs > 24 hrs old are archived) to prevent the
+    jobs table from becoming bloated and huge.
+
+    All Bj's tables are namespaced and accessible via the Bj module:
+
+      Bj.table.job.find(:all)         # jobs table
+      Bj.table.job_archive.find(:all) # archived jobs
+      Bj.table.config.find(:all)      # configuration and runner state
+
+    Bj always arranges for submitted jobs to run with a current working directory
+    of RAILS_ROOT and with the correct RAILS_ENV setting.  For example, if you
+    submit a job in production it will have ENV['RAILS_ENV'] == 'production'.
+
+    When Bj manages the background runner it will never outlive the rails
+    application - it is started and stopped on demand as the rails app is started
+    and stopped.  This is also true for ./script/console - Bj will automatically
+    fire off the background runner to process jobs submitted using the console.
+
+    Bj ensures that only one background process is running for your application -
+    firing up three mongrels or fcgi processes will result in only one background
+    runner being started.  Note that the number of background runners does not
+    determine throughput - that is determined primarily by the nature of the jobs
+    themselves and how much work they perform per process.
+
+
+    ________________________________
+    Architecture
+    --------------------------------
+
+    If one ignores platform specific details the design of Bj is quite simple: the
+    main Rails application submits jobs to table, stored in the database.  The act
+    of submitting triggers exactly one of two things to occur:
+
+      1) a new long running background runner to be started
+
+      2) an existing background runner to be signaled
+
+    The background runner refuses to run two copies of itself for a given
+    hostname/rails_env combination.  For example you may only have one background
+    runner processing jobs on localhost in development mode.
+
+    The background runner, under normal circumstances, is managed by Bj itself -
+    you need do nothing to start, monitor, or stop it - it just works.  However,
+    some people will prefer manage their own background process, see 'External
+    Runner' section below for more on this.
+
+    The runner simply processes each job in a highest priority oldest-in fashion,
+    capturing stdout, stderr, exit_status, etc. and storing the information back
+    into the database while logging it's actions.  When there are no jobs to run
+    the runner goes to sleep for 42 seconds; however this sleep is interuptable,
+    such as when the runner is signaled that a new job has been submitted so,
+    under normal circumstances there will be zero lag between job submission and
+    job running for an empty queue.
+
+
+    ________________________________
+    External Runner / Clustering
+    --------------------------------
+
+    For the paranoid control freaks out there (myself included) it is quite
+    possible to manage and monitor the runner process manually.  This can be
+    desirable in production setups where monitoring software may kill leaking
+    rails apps periodically.
+
+    Recalling that Bj will only allow one copy of itself to process jobs per
+    hostname/rails_env pair we can simply do something like this in cron
+
+      cmd = bj run --forever \\
+                   --rails_env=development \\
+                   --rails_root=/Users/ahoward/rails_root
+
+      */15 * * * * $cmd 
+
+    this will simply attempt the start the background runner every 15 minutes if,
+    and only if, it's not *already* running.
+
+    In addtion to this you'll want to tell Bj not to manage the runner itself
+    using
+
+      Bj.config["production.no_tickle"] = true
+
+    Note that, for clusting setups, it's as simple as adding a crontab and config
+    entry like this for each host.  Because Bj throttles background runners per
+    hostname this will allow one runner per hostname - making it quite simple to
+    cluster three nodes behind a besieged rails application.
+
+
+    ________________________________
+    Designing Jobs
+    --------------------------------
+
+    Bj runs it's jobs as command line applications.  It ensures that all jobs run
+    in RAILS_ROOT so it's quite natural to apply a pattern such as
+
+      mkdir ./jobs
+      edit ./jobs/background_job_to_run
+
+      ...
+
+      Bj.submit "./jobs/background_job_to_run"
+
+    If you need to run you jobs under an entire rails environment you'll need to
+    do this:
+
+      Bj.submit "./script/runner ./jobs/background_job_to_run"
+
+    Obviously "./script/runner" loads the rails environment for you.  It's worth
+    noting that this happens for each job and that this is by design: the reason
+    is that most rails applications leak memory like a sieve so, if one were to
+    spawn a long running process that used the application code base you'd have a
+    lovely doubling of memory usage on you app servers.  Although loading the
+    rails environment for each background job requires a little time, a little
+    cpu, and a lot less memory.  A future version of Bj will provide a way to load
+    the rails environment once and to process background jobs in this environment,
+    but anyone wanting to use this in production will be required to duct tape
+    their entire chest and have a team of oxen rip off the tape without screaming
+    to prove steelyness of spirit and profound understanding of the other side.
+
+    Don't forget that you can submit jobs with command line arguments:
+
+      Bj.submit "./jobs/a.rb 1 foobar --force" 
+
+    and that you can do powerful things by passing stdin to a job that powers
+    through a list of work.  For instance, assume a "./jobs/bulkmail" job
+    resembling
+
+      STDIN.each do |line|
+        address = line.strip
+        mail_message_to address
+      end
+      
+    then you could
+
+      stdin = [
+        "foo@bar.com",
+        "bar@foo.com",
+        "ara.t.howard@codeforpeople.com",
+      ]
+
+      Bj.submit "./script/runner ./jobs/bulkmail", :stdin => stdin
+
+    and all those emails would be sent in the background.
+
+    Bj's power is putting jobs in the background in a simple and robust fashion.
+    It's your task to build intelligent jobs that leverage batch processing, and
+    other, possibilities.  The upshot of building tasks this way is that they are
+    quite easy to test before submitting them from inside your application.
+
+
+    ________________________________
+    Install 
+    --------------------------------
+      
+      Bj can be installed two ways: as a plugin or via rubygems
+
+        plugin:
+          1) ./script/plugin install http://codeforpeople.rubyforge.org/svn/rails/plugins/bj
+          2) ./script/bj setup
+
+        gem:
+          1) $sudo gem install bj
+          2) add "require 'bj'" to config/environment.rb
+          3) bj setup
+
+    ________________________________
+    Api
+    --------------------------------
+
+    submit jobs for background processing.  'jobs' can be a string or array of
+    strings.  options are applied to each job in the 'jobs', and the list of
+    submitted jobs is always returned.  options (string or symbol) can be
+      
+      :rails_env => production|development|key_in_database_yml 
+                    when given this keyword causes bj to submit jobs to the
+                    specified database.  default is RAILS_ENV.
+
+      :priority => any number, including negative ones.  default is zero.
+
+      :tag => a tag added to the job.  simply makes searching easier.
+
+      :env => a hash specifying any additional environment vars the background
+              process should have.
+
+      :stdin => any stdin the background process should have.  must respond_to
+                to_s
+      
+    eg:
+
+      jobs = Bj.submit 'echo foobar', :tag => 'simple job'
+
+      jobs = Bj.submit '/bin/cat', :stdin => 'in the hat', :priority => 42
+
+      jobs = Bj.submit './script/runner ./scripts/a.rb', :rails_env => 'production'
+
+      jobs = Bj.submit './script/runner /dev/stdin',
+                       :stdin => 'p RAILS_ENV',
+                       :tag => 'dynamic ruby code'
+
+      jobs Bj.submit array_of_commands, :priority => 451 
+      
+    when jobs are run, they are run in RAILS_ROOT.  various attributes are
+    available *only* once the job has finished.  you can check whether or not a
+    job is finished by using the #finished method, which simple does a reload and
+    checks to see if the exit_status is non-nil.
+      
+      eg:
+      
+        jobs = Bj.submit list_of_jobs, :tag => 'important'
+        ...
+        
+        jobs.each do |job|
+          if job.finished?
+            p job.exit_status
+            p job.stdout
+            p job.stderr
+          end
+        end
+
+    See lib/bj/api.rb for more details.
+
+    ________________________________
+    Sponsors 
+    --------------------------------
+      http://quintess.com/
+      http://www.engineyard.com/
+      http://igicom.com/
+      http://eparklabs.com/
+
+      http://your_company.com/      <<-- (targeted marketing aimed at *you*)
+
+    ________________________________
+    Version
+    --------------------------------
+    #{ Bj.version }
+txt
+
+  usage["uris"] = <<-txt
+    http://codeforpeople.com/lib/ruby/
+    http://rubyforge.org/projects/codeforpeople/
+    http://codeforpeople.rubyforge.org/svn/rails/plugins/
+  txt
+
+  author "ara.t.howard@gmail.com"
+
+  option("rails_root", "R"){
+    description "the rails_root will be guessed unless you set this"
+    argument_required
+    default RAILS_ROOT
+  }
+
+  option("rails_env", "E"){
+    description "set the rails_env"
+    argument_required
+    default RAILS_ENV
+  }
+
+  option("log", "l"){
+    description "set the logfile"
+    argument_required
+    default STDERR
+  }
+
+
+  mode "migration_code" do
+    description "dump migration code on stdout"
+
+    def run
+      puts Bj.table.migration_code
+    end
+  end
+
+  mode "generate_migration" do
+    description "generate a migration"
+
+    def run
+      Bj.generate_migration
+    end
+  end
+
+  mode "migrate" do
+    description "migrate the db"
+
+    def run
+      Bj.migrate
+    end
+  end
+
+  mode "setup" do
+    description "generate a migration and migrate"
+
+    def run
+      set_rails_env(argv.first) if argv.first
+      Bj.setup
+    end
+  end
+
+  mode "plugin" do
+    description "dump the plugin into rails_root"
+
+    def run
+      Bj.plugin
+    end
+  end
+
+  mode "run" do
+    description "start a job runnner, possibly as a daemon"
+
+    option("--forever"){}
+    option("--ppid"){
+      argument :required
+      cast :integer
+    }
+    option("--wait"){
+      argument :required
+      cast :integer
+    }
+    option("--instance"){
+      argument :required
+      cast :string
+    }
+    option("--limit"){
+      argument :required
+      cast :integer
+    }
+    option("--redirect"){
+      argument :required
+    }
+    option("--daemon"){}
+
+    def run
+      options = {}
+
+=begin
+      %w[ forever ].each do |key|
+        options[key.to_sym] = true if param[key].given?
+      end
+=end
+
+      %w[ forever ppid wait limit instance].each do |key|
+        options[key.to_sym] = param[key].value if param[key].given?
+      end
+
+#p options
+#exit
+      if param["redirect"].given?
+        open(param["redirect"].value, "a+") do |fd|
+          STDERR.reopen fd 
+          STDOUT.reopen fd 
+        end
+        STDERR.sync = true
+        STDOUT.sync = true
+      end
+
+      trap("SIGTERM"){
+        info{ "SIGTERM" }
+        exit
+      }
+      
+      if param["instance"].given?
+        Bj.hostname = Bj.hostname + "." + param["instance"].value
+      end
+   
+      if param["daemon"].given?
+        daemon{ Bj.run options }
+      else
+        Bj.run options 
+      end
+    end
+  end
+
+  mode "submit" do
+    keyword("file"){
+      argument :required
+      attr
+    }
+
+    def run
+      joblist = Bj.joblist.for argv.join(' ') 
+
+      case file
+        when "-"
+          joblist.push(Bj.joblist.jobs_from_io(STDIN))
+        when "--", "---"
+          joblist.push(Bj.joblist.jobs_from_yaml(STDIN))
+        else
+          open(file){|io| joblist.push(Bj.joblist.jobs_from_io(io)) }
+      end
+
+      jobs = Bj.submit joblist, :no_tickle => true
+
+      oh = lambda{|job| OrderedHash["id", job.id, "command", job.command]}
+
+      y jobs.map{|job| oh[job]} 
+    end
+  end
+
+  mode "list" do
+    def run
+      Bj.transaction do
+        y Bj::Table::Job.find(:all).map(&:to_hash)
+      end
+    end
+  end
+
+  mode "set" do
+    argument("key"){ attr }
+
+    argument("value"){ attr }
+
+    option("hostname", "H"){
+      argument :required
+      default Bj.hostname
+      attr
+    }
+
+    option("cast", "c"){
+      argument :required
+      default "to_s" 
+      attr
+    }
+
+    def run
+      Bj.transaction do
+        Bj.config.set(key, value, :hostname => hostname, :cast => cast)
+        y Bj.table.config.for(:hostname => hostname)
+      end
+    end
+  end
+
+  mode "config" do
+    option("hostname", "H"){
+      argument :required
+      default Bj.hostname
+    }
+
+    def run
+      Bj.transaction do
+        y Bj.table.config.for(:hostname => param["hostname"].value)
+      end
+    end
+  end
+
+  mode "pid" do
+    option("hostname", "H"){
+      argument :required
+      default Bj.hostname
+    }
+
+    def run
+      Bj.transaction do
+        config = Bj.table.config.for(:hostname => param["hostname"].value)
+        puts config[ "#{ RAILS_ENV }.pid" ] if config
+      end
+    end
+  end
+
+
+  def run 
+    help!
+  end
+
+  def before_run 
+    self.logger = param["log"].value
+    Bj.logger = logger
+    set_rails_root(param["rails_root"].value) if param["rails_root"].given?
+    set_rails_env(param["rails_env"].value) if param["rails_env"].given?
+  end
+
+  def set_rails_root rails_root
+    ENV["RAILS_ROOT"] = rails_root 
+    ::Object.instance_eval do
+      remove_const :RAILS_ROOT
+      const_set :RAILS_ROOT, rails_root
+    end
+  end
+
+  def set_rails_env rails_env
+    ENV["RAILS_ENV"] = rails_env 
+    ::Object.instance_eval do
+      remove_const :RAILS_ENV
+      const_set :RAILS_ENV, rails_env
+    end
+  end
+
+  def daemon
+    ra, wa = IO.pipe
+    rb, wb = IO.pipe
+    if fork
+      at_exit{ exit! }
+      wa.close
+      r = ra
+      rb.close
+      w = wb
+      pid = r.gets
+      w.puts pid
+      Integer pid.strip
+    else
+      ra.close
+      w = wa
+      wb.close
+      r = rb
+      open("/dev/null", "r+") do |fd|
+        STDIN.reopen fd 
+        STDOUT.reopen fd 
+        STDERR.reopen fd 
+      end
+      Process::setsid rescue nil
+      pid =
+        fork do
+          Dir::chdir RAILS_ROOT
+          File::umask 0
+          $DAEMON = true
+          yield
+          exit!
+        end
+      w.puts pid
+      r.gets
+      exit!
+    end
+  end
+}
+
+
+
+
+
+#
+# we setup a few things so the script works regardless of whether it was
+# called out of /usr/local/bin, ./script, or wherever.  note that the script
+# does *not* require the entire rails application to be loaded into memory!
+# we could just load boot.rb and environment.rb, but this method let's
+# submitting and running jobs be infinitely more lightweight.
+#
+
+BEGIN {
+#
+# see if we're running out of RAILS_ROOT/script/
+#
+unless defined?(BJ_SCRIPT)
+  BJ_SCRIPT =
+    if %w[ script config app ].map{|d| test ?d, "#{ File.dirname __FILE__ }/../#{ d }"}.all?
+      __FILE__
+    else
+      nil
+    end
+end
+#
+# setup RAILS_ROOT
+#
+unless defined?(RAILS_ROOT)
+  ### grab env var first
+  rails_root = ENV["RAILS_ROOT"]
+
+  ### commandline usage clobbers
+  kv = nil
+  ARGV.delete_if{|arg| arg =~ %r/^RAILS_ROOT=/ and kv = arg}
+  rails_root = kv.split(%r/=/,2).last if kv
+
+  ### we know the rails_root if we are in RAILS_ROOT/script/ 
+  unless rails_root 
+    if BJ_SCRIPT
+      rails_root = File.expand_path "#{ File.dirname __FILE__ }/.."
+    end
+  end
+
+  ### perhaps the current directory is a rails_root?
+  unless rails_root 
+    if %w[ script config app ].map{|d| test(?d, d)}.all?
+      rails_root = File.expand_path "."
+    end
+  end
+
+  ### bootstrap 
+  RAILS_ROOT = rails_root
+end
+#
+# setup RAILS_ENV
+#
+unless defined?(RAILS_ENV)
+  ### grab env var first
+  rails_env = ENV["RAILS_ENV"]
+
+  ### commandline usage clobbers
+  kv = nil
+  ARGV.delete_if{|arg| arg =~ %r/^RAILS_ENV=/ and kv = arg}
+  rails_env = kv.split(%r/=/,2).last if kv
+
+  ### fallback to development
+  unless rails_env
+    rails_env = "development"
+  end 
+
+  ### bootstrap 
+  RAILS_ENV = rails_env
+end
+#
+# ensure that rubygems is loaded
+#
+  begin
+    require "rubygems"
+  rescue
+    42
+  end
+#
+# load gems from plugin dir iff installed as plugin - otherwise load normally
+#
+if RAILS_ROOT and BJ_SCRIPT
+=begin
+  dir = Gem.dir
+  path = Gem.path
+  gem_home = File.join RAILS_ROOT, "vendor", "plugins", "bj", "gem_home"
+  gem_path = [gem_home]
+  Gem.send :use_paths, gem_home, gem_path
+  gem "bj"
+  require "bj"
+  gem "main"
+  require "main"
+=end
+  libdir = File.join(RAILS_ROOT, "vendor", "plugins", "bj", "lib")
+  $LOAD_PATH.unshift libdir
+end
+#
+# hack of #to_s of STDERR/STDOUT for nice help messages
+#
+  class << STDERR
+    def to_s() 'STDERR' end
+  end
+  class << STDOUT
+    def to_s() 'STDOUT' end
+  end
+}