backgroundrb-rails3 1.1

data/config/backgroundrb.yml

@@ -0,0 +1,11 @@
+## YAML Template.
+:backgroundrb:
+  :ip: 0.0.0.0
+  :port: 11006
+
+:schedules:
+  :foo_worker:
+    :worker_method: foobar
+    :trigger_args: */5 * * * * * *
+
+

data/doc/Rakefile

@@ -0,0 +1,5 @@
+Dir['tasks/**/*.rake'].sort.each { |rakefile| load rakefile }
+
+task :default do
+  puts 'This is an example rake task.'
+end

data/doc/config.yaml

@@ -0,0 +1,2 @@
+output_dir:  "output"
+data_source: "filesystem"

data/doc/content/advanced/advanced.txt

@@ -0,0 +1,76 @@
+<div id="content">
+
+%(entry-title)<a name="advanced"> Advanced Stuff </a>%
+
+Each worker comes with an Event loop of its own and can potentially do lots of fancy stuff. Two noteworthy methods are:
+
+<pre class="multiline">connect(ip,port,Handler)
+start_server(ip,port,Handler) </pre>
+
+If you are familiar with the EventMachine or Twisted style of network programming, the above methods allow you to
+start tcp servers inside your workers or let you connect to external tcp servers. For Each accepted client or
+connected socket, an instance of Handler class would be created and integrated with main event loop.
+This can be used for worker to worker communication between backgroundrb servers running on two machines.
+
+p(sub-title). @BackgrounDRb::MetaWorker#connect@ :
+
+@connect@ lets you connect to an external TCP Server and integrates the connection within reactor loop
+of worker. For example:
+
+<pre class="multiline">class TimeClient
+  def receive_data(p_data)
+    worker.get_external_data(p_data)
+  end
+
+  def post_init
+      p "***************** : connection completed"
+  end
+end
+
+class FooWorker < BackgrounDRb::MetaWorker
+  set_worker_name :foo_worker
+  def create(args = nil)
+    external_connection = nil
+    connect("localhost",11009,TimeClient) { |conn| external_connection = conn }
+  end
+  def get_external_data(p_data)
+    puts "And external data is : #{p_data}"
+  end
+end </pre>
+
+p(sub-title). @BackgrounDRb::MetaWorker#start_server@ :
+
+Above method allows you to start a tcp server from your worker, all the accepted connections are integrated with event loop of worker.
+
+<pre class="multiline"> class TimeServer
+
+   def receive_data(p_data)
+   end
+
+   def post_init
+     add_periodic_timer(2) { say_hello_world }
+   end
+
+   def connection_completed
+   end
+
+   def say_hello_world
+     p "***************** : invoking hello world #{Time.now}"
+     send_data("Hello World\n")
+   end
+ end
+
+ class ServerWorker < BackgrounDRb::MetaWorker
+   set_worker_name :server_worker
+   def create(args = nil)
+     # start the server when worker starts
+     start_server("0.0.0.0",11009,TimeServer) do |client_connection|
+       client_connection.say_hello_world
+     end
+   end
+ end </pre>
+
+
+
+
+</div>

data/doc/content/advanced/advanced.yaml

@@ -0,0 +1,4 @@
+filters_pre: [ "redcloth" ]
+# Custom
+title: "Advanced Stuff"
+

data/doc/content/bugs/bugs.txt

@@ -0,0 +1,20 @@
+<div id="content">
+
+%(entry-title)<a name="trac">Submit Bug Report or Patch </a>%
+
+_BackgrounDRb_ causing trouble, or you have a potential fix? Why not submit the bug report "here":http://backgroundrb.devjavu.com/report/ .
+You can also submit potential patches that fix known bugs or add exciting features to the above trac url.
+
+Also, if you are using older version of _BackgrounDRb_ and having a problem, why not upgrade to latest code from git, your bug
+might have been fixed there.
+
+%(entry-title)<a name="known_bugs">Known Problems </a>%
+
+Since _BackgrounDRb_ has feature of thread pools, which allows users to execute tasks
+concurrently, we are using @allow_concurrency = true@ in @ActiveRecord@ model objects.
+This has been known to create some issues with Oracle database Adapters. If you are sure of
+what you are doing and don't need thread pool feature, you can go ahead and remove @allow_concurrency = true@
+from _BackgrounDRb_ code and it should work.
+
+</div>
+

data/doc/content/bugs/bugs.yaml

@@ -0,0 +1,5 @@
+# Built-in
+filters_pre: [ "redcloth" ]
+# Custom
+title: "Report Bugs"
+sidebar_items: [["Trac", "#trac"], ["KnownBugs", "#known_bugs"]]

data/doc/content/community/community.txt

@@ -0,0 +1,36 @@
+<div id="content">
+
+%(entry-title)<a name="people"> People </a>%
+
+_BackgrounDRb_ has been brought to you by, "Hemant Kumar":http://gnufied.org with help of folks
+from Ruby and Ruby on Rails community. _BackgrounDRb_ was an original idea of "Ezra Zygmuntowicz":http://brainspl.at .
+
+Today it pulls in ideas from "EventMachine":http://eventmachine.rubyforge.org , "Twisted":http://twistedmatrix.com .
+I am eternally grateful to following people ( in no particular order ):
+
+*(content_list) Francis Cianfrocca : For EventMachine
+* Matz : For creating Ruby
+* Dale Cook : For Documentation
+* "Jason LaPier":http://offtheline.net : for initial testing and bug reports.
+* Kevin Russell : For Letting me work on his Mac machine all day and night and helping out with Mac OSX issues.
+* Richard Stallman: For giving us free software and Emacs.
+
+
+%(entry-title)<a name="mailing_list"> Mailing List </a>%
+
+You can join BackgrounDRb mailing list "here":http://rubyforge.org/mailman/listinfo/backgroundrb-devel .
+Its a perfect place for discussing new ideas, asking questions, submitting patches and stuff.
+
+%(entry-title)<a name="irc"> IRC Help </a>%
+
+You can also try #backgroundrb on freenode for help.
+
+%(entry-title)<a name="contribute"> Contribute </a>%
+
+Your contributions/feedback is most welcome. We have very simple rule for giving new coders
+commit access. If you submit one patch which is accepted for inclusion in the _BackgrounDRb_
+code repository, you are elligible for commit access. Create an account on "Github":http://github.com/
+and let me (gethemant at gmail dot com) know your login handle.
+
+</div>
+

data/doc/content/community/community.yaml

@@ -0,0 +1,5 @@
+# Built-in
+filters_pre: [ "redcloth" ]
+# Custom
+title: "Join the BackgrounDRb Community"
+sidebar_items: [["People", "people"], ["MailingList", "#mailing_list"], ["IRC", "#irc"], ["Contribute", "#contribute"]]

data/doc/content/content.txt

@@ -0,0 +1,168 @@
+<div id="content">
+
+%(entry-title)<a name="introduction"> Introduction </a>%
+
+*BackgrounDRb* is a Ruby job server and scheduler. Its main intent is to be
+used with Ruby on Rails applications for offloading long-running tasks.
+Since a Rails application blocks while serving a request it is best to
+move long-running tasks off into a background process that is divorced
+from http request/response cycle.
+
+%(entry-title)<a name="installation"> Installation </a>%
+
+p(sub-title). Installing the dependencies :
+
+As of version 1.0.4 _BackgrounDRb_ depends on _chronic_ and _packet_ gems. Thus lets get
+started by installing these two gems:
+
+<pre class="boxed">sudo gem install chronic packet </pre>
+
+Please note that, this version of _BackgrounDRb_ needs packet version 0.1.7 or greater, so make
+sure you have that.
+
+p(sub-title). Getting the code from Subversion :
+
+<pre class="boxed"> svn co http://svn.devjavu.com/backgroundrb/trunk </pre>
+
+p(sub-title). Installing from Git:
+
+As of version 1.0.4 __BackgrounDRb__ development has moved to <a href="http://github.com/gnufied/backgroundrb/tree/master">github</a>. Above SVN url
+will always mirror stable releases from Git. However to enjoy latest and greatest of features
+you can install the plugin from git:
+
+<pre class="multiline">
+git clone git://github.com/gnufied/backgroundrb.git </pre>
+
+<p style="text-decoration: line-through;"> Also for running git version of BackgrounDRb you will need, git version of packet.</p>
+
+p(sub-title). Installation using Piston
+
+<pre class="boxed">piston import http://svn.devjavu.com/backgroundrb/trunk/ backgroundrb </pre>
+
+%(entry-title)<a name="configuration"> Configuration </a>%
+
+After getting the plugin, you must copy it into your vendor/plugins and
+then configure it for use. _BackgrounDRb_ comes with a rake task for
+automating plugin configuration. Before running rake task, remove if
+any old @backgroundrb@ or @load_worker_env.rb@ script is there in script folder of your rails
+app after that run, following command from root directory of your
+rails application:
+
+<pre class="boxed">rake backgroundrb:setup </pre>
+
+Above Command does following things :
+
+*(content_list) Creates a config file @backgroundrb.yml@ in config directory of your rails application.
+* Creates @RAILS_ROOT/lib/workers@ directory for keeping BackgrounDRb workers in one place.
+* Creates @RAILS_ROOT/test/bdrb_test_helper.rb@ as a test helper for your workers
+* Creates migration required for creating persistent job queue table.
+
+After above make sure, that generated migration is ran using:
+
+<pre class="boxed">rake db:migrate </pre>
+
+p(sub-title). Configuration Options
+
+A default @backgroundrb.yml@ file looks like this:
+
+<pre class="multiline">
+:backgroundrb:
+  :port: 11006
+  :ip: 0.0.0.0 </pre>
+
+However, various other configuration options are available. For example, to load @production@
+environment in your workers:
+
+<pre class="multiline">
+:backgroundrb:
+  :port: 11006
+  :ip: 0.0.0.0
+  :environment: production </pre>
+
+
+Following file demonstrates other available configuration options:
+
+<pre class="multiline">
+---
+:backgroundrb:
+  :port: 11006 # port to start listen
+  :ip: localhost # host to listen
+  :environment: production # rails environment to load
+  :log: foreground # foreground mode,print log messages on console
+  :debug_log: false # disable log workers and other logging
+  :persistent_disabled: false # turn this off if your application doesn't use backgroundrb's persistent/enqueued tasks system
+  :persistent_delay: 10 # the time (seconds) between each time backgroundrb checks the database for enqueued tasks
+:schedules: # optional task scheduling
+  : # look in scheduling section </pre>
+
+%(entry-title)<a name="worker"> Workers </a>%
+
+Once you are set with initial configuration, you can proceed to create worker and start
+_BackgrounDRb_ server. To generate a worker:
+
+<pre class="boxed"> ./script/generate worker billing </pre>
+
+Output will look something like:
+
+<pre class="multiline">exists  lib/workers/
+create  lib/workers/billing_worker.rb </pre>
+
+And generated worker will look like:
+
+<pre class="multiline">class BillingWorker < BackgrounDRb::MetaWorker
+  set_worker_name :billing_worker
+  def create(args = nil)
+    # method gets called, when new instance of worker is created.
+   end
+  end </pre>
+
+
+You can define worker specific initialization in @create@ method. Tasks that are to be executed
+in this worker should be defined as seperate methods. For example:
+
+<pre class="multiline">class BillingWorker < BackgrounDRb::MetaWorker
+  set_worker_name :billing_worker
+  def create(args = nil)
+    # this method is called, when worker is loaded for the first time
+  end
+
+  def charge_customer(customer_id = nil)
+    logger.info 'charging customer now'
+  end
+end </pre>
+
+%(entry-title)<a name="invoking_tasks"> Invoking Tasks </a>%
+
+Task @charge_customer@ defined in @BillingWorker@ can be invoked in several ways. To beging with
+it can be invoked from rails or can be scheduled to execute at particular interval using
+cron like syntax.
+
+p(sub-title). Invoking Task from Rails :
+
+In your Rails controllers you have access to proxy class @MiddleMan@ which can be used
+to interact with BackgrounDRb server, either from Rails controllers/Models or from Rails console.
+For example to invoke @charge_customer@ method asychronously one can use:
+
+<pre class="boxed">MiddleMan.worker(:billing_worker).async_charge_customer(:arg => current_customer.id) </pre>
+
+Above code can be also executed from Rails console.
+
+p(sub-title). Start the BackgrounDRb server :
+
+You can use:
+
+<pre class="boxed">./script/backgroundrb start </pre>
+
+For more options:
+
+<pre class="boxed">./script/backgroundrb --help </pre>
+
+As documented above, you can use @backgroundrb.yml@ file to load different rails environments, however you can use:
+
+<pre class="boxed">./script/backgroundrb -e development </pre>
+
+</div>
+
+
+
+

data/doc/content/content.yaml

@@ -0,0 +1,5 @@
+# Built-in
+filters_pre: [ "redcloth" ]
+# Custom
+title: "Introduction BackgrounDRb"
+sidebar_items: [["Introduction", "#introduction"], ["Installation", "#installation"], ["Configuration", "#configuration"], ["Workers", "#create_worker"]]

data/doc/content/faq/faq.txt

@@ -0,0 +1,41 @@
+<div id="content">
+
+%(entry-title)<a name="faq"> Frequently asked Questions </a>%
+
+*(content_list) *How does it work with ActiveRecord in multi-threaded environment?*<br/>
+You can use _BackgrounDRb_'s inbuilt thread pool for executing tasks
+concurrently. We are using @allow_concurrency = true@ for @ActiveRecord@
+models, which is known to create problem with Oracle database adapters, if
+you are using Oracle db, you can turn it off, by removing @allow_concurrency = true@.
+* *Whats the deal with persistent job queues?*<br />
+Well, its used for persisting tasks to the table, so as later
+then can be inspected for status and results. Before using this feature.You should
+run @rake backgroundrb:setup@ and run @rake db:migrate@ for creation of required
+table.
+* *My worker dies silently*<br />
+First,make sure you have logging enabled (Meaning, you are not using @:debug_log false@
+in your config file).Second, if a worker dies, the backtrace should be written to the
+file @RAILS_ROOT/log/backgroundrb_debug_xxxx.log@. Third, You should not pass
+entire @ActiveRecord@ model's around, this could cause hard to debug problems.AR objects
+don't serialize well across tcp sockets. If none of this helps, hit us on the mailing list.
+* *Master server died silently*<br/>
+If _BackgrounDRb_ master server dies, its definetely a bug in _BackgrounDRb_, exception
+should be still logged in same @RAILS_ROOT/log/backgroundrb_debug_xxxx.log@ file. Copy
+the exception and file a bug report.
+* *Where do I file bug reports?* <br />
+You can submit bug reports at: "http://backgroundrb.devjavu.com/report/":http://backgroundrb.devjavu.com/report/.
+* *Result caching is not working properly* <br />
+Switch to memcache for storing results. More information "here:"http://localhost:3000/workers/#result_caching.
+* *Dynamically started workers through @new_worker@ are not working* <br />
+Use, the worker key that was used for accessing dynamically started workers.
+* *BackgrounDRb server is not starting*<br />
+Make sure, you have latest version of chronic and packet gems installed.After
+that, clone the git repository (or download a snapshot from github) and run
+@rake backgroundrb:setup@. If you are upgrading make sure, that older
+autogenerated scripts namely, @load_worker_env.rb@ and @backgroundrb@
+are removed from script directory of your rails application before running
+rake task.If you are upgrading to 1.1, check if @scheduled_at@ column
+is there in @bdrb_job_queues@ table.
+
+</div>
+

data/doc/content/faq/faq.yaml

@@ -0,0 +1,5 @@
+# Built-in
+filters_pre: [ "redcloth" ]
+# Custom
+title: "FAQs"
+sidebar_items: [["Trac", "#trac"], ["KnownBugs", "#known_bugs"]]

data/doc/content/rails/rails.txt

@@ -0,0 +1,182 @@
+<div id="content">
+
+%(entry-title)<a name="introduction"> Introduction </a>%
+
+BackgrounDRb offers seamless integration with rails. You can invoke random tasks defined in your
+workers from rails. You can pass arguments, collect results, monitor status of workers and other
+stuff.
+
+%(entry-title)<a name="async_task"> Invoke a task asynchronously on a worker </a>%
+
+Let's say, you have following worker code:
+
+<pre class="multiline">class FooWorker < BackgrounDRb::MetaWorker
+  set_worker_name :foo_worker
+  def create(args = nil)
+    # this method is called, when worker is loaded for the first time
+  end
+
+  def some_task args
+    # perform a long running task
+  end
+end
+</pre>
+
+And you want to invoke @some_task@ method with appropriate arguments from rails.
+Following snippet will invoke method @some_task@ with argument @data@ in @foo_worker@. Also, method will
+be invoked asynchronously and Rails won't wait for result from BackgrounDRb server.
+
+<pre class="multiline">worker = MiddleMan.worker(:foo_worker)
+worker.async_some_task(:arg => data) </pre>
+
+It should be noted that, since @some_task@ method is being
+executed asynchronously, don't expect any meaningful return values from method invocation.
+If you want to invoke a method on worker and collect results returned by it, you
+should read next section (Invoke method and collect results).
+
+When you invoke @MiddleMan.worker(:foo_worker)@ it returns a worker proxy, hence you can combine above two lines in
+one as follows:
+
+<pre class="multiline">MiddleMan.worker(:foo_worker,<optional_worker_key>).
+     async_some_task(:arg => data) </pre>
+
+Above snippet also demonstrates that, if your worker was started with a @worker_key@ you can use it to
+get correct worker proxy.
+
+%(entry-title)<a name="sync_task"> Synchronous Task invocation (Invoke task and wait for results) </a>%
+
+Following snippet will invoke method @some_task@ with argument @data@ in @foo_worker@. Also, method will block
+until BackgrounDRb server returns a result.
+
+<pre class="multiline">worker = MiddleMan.worker(:foo_worker)
+result = worker.some_task(:arg => data) </pre>
+
+Since, now you are expecting a return value from your worker method, new worker code will look like:
+
+<pre class="multiline">class FooWorker < BackgrounDRb::MetaWorker
+  set_worker_name :foo_worker
+  def create(args = nil)
+    # this method is called, when worker is loaded for the first time
+  end
+
+  def some_task args
+    billing_result = UserPayment.bill!
+    return billing_result
+  end
+end
+</pre>
+
+As illustrated above, you can use @worker_key@ or make them in single line too.
+
+%(entry-title)<a name="worker_results"> Retrieve Cached Worker results </a>%
+
+If you are using @cache@ in your worker code to store result objects, you can retrieve them from
+rails using:
+
+<pre class="boxed">status_obj = MiddleMan.worker(:foo_worker).ask_result(cache_key) </pre>
+
+You must use @worker_key@ if *worker was started with a worker_key*.
+
+From controller, you can also reset result stored for a particular worker, with particular cache key.
+This is only applicable, if you are using memcache for storing results.
+
+<pre class="multiline">
+MiddleMan.worker(:foo_worker).reset_memcache_result(cache_key) # or
+MiddleMan.worker(:foo_worker,"worker_key").reset_memcache_result(cache_key)
+</pre>
+
+%(entry-title)<a name="persistent_task"> Enqueue task to the persistent job queue : </a>%
+
+Jobs executed via synchronous and asynchronous APIs are fine, but these tasks are usually
+kept in memory(and hence they are fast) and hence aren't entirely failsafe.
+
+To solve this _BackgrounDRb_ also lets you add jobs to a persistent job queue, which is
+automatically picked by responsible worker and invoked. To use this:
+
+<pre class="boxed">MiddleMan(:hello_worker).enq_some_task(:arg => "hello_world",:job_key => "boy")</pre>
+
+With _BackgrounDRb_ version >= 1.1, you can also schedule a persistent task to be executed at a particular time,
+
+<pre class="multiline">MiddleMan(:hello_worker).enq_some_task(:arg => "hello_world",
+                      :job_key => "boy",:scheduled_at => (Time.now + 1.hour))</pre>
+
+Above line will add specified task to the job queue and set to be invoked at specified time. For more information
+about scheduling see scheduling section.
+
+
+%(entry-title)<a name="new_worker"> Start a new worker from controller </a>%
+
+To start a worker from rails:
+
+<pre class="multiline">used_job_key = MiddleMan.new_worker(:worker => :foo_worker,\
+     :worker_key => "my_secret_job_key") </pre>
+
+Worker key passed here, while starting the worker can be used later for invoking tasks on started
+worker or for accessing cached result objects and stuff like that.
+
+Important thing to be kept in mind is, when you are creating a worker using above approach, you
+must use a unique @worker_key@ while starting the worker. Also, while invoking any of the other methods
+like @ask_result@, @worker_info@ or one of the worker methods, you must user same @worker_key@.
+
+%(entry-title)<a name="worker_info"> Worker Info </a>%
+
+You can get worker specific information using:
+
+<pre class="boxed">MiddleMan.worker(:foo_worker).worker_info </pre>
+
+The return value will look something like:
+
+<pre class="boxed">{:worker=>:foo_worker, :status=>:running, :worker_key=>"hello"} </pre>
+
+Information about all currently running workers can be obtained using:
+
+<pre class="boxed">MiddleMan.all_worker_info </pre>
+
+Return value will look like:
+
+<pre class="multiline">{"0.0.0.0:11006"=>nil, "0.0.0.0:11008"=>
+[{:worker_key=>"", :status=>:running, :worker=>:log_worker},
+{:worker_key=>"", :status=>:running, :worker=>:foo_worker}]}</pre>
+
+%(entry-title)<a name="clustering"> BackgrounDRb Clustering </a>%
+
+By using following option in your @backgroundrb.yml@ you can cluster more than
+one backgroundrb server.
+
+
+<pre class="multiline">:backgroundrb:
+  :ip: 0.0.0.0
+  :port: 11006
+  :environment: production
+:client: "10.0.0.1:11006,10.0.0.2:11007"</pre>
+
+
+So what happens here is, now BackgrounDRb client will talk to bdrb
+servers running on both @10.0.0.1:11006@ and @10.0.0.2:11007@. So when you invoke
+a task like this:
+
+<pre class="boxed">MiddleMan.worker(:foo_worker).async_some_task(:arg => data) </pre>
+
+Your task gets executed in round robin manner in specified servers by default.
+Also, once a server goes down, it will automatically stop participating in clustering and
+when it comes back, it will be automatically start participating in clustering.
+
+
+In addition to default round robin task distribution, you can override this behaviour
+by passing additional @:host@ option while invoking task from rails.For example:
+
+
+<pre class="multiline">
+# run method 'some_task' on all backgroundrb servers
+MiddleMan.worker(:hello_worker).async_some_task(:arg => data,
+               :job_key => session[:user_id],:host => :all)
+
+# run method 'some_task' on only locally configured server
+MiddleMan.worker(:hello_worker).async_some_task(:arg => data,
+               :job_key => session[:user_id],:host => :local)
+
+# run the task on specified server
+MiddleMan.worker(:hello_worker).async_some_task(:arg => data,:job_key => \
+               session[:user_id],:host => "10.0.0.2:11210")
+</pre>
+</div>