que 1.0.0.beta → 1.0.0.beta5

data/docs/multiple_queues.md

@@ -1,31 +0,0 @@
-## Multiple Queues
-
-Que supports the use of multiple queues in a single job table. Please note that this feature is intended to support the case where multiple codebases are sharing the same job queue - if you want to support jobs of differing priorities, the numeric priority system offers better flexibility and performance.
-
-For instance, you might have a separate Ruby application that handles only processing credit cards. In that case, you can run that application's workers against a specific queue:
-
-```shell
-que --queue-name credit_cards
-# The -q flag is equivalent, and either can be passed multiple times.
-que -q default -q credit_cards
-```
-
-Then you can set jobs to be enqueued in that queue specifically:
-
-```ruby
-ProcessCreditCard.enqueue current_user.id, queue: 'credit_cards'
-
-# Or:
-
-class ProcessCreditCard < Que::Job
-  # Set a default queue for this job class; this can be overridden by
-  # passing the :queue parameter to enqueue like above.
-  self.queue = 'credit_cards'
-end
-```
-
-In some cases, the ProcessCreditCard class may not be defined in the application that is enqueueing the job. In that case, you can specify the job class as a string:
-
-```ruby
-Que.enqueue current_user.id, job_class: 'ProcessCreditCard', queue: 'credit_cards'
-```

data/docs/shutting_down_safely.md

@@ -1,7 +0,0 @@
-## Shutting Down Safely
-
-To ensure safe operation, Que needs to be very careful in how it shuts down. When a Ruby process ends normally, it calls Thread#kill on any threads that are still running - unfortunately, if a thread is in the middle of a transaction when this happens, there is a risk that it will be prematurely commited, resulting in data corruption. See [here](http://blog.headius.com/2008/02/ruby-threadraise-threadkill-timeoutrb.html) and [here](http://coderrr.wordpress.com/2011/05/03/beware-of-threadkill-or-your-activerecord-transactions-are-in-danger-of-being-partially-committed/) for more detail on this.
-
-To prevent this, Que will block the worker process from exiting until all jobs it is working have completed normally. Unfortunately, if you have long-running jobs, this may take a very long time (and if something goes wrong with a job's logic, it may never happen). The solution in this case is SIGKILL - luckily, Ruby processes that are killed via SIGKILL will end without using Thread#kill on its running threads. This is safer than exiting normally - when PostgreSQL loses the connection it will simply roll back the open transaction, if any, and unlock the job so it can be retried later by another worker. Be sure to read [Writing Reliable Jobs](https://github.com/chanks/que/blob/master/docs/writing_reliable_jobs.md) for information on how to design your jobs to fail safely.
-
-So, be prepared to use SIGKILL on your Ruby processes if they run for too long. For example, Heroku takes a good approach to this - when Heroku's platform is shutting down a process, it sends SIGTERM, waits ten seconds, then sends SIGKILL if the process still hasn't exited. This is a nice compromise - it will give each of your currently running jobs ten seconds to complete, and any jobs that haven't finished by then will be interrupted and retried later.

data/docs/using_plain_connections.md

@@ -1,65 +0,0 @@
-## Using Plain Postgres Connections
-
-If you're not using an ORM like ActiveRecord or Sequel, you can use a distinct connection pool to manage your Postgres connections. Please be aware that if you **are** using ActiveRecord or Sequel, there's no reason for you to be using any of these methods - it's less efficient (unnecessary connections will waste memory on your database server) and you lose the reliability benefits of wrapping jobs in the same transactions as the rest of your data.
-
-## Using ConnectionPool or Pond
-
-Support for two connection pool gems is included in Que. The first is the ConnectionPool gem (be sure to add `gem 'connection_pool'` to your Gemfile):
-
-```ruby
-require 'uri'
-require 'pg'
-require 'connection_pool'
-
-uri = URI.parse(ENV['DATABASE_URL'])
-
-Que.connection = ConnectionPool.new(size: 10) do
-  PG::Connection.open(
-    host:     uri.host,
-    user:     uri.user,
-    password: uri.password,
-    port:     uri.port || 5432,
-    dbname:   uri.path[1..-1]
-  )end
-```
-
-Be sure to pick your pool size carefully - if you use 10 for the size, you'll incur the overhead of having 10 connections open to Postgres even if you never use more than a couple of them.
-
-The Pond gem doesn't have this drawback - it is very similar to ConnectionPool, but establishes connections lazily (add `gem 'pond'` to your Gemfile):
-
-```ruby
-require 'uri'
-require 'pg'
-require 'pond'
-
-uri = URI.parse(ENV['DATABASE_URL'])
-
-Que.connection = Pond.new(maximum_size: 10) do
-  PG::Connection.open(
-    host:     uri.host,
-    user:     uri.user,
-    password: uri.password,
-    port:     uri.port || 5432,
-    dbname:   uri.path[1..-1]
-  )
-end
-```
-
-## Using Any Other Connection Pool
-
-You can use any other in-process connection pool by defining access to it in a proc that's passed to `Que.connection_proc = proc`. The proc you pass should accept a block and call it with a connection object. For instance, Que's built-in interface to Sequel's connection pool is basically implemented like:
-
-```ruby
-Que.connection_proc = proc do |&block|
-  DB.synchronize do |connection|
-    block.call(connection)
-  end
-end
-```
-
-This proc must meet a few requirements:
-- The yielded object must be an instance of `PG::Connection`.
-- It must be reentrant - if it is called with a block, and then called again inside that block, it must return the same object. For example, in `proc.call{|conn1| proc.call{|conn2| conn1.object_id == conn2.object_id}}` the innermost condition must be true.
-- It must lock the connection object and prevent any other thread from accessing it for the duration of the block.
-
-If any of these conditions aren't met, Que will raise an error.

data/docs/using_sequel.md

@@ -1,33 +0,0 @@
-## Using Sequel
-
-If you're using Sequel, with or without Rails, you'll need to give Que a specific database instance to use:
-
-```ruby
-DB = Sequel.connect(ENV['DATABASE_URL'])
-Que.connection = DB
-```
-
-Then you can safely use the same database object to transactionally protect your jobs:
-
-```ruby
-class MyJob < Que::Job
-  def run(user_id:)
-    # Do stuff.
-
-    DB.transaction do
-      # Make changes to the database.
-
-      # Destroying this job will be protected by the same transaction.
-      destroy
-    end
-  end
-end
-
-# Or, in your controller action:
-DB.transaction do
-  @user = User.create(params[:user])
-  MyJob.enqueue user_id: @user.id
-end
-```
-
-Sequel automatically wraps model persistance actions (create, update, destroy) in transactions, so you can simply call #enqueue methods from your models' callbacks, if you wish.

data/docs/writing_reliable_jobs.md

@@ -1,108 +0,0 @@
-## Writing Reliable Jobs
-
-Que does everything it can to ensure that jobs are worked exactly once, but if something bad happens when a job is halfway completed, there's no way around it - the job will need be repeated over again from the beginning, probably by a different worker. When you're writing jobs, you need to be prepared for this to happen.
-
-The safest type of job is one that reads in data, either from the database or from external APIs, then does some number crunching and writes the results to the database. These jobs are easy to make safe - simply write the results to the database inside a transaction, and also destroy the job inside that transaction, like so:
-
-```ruby
-class UpdateWidgetPrice < Que::Job
-  def run(widget_id)
-    widget = Widget[widget_id]
-    price  = ExternalService.get_widget_price(widget_id)
-
-    ActiveRecord::Base.transaction do
-      # Make changes to the database.
-      widget.update price: price
-
-      # Mark the job as destroyed, so it doesn't run again.
-      destroy
-    end
-  end
-end
-```
-
-Here, you're taking advantage of the guarantees of an [ACID](https://en.wikipedia.org/wiki/ACID) database. The job is destroyed along with the other changes, so either the write will succeed and the job will be run only once, or it will fail and the database will be left untouched. But even if it fails, the job can simply be retried, and there are no lingering effects from the first attempt, so no big deal.
-
-The more difficult type of job is one that makes changes that can't be controlled transactionally. For example, writing to an external service:
-
-```ruby
-class ChargeCreditCard < Que::Job
-  def run(user_id, credit_card_id)
-    CreditCardService.charge(credit_card_id, amount: "$10.00")
-
-    ActiveRecord::Base.transaction do
-      User.where(id: user_id).update_all charged_at: Time.now
-      destroy
-    end
-  end
-end
-```
-
-What if the process abruptly dies after we tell the provider to charge the credit card, but before we finish the transaction? Que will retry the job, but there's no way to tell where (or even if) it failed the first time. The credit card will be charged a second time, and then you've got an angry customer. The ideal solution in this case is to make the job [idempotent](https://en.wikipedia.org/wiki/Idempotence), meaning that it will have the same effect no matter how many times it is run:
-
-```ruby
-class ChargeCreditCard < Que::Job
-  def run(user_id, credit_card_id)
-    unless CreditCardService.check_for_previous_charge(credit_card_id)
-      CreditCardService.charge(credit_card_id, amount: "$10.00")
-    end
-
-    ActiveRecord::Base.transaction do
-      User.where(id: user_id).update_all charged_at: Time.now
-      destroy
-    end
-  end
-end
-```
-
-This makes the job slightly more complex, but reliable (or, at least, as reliable as your credit card service).
-
-Finally, there are some jobs where you won't want to write to the database at all:
-
-```ruby
-class SendVerificationEmail < Que::Job
-  def run(email_address)
-    Mailer.verification_email(email_address).deliver
-  end
-end
-```
-
-In this case, we don't have any no way to prevent the occasional double-sending of an email. But, for ease of use, you can leave out the transaction and the `destroy` call entirely - Que will recognize that the job wasn't destroyed and will clean it up for you.
-
-### Timeouts
-
-Long-running jobs aren't necessarily a problem for the database, since the overhead of an individual job is very small (just an advisory lock held in memory). But jobs that hang indefinitely can tie up a worker and [block the Ruby process from exiting gracefully](https://github.com/chanks/que/blob/master/docs/shutting_down_safely.md), which is a pain.
-
-If there's part of your job that is prone to hang (due to an API call or other HTTP request that never returns, for example), you can (and should) timeout those parts of your job. For example, consider a job that needs to make an HTTP request and then write to the database:
-
-```ruby
-class ScrapeStuff < Que::Job
-  def run(url_to_scrape)
-    result = YourHTTPLibrary.get(url_to_scrape)
-
-    ActiveRecord::Base.transaction do
-      # Insert result...
-
-      destroy
-    end
-  end
-end
-```
-
-That request could take a very long time, or never return at all. Let's use the timeout feature that almost all HTTP libraries offer some version of:
-
-```ruby
-class ScrapeStuff < Que::Job
-  def run(url_to_scrape)
-    result = YourHTTPLibrary.get(url_to_scrape, timeout: 5)
-
-    ActiveRecord::Base.transaction do
-      # Insert result...
-
-      destroy
-    end
-  end
-end
-```
-
-Now, if the request takes more than five seconds, an error will be raised (probably - check your library's documentation) and Que will just retry the job later.