eycloud-recipe-resque 1.0.3 → 1.1.0

data/.gitignore

@@ -1,2 +1,3 @@
 pkg
-tmp
+tmp
+Gemfile.lock

data/ChangeLog.md

@@ -0,0 +1,11 @@
+# ChangeLog
+
+## v1.1.0
+
+Fixes based on testing with [todo_with_resque](https://github.com/engineyard/todo_with_resque) Rails 3 app.
+
+Most important change: Instructions on how to restart resque after deploy!
+
+## v1.0.0
+
+First release based on various Resque projects found.

data/README.md

@@ -1,16 +1,26 @@
+
 # Resque recipe for EY Cloud
 
-Resque is a Redis-backed Ruby library for creating background jobs, placing those jobs on multiple queues, and processing them later.
+[Resque](https://github.com/defunkt/resque) is a Redis-backed Ruby library for creating background jobs, placing those jobs on multiple queues, and processing them later.
 
-This recipe will setup `resque` on a Solo instance environment or on named Utility instances in a cluster environment. 
+This recipe will setup Resque on a Solo instance environment or on named Utility instances in a cluster environment.
 
-Name your Utility instances with prefixes: `resque`. For example, `resque`, or `resque123`. Naming them all `resque` works just fine.
+## Installation
 
-This recipe assumes that redis is running within the environment. Fortunately, it is. By default EY Cloud runs redis on your `solo` or `db_master` instance. Optionally, this recipe will use a custom `redis` utility instance that you might have in your environment. It's clever like that.
+1. Install the recipe
+2. Add a deploy hook to restart resque workers after the application restarts
+3. Name all utility instances `resque`.
 
-## Installation
+```ruby
+# deploy/after_restart.rb
+on_utilities(:resque) do
+  node[:applications].each do |app_name, data|
+    sudo 'echo "sleep 20 && monit -g resque_#{app_name} restart all" | at now'
+  end
+end
+```
 
-## Simple Installation
+### Simple Installation
 
 To add this recipe to your collection of recipes, or as your first recipe, you can use the helpful `ey-recipes` command line tool:
 
@@ -27,7 +37,7 @@ If you want to have your recipes run during deploy (rather than the separate `ey
     git add .; git commit -m "added delayed job recipe"; git push origin master
     ey deploy
 
-## Manual Installation
+### Manual Installation
 
 Clone/copy this repository into a `cookbooks/resque` folder (such that you have a `cookbooks/resque/recipes/default.rb` file). This recipe must be installed as `resque` and not `eycloud-recipe-resque` or anything fancy.
 
@@ -41,3 +51,68 @@ repository.
 Then to upload and apply to EY Cloud for a given environment:
 
     ey recipes upload --apply -e target-environment
+
+### Rails 3
+
+To ensure that Resque can discover where Redis installed, create a `config/initializers/resque.rb` file:
+
+```ruby
+resque_yml = File.expand_path('../../resque.yml', __FILE__)
+if File.exist?(resque_yml)
+  Resque.redis = YAML.load_file(resque_yml)["redis_uri"]
+end
+```
+
+## How do I get some Resque workers?
+
+Resque workers can process 1 task/job at a time.
+
+If you have a solo instance, then one Resque worker is provisioned. 
+
+If you have one or more app instances, then no Resque workers are provisioned. Instead, add dedicated utility instances to provision Resque workers.
+
+Name your Utility instances `resque`.
+
+## How many Resque workers will I get?
+
+All the available memory of an instance is used for Resque workers, with 300Mb set aside for operating system tasks. Resque workers are allocated 300Mb each.
+
+As above, solo instances of any time have 1 worker. 
+
+For utility instances you get approximately the following number of workers:
+
+* Small -> 4 workers within 1.7 GB
+* Large -> 24 workers within 7.5 GB
+* Extra Large -> 49 workers within 15 GB
+
+* High Memory Extra Large -> 55 workers within 17.1 GB
+* High-Memory Double Extra Large -> 113 workers within 34.2 GB
+* High-Memory Quadruple Extra Large -> 213 workers within 64.4 GB
+
+* High-CPU Medium -> 4 workers within 1.7 GB 
+* High-CPU Extra Large -> 22 workers within 7 GB
+
+## How do I setup Redis?
+
+This recipe assumes that Redis is running within the environment. Fortunately, it is. Redis is automatically running on all Engine Yard Cloud environments.
+
+If you have a dedicate Redis instance called `redis` then this recipe will discover and use that version of Redis.
+
+## How do I use resque-web?
+
+In a Rails 3 app, you can access "resque-web" via your normal Rails app using mounted apps.
+
+As a very simple, moderately secure method, add the following into your `routes.rb` routes:
+
+```ruby
+require "resque/server"
+mount Resque::Server.new, :at => "/resque/34257893542"
+```
+
+For more security, consider the ideas in this [gist](https://gist.github.com/1575082).
+
+## Helpers
+
+* `resque_instances` - returns list of instances that have resque running on them
+* `resque_instance?` - does this instance have resque running on it?
+

data/eycloud-recipe-resque.gemspec

@@ -1,7 +1,7 @@
 # -*- encoding: utf-8 -*-
 $:.push File.expand_path("../lib", __FILE__)
 
-version = "1.0.3" # get from metadata.json or .rb
+version = "1.1.0" # get from metadata.json or .rb
 
 Gem::Specification.new do |s|
   s.name        = "eycloud-recipe-resque"

data/libraries/find_resque_instances.rb

@@ -0,0 +1,21 @@
+class Chef
+  class Recipe
+    # Return which instance is to have redis installed on it
+    # This is determind as follows:
+    #   1. A utility prefixed with 'resque'
+    #   2. A solo
+    # Returns array of hash of instance data, including { "id" => "i-123456", }
+    def resque_instances
+      @resque_instances ||= node[:engineyard][:environment][:instances].find do |x|
+        x[:role] == "solo" || node[:instance_role] == "eylocal" || 
+                        (node[:role] == "util" && node[:name] =~ /^(resque)/)
+      end
+    end
+    
+    # Does this instance run resque?
+    def resque_instance?
+      node[:instance_role] == "solo" || node[:instance_role] == "eylocal" || 
+              (node[:instance_role] == "util" && node[:name] =~ /^(resque)/)
+    end
+  end
+end

data/libraries/get_resque_worker_count.rb

@@ -1,6 +1,8 @@
 class Chef
   class Recipe
-    def get_resque_worker_count      
+    def get_resque_worker_count
+      return 1 if node[:instance_role] == 'solo' || node[:instance_role] == 'eylocal'
+           
       # $ sudo cat /proc/meminfo
       # MemTotal:        1759228 kB
       # ...
@@ -11,7 +13,6 @@ class Chef
       # in other words, if all workers on the instance are running jobs, there should
       #   be memory left over
       result = (mem_total_mb - node[:worker_memory].to_i) / node[:worker_memory].to_i
-      result /= 2 if node[:instance_role] == 'solo' || node[:instance_role] == 'eylocal'
       result
     end
   end

data/metadata.json

@@ -2,7 +2,7 @@
   "license": "MIT",
   "maintainer": "Engine Yard LLC",
   "maintainer_email": "drnic@engineyard.com",
-  "version": "1.0.3",
+  "version": "1.1.0",
   "description": "Resque for EY Cloud",
   "name": "resque",
   "attributes": {

data/metadata.rb

@@ -3,5 +3,5 @@ description       "Resque for EY Cloud"
 long_description  IO.read(File.join(File.dirname(__FILE__), 'README.md'))
 maintainer        "Engine Yard LLC"
 maintainer_email  "drnic@engineyard.com"
-version           "1.0.3"
+version           "1.1.0"
 depends           "cronjobs"

data/recipes/configure.rb

@@ -3,8 +3,7 @@
 # Recipe:: configure
 #
 
-if node[:instance_role] == "solo" || node[:instance_role] == "eylocal" || 
-                (node[:instance_role] == "util" && node[:name] =~ /^(resque)/)
+if resque_instance?
   resque_workers_count = get_resque_worker_count()
 
   directory "/tmp/resque_ttls" do

data/recipes/install.rb

@@ -3,8 +3,7 @@
 # Recipe:: install
 #
 
-if node[:instance_role] == "solo" || node[:instance_role] == "eylocal" || 
-                (node[:instance_role] == "util" && node[:name] =~ /^(resque)/)
+if resque_instance?
   
   file "/usr/local/bin/resque_kill_stale" do
     owner 'root'

data/recipes/restart.rb

@@ -3,15 +3,14 @@
 # Recipe:: default
 #
 
-if node[:instance_role] == "solo" || node[:instance_role] == "eylocal" || 
-                (node[:instance_role] == "util" && node[:name] =~ /^(resque)/)
+if resque_instance?
   node[:applications].each do |app_name, data|
     execute "ensure-resque-is-setup-with-monit" do 
       command %Q{monit reload}
     end
 
     execute "restart-resque" do 
-      command %Q{echo "sleep 20 && monit -g #{app_name}_resque restart all" | at now }
+      command %Q{echo "sleep 20 && monit -g resque_#{app_name} restart all" | at now }
     end
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: eycloud-recipe-resque
 version: !ruby/object:Gem::Version
-  version: 1.0.3
+  version: 1.1.0
   prerelease: 
 platform: ruby
 authors:
@@ -9,11 +9,11 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2012-03-17 00:00:00.000000000 Z
+date: 2012-03-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: eycloud-helper-cronjobs
-  requirement: &70200712055240 !ruby/object:Gem::Requirement
+  requirement: &70279512580460 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -21,10 +21,10 @@ dependencies:
         version: '0'
   type: :runtime
   prerelease: false
-  version_requirements: *70200712055240
+  version_requirements: *70279512580460
 - !ruby/object:Gem::Dependency
   name: rake
-  requirement: &70200712098120 !ruby/object:Gem::Requirement
+  requirement: &70279512579700 !ruby/object:Gem::Requirement
     none: false
     requirements:
     - - ! '>='
@@ -32,7 +32,7 @@ dependencies:
         version: '0'
   type: :development
   prerelease: false
-  version_requirements: *70200712098120
+  version_requirements: *70279512579700
 description: Resque for EY Cloud
 email:
 - drnicwilliams@gmail.com
@@ -42,8 +42,8 @@ extra_rdoc_files: []
 files:
 - .DS_Store
 - .gitignore
+- ChangeLog.md
 - Gemfile
-- Gemfile.lock
 - README.md
 - Rakefile
 - attributes/default.rb
@@ -51,6 +51,7 @@ files:
 - files/default/resque
 - files/default/resque-web
 - files/default/resque_kill_stale
+- libraries/find_resque_instances.rb
 - libraries/get_resque_worker_count.rb
 - metadata.json
 - metadata.rb
@@ -76,7 +77,7 @@ required_ruby_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -116094860101360678
+      hash: -3022652905535487665
 required_rubygems_version: !ruby/object:Gem::Requirement
   none: false
   requirements:
@@ -85,7 +86,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
       segments:
       - 0
-      hash: -116094860101360678
+      hash: -3022652905535487665
 requirements: []
 rubyforge_project: 
 rubygems_version: 1.8.17

data/Gemfile.lock

@@ -1,18 +0,0 @@
-PATH
-  remote: .
-  specs:
-    eycloud-recipe-resque (1.0.3)
-      eycloud-helper-cronjobs
-
-GEM
-  remote: https://rubygems.org/
-  specs:
-    eycloud-helper-cronjobs (1.0.1)
-    rake (0.9.2.2)
-
-PLATFORMS
-  ruby
-
-DEPENDENCIES
-  eycloud-recipe-resque!
-  rake