flyerhzm-bullet 1.0.0

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2009 Richard Huang (flyerhzm@gmail.com)
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.textile

@@ -0,0 +1,243 @@
+h1. Bullet
+
+This plugin is aimed to give you some performance suggestion about ActiveRecord usage,  what should use but not use, such as eager loading, counter cache and so on, what should not use but use, such as unused eager loading.
+
+Now it provides you the suggestion of eager loading and unused eager loading.
+
+The others are todo, next may be couter cache.
+
+****************************************************************************
+
+h2. Install
+
+install as gem:
+<pre><code>
+gem sources -a http://gems.github.com
+gem install flyerhzm-bullet
+</code></pre>
+
+install as plugin:
+<pre><code>script/plugin install git://github.com/flyerhzm/bullet.git</code></pre>
+
+****************************************************************************
+
+h2. Usage
+
+* "Eager Loading, protect N+1 query and protect from unused eager loading":#EagerLoading
+
+****************************************************************************
+
+h2. Step by step example
+
+* "Eager Loading, protect N+1 query and protect from unused eager loading":#EagerLoadingExample
+
+
+****************************************************************************
+
+h3. Eager Loading, protect N+1 query and protect from unused eager loading, usage
+<a id="EagerLoading"/></a>
+
+*important*: It is strongly recommended to disable cache in browser.
+
+* add configuration to environment 
+<pre><code>
+Bullet.enable = true
+Bullet::Association.logger = true 
+Bullet::Association.alert = true 
+</code></pre>
+** Bullet.enable (required), if enable is true (default is false), Bullet plugin is enabled. Otherwise, Bullet plugin is disabled.
+** Bullet::Association.logger (optional), if logger is true (default is true), the N+1 query hints will be appended to <code>log/bullet.log</code> with N+1 query method call stack. Otherwise, no hint to log/bullet.log.
+** Bullet::Association.alert (optional), if alert is true (default value), alert box will popup if there is N+1 query when browsing web page. Otherwise, no alert box.
+
+* browse the webpage, if there are N+1 queries or unused eager loading, alert box and bullet log will generate according to configurations. Alert box will only popup when the request's Content-Type is text/html, and <code>log/bullet.log</code> will produce whatever the request is.
+
+* example of <code>log/bullet.log</code>
+<pre><code>
+2009-08-25 20:40:17[INFO] N+1 Query: PATH_INFO: /posts;    model: Post => associations: [comments]·
+Add to your finder: :include => [:comments]
+2009-08-25 20:40:17[INFO] N+1 Query: method call stack:·
+/Users/richard/Downloads/test/app/views/posts/index.html.erb:11:in `_run_erb_app47views47posts47index46html46erb'
+/Users/richard/Downloads/test/app/views/posts/index.html.erb:8:in `each'
+/Users/richard/Downloads/test/app/views/posts/index.html.erb:8:in `_run_erb_app47views47posts47index46html46erb'
+/Users/richard/Downloads/test/app/controllers/posts_controller.rb:7:in `index'
+</code></pre>
+It represents that in request '/posts', there is a N+1 query from Post to comments. It means you may have a logic code in controller <code>@posts = Post.find(:all)</code> should be changed to <code>@posts = Post.find(:all, :include => :comments)</code>
+<pre><code>
+2009-08-25 20:53:56[INFO] Unused preload associations: PATH_INFO: /posts;    model: Post => associations: [comments]·
+Remove from your finder: :include => [:comments]
+</code></pre>
+It represents that in request '/posts', there is a used eager loading from Post to comments. It means you may have a logic code in controller <code>@posts = Post.find(:all, :include => :comments)</code> should be changed to <code>@posts = Post.find(:all)</code>
+
+* To see what causes N+1 queries, check the <code>spec/bullet_association_spec.rb</code>
+
+* Rake tasks
+<code>rake bullet:log:clear</code>, clear the <code>log/bullet.log</code>
+
+****************************************************************************
+
+h3. Eager Loading, protect N+1 query and protect from unused eager loading, step by step example
+<a id="EagerLoadingExample"/></a>
+
+1. setup test environment
+
+<pre><code>
+$ rails test
+$ cd test
+$ script/generate scaffold post name:string 
+$ script/generate scaffold comment name:string post_id:integer
+$ rake db:migrate
+</code></pre>
+
+2. change <code>app/model/post.rb</code> and <code>app/model/comment.rb</code>
+
+<pre><code>
+class Post < ActiveRecord::Base
+  has_many :comments
+end
+
+class Comment < ActiveRecord::Base
+  belongs_to :post
+end
+</code></pre>
+
+3. go to script/console and execute
+
+<pre><code>
+post1 = Post.create(:name => 'first')
+post2 = Post.create(:name => 'second')
+post1.comments.create(:name => 'first')
+post1.comments.create(:name => 'second')
+post2.comments.create(:name => 'third')
+post2.comments.create(:name => 'fourth')
+</code></pre>
+
+4. change the <code>app/views/posts/index.html.erb</code> to produce a N+1 query
+
+<pre><code>
+<% @posts.each do |post| %>
+  <tr>
+    <td><%=h post.name %></td>
+    <td><%= post.comments.collect(&:name) %></td>
+    <td><%= link_to 'Show', post %></td>
+    <td><%= link_to 'Edit', edit_post_path(post) %></td>
+    <td><%= link_to 'Destroy', post, :confirm => 'Are you sure?', :method => :delete %></td>
+  </tr>
+<% end %>
+</code></pre>
+
+5. add bullet plugin
+
+<pre><code>
+$ script/plugin install git://github.com/flyerhzm/bullet.git
+</code></pre>
+
+6. enable the bullet plugin in development, add a line to <code>config/environments/development.rb</code>
+
+<pre><code>
+Bullet.enable = true
+</code></pre>
+
+7. start server
+
+<pre><code>
+$ script/server
+</code></pre>
+
+8. input http://localhost:3000/posts in browser, then you will see a popup alert box says
+
+<pre><code>
+The request has unused preload associations as follows:
+None
+The request has N+1 queries as follows:
+model: Post => associations: [comment]
+</code></pre>
+
+which means there is a N+1 query from post object to comments associations.
+
+In the meanwhile, there's a log appended into <code>log/bullet.log</code> file
+
+<pre><code>
+2009-08-20 09:12:19[INFO] N+1 Query: PATH_INFO: /posts;    model: Post => assocations: [comments]
+Add your finder: :include => [:comments]
+2009-08-20 09:12:19[INFO] N+1 Query: method call stack:
+/Users/richard/Downloads/test/app/views/posts/index.html.erb:11:in `_run_erb_app47views47posts47index46html46erb'
+/Users/richard/Downloads/test/app/views/posts/index.html.erb:8:in `each'
+/Users/richard/Downloads/test/app/views/posts/index.html.erb:8:in `_run_erb_app47views47posts47index46html46erb'
+/Users/richard/Downloads/test/app/controllers/posts_controller.rb:7:in `index'
+</code></pre>
+
+The generated SQLs are
+
+<pre><code>
+  Post Load (1.0ms)   SELECT * FROM "posts" 
+  Comment Load (0.4ms)   SELECT * FROM "comments" WHERE ("comments".post_id = 1) 
+  Comment Load (0.3ms)   SELECT * FROM "comments" WHERE ("comments".post_id = 2) 
+</code></pre>
+
+
+9. fix the N+1 query, change <code>app/controllers/posts_controller.rb</code> file
+
+<pre><code>
+  def index
+    @posts = Post.find(:all, :include => :comments)
+
+    respond_to do |format|
+      format.html # index.html.erb
+      format.xml  { render :xml => @posts }
+    end 
+  end 
+</code></pre>
+
+10. refresh http://localhost:3000/posts page, no alert box and no log appended.
+
+The generated SQLs are
+
+<pre><code>
+  Post Load (0.5ms)   SELECT * FROM "posts" 
+  Comment Load (0.5ms)   SELECT "comments".* FROM "comments" WHERE ("comments".post_id IN (1,2)) 
+</code></pre>
+
+a N+1 query fixed. Cool!
+
+11. now simulate unused eager loading. Change <code>app/controllers/posts_controller.rb</code> and <code>app/views/posts/index.html.erb</code>
+
+<pre><code>
+  def index
+    @posts = Post.find(:all, :include => :comments)
+
+    respond_to do |format|
+      format.html # index.html.erb
+      format.xml  { render :xml => @posts }
+    end 
+  end 
+</code></pre>
+
+<pre><code>
+<% @posts.each do |post| %>
+  <tr>
+    <td><%=h post.name %></td>
+    <td><%= link_to 'Show', post %></td>
+    <td><%= link_to 'Edit', edit_post_path(post) %></td>
+    <td><%= link_to 'Destroy', post, :confirm => 'Are you sure?', :method => :delete %></td>
+  </tr>
+<% end %>
+</code></pre>
+
+12. refresh http://localhost:3000/posts page, then you will see a popup alert box says
+
+<pre><code>
+The request has unused preload associations as follows:
+model: Post => associations: [comment]
+The request has N+1 queries as follows:
+None
+</code></pre>
+
+In the meanwhile, there's a log appended into <code>log/bullet.log</code> file
+
+<pre><code>
+2009-08-25 21:13:22[INFO] Unused preload associations: PATH_INFO: /posts;    model: Post => associations: [comments]·
+Remove from your finder: :include => [:comments]
+</code></pre>
+
+
+Copyright (c) 2009 Richard Huang (flyerhzm@gmail.com), released under the MIT license

data/Rakefile

@@ -0,0 +1,32 @@
+require 'rake'
+require 'spec/rake/spectask'
+require 'rake/rdoctask'
+require 'jeweler'
+
+desc 'Default: run unit tests.'
+task :default => :spec
+
+desc 'Generate documentation for the sitemap plugin.'
+Rake::RDocTask.new(:rdoc) do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title    = 'Bullet'
+  rdoc.options << '--line-numbers' << '--inline-source'
+  rdoc.rdoc_files.include('README')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+desc "Run all specs in spec directory"
+Spec::Rake::SpecTask.new(:spec) do |t|
+  t.spec_files = FileList['spec/spec_helper.rb', 'spec/**/*_spec.rb']
+end
+
+Jeweler::Tasks.new do |gemspec|
+  gemspec.name = "bullet"
+  gemspec.summary = "A plugin to kill N+1 queries and unused eager loading"
+  gemspec.description = "This plugin is aimed to give you some performance suggestion about ActiveRecord usage, what should use but not use, such as eager loading, counter cache and so on, what should not use but use, such as unused eager loading. Now it provides you the suggestion of eager loading and unused eager loading. The others are todo, next may be couter cache."
+  gemspec.email = "flyerhzm@gmail.com"
+  gemspec.homepage = "http://www.huangzhimin.com/projects/4-bullet"
+  gemspec.authors = ["Richard Huang"]
+  gemspec.files.exclude '.gitignore'
+  gemspec.files.exclude 'log/'
+end

data/VERSION

@@ -0,0 +1 @@
+1.0.0

data/bullet.gemspec

@@ -0,0 +1,54 @@
+# Generated by jeweler
+# DO NOT EDIT THIS FILE
+# Instead, edit Jeweler::Tasks in Rakefile, and run `rake gemspec`
+# -*- encoding: utf-8 -*-
+
+Gem::Specification.new do |s|
+  s.name = %q{bullet}
+  s.version = "1.0.0"
+
+  s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
+  s.authors = ["Richard Huang"]
+  s.date = %q{2009-08-26}
+  s.description = %q{This plugin is aimed to give you some performance suggestion about ActiveRecord usage, what should use but not use, such as eager loading, counter cache and so on, what should not use but use, such as unused eager loading. Now it provides you the suggestion of eager loading and unused eager loading. The others are todo, next may be couter cache.}
+  s.email = %q{flyerhzm@gmail.com}
+  s.extra_rdoc_files = [
+    "README.textile"
+  ]
+  s.files = [
+    "MIT-LICENSE",
+     "README.textile",
+     "Rakefile",
+     "VERSION",
+     "bullet.gemspec",
+     "lib/bullet.rb",
+     "lib/bullet/association.rb",
+     "lib/bullet/logger.rb",
+     "lib/bulletware.rb",
+     "lib/hack/active_record.rb",
+     "rails/init.rb",
+     "spec/bullet_association_spec.rb",
+     "spec/spec.opts",
+     "spec/spec_helper.rb",
+     "tasks/bullet_tasks.rake"
+  ]
+  s.homepage = %q{http://www.huangzhimin.com/projects/4-bullet}
+  s.rdoc_options = ["--charset=UTF-8"]
+  s.require_paths = ["lib"]
+  s.rubygems_version = %q{1.3.5}
+  s.summary = %q{A plugin to kill N+1 queries and unused eager loading}
+  s.test_files = [
+    "spec/spec_helper.rb",
+     "spec/bullet_association_spec.rb"
+  ]
+
+  if s.respond_to? :specification_version then
+    current_version = Gem::Specification::CURRENT_SPECIFICATION_VERSION
+    s.specification_version = 3
+
+    if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
+    else
+    end
+  else
+  end
+end

data/lib/bullet/association.rb

@@ -0,0 +1,150 @@
+module Bullet
+  class Association
+    class <<self
+      @@logger_file = File.open(Bullet::BulletLogger::LOG_FILE, 'a+')
+      @@logger = Bullet::BulletLogger.new(@@logger_file)
+      @@alert = true
+      
+      def start_request
+        # puts "start request"
+        @@object_associations ||= {}
+        @@call_object_associations ||= {}
+        @@unpreload_associations ||= {}
+        @@unused_preload_associations ||= {}
+        @@callers ||= []
+        @@possible_objects ||= {}
+        @@impossible_objects ||= {}
+      end
+
+      def end_request
+        # puts "end request"
+        @@object_associations = nil
+        @@unpreload_associations = nil
+        @@unused_preload_associations = nil
+        @@callers = nil
+        @@possible_objects = nil
+        @@impossible_objects = nil
+        @@call_object_associations = nil
+      end
+
+      def alert=(alert)
+        @@alert = alert
+      end
+
+      def logger=(logger)
+        if logger == false
+          @@logger = nil
+        end
+      end
+      
+      def check_unused_preload_associations
+        @@object_associations.each do |object, association|
+          call_association = @@call_object_associations[object] || []
+          association.uniq! unless association.flatten!.nil?
+          call_association.uniq! unless call_association.flatten!.nil?
+          klazz = object.class
+          unless (association - call_association).empty?
+            @@unused_preload_associations[klazz] ||= []
+            @@unused_preload_associations[klazz] << (association - call_association) 
+            @@unused_preload_associations[klazz].flatten!.uniq!
+          end
+        end
+      end
+      
+      def has_bad_assocations?
+        check_unused_preload_associations
+        has_unpreload_associations? or has_unused_preload_associations?
+      end
+
+      def has_unused_preload_associations?
+        !@@unused_preload_associations.empty?
+      end
+      
+      def has_unpreload_associations?
+        !@@unpreload_associations.empty?
+      end
+
+      def bad_associations_alert
+        str = ''
+        if @@alert
+          str = "<script type='text/javascript'>"
+          str << "alert('The request has unused preload assocations as follows:\\n"
+          str << (has_unused_preload_associations? ? @@unused_preload_associations.to_a.collect{|klazz, associations| "model: #{klazz} => associations: [#{associations.join(', ')}]"}.join('\\n') : "None")
+          str << "\\nThe request has N+1 queries as follows:\\n"
+          str << (has_unpreload_associations? ? @@unpreload_associations.to_a.collect{|klazz, associations| "model: #{klazz} => associations: [#{associations.join(', ')}]"}.join('\\n') : "None")
+          str << "')"
+          str << "</script>\n"
+        end
+        str
+      end
+
+      def log_bad_associations(path)
+        if @@logger
+          @@unused_preload_associations.each do |klazz, associations|
+            @@logger.info "Unused preload associations: PATH_INFO: #{path};    model: #{klazz} => associations: [#{associations.join(', ')}] \n Remove from your finder: :include => #{associations.map{|a| a.to_sym}.inspect}"
+          end
+          @@unpreload_associations.each do |klazz, associations| 
+            @@logger.info "N+1 Query: PATH_INFO: #{path};    model: #{klazz} => associations: [#{associations.join(', ')}] \n Add to your finder: :include => #{associations.map{|a| a.to_sym}.inspect}"
+          end  
+          @@callers.each do |c|
+            @@logger.info "N+1 Query: method call stack: \n" + c.join("\n")
+          end
+          @@logger_file.flush
+        end
+      end
+
+      def has_klazz_association(klazz)
+        !@@klazz_associations[klazz].nil? and @@klazz_associations.keys.include?(klazz)
+      end
+      
+      def define_association(klazz, associations)
+        # puts "define association, #{klazz} => #{associations}"
+        @@klazz_associations ||= {}
+        @@klazz_associations[klazz] ||= []
+        @@klazz_associations[klazz] << associations
+      end
+
+      def add_possible_objects(objects)
+        # puts "add possible object, #{objects}"
+        klazz= objects.first.class
+        @@possible_objects[klazz] ||= []
+        @@possible_objects[klazz] << objects
+        @@possible_objects[klazz].flatten!.uniq!
+      end
+      
+      def add_impossible_object(object)
+        # puts "add impossible object, #{object}"
+        klazz = object.class
+        @@impossible_objects[klazz] ||= []
+        @@impossible_objects[klazz] << object
+      end
+
+      def add_association(object, associations)
+        # puts "add association, #{object} => #{associations}"
+        @@object_associations[object] ||= []
+        @@object_associations[object] << associations
+      end
+
+      def call_association(object, associations)
+        # puts "call association, #{object} => #{associations}"
+        klazz = object.class
+        @@possible_objects ||= {}
+        @@impossible_objects ||= {}
+        if (!@@possible_objects[klazz].nil? and @@possible_objects[klazz].include?(object)) and (@@impossible_objects[klazz].nil? or !@@impossible_objects[klazz].include?(object)) and (@@object_associations[object].nil? or !@@object_associations[object].include?(associations))
+          @@unpreload_associations[klazz] ||= []
+          @@unpreload_associations[klazz] << associations
+          @@unpreload_associations[klazz].uniq!
+          @@call_object_associations[object] ||= []
+          @@call_object_associations[object] << associations
+          caller_in_project
+        end
+      end
+      
+      VENDOR_ROOT = File.join(RAILS_ROOT, 'vendor')
+      def caller_in_project
+        @@callers << caller.select {|c| c =~ /#{RAILS_ROOT}/}.reject {|c| c =~ /#{VENDOR_ROOT}/}
+        @@callers.uniq!
+      end
+    end
+  end
+end

data/lib/bullet/logger.rb

@@ -0,0 +1,9 @@
+module Bullet
+  class BulletLogger < Logger
+    LOG_FILE = ::RAILS_ROOT + '/log/bullet.log'
+
+    def format_message(severity, timestamp, progname, msg)
+      "#{timestamp.to_formatted_s(:db)}[#{severity}] #{msg}\n"
+    end
+  end
+end

data/lib/bullet.rb

@@ -0,0 +1,14 @@
+module Bullet
+  class <<self
+    def enable=(enable)
+      @@enable = enable
+    end
+
+    def enable?
+      class_variables.include?('@@enable') and @@enable == true
+    end
+  end
+
+  autoload :Association, 'bullet/association'
+  autoload :BulletLogger, 'bullet/logger'
+end

data/lib/bulletware.rb

@@ -0,0 +1,25 @@
+class Bulletware
+  def initialize(app)
+    @app = app
+  end
+
+  def call(env)
+    return @app.call(env) unless Bullet.enable?
+
+    Bullet::Association.start_request
+    status, headers, response = @app.call(env)
+    return [status, headers, response] if response.empty?
+
+    if Bullet::Association.has_bad_assocations?
+      if !headers['Content-Type'].nil? and headers['Content-Type'].include? 'text/html'
+        response_body = response.body.insert(-17, Bullet::Association.bad_associations_alert)
+        headers['Content-Length'] = response_body.length.to_s
+      end
+
+      Bullet::Association.log_bad_associations(env['PATH_INFO'])
+    end
+    response_body ||= response.body
+    Bullet::Association.end_request
+    [status, headers, response_body]
+  end
+end

data/lib/hack/active_record.rb

@@ -0,0 +1,95 @@
+if Bullet.enable?
+  ActiveRecord::ActiveRecordError # An ActiveRecord bug
+
+  module ActiveRecord
+    class Base
+      class <<self
+        # if select a collection of objects, then these objects have possible to cause N+1 query
+        # if select only one object, then the only one object has impossible to cause N+1 query
+        alias_method :origin_find_every, :find_every
+        
+        def find_every(options)
+          records = origin_find_every(options)
+
+          if records 
+            if records.size > 1
+              Bullet::Association.add_possible_objects(records)
+            elsif records.size == 1
+              Bullet::Association.add_impossible_object(records.first)
+            end
+          end
+
+          records
+        end
+      end
+    end
+
+    module AssociationPreload
+      module ClassMethods
+        # add include for one to many associations query
+        alias_method :origin_preload_associations, :preload_associations
+        
+        def preload_associations(records, associations, preload_options={})
+          records = [records].flatten.compact.uniq
+          return if records.empty?
+          records.each do |record|
+            Bullet::Association.add_association(record, associations)
+          end
+          origin_preload_associations(records, associations, preload_options={})
+        end
+      end
+    end
+    
+    module Associations
+      module ClassMethods
+        # define one to many associations
+        alias_method :origin_collection_reader_method, :collection_reader_method
+        
+        def collection_reader_method(reflection, association_proxy_class)
+          Bullet::Association.define_association(self, reflection.name)
+          origin_collection_reader_method(reflection, association_proxy_class)
+        end
+      end
+      
+      class AssociationCollection
+        # call one to many associations
+        alias_method :origin_load_target, :load_target
+
+        def load_target
+          Bullet::Association.call_association(@owner, @reflection.name)
+          origin_load_target
+        end  
+      end
+
+      class HasOneAssociation
+        # call has_one association
+        alias_method :origin_find_target, :find_target
+
+        def find_target
+          Bullet::Association.call_association(@owner, @reflection.name)
+          origin_find_target
+        end
+      end
+
+      class BelongsToAssociation
+        # call belongs_to association
+        alias_method :origin_find_target, :find_target
+
+        def find_target
+          Bullet::Association.call_association(@owner, @reflection.name)
+          origin_find_target
+        end
+      end
+
+      class BelongsToPolymorphicAssociation
+        # call belongs_to association
+        alias_method :origin_find_target, :find_target
+
+        def find_target
+          Bullet::Association.call_association(@owner, @reflection.name)
+          origin_find_target
+        end
+      end
+    end
+  end
+end

data/rails/init.rb

@@ -0,0 +1,4 @@
+# Include hook code here
+require 'bullet'
+require 'hack/active_record'
+ActionController::Dispatcher.middleware.use Bulletware