newrelic_ia 0.1.0 → 0.2.0

data/CHANGELOG

@@ -1,3 +1,7 @@
+== 0.2.0 2010-04-14
+
+* Pull in memcached instrumentation from rubyorchard/memcached_newrelic_ia
+
 == 0.1.0 2009-06-15
 
 * 1 major enhancement:

data/LICENSE

@@ -0,0 +1,22 @@
+(The MIT License)
+
+Copyright (c) 2009-2010 New Relic, Inc.
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+'Software'), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
+IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
+CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
+TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
+SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.rdoc

@@ -1,23 +1,17 @@
 = New Relic Infrastructure Agent
 
-Bare bones Gem for collecting disk, cpu and IO statistics and 
-sending them to the New Relic RPM application.
-
-This is a work in progress.  We hope to add more statistics
-to the agent as well as more useful views on the RPM site
-for viewing these metrics.
-
-== DESCRIPTION:
+Gem for collecting memcached, disk, cpu and IO statistics.  Collected
+stats are sent to the New Relic RPM application.
 
 Use this gem to collect statistics from a given host by running
 newrelic_ia -a as a daemon.  This will contribute System/* metrics
 to your RPM console which can be viewed with custom dashboards.
 
-== SYNOPSIS:
+== Synopsis
 
     newrelic_ia [ options ] aspect, aspect.. 
 
-    aspect: one or more of 'iostat' or 'disk' (more to come)
+    aspect: one or more of 'iostat' or 'disk' or 'memcached' (more to come)
 
     -a, --all                        use all available aspects
     -v, --verbose                    debug output
@@ -26,15 +20,14 @@ to your RPM console which can be viewed with custom dashboards.
         --install                    create a default newrelic.yml
     -h, --help                       Show this help message.
 
-== REQUIREMENTS:
+== Requirements
 
-* New Relic Agent version 2.9.2 or later
+* New Relic Agent version 2.10.6 or later
 * Gold Subscription for creating and viewing custom dashboards
 
-== INSTALL:
+== Install
 
-    sudo gem sources -a http://gems.github.com
-    sudo gem install newrelic-ia
+    sudo gem install newrelic_ia
 
 Once installed, run from the home directory of your Rails 
 application, or create a separate working directory and run
@@ -44,27 +37,97 @@ application, or create a separate working directory and run
 to create a template newrelic.yml file you can use.  Edit this file
 and substitute your license key and app_name value.
 
-== LICENSE:
-
-(The MIT License)
-
-Copyright (c) 2009 FIXME full name
-
-Permission is hereby granted, free of charge, to any person obtaining
-a copy of this software and associated documentation files (the
-'Software'), to deal in the Software without restriction, including
-without limitation the rights to use, copy, modify, merge, publish,
-distribute, sublicense, and/or sell copies of the Software, and to
-permit persons to whom the Software is furnished to do so, subject to
-the following conditions:
-
-The above copyright notice and this permission notice shall be
-included in all copies or substantial portions of the Software.
+specify memcached nodes in memcached-nodes.txt. 
+
+== Credits
+
+Special thanks to Chandra Patni and Tony Ford of IGN for the Memcached
+sampler.
+
+== Memcached Details
+
+Memcached statistics sampler collects subset of stats provided by
+memcached daemon, once a minute.  In addition it computes other stats
+dervied from memcached stats. Also support stats derivatives over time
+to compute rate etc.  Specify Memcached nodes configuration in the
+newrelic.yml file.
+
+Memcached stats are reported for each node and also sample reports aggregated stats
+
+=== Stats Detail
+
+    uptime                     Number of seconds this server has been running
+    curr_items                 Current number of items stored by the server
+    total_items                Total number of items stored by this server 
+                               ever since it started
+    bytes                      Current number of bytes used by this server to store items
+    curr_connections           Number of open connections
+    total_connections          Total number of connections opened since 
+                               the server started running
+    connection_structures      Number of connection structures allocated by the server
+    cmd_get                    Cumulative number of retrieval requests
+    cmd_set                    Cumulative number of storage requests
+    get_hits                   Number of keys that have been requested and found present
+    get_misses                 Number of items that have been requested and not found
+    evictions                  Number of valid items removed from cache to free 
+                               memory for new items
+    bytes_read                 Total number of bytes read by this server from network
+    bytes_written              Total number of bytes sent by this server to network
+    limit_maxbytes             Number of bytes this server is allowed to use for storage
+    threads                    Number of worker threads requested
+    
+Derived Stats (computed stats):
+
+    free_maxbytes              Number of free bytes this server has available for storage
+
+Stats Derivatives (time derivatives of stats):
+
+    hit_ratio                  Percent of keys that have been requested and found present
+    miss_ratio                 Percent of keys that have been requested and found missing
+    rpm                        Requests per minutes 
+    gpm                        Gets per minutes
+    hpm                        Hits per minutes
+    mpm                        Misses per minutes
+    spm                        Sets per minutes
+    fpm                        Flushes per minutes
+    epm                        Evictions per minutes
+
+
+=== Sample Custom View
+
+Liquid template for NewRelic custom dashboard:
+
+    <div>
+        <div style="float: left; width: 50%;">
+            {% line_chart value:average_value title:'Cache Miss Ratio' metric:'System/Memcached/Miss Ratio' simple_tooltip:true label:segment_3 value_suffix:'%' %}
+        </div>
+        <div style="float: left; width: 50%;">
+            {% line_chart value:average_value title:'Cache Memory' regexp:'System/Memcached/(Bytes|Free Bytes|Limit Maxbytes)' simple_tooltip:true label:segment_3 %}
+        </div>
+        <br style="clear: both" />
+    </div>
+    
+    <div style="padding-top: 20px">&nbsp;</div>
+    
+    <div>
+        <div style="float: left; width: 50%;">
+            {% line_chart value:average_value title:'Cache Gets & Sets' regexp:'System/Memcached/(Gpm|Spm)' simple_tooltip:true label:segment_3 %}
+        </div>
+        <div style="float: left; width: 50%;">
+            {% line_chart value:average_value title:'Cache Misses, Flushes, & Evictions' regexp:'System/Memcached/(Mpm|Epm|Fpm)' simple_tooltip:true label:segment_3 %}
+        </div>
+        <br style="clear: both" />
+    </div>
+    
+    <div style="padding-top: 20px">&nbsp;</div>
+    
+    <div>
+        <div style="float: left; width: 50%;">
+        {% line_chart value:average_value title:'Active Connections' metric:'System/Memcached/Curr Connections' simple_tooltip:true hide_legend:true %}
+        </div>
+        <div style="float: left; width: 50%;">
+        {% line_chart value:average_value title:'Cache Objects' metric:'System/Memcached/Curr Items' simple_tooltip:true hide_legend:true %}
+        </div>
+        <br style="clear: both" />
+    </div>
 
-THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
-EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
-MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
-IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
-CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
-TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
-SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/Rakefile

@@ -1,43 +1,87 @@
-require 'rubygems'
-require 'echoe'
-%w[rake rake/clean fileutils rubigen].each { |f| require f }
-require File.dirname(__FILE__) + '/lib/newrelic_ia'
-
-GEM_NAME = "newrelic_ia"
-GEM_VERSION = NewRelic::IA::VERSION
-AUTHOR = "Bill Kayser"
-EMAIL = "bkayser@newrelic.com"
-HOMEPAGE = "http://www.newrelic.com"
-SUMMARY = "New Relic Gem for gathering system metrics"
-DESCRIPTION = <<EOF
-The New Relic Infrastructure Agent (IA) collects system metrics and transmits
-them to the RPM server where they can be viewed with custom dashboards.
-EOF
-
-# Generate all the Rake tasks
-# Run 'rake -T' to see list of generated tasks (from gem root directory)
-Echoe.new('newrelic_ia', NewRelic::IA::VERSION) do |p|
-  p.author = AUTHOR
-  p.email = EMAIL
-  p.summary = SUMMARY
-  p.description = DESCRIPTION
-  p.url = HOMEPAGE
-  p.project = 'newrelic'
-  p.need_tar_gz = false
-  p.need_gem = true
-  p.runtime_dependencies = [
-     ['newrelic_rpm','>= 2.9.2'],
-  ]
-  p.development_dependencies = [
-    #['newgem', ">= #{::Newgem::VERSION}"]
-  ]
-  p.bin_files = 'bin/newrelic_ia'
-  p.test_pattern = "spec/*.rb"
-  p.install_message = File.read('PostInstall.txt')
-  p.ignore_pattern = %w[PostInstall.txt newrelic.yml]
-  p.clean_pattern |= %w[**/.DS_Store tmp *.log]
-  path = (p.rubyforge_name == p.name) ? p.rubyforge_name : "\#{p.rubyforge_name}/\#{p.name}"
-end
-
-Dir['tasks/**/*.rake'].each { |t| load t }
-
+require 'rubygems'
+require 'rake'
+require File.dirname(__FILE__) + '/lib/new_relic/ia/version.rb'
+require 'rake/testtask'
+
+GEM_NAME = "newrelic_ia"
+GEM_VERSION = NewRelic::IA::VERSION
+AUTHOR = "Bill Kayser"
+EMAIL = "bkayser@newrelic.com"
+HOMEPAGE = "http://www.newrelic.com"
+SUMMARY = "New Relic Gem for gathering system metrics"
+DESCRIPTION = <<EOF
+The New Relic Infrastructure Agent (IA) collects system metrics and transmits
+them to the RPM server where they can be viewed with custom dashboards.
+EOF
+
+# See http://www.rubygems.org/read/chapter/20 
+begin
+  require 'jeweler'
+  Jeweler::Tasks.new do |gem|
+    gem.name = GEM_NAME
+    gem.summary = SUMMARY
+    gem.description = DESCRIPTION
+    gem.email = EMAIL
+    gem.homepage = HOMEPAGE
+    gem.author = AUTHOR
+    gem.version = GEM_VERSION
+    gem.files = FileList['Rakefile', 'README*', 'CHANGELOG', 'spec/**/*','tasks/*', 'lib/**/*'].to_a
+    gem.test_files = FileList['spec/**/*.rb']
+    gem.rdoc_options <<
+      "--line-numbers" <<
+      "--inline-source" <<
+      "--title" << SUMMARY <<
+      "-m" << "README.rdoc"
+    
+    gem.files.reject! { |fn| fn =~ /PostInstall.txt|pkg\/|rdoc\// }
+    gem.extra_rdoc_files = %w[CHANGELOG LICENSE README.rdoc bin/newrelic_ia]
+    gem.add_dependency 'newrelic_rpm', '>=2.10.6'
+    gem.post_install_message = File.read 'PostInstall.txt'
+  end
+  Jeweler::GemcutterTasks.new
+rescue LoadError
+  puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
+end
+
+load "#{File.dirname(__FILE__)}/tasks/rspec.rake"
+
+task :manifest do
+  puts "Manifest task is no longer used since switching to jeweler."
+end
+
+begin
+  require 'rcov/rcovtask'
+  Rcov::RcovTask.new do |test|
+    test.libs << 'test'
+    test.pattern = 'spec/**/*.rb'
+    test.verbose = true
+  end
+rescue LoadError
+  task :rcov do
+    abort "RCov is not available. In order to run rcov, you must: sudo gem install spicycode-rcov"
+  end
+end
+
+require 'spec/rake/spectask'
+Spec::Rake::SpecTask.new(:spec) do |spec|
+  spec.libs << 'lib' << 'spec'
+  spec.spec_files = FileList['spec/**/*_spec.rb']
+end
+
+task :spec => :check_dependencies
+
+task :default => :spec
+
+require 'rake/rdoctask'
+Rake::RDocTask.new do |rdoc|
+  rdoc.title = SUMMARY
+  rdoc.main = "README.rdoc"
+  rdoc.rdoc_files << 'LICENSE' << 'README*' << 'CHANGELOG' << 'lib/**/*.rb' << 'bin/**/*'
+  rdoc.inline_source = true
+end
+
+begin
+  require 'sdoc_helpers'
+rescue LoadError
+  puts "sdoc support not enabled. Please gem install sdoc-helpers."
+end

data/bin/newrelic_ia

@@ -3,10 +3,18 @@
 #  Created on 2009-6-15.
 #  Copyright (c) 2009. All rights reserved.
 
-require File.expand_path(File.dirname(__FILE__) + "/../lib/newrelic_ia")
-
+begin
+  require 'newrelic_ia'
+rescue LoadError
+  require 'rubygems' unless ENV['NO_RUBYGEMS']
+  begin
+    require 'newrelic_ia'
+  rescue Exception => e
+    # No gem.  Must be in development
+    require "#{File.dirname(__FILE__)}/../lib/newrelic_ia"
+  end
+end
 require "new_relic/ia/cli"
-
 status = NewRelic::IA::CLI.execute(STDOUT, ARGV)
 exit status if status
 # if execute returned nil, we are running forever:

data/lib/new_relic/ia/cli.rb

@@ -1,104 +1,157 @@
 require 'optparse'
 require 'logger'
 
-class NewRelic::IA::CLI
-  
-  LOGFILE = "newrelic_ia.log"
-  @log = Logger.new(STDOUT)
+module NewRelic::IA
+
+  class InitError < StandardError;  end
   
-  class << self
-    attr_accessor :log
-    def level= l
-      @log.level = l      
-    end
+  class CLI
     
-    # Run the command line args.  Return nil if running
-    # or an exit status if not.
-    def execute(stdout, arguments=[])
-      @aspects = []
-      @log = Logger.new LOGFILE
-      @log_level = Logger::INFO
-      parser = OptionParser.new do |opts|
-        opts.banner = <<-BANNER.gsub(/^ */,'')
-
+    LOGFILE = "newrelic_ia.log"
+    
+    class << self
+      
+      def log
+        @log ||= Logger.new LOGFILE
+      end
+      
+      def level= l
+        log.level = l
+      end
+      
+      # Run the command line args.  Return nil if running
+      # or an exit status if not.
+      def execute(stdout, arguments=[])
+        @aspects = []
+        parser = OptionParser.new do |opts|
+          opts.banner = <<-BANNER.gsub(/^ */,'')
+          New Relic Infrastructure Agent (IA) version #{NewRelic::IA::VERSION}
           Monitor different aspects of your environment with New Relic RPM.  
 
           Usage: #{File.basename($0)} [ options ] aspect, aspect.. 
 
-          aspect: one or more of 'iostat' or 'disk' (more to come)
+          aspect: one or more of 'memcached', 'iostat' or 'disk' (more to come)
         BANNER
-        opts.separator ""
-        opts.on("-a", "--all",
-                "use all available aspects") { @aspects = %w[iostat disk] }
-        opts.on("-v", "--verbose",
-                "debug output") { @log_level = Logger::DEBUG }
-        opts.on("-q", "--quiet",
-                "quiet output") { @log_level = Logger::ERROR }
-        opts.on("-e", "--environment=ENV",
-                "use ENV section in newrelic.yml") { |e| @env = e }
-        opts.on("--install",
-                "create a default newrelic.yml") { |e| return self.install(stdout) }
-        
-        opts.on("-h", "--help",
-                "Show this help message.") { stdout.puts "#{opts}\n"; return 0 }
-        begin
-          args = opts.parse! arguments
-          unless args.empty?
-            @aspects = args
+          opts.separator ""
+          opts.on("-a", "--all",
+                  "use all available aspects") { @aspects = %w[iostat disk memcached] }
+          opts.on("-v", "--verbose",
+                  "debug output") { NewRelic::IA::CLI.log.level = Logger::DEBUG }
+          opts.on("-q", "--quiet",
+                  "quiet output") { NewRelic::IA::CLI.log.level = Logger::ERROR }
+          opts.on("-e", "--environment=ENV",
+                  "use ENV section in newrelic.yml") { |e| @env = e }
+          opts.on("--install",
+                  "create a default newrelic.yml") { |e| return self.install(stdout) }
+          
+          opts.on("-h", "--help",
+                  "Show this help message.") { stdout.puts "#{opts}\n"; return 0 }
+          begin
+            args = opts.parse! arguments
+            unless args.empty?
+              @aspects = args
+            end
+          rescue => e
+            stdout.puts e
+            stdout.puts opts
+            return 1
+          end
+        end
+        @aspects.delete_if do |aspect|
+          unless self.instance_methods(false).include? aspect
+            stdout.puts "Unknown aspect: #{aspect}"
+            true
           end
-        rescue => e
-          stdout.puts e
-          stdout.puts opts
+        end
+        if @aspects.empty?
+          stdout.puts "No aspects specified."
+          stdout.puts parser
           return 1
         end
-      end
-      @aspects.delete_if do |aspect|
-        unless self.instance_methods(false).include? aspect
-          stdout.puts "Unknown aspect: #{aspect}"
-          true
+        require_newrelic_rpm        
+        NewRelic::Agent.manual_start  :env => @env, :monitor_mode => true, :log => self.log
+        # connected? due in a future version
+        if not (NewRelic::Agent.instance.connected? rescue true)
+          raise InitError, "Unable to connect to RPM server.  Agent not started."
         end
-      end
-      if @aspects.empty?
-        stdout.puts "No aspects specified."
-        stdout.puts parser
+        cli = new
+        @aspects.each do | aspect |
+          cli.send aspect
+        end
+        return nil
+      rescue InitError => e
+        stdout.puts e.message
         return 1
       end
       
-      @log.level = @log_level 
-      gem 'newrelic_rpm'
-      require 'newrelic_rpm'
-      NewRelic::Agent.manual_start :log => @log, :env => @env, :enabled => true
-      cli = new
-      @aspects.each do | aspect |
-        cli.send aspect
+    end
+    # Aspect definitions
+    def iostat # :nodoc:
+      self.class.log.info "Starting iostat monitor..."
+      require 'new_relic/ia/iostat_reader'
+      reader = NewRelic::IA::IostatReader.new
+      Thread.new { reader.run }
+    end
+    
+    def disk
+      self.class.log.info "Starting disk sampler..."
+      require 'new_relic/ia/disk_sampler'
+      NewRelic::Agent.instance.stats_engine.add_harvest_sampler NewRelic::IA::DiskSampler.new    
+    end
+    
+    def memcached
+      require 'new_relic/ia/memcached_sampler'
+      s = NewRelic::IA::MemcachedSampler.new
+      s.check
+      NewRelic::Agent.instance.stats_engine.add_harvest_sampler s
+    end
+
+    private
+    
+    def log
+      self.class.log
+    end
+    
+    def self.require_newrelic_rpm
+      begin
+        require 'newrelic_rpm'
+      rescue Exception => e
+        begin
+          require 'rubygems' unless ENV['NO_RUBYGEMS']
+          require 'newrelic_rpm'
+        rescue LoadError
+          $stderr.puts "Unable to load required gem newrelic_rpm"
+          $stderr.puts "Try `gem install newrelic_rpm`"
+          Kernel.exit 1
+        end
       end
-      return nil
     end
-  end
-  # Aspect definitions
-  def iostat # :nodoc:
-    self.class.log.info "Starting iostat monitor..."
-    require 'new_relic/ia/iostat_reader'
-    reader = NewRelic::IA::IostatReader.new
-    Thread.new { reader.run }
-  end
-  
-  def disk
-    self.class.log.info "Starting disk sampler..."
-    require 'new_relic/ia/disk_sampler'
-    NewRelic::Agent.instance.stats_engine.add_harvest_sampler NewRelic::IA::DiskSampler.new    
-  end
-  
-  private 
-  def self.install(stdio)
-    if File.exists? "newrelic.yml"
-      stdio.puts "A newrelic.yml file already exists.  Please remove it before installing another."
-      1 # error
-    else      
-      FileUtils.copy File.join(File.dirname(__FILE__), "newrelic.yml"), "."
-      stdio.puts "A newrelic.yml template was copied to #{File.expand_path('.')}."
-      stdio.puts "Please add a license key to the file before starting."
-      0 # normal
+    
+    def self.install stdout
+      require_newrelic_rpm
+      if NewRelic::VersionNumber.new(NewRelic::VERSION::STRING) < '2.12'
+        if File.exists? "newrelic.yml"
+          stdout.puts "A newrelic.yml file already exists.  Please remove it before installing another."
+          return 1 # error
+        else      
+          FileUtils.copy File.join(File.dirname(__FILE__), "newrelic.yml"), "."
+          stdout.puts "A newrelic.yml template was copied to #{File.expand_path('.')}."
+          stdout.puts "Please add a license key to the file before starting."
+          return 0 # normal
+        end
+      else
+        begin
+          require 'new_relic/command'
+          cmd = NewRelic::Command::Install.new \
+          :src_file => File.join(File.dirname(__FILE__), "newrelic.yml"),
+          :generated_for_user => "Generated on #{Time.now.strftime('%b %d, %Y')}, from version #{NewRelic::IA::VERSION}"
+          cmd.run 
+          0 # normal
+        rescue NewRelic::Command::CommandFailure => e
+          stdout.puts e.message
+          1 # error
+        end
+      end
     end
   end
-end
+end