mouth 0.8.0

data/.gitignore

@@ -0,0 +1,7 @@
+.rvmrc
+*.log
+*.pid
+log/*.log
+log/*.pid
+Gemfile.lock
+

data/Capfile

@@ -0,0 +1,26 @@
+load 'deploy'
+
+# Simply way to deploy the app if you're developing it
+# I will probably remove this file later.
+set :application, "mouth"
+default_run_options[:pty] = true 
+
+set :scm, :none
+role :app, "sugar"
+
+task :omg do
+  res = `gem build mouth.gemspec`
+  if res =~ /Success/
+    home = "#{capture('pwd').strip}/mouth"
+    file = res.match(/File: (.+)$/)[1]
+    run "mkdir -p ~/mouth"
+    loc = "#{home}/#{file}"
+    upload file, loc
+    sudo "gem install #{loc}"
+    run "mouth-endoscope -K"
+    run "mouth --pidfile #{home}/mouth.pid -K"
+    run "sleep 5"
+    run "mouth-endoscope --mongohost 10.36.103.70"
+    run "mouth --pidfile #{home}/mouth.pid --logfile #{home}/mouth.log -H 10.18.23.135 -P 8889 --mongohost 10.36.103.70"
+  end
+end

data/Gemfile

@@ -0,0 +1,3 @@
+source "http://rubygems.org"
+
+gemspec

data/MIT-LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2011 Jonathan Novak - http://github.com/cypriss
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,138 @@
+# Mouth
+
+Mouth is a Ruby daemon that collects stats via UDP and stores them in Mongo.  It comes with a modern UI that allows you to view graphs and create dashboards of these statistics.  Mouth is very similar to [StatsD](https://github.com/etsy/statsd) + [Graphite](http://graphite.wikidot.com/) + [Graphene](http://jondot.github.com/graphene/).
+
+
+## Why
+
+Why duplicate effort of the excellent StatsD / Graphite packages?  I wanted a graphing and monitoring tool that offered:
+
+
+* **Accessible data**.  Because the metrics are stored in Mongo, they are very accessible to any scripts you want to write to read and act on your data.
+* **Easy installation**.  Mouth depends on just Ruby and Mongo, both incredibly easy to install.
+* **Modern, friendly UI**.  Mouth graphs are beautiful and easy to work with.  Exploring data and building dashboards are a joy.
+
+## Kinds of Metrics
+
+There are two kinds of metrics currently: counters and timers.  Both are stored in a time series with minute granularity.
+
+* **Counters**:
+  * how many occurances of something happen per minute?
+  * Can be sampled
+  * Standard usage: Mouth.increment("myapp.happenings")
+  * Advanced usage: Mouth.increment("myapp.happenings", 1, 0.1) # 1/10 sample rate.  UDP packets are sent 1 in 10 times, but count for 1 * 10 each.
+  * Gives you a time series of happenings by minute: {"myapp.happenings" => [1, 2, 5, 20, ...]}
+* **Timers**:
+  * How long does something take?
+  * Standard usage: Mouth.measure("myapp.occurances", 3.3) # This occurance took 3.3 ms to occur
+  * Standard usage 2: Mouth.measure("myapp.occurances") { do_occurance() }
+  * Gives you a time series of occurance timings by minute: {"myapp.happenings" => [1, 2, 5, 20, ...]}
+* **Gauges**: (Coming soon)
+
+## Installation on OSX
+
+Install mouth:
+
+    gem install mouth
+
+Install MongoDB if you haven't:
+
+    brew install mongodb
+
+Start collector:
+
+    mouth
+
+Start web UI:
+
+    mouth-endoscope
+
+Record a metric:
+
+    ruby -e 'require "mouth"; require "mouth/instrument"; Mouth.increment("gorets")'
+
+To load the web UI, go to http://0.0.0.0:5678/ (or whatever port got chosen -- see the Terminal).  Click 'Add Graph' in the lower right-hand corner.
+
+## Installation in Production
+
+You'll want to follow the general gist of what you did for OSX, but make sure to specify your hosts, ports, and log locations.
+NOTE: there is no config file -- all options are via command-line.
+
+    sudo gem install mouth
+    mouth --pidfile /path/to/log/mouth.pid --logfile /path/to/log/mouth.log -H x.x.x.x -P 8889 --mongohost y.y.y.y --verbosity 1
+    mouth-endoscope --mongohost x.x.x.x
+
+## Instrumenting Your Application to Record Metrics
+
+There are many ways to instrument your application:
+
+### Using the mouth gem
+
+Mouth comes with a built-in facility to instrument your apps:
+    
+    require 'mouth'
+    require 'mouth/instrument'
+    
+    Mouth.server = "0.0.0.0:8889"
+    Mouth.increment('hello.world')
+    Mouth.measure('hello.happening', 42.9)
+    
+### Using mouth-instrument
+  
+mouth-instrument is a lightweight gem that doesn't have the baggage of the various gems that come with mouth. Its usage is nearly identical:
+    
+    gem install mouth-instrument
+
+    require 'mouth-instrument'
+    
+    Mouth.server = "0.0.0.0:8889"
+    Mouth.increment('hello.world')
+    Mouth.measure('hello.happening', 42.9)
+    
+### Using any StatsD instrumentation
+  
+Mouth is StatsD compatible -- if you've instrumented your application to record StatsD metrics, it should work on Mouth.  Just replace your StatsD server with a mouth process.
+
+## Accessing Your Data Via Scripts
+
+You can access and act on your metrics quite easily.
+
+  require 'mouth'
+  require 'mouth/sequence'
+  
+  Mouth.host = "0.0.0.0:8889"
+  Sequence.new("exceptions.app", :kind => :counter).sequence
+  # => [4, 9, 0, ...]
+  
+  Sequence.new("app.requests", :kind => :timer, :granularity_in_minutes => 15, :start_time => Time.now - 86400, :end_time => Time.now).sequence
+  # => [{:count => 3, :min => 1, :max => 30, :mean => 17.0, :sum => 51.0, :median => 20, :stddev => 12.02}, ...]
+
+## Tech
+
+* **Ruby** - 1.9.2+ is required.  Ruby was chosen because many Ruby shops already have it deployed as part of their infrastucture.  By putting everything in Ruby, node + python aren't needed.
+* **MongoDB** -  Mouth stores metrics in Mongo.  Mongo was chosen for 3 reasons:
+  * It's very easy to install and get going
+  * It has drivers for everything, so getting at your data is super easy
+  * It's schemaless design is fairly good for storing time series metrics.
+* **EventMachine** - Mouth is powered by EM, the Ruby way of doing nonblocking IO.
+* **Sinatra** - The web UI is served with a simple Sinatra app.
+* **Backbone.js** - The web UI is powered by Backbone.js
+* **D3.js** - The graphs are powered by D3.js
+
+
+## Contributing
+
+You're interested in contributing to Mouth? *AWESOME*. Here are the basic steps:
+
+fork Mouth from here: http://github.com/cypriss/mouth
+
+1. Clone your fork
+2. Hack away
+3. If you are adding new functionality, document it in the README
+4. If necessary, rebase your commits into logical chunks, without errors
+5. Push the branch up to GitHub
+6. Send a pull request to the cypriss/mouth project.
+
+## Thanks
+
+Thanks to UserVoice.com for sponsoring this project.  Thanks to the [StatsD](https://github.com/etsy/statsd) project for massive inspiration.  Other contributors: https://github.com/cypriss/mouth/graphs/contributors

data/Rakefile

@@ -0,0 +1,19 @@
+require 'rubygems'
+require 'bundler'
+begin
+  Bundler.setup(:default, :development)
+rescue Bundler::BundlerError => e
+  $stderr.puts e.message
+  $stderr.puts "Run `bundle install` to install missing gems"
+  exit e.status_code
+end
+require 'rake'
+
+require 'rake/testtask'
+Rake::TestTask.new(:test) do |test|
+  test.libs << 'lib' << 'test'
+  test.pattern = 'test/*_test.rb'
+  test.verbose = true
+end
+
+task :default => :test

data/TODO

@@ -0,0 +1,32 @@
+Next
+ - Cookie-ize the timespan settings
+ - Re implement graph autoplacement
+ - Better backend impl of source list
+ - Figure out how dashboards can scroll
+ - Show period averages / rates
+ - Full keyboard navigation
+ - Build in some mechanisms to diagnose mongo connection issues
+ - Auto refreshing source list (every minute?)
+ - More Tests
+ - Figure out time zones
+
+Someday
+ - Picker:
+   - When you re-pick sources, it should scroll to the existing picked items
+   - Selected rows should also be highlighted in addition to checked
+   - should show a little sparkline or some sense of how old the data is
+   - Source picker textbox filter
+   - Manual refresh source list icon
+ - Graphing
+   - Restructure charting code to not remove everything when updating with new data
+ - Plugin system
+   - 'Dashboards' should be a module
+   - 'Explorer' module
+   - 'Monitor' module
+
+Bugs:
+ - When picker is open, changing dashboards should hide picker
+ - tooltip spawns multiple copies on large datasets
+
+Refactorings:
+ - reconcile naming of 'sources' sometimes being array of strings and sometimes array of objects

data/bin/mouth

@@ -0,0 +1,77 @@
+#!/usr/bin/env ruby
+
+$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/../lib')
+
+require 'optparse'
+require 'mouth'
+require 'mouth/runner'
+require 'mouth/sucker'
+
+options = {
+  :verbosity => 0,
+  :host => "localhost",
+  :port => 8889
+}
+
+parser = OptionParser.new do |op|
+  op.banner = "Usage: mouth [options]"
+
+  op.separator ""
+  op.separator "Options:"
+
+  ##
+  ## Daemonization / Logging options
+  ##
+
+  op.on("--pidfile PATH", "DO YOU WANT ME TO WRITE A PIDFILE SOMEWHERE FOR U?") do |pid_file|
+    options[:pid_file] = pid_file
+  end
+  
+  op.on("--logfile PATH", "I'LL POOP OUT LOGS HERE FOR U") do |log_file|
+    options[:log_file] = log_file
+  end
+  
+  op.on("-v", "--verbosity LEVEL", "HOW MUCH POOP DO U WANT IN UR LOGS? [LEVEL=0:errors,1:some,2:lots of poop]") do |verbosity|
+    options[:verbosity] = verbosity.to_i
+  end
+  
+  op.on("-K", "--kill", "SHUT THE MOUTH") do
+    options[:kill] = true
+  end
+  
+  ##
+  ## Socket to suck from
+  ##
+  
+  op.on("-H", "--host HOST", "I SUCK ON THIS NETWORK INTERFACE") do |host|
+    options[:host] = host
+  end
+  
+  op.on("-P", "--port PORT", "I SUCK FROM THIS HOLE") do |port|
+    options[:port] = port.to_i
+  end
+  
+  ##
+  ## Mongo
+  ##
+  op.on("--mongodb DATABASE", "STORE SUCKINGS IN THIS DB") do |mongo_db|
+    options[:mongo_db] = mongo_db
+  end
+  
+  # NOTE: this option can be given multiple times for a replica set
+  op.on("--mongohost HOSTPORT", "STORE SUCKINGS IN THIS MONGO") do |mongo_host|
+    options[:mongo_hosts] ||= []
+    options[:mongo_hosts] << mongo_host
+  end
+
+  op.on("-h", "--help", "I WANT MOAR HALP") do
+    puts op
+    exit
+  end
+
+  op.separator ""
+end
+
+parser.parse!
+
+Mouth::Runner.new(options).run!

data/bin/mouth-console

@@ -0,0 +1,18 @@
+#!/usr/bin/env ruby
+
+# Running bin/mouth-console will put you in an IRB session with the mouth files loaded.  Convenient for development and experimentation.
+
+require 'irb'
+require 'irb/completion'
+
+$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/../lib')
+
+require 'mouth'
+require 'mouth/sequence'
+require 'mouth/record'
+require 'mouth/graph'
+require 'mouth/dashboard'
+require 'mouth/source'
+require 'mouth/instrument'
+
+IRB.start

data/bin/mouth-endoscope

@@ -0,0 +1,18 @@
+#!/usr/bin/env ruby
+
+$LOAD_PATH.unshift File.expand_path(File.dirname(__FILE__) + '/../lib')
+
+require 'vegas'
+require 'mouth'
+require 'mouth/endoscope'
+
+begin
+  require 'thin'
+rescue LoadError
+end 
+
+Vegas::Runner.new(Mouth::Endoscope, 'mouth-endoscope') do |runner, opts, app|
+  opts.on("--mongohost HOSTPORT", "STORE SUCKINGS IN THIS MONGO") do |mongo_host|
+    Mouth.mongo_host = mongo_host
+  end
+end

data/lib/mouth.rb

@@ -0,0 +1,61 @@
+# Since the gem can be used in many ways (sucker daemon, sinatra app, instrument, ad-hoc, don't include anything)
+require 'mouth/version'
+
+module Mouth
+  class << self
+    attr_accessor :logger
+    attr_accessor :mongo_host
+
+    # Returns a mongo connection (NOT an em-mongo connection)
+    def mongo
+      @mongo ||= begin
+        require 'mongo'   # require mongo here, as opposed to the top, because we don't want mongo included in the reactor (use em-mongo for that)
+        Mongo::Connection.new(self.mongo_host || "localhost").db("mouth")
+      end
+    end
+    
+    def collection(collection_name)
+      @collections ||= {}
+      @collections[collection_name] ||= begin
+        c = mongo.collection(collection_name)
+        c.ensure_index([["t", 1]], {:background => true, :unique => true})
+        c
+      end
+    end
+    
+    def mongo_collection_name(namespace)
+      "mouth_#{namespace}"
+    end
+    
+    def sanitize_namespace(key)
+      key.gsub(/\s+/, '_').gsub(/\//, '-').gsub(/[^a-zA-Z_\-0-9]/, '')
+    end
+    
+    def sanitize_metric(key)
+      key.gsub(/\s+/, '_').gsub(/[^a-zA-Z0-9\-_\/]/, '')
+    end
+    
+    # Parses a key into two parts: namespace, and metric.  Also sanitizes each field
+    # Returns an array of values: [namespace, metric]
+    # eg,
+    # parse_key("Ticket.process_new_ticket") # => ["Ticket", "process_new_ticket"]
+    # parse_key("Forum List.other! stuff.ok") # => ["Forum_List", "other_stuff-ok"]
+    # parse_key("no_namespace") # => ["default", "no_namespace"]
+    def parse_key(key)
+      parts = key.split(".")
+      namespace = nil
+      metric = nil
+      if parts.length > 1
+        namespace = parts.shift
+        metric = parts.join("-")
+      else
+        namespace = "default"
+        metric = parts.shift
+      end
+      
+      [sanitize_namespace(namespace), sanitize_metric(metric)]
+    end 
+    
+
+  end  
+end

data/lib/mouth/dashboard.rb

@@ -0,0 +1,25 @@
+module Mouth
+  
+  # A dashboard has these fields:
+  #  id
+  #  name
+  #  todo: order
+  class Dashboard < Record
+    
+    def all_attributes
+      self.attributes.tap {|attrs| attrs[:graphs] = graphs.collect(&:all_attributes) }
+    end
+    
+    def destroy
+      self.graphs.each(&:destroy)
+      super
+    end
+    
+    # An array of graphs
+    def graphs
+      @graphs ||= begin
+        Graph.for_dashboard(self.attributes[:id])
+      end
+    end
+  end
+end