batsd-dash 0.3.1 → 0.5.0

data/.rbenv-version

@@ -0,0 +1 @@
+jruby-1.7.0

data/Gemfile

@@ -7,3 +7,6 @@ group :test do
   gem "guard-minitest"
 end
 
+group :development do
+  gem "compass"
+end

data/README.md

@@ -1,91 +1,48 @@
-batsd-dash
+Batsd-dash
 ==========
 
-Configurable dashboard for [batsd-server](https://github.com/noahhl/batsd).
+Batds-dash is a configurable dashboard for [batsd-server](https://github.com/noahhl/batsd).
+The front-end uses [NVD3](http://nvd3.org/) for rendering graphs. The back-end uses 
+[Sinatra](github.com/sinatra/sinatra/) for the actual web application.
 
-## Setup
+The application server is designed to use a threaded [connection pool](https://github.com/mperham/connection_pool).
+Thus, it is suggested that you run Batsd-dash on JRuby or Rubinius in order 
+to take full advantage of threaded processing.
 
-### Install
+### Documentation
 
-To install batsd-dash, simply install the gem
+  * [Installation and
+    Configuration](https://github.com/mikeycgto/batsd-dash/wiki/Installation-and-Configuration)
+  * [Running the
+    Application](https://github.com/mikeycgto/batsd-dash/wiki/Running-the-Application)
+  * [Viewing
+    Graphs](https://github.com/mikeycgto/batsd-dash/wiki/Viewing-Graphs)
+  * [Data API](https://github.com/mikeycgto/batsd-dash/wiki/Data-API)
+  * [Custom Pages](https://github.com/mikeycgto/batsd-dash/wiki/Custom-Pages)
+  * [Contributing](https://github.com/mikeycgto/batsd-dash/wiki/Contributing)
 
-    gem install batsd-dash
+### About
 
-### Configuration
+This is project is maintained and developed by [@mikeycgto](https://twitter.com/mikeycgto) 
+and [@btoconnor](https://twitter.com/btoconnor) mainly for use on [BreakBase](http://breakbase.com).
 
-Here is a sample rackup file (`config.ru`):
-    
-    require 'batsd-dash'
+### License
 
-    # set batsd server setting 
-    BatsdDash::ConnectionPool.settings = { host:'localhost', port: 8127, pool_size: 8 }
+Copyright (c) 2012 Michael J Coyne & Brian O'Connor
 
-    # run the app 
-    run BatsdDash::App
+Permission is hereby granted, free of charge, to any person obtaining a copy of this 
+software and associated documentation files (the "Software"), to deal in the Software
+without restriction, including without limitation the rights to use, copy, modify,
+merge, publish, distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to the following
+conditions:
 
-Rack is very powerful. You can password protect your batsd-dash instance 
-by using `Rack::Auth::Basic` or `Rack::Auth::Digest::MD5`.
+The above copyright notice and this permission notice shall be included in all copies
+or substantial portions of the Software.
 
-## Viewing Graphs
-
-Graphs are rendered using [nv.d3](http://nvd3.com/), a powerful graph
-and visualization library.
-
-Since rendering is all done on the client, we make use of hash based
-navigation in order to reduce the amount of requests while still
-maintaining 'linkability'.
-
-For example, to view a graph for the counter `a.b` statistic, you would need
-to make the following request from your browser:
-
-    /graph#counters=a.b
-
-The graph view will provide you with a date time picker to make selecting
-different time ranges easier. Graphs are updated when you press the 
-'View' button or when the URL is updated.
-
-It's possible to view more than one metric at the same time. To do this, 
-visit the following route from your browser:
-
-    /graph#counters=a.b,c.d
-
-You can also view different datatypes at the same time:
-
-    /graph#counters=a.b&timers=x.y
-
-__NOTE__: As of now, a single y-axis is used when datatypes are mixed.
-Soon, we will add support for multiple axis when viewing mixed types.
-
-## Data API
-
-The application provides a simple JSON-based API for accessing data from
-the batds data server. The data API accepts similar parameters as the
-graph view but uses traditional query strings instead: 
-
-    /data?counters[]=a.b&counters[]=c.d&timers[]=x.y
-
-The data API also accepts a `start` and `stop` unix timestamp parameter 
-for accessing different ranges of data. Note that, the data API will
-only respond with JSON if the `Accept` header to set to `application/json`!
-
-## Graph and Render Options
-
-1. Zerofill:
-   __TODO__ Add details about zerofill
-
-
-## Development
-
-### Asset Management
-
-We use Sass for CSS within this project. If you make any changes to the Sass
-files, ensure you recompile the CSS. This is done by running:
-    
-    compass compile --force --output-style compact --environment production --sass-dir lib/batsd-dash/sass --css-dir lib/public/css
-
-Additionally, it is highly recommended you use thin for development since this
-app uses EventMachine.
-
-## About
-
-This is project is maintained and developed by the people behind [BreakBase](http://breakbase.com) ([@mikeycgto](https://twitter.com/mikeycgto) and [@btoconnor](https://twitter.com/btoconnor))
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED,
+INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR
+PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE
+FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR
+OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER
+DEALINGS IN THE SOFTWARE.

data/batsd-dash.gemspec

@@ -4,7 +4,7 @@ require 'batsd-dash/version'
 
 Gem::Specification.new do |s|
   s.name        = "batsd-dash"
-  s.version     = BatsdDash::VERSION
+  s.version     = Batsd::Dash::VERSION
 
   s.authors     = ["mikeycgto", "btoconnor"]
   s.email       = ["mikeycgto@gmail.com", "gatzby3jr@gmail.com"]
@@ -21,18 +21,14 @@ Gem::Specification.new do |s|
   s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
   s.require_paths = ["lib"]
 
-  # specify any dependencies here; for example:
-  s.add_dependency "sinatra"
-  s.add_dependency "sinatra-contrib"
-  s.add_dependency "sinatra-synchrony", "~> 0.3.2"
+  s.add_dependency "connection_pool"
 
+  s.add_dependency "sinatra"
   s.add_dependency "haml"
-  s.add_dependency "yajl-ruby"
 
   s.add_development_dependency "rake"
   s.add_development_dependency "minitest"
-  s.add_development_dependency "mocha"
-  s.add_development_dependency "turn"
 
-  s.add_development_dependency "thin"
+  s.add_development_dependency "sinatra-contrib"
+  s.add_development_dependency "mocha"
 end

data/lib/batsd-dash.rb

@@ -1,83 +1,12 @@
-require 'yajl'
-require 'sinatra/base'
-require 'sinatra/synchrony'
-#require 'sinatra/reloader' if ENV['RACK_ENV'] == 'development'
-
-%w[connection_pool graph params version].each { |file| require "batsd-dash/#{file}" }
-
-module BatsdDash
-  class App < Sinatra::Base
-    #configure(:development) { register Sinatra::Reloader }
-
-    configure do
-      register Sinatra::Synchrony
-      helpers ParamsHelper, GraphHelper, ConnectionHelpers
-
-      set :haml, :format => :html5
-
-      EM::Synchrony.next_tick { ConnectionPool::initialize_connection_pool }
-    end
-
-    helpers do
-      def render_error(msg)
-        render_json 400, error: msg
-      end
-
-      def render_json(code = 200, json)
-        halt code, String === json ? json : Yajl::Encoder.encode(json)
-      end
-    end
-
-    get "/" do
-      haml :root
-    end
-
-    get "/version", :provides => :json do
-      render_json version: BatsdDash::VERSION
-    end
-
-    get "/available", :provides => :json do
-      connection_pool.async_available_list.callback do |json|
-        render_json json
-      end
-    end
-
-    # this route renders the template (with codes for the graph)
-    get "/graph", :provides => :html do
-      haml :view
-    end
-
-    # actual data API route
-    get "/data", :provides => :json do
-      statistics = parse_statistics
-      range = parse_time_range
-
-      return render_error('Invalid time range') unless range
-      return render_error('Invalid metrics') if statistics.empty?
-
-      results = []
-      options = { range: range.dup.map { |n| n * 1000 } }
-
-      statistics.each do |datatype, metrics|
-        metrics.each do |metric|
-          statistic = "#{datatype}:#{metric}"
-          deferrable = connection_pool.async_values(statistic, range)
-
-          deferrable.errback { |e| return render_error(e.message) }
-          deferrable.callback do |json|
-            options[:interval] ||= json['interval']
-            options[:zero_fill] = !statistic.start_with?('gauges') && params[!:no_zero_fill]
-
-            points = json[statistic] || []
-            values = values_for_graph(points, options)
-
-            results << { key: metric, type: datatype[0..-2], values: values }
-          end
-        end
-      end
-
-      cache_control :no_cache, :no_store
-      render_json range: options[:range], interval: options[:interval], results: results
+module Batsd
+  module Dash
+    class << self
+      attr_accessor :config
     end
   end
 end
+
+require 'batsd-dash/version'
+require 'batsd-dash/connection'
+require 'batsd-dash/params'
+require 'batsd-dash/graph'

data/lib/batsd-dash/app.rb

@@ -0,0 +1,113 @@
+require 'sinatra/base'
+require 'haml'
+require 'json'
+
+module Batsd::Dash
+  class App < Sinatra::Base
+    configure(:development) { disable :show_exceptions }
+
+    configure do
+      helpers ParamsHelper, GraphHelper
+      set :haml, format: :html5
+      set :raise_errors, false
+
+      config = Batsd::Dash::config.dup || {}
+
+      set :views, [views, config.delete(:view_path)].compact
+      set :host, config.delete(:host) || 'localhost'
+      set :port, config.delete(:port) || 8127
+
+      set :view_names, [:root, :view, :missing, :layout, :loading]
+
+      @connection_pool = ConnectionPool.new(config) do
+        Connection.new(host, port)
+      end
+    end
+
+    helpers do
+      def find_template(views, name, engine, &block)
+        path = settings.view_names.include?(name) ? views.first : views.last
+
+        super path, name, engine, &block
+      end
+
+      def render_error(msg)
+        render_json 400, error: msg
+      end
+
+      def render_json(code = 200, json)
+        halt code, String === json ? json : JSON.dump(json)
+      end
+
+      def connection_pool
+        @connection_pool ||= self.class.instance_variable_get(:@connection_pool)
+      end
+    end
+
+    error Connection::SocketError do
+      render_json error: env['sinatra.error'].message
+    end
+
+    get "/", provides: :html do
+      haml :root
+    end
+
+    get "/graph", provides: :html do
+      haml :view
+    end
+
+    get %r[/([A-Za-z0-9-_]+)$], provides: :html do
+      begin
+        haml params[:captures].first.to_sym, locals: { user_template: true }
+      rescue Errno::ENOENT
+        haml :missing
+      end
+    end
+
+    get "/version", provides: :json do
+      render_json version: BatsdDash::VERSION
+    end
+
+    get "/available", provides: :json do
+      connection_pool.with do |conn|
+        render_json conn.available
+      end
+    end
+
+    # actual data API route
+    get "/data", :provides => :json do
+      cache_control :no_cache, :no_store
+
+      statistics = parse_statistics
+      range = parse_time_range
+
+      return render_error('Invalid time range') unless range
+      return render_error('Invalid metrics') if statistics.empty?
+
+      options = { range: range.dup.map { |n| n * 1000 } }
+      results = []
+
+      connection_pool.with do |conn|
+        statistics.each do |datatype, metrics|
+          metrics.each do |metric|
+            statistic = "#{datatype}:#{metric}"
+            json = conn.values(statistic, range)
+
+            next unless json
+
+            options[:interval] ||= json['interval']
+            options[:zero_fill] = statistic !~ /^gauges/i && !params[:no_zero_fill]
+
+            points = json[statistic] || []
+            values = values_for_graph(points, options)
+
+            results << { key: metric, type: datatype[0..-2], values: values }
+          end
+        end
+      end
+
+      render_json range: options[:range], interval: options[:interval], results: results
+    end
+  end
+end
+

data/lib/batsd-dash/connection.rb

@@ -0,0 +1,54 @@
+require 'socket'
+require 'connection_pool'
+
+module Batsd::Dash
+  class Connection
+    class SocketError < Exception; end
+
+    attr_reader :socket
+
+    def initialize(host, port)
+      @host = host
+      @port = port
+
+      connect_socket
+    end
+
+    def available
+      query 'available'
+    end
+
+    def values(statistic, range)
+      query "values #{statistic} #{range[0]} #{range[1]}"
+    end
+
+    private
+
+    def reset_socket
+      @socket = nil
+    end
+
+    def connect_socket
+      @socket = TCPSocket.new(@host, @port)
+
+    rescue Errno::ECONNREFUSED, Errno::ETIMEDOUT
+      reset_socket
+    end
+
+    def query(command, attempts = 0)
+      connect_socket unless socket
+
+      raise SocketError.new('Socket not Connected') if socket.nil?
+
+      socket.puts command
+      JSON.parse socket.gets
+
+    rescue Errno::EPIPE
+      reset_socket
+
+      if attempts < 5
+        query command, attempts + 1
+      end
+    end
+  end
+end

data/lib/batsd-dash/graph.rb

@@ -1,10 +1,14 @@
 # helpers for parsing and validating input
-module BatsdDash
+module Batsd::Dash
   module GraphHelper
     ##
     # This method works directly against values. It will tranform all
     # datapoint to an array where the first element is a milisecond
     # timestamp and the second is a float value data point.
+    #
+    # @param [Array] values array of datapoints
+    # @param [Hash] opts optional options hash
+    # @return [Array] modified values array
     def values_for_graph(values, opts = {})
       return values if values.empty?
 
@@ -37,7 +41,11 @@ module BatsdDash
     end
 
     ##
-    # Transform a point at a given index
+    # Transform a point at a given index. Works against
+    # values array
+    #
+    # @param [Integer] index
+    # @param [Array] values array of values
     def transform_point_at!(index, values)
       data_pt = values[index]