batsd-dash 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (40) hide show
  1. data/Gemfile +2 -0
  2. data/README.md +94 -0
  3. data/Rakefile +10 -0
  4. data/batsd-dash.gemspec +38 -0
  5. data/lib/batsd-dash.rb +82 -0
  6. data/lib/batsd-dash/connection_pool.rb +87 -0
  7. data/lib/batsd-dash/graph.rb +46 -0
  8. data/lib/batsd-dash/params.rb +30 -0
  9. data/lib/batsd-dash/sass/public.scss +56 -0
  10. data/lib/batsd-dash/version.rb +3 -0
  11. data/lib/public/css/datetimepicker.css +6 -0
  12. data/lib/public/css/images/ui-bg_flat_0_aaaaaa_40x100.png +0 -0
  13. data/lib/public/css/images/ui-bg_flat_75_ffffff_40x100.png +0 -0
  14. data/lib/public/css/images/ui-bg_glass_55_fbf9ee_1x400.png +0 -0
  15. data/lib/public/css/images/ui-bg_glass_65_ffffff_1x400.png +0 -0
  16. data/lib/public/css/images/ui-bg_glass_75_dadada_1x400.png +0 -0
  17. data/lib/public/css/images/ui-bg_glass_75_e6e6e6_1x400.png +0 -0
  18. data/lib/public/css/images/ui-bg_glass_95_fef1ec_1x400.png +0 -0
  19. data/lib/public/css/images/ui-bg_highlight-soft_75_cccccc_1x100.png +0 -0
  20. data/lib/public/css/images/ui-icons_222222_256x240.png +0 -0
  21. data/lib/public/css/images/ui-icons_2e83ff_256x240.png +0 -0
  22. data/lib/public/css/images/ui-icons_454545_256x240.png +0 -0
  23. data/lib/public/css/images/ui-icons_888888_256x240.png +0 -0
  24. data/lib/public/css/images/ui-icons_cd0a0a_256x240.png +0 -0
  25. data/lib/public/css/jquery-ui.css +398 -0
  26. data/lib/public/css/public.css +41 -0
  27. data/lib/public/js/dash.js +84 -0
  28. data/lib/public/js/datetimepicker.js +13 -0
  29. data/lib/public/js/datetimepicker.js~ +1 -0
  30. data/lib/public/js/flot.js +6 -0
  31. data/lib/public/js/jquery-ui.js +33 -0
  32. data/lib/public/js/jquery.js +4 -0
  33. data/lib/views/layout.haml +29 -0
  34. data/lib/views/missing.haml +6 -0
  35. data/lib/views/root.haml +0 -0
  36. data/lib/views/view.haml +10 -0
  37. data/test/helper.rb +41 -0
  38. data/test/test_connection_pool.rb +20 -0
  39. data/test/test_params_helper.rb +86 -0
  40. metadata +197 -0
data/Gemfile ADDED
@@ -0,0 +1,2 @@
1
+ source "http://rubygems.org"
2
+ gemspec
data/README.md ADDED
@@ -0,0 +1,94 @@
1
+ batsd-dash
2
+ ==========
3
+
4
+ Configurable dashboard for [batsd-server](https://github.com/noahhl/batsd).
5
+
6
+ ## Setup
7
+
8
+ ### Install
9
+
10
+ To install batsd-dash, simply install the gem
11
+
12
+ gem install batsd-dash
13
+
14
+ ### Configuration
15
+
16
+ Here is a sample rackup file (`config.ru`):
17
+
18
+ require 'batsd-dash'
19
+
20
+ # set batsd server setting BatsdDash::ConnectionPool.settings = { host:'localhost', port: 8127, pool_size: 4 }
21
+
22
+ # run the app run BatsdDash::App
23
+
24
+ Rack is very powerful. You can password protect your batsd-dash instance by
25
+ using `Rack::Auth::Basic` or `Rack::Auth::Digest::MD5`.
26
+
27
+ ## Usage
28
+
29
+ ### Data API
30
+
31
+ The application provides a simple JSON-based API for accessing data from the
32
+ batds data server. There are 3 main routes provide, one for each datatype. These
33
+ routes are `/counters`, `/timers` and `/gauges`. For example, the following
34
+ request would access data for counter based metric:
35
+
36
+ /counters?metric=a.b
37
+
38
+ It's possible to access data for more than one metric within a single request.
39
+ For example, the following request route will return data for both the `a.b`
40
+ metric and the `c.d` metric:
41
+
42
+ /counters?metrics[]=a.b&metrics[]=c.d
43
+
44
+ The data API also accepts a `start` and `stop` unix timestamp parameter for
45
+ accessing different ranges of data.
46
+
47
+ Note that, the data API will only respond with JSON if the `Accept` header to
48
+ set to `application/json`!
49
+
50
+ ### Viewing Graphs
51
+
52
+ Graphs are rendered using Flot, a JavaScript library which uses the canvas
53
+ element to create graphs. Since rendering is all done on the client, we make use
54
+ of hash based navigation in order to reduce the amount of requests and while
55
+ maintaining 'linkability'.
56
+
57
+ For example, to view a graph for the `a.b` metric, you would make the following
58
+ request from your browser:
59
+
60
+ /counters#metrics=a.b
61
+
62
+ The graph view will provide you with a date time picker to make selecting
63
+ different start and stop time ranges easier. Graphs are updated when you press
64
+ the 'View' button.
65
+
66
+ Much like the data API, it's possible to view more than one metric at the same
67
+ time. To do this, visit the following route from your browser:
68
+
69
+ /counters#metrics=a.b,c.d
70
+
71
+ _TODO_ when no data or only a single point is available, the graph is a little
72
+ strange looking. This is something we will improve upon. Additionally, we also
73
+ plan to add some sort of tree-based widget for selecting different metrics to
74
+ view.
75
+
76
+ Feel free to submit pull requests with these features!
77
+
78
+ ### Zerofill
79
+
80
+ _TODO_ add details about zerofill.
81
+
82
+ _TODO_ Setup client to accept pass along no-zerofill options.
83
+
84
+ ## Development
85
+
86
+ ### Asset Management
87
+
88
+ We use Sass for CSS within this project. If you make any changes to the Sass
89
+ files, ensure you recompile the CSS. This is done by running:
90
+
91
+ compass compile --force --output-style compact --environment production --sass-dir lib/batsd-dash/sass --css-dir lib/public/css
92
+
93
+ Additionally, it is highly recommended you use thin for development since this
94
+ app uses EventMachine.
data/Rakefile ADDED
@@ -0,0 +1,10 @@
1
+ require 'bundler/gem_tasks'
2
+ require 'rake/testtask'
3
+
4
+ Rake::TestTask.new do |t|
5
+ t.libs << 'test'
6
+ t.test_files = ENV['TEST'] || Dir['test/test_*.rb']
7
+ t.verbose = true
8
+ #t.warning = true
9
+ end
10
+
@@ -0,0 +1,38 @@
1
+ # -*- encoding: utf-8 -*-
2
+ $:.push File.expand_path("../lib", __FILE__)
3
+ require 'batsd-dash/version'
4
+
5
+ Gem::Specification.new do |s|
6
+ s.name = "batsd-dash"
7
+ s.version = BatsdDash::VERSION
8
+
9
+ s.authors = ["mikeycgto", "btoconnor"]
10
+ s.email = ["mikeycgto@gmail.com", "gatzby3jr@gmail.com"]
11
+
12
+ s.homepage = "https://github.com/mikeycgto/batsd-dash"
13
+
14
+ s.summary = %q{batsd-dash}
15
+ s.description = %q{batsd-dash - graphs and stuff from batds. yay.}
16
+
17
+ s.rubyforge_project = "batsd-dash"
18
+
19
+ s.files = `git ls-files`.split("\n")
20
+ s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
21
+ s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
22
+ s.require_paths = ["lib"]
23
+
24
+ # specify any dependencies here; for example:
25
+ s.add_dependency "sinatra"
26
+ s.add_dependency "sinatra-contrib"
27
+ s.add_dependency "sinatra-synchrony", "~> 0.3.2"
28
+
29
+ s.add_dependency "haml"
30
+ s.add_dependency "yajl-ruby"
31
+
32
+ s.add_development_dependency "rake"
33
+ s.add_development_dependency "minitest"
34
+ s.add_development_dependency "mocha"
35
+ s.add_development_dependency "turn"
36
+
37
+ s.add_development_dependency "thin"
38
+ end
data/lib/batsd-dash.rb ADDED
@@ -0,0 +1,82 @@
1
+ require 'yajl'
2
+ require 'sinatra/base'
3
+ require 'sinatra/synchrony'
4
+ #require 'sinatra/reloader' if ENV['RACK_ENV'] == 'development'
5
+
6
+ %w[connection_pool graph params version].each { |file| require "batsd-dash/#{file}" }
7
+
8
+ module BatsdDash
9
+ class App < Sinatra::Base
10
+ #configure(:development) { register Sinatra::Reloader }
11
+
12
+ configure do
13
+ register Sinatra::Synchrony
14
+ helpers ParamsHelper, GraphHelper, ConnectionHelpers
15
+
16
+ set :haml, :format => :html5
17
+
18
+ EM::Synchrony.next_tick { ConnectionPool::initialize_connection_pool }
19
+ end
20
+
21
+ helpers do
22
+ def render_error(msg)
23
+ render_json 400, error: msg
24
+ end
25
+
26
+ def render_json(code = 200, json)
27
+ halt code, String === json ? json : Yajl::Encoder.encode(json)
28
+ end
29
+ end
30
+
31
+ get "/" do
32
+ haml :root
33
+ end
34
+
35
+ get "/version", :provides => :json do
36
+ render_json version: BatsdDash::VERSION
37
+ end
38
+
39
+ get "/available", :provides => :json do
40
+ connection_pool.async_available_list.callback do |json|
41
+ render_json json
42
+ end
43
+ end
44
+
45
+ %w[ counters timers gauges ].each do |datatype|
46
+ # this route renders the template (with codes for the graph)
47
+ get "/#{datatype}", :provides => :html do
48
+ haml :view
49
+ end
50
+
51
+ # actual data API route
52
+ get "/#{datatype}", :provides => :json do
53
+ metrics = parse_metrics
54
+ range = parse_time_range
55
+
56
+ return render_error('Invalid time range') unless range
57
+ return render_error('Invalid metrics') if metrics.empty?
58
+
59
+ results = { range: range.dup.map! { |n| n * 1000 }, metrics: [] }
60
+ collect_opts = { zero_fill: !params[:no_zero_fill], range: results[:range] }
61
+
62
+ metrics.each do |metric|
63
+ statistic = "#{datatype}:#{metric}"
64
+ deferrable = connection_pool.async_values(statistic, range)
65
+
66
+ deferrable.errback { |e| return render_error(e.message) }
67
+ deferrable.callback do |json|
68
+ values = json[statistic]
69
+
70
+ # merge in interval if its not already; interval is always same
71
+ collect_opts.merge!(interval: json['interval'] || 0) unless collect_opts.has_key?(:interval)
72
+ # process values for graphing and add to results
73
+ results[:metrics] << { label: metric, data: collect_for_graph(values, collect_opts) }
74
+ end
75
+ end
76
+
77
+ cache_control :no_cache, :no_store
78
+ render_json results
79
+ end
80
+ end
81
+ end
82
+ end
@@ -0,0 +1,87 @@
1
+ module BatsdDash
2
+ module ConnectionHelpers
3
+ def connection_pool
4
+ ConnectionPool.pool or render_error('Connect pool failed to connect to Batsd')
5
+ end
6
+ end
7
+
8
+ class ConnectionPool
9
+ class << self
10
+ attr_accessor :settings
11
+ attr_reader :pool
12
+
13
+ def initialize_connection_pool
14
+ # default settings (which are defaults from batsd itself)
15
+ @settings ||= { host: 'localhost', port: 8127, pool_size: 4 }
16
+
17
+ if @reconnect_timer
18
+ @reconnect_timer.cancel
19
+ @reconnect_timer = nil
20
+ end
21
+
22
+ begin
23
+ @pool = EventMachine::Synchrony::ConnectionPool.new(size: settings[:pool_size]) do
24
+ Client.new(settings[:host], settings[:port])
25
+ end
26
+
27
+ rescue Exception => e
28
+ warn "Connection Pool Error: #{e.message}"
29
+
30
+ end
31
+ end
32
+
33
+ def close_connection_pool
34
+ @pool.close if @pool
35
+ @pool = nil
36
+ end
37
+
38
+ def start_reconnect_timer
39
+ unless @reconnect_timer
40
+ # try to reconnect every 30 seconds
41
+ @reconnect_timer = EventMachine::Synchrony.add_timer(30) { initialize_connection_pool }
42
+ end
43
+ end
44
+ end
45
+
46
+ class Client < EventMachine::Synchrony::TCPSocket
47
+ def write_and_read_json(request)
48
+ EventMachine::DefaultDeferrable.new.tap do |df|
49
+ response = String.new
50
+ parser = Yajl::Parser.new
51
+ parser.on_parse_complete = Proc.new { |json| df.succeed(json) }
52
+
53
+ begin
54
+ write request
55
+
56
+ # keep reading till we hit new line
57
+ while response[-1] != "\n"
58
+ response << read(1)
59
+ end
60
+
61
+ parser.parse response
62
+
63
+ rescue Exception => e
64
+ # TODO handle broken pipe
65
+ #unbind if Errno::EPIPE === e
66
+
67
+ df.fail(e)
68
+ end
69
+ end
70
+ end
71
+
72
+ def async_available_list
73
+ write_and_read_json "available"
74
+ end
75
+
76
+ def async_values(statistic, range)
77
+ write_and_read_json "values #{statistic} #{range[0]} #{range[1]}"
78
+ end
79
+
80
+ def unbind(reason = nil)
81
+ super(reason)
82
+
83
+ ConnectionPool::start_reconnect_timer
84
+ end
85
+ end
86
+ end
87
+ end
@@ -0,0 +1,46 @@
1
+ # helpers for parsing and validating input
2
+ module BatsdDash
3
+ module GraphHelper
4
+ ##
5
+ # This method works directly against values
6
+ def collect_for_graph(values, opts = {})
7
+ values.tap do |pts|
8
+ # remap the values
9
+ pts.map! { |pt| [pt['timestamp'].to_i * 1000, pt['value'].to_f] }
10
+
11
+ # apply zerofill
12
+ zero_fill!(pts, opts[:range], opts[:interval]) unless pts.empty? || !opts[:zero_fill]
13
+ end
14
+ end
15
+
16
+ ##
17
+ # The data better be normalized to the interval otherwise
18
+ # this method may get pissed
19
+ def zero_fill!(values, range, step)
20
+ return values if step.zero?
21
+
22
+ # convert to milisec
23
+ step *= 1000
24
+
25
+ values.tap do |data|
26
+ # start from the first timestamp
27
+ time = data.first.first + step
28
+ index = 0
29
+
30
+ while obj = data[index += 1]
31
+ current = obj.first
32
+
33
+ next if current <= time
34
+ if current == time
35
+ time += step
36
+ next
37
+ end
38
+
39
+ data.insert(index, [time, 0])
40
+
41
+ time += step
42
+ end
43
+ end
44
+ end
45
+ end
46
+ end
@@ -0,0 +1,30 @@
1
+ # helpers for processing params and validating input
2
+ module BatsdDash
3
+ module ParamsHelper
4
+ def parse_metrics
5
+ metrics = params[:metrics]
6
+ metrics = [metrics] unless Array === metrics
7
+
8
+ metrics.tap { |list| list.reject! { |m| m.nil? || m.empty? } }
9
+ end
10
+
11
+ def parse_time_range
12
+ start, stop = params[:start], params[:stop]
13
+
14
+ if start.nil? && stop.nil?
15
+ now = Time.now.to_i
16
+
17
+ # 1 hr range
18
+ # TODO make this setting?
19
+ [ now - 3600 + 1, now ]
20
+
21
+ else
22
+ [start.to_i, stop.to_i].tap do |range|
23
+ if range[0] <= 0 || range[1] <= 0 || range[0] >= range[1]
24
+ return nil
25
+ end
26
+ end
27
+ end
28
+ end
29
+ end
30
+ end
@@ -0,0 +1,56 @@
1
+ @import "compass/utilities/general/clearfix";
2
+ @import "compass/css3";
3
+ @import "compass/reset";
4
+
5
+ html, body { height:100%; min-height:100%; background-color:#222; }
6
+ h1, h2 { color:#0F0; font-family:"Courier New",Courier,monospace; }
7
+ a { text-decoration:none; color:inherit; }
8
+
9
+ .wrap { width:1000px; margin:0 auto; }
10
+
11
+ #header {
12
+ height:60px; border:1px solid #0F0; border-width:0 0 1px 0; margin:0 0 20px;
13
+
14
+ background-color: #000000;
15
+ @include filter-gradient(#000000, #222, vertical);
16
+ @include background-image(linear-gradient(top, #000000 0%,#222 100%));
17
+
18
+ @include pie-clearfix;
19
+
20
+ h1 { float:left; font-size:28px; padding:15px 0 0; }
21
+ ul {
22
+ float:right; width:300px; margin:15px 0;
23
+
24
+ li {
25
+ float:left; margin:0 20px;
26
+ a { cursor:pointer; }
27
+ }
28
+ }
29
+ }
30
+
31
+
32
+ #content {
33
+ color:#FFF; width:1000px; margin:0 auto; font-size:18px;
34
+
35
+ h1 { font-size:26px; margin:30px 0; }
36
+ h2 { font-size:24px; margin:20px 0; }
37
+
38
+ p {
39
+ em { font-style:italic; margin:0 -4px 0 0; }
40
+ }
41
+
42
+ .graph {
43
+ width:900px; height:400px; margin:0 auto;
44
+
45
+ h2 { text-align:center; margin:180px; }
46
+ }
47
+
48
+ .inputs {
49
+ width:900px; margin:0 auto 30px; text-align:center;
50
+
51
+ label { font-weight:bold; }
52
+ input { width:200px; }
53
+ }
54
+ }
55
+
56
+