batsd-dash 0.2.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/Gemfile +2 -0
- data/README.md +94 -0
- data/Rakefile +10 -0
- data/batsd-dash.gemspec +38 -0
- data/lib/batsd-dash.rb +82 -0
- data/lib/batsd-dash/connection_pool.rb +87 -0
- data/lib/batsd-dash/graph.rb +46 -0
- data/lib/batsd-dash/params.rb +30 -0
- data/lib/batsd-dash/sass/public.scss +56 -0
- data/lib/batsd-dash/version.rb +3 -0
- data/lib/public/css/datetimepicker.css +6 -0
- data/lib/public/css/images/ui-bg_flat_0_aaaaaa_40x100.png +0 -0
- data/lib/public/css/images/ui-bg_flat_75_ffffff_40x100.png +0 -0
- data/lib/public/css/images/ui-bg_glass_55_fbf9ee_1x400.png +0 -0
- data/lib/public/css/images/ui-bg_glass_65_ffffff_1x400.png +0 -0
- data/lib/public/css/images/ui-bg_glass_75_dadada_1x400.png +0 -0
- data/lib/public/css/images/ui-bg_glass_75_e6e6e6_1x400.png +0 -0
- data/lib/public/css/images/ui-bg_glass_95_fef1ec_1x400.png +0 -0
- data/lib/public/css/images/ui-bg_highlight-soft_75_cccccc_1x100.png +0 -0
- data/lib/public/css/images/ui-icons_222222_256x240.png +0 -0
- data/lib/public/css/images/ui-icons_2e83ff_256x240.png +0 -0
- data/lib/public/css/images/ui-icons_454545_256x240.png +0 -0
- data/lib/public/css/images/ui-icons_888888_256x240.png +0 -0
- data/lib/public/css/images/ui-icons_cd0a0a_256x240.png +0 -0
- data/lib/public/css/jquery-ui.css +398 -0
- data/lib/public/css/public.css +41 -0
- data/lib/public/js/dash.js +84 -0
- data/lib/public/js/datetimepicker.js +13 -0
- data/lib/public/js/datetimepicker.js~ +1 -0
- data/lib/public/js/flot.js +6 -0
- data/lib/public/js/jquery-ui.js +33 -0
- data/lib/public/js/jquery.js +4 -0
- data/lib/views/layout.haml +29 -0
- data/lib/views/missing.haml +6 -0
- data/lib/views/root.haml +0 -0
- data/lib/views/view.haml +10 -0
- data/test/helper.rb +41 -0
- data/test/test_connection_pool.rb +20 -0
- data/test/test_params_helper.rb +86 -0
- metadata +197 -0
data/Gemfile
ADDED
data/README.md
ADDED
@@ -0,0 +1,94 @@
|
|
1
|
+
batsd-dash
|
2
|
+
==========
|
3
|
+
|
4
|
+
Configurable dashboard for [batsd-server](https://github.com/noahhl/batsd).
|
5
|
+
|
6
|
+
## Setup
|
7
|
+
|
8
|
+
### Install
|
9
|
+
|
10
|
+
To install batsd-dash, simply install the gem
|
11
|
+
|
12
|
+
gem install batsd-dash
|
13
|
+
|
14
|
+
### Configuration
|
15
|
+
|
16
|
+
Here is a sample rackup file (`config.ru`):
|
17
|
+
|
18
|
+
require 'batsd-dash'
|
19
|
+
|
20
|
+
# set batsd server setting BatsdDash::ConnectionPool.settings = { host:'localhost', port: 8127, pool_size: 4 }
|
21
|
+
|
22
|
+
# run the app run BatsdDash::App
|
23
|
+
|
24
|
+
Rack is very powerful. You can password protect your batsd-dash instance by
|
25
|
+
using `Rack::Auth::Basic` or `Rack::Auth::Digest::MD5`.
|
26
|
+
|
27
|
+
## Usage
|
28
|
+
|
29
|
+
### Data API
|
30
|
+
|
31
|
+
The application provides a simple JSON-based API for accessing data from the
|
32
|
+
batds data server. There are 3 main routes provide, one for each datatype. These
|
33
|
+
routes are `/counters`, `/timers` and `/gauges`. For example, the following
|
34
|
+
request would access data for counter based metric:
|
35
|
+
|
36
|
+
/counters?metric=a.b
|
37
|
+
|
38
|
+
It's possible to access data for more than one metric within a single request.
|
39
|
+
For example, the following request route will return data for both the `a.b`
|
40
|
+
metric and the `c.d` metric:
|
41
|
+
|
42
|
+
/counters?metrics[]=a.b&metrics[]=c.d
|
43
|
+
|
44
|
+
The data API also accepts a `start` and `stop` unix timestamp parameter for
|
45
|
+
accessing different ranges of data.
|
46
|
+
|
47
|
+
Note that, the data API will only respond with JSON if the `Accept` header to
|
48
|
+
set to `application/json`!
|
49
|
+
|
50
|
+
### Viewing Graphs
|
51
|
+
|
52
|
+
Graphs are rendered using Flot, a JavaScript library which uses the canvas
|
53
|
+
element to create graphs. Since rendering is all done on the client, we make use
|
54
|
+
of hash based navigation in order to reduce the amount of requests and while
|
55
|
+
maintaining 'linkability'.
|
56
|
+
|
57
|
+
For example, to view a graph for the `a.b` metric, you would make the following
|
58
|
+
request from your browser:
|
59
|
+
|
60
|
+
/counters#metrics=a.b
|
61
|
+
|
62
|
+
The graph view will provide you with a date time picker to make selecting
|
63
|
+
different start and stop time ranges easier. Graphs are updated when you press
|
64
|
+
the 'View' button.
|
65
|
+
|
66
|
+
Much like the data API, it's possible to view more than one metric at the same
|
67
|
+
time. To do this, visit the following route from your browser:
|
68
|
+
|
69
|
+
/counters#metrics=a.b,c.d
|
70
|
+
|
71
|
+
_TODO_ when no data or only a single point is available, the graph is a little
|
72
|
+
strange looking. This is something we will improve upon. Additionally, we also
|
73
|
+
plan to add some sort of tree-based widget for selecting different metrics to
|
74
|
+
view.
|
75
|
+
|
76
|
+
Feel free to submit pull requests with these features!
|
77
|
+
|
78
|
+
### Zerofill
|
79
|
+
|
80
|
+
_TODO_ add details about zerofill.
|
81
|
+
|
82
|
+
_TODO_ Setup client to accept pass along no-zerofill options.
|
83
|
+
|
84
|
+
## Development
|
85
|
+
|
86
|
+
### Asset Management
|
87
|
+
|
88
|
+
We use Sass for CSS within this project. If you make any changes to the Sass
|
89
|
+
files, ensure you recompile the CSS. This is done by running:
|
90
|
+
|
91
|
+
compass compile --force --output-style compact --environment production --sass-dir lib/batsd-dash/sass --css-dir lib/public/css
|
92
|
+
|
93
|
+
Additionally, it is highly recommended you use thin for development since this
|
94
|
+
app uses EventMachine.
|
data/Rakefile
ADDED
data/batsd-dash.gemspec
ADDED
@@ -0,0 +1,38 @@
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
2
|
+
$:.push File.expand_path("../lib", __FILE__)
|
3
|
+
require 'batsd-dash/version'
|
4
|
+
|
5
|
+
Gem::Specification.new do |s|
|
6
|
+
s.name = "batsd-dash"
|
7
|
+
s.version = BatsdDash::VERSION
|
8
|
+
|
9
|
+
s.authors = ["mikeycgto", "btoconnor"]
|
10
|
+
s.email = ["mikeycgto@gmail.com", "gatzby3jr@gmail.com"]
|
11
|
+
|
12
|
+
s.homepage = "https://github.com/mikeycgto/batsd-dash"
|
13
|
+
|
14
|
+
s.summary = %q{batsd-dash}
|
15
|
+
s.description = %q{batsd-dash - graphs and stuff from batds. yay.}
|
16
|
+
|
17
|
+
s.rubyforge_project = "batsd-dash"
|
18
|
+
|
19
|
+
s.files = `git ls-files`.split("\n")
|
20
|
+
s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
|
21
|
+
s.executables = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
|
22
|
+
s.require_paths = ["lib"]
|
23
|
+
|
24
|
+
# specify any dependencies here; for example:
|
25
|
+
s.add_dependency "sinatra"
|
26
|
+
s.add_dependency "sinatra-contrib"
|
27
|
+
s.add_dependency "sinatra-synchrony", "~> 0.3.2"
|
28
|
+
|
29
|
+
s.add_dependency "haml"
|
30
|
+
s.add_dependency "yajl-ruby"
|
31
|
+
|
32
|
+
s.add_development_dependency "rake"
|
33
|
+
s.add_development_dependency "minitest"
|
34
|
+
s.add_development_dependency "mocha"
|
35
|
+
s.add_development_dependency "turn"
|
36
|
+
|
37
|
+
s.add_development_dependency "thin"
|
38
|
+
end
|
data/lib/batsd-dash.rb
ADDED
@@ -0,0 +1,82 @@
|
|
1
|
+
require 'yajl'
|
2
|
+
require 'sinatra/base'
|
3
|
+
require 'sinatra/synchrony'
|
4
|
+
#require 'sinatra/reloader' if ENV['RACK_ENV'] == 'development'
|
5
|
+
|
6
|
+
%w[connection_pool graph params version].each { |file| require "batsd-dash/#{file}" }
|
7
|
+
|
8
|
+
module BatsdDash
|
9
|
+
class App < Sinatra::Base
|
10
|
+
#configure(:development) { register Sinatra::Reloader }
|
11
|
+
|
12
|
+
configure do
|
13
|
+
register Sinatra::Synchrony
|
14
|
+
helpers ParamsHelper, GraphHelper, ConnectionHelpers
|
15
|
+
|
16
|
+
set :haml, :format => :html5
|
17
|
+
|
18
|
+
EM::Synchrony.next_tick { ConnectionPool::initialize_connection_pool }
|
19
|
+
end
|
20
|
+
|
21
|
+
helpers do
|
22
|
+
def render_error(msg)
|
23
|
+
render_json 400, error: msg
|
24
|
+
end
|
25
|
+
|
26
|
+
def render_json(code = 200, json)
|
27
|
+
halt code, String === json ? json : Yajl::Encoder.encode(json)
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
get "/" do
|
32
|
+
haml :root
|
33
|
+
end
|
34
|
+
|
35
|
+
get "/version", :provides => :json do
|
36
|
+
render_json version: BatsdDash::VERSION
|
37
|
+
end
|
38
|
+
|
39
|
+
get "/available", :provides => :json do
|
40
|
+
connection_pool.async_available_list.callback do |json|
|
41
|
+
render_json json
|
42
|
+
end
|
43
|
+
end
|
44
|
+
|
45
|
+
%w[ counters timers gauges ].each do |datatype|
|
46
|
+
# this route renders the template (with codes for the graph)
|
47
|
+
get "/#{datatype}", :provides => :html do
|
48
|
+
haml :view
|
49
|
+
end
|
50
|
+
|
51
|
+
# actual data API route
|
52
|
+
get "/#{datatype}", :provides => :json do
|
53
|
+
metrics = parse_metrics
|
54
|
+
range = parse_time_range
|
55
|
+
|
56
|
+
return render_error('Invalid time range') unless range
|
57
|
+
return render_error('Invalid metrics') if metrics.empty?
|
58
|
+
|
59
|
+
results = { range: range.dup.map! { |n| n * 1000 }, metrics: [] }
|
60
|
+
collect_opts = { zero_fill: !params[:no_zero_fill], range: results[:range] }
|
61
|
+
|
62
|
+
metrics.each do |metric|
|
63
|
+
statistic = "#{datatype}:#{metric}"
|
64
|
+
deferrable = connection_pool.async_values(statistic, range)
|
65
|
+
|
66
|
+
deferrable.errback { |e| return render_error(e.message) }
|
67
|
+
deferrable.callback do |json|
|
68
|
+
values = json[statistic]
|
69
|
+
|
70
|
+
# merge in interval if its not already; interval is always same
|
71
|
+
collect_opts.merge!(interval: json['interval'] || 0) unless collect_opts.has_key?(:interval)
|
72
|
+
# process values for graphing and add to results
|
73
|
+
results[:metrics] << { label: metric, data: collect_for_graph(values, collect_opts) }
|
74
|
+
end
|
75
|
+
end
|
76
|
+
|
77
|
+
cache_control :no_cache, :no_store
|
78
|
+
render_json results
|
79
|
+
end
|
80
|
+
end
|
81
|
+
end
|
82
|
+
end
|
@@ -0,0 +1,87 @@
|
|
1
|
+
module BatsdDash
|
2
|
+
module ConnectionHelpers
|
3
|
+
def connection_pool
|
4
|
+
ConnectionPool.pool or render_error('Connect pool failed to connect to Batsd')
|
5
|
+
end
|
6
|
+
end
|
7
|
+
|
8
|
+
class ConnectionPool
|
9
|
+
class << self
|
10
|
+
attr_accessor :settings
|
11
|
+
attr_reader :pool
|
12
|
+
|
13
|
+
def initialize_connection_pool
|
14
|
+
# default settings (which are defaults from batsd itself)
|
15
|
+
@settings ||= { host: 'localhost', port: 8127, pool_size: 4 }
|
16
|
+
|
17
|
+
if @reconnect_timer
|
18
|
+
@reconnect_timer.cancel
|
19
|
+
@reconnect_timer = nil
|
20
|
+
end
|
21
|
+
|
22
|
+
begin
|
23
|
+
@pool = EventMachine::Synchrony::ConnectionPool.new(size: settings[:pool_size]) do
|
24
|
+
Client.new(settings[:host], settings[:port])
|
25
|
+
end
|
26
|
+
|
27
|
+
rescue Exception => e
|
28
|
+
warn "Connection Pool Error: #{e.message}"
|
29
|
+
|
30
|
+
end
|
31
|
+
end
|
32
|
+
|
33
|
+
def close_connection_pool
|
34
|
+
@pool.close if @pool
|
35
|
+
@pool = nil
|
36
|
+
end
|
37
|
+
|
38
|
+
def start_reconnect_timer
|
39
|
+
unless @reconnect_timer
|
40
|
+
# try to reconnect every 30 seconds
|
41
|
+
@reconnect_timer = EventMachine::Synchrony.add_timer(30) { initialize_connection_pool }
|
42
|
+
end
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
class Client < EventMachine::Synchrony::TCPSocket
|
47
|
+
def write_and_read_json(request)
|
48
|
+
EventMachine::DefaultDeferrable.new.tap do |df|
|
49
|
+
response = String.new
|
50
|
+
parser = Yajl::Parser.new
|
51
|
+
parser.on_parse_complete = Proc.new { |json| df.succeed(json) }
|
52
|
+
|
53
|
+
begin
|
54
|
+
write request
|
55
|
+
|
56
|
+
# keep reading till we hit new line
|
57
|
+
while response[-1] != "\n"
|
58
|
+
response << read(1)
|
59
|
+
end
|
60
|
+
|
61
|
+
parser.parse response
|
62
|
+
|
63
|
+
rescue Exception => e
|
64
|
+
# TODO handle broken pipe
|
65
|
+
#unbind if Errno::EPIPE === e
|
66
|
+
|
67
|
+
df.fail(e)
|
68
|
+
end
|
69
|
+
end
|
70
|
+
end
|
71
|
+
|
72
|
+
def async_available_list
|
73
|
+
write_and_read_json "available"
|
74
|
+
end
|
75
|
+
|
76
|
+
def async_values(statistic, range)
|
77
|
+
write_and_read_json "values #{statistic} #{range[0]} #{range[1]}"
|
78
|
+
end
|
79
|
+
|
80
|
+
def unbind(reason = nil)
|
81
|
+
super(reason)
|
82
|
+
|
83
|
+
ConnectionPool::start_reconnect_timer
|
84
|
+
end
|
85
|
+
end
|
86
|
+
end
|
87
|
+
end
|
@@ -0,0 +1,46 @@
|
|
1
|
+
# helpers for parsing and validating input
|
2
|
+
module BatsdDash
|
3
|
+
module GraphHelper
|
4
|
+
##
|
5
|
+
# This method works directly against values
|
6
|
+
def collect_for_graph(values, opts = {})
|
7
|
+
values.tap do |pts|
|
8
|
+
# remap the values
|
9
|
+
pts.map! { |pt| [pt['timestamp'].to_i * 1000, pt['value'].to_f] }
|
10
|
+
|
11
|
+
# apply zerofill
|
12
|
+
zero_fill!(pts, opts[:range], opts[:interval]) unless pts.empty? || !opts[:zero_fill]
|
13
|
+
end
|
14
|
+
end
|
15
|
+
|
16
|
+
##
|
17
|
+
# The data better be normalized to the interval otherwise
|
18
|
+
# this method may get pissed
|
19
|
+
def zero_fill!(values, range, step)
|
20
|
+
return values if step.zero?
|
21
|
+
|
22
|
+
# convert to milisec
|
23
|
+
step *= 1000
|
24
|
+
|
25
|
+
values.tap do |data|
|
26
|
+
# start from the first timestamp
|
27
|
+
time = data.first.first + step
|
28
|
+
index = 0
|
29
|
+
|
30
|
+
while obj = data[index += 1]
|
31
|
+
current = obj.first
|
32
|
+
|
33
|
+
next if current <= time
|
34
|
+
if current == time
|
35
|
+
time += step
|
36
|
+
next
|
37
|
+
end
|
38
|
+
|
39
|
+
data.insert(index, [time, 0])
|
40
|
+
|
41
|
+
time += step
|
42
|
+
end
|
43
|
+
end
|
44
|
+
end
|
45
|
+
end
|
46
|
+
end
|
@@ -0,0 +1,30 @@
|
|
1
|
+
# helpers for processing params and validating input
|
2
|
+
module BatsdDash
|
3
|
+
module ParamsHelper
|
4
|
+
def parse_metrics
|
5
|
+
metrics = params[:metrics]
|
6
|
+
metrics = [metrics] unless Array === metrics
|
7
|
+
|
8
|
+
metrics.tap { |list| list.reject! { |m| m.nil? || m.empty? } }
|
9
|
+
end
|
10
|
+
|
11
|
+
def parse_time_range
|
12
|
+
start, stop = params[:start], params[:stop]
|
13
|
+
|
14
|
+
if start.nil? && stop.nil?
|
15
|
+
now = Time.now.to_i
|
16
|
+
|
17
|
+
# 1 hr range
|
18
|
+
# TODO make this setting?
|
19
|
+
[ now - 3600 + 1, now ]
|
20
|
+
|
21
|
+
else
|
22
|
+
[start.to_i, stop.to_i].tap do |range|
|
23
|
+
if range[0] <= 0 || range[1] <= 0 || range[0] >= range[1]
|
24
|
+
return nil
|
25
|
+
end
|
26
|
+
end
|
27
|
+
end
|
28
|
+
end
|
29
|
+
end
|
30
|
+
end
|
@@ -0,0 +1,56 @@
|
|
1
|
+
@import "compass/utilities/general/clearfix";
|
2
|
+
@import "compass/css3";
|
3
|
+
@import "compass/reset";
|
4
|
+
|
5
|
+
html, body { height:100%; min-height:100%; background-color:#222; }
|
6
|
+
h1, h2 { color:#0F0; font-family:"Courier New",Courier,monospace; }
|
7
|
+
a { text-decoration:none; color:inherit; }
|
8
|
+
|
9
|
+
.wrap { width:1000px; margin:0 auto; }
|
10
|
+
|
11
|
+
#header {
|
12
|
+
height:60px; border:1px solid #0F0; border-width:0 0 1px 0; margin:0 0 20px;
|
13
|
+
|
14
|
+
background-color: #000000;
|
15
|
+
@include filter-gradient(#000000, #222, vertical);
|
16
|
+
@include background-image(linear-gradient(top, #000000 0%,#222 100%));
|
17
|
+
|
18
|
+
@include pie-clearfix;
|
19
|
+
|
20
|
+
h1 { float:left; font-size:28px; padding:15px 0 0; }
|
21
|
+
ul {
|
22
|
+
float:right; width:300px; margin:15px 0;
|
23
|
+
|
24
|
+
li {
|
25
|
+
float:left; margin:0 20px;
|
26
|
+
a { cursor:pointer; }
|
27
|
+
}
|
28
|
+
}
|
29
|
+
}
|
30
|
+
|
31
|
+
|
32
|
+
#content {
|
33
|
+
color:#FFF; width:1000px; margin:0 auto; font-size:18px;
|
34
|
+
|
35
|
+
h1 { font-size:26px; margin:30px 0; }
|
36
|
+
h2 { font-size:24px; margin:20px 0; }
|
37
|
+
|
38
|
+
p {
|
39
|
+
em { font-style:italic; margin:0 -4px 0 0; }
|
40
|
+
}
|
41
|
+
|
42
|
+
.graph {
|
43
|
+
width:900px; height:400px; margin:0 auto;
|
44
|
+
|
45
|
+
h2 { text-align:center; margin:180px; }
|
46
|
+
}
|
47
|
+
|
48
|
+
.inputs {
|
49
|
+
width:900px; margin:0 auto 30px; text-align:center;
|
50
|
+
|
51
|
+
label { font-weight:bold; }
|
52
|
+
input { width:200px; }
|
53
|
+
}
|
54
|
+
}
|
55
|
+
|
56
|
+
|