heroku_mongo_watcher 0.0.2.alpha → 0.0.4.beta
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +42 -3
- data/heroku_mongo_watcher.gemspec +1 -1
- data/lib/heroku_mongo_watcher/cli.rb +41 -205
- data/lib/heroku_mongo_watcher/configuration.rb +10 -2
- data/lib/heroku_mongo_watcher/data_row.rb +188 -0
- data/lib/heroku_mongo_watcher/version.rb +1 -1
- metadata +10 -9
data/README.md
CHANGED
@@ -1,11 +1,18 @@
|
|
1
1
|
# Heroku Mongo Watcher
|
2
2
|
|
3
|
-
|
3
|
+
Command line utility to monitor both your mongo and heroku instances, and to alert you when things are heating up
|
4
|
+
|
5
|
+
## The Origin
|
6
|
+
|
7
|
+
I have a pretty 'spiky' application that can go from having 10_000 requests per minute to 100_000, we need to notified
|
8
|
+
when things are heating up so we can turn the appropriate dials. We found new relic to be too slow
|
9
|
+
(and not accurate enough once throughput levels got high), so we built this.
|
4
10
|
|
5
11
|
It needed to accomplish the following:
|
6
12
|
|
7
|
-
* See Mongostats and heroku stats at the same time, the key ones being requests per minute, average response time
|
8
|
-
|
13
|
+
* See Mongostats and heroku stats at the same time, the key ones being requests per minute, average response time
|
14
|
+
lock %, and error counts
|
15
|
+
* Have multiple ways of notifying stake holders: colors, beeps and email notifications
|
9
16
|
* Be able to parse the web log for certain errors and other logged events and aggregate data on them
|
10
17
|
|
11
18
|
The output looks like the following ...
|
@@ -19,12 +26,44 @@ The output looks like the following ...
|
|
19
26
|
[11] Cannot find impression when looking for asset
|
20
27
|
20 23935 190 7144 0 43 0 0 /crossdomain.xml| 307 0 618 1 21.6 0|0 260k 221k 15:05:19
|
21
28
|
|
29
|
+
### Legend
|
30
|
+
<table>
|
31
|
+
<tr><td>dynos</td><td>Number of running web instances</td></tr>
|
32
|
+
<tr><td>reqs</td><td>number of requests per sample</td></tr>
|
33
|
+
<tr><td>art</td><td>average request time</td></tr>
|
34
|
+
<tr><td>max</td><td>max request time</td></tr>
|
35
|
+
<tr><td>r_err</td><td>number of router errors, i.e. timeouts</td></tr>
|
36
|
+
<tr><td>w_err</td><td>number of web errros (see below)</td></tr>
|
37
|
+
<tr><td>wait</td><td>average router wait</td></tr>
|
38
|
+
<tr><td>queue</td><td>average router queue</td></tr>
|
39
|
+
<tr><td>slowest</td><td>path of the url that corresponds to the max request time</td></tr>
|
40
|
+
<tr><td>insert</td><td>number of mongo inserts</td></tr>
|
41
|
+
<tr><td>query</td><td>number of mongo queries</td></tr>
|
42
|
+
<tr><td>update</td><td>number of mongo updates</td></tr>
|
43
|
+
<tr><td>faults</td><td>number of mongo page faults</td></tr>
|
44
|
+
<tr><td>qr|qw</td><td>number of mongo's queued read and writes</td></tr>
|
45
|
+
<tr><td>netIn</td><td>size on mongo net in</td></tr>
|
46
|
+
<tr><td>netOut</td><td>size on mongo net out</td></tr>
|
47
|
+
<tr><td>time</td><td>the time sampled</td></tr>
|
48
|
+
</table>
|
49
|
+
|
50
|
+
### Web Errors (w_err)
|
51
|
+
At least for me, one of the key features is aggregating signals from my web log (I look out for certain race conditions,
|
52
|
+
and other errors). You can can configure the `error_messages` array in your .watcher file to define which String we
|
53
|
+
should report on.
|
54
|
+
|
55
|
+
In concert with that is the `display_errors` configuration. If set to true it will aggregate and display the errors
|
56
|
+
found (see output above), set to false it will just put the total in the summary row
|
57
|
+
|
58
|
+
|
22
59
|
## Prereqs
|
23
60
|
|
24
61
|
1. need to have a heroku account, and have the heroku gem running on your machine
|
62
|
+
2. need to be able to run mongostat (e.q. has at least read-only admin access to your mongos)
|
25
63
|
|
26
64
|
## To install
|
27
65
|
|
28
66
|
1. bundle install heroku_mongo_watcher
|
29
67
|
2. create a .watcher file (see the examples) in your user directory ~/.watcher
|
30
68
|
3. then run `bundle exec watcher`
|
69
|
+
4. Ctrl-C out to quit
|
@@ -7,7 +7,7 @@ Gem::Specification.new do |s|
|
|
7
7
|
s.version = HerokuMongoWatcher::VERSION
|
8
8
|
s.authors = ["Jonathan Tushman"]
|
9
9
|
s.email = ["jtushman@gmail.com"]
|
10
|
-
s.homepage = "
|
10
|
+
s.homepage = "https://github.com/jtushman/heroku_mongo_watcher"
|
11
11
|
s.summary = %q{Watches Mongostat and heroku logs to give you a pulse of your application}
|
12
12
|
s.description = %q{Also notifies you when certain thresholds are hit. I have found this much more accurate than New Relic}
|
13
13
|
|
@@ -1,5 +1,5 @@
|
|
1
1
|
require 'heroku_mongo_watcher/configuration'
|
2
|
-
require '
|
2
|
+
require 'heroku_mongo_watcher/data_row'
|
3
3
|
|
4
4
|
#http://stackoverflow.com/a/9117903/192791
|
5
5
|
require 'net/smtp'
|
@@ -16,11 +16,13 @@ require 'tlsmail'
|
|
16
16
|
|
17
17
|
class HerokuMongoWatcher::CLI
|
18
18
|
|
19
|
-
|
19
|
+
def self.config
|
20
|
+
HerokuMongoWatcher::Configuration.instance.config
|
21
|
+
end
|
20
22
|
|
21
23
|
def self.watch(*args)
|
22
|
-
|
23
|
-
|
24
|
+
|
25
|
+
notify("Mongo Watcher enabled!")
|
24
26
|
|
25
27
|
# lock warnings flags
|
26
28
|
@lock_critical_notified = false
|
@@ -30,17 +32,8 @@ class HerokuMongoWatcher::CLI
|
|
30
32
|
@art_critical_notified = false
|
31
33
|
@art_warning_notified = false
|
32
34
|
|
33
|
-
|
34
|
-
@
|
35
|
-
@total_service = 0
|
36
|
-
@total_wait = 0
|
37
|
-
@total_queue = 0
|
38
|
-
@total_router_errors = 0
|
39
|
-
@total_web_errors = 0
|
40
|
-
@max_service = 0
|
41
|
-
@dynos = 0
|
42
|
-
@slowest_request = nil
|
43
|
-
@errors = {}
|
35
|
+
@current_row = HerokuMongoWatcher::DataRow.new
|
36
|
+
@last_row = HerokuMongoWatcher::DataRow.new
|
44
37
|
|
45
38
|
@mutex = Mutex.new
|
46
39
|
|
@@ -48,8 +41,10 @@ class HerokuMongoWatcher::CLI
|
|
48
41
|
heroku_watcher = Thread.new('heroku_logs') do
|
49
42
|
IO.popen("heroku logs --tail --app #{config[:heroku_appname]} --account #{config[:heroku_account]}") do |f|
|
50
43
|
while line = f.gets
|
51
|
-
|
52
|
-
|
44
|
+
@mutex.synchronize do
|
45
|
+
@current_row.process_heroku_router_line(line) if line.include? 'heroku[router]'
|
46
|
+
@current_row.process_heroku_web_line(line) if line.include? 'app[web'
|
47
|
+
end
|
53
48
|
end
|
54
49
|
end
|
55
50
|
end
|
@@ -63,176 +58,42 @@ class HerokuMongoWatcher::CLI
|
|
63
58
|
dynos += 1 if line =~ /^web/ && line.split(' ')[1] == 'up'
|
64
59
|
end
|
65
60
|
end
|
66
|
-
@mutex.synchronize { @dynos = dynos }
|
61
|
+
@mutex.synchronize { @current_row.dynos = dynos }
|
67
62
|
sleep(30)
|
68
63
|
end
|
69
64
|
end
|
70
65
|
|
71
|
-
|
72
|
-
|
73
|
-
puts "dyno reqs art max r_err w_err wait queue slowest | insert query update faults locked qr|qw netIn netOut time |"
|
66
|
+
|
67
|
+
HerokuMongoWatcher::DataRow.print_header
|
74
68
|
|
75
69
|
IO.popen("mongostat --rowcount 0 #{config[:interval]} --host #{config[:mongo_host]} --username #{config[:mongo_username]} --password #{config[:mongo_password]} --noheaders") do |f|
|
76
70
|
while line = f.gets
|
77
|
-
|
78
|
-
|
79
|
-
|
80
|
-
|
81
|
-
heroku_watcher.join
|
82
|
-
heroku_ps.join
|
83
|
-
|
84
|
-
end
|
85
|
-
|
86
|
-
protected
|
87
|
-
|
88
|
-
def self.process_mongo_line(line)
|
89
|
-
items = line.split
|
90
|
-
|
91
|
-
inserts = items[0]
|
92
|
-
query = items[1]
|
93
|
-
update = items[2]
|
94
|
-
delete = items[3]
|
95
|
-
getmore = items[4]
|
96
|
-
command = items[5]
|
97
|
-
flushes = items[6]
|
98
|
-
mapped = items[7]
|
99
|
-
vsize = items[8]
|
100
|
-
res = items[9]
|
101
|
-
faults = items[10]
|
102
|
-
locked = items[11]
|
103
|
-
idx_miss = items[12]
|
104
|
-
qrw = items[13]
|
105
|
-
arw = items[14]
|
106
|
-
netIn = items[15]
|
107
|
-
netOut = items[16]
|
108
|
-
conn = items[17]
|
109
|
-
set = items[18]
|
110
|
-
repl = items[19]
|
111
|
-
time = items[20]
|
112
|
-
|
113
|
-
@mutex.synchronize do
|
114
|
-
art = average_response_time
|
115
|
-
err = @total_router_errors
|
116
|
-
average_wait = @total_lines > 0 ? @total_wait / @total_lines : 'N/A'
|
117
|
-
average_queue = @total_lines > 0 ? @total_queue / @total_lines : 'N/A'
|
118
|
-
|
119
|
-
print_errors
|
120
|
-
|
121
|
-
color_print @dynos, length: 4
|
122
|
-
color_print @total_lines
|
123
|
-
color_print art, warning: 1000, critical: 10_000, bold: true
|
124
|
-
color_print @max_service, warning: 10_000, critical: 20_000
|
125
|
-
color_print err, warning: 1, critical: 10
|
126
|
-
color_print @total_web_errors, warning: 1, critical: 10
|
127
|
-
color_print average_wait, warning: 10, critical: 100
|
128
|
-
color_print average_queue, warning: 10, critical: 100
|
129
|
-
color_print @slowest_request.slice(0,25), length: 28
|
130
|
-
print '|'
|
131
|
-
color_print inserts
|
132
|
-
color_print query
|
133
|
-
color_print update
|
134
|
-
color_print faults
|
135
|
-
color_print locked, bold: true, warning: 70, critical: 90
|
136
|
-
color_print qrw
|
137
|
-
color_print netIn
|
138
|
-
color_print netOut
|
139
|
-
color_print time, length: 10
|
140
|
-
printf "\n"
|
141
|
-
|
142
|
-
check_and_notify_locks(locked)
|
143
|
-
check_and_notify_response_time(art, @total_lines)
|
144
|
-
|
145
|
-
reset_memos
|
146
|
-
|
147
|
-
end
|
148
|
-
|
71
|
+
next unless line =~ /^/ && !(line =~ /^connected/)
|
72
|
+
@mutex.synchronize do
|
73
|
+
@current_row.process_mongo_line(line)
|
149
74
|
|
150
|
-
|
75
|
+
@current_row.print_row
|
151
76
|
|
152
|
-
|
153
|
-
|
154
|
-
end
|
155
|
-
|
156
|
-
def self.print_errors
|
157
|
-
if config[:print_errors] &&@errors && @errors.keys && @errors.keys.length > 0
|
158
|
-
@errors.each do |error,count|
|
159
|
-
puts "\t\t[#{count}] #{error}"
|
160
|
-
end
|
161
|
-
end
|
162
|
-
end
|
77
|
+
check_and_notify_locks
|
78
|
+
check_and_notify_response_time
|
163
79
|
|
164
|
-
|
165
|
-
|
166
|
-
|
80
|
+
@last_row = @current_row
|
81
|
+
@current_row = HerokuMongoWatcher::DataRow.new
|
82
|
+
@current_row.dynos = @last_row.dynos
|
167
83
|
|
168
|
-
if error_messages.any? { |mes| line.include? mes }
|
169
|
-
items = line.split
|
170
|
-
time = items[0]
|
171
|
-
process = items[1]
|
172
|
-
clean_line = line.sub(time,'').sub(process,'').strip
|
173
|
-
@mutex.synchronize do
|
174
|
-
@total_web_errors += 1
|
175
|
-
if @errors.has_key? clean_line
|
176
|
-
@errors[clean_line] = @errors[clean_line] + 1
|
177
|
-
else
|
178
|
-
@errors[clean_line] = 1
|
179
84
|
end
|
180
85
|
end
|
181
86
|
end
|
182
87
|
|
183
|
-
|
184
|
-
|
185
|
-
end
|
186
|
-
|
187
|
-
def self.process_heroku_router_line(line)
|
188
|
-
# 2012-07-05T20:24:10+00:00 heroku[router]: GET myapp.com/pxl/4fdbc97dc6b36c0030001160?value=1 dyno=web.14 queue=0 wait=0ms service=8ms status=200 bytes=35
|
189
|
-
|
190
|
-
# or if error
|
191
|
-
|
192
|
-
#2012-07-05T20:17:12+00:00 heroku[router]: Error H12 (Request timeout) -> GET myapp.com/crossdomain.xml dyno=web.4 queue= wait= service=30000ms status=503 bytes=0
|
193
|
-
|
194
|
-
items = line.split
|
195
|
-
|
196
|
-
if line =~ /Error/
|
197
|
-
@mutex.synchronize { @total_router_errors += 1 }
|
198
|
-
else
|
199
|
-
|
200
|
-
|
201
|
-
time = items[0]
|
202
|
-
process = items[1]
|
203
|
-
http_type = items[2]
|
204
|
-
url = items[3]
|
205
|
-
dyno = items[4].split('=').last if items[4]
|
206
|
-
queue = items[5].split('=').last.sub('ms', '') if items[5]
|
207
|
-
wait = items[6].split('=').last.sub('ms', '') if items[6]
|
208
|
-
service = items[7].split('=').last.sub('ms', '') if items[7]
|
209
|
-
status = items[8].split('=').last if items[8]
|
210
|
-
bytes = items[9].split('=').last if items[9]
|
211
|
-
|
212
|
-
if is_number?(service) && is_number?(wait) && is_number?(queue)
|
213
|
-
@mutex.synchronize do
|
214
|
-
@total_lines +=1
|
215
|
-
@total_service += Integer(service) if service
|
216
|
-
@total_wait += Integer(wait) if wait
|
217
|
-
@total_queue += Integer(queue) if queue
|
218
|
-
if Integer(service) > @max_service
|
219
|
-
@max_service = Integer(service)
|
220
|
-
@slowest_request = URI('http://' + url).path
|
221
|
-
end
|
222
|
-
end
|
223
|
-
end
|
224
|
-
|
225
|
-
end
|
88
|
+
heroku_watcher.join
|
89
|
+
heroku_ps.join
|
226
90
|
|
227
91
|
end
|
228
92
|
|
229
|
-
|
230
|
-
@total_service = @total_lines = @total_wait = @total_queue = @total_router_errors = @total_web_errors = @max_service = 0
|
231
|
-
@errors = {}
|
232
|
-
end
|
93
|
+
protected
|
233
94
|
|
234
|
-
def self.check_and_notify_locks
|
235
|
-
l = Float(
|
95
|
+
def self.check_and_notify_locks
|
96
|
+
l = Float(@current_row.lock)
|
236
97
|
if l > 90
|
237
98
|
notify '[CRITICAL] Locks above 90%' unless @lock_critical_notified
|
238
99
|
elsif l > 70
|
@@ -247,15 +108,15 @@ class HerokuMongoWatcher::CLI
|
|
247
108
|
end
|
248
109
|
end
|
249
110
|
|
250
|
-
def self.check_and_notify_response_time
|
251
|
-
return unless
|
252
|
-
if
|
253
|
-
notify "[SEVERE WARNING] Application not healthy | [#{@
|
111
|
+
def self.check_and_notify_response_time
|
112
|
+
return unless @current_row.total_requests > 200
|
113
|
+
if @current_row.average_response_time > 10_000 || @current_row.total_router_errors > 100
|
114
|
+
notify "[SEVERE WARNING] Application not healthy | [#{@current_row.total_requests} rpm,#{@current_row.average_response_time} art]" unless @art_critical_notified
|
254
115
|
# @art_critical_notified = true
|
255
|
-
elsif
|
256
|
-
notify "[WARNING] Application heating up | [#{@
|
116
|
+
elsif @current_row.average_response_time > 500 || @current_row.total_router_errors > 10 || @current_row.total_requests > 30_000
|
117
|
+
notify "[WARNING] Application heating up | [#{@current_row.total_requests} rpm,#{@current_row.average_response_time} art]" unless @art_warning_notified
|
257
118
|
# @art_warning_notified = true
|
258
|
-
elsif
|
119
|
+
elsif @current_row.average_response_time < 300
|
259
120
|
if @art_warning_notified || @art_critical_notified
|
260
121
|
@art_warning_notified = false
|
261
122
|
@art_critical_notified = false
|
@@ -270,16 +131,6 @@ class HerokuMongoWatcher::CLI
|
|
270
131
|
end
|
271
132
|
end
|
272
133
|
|
273
|
-
def self.is_number?(string)
|
274
|
-
_is_number = true
|
275
|
-
begin
|
276
|
-
num = Integer(string)
|
277
|
-
rescue
|
278
|
-
_is_number = false
|
279
|
-
end
|
280
|
-
_is_number
|
281
|
-
end
|
282
|
-
|
283
134
|
def self.send_email(to, msg)
|
284
135
|
return unless config[:gmail_username] && config[:gmail_password]
|
285
136
|
content = [
|
@@ -287,11 +138,11 @@ class HerokuMongoWatcher::CLI
|
|
287
138
|
"To: #{to}",
|
288
139
|
"Subject: #{msg}",
|
289
140
|
"",
|
290
|
-
"RPM: #{@
|
291
|
-
"Average Reponse Time: #{average_response_time}",
|
292
|
-
"Application Errors: #{@total_web_errors}",
|
293
|
-
"Router Errors (timeouts): #{@total_router_errors}",
|
294
|
-
"Dynos: #{@dynos}"
|
141
|
+
"RPM: #{@last_row.total_requests}",
|
142
|
+
"Average Reponse Time: #{@last_row.average_response_time}",
|
143
|
+
"Application Errors: #{@last_row.total_web_errors}",
|
144
|
+
"Router Errors (timeouts): #{@last_row.total_router_errors}",
|
145
|
+
"Dynos: #{@last_row.dynos}"
|
295
146
|
].join("\r\n")
|
296
147
|
|
297
148
|
Net::SMTP.enable_tls(OpenSSL::SSL::VERIFY_NONE)
|
@@ -300,19 +151,4 @@ class HerokuMongoWatcher::CLI
|
|
300
151
|
end
|
301
152
|
end
|
302
153
|
|
303
|
-
def self.color_print(field, options ={})
|
304
|
-
options[:length] = 7 unless options[:length]
|
305
|
-
print Term::ANSIColor.bold if options[:bold] == true
|
306
|
-
if options[:critical] && is_number?(field) && Integer(field) > options[:critical]
|
307
|
-
print "\a" #beep
|
308
|
-
print Term::ANSIColor.red
|
309
|
-
print Term::ANSIColor.bold
|
310
|
-
elsif options[:warning] && is_number?(field) && Integer(field) > options[:warning]
|
311
|
-
print Term::ANSIColor.yellow
|
312
|
-
end
|
313
|
-
printf "%#{options[:length]}s", field
|
314
|
-
print Term::ANSIColor.clear
|
315
|
-
end
|
316
|
-
|
317
|
-
|
318
154
|
end
|
@@ -1,4 +1,7 @@
|
|
1
|
-
|
1
|
+
require 'singleton'
|
2
|
+
|
3
|
+
class HerokuMongoWatcher::Configuration
|
4
|
+
include Singleton
|
2
5
|
|
3
6
|
@@config = {
|
4
7
|
error_messages: ['Error', 'Exception', 'Cannot find impression', 'Timed out running'],
|
@@ -11,11 +14,16 @@ module HerokuMongoWatcher::Configuration
|
|
11
14
|
mongo_password: '',
|
12
15
|
heroku_appname: '',
|
13
16
|
heroku_account: '',
|
14
|
-
|
17
|
+
print_errors: true
|
15
18
|
}
|
16
19
|
|
17
20
|
@@valid_config_keys = @@config.keys
|
18
21
|
|
22
|
+
def initialize
|
23
|
+
f = File.join(File.expand_path('~'),'.watcher')
|
24
|
+
configure_with(f)
|
25
|
+
end
|
26
|
+
|
19
27
|
def config
|
20
28
|
@@config
|
21
29
|
end
|
@@ -0,0 +1,188 @@
|
|
1
|
+
require 'term/ansicolor'
|
2
|
+
require 'heroku_mongo_watcher/configuration'
|
3
|
+
class HerokuMongoWatcher::DataRow
|
4
|
+
|
5
|
+
# Heroku Attributes
|
6
|
+
@@attributes = [:total_requests, :total_service, :total_wait,
|
7
|
+
:total_queue, :total_router_errors, :total_web_errors,
|
8
|
+
:max_service, :slowest_request, :errors, :dynos]
|
9
|
+
|
10
|
+
# Mongo Attributes
|
11
|
+
@@attributes.concat [:inserts, :queries, :ops, :updates, :deletes,
|
12
|
+
:faults, :lock, :qrw, :net_in, :net_out]
|
13
|
+
|
14
|
+
@@attributes.each { |attr| attr_accessor attr }
|
15
|
+
|
16
|
+
def initialize
|
17
|
+
@@attributes.each { |attr| send "#{attr}=", 0 }
|
18
|
+
self.slowest_request = nil
|
19
|
+
self.errors = {}
|
20
|
+
end
|
21
|
+
|
22
|
+
def config
|
23
|
+
HerokuMongoWatcher::Configuration.instance.config
|
24
|
+
end
|
25
|
+
|
26
|
+
def average_response_time
|
27
|
+
total_requests > 0 ? total_service / total_requests : 'N/A'
|
28
|
+
end
|
29
|
+
|
30
|
+
def average_wait
|
31
|
+
total_requests > 0 ? total_wait / total_requests : 'N/A'
|
32
|
+
end
|
33
|
+
|
34
|
+
def average_queue
|
35
|
+
total_requests > 0 ? total_queue / total_requests : 'N/A'
|
36
|
+
end
|
37
|
+
|
38
|
+
def process_heroku_router_line(line)
|
39
|
+
items = line.split
|
40
|
+
|
41
|
+
if line =~ /Error/
|
42
|
+
# Note: The lion share of these are timeouts
|
43
|
+
#Full list here: https://devcenter.heroku.com/articles/error-codes
|
44
|
+
self.total_router_errors += 1
|
45
|
+
else
|
46
|
+
time = items[0]
|
47
|
+
process = items[1]
|
48
|
+
http_type = items[2]
|
49
|
+
url = items[3]
|
50
|
+
dyno = items[4].split('=').last if items[4]
|
51
|
+
queue = items[5].split('=').last.sub('ms', '') if items[5]
|
52
|
+
wait = items[6].split('=').last.sub('ms', '') if items[6]
|
53
|
+
service = items[7].split('=').last.sub('ms', '') if items[7]
|
54
|
+
status = items[8].split('=').last if items[8]
|
55
|
+
bytes = items[9].split('=').last if items[9]
|
56
|
+
|
57
|
+
if is_number?(service) && is_number?(wait) && is_number?(queue)
|
58
|
+
self.total_requests +=1
|
59
|
+
self.total_service += Integer(service) if service
|
60
|
+
self.total_wait += Integer(wait) if wait
|
61
|
+
self.total_queue += Integer(queue) if queue
|
62
|
+
if Integer(service) > self.max_service
|
63
|
+
self.max_service = Integer(service)
|
64
|
+
self.slowest_request = URI('http://' + url).path
|
65
|
+
end
|
66
|
+
end
|
67
|
+
|
68
|
+
end
|
69
|
+
end
|
70
|
+
|
71
|
+
def process_heroku_web_line(line)
|
72
|
+
# Only care about errors
|
73
|
+
error_messages = config[:error_messages]
|
74
|
+
|
75
|
+
if error_messages.any? { |mes| line.include? mes }
|
76
|
+
items = line.split
|
77
|
+
time = items[0]
|
78
|
+
process = items[1]
|
79
|
+
clean_line = line.sub(time, '').sub(process, '').strip
|
80
|
+
|
81
|
+
self.total_web_errors += 1
|
82
|
+
if self.errors.has_key? clean_line
|
83
|
+
self.errors[clean_line] = self.errors[clean_line] + 1
|
84
|
+
else
|
85
|
+
self.errors[clean_line] = 1
|
86
|
+
end
|
87
|
+
end
|
88
|
+
|
89
|
+
end
|
90
|
+
|
91
|
+
def process_mongo_line(line)
|
92
|
+
items = line.split
|
93
|
+
|
94
|
+
@inserts = items[0]
|
95
|
+
@queries = items[1]
|
96
|
+
@updates = items[2]
|
97
|
+
delete = items[3]
|
98
|
+
getmore = items[4]
|
99
|
+
command = items[5]
|
100
|
+
flushes = items[6]
|
101
|
+
mapped = items[7]
|
102
|
+
vsize = items[8]
|
103
|
+
res = items[9]
|
104
|
+
@faults = items[10]
|
105
|
+
@locked = items[11]
|
106
|
+
idx_miss = items[12]
|
107
|
+
@qrw = items[13]
|
108
|
+
arw = items[14]
|
109
|
+
@net_in = items[15]
|
110
|
+
@net_out = items[16]
|
111
|
+
conn = items[17]
|
112
|
+
set = items[18]
|
113
|
+
repl = items[19]
|
114
|
+
@mongo_time = items[20]
|
115
|
+
|
116
|
+
end
|
117
|
+
|
118
|
+
def self.print_header
|
119
|
+
puts
|
120
|
+
puts "|<---- heroku stats ------------------------------------------------------------>|<----mongo stats ------------------------------------------------------->|"
|
121
|
+
puts "dyno reqs art max r_err w_err wait queue slowest | insert query update faults locked qr|qw netIn netOut time |"
|
122
|
+
end
|
123
|
+
|
124
|
+
def print_row
|
125
|
+
print_errors
|
126
|
+
|
127
|
+
color_print @dynos, length: 4
|
128
|
+
color_print @total_requests, warning: 30_000, critical: 50_000
|
129
|
+
color_print average_response_time, warning: 1000, critical: 10_000, bold: true
|
130
|
+
color_print @max_service, warning: 20_000, critical: 27_000
|
131
|
+
color_print @total_router_errors, warning: 1, critical: 10
|
132
|
+
color_print @total_web_errors, warning: 1, critical: 10
|
133
|
+
color_print average_wait, warning: 10, critical: 100
|
134
|
+
color_print average_queue, warning: 10, critical: 100
|
135
|
+
color_print @slowest_request, length: 28, slice: 25
|
136
|
+
print '|'
|
137
|
+
color_print @inserts
|
138
|
+
color_print @queries
|
139
|
+
color_print @updates
|
140
|
+
color_print @faults
|
141
|
+
color_print @locked, bold: true, warning: 40, critical: 70
|
142
|
+
color_print @qrw
|
143
|
+
color_print @net_in
|
144
|
+
color_print @net_out
|
145
|
+
color_print @mongo_time, length: 10
|
146
|
+
printf "\n"
|
147
|
+
end
|
148
|
+
|
149
|
+
def print_errors
|
150
|
+
if config[:print_errors] && @errors && @errors.keys && @errors.keys.length > 0
|
151
|
+
@errors.each do |error,count|
|
152
|
+
puts "\t\t[#{count}] #{error}"
|
153
|
+
end
|
154
|
+
end
|
155
|
+
end
|
156
|
+
|
157
|
+
private
|
158
|
+
|
159
|
+
def is_number?(string)
|
160
|
+
_is_number = true
|
161
|
+
begin
|
162
|
+
num = Integer(string)
|
163
|
+
rescue
|
164
|
+
_is_number = false
|
165
|
+
end
|
166
|
+
_is_number
|
167
|
+
end
|
168
|
+
|
169
|
+
def color_print(field, options ={})
|
170
|
+
options[:length] = 7 unless options[:length]
|
171
|
+
print Term::ANSIColor.bold if options[:bold] == true
|
172
|
+
if options[:critical] && is_number?(field) && Integer(field) > options[:critical]
|
173
|
+
print "\a" #beep
|
174
|
+
print Term::ANSIColor.red
|
175
|
+
print Term::ANSIColor.bold
|
176
|
+
elsif options[:warning] && is_number?(field) && Integer(field) > options[:warning]
|
177
|
+
print Term::ANSIColor.yellow
|
178
|
+
end
|
179
|
+
|
180
|
+
str = field.to_s
|
181
|
+
str = str.slice(0, options[:slice]) if options[:slice] && str && str.length > 0
|
182
|
+
|
183
|
+
printf "%#{options[:length]}s", str
|
184
|
+
print Term::ANSIColor.clear
|
185
|
+
end
|
186
|
+
|
187
|
+
|
188
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: heroku_mongo_watcher
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.4.beta
|
5
5
|
prerelease: 6
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,11 +9,11 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2012-07-
|
12
|
+
date: 2012-07-23 00:00:00.000000000Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: term-ansicolor
|
16
|
-
requirement: &
|
16
|
+
requirement: &2157942700 !ruby/object:Gem::Requirement
|
17
17
|
none: false
|
18
18
|
requirements:
|
19
19
|
- - ! '>='
|
@@ -21,10 +21,10 @@ dependencies:
|
|
21
21
|
version: '0'
|
22
22
|
type: :runtime
|
23
23
|
prerelease: false
|
24
|
-
version_requirements: *
|
24
|
+
version_requirements: *2157942700
|
25
25
|
- !ruby/object:Gem::Dependency
|
26
26
|
name: tlsmail
|
27
|
-
requirement: &
|
27
|
+
requirement: &2157942040 !ruby/object:Gem::Requirement
|
28
28
|
none: false
|
29
29
|
requirements:
|
30
30
|
- - ! '>='
|
@@ -32,10 +32,10 @@ dependencies:
|
|
32
32
|
version: '0'
|
33
33
|
type: :runtime
|
34
34
|
prerelease: false
|
35
|
-
version_requirements: *
|
35
|
+
version_requirements: *2157942040
|
36
36
|
- !ruby/object:Gem::Dependency
|
37
37
|
name: heroku
|
38
|
-
requirement: &
|
38
|
+
requirement: &2157941260 !ruby/object:Gem::Requirement
|
39
39
|
none: false
|
40
40
|
requirements:
|
41
41
|
- - ! '>='
|
@@ -43,7 +43,7 @@ dependencies:
|
|
43
43
|
version: '0'
|
44
44
|
type: :runtime
|
45
45
|
prerelease: false
|
46
|
-
version_requirements: *
|
46
|
+
version_requirements: *2157941260
|
47
47
|
description: Also notifies you when certain thresholds are hit. I have found this
|
48
48
|
much more accurate than New Relic
|
49
49
|
email:
|
@@ -63,8 +63,9 @@ files:
|
|
63
63
|
- lib/heroku_mongo_watcher.rb
|
64
64
|
- lib/heroku_mongo_watcher/cli.rb
|
65
65
|
- lib/heroku_mongo_watcher/configuration.rb
|
66
|
+
- lib/heroku_mongo_watcher/data_row.rb
|
66
67
|
- lib/heroku_mongo_watcher/version.rb
|
67
|
-
homepage:
|
68
|
+
homepage: https://github.com/jtushman/heroku_mongo_watcher
|
68
69
|
licenses: []
|
69
70
|
post_install_message:
|
70
71
|
rdoc_options: []
|