cloud66-bluepill 0.0.62

Sign up to get free protection for your applications and to get access to all the features.
Files changed (50) hide show
  1. data/.gitignore +10 -0
  2. data/.rspec +1 -0
  3. data/DESIGN.md +10 -0
  4. data/Gemfile +10 -0
  5. data/LICENSE +22 -0
  6. data/README.md +349 -0
  7. data/Rakefile +38 -0
  8. data/bin/bluepill +124 -0
  9. data/bin/bpsv +3 -0
  10. data/bin/sample_forking_server +53 -0
  11. data/bluepill.gemspec +37 -0
  12. data/examples/example.rb +87 -0
  13. data/examples/new_example.rb +89 -0
  14. data/examples/new_runit_example.rb +29 -0
  15. data/examples/runit_example.rb +26 -0
  16. data/lib/bluepill.rb +38 -0
  17. data/lib/bluepill/application.rb +215 -0
  18. data/lib/bluepill/application/client.rb +8 -0
  19. data/lib/bluepill/application/server.rb +23 -0
  20. data/lib/bluepill/condition_watch.rb +51 -0
  21. data/lib/bluepill/controller.rb +122 -0
  22. data/lib/bluepill/dsl.rb +12 -0
  23. data/lib/bluepill/dsl/app_proxy.rb +25 -0
  24. data/lib/bluepill/dsl/process_factory.rb +122 -0
  25. data/lib/bluepill/dsl/process_proxy.rb +44 -0
  26. data/lib/bluepill/group.rb +72 -0
  27. data/lib/bluepill/logger.rb +63 -0
  28. data/lib/bluepill/process.rb +514 -0
  29. data/lib/bluepill/process_conditions.rb +14 -0
  30. data/lib/bluepill/process_conditions/always_true.rb +18 -0
  31. data/lib/bluepill/process_conditions/cpu_usage.rb +19 -0
  32. data/lib/bluepill/process_conditions/file_time.rb +26 -0
  33. data/lib/bluepill/process_conditions/http.rb +58 -0
  34. data/lib/bluepill/process_conditions/mem_usage.rb +32 -0
  35. data/lib/bluepill/process_conditions/process_condition.rb +22 -0
  36. data/lib/bluepill/process_journal.rb +219 -0
  37. data/lib/bluepill/process_statistics.rb +27 -0
  38. data/lib/bluepill/socket.rb +58 -0
  39. data/lib/bluepill/system.rb +265 -0
  40. data/lib/bluepill/trigger.rb +60 -0
  41. data/lib/bluepill/triggers/flapping.rb +56 -0
  42. data/lib/bluepill/util/rotational_array.rb +20 -0
  43. data/lib/bluepill/version.rb +4 -0
  44. data/local-bluepill +129 -0
  45. data/spec/lib/bluepill/logger_spec.rb +3 -0
  46. data/spec/lib/bluepill/process_spec.rb +96 -0
  47. data/spec/lib/bluepill/process_statistics_spec.rb +24 -0
  48. data/spec/lib/bluepill/system_spec.rb +36 -0
  49. data/spec/spec_helper.rb +15 -0
  50. metadata +302 -0
data/.gitignore ADDED
@@ -0,0 +1,10 @@
1
+ *.sw?
2
+ .DS_Store
3
+ coverage/
4
+ .yardoc/
5
+ doc/
6
+ pkg/
7
+ *.gem
8
+ .bundle/
9
+ Gemfile.lock
10
+ .idea
data/.rspec ADDED
@@ -0,0 +1 @@
1
+ -r ./spec/spec_helper.rb -c -f progress
data/DESIGN.md ADDED
@@ -0,0 +1,10 @@
1
+ ## Bluepill Design
2
+ Here are just some bullet points of the design. We'll add details later.
3
+
4
+ * Each process monitors a single _application_, so you can have multiple bluepill processes on a system
5
+ * Use rotational arrays for storing historical data for monitoring process conditions
6
+ * Memo-ize output of _ps_ per tick as an optimization for applications with many processes
7
+ * Use socket files to communicate between CLI and daemon
8
+ * DSL is a separate layer, the core of the monitoring just uses regular initializers, etc. DSL is simply for ease of use and should not interfere with business logic
9
+ * Sequentially process user issued commands so no weird race cases occur
10
+ * Triggers are notified by the state machine on any process state transitions
data/Gemfile ADDED
@@ -0,0 +1,10 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in bluepill.gemspec
4
+ gemspec
5
+
6
+ # YARD helper for ruby 1.8 (already embedded into ruby 1.9)
7
+ gem "ripper", :platforms => :ruby_18, :group => :development
8
+
9
+ # Code coverage tool that works on Ruby 1.9
10
+ gem "simplecov", ">= 0.4.0", :platforms => :ruby_19, :group => :test
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2009 Arya Asemanfar, Rohith Ravi, Gary Tsang
2
+
3
+ Permission is hereby granted, free of charge, to any person
4
+ obtaining a copy of this software and associated documentation
5
+ files (the "Software"), to deal in the Software without
6
+ restriction, including without limitation the rights to use,
7
+ copy, modify, merge, publish, distribute, sublicense, and/or sell
8
+ copies of the Software, and to permit persons to whom the
9
+ Software is furnished to do so, subject to the following
10
+ conditions:
11
+
12
+ The above copyright notice and this permission notice shall be
13
+ included in all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
16
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES
17
+ OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
18
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT
19
+ HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY,
20
+ WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
21
+ FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR
22
+ OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,349 @@
1
+ # Bluepill
2
+ Bluepill is a simple process monitoring tool written in Ruby.
3
+
4
+ ## Installation
5
+ It's hosted on [rubygems.org][rubygems].
6
+
7
+ sudo gem install bluepill
8
+
9
+ In order to take advantage of logging with syslog, you also need to setup your syslog to log the local6 facility. Edit the appropriate config file for your syslogger (/etc/syslog.conf for syslog) and add a line for local6:
10
+
11
+ local6.* /var/log/bluepill.log
12
+
13
+ You'll also want to add _/var/log/bluepill.log_ to _/etc/logrotate.d/syslog_ so that it gets rotated.
14
+
15
+ Lastly, create the _/var/run/bluepill_ directory for bluepill to store its pid and sock files.
16
+
17
+ ## Usage
18
+ ### Config
19
+ Bluepill organizes processes into 3 levels: application -> group -> process. Each process has a few attributes that tell bluepill how to start, stop, and restart it, where to look or put the pid file, what process conditions to monitor and the options for each of those.
20
+
21
+ The minimum config file looks something like this:
22
+
23
+ ```ruby
24
+ Bluepill.application("app_name") do |app|
25
+ app.process("process_name") do |process|
26
+ process.start_command = "/usr/bin/some_start_command"
27
+ process.pid_file = "/tmp/some_pid_file.pid"
28
+ end
29
+ end
30
+ ```
31
+
32
+ Note that since we specified a PID file and start command, bluepill assumes the process will daemonize itself. If we wanted bluepill to daemonize it for us, we can do (note we still need to specify a PID file):
33
+
34
+ ```ruby
35
+ Bluepill.application("app_name") do |app|
36
+ app.process("process_name") do |process|
37
+ process.start_command = "/usr/bin/some_start_command"
38
+ process.pid_file = "/tmp/some_pid_file.pid"
39
+ process.daemonize = true
40
+ end
41
+ end
42
+ ```
43
+
44
+ If you don't specify a stop command, a TERM signal will be sent by default. Similarly, the default restart action is to issue stop and then start.
45
+
46
+ Now if we want to do something more meaningful, like actually monitor the process, we do:
47
+
48
+ ```ruby
49
+ Bluepill.application("app_name") do |app|
50
+ app.process("process_name") do |process|
51
+ process.start_command = "/usr/bin/some_start_command"
52
+ process.pid_file = "/tmp/some_pid_file.pid"
53
+
54
+ process.checks :cpu_usage, :every => 10.seconds, :below => 5, :times => 3
55
+ end
56
+ end
57
+ ```
58
+
59
+ We added a line that checks every 10 seconds to make sure the cpu usage of this process is below 5 percent; 3 failed checks results in a restart. We can specify a two-element array for the _times_ option to say that it 3 out of 5 failed attempts results in a restart.
60
+
61
+ To watch memory usage, we just add one more line:
62
+
63
+ ```ruby
64
+ Bluepill.application("app_name") do |app|
65
+ app.process("process_name") do |process|
66
+ process.start_command = "/usr/bin/some_start_command"
67
+ process.pid_file = "/tmp/some_pid_file.pid"
68
+
69
+ process.checks :cpu_usage, :every => 10.seconds, :below => 5, :times => 3
70
+ process.checks :mem_usage, :every => 10.seconds, :below => 100.megabytes, :times => [3,5]
71
+ end
72
+ end
73
+ ```
74
+
75
+ To watch the modification time of a file, e.g. a log file to ensure the process is actually working add one more line:
76
+
77
+ ```ruby
78
+ Bluepill.application("app_name") do |app|
79
+ app.process("process_name") do |process|
80
+ process.start_command = "/usr/bin/some_start_command"
81
+ process.pid_file = "/tmp/some_pid_file.pid"
82
+
83
+ process.checks :cpu_usage, :every => 10.seconds, :below => 5, :times => 3
84
+ process.checks :mem_usage, :every => 10.seconds, :below => 100.megabytes, :times => [3,5]
85
+ process.checks :file_time, :every => 60.seconds, :below => 3.minutes, :filename => "/tmp/some_file.log", :times => 2
86
+ end
87
+ end
88
+ ```
89
+
90
+
91
+
92
+ We can tell bluepill to give a process some grace time to start/stop/restart before resuming monitoring:
93
+
94
+ ```ruby
95
+ Bluepill.application("app_name") do |app|
96
+ app.process("process_name") do |process|
97
+ process.start_command = "/usr/bin/some_start_command"
98
+ process.pid_file = "/tmp/some_pid_file.pid"
99
+ process.start_grace_time = 3.seconds
100
+ process.stop_grace_time = 5.seconds
101
+ process.restart_grace_time = 8.seconds
102
+
103
+ process.checks :cpu_usage, :every => 10.seconds, :below => 5, :times => 3
104
+ process.checks :mem_usage, :every => 10.seconds, :below => 100.megabytes, :times => [3,5]
105
+ end
106
+ end
107
+ ```
108
+
109
+ We can group processes by name:
110
+
111
+ ```ruby
112
+ Bluepill.application("app_name") do |app|
113
+ 5.times do |i|
114
+ app.process("process_name_#{i}") do |process|
115
+ process.group = "mongrels"
116
+ process.start_command = "/usr/bin/some_start_command"
117
+ process.pid_file = "/tmp/some_pid_file.pid"
118
+ end
119
+ end
120
+ end
121
+ ```
122
+
123
+ If you want to run the process as someone other than root:
124
+
125
+ ```ruby
126
+ Bluepill.application("app_name") do |app|
127
+ app.process("process_name") do |process|
128
+ process.start_command = "/usr/bin/some_start_command"
129
+ process.pid_file = "/tmp/some_pid_file.pid"
130
+ process.uid = "deploy"
131
+ process.gid = "deploy"
132
+
133
+ process.checks :cpu_usage, :every => 10.seconds, :below => 5, :times => 3
134
+ process.checks :mem_usage, :every => 10.seconds, :below => 100.megabytes, :times => [3,5]
135
+ end
136
+ end
137
+ ```
138
+
139
+ If you want to include one or more supplementary groups:
140
+
141
+ ```ruby
142
+ Bluepill.application("app_name") do |app|
143
+ app.process("process_name") do |process|
144
+ process.start_command = "/usr/bin/some_start_command"
145
+ process.pid_file = "/tmp/some_pid_file.pid"
146
+ process.uid = "deploy"
147
+ process.gid = "deploy"
148
+ process.supplementary_groups = ['rvm']
149
+
150
+ process.checks :cpu_usage, :every => 10.seconds, :below => 5, :times => 3
151
+ process.checks :mem_usage, :every => 10.seconds, :below => 100.megabytes, :times => [3,5]
152
+ end
153
+ end
154
+ ```
155
+
156
+ You can also set an app-wide uid/gid:
157
+
158
+ ```ruby
159
+ Bluepill.application("app_name") do |app|
160
+ app.uid = "deploy"
161
+ app.gid = "deploy"
162
+ app.process("process_name") do |process|
163
+ process.start_command = "/usr/bin/some_start_command"
164
+ process.pid_file = "/tmp/some_pid_file.pid"
165
+ end
166
+ end
167
+ ```
168
+
169
+ To track resources of child processes, use :include_children:
170
+ ```ruby
171
+ Bluepill.application("app_name") do |app|
172
+ app.process("process_name") do |process|
173
+ process.start_command = "/usr/bin/some_start_command"
174
+ process.pid_file = "/tmp/some_pid_file.pid"
175
+ process.checks :mem_usage, :every => 1.seconds, :below => 5.megabytes, :times => [3,5], :include_children => true
176
+ end
177
+ end
178
+ ```
179
+
180
+ To check for flapping:
181
+
182
+ ```ruby
183
+ process.checks :flapping, :times => 2, :within => 30.seconds, :retry_in => 7.seconds
184
+ ```
185
+
186
+ To set the working directory to _cd_ into when starting the command:
187
+
188
+ ```ruby
189
+ Bluepill.application("app_name") do |app|
190
+ app.process("process_name") do |process|
191
+ process.start_command = "/usr/bin/some_start_command"
192
+ process.pid_file = "/tmp/some_pid_file.pid"
193
+ process.working_dir = "/path/to/some_directory"
194
+ end
195
+ end
196
+ ```
197
+
198
+ You can also have an app-wide working directory:
199
+
200
+ ```ruby
201
+ Bluepill.application("app_name") do |app|
202
+ app.working_dir = "/path/to/some_directory"
203
+ app.process("process_name") do |process|
204
+ process.start_command = "/usr/bin/some_start_command"
205
+ process.pid_file = "/tmp/some_pid_file.pid"
206
+ end
207
+ end
208
+ ```
209
+
210
+ Note: We also set the PWD in the environment to the working dir you specify. This is useful for when the working dir is a symlink. Unicorn in particular will cd into the environment variable in PWD when it re-execs to deal with a change in the symlink.
211
+
212
+ By default, bluepill will send a SIGTERM to your process when stopping.
213
+ To change the stop command:
214
+
215
+ ```ruby
216
+ Bluepill.application("app_name") do |app|
217
+ app.process("process_name") do |process|
218
+ process.start_command = "/usr/bin/some_start_command"
219
+ process.pid_file = "/tmp/some_pid_file.pid"
220
+ process.stop_command = "/user/bin/some_stop_command"
221
+ end
222
+ end
223
+ ```
224
+
225
+ If you'd like to send a signal or signals to your process to stop it:
226
+
227
+ ```ruby
228
+ Bluepill.application("app_name") do |app|
229
+ app.process("process_name") do |process|
230
+ process.start_command = "/usr/bin/some_start_command"
231
+ process.pid_file = "/tmp/some_pid_file.pid"
232
+ process.stop_signals = [:quit, 30.seconds, :term, 5.seconds, :kill]
233
+ end
234
+ end
235
+ ```
236
+
237
+ We added a line that will send a SIGQUIT, wait 30 seconds and check to
238
+ see if the process is still up, send a SIGTERM, wait 5 seconds and check
239
+ to see if the process is still up, and finally send a SIGKILL.
240
+
241
+ And lastly, to monitor child processes:
242
+
243
+ ```ruby
244
+ process.monitor_children do |child_process|
245
+ child_process.checks :cpu_usage, :every => 10, :below => 5, :times => 3
246
+ child_process.checks :mem_usage, :every => 10, :below => 100.megabytes, :times => [3, 5]
247
+
248
+ child_process.stop_command = "kill -QUIT {{PID}}"
249
+ end
250
+ ```
251
+
252
+ Note {{PID}} will be substituted for the pid of process in both the stop and restart commands.
253
+
254
+ ### A Note About Output Redirection
255
+
256
+ While you can specify shell tricks like the following in the start_command of a process:
257
+
258
+ ```ruby
259
+ Bluepill.application("app_name") do |app|
260
+ app.process("process_name") do |process|
261
+ process.start_command = "cd /tmp/some_dir && SOME_VAR=1 /usr/bin/some_start_command > /tmp/server.log 2>&1"
262
+ process.pid_file = "/tmp/some_pid_file.pid"
263
+ end
264
+ end
265
+ ```
266
+
267
+ We recommend that you _not_ do that and instead use the config options to capture output from your daemons. Like so:
268
+
269
+ ```ruby
270
+ Bluepill.application("app_name") do |app|
271
+ app.process("process_name") do |process|
272
+ process.start_command = "/usr/bin/env SOME_VAR=1 /usr/bin/some_start_command"
273
+
274
+ process.working_dir = "/tmp/some_dir"
275
+ process.stdout = process.stderr = "/tmp/server.log"
276
+
277
+ process.pid_file = "/tmp/some_pid_file.pid"
278
+ end
279
+ end
280
+ ```
281
+
282
+ The main benefit of using the config options is that Bluepill will be able to monitor the correct process instead of just watching the shell that spawned your actual server.
283
+
284
+ ### CLI
285
+
286
+ #### Usage
287
+
288
+ bluepill [app_name] command [options]
289
+
290
+ For the "load" command, the _app_name_ is specified in the config file, and
291
+ must not be provided on the command line.
292
+
293
+ For all other commands, the _app_name_ is optional if there is only
294
+ one bluepill daemon running. Otherwise, the _app_name_ must be
295
+ provided, because the command will fail when there are multiple
296
+ bluepill daemons running. The example commands below leaves out the
297
+ _app_name_.
298
+
299
+ #### Commands
300
+
301
+ To start a bluepill daemon and load the config for an application:
302
+
303
+ sudo bluepill load /path/to/production.pill
304
+
305
+ To act on a process or group for an application:
306
+
307
+ sudo bluepill <start|stop|restart|unmonitor> <process_or_group_name>
308
+
309
+ To view process statuses for an application:
310
+
311
+ sudo bluepill status
312
+
313
+ To view the log for a process or group for an application:
314
+
315
+ sudo bluepill log <process_or_group_name>
316
+
317
+ To quit the bluepill daemon for an application:
318
+
319
+ sudo bluepill quit
320
+
321
+ ### Logging
322
+ By default, bluepill uses syslog local6 facility as described in the installation section. But if for any reason you don&apos;t want to use syslog, you can use a log file. You can do this by setting the :log\_file option in the config:
323
+
324
+ ```ruby
325
+ Bluepill.application("app_name", :log_file => "/path/to/bluepill.log") do |app|
326
+ # ...
327
+ end
328
+ ```
329
+
330
+ Keep in mind that you still need to set up log rotation (described in the installation section) to keep the log file from growing huge.
331
+
332
+ ### Extra options
333
+ You can run bluepill in the foreground:
334
+
335
+ ```ruby
336
+ Bluepill.application("app_name", :foreground => true) do |app|
337
+ # ...
338
+ end
339
+ ```
340
+
341
+ Note that You must define only one application per config when using foreground mode.
342
+
343
+ ## Links
344
+ Code: [http://github.com/arya/bluepill](http://github.com/arya/bluepill)
345
+ Bugs/Features: [http://github.com/arya/bluepill/issues](http://github.com/arya/bluepill/issues)
346
+ Mailing List: [http://groups.google.com/group/bluepill-rb](http://groups.google.com/group/bluepill-rb)
347
+
348
+
349
+ [rubygems]: http://rubygems.org/gems/bluepill
data/Rakefile ADDED
@@ -0,0 +1,38 @@
1
+ # encoding: utf-8
2
+
3
+ begin
4
+ require 'bundler'
5
+ Bundler::GemHelper.install_tasks
6
+ rescue LoadError
7
+ $stderr.puts "Bundler not installed. You should install it with: gem install bundler"
8
+ end
9
+
10
+ $LOAD_PATH << File.expand_path('./lib', File.dirname(__FILE__))
11
+ require 'bluepill/version'
12
+
13
+ begin
14
+ require 'rspec/core/rake_task'
15
+
16
+ RSpec::Core::RakeTask.new
17
+
18
+ if RUBY_VERSION >= '1.9'
19
+ RSpec::Core::RakeTask.new(:cov) do |t|
20
+ ENV['ENABLE_SIMPLECOV'] = '1'
21
+ t.ruby_opts = '-w'
22
+ t.rcov_opts = %q[-Ilib --exclude "spec/*,gems/*"]
23
+ end
24
+ end
25
+ rescue LoadError
26
+ $stderr.puts "RSpec not available. Install it with: gem install rspec-core rspec-expectations rr faker"
27
+ end
28
+
29
+ begin
30
+ require 'yard'
31
+ YARD::Rake::YardocTask.new do |yard|
32
+ yard.options << "--title='bluepill #{Bluepill::VERSION}'"
33
+
34
+ end
35
+ rescue LoadError
36
+ $stderr.puts "Please install YARD with: gem install yard"
37
+ end
38
+