skywatch 0.0.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/.gitignore +17 -0
- data/Gemfile +4 -0
- data/LICENSE +22 -0
- data/README.md +175 -0
- data/Rakefile +5 -0
- data/bin/skywatch +231 -0
- data/lib/skywatch.rb +3 -0
- data/lib/skywatch/builtin/alerts/email +14 -0
- data/lib/skywatch/builtin/checks/30.example +3 -0
- data/lib/skywatch/builtin/checks/3600.skywatch_watchers +18 -0
- data/lib/skywatch/tool.rb +211 -0
- data/lib/skywatch/version.rb +3 -0
- data/lib/skywatch/watcher/Gemfile +2 -0
- data/lib/skywatch/watcher/Procfile +1 -0
- data/lib/skywatch/watcher/watcher.rb +56 -0
- data/skywatch.gemspec +25 -0
- data/spec/skywatch_spec.rb +8 -0
- metadata +129 -0
data/.gitignore
ADDED
data/Gemfile
ADDED
data/LICENSE
ADDED
@@ -0,0 +1,22 @@
|
|
1
|
+
Copyright (c) 2012 Jeff Lindsay
|
2
|
+
|
3
|
+
MIT License
|
4
|
+
|
5
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
6
|
+
a copy of this software and associated documentation files (the
|
7
|
+
"Software"), to deal in the Software without restriction, including
|
8
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
9
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
10
|
+
permit persons to whom the Software is furnished to do so, subject to
|
11
|
+
the following conditions:
|
12
|
+
|
13
|
+
The above copyright notice and this permission notice shall be
|
14
|
+
included in all copies or substantial portions of the Software.
|
15
|
+
|
16
|
+
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
|
17
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
18
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
|
19
|
+
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
|
20
|
+
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
|
21
|
+
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
|
22
|
+
WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/README.md
ADDED
@@ -0,0 +1,175 @@
|
|
1
|
+
# Skywatch (alpha)
|
2
|
+
|
3
|
+
Simple alerting system that lets you define checks and alerts in any
|
4
|
+
language that are then magically run on Heroku.
|
5
|
+
|
6
|
+
NoOps! Polyglot! Free monitoring of anything!
|
7
|
+
|
8
|
+
## Installation
|
9
|
+
|
10
|
+
$ gem install skywatch
|
11
|
+
|
12
|
+
## Usage
|
13
|
+
|
14
|
+
It's a fairly powerful tool. Run help to see a list of subcommands.
|
15
|
+
|
16
|
+
$ skywatch -h
|
17
|
+
|
18
|
+
## What the hell is this amazing thing??
|
19
|
+
|
20
|
+
Skywatch is a command-line utility that manages checking and alerting
|
21
|
+
scripts used by a builtin watcher service. Skywatch deploys these
|
22
|
+
scripts and service for you on Heroku where they can run and monitor
|
23
|
+
anything from the cloud for free.
|
24
|
+
|
25
|
+
The watcher service runs check scripts that can assert
|
26
|
+
anything at any frequency. If a check script returns a non-zero exit status,
|
27
|
+
it will fire any enabled alert scripts, passing it the output of the
|
28
|
+
check script and other useful data. Alert scripts can then act on this
|
29
|
+
assertion failure, such as send email, SMS, or webhook.
|
30
|
+
|
31
|
+
The check script will continue to run and potentially fail, but the
|
32
|
+
alert script only runs once, if without error. Only until a reset signal
|
33
|
+
is sent will it be ready to fire the alert again for any failed check
|
34
|
+
script. In this way, alerts work like [clip
|
35
|
+
indicators](http://help.adobe.com/en_US/audition/cs/using/WS58a04a822e3e5010548241038980c2c5-7f93.html)
|
36
|
+
in the audio recording world. They turn on once any clipping happens and
|
37
|
+
remain on until you manually reset them.
|
38
|
+
|
39
|
+
You manage your scripts locally with the skywatch command, or by hand
|
40
|
+
since they're just files in directories. When you want to deploy script
|
41
|
+
changes, toggle enabled scripts, or reset the alert state, you can run a
|
42
|
+
skywatch command and it will handle pushing changes to Heroku for you.
|
43
|
+
|
44
|
+
## Getting Started
|
45
|
+
|
46
|
+
The skywatch command manages a directory containing check scripts and
|
47
|
+
alert scripts. You can make a new directory and let skywatch set this up
|
48
|
+
for you:
|
49
|
+
|
50
|
+
$ mkdir my-watcher
|
51
|
+
$ cd my-watcher
|
52
|
+
$ skywatch init
|
53
|
+
|
54
|
+
At this point, it will have you authenticate with your Heroku
|
55
|
+
credentials if you haven't already. Grab a free account if you don't
|
56
|
+
have one.
|
57
|
+
|
58
|
+
When you run `skywatch init` authenticated it will create some example
|
59
|
+
alerts and checks for you, and deploy a Heroku app. None of the checks
|
60
|
+
or alerts are enabled by default. See everything by just running
|
61
|
+
`skywatch` from the directory:
|
62
|
+
|
63
|
+
$ skywatch
|
64
|
+
Checks for fathomless-crag-3169
|
65
|
+
example every 30s disabled
|
66
|
+
skywatch_watchers every 3600s disabled
|
67
|
+
Alerts for fathomless-crag-3169
|
68
|
+
email disabled
|
69
|
+
|
70
|
+
Take a look at all the files in the directory. Checks and alerts are nothing
|
71
|
+
more than scripts. Checks have a naming convention of `<interval>.<name>`,
|
72
|
+
and enabling and disabling is just setting the execute bit on the
|
73
|
+
scripts. There's nothing the `skywatch` command does that you can't
|
74
|
+
easily do by hand. It just happens to be terribly convenient.
|
75
|
+
|
76
|
+
$ skywatch edit alert email
|
77
|
+
|
78
|
+
This will open your editor and you can see the example email alert
|
79
|
+
script is using SendGrid. In fact, when you ran `skywatch init`, you
|
80
|
+
were set up with a free SendGrid starter addon for 600 emails a day. So
|
81
|
+
let's try it by putting your email address in the `TO` variable of the
|
82
|
+
script. Now enable the alert:
|
83
|
+
|
84
|
+
$ skywatch enable alert email
|
85
|
+
|
86
|
+
Let's create a new check script in bash that fails so we can get the
|
87
|
+
alert.
|
88
|
+
|
89
|
+
$ skywatch create check failure_test 30
|
90
|
+
|
91
|
+
The last argument is the interval. Intervals are always in seconds. All
|
92
|
+
this did was create a new file under the `checks` directory with a
|
93
|
+
little bit of boiler plate. Let's replace its contents with this:
|
94
|
+
|
95
|
+
#!/usr/bin/env bash
|
96
|
+
echo "Oh no, a failed check."
|
97
|
+
exit 255
|
98
|
+
|
99
|
+
Enable the check and then we'll deploy:
|
100
|
+
|
101
|
+
$ skywatch enable check failure_test
|
102
|
+
$ skywatch deploy
|
103
|
+
|
104
|
+
It's going to move some files around and then deploy to Heroku. It keeps
|
105
|
+
a staging directory called `.skywatch`, which is a Git repo used to push
|
106
|
+
to Heroku. It automatically adds this to a `.gitignore` file, so you can
|
107
|
+
version your scripts with Git and not worry about this implementation
|
108
|
+
detail.
|
109
|
+
|
110
|
+
Once it's finished, you might want to run monitor to see how it went and
|
111
|
+
what's going on. This is just tailing the Heroku logs of the watcher
|
112
|
+
service:
|
113
|
+
|
114
|
+
$ skywatch monitor
|
115
|
+
|
116
|
+
You can run this whenever to see what it's doing. You'll probably see
|
117
|
+
that it triggered the alert. Go check your email! That will be the only
|
118
|
+
email you get, regardless of whether the check starts to work again and
|
119
|
+
then fail again. No flapping. You have to manually reset:
|
120
|
+
|
121
|
+
$ skywatch reset
|
122
|
+
|
123
|
+
This should cause another alert email within 30 seconds. And of course,
|
124
|
+
you can tear everything down with destroy:
|
125
|
+
|
126
|
+
$ skywatch destroy
|
127
|
+
|
128
|
+
This destroys the Heroku app and the `.skywatch` directory. It doesn't
|
129
|
+
touch your scripts at all. In fact, you can run `skywatch init` again if
|
130
|
+
you'd like.
|
131
|
+
|
132
|
+
The source code to all this is terribly simple. The watcher service is
|
133
|
+
only about 50 lines of Ruby. Everything else is just file operations.
|
134
|
+
In fact, the little state it maintains is kept in file metadata. For how
|
135
|
+
automated it is, skywatch has to be one of the simplest monitoring services
|
136
|
+
ever.
|
137
|
+
|
138
|
+
## Writing Check Scripts
|
139
|
+
|
140
|
+
Check scripts are any executable script using the shebang to define the
|
141
|
+
interpreter. Heroku has most common languages built-in to its Cedar
|
142
|
+
stack, so feel free to use Python, Perl, Ruby, whatever. I like bash.
|
143
|
+
|
144
|
+
The only conventions of check scripts are the interval-in-the-filename and that a non-zero exit status will fire the alerts. Any output of the check script will be piped into STDIN of the alert script, so try be verbose but not too
|
145
|
+
verbose.
|
146
|
+
|
147
|
+
If you're using bash, it's a good idea to use `set -e` so any failed
|
148
|
+
subcommand will bubble up.
|
149
|
+
|
150
|
+
## Writing Alert Scripts
|
151
|
+
|
152
|
+
Like check scripts, alert scripts can be written in any language. Also,
|
153
|
+
like check scripts, the exit status is important. If an alert script
|
154
|
+
exit status is non-zero, it will run again with the next failure of the check
|
155
|
+
script.
|
156
|
+
|
157
|
+
The alert script is given the output of the check script via STDIN. It's
|
158
|
+
also given 2 arguments. The first is the name of the check script. The
|
159
|
+
second is the exit status of the failed check script.
|
160
|
+
|
161
|
+
The output of an alert script is ignored. It might be a good idea to log
|
162
|
+
the output of failed alert scripts. You'd then be able to see it via
|
163
|
+
`skywatch monitor`. Sounds like a contribution idea.
|
164
|
+
|
165
|
+
## Contributing
|
166
|
+
|
167
|
+
1. Fork it
|
168
|
+
2. Create your feature branch (`git checkout -b my-new-feature`)
|
169
|
+
3. Commit your changes (`git commit -am 'Add some feature'`)
|
170
|
+
4. Push to the branch (`git push origin my-new-feature`)
|
171
|
+
5. Create new Pull Request
|
172
|
+
|
173
|
+
## License
|
174
|
+
|
175
|
+
MIT
|
data/Rakefile
ADDED
data/bin/skywatch
ADDED
@@ -0,0 +1,231 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
require 'rubygems'
|
3
|
+
require 'commander/import'
|
4
|
+
require 'highline/import'
|
5
|
+
|
6
|
+
require 'skywatch'
|
7
|
+
|
8
|
+
include Skywatch
|
9
|
+
|
10
|
+
program :name, 'skywatch'
|
11
|
+
program :version, Skywatch::VERSION
|
12
|
+
program :description, 'Simple, Unix-oriented alerting system'
|
13
|
+
default_command :all
|
14
|
+
|
15
|
+
def display_check(check)
|
16
|
+
status = check.enabled? ? "'enabled', :green" : "'disabled', :red"
|
17
|
+
say " %-24s %-16s %s" % [
|
18
|
+
check.name, "every #{check.interval}s", "<%= color #{status} %>"]
|
19
|
+
end
|
20
|
+
|
21
|
+
def display_alert(alert)
|
22
|
+
status = alert.enabled? ? "'enabled', :green" : "'disabled', :red"
|
23
|
+
say " %-16s %s" % [
|
24
|
+
alert.name, "<%= color #{status} %>"]
|
25
|
+
end
|
26
|
+
|
27
|
+
def ensure_logged_in
|
28
|
+
if not logged_in?
|
29
|
+
say_warning "You're not logged into Heroku. Please login then try again"
|
30
|
+
login
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
command :init do |c|
|
35
|
+
c.syntax = 'skywatch init [name]'
|
36
|
+
c.description = 'Initializes skywatch in the current directory'
|
37
|
+
c.action do |args, options|
|
38
|
+
ensure_logged_in
|
39
|
+
init args.first
|
40
|
+
say_ok "Skywatch initialized as #{name}"
|
41
|
+
end
|
42
|
+
end
|
43
|
+
|
44
|
+
command :checks do |c|
|
45
|
+
c.syntax = 'skywatch checks'
|
46
|
+
c.description = 'Lists checks'
|
47
|
+
c.action do |args, options|
|
48
|
+
puts " Checks for #{name}"
|
49
|
+
checks.each do |check|
|
50
|
+
display_check check
|
51
|
+
end
|
52
|
+
end
|
53
|
+
end
|
54
|
+
|
55
|
+
command :alerts do |c|
|
56
|
+
c.syntax = 'skywatch alerts'
|
57
|
+
c.description = 'Lists alerts'
|
58
|
+
c.action do |args, options|
|
59
|
+
puts " Alerts for #{name}"
|
60
|
+
alerts.each do |alert|
|
61
|
+
display_alert alert
|
62
|
+
end
|
63
|
+
end
|
64
|
+
end
|
65
|
+
|
66
|
+
command :all do |c|
|
67
|
+
c.syntax = 'skywatch all'
|
68
|
+
c.description = 'Lists checks and alerts'
|
69
|
+
c.action do |args, options|
|
70
|
+
puts " Checks for #{name}"
|
71
|
+
checks.each do |check|
|
72
|
+
display_check check
|
73
|
+
end
|
74
|
+
puts " Alerts for #{name}"
|
75
|
+
alerts.each do |alert|
|
76
|
+
display_alert alert
|
77
|
+
end
|
78
|
+
end
|
79
|
+
end
|
80
|
+
|
81
|
+
command :'enable check' do |c|
|
82
|
+
c.syntax = 'skywatch enable check <name>'
|
83
|
+
c.description = 'Enable a check'
|
84
|
+
c.action do |args, options|
|
85
|
+
fail unless args.length > 0
|
86
|
+
check = check(args.first)
|
87
|
+
check.enable
|
88
|
+
display_check check
|
89
|
+
end
|
90
|
+
end
|
91
|
+
|
92
|
+
command :'disable check' do |c|
|
93
|
+
c.syntax = 'skywatch disable check <name>'
|
94
|
+
c.description = 'Disable a check'
|
95
|
+
c.action do |args, options|
|
96
|
+
fail unless args.length > 0
|
97
|
+
check = check(args.first)
|
98
|
+
check.disable
|
99
|
+
display_check check
|
100
|
+
end
|
101
|
+
end
|
102
|
+
|
103
|
+
command :'enable alert' do |c|
|
104
|
+
c.syntax = 'skywatch enable alert <name>'
|
105
|
+
c.description = 'Enable an alert'
|
106
|
+
c.action do |args, options|
|
107
|
+
fail unless args.length > 0
|
108
|
+
alert = alert(args.first)
|
109
|
+
alert.enable
|
110
|
+
display_alert alert
|
111
|
+
end
|
112
|
+
end
|
113
|
+
|
114
|
+
command :'disable alert' do |c|
|
115
|
+
c.syntax = 'skywatch disable alert <name>'
|
116
|
+
c.description = 'Disable an alert'
|
117
|
+
c.action do |args, options|
|
118
|
+
fail unless args.length > 0
|
119
|
+
alert = alert(args.first)
|
120
|
+
alert.disable
|
121
|
+
display_alert alert
|
122
|
+
end
|
123
|
+
end
|
124
|
+
|
125
|
+
command :'create alert' do |c|
|
126
|
+
c.syntax = 'skywatch create alert <name> [interpreter]'
|
127
|
+
c.description = 'Create an alert script. Default interpreter is bash'
|
128
|
+
c.action do |args, options|
|
129
|
+
fail unless args.length > 0
|
130
|
+
interpreter = args.length > 1 ? args[1] : 'bash'
|
131
|
+
alert = create_alert(args.first, interpreter)
|
132
|
+
display_alert alert
|
133
|
+
end
|
134
|
+
end
|
135
|
+
|
136
|
+
command :'create check' do |c|
|
137
|
+
c.syntax = 'skywatch create check <name> <interval> [interpreter]'
|
138
|
+
c.description = 'Create a check script. Default interpreter is bash'
|
139
|
+
c.action do |args, options|
|
140
|
+
fail unless args.length > 1
|
141
|
+
interpreter = args.length > 2 ? args[2] : 'bash'
|
142
|
+
check = create_check(args[0], args[1], interpreter)
|
143
|
+
display_check check
|
144
|
+
end
|
145
|
+
end
|
146
|
+
|
147
|
+
command :'edit alert' do |c|
|
148
|
+
c.syntax = 'skywatch edit alert <name>'
|
149
|
+
c.description = 'Open the alert script in an editor'
|
150
|
+
c.action do |args|
|
151
|
+
fail unless args.length > 0
|
152
|
+
if ENV['EDITOR'].to_s.empty?
|
153
|
+
puts "The EDITOR environment variable is not defined."
|
154
|
+
else
|
155
|
+
alert = alert(args.first)
|
156
|
+
exec "#{ENV['EDITOR']} #{alert.path}"
|
157
|
+
end
|
158
|
+
end
|
159
|
+
end
|
160
|
+
|
161
|
+
command :'edit check' do |c|
|
162
|
+
c.syntax = 'skywatch edit check <name>'
|
163
|
+
c.description = 'Open the check script in an editor'
|
164
|
+
c.action do |args|
|
165
|
+
fail unless args.length > 0
|
166
|
+
if ENV['EDITOR'].to_s.empty?
|
167
|
+
puts "The EDITOR environment variable is not defined."
|
168
|
+
else
|
169
|
+
check = check(args.first)
|
170
|
+
exec "#{ENV['EDITOR']} #{check.path}"
|
171
|
+
end
|
172
|
+
end
|
173
|
+
end
|
174
|
+
|
175
|
+
command :reset do |c|
|
176
|
+
c.syntax = 'skywatch reset'
|
177
|
+
c.description = 'Reset running check states'
|
178
|
+
c.action do
|
179
|
+
ensure_logged_in
|
180
|
+
reset
|
181
|
+
end
|
182
|
+
end
|
183
|
+
|
184
|
+
command :monitor do |c|
|
185
|
+
c.syntax = 'skywatch monitor'
|
186
|
+
c.description = 'Monitor the running checks'
|
187
|
+
c.action do
|
188
|
+
ensure_logged_in
|
189
|
+
monitor
|
190
|
+
end
|
191
|
+
end
|
192
|
+
|
193
|
+
command :deploy do |c|
|
194
|
+
c.syntax = 'skywatch deploy'
|
195
|
+
c.description = 'Deploy this skywatch'
|
196
|
+
c.action do
|
197
|
+
ensure_logged_in
|
198
|
+
deploy
|
199
|
+
end
|
200
|
+
end
|
201
|
+
|
202
|
+
command :destroy do |c|
|
203
|
+
c.syntax = 'skywatch destroy'
|
204
|
+
c.description = 'Destroy skywatch Heroku app and stage directory'
|
205
|
+
c.action do
|
206
|
+
ensure_logged_in
|
207
|
+
destroy
|
208
|
+
end
|
209
|
+
end
|
210
|
+
|
211
|
+
#command :'destroy alert' do |c|
|
212
|
+
# c.syntax = 'skywatch destroy alert <name>'
|
213
|
+
# c.description = 'Delete an alert script'
|
214
|
+
# c.action do |args, options|
|
215
|
+
# fail unless args.length > 0
|
216
|
+
# alert = alert(args.first)
|
217
|
+
# alert.destroy
|
218
|
+
# say " Alert <%= color '#{alert.name}', :bold %> has been destroyed"
|
219
|
+
# end
|
220
|
+
#end
|
221
|
+
|
222
|
+
#command :'destroy check' do |c|
|
223
|
+
# c.syntax = 'skywatch destroy check <name>'
|
224
|
+
# c.description = 'Delete a check script'
|
225
|
+
# c.action do |args, options|
|
226
|
+
# fail unless args.length > 0
|
227
|
+
# check = check(args.first)
|
228
|
+
# check.destroy
|
229
|
+
# say " Check <%= color '#{check.name}', :bold %> has been destroyed"
|
230
|
+
# end
|
231
|
+
#end
|
data/lib/skywatch.rb
ADDED
@@ -0,0 +1,14 @@
|
|
1
|
+
#!/usr/bin/env bash
|
2
|
+
TO=
|
3
|
+
SUBJECT="[skywatch] $1"
|
4
|
+
BODY=`echo -e "Failure with status $2:\n\n$(cat)"`
|
5
|
+
set -e
|
6
|
+
curl \
|
7
|
+
-X 'POST' \
|
8
|
+
-F "api_user=$SENDGRID_USERNAME" \
|
9
|
+
-F "api_key=$SENDGRID_PASSWORD" \
|
10
|
+
-F "to=$TO" \
|
11
|
+
-F "subject=$SUBJECT" \
|
12
|
+
-F "text=$BODY" \
|
13
|
+
-F "from=$TO" \
|
14
|
+
--silent --fail "https://sendgrid.com/api/mail.send.json"
|
@@ -0,0 +1,18 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
require "net/https"
|
3
|
+
require "uri"
|
4
|
+
require "json"
|
5
|
+
|
6
|
+
uri = URI.parse("https://api.github.com/repos/progrium/skywatch")
|
7
|
+
http = Net::HTTP.new(uri.host, uri.port)
|
8
|
+
http.use_ssl = true
|
9
|
+
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
|
10
|
+
|
11
|
+
request = Net::HTTP::Get.new(uri.request_uri)
|
12
|
+
|
13
|
+
response = http.request(request)
|
14
|
+
repo = JSON.parse(response.body)
|
15
|
+
|
16
|
+
if repo['watchers_count'] > 1000
|
17
|
+
raise "#{repo['watchers_count']} watchers!"
|
18
|
+
end
|
@@ -0,0 +1,211 @@
|
|
1
|
+
require 'fileutils'
|
2
|
+
|
3
|
+
module Skywatch
|
4
|
+
include FileUtils
|
5
|
+
|
6
|
+
def init(name="")
|
7
|
+
fail if skywatch?
|
8
|
+
mkdir '.skywatch'
|
9
|
+
begin
|
10
|
+
cd '.skywatch' do
|
11
|
+
system "git init"
|
12
|
+
puts `heroku create #{name}`
|
13
|
+
fail if not $?.exitstatus.zero?
|
14
|
+
puts `heroku addons:add sendgrid:starter`
|
15
|
+
system "cp #{watcher_path}/* ."
|
16
|
+
system "bundle install > /dev/null 2>&1"
|
17
|
+
end
|
18
|
+
mkdir_p 'alerts'
|
19
|
+
if Dir['alerts/*'].empty?
|
20
|
+
system "cp #{builtin_path}/alerts/* alerts"
|
21
|
+
end
|
22
|
+
mkdir_p 'checks'
|
23
|
+
if Dir['checks/*'].empty?
|
24
|
+
system "cp #{builtin_path}/checks/* checks"
|
25
|
+
end
|
26
|
+
system "echo '.skywatch' >> .gitignore"
|
27
|
+
deploy
|
28
|
+
cd '.skywatch' do
|
29
|
+
system "heroku ps:scale watcher=1"
|
30
|
+
end
|
31
|
+
rescue
|
32
|
+
rm_rf '.skywatch'
|
33
|
+
end
|
34
|
+
end
|
35
|
+
|
36
|
+
def logged_in?
|
37
|
+
`heroku auth:token < /dev/null`
|
38
|
+
$?.exitstatus.zero?
|
39
|
+
end
|
40
|
+
|
41
|
+
def login
|
42
|
+
exec 'heroku auth:login'
|
43
|
+
end
|
44
|
+
|
45
|
+
def watcher_path
|
46
|
+
File.dirname(__FILE__)+"/watcher"
|
47
|
+
end
|
48
|
+
|
49
|
+
def builtin_path
|
50
|
+
File.dirname(__FILE__)+"/builtin"
|
51
|
+
end
|
52
|
+
|
53
|
+
def skywatch?
|
54
|
+
not Dir["./.skywatch"].empty?
|
55
|
+
end
|
56
|
+
|
57
|
+
def checks
|
58
|
+
fail unless skywatch?
|
59
|
+
Dir["./checks/*"].collect{|c| Check[c] }
|
60
|
+
end
|
61
|
+
|
62
|
+
def alerts
|
63
|
+
fail unless skywatch?
|
64
|
+
Dir["./alerts/*"].collect{|c| Alert[c] }
|
65
|
+
end
|
66
|
+
|
67
|
+
def check(name)
|
68
|
+
fail unless skywatch?
|
69
|
+
Check[Dir["./checks/*.#{name}"].first]
|
70
|
+
end
|
71
|
+
|
72
|
+
def alert(name)
|
73
|
+
fail unless skywatch?
|
74
|
+
Alert[Dir["./alerts/#{name}"].first]
|
75
|
+
end
|
76
|
+
|
77
|
+
def create_alert(name, interpreter)
|
78
|
+
fail unless skywatch?
|
79
|
+
fail if File.exist? "./alerts/#{name}"
|
80
|
+
File.open("./alerts/#{name}", 'w') do |f|
|
81
|
+
f.write "#!/usr/bin/env #{interpreter}\n"
|
82
|
+
if interpreter == 'bash'
|
83
|
+
f.write "set -e\n"
|
84
|
+
end
|
85
|
+
f.write "# Write your alert here"
|
86
|
+
end
|
87
|
+
alert(name)
|
88
|
+
end
|
89
|
+
|
90
|
+
def create_check(name, interval, interpreter)
|
91
|
+
fail unless skywatch?
|
92
|
+
fail if File.exist? "./checks/#{interval}.#{name}"
|
93
|
+
File.open("./checks/#{interval}.#{name}", 'w') do |f|
|
94
|
+
f.write "#!/usr/bin/env #{interpreter}\n"
|
95
|
+
if interpreter == 'bash'
|
96
|
+
f.write "set -e\n"
|
97
|
+
end
|
98
|
+
f.write "# Write your check here"
|
99
|
+
end
|
100
|
+
check(name)
|
101
|
+
end
|
102
|
+
|
103
|
+
def name
|
104
|
+
fail unless skywatch?
|
105
|
+
cd ".skywatch" do
|
106
|
+
if not File.exist? 'name'
|
107
|
+
name = `eval $(heroku apps:info -s | grep "^name=") && echo "$name"`.strip
|
108
|
+
File.open('name', 'w') {|f| f.write(name) }
|
109
|
+
end
|
110
|
+
@@name ||= File.read('name')
|
111
|
+
end
|
112
|
+
@@name
|
113
|
+
end
|
114
|
+
|
115
|
+
def reset
|
116
|
+
fail unless skywatch?
|
117
|
+
cd ".skywatch" do
|
118
|
+
puts `heroku restart`
|
119
|
+
end
|
120
|
+
end
|
121
|
+
|
122
|
+
def monitor
|
123
|
+
fail unless skywatch?
|
124
|
+
cd ".skywatch" do
|
125
|
+
exec "heroku logs -t -n 10"
|
126
|
+
end
|
127
|
+
end
|
128
|
+
|
129
|
+
def stage
|
130
|
+
fail unless skywatch?
|
131
|
+
cd '.skywatch' do
|
132
|
+
system 'git rm -f alerts/* > /dev/null 2>&1'
|
133
|
+
system 'git rm -f checks/* > /dev/null 2>&1'
|
134
|
+
end
|
135
|
+
mkdir_p '.skywatch/alerts'
|
136
|
+
mkdir_p '.skywatch/checks'
|
137
|
+
system 'cp alerts/* .skywatch/alerts'
|
138
|
+
system 'cp checks/* .skywatch/checks'
|
139
|
+
commit
|
140
|
+
end
|
141
|
+
|
142
|
+
def commit
|
143
|
+
fail unless skywatch?
|
144
|
+
cd '.skywatch' do
|
145
|
+
system 'git add .'
|
146
|
+
if not `git status`.include? "nothing to commit"
|
147
|
+
system 'git commit -m "skywatch commit"'
|
148
|
+
end
|
149
|
+
end
|
150
|
+
end
|
151
|
+
|
152
|
+
def deploy
|
153
|
+
stage
|
154
|
+
cd '.skywatch' do
|
155
|
+
puts `git push heroku master`
|
156
|
+
end
|
157
|
+
end
|
158
|
+
|
159
|
+
def destroy
|
160
|
+
fail unless skywatch?
|
161
|
+
puts `heroku apps:destroy -a #{name} --confirm #{name}`
|
162
|
+
rm_rf '.skywatch'
|
163
|
+
end
|
164
|
+
|
165
|
+
class Script
|
166
|
+
attr :path
|
167
|
+
attr :name
|
168
|
+
|
169
|
+
def self.[](path)
|
170
|
+
self.new(path)
|
171
|
+
end
|
172
|
+
|
173
|
+
def initialize(path)
|
174
|
+
@path = path
|
175
|
+
@name = File.basename(path)
|
176
|
+
end
|
177
|
+
|
178
|
+
def enabled?
|
179
|
+
File.executable? @path
|
180
|
+
end
|
181
|
+
|
182
|
+
def enable
|
183
|
+
chmod File.stat(@path).mode | 0700, @path
|
184
|
+
end
|
185
|
+
|
186
|
+
def disable
|
187
|
+
enable # since this is relative, force 0700
|
188
|
+
chmod File.stat(@path).mode & ~0100, @path
|
189
|
+
end
|
190
|
+
|
191
|
+
def destroy
|
192
|
+
rm @path
|
193
|
+
end
|
194
|
+
end
|
195
|
+
|
196
|
+
class Check < Script
|
197
|
+
attr :interval
|
198
|
+
|
199
|
+
def initialize(path)
|
200
|
+
interval, name = File.basename(path).split('.', 2)
|
201
|
+
@path = path
|
202
|
+
@name = name
|
203
|
+
@interval = interval.to_i
|
204
|
+
end
|
205
|
+
end
|
206
|
+
|
207
|
+
class Alert < Script
|
208
|
+
end
|
209
|
+
|
210
|
+
end
|
211
|
+
|
@@ -0,0 +1 @@
|
|
1
|
+
watcher: bundle exec clockwork watcher.rb
|
@@ -0,0 +1,56 @@
|
|
1
|
+
require 'clockwork'
|
2
|
+
require 'fileutils'
|
3
|
+
|
4
|
+
include Clockwork
|
5
|
+
include FileUtils
|
6
|
+
|
7
|
+
Clockwork.configure do |config|
|
8
|
+
config[:logger] = Logger.new(STDOUT)
|
9
|
+
config[:logger].level = Logger::ERROR
|
10
|
+
end
|
11
|
+
|
12
|
+
def mark_pass(check); chmod File.stat(check).mode & ~0003, check; end
|
13
|
+
def mark_fail(check); chmod File.stat(check).mode | 0007, check; end
|
14
|
+
def mark_alerted(check); chmod File.stat(check).mode | 0070, check; end
|
15
|
+
def marked_alerted?(check)
|
16
|
+
`stat -c %A #{check} | sed 's/......\\(.\\).\\+/\\1/'` == "x\n"
|
17
|
+
end
|
18
|
+
def executables(glob)
|
19
|
+
Dir[glob].select {|path| File.executable? path }
|
20
|
+
end
|
21
|
+
|
22
|
+
mkdir_p 'output'
|
23
|
+
mkdir_p 'checks'
|
24
|
+
mkdir_p 'alerts'
|
25
|
+
|
26
|
+
executables('checks/*').each do |check|
|
27
|
+
interval, name = File.basename(check).split('.', 2)
|
28
|
+
|
29
|
+
puts "loading check #{name} for every #{interval} seconds"
|
30
|
+
every interval.to_i.seconds, name do
|
31
|
+
|
32
|
+
puts "checking #{name} (#{File.stat(check).mode.to_s(8)})"
|
33
|
+
`#{check} > output/#{name} 2>&1`
|
34
|
+
|
35
|
+
status = $?.exitstatus
|
36
|
+
if status.zero?
|
37
|
+
mark_pass check
|
38
|
+
else
|
39
|
+
puts " #{name} check failed with status #{status}"
|
40
|
+
mark_fail check
|
41
|
+
|
42
|
+
if not marked_alerted? check
|
43
|
+
executables('alerts/*').each do |alert|
|
44
|
+
puts " sending #{File.basename(alert)} alert for #{name}"
|
45
|
+
`cat output/#{name} | #{alert} #{name} #{status} > /dev/null 2>&1`
|
46
|
+
|
47
|
+
if $?.exitstatus.zero?
|
48
|
+
mark_alerted check
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
52
|
+
|
53
|
+
end
|
54
|
+
end # every
|
55
|
+
|
56
|
+
end
|
data/skywatch.gemspec
ADDED
@@ -0,0 +1,25 @@
|
|
1
|
+
# -*- encoding: utf-8 -*-
|
2
|
+
lib = File.expand_path('../lib', __FILE__)
|
3
|
+
$LOAD_PATH.unshift(lib) unless $LOAD_PATH.include?(lib)
|
4
|
+
require 'skywatch/version'
|
5
|
+
|
6
|
+
Gem::Specification.new do |gem|
|
7
|
+
gem.name = "skywatch"
|
8
|
+
gem.version = Skywatch::VERSION
|
9
|
+
gem.authors = ["Jeff Lindsay"]
|
10
|
+
gem.email = ["progrium@gmail.com"]
|
11
|
+
gem.description = %q{Simple alerting system that lets you define checks and alerts in any language that are then magically run on Heroku.}
|
12
|
+
gem.summary = %q{Simple, polyglot alerting system}
|
13
|
+
gem.homepage = "http://github.com/progrium/skywatch"
|
14
|
+
|
15
|
+
gem.files = `git ls-files`.split($/)
|
16
|
+
gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
|
17
|
+
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
18
|
+
gem.require_paths = ["lib"]
|
19
|
+
|
20
|
+
gem.add_development_dependency "rspec", "~> 2.6"
|
21
|
+
|
22
|
+
gem.add_dependency "commander", "~> 4.1"
|
23
|
+
gem.add_dependency "heroku", "~> 2.33"
|
24
|
+
gem.add_dependency "bundler", "~> 1.2"
|
25
|
+
end
|
metadata
ADDED
@@ -0,0 +1,129 @@
|
|
1
|
+
--- !ruby/object:Gem::Specification
|
2
|
+
name: skywatch
|
3
|
+
version: !ruby/object:Gem::Version
|
4
|
+
version: 0.0.1
|
5
|
+
prerelease:
|
6
|
+
platform: ruby
|
7
|
+
authors:
|
8
|
+
- Jeff Lindsay
|
9
|
+
autorequire:
|
10
|
+
bindir: bin
|
11
|
+
cert_chain: []
|
12
|
+
date: 2012-12-20 00:00:00.000000000 Z
|
13
|
+
dependencies:
|
14
|
+
- !ruby/object:Gem::Dependency
|
15
|
+
name: rspec
|
16
|
+
requirement: !ruby/object:Gem::Requirement
|
17
|
+
none: false
|
18
|
+
requirements:
|
19
|
+
- - ~>
|
20
|
+
- !ruby/object:Gem::Version
|
21
|
+
version: '2.6'
|
22
|
+
type: :development
|
23
|
+
prerelease: false
|
24
|
+
version_requirements: !ruby/object:Gem::Requirement
|
25
|
+
none: false
|
26
|
+
requirements:
|
27
|
+
- - ~>
|
28
|
+
- !ruby/object:Gem::Version
|
29
|
+
version: '2.6'
|
30
|
+
- !ruby/object:Gem::Dependency
|
31
|
+
name: commander
|
32
|
+
requirement: !ruby/object:Gem::Requirement
|
33
|
+
none: false
|
34
|
+
requirements:
|
35
|
+
- - ~>
|
36
|
+
- !ruby/object:Gem::Version
|
37
|
+
version: '4.1'
|
38
|
+
type: :runtime
|
39
|
+
prerelease: false
|
40
|
+
version_requirements: !ruby/object:Gem::Requirement
|
41
|
+
none: false
|
42
|
+
requirements:
|
43
|
+
- - ~>
|
44
|
+
- !ruby/object:Gem::Version
|
45
|
+
version: '4.1'
|
46
|
+
- !ruby/object:Gem::Dependency
|
47
|
+
name: heroku
|
48
|
+
requirement: !ruby/object:Gem::Requirement
|
49
|
+
none: false
|
50
|
+
requirements:
|
51
|
+
- - ~>
|
52
|
+
- !ruby/object:Gem::Version
|
53
|
+
version: '2.33'
|
54
|
+
type: :runtime
|
55
|
+
prerelease: false
|
56
|
+
version_requirements: !ruby/object:Gem::Requirement
|
57
|
+
none: false
|
58
|
+
requirements:
|
59
|
+
- - ~>
|
60
|
+
- !ruby/object:Gem::Version
|
61
|
+
version: '2.33'
|
62
|
+
- !ruby/object:Gem::Dependency
|
63
|
+
name: bundler
|
64
|
+
requirement: !ruby/object:Gem::Requirement
|
65
|
+
none: false
|
66
|
+
requirements:
|
67
|
+
- - ~>
|
68
|
+
- !ruby/object:Gem::Version
|
69
|
+
version: '1.2'
|
70
|
+
type: :runtime
|
71
|
+
prerelease: false
|
72
|
+
version_requirements: !ruby/object:Gem::Requirement
|
73
|
+
none: false
|
74
|
+
requirements:
|
75
|
+
- - ~>
|
76
|
+
- !ruby/object:Gem::Version
|
77
|
+
version: '1.2'
|
78
|
+
description: Simple alerting system that lets you define checks and alerts in any
|
79
|
+
language that are then magically run on Heroku.
|
80
|
+
email:
|
81
|
+
- progrium@gmail.com
|
82
|
+
executables:
|
83
|
+
- skywatch
|
84
|
+
extensions: []
|
85
|
+
extra_rdoc_files: []
|
86
|
+
files:
|
87
|
+
- .gitignore
|
88
|
+
- Gemfile
|
89
|
+
- LICENSE
|
90
|
+
- README.md
|
91
|
+
- Rakefile
|
92
|
+
- bin/skywatch
|
93
|
+
- lib/skywatch.rb
|
94
|
+
- lib/skywatch/builtin/alerts/email
|
95
|
+
- lib/skywatch/builtin/checks/30.example
|
96
|
+
- lib/skywatch/builtin/checks/3600.skywatch_watchers
|
97
|
+
- lib/skywatch/tool.rb
|
98
|
+
- lib/skywatch/version.rb
|
99
|
+
- lib/skywatch/watcher/Gemfile
|
100
|
+
- lib/skywatch/watcher/Procfile
|
101
|
+
- lib/skywatch/watcher/watcher.rb
|
102
|
+
- skywatch.gemspec
|
103
|
+
- spec/skywatch_spec.rb
|
104
|
+
homepage: http://github.com/progrium/skywatch
|
105
|
+
licenses: []
|
106
|
+
post_install_message:
|
107
|
+
rdoc_options: []
|
108
|
+
require_paths:
|
109
|
+
- lib
|
110
|
+
required_ruby_version: !ruby/object:Gem::Requirement
|
111
|
+
none: false
|
112
|
+
requirements:
|
113
|
+
- - ! '>='
|
114
|
+
- !ruby/object:Gem::Version
|
115
|
+
version: '0'
|
116
|
+
required_rubygems_version: !ruby/object:Gem::Requirement
|
117
|
+
none: false
|
118
|
+
requirements:
|
119
|
+
- - ! '>='
|
120
|
+
- !ruby/object:Gem::Version
|
121
|
+
version: '0'
|
122
|
+
requirements: []
|
123
|
+
rubyforge_project:
|
124
|
+
rubygems_version: 1.8.23
|
125
|
+
signing_key:
|
126
|
+
specification_version: 3
|
127
|
+
summary: Simple, polyglot alerting system
|
128
|
+
test_files:
|
129
|
+
- spec/skywatch_spec.rb
|