jobmanager 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- data/FAQ.txt +151 -0
- data/History.txt +2 -0
- data/LICENSE +20 -0
- data/Manifest.txt +42 -0
- data/README.txt +395 -0
- data/Rakefile +17 -0
- data/bin/jobmanager +20 -0
- data/examples/email_settings.rb +14 -0
- data/examples/example.rb +23 -0
- data/examples/jobmanager.yaml +66 -0
- data/examples/mysql_backup.rb +18 -0
- data/lib/jobmanager.rb +1 -0
- data/lib/jobmanager/application.rb +455 -0
- data/lib/jobmanager/applicationconfig.rb +89 -0
- data/lib/jobmanager/applicationlogger.rb +306 -0
- data/lib/jobmanager/applicationoptionparser.rb +163 -0
- data/lib/jobmanager/system.rb +240 -0
- data/lib/jobmanager/teestream.rb +78 -0
- data/lib/jobmanager/util.rb +25 -0
- data/test/configs/all_values.yaml +33 -0
- data/test/configs/bad_email_condition.yaml +7 -0
- data/test/configs/bad_no_central_log_file.yaml +7 -0
- data/test/configs/bad_number_of_job_logs.yaml +7 -0
- data/test/configs/bad_timeout_zero.yaml +7 -0
- data/test/configs/email_settings.rb +16 -0
- data/test/configs/incomplete_1.yaml +23 -0
- data/test/configs/jobmanager_1.yaml +9 -0
- data/test/configs/jobmanager_2.yaml +11 -0
- data/test/configs/jobmanager_3.yaml +13 -0
- data/test/configs/jobmanager_4_bad_central_log_file.yaml +9 -0
- data/test/configs/jobmanager_4_bad_email_settings_file.yaml +9 -0
- data/test/configs/jobmanager_4_bad_job_logs_directory_1.yaml +9 -0
- data/test/configs/jobmanager_4_bad_job_logs_directory_2.yaml +9 -0
- data/test/configs/minimum_plus_email_settings.yaml +9 -0
- data/test/configs/minimum_required.yaml +5 -0
- data/test/helpers.rb +7 -0
- data/test/mock_syslog.rb +68 -0
- data/test/test_applicationlogger.rb +145 -0
- data/test/test_config.rb +208 -0
- data/test/test_jobmanager.rb +443 -0
- data/test/test_system.rb +206 -0
- data/test/test_teestream.rb +55 -0
- metadata +172 -0
data/FAQ.txt
ADDED
@@ -0,0 +1,151 @@
|
|
1
|
+
=== Does +jobmanager+ rotate its own log file?
|
2
|
+
|
3
|
+
No. If +jobmanager+ is configured to log directly to a file
|
4
|
+
(+central_log_mode+ is set to +file+), +jobmanager+ does not rotate
|
5
|
+
this file. This is because most versions of Unix ship with a standard
|
6
|
+
log rotation utility that already provides a large number of features.
|
7
|
+
Most variants of Linux ship with +logrotate+, Solaris ships with
|
8
|
+
+logadm+, FreeBSD with +newsyslog+.
|
9
|
+
|
10
|
+
For example, in Fedora, one can specify that +jobmanager+'s central
|
11
|
+
log file be rotated weekly (via the +logrotate+ utility) by creating
|
12
|
+
the following file.
|
13
|
+
|
14
|
+
======<tt>/etc/logrotate.d/jobmanager</tt>:
|
15
|
+
/var/log/jobmanager.log {
|
16
|
+
weekly
|
17
|
+
missingok
|
18
|
+
}
|
19
|
+
|
20
|
+
|
21
|
+
If +jobmanager+ is configured to log via syslog (+central_log_mode+
|
22
|
+
is set to +syslog+), the +jobmanager+ output will be directed to a
|
23
|
+
central system log file (eg. +/var/log/messages+ on many Unix systems)
|
24
|
+
for which log rotation will already be in place.
|
25
|
+
|
26
|
+
=== Can I run +jobmanager+ out of my own user crontab even though I don't have permissions to edit +/etc/croncan.yaml+?
|
27
|
+
|
28
|
+
Yes. You can do this by setting the environment variable
|
29
|
+
+JOBMANAGER_CONFIG_FILE+ to a config file location of your choosing.
|
30
|
+
|
31
|
+
=== What should I set the +central_log_mode+ configuration parameter to? +syslog+ or +file+?
|
32
|
+
|
33
|
+
This depends on how +jobmanager+ will be used as well as the user's
|
34
|
+
personal preference. One caveat with using the +file+ option is that
|
35
|
+
if +jobmanager+ is invoked from the cron and there is an error
|
36
|
+
reading the configuration file, +jobmanager+ will not be able to log
|
37
|
+
this error to the central log file because it doesn't yet know the
|
38
|
+
name of the central log file- it's in the configuration file! Thus,
|
39
|
+
+jobmanager+ will attempt to log this error to syslog. This can be
|
40
|
+
avoided by doing a test run and invoking jobmanager with your new
|
41
|
+
cron job via the command line before editing your crontab.
|
42
|
+
|
43
|
+
A second point to keep in mind is that if you plan to have multiple
|
44
|
+
users share the configuration file, and log to the same central log
|
45
|
+
file, you will have to be careful about write permissions on the
|
46
|
+
central log file (the central log file must be world writable or group
|
47
|
+
writable with all users sharing the same group). If you make the
|
48
|
+
central log file group writable (and not world writable), you must
|
49
|
+
make sure that the central_log_file is created (after rotation) with
|
50
|
+
the proper group name. You need not consider these concerns if you
|
51
|
+
have multiple users logging via syslog as no permissions are required
|
52
|
+
for this.
|
53
|
+
|
54
|
+
=== When I run +jobmanager+ on the command line everything works, but when I run it through cron, it fails! Why?
|
55
|
+
|
56
|
+
There are a number reasons this could be the case all arising from the
|
57
|
+
fact that the environment from which you invoke +jobmanager+ on the
|
58
|
+
command line is different than the environment from which cron
|
59
|
+
invokes +jobmanager+. When you run a command from the shell, your
|
60
|
+
+.profile+ has already been loaded (which has set a number of
|
61
|
+
environment variables). When cron invokes a command, the +.profile+
|
62
|
+
will not be loaded unless you explicitly load it yourself.
|
63
|
+
|
64
|
+
Here is an example crontab entry which calls upon bash to load your
|
65
|
+
+.profile+ before running +jobmanager+.
|
66
|
+
00 22 * * * bash -lc "jobmanager --job_name nightly_backup 'mysql_backup test'"
|
67
|
+
|
68
|
+
If you have set the environment variable +JOBMANAGER_CONFIG_FILE+ in
|
69
|
+
your +.profile+, you will need to explicitly load your +.profile+ before
|
70
|
+
calling +jobmanager+ in the crontab entry.
|
71
|
+
|
72
|
+
=== +jobmanager+ fails when invoked through the cron. But there are no error messages in the central log file! Why?
|
73
|
+
If +jobmanager+ encounters an error before opening the central log
|
74
|
+
file, it will not be able to log this error to the central log file!
|
75
|
+
Such errors will be logged to syslog. Possible errors include:
|
76
|
+
failure to open or parse the +jobmanager+ configuration file, failure
|
77
|
+
to open or parse the email settings file, and failure to open the
|
78
|
+
central log file.
|
79
|
+
|
80
|
+
=== I have configured +jobmanager+ to log via syslog. Where does this go?
|
81
|
+
|
82
|
+
Most if not all standard Unix systems ship with a syslog daemon
|
83
|
+
(eg. {+rsyslogd+}[http://www.rsyslog.com/] on Fedora 8 and above,
|
84
|
+
{+syslog-ng+}[http://www.balabit.com/network-security/syslog-ng/] on a
|
85
|
+
number of Linux distributions). The syslog daemon writes all log
|
86
|
+
messages to a set of log files. Which log messages are written to
|
87
|
+
which file depends on the syslog daemon's configuration. There is
|
88
|
+
usually a default log file (eg. +/var/log/messages+) where the majority
|
89
|
+
of syslog log messages are written to.
|
90
|
+
|
91
|
+
=== I have configured +jobmanager+ to log via syslog. Can all the +jobmanager+ specific output be sent to a separate file?
|
92
|
+
|
93
|
+
Yes. This requires configuring the syslog daemon.
|
94
|
+
|
95
|
+
Here, I will provide an example of how to edit +rsyslogd+'s
|
96
|
+
configuration file (<tt>/etc/rsyslog.conf</tt>) to send all
|
97
|
+
+jobmanager+ output to +/var/log/jobmanager.log+.
|
98
|
+
{+rsyslogd+}[http://www.rsyslog.com/] is the syslog daemon that ships
|
99
|
+
with Fedora 8 (and above). At the top of the Rules section, add the
|
100
|
+
lines below.
|
101
|
+
|
102
|
+
======<tt>/etc/rsyslog.conf</tt>:
|
103
|
+
# RULES
|
104
|
+
...
|
105
|
+
|
106
|
+
:programname, isequal, "jobmanager" /var/log/jobmanager.log
|
107
|
+
|
108
|
+
# Prevent the jobmanager specific messages from being written to any other log files.
|
109
|
+
:programname, isequal, "jobmanager" ~
|
110
|
+
|
111
|
+
...
|
112
|
+
|
113
|
+
|
114
|
+
|
115
|
+
See the {<tt>rsyslog home page</tt>}[http://www.rsyslog.com/] for further details. For other Unix systems, consult the appropriate syslog daemon's manual.
|
116
|
+
|
117
|
+
If you configure the syslog daemon to write +jobmanager+ specific
|
118
|
+
output to a separate file, you may also want to configure your Unix
|
119
|
+
system's log rotation utility to rotate this file as well.
|
120
|
+
|
121
|
+
=== Can multiple users use the same configuration file? Are there any caveats with doing this?
|
122
|
+
|
123
|
+
Yes, multiple users can use the same configuration file. In fact, if
|
124
|
+
so desired, you can specify that the +job_logs_directory+,
|
125
|
+
+central_log_file+, and/or +email_settings_file+ each be different for
|
126
|
+
each user. This can be achieved by including the +user_name+ variable
|
127
|
+
(with ERB syntax) in the desired configuration parameter's value.
|
128
|
+
|
129
|
+
Here is an example configuration file excerpt that includes the +user_name+ variable.
|
130
|
+
|
131
|
+
|
132
|
+
job_logs_directory: /home/<%=user_name%>/cron_jobs
|
133
|
+
|
134
|
+
If you have configured +jobmanager+ to log directly to a file, and the
|
135
|
+
central log file is the same across multiple users, you must ensure
|
136
|
+
that all users have permissions to write to this central log file.
|
137
|
+
Refer to the FAQ question concerning whether to set the central log
|
138
|
+
mode to file or syslog for more details concerning this.
|
139
|
+
|
140
|
+
=== Should I specify the to/from email addresses in the {+simpleemail+}[http://simpleemail.rubyforge.org] configuration file or directly in +jobmanager.yaml+?
|
141
|
+
|
142
|
+
Both options will work fine. On my system, I use the
|
143
|
+
{+simpleemail+}[http://simpleemail.rubyforge.org] gem in other pieces
|
144
|
+
of Ruby code, and in most cases, send email from one default email
|
145
|
+
address. As a result, I prefer to specify the +default_to+,
|
146
|
+
+default_from+ email address parameters in the
|
147
|
+
{+simpleemail+}[http://simpleemail.rubyforge.org] configuration file
|
148
|
+
to minimize duplication.
|
149
|
+
|
150
|
+
=== Can +jobmanager+ send email via smtp with tls/ssl enabled?
|
151
|
+
Yes. See the {+simpleemail+}[http://simpleemail.rubyforge.org] gem for details.
|
data/History.txt
ADDED
data/LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2008 Designing Patterns
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
'Software'), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
17
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
18
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
19
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
20
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/Manifest.txt
ADDED
@@ -0,0 +1,42 @@
|
|
1
|
+
Manifest.txt
|
2
|
+
LICENSE
|
3
|
+
History.txt
|
4
|
+
FAQ.txt
|
5
|
+
README.txt
|
6
|
+
Rakefile
|
7
|
+
bin/jobmanager
|
8
|
+
examples/email_settings.rb
|
9
|
+
examples/example.rb
|
10
|
+
examples/mysql_backup.rb
|
11
|
+
examples/jobmanager.yaml
|
12
|
+
lib/jobmanager.rb
|
13
|
+
lib/jobmanager/applicationconfig.rb
|
14
|
+
lib/jobmanager/applicationlogger.rb
|
15
|
+
lib/jobmanager/applicationoptionparser.rb
|
16
|
+
lib/jobmanager/application.rb
|
17
|
+
lib/jobmanager/system.rb
|
18
|
+
lib/jobmanager/teestream.rb
|
19
|
+
lib/jobmanager/util.rb
|
20
|
+
test/helpers.rb
|
21
|
+
test/mock_syslog.rb
|
22
|
+
test/test_applicationlogger.rb
|
23
|
+
test/test_config.rb
|
24
|
+
test/test_jobmanager.rb
|
25
|
+
test/test_system.rb
|
26
|
+
test/test_teestream.rb
|
27
|
+
test/configs/email_settings.rb
|
28
|
+
test/configs/all_values.yaml
|
29
|
+
test/configs/bad_email_condition.yaml
|
30
|
+
test/configs/bad_no_central_log_file.yaml
|
31
|
+
test/configs/bad_number_of_job_logs.yaml
|
32
|
+
test/configs/bad_timeout_zero.yaml
|
33
|
+
test/configs/incomplete_1.yaml
|
34
|
+
test/configs/jobmanager_1.yaml
|
35
|
+
test/configs/jobmanager_2.yaml
|
36
|
+
test/configs/jobmanager_3.yaml
|
37
|
+
test/configs/jobmanager_4_bad_central_log_file.yaml
|
38
|
+
test/configs/jobmanager_4_bad_email_settings_file.yaml
|
39
|
+
test/configs/jobmanager_4_bad_job_logs_directory_1.yaml
|
40
|
+
test/configs/jobmanager_4_bad_job_logs_directory_2.yaml
|
41
|
+
test/configs/minimum_plus_email_settings.yaml
|
42
|
+
test/configs/minimum_required.yaml
|
data/README.txt
ADDED
@@ -0,0 +1,395 @@
|
|
1
|
+
= jobmanager
|
2
|
+
* Project Page: http://rubyforge.org/projects/jobmanager/
|
3
|
+
* Documentation: http://jobmanager.rubyforge.org/
|
4
|
+
|
5
|
+
== DESCRIPTION:
|
6
|
+
|
7
|
+
This package makes managing cron jobs on unix systems a breeze! It is
|
8
|
+
composed of a single program, +jobmanager+, which is intended to be
|
9
|
+
invoked by cron, or a cron-like application. +jobmanager+, in turn,
|
10
|
+
invokes the specified cron job, in addition to providing the following
|
11
|
+
added value:
|
12
|
+
|
13
|
+
* Logs the output of the cron jobs.
|
14
|
+
* Rotates the cron job logs.
|
15
|
+
* Manages a central log which records all cron events (when each job
|
16
|
+
starts, finishes, the job's success, etc).
|
17
|
+
* Emails the user the results of the cron job on a configurable
|
18
|
+
condition (on failure only, on success only, etc.) via sendmail or
|
19
|
+
smtp.
|
20
|
+
* Times out a cron job that runs past the configurable timeout.
|
21
|
+
* Easily scales to handle multiple users' cron jobs.
|
22
|
+
* Both the default behavior and the per-cron job behavior are highly
|
23
|
+
configurable.
|
24
|
+
|
25
|
+
== SYNOPSIS:
|
26
|
+
|
27
|
+
Lets walk through an example invocation of +jobmanager+ via cron.
|
28
|
+
|
29
|
+
Here is an example +crontab+ entry that invokes a simplified database
|
30
|
+
backup script via +jobmanager+ every day at 22:00.
|
31
|
+
00 22 * * * jobmanager --job_name nightly_backup "mysql_backup test"
|
32
|
+
|
33
|
+
Here is the example +mysql_backup+ script referenced above:
|
34
|
+
======<tt>examples/mysql_backup.rb</tt>:
|
35
|
+
#!/usr/bin/env ruby
|
36
|
+
|
37
|
+
print "=================================================\n"
|
38
|
+
print "Backing up Database\n"
|
39
|
+
print "=================================================\n"
|
40
|
+
|
41
|
+
database = ARGV[0]
|
42
|
+
|
43
|
+
command = "mysqldump #{database} > /tmp/#{database}.mysql"
|
44
|
+
print command, "\n"
|
45
|
+
|
46
|
+
if (system("#{command}"))
|
47
|
+
print"\nSuccess!\n"
|
48
|
+
else
|
49
|
+
print "\nFailure!\n"
|
50
|
+
exit(1)
|
51
|
+
end
|
52
|
+
|
53
|
+
|
54
|
+
|
55
|
+
Here is the configuration file to be read by +jobmanager+.
|
56
|
+
======<tt>/etc/jobmanager.yaml</tt>:
|
57
|
+
################################################################################
|
58
|
+
# The jobmanager configuration file.
|
59
|
+
#
|
60
|
+
# For a full description of each of the fields, and their default values,
|
61
|
+
# see the CONFIGURATION FILE section in README.txt.
|
62
|
+
################################################################################
|
63
|
+
|
64
|
+
# The number of log files to be kept per job.
|
65
|
+
number_of_job_logs: 5
|
66
|
+
|
67
|
+
# The directory in which the job logs will be kept, to be interpreted
|
68
|
+
# by ERB. Allowed variables: user_name.
|
69
|
+
job_logs_directory: /tmp/logs
|
70
|
+
|
71
|
+
# jobmanager logs operational information (when jobs are launched,
|
72
|
+
# exit, etc). This can be done via syslog or directly to a log file.
|
73
|
+
# Possible values are: syslog, file.
|
74
|
+
central_log_mode: file
|
75
|
+
|
76
|
+
# If the central_log_mode is set to "file", the log file must be
|
77
|
+
# specified. This parameter will be interpreted by ERB. Allowed
|
78
|
+
# variables: user_name.
|
79
|
+
central_log_file: /tmp/logs/jobmanager.log
|
80
|
+
|
81
|
+
# The path to search for the command that jobmanager is invoked with.
|
82
|
+
#command_path: /usr/bin:/usr/designingpatterns/bin
|
83
|
+
|
84
|
+
# Whether jobmanager should print debug trace to the central log file.
|
85
|
+
debug: false
|
86
|
+
|
87
|
+
# The condition upon which condition results should be emailed.
|
88
|
+
# Possible values are: always, never, on_failure.
|
89
|
+
email_condition: always
|
90
|
+
|
91
|
+
# The configuration file for the simpleemail gem. This parameter will
|
92
|
+
# be interpreted by ERB. Allowed variables: user_name. Note: If a
|
93
|
+
# relative path is specified, it will be considered to be relative to
|
94
|
+
# the location of this configuration file.
|
95
|
+
email_settings_file: email_settings.rb
|
96
|
+
|
97
|
+
# The email subject, to be interpreted by ERB.
|
98
|
+
# Allowed variables: job_name, command, host_name, results, user_name.
|
99
|
+
email_subject: "jobmanager results for <%=job_name%> on <%=host_name%> : <%=result%>"
|
100
|
+
|
101
|
+
# The extension of the rotated job log file. If true, the extension
|
102
|
+
# is a date_time. Otherwise, it is a simple count (.1, .2, etc.).
|
103
|
+
date_time_extension: true
|
104
|
+
|
105
|
+
# If date_time_extension is true, this field will be used as the
|
106
|
+
# format for the date/time extension of the rotated job log file
|
107
|
+
# name.
|
108
|
+
date_time_extension_format: '%F_%T'
|
109
|
+
|
110
|
+
# Whether the rotated job log file should be zipped.
|
111
|
+
zip_rotated_log_file: false
|
112
|
+
|
113
|
+
# The timeout (in seconds). If not present, no timeout will be used.
|
114
|
+
timeout: 1800
|
115
|
+
|
116
|
+
# The sender of emails sent from jobmanager. This can also be
|
117
|
+
# specified in the email_settings_file (attribute default_from).
|
118
|
+
#email_from_address: "Sender <sender@gmail.com>"
|
119
|
+
|
120
|
+
# The receiver of emails sent from jobmanager. This can also be
|
121
|
+
# specified in the email_settings_file (attributes default_to).
|
122
|
+
#email_to_address: "Receiver <receiver@gmail.com>"
|
123
|
+
|
124
|
+
|
125
|
+
Here is the email configuration file referenced in +jobmanager.yaml+
|
126
|
+
above. In this example, we choose to email via smtp, although
|
127
|
+
sendmail is also an option. This file is documented further in the
|
128
|
+
{+simpleemail+}[http://simpleemail.rubyforge.org] gem.
|
129
|
+
======<tt>/etc/email_settings.rb</tt>:
|
130
|
+
ActionMailer::Base.delivery_method = :smtp
|
131
|
+
|
132
|
+
ActionMailer::Base.smtp_settings = {
|
133
|
+
:address => 'smtp.gmail.com',
|
134
|
+
:port => 587,
|
135
|
+
:domain => "gmail.com",
|
136
|
+
:authentication => "login",
|
137
|
+
:user_name => "sender_address@gmail.com",
|
138
|
+
:password => "password",
|
139
|
+
:tls => :auto
|
140
|
+
}
|
141
|
+
|
142
|
+
SimpleEmail::Email.default_from = "Sender Address <sender_address@gmail.com>"
|
143
|
+
SimpleEmail::Email.default_to = "Receiver Address <receiver_address@gmail.com>"
|
144
|
+
|
145
|
+
|
146
|
+
Here is the central log file that +jobmanager+ updated.
|
147
|
+
======<tt>/tmp/logs/jobmanager.log</tt>:
|
148
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Command (/usr/bin/mysql_backup test) launched, pid = 2302.
|
149
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Exited successfully
|
150
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Job log rotated to /tmp/logs/nightly_backup.log.2008-07-30_22:00:00.
|
151
|
+
|
152
|
+
|
153
|
+
Here is the rotated job log file.
|
154
|
+
======<tt>/tmp/logs/nightly_backup.log.2008-07-30_22:00:00</tt>:
|
155
|
+
=================================================
|
156
|
+
Backing up Database
|
157
|
+
=================================================
|
158
|
+
mysqldump test > /tmp/test.mysql
|
159
|
+
|
160
|
+
Success!
|
161
|
+
|
162
|
+
Here is the subject and body of the email that was sent upon job completion.
|
163
|
+
|
164
|
+
======<tt>Email Subject</tt>:
|
165
|
+
jobmanager results for nightly_backup on jackfruit : Success
|
166
|
+
|
167
|
+
======<tt>Email Body</tt>:
|
168
|
+
---------------------------------------------------------
|
169
|
+
jobmanager Output:
|
170
|
+
---------------------------------------------------------
|
171
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Command (/usr/bin/mysql_backup test) launched, pid = 2302.
|
172
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Exited successfully
|
173
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Job log rotated to /tmp/logs/nightly_backup.log.2008-08-01_11:29:47.
|
174
|
+
|
175
|
+
---------------------------------------------------------
|
176
|
+
Job Output:
|
177
|
+
---------------------------------------------------------
|
178
|
+
=================================================
|
179
|
+
Backing up Database
|
180
|
+
=================================================
|
181
|
+
mysqldump test > /tmp/test.mysql
|
182
|
+
|
183
|
+
Success!
|
184
|
+
|
185
|
+
|
186
|
+
|
187
|
+
== CONFIGURATION FILE:
|
188
|
+
|
189
|
+
A configuration file is required for +jobmanager+ to run. By default,
|
190
|
+
+jobmanager+ will expect the configuration file to be located at
|
191
|
+
+/etc/jobmanager.yaml+. If the environment variable
|
192
|
+
+JOBMANAGER_CONFIG_FILE+ is set, +jobmanager+ will instead read the
|
193
|
+
configuration file specified by this variable.
|
194
|
+
|
195
|
+
The configuration file provides the following parameters:
|
196
|
+
|
197
|
+
[+job_logs_directory+: required]
|
198
|
+
The directory in which the job log files will be kept. The
|
199
|
+
+job_logs_directory+ parameter will be interpreted by ERB. The
|
200
|
+
allowed variables are: +user_name+. Specifying the +user_name+
|
201
|
+
variable in the +job_logs_directory+ value can be useful if you
|
202
|
+
want multiple users to share the same configuration file, but have
|
203
|
+
different job logs directories.
|
204
|
+
|
205
|
+
[+central_log_mode+: optional, default: +syslog+]
|
206
|
+
+jobmanager+ logs operational information (when jobs are
|
207
|
+
launched, exit, etc). This can be done via +syslog+ or directly to a
|
208
|
+
log file. Possible values are: <tt>syslog, file</tt>.
|
209
|
+
|
210
|
+
[+central_log_file+: optional]
|
211
|
+
If +central_log_mode+ is set to +file+, this parameter must be set.
|
212
|
+
The +central_log_file+ parameter will be interpreted by ERB. The
|
213
|
+
allowed variables are: +user_name+. Specifying the +user_name+
|
214
|
+
variable in the +central_log_file+ value can be useful if you want
|
215
|
+
multiple users to share the same configuration file, but have
|
216
|
+
different central log files.
|
217
|
+
|
218
|
+
[+number_of_job_logs+: optional, default: +3+]
|
219
|
+
The number of logs to be kept per job.
|
220
|
+
|
221
|
+
[+command_path+: optional, default: <tt>ENV['PATH']</tt> ]
|
222
|
+
The path to be searched for the command that +jobmanager+ is
|
223
|
+
invoked with. If unspecified, the environment path will be used.
|
224
|
+
|
225
|
+
[+debug+: optional, default: +false+]
|
226
|
+
Whether +jobmanager+ should print debug trace to the central log file.
|
227
|
+
|
228
|
+
[+email_condition+: optional, default: +on_failure+]
|
229
|
+
The condition upon which results should be emailed. Possible
|
230
|
+
values are: <tt>always, never, on_failure</tt>.
|
231
|
+
|
232
|
+
[+email_settings_file+: optional, default: +email_settings.rb+]
|
233
|
+
The configuration file for the
|
234
|
+
{+simpleemail+}[http://simpleemail.rubyforge.org] gem. This
|
235
|
+
specifies the delivery method (+smtp+, +sendmail+), as well as
|
236
|
+
delivery method specific settings. Default to/from emails can be
|
237
|
+
specified in here. If the path specified is relative, it is assumed
|
238
|
+
that it is relative to the directory in which the +jobmanager+
|
239
|
+
configuration file is located. If +email_condition+ is set to a
|
240
|
+
value other than +never+, this parameter must specified. The
|
241
|
+
+email_settings_file+ parameter will be interpreted by ERB. The
|
242
|
+
allowed variables are: +user_name+. Specifying the +user_name+
|
243
|
+
variable in the +email_settings_file+ value can be useful if you
|
244
|
+
want multiple users to share the same configuration file, but have
|
245
|
+
separate email settings files.
|
246
|
+
|
247
|
+
|
248
|
+
[+email_subject+, optional, default: <tt>"jobmanager results for <%=job_name%> on <%=host_name%> : <%=result%>"</tt>]
|
249
|
+
The email subject, to be interpreted by ERB. Allowed variables
|
250
|
+
inside the string are: <tt>job_name, command, host_name, results, user_name</tt>.
|
251
|
+
|
252
|
+
[+date_time_extension+, optional, default: +true+]
|
253
|
+
This refers to the extension of the rotated job log file. If
|
254
|
+
+date_time_extension+ is present and set to +true+, the extension
|
255
|
+
represents a date and time. Otherwise, it is a simple count (.1, .2, etc.).
|
256
|
+
|
257
|
+
[+date_time_extension_format+, optional, default: <tt>'%F_%T'</tt>]
|
258
|
+
If +date_time_extension+ is +true+, this parameter will be used as the
|
259
|
+
format for the date and time extension of the rotated job log file.
|
260
|
+
|
261
|
+
[+zip_rotated_log_file+, optional, default: +false+]
|
262
|
+
Whether the rotated job log file should be gzip'ed.
|
263
|
+
|
264
|
+
[+timeout+, optional]
|
265
|
+
The timeout (in seconds). If the job spawned by +jobmanager+ does not
|
266
|
+
exit after +timeout+ seconds, +jobmanager+ will kill the process, and
|
267
|
+
report failure. If +timeout+ is unspecifed, +jobmanager+ will let the
|
268
|
+
process run for an unlimited amount of time (+jobmanager+ will simply
|
269
|
+
wait til it returns).
|
270
|
+
|
271
|
+
[+email_from_address+, optional]
|
272
|
+
The sender of emails sent from +jobmanager+. This can also be
|
273
|
+
specified in the +email_settings_file+ (attribute +default_from+).
|
274
|
+
|
275
|
+
[+email_to_address+, optional]
|
276
|
+
The receiver of emails sent from +jobmanager+. This can also be
|
277
|
+
specified in the +email_settings_file+ (attribute +default_to+).
|
278
|
+
|
279
|
+
== COMMAND LINE ARGUMENTS:
|
280
|
+
|
281
|
+
Usage:
|
282
|
+
Usage: jobmanager.rb [options] <command>
|
283
|
+
|
284
|
+
Notable command line options:
|
285
|
+
[<tt>-j, --job_name</tt>]
|
286
|
+
The name of the job to be run. If this is unspecified, the
|
287
|
+
job name will be set to the basename of the COMMAND specified.
|
288
|
+
The job name is used when logging to the central log file, and
|
289
|
+
in the name of the job log file.
|
290
|
+
|
291
|
+
The remaining command line options are duplicates of the configuration
|
292
|
+
file parameters. In fact, all of the configuration parameters
|
293
|
+
specified in the configuration file can be overriden via +jobmanager+
|
294
|
+
command line options.
|
295
|
+
|
296
|
+
== STARTING JOBMANAGER FROM THE COMMAND LINE VS. CRON:
|
297
|
+
|
298
|
+
The most common use case for +jobmanager+ is to be run from cron, or a
|
299
|
+
cron-like application. However, invoking +jobmanager+ from the
|
300
|
+
command line can be useful for testing, debugging, and one-off runs.
|
301
|
+
As +jobmanager+ is commonly invoked via the cron, +jobmanager+ logs
|
302
|
+
all output to either +syslog+ or a central log file (specified by the
|
303
|
+
configuration parameters +central_log_mode+, +central_log_file+).
|
304
|
+
When +jobmanager+ is run from the command line (or any script whose
|
305
|
+
STDIN, STDOUT, or STDERR is connected to a terminal), these two
|
306
|
+
configuration parameters will be ignored and +jobmanager+'s output
|
307
|
+
will be sent to the terminal.
|
308
|
+
|
309
|
+
== REQUIREMENTS:
|
310
|
+
|
311
|
+
The +jobmanager+ application requires a number of gems: hoe,
|
312
|
+
SyslogLogger, sys-host, configtoolkit, simpleemail, logrotate,
|
313
|
+
assertions, and relative.
|
314
|
+
|
315
|
+
Hoe and assertions are required only for running the tests. Relative is
|
316
|
+
required for running the tests and the examples. The rest of the gems
|
317
|
+
listed above are required for the proper functioning of the
|
318
|
+
+jobmanager+ application. The sys-host gem is required only for the
|
319
|
+
purpose of including the hostname in the email subject line.
|
320
|
+
|
321
|
+
== INSTALL:
|
322
|
+
|
323
|
+
1. <tt>sudo gem install jobmanager</tt>
|
324
|
+
|
325
|
+
2. Set up the +jobmanager+ configuration file (+jobmanager.yaml+).
|
326
|
+
For more detailed instructions, see the CONFIGURATION FILE section
|
327
|
+
above.
|
328
|
+
|
329
|
+
3. Set up the simpleemail configuration file (+email_settings.rb+).
|
330
|
+
Alternately, if you want to disable email, set the configuration
|
331
|
+
parameter +email_condition+ to +never+ in +jobmanager.yaml+. See
|
332
|
+
the {+simpleemail+}[http://simpleemail.rubyforge.org] gem rdoc for
|
333
|
+
more details.
|
334
|
+
|
335
|
+
4. Confirm that the configuration files are valid by invoking
|
336
|
+
+jobmanager+ from the command line with one of your cron jobs or a
|
337
|
+
test command.
|
338
|
+
|
339
|
+
5. Edit your +crontab+ to run new and/or existing cron jobs through
|
340
|
+
+jobmanager+.
|
341
|
+
|
342
|
+
For possible reasons why running +jobmanager+ via the command line may
|
343
|
+
be successful, but running +jobmanager+ with the same configuration
|
344
|
+
via the cron may fail, consult FAQ.txt.
|
345
|
+
|
346
|
+
== PROBLEMS:
|
347
|
+
|
348
|
+
None (known).
|
349
|
+
|
350
|
+
== AUTHORS:
|
351
|
+
=== Designing Patterns
|
352
|
+
* Homepage: http://www.designingpatterns.com
|
353
|
+
* Blogs: http://blogs.designingpatterns.com
|
354
|
+
|
355
|
+
== SUPPORT:
|
356
|
+
Please post questions, concerns, or requests for enhancement to the forums on
|
357
|
+
the project page. Alternatively, direct contact information for
|
358
|
+
Designing Patterns can be found on the project page for this gem.
|
359
|
+
|
360
|
+
== ENHANCEMENTS:
|
361
|
+
Please feel free to contact us with any ideas; we will try our best to
|
362
|
+
enhance the software and respond to user requests. Of course, we are more
|
363
|
+
likely to work on a particular enhancement if we know that there are users
|
364
|
+
who want it. Designing Patterns provides contracting and consulting services,
|
365
|
+
so if there is an enhancement that *must* get done (and in a specified time
|
366
|
+
frame), please inquire about retaining our services!
|
367
|
+
|
368
|
+
== LICENSE:
|
369
|
+
The license text can be found in the +LICENSE+ file at the root of the
|
370
|
+
distribution.
|
371
|
+
|
372
|
+
This package is licensed with an MIT license:
|
373
|
+
|
374
|
+
Copyright (c) 2008 Designing Patterns
|
375
|
+
|
376
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
377
|
+
a copy of this software and associated documentation files (the
|
378
|
+
'Software'), to deal in the Software without restriction, including
|
379
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
380
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
381
|
+
permit persons to whom the Software is furnished to do so, subject to
|
382
|
+
the following conditions:
|
383
|
+
|
384
|
+
The above copyright notice and this permission notice shall be
|
385
|
+
included in all copies or substantial portions of the Software.
|
386
|
+
|
387
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
388
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
389
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
390
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
391
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
392
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
393
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
394
|
+
|
395
|
+
== SHARE AND ENJOY!
|