jobmanager 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/FAQ.txt +151 -0
- data/History.txt +2 -0
- data/LICENSE +20 -0
- data/Manifest.txt +42 -0
- data/README.txt +395 -0
- data/Rakefile +17 -0
- data/bin/jobmanager +20 -0
- data/examples/email_settings.rb +14 -0
- data/examples/example.rb +23 -0
- data/examples/jobmanager.yaml +66 -0
- data/examples/mysql_backup.rb +18 -0
- data/lib/jobmanager.rb +1 -0
- data/lib/jobmanager/application.rb +455 -0
- data/lib/jobmanager/applicationconfig.rb +89 -0
- data/lib/jobmanager/applicationlogger.rb +306 -0
- data/lib/jobmanager/applicationoptionparser.rb +163 -0
- data/lib/jobmanager/system.rb +240 -0
- data/lib/jobmanager/teestream.rb +78 -0
- data/lib/jobmanager/util.rb +25 -0
- data/test/configs/all_values.yaml +33 -0
- data/test/configs/bad_email_condition.yaml +7 -0
- data/test/configs/bad_no_central_log_file.yaml +7 -0
- data/test/configs/bad_number_of_job_logs.yaml +7 -0
- data/test/configs/bad_timeout_zero.yaml +7 -0
- data/test/configs/email_settings.rb +16 -0
- data/test/configs/incomplete_1.yaml +23 -0
- data/test/configs/jobmanager_1.yaml +9 -0
- data/test/configs/jobmanager_2.yaml +11 -0
- data/test/configs/jobmanager_3.yaml +13 -0
- data/test/configs/jobmanager_4_bad_central_log_file.yaml +9 -0
- data/test/configs/jobmanager_4_bad_email_settings_file.yaml +9 -0
- data/test/configs/jobmanager_4_bad_job_logs_directory_1.yaml +9 -0
- data/test/configs/jobmanager_4_bad_job_logs_directory_2.yaml +9 -0
- data/test/configs/minimum_plus_email_settings.yaml +9 -0
- data/test/configs/minimum_required.yaml +5 -0
- data/test/helpers.rb +7 -0
- data/test/mock_syslog.rb +68 -0
- data/test/test_applicationlogger.rb +145 -0
- data/test/test_config.rb +208 -0
- data/test/test_jobmanager.rb +443 -0
- data/test/test_system.rb +206 -0
- data/test/test_teestream.rb +55 -0
- metadata +172 -0
data/FAQ.txt
ADDED
@@ -0,0 +1,151 @@
|
|
1
|
+
=== Does +jobmanager+ rotate its own log file?
|
2
|
+
|
3
|
+
No. If +jobmanager+ is configured to log directly to a file
|
4
|
+
(+central_log_mode+ is set to +file+), +jobmanager+ does not rotate
|
5
|
+
this file. This is because most versions of Unix ship with a standard
|
6
|
+
log rotation utility that already provides a large number of features.
|
7
|
+
Most variants of Linux ship with +logrotate+, Solaris ships with
|
8
|
+
+logadm+, FreeBSD with +newsyslog+.
|
9
|
+
|
10
|
+
For example, in Fedora, one can specify that +jobmanager+'s central
|
11
|
+
log file be rotated weekly (via the +logrotate+ utility) by creating
|
12
|
+
the following file.
|
13
|
+
|
14
|
+
======<tt>/etc/logrotate.d/jobmanager</tt>:
|
15
|
+
/var/log/jobmanager.log {
|
16
|
+
weekly
|
17
|
+
missingok
|
18
|
+
}
|
19
|
+
|
20
|
+
|
21
|
+
If +jobmanager+ is configured to log via syslog (+central_log_mode+
|
22
|
+
is set to +syslog+), the +jobmanager+ output will be directed to a
|
23
|
+
central system log file (eg. +/var/log/messages+ on many Unix systems)
|
24
|
+
for which log rotation will already be in place.
|
25
|
+
|
26
|
+
=== Can I run +jobmanager+ out of my own user crontab even though I don't have permissions to edit +/etc/croncan.yaml+?
|
27
|
+
|
28
|
+
Yes. You can do this by setting the environment variable
|
29
|
+
+JOBMANAGER_CONFIG_FILE+ to a config file location of your choosing.
|
30
|
+
|
31
|
+
=== What should I set the +central_log_mode+ configuration parameter to? +syslog+ or +file+?
|
32
|
+
|
33
|
+
This depends on how +jobmanager+ will be used as well as the user's
|
34
|
+
personal preference. One caveat with using the +file+ option is that
|
35
|
+
if +jobmanager+ is invoked from the cron and there is an error
|
36
|
+
reading the configuration file, +jobmanager+ will not be able to log
|
37
|
+
this error to the central log file because it doesn't yet know the
|
38
|
+
name of the central log file- it's in the configuration file! Thus,
|
39
|
+
+jobmanager+ will attempt to log this error to syslog. This can be
|
40
|
+
avoided by doing a test run and invoking jobmanager with your new
|
41
|
+
cron job via the command line before editing your crontab.
|
42
|
+
|
43
|
+
A second point to keep in mind is that if you plan to have multiple
|
44
|
+
users share the configuration file, and log to the same central log
|
45
|
+
file, you will have to be careful about write permissions on the
|
46
|
+
central log file (the central log file must be world writable or group
|
47
|
+
writable with all users sharing the same group). If you make the
|
48
|
+
central log file group writable (and not world writable), you must
|
49
|
+
make sure that the central_log_file is created (after rotation) with
|
50
|
+
the proper group name. You need not consider these concerns if you
|
51
|
+
have multiple users logging via syslog as no permissions are required
|
52
|
+
for this.
|
53
|
+
|
54
|
+
=== When I run +jobmanager+ on the command line everything works, but when I run it through cron, it fails! Why?
|
55
|
+
|
56
|
+
There are a number reasons this could be the case all arising from the
|
57
|
+
fact that the environment from which you invoke +jobmanager+ on the
|
58
|
+
command line is different than the environment from which cron
|
59
|
+
invokes +jobmanager+. When you run a command from the shell, your
|
60
|
+
+.profile+ has already been loaded (which has set a number of
|
61
|
+
environment variables). When cron invokes a command, the +.profile+
|
62
|
+
will not be loaded unless you explicitly load it yourself.
|
63
|
+
|
64
|
+
Here is an example crontab entry which calls upon bash to load your
|
65
|
+
+.profile+ before running +jobmanager+.
|
66
|
+
00 22 * * * bash -lc "jobmanager --job_name nightly_backup 'mysql_backup test'"
|
67
|
+
|
68
|
+
If you have set the environment variable +JOBMANAGER_CONFIG_FILE+ in
|
69
|
+
your +.profile+, you will need to explicitly load your +.profile+ before
|
70
|
+
calling +jobmanager+ in the crontab entry.
|
71
|
+
|
72
|
+
=== +jobmanager+ fails when invoked through the cron. But there are no error messages in the central log file! Why?
|
73
|
+
If +jobmanager+ encounters an error before opening the central log
|
74
|
+
file, it will not be able to log this error to the central log file!
|
75
|
+
Such errors will be logged to syslog. Possible errors include:
|
76
|
+
failure to open or parse the +jobmanager+ configuration file, failure
|
77
|
+
to open or parse the email settings file, and failure to open the
|
78
|
+
central log file.
|
79
|
+
|
80
|
+
=== I have configured +jobmanager+ to log via syslog. Where does this go?
|
81
|
+
|
82
|
+
Most if not all standard Unix systems ship with a syslog daemon
|
83
|
+
(eg. {+rsyslogd+}[http://www.rsyslog.com/] on Fedora 8 and above,
|
84
|
+
{+syslog-ng+}[http://www.balabit.com/network-security/syslog-ng/] on a
|
85
|
+
number of Linux distributions). The syslog daemon writes all log
|
86
|
+
messages to a set of log files. Which log messages are written to
|
87
|
+
which file depends on the syslog daemon's configuration. There is
|
88
|
+
usually a default log file (eg. +/var/log/messages+) where the majority
|
89
|
+
of syslog log messages are written to.
|
90
|
+
|
91
|
+
=== I have configured +jobmanager+ to log via syslog. Can all the +jobmanager+ specific output be sent to a separate file?
|
92
|
+
|
93
|
+
Yes. This requires configuring the syslog daemon.
|
94
|
+
|
95
|
+
Here, I will provide an example of how to edit +rsyslogd+'s
|
96
|
+
configuration file (<tt>/etc/rsyslog.conf</tt>) to send all
|
97
|
+
+jobmanager+ output to +/var/log/jobmanager.log+.
|
98
|
+
{+rsyslogd+}[http://www.rsyslog.com/] is the syslog daemon that ships
|
99
|
+
with Fedora 8 (and above). At the top of the Rules section, add the
|
100
|
+
lines below.
|
101
|
+
|
102
|
+
======<tt>/etc/rsyslog.conf</tt>:
|
103
|
+
# RULES
|
104
|
+
...
|
105
|
+
|
106
|
+
:programname, isequal, "jobmanager" /var/log/jobmanager.log
|
107
|
+
|
108
|
+
# Prevent the jobmanager specific messages from being written to any other log files.
|
109
|
+
:programname, isequal, "jobmanager" ~
|
110
|
+
|
111
|
+
...
|
112
|
+
|
113
|
+
|
114
|
+
|
115
|
+
See the {<tt>rsyslog home page</tt>}[http://www.rsyslog.com/] for further details. For other Unix systems, consult the appropriate syslog daemon's manual.
|
116
|
+
|
117
|
+
If you configure the syslog daemon to write +jobmanager+ specific
|
118
|
+
output to a separate file, you may also want to configure your Unix
|
119
|
+
system's log rotation utility to rotate this file as well.
|
120
|
+
|
121
|
+
=== Can multiple users use the same configuration file? Are there any caveats with doing this?
|
122
|
+
|
123
|
+
Yes, multiple users can use the same configuration file. In fact, if
|
124
|
+
so desired, you can specify that the +job_logs_directory+,
|
125
|
+
+central_log_file+, and/or +email_settings_file+ each be different for
|
126
|
+
each user. This can be achieved by including the +user_name+ variable
|
127
|
+
(with ERB syntax) in the desired configuration parameter's value.
|
128
|
+
|
129
|
+
Here is an example configuration file excerpt that includes the +user_name+ variable.
|
130
|
+
|
131
|
+
|
132
|
+
job_logs_directory: /home/<%=user_name%>/cron_jobs
|
133
|
+
|
134
|
+
If you have configured +jobmanager+ to log directly to a file, and the
|
135
|
+
central log file is the same across multiple users, you must ensure
|
136
|
+
that all users have permissions to write to this central log file.
|
137
|
+
Refer to the FAQ question concerning whether to set the central log
|
138
|
+
mode to file or syslog for more details concerning this.
|
139
|
+
|
140
|
+
=== Should I specify the to/from email addresses in the {+simpleemail+}[http://simpleemail.rubyforge.org] configuration file or directly in +jobmanager.yaml+?
|
141
|
+
|
142
|
+
Both options will work fine. On my system, I use the
|
143
|
+
{+simpleemail+}[http://simpleemail.rubyforge.org] gem in other pieces
|
144
|
+
of Ruby code, and in most cases, send email from one default email
|
145
|
+
address. As a result, I prefer to specify the +default_to+,
|
146
|
+
+default_from+ email address parameters in the
|
147
|
+
{+simpleemail+}[http://simpleemail.rubyforge.org] configuration file
|
148
|
+
to minimize duplication.
|
149
|
+
|
150
|
+
=== Can +jobmanager+ send email via smtp with tls/ssl enabled?
|
151
|
+
Yes. See the {+simpleemail+}[http://simpleemail.rubyforge.org] gem for details.
|
data/History.txt
ADDED
data/LICENSE
ADDED
@@ -0,0 +1,20 @@
|
|
1
|
+
Copyright (c) 2008 Designing Patterns
|
2
|
+
|
3
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
4
|
+
a copy of this software and associated documentation files (the
|
5
|
+
'Software'), to deal in the Software without restriction, including
|
6
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
7
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
8
|
+
permit persons to whom the Software is furnished to do so, subject to
|
9
|
+
the following conditions:
|
10
|
+
|
11
|
+
The above copyright notice and this permission notice shall be
|
12
|
+
included in all copies or substantial portions of the Software.
|
13
|
+
|
14
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
15
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
16
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
17
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
18
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
19
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
20
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
data/Manifest.txt
ADDED
@@ -0,0 +1,42 @@
|
|
1
|
+
Manifest.txt
|
2
|
+
LICENSE
|
3
|
+
History.txt
|
4
|
+
FAQ.txt
|
5
|
+
README.txt
|
6
|
+
Rakefile
|
7
|
+
bin/jobmanager
|
8
|
+
examples/email_settings.rb
|
9
|
+
examples/example.rb
|
10
|
+
examples/mysql_backup.rb
|
11
|
+
examples/jobmanager.yaml
|
12
|
+
lib/jobmanager.rb
|
13
|
+
lib/jobmanager/applicationconfig.rb
|
14
|
+
lib/jobmanager/applicationlogger.rb
|
15
|
+
lib/jobmanager/applicationoptionparser.rb
|
16
|
+
lib/jobmanager/application.rb
|
17
|
+
lib/jobmanager/system.rb
|
18
|
+
lib/jobmanager/teestream.rb
|
19
|
+
lib/jobmanager/util.rb
|
20
|
+
test/helpers.rb
|
21
|
+
test/mock_syslog.rb
|
22
|
+
test/test_applicationlogger.rb
|
23
|
+
test/test_config.rb
|
24
|
+
test/test_jobmanager.rb
|
25
|
+
test/test_system.rb
|
26
|
+
test/test_teestream.rb
|
27
|
+
test/configs/email_settings.rb
|
28
|
+
test/configs/all_values.yaml
|
29
|
+
test/configs/bad_email_condition.yaml
|
30
|
+
test/configs/bad_no_central_log_file.yaml
|
31
|
+
test/configs/bad_number_of_job_logs.yaml
|
32
|
+
test/configs/bad_timeout_zero.yaml
|
33
|
+
test/configs/incomplete_1.yaml
|
34
|
+
test/configs/jobmanager_1.yaml
|
35
|
+
test/configs/jobmanager_2.yaml
|
36
|
+
test/configs/jobmanager_3.yaml
|
37
|
+
test/configs/jobmanager_4_bad_central_log_file.yaml
|
38
|
+
test/configs/jobmanager_4_bad_email_settings_file.yaml
|
39
|
+
test/configs/jobmanager_4_bad_job_logs_directory_1.yaml
|
40
|
+
test/configs/jobmanager_4_bad_job_logs_directory_2.yaml
|
41
|
+
test/configs/minimum_plus_email_settings.yaml
|
42
|
+
test/configs/minimum_required.yaml
|
data/README.txt
ADDED
@@ -0,0 +1,395 @@
|
|
1
|
+
= jobmanager
|
2
|
+
* Project Page: http://rubyforge.org/projects/jobmanager/
|
3
|
+
* Documentation: http://jobmanager.rubyforge.org/
|
4
|
+
|
5
|
+
== DESCRIPTION:
|
6
|
+
|
7
|
+
This package makes managing cron jobs on unix systems a breeze! It is
|
8
|
+
composed of a single program, +jobmanager+, which is intended to be
|
9
|
+
invoked by cron, or a cron-like application. +jobmanager+, in turn,
|
10
|
+
invokes the specified cron job, in addition to providing the following
|
11
|
+
added value:
|
12
|
+
|
13
|
+
* Logs the output of the cron jobs.
|
14
|
+
* Rotates the cron job logs.
|
15
|
+
* Manages a central log which records all cron events (when each job
|
16
|
+
starts, finishes, the job's success, etc).
|
17
|
+
* Emails the user the results of the cron job on a configurable
|
18
|
+
condition (on failure only, on success only, etc.) via sendmail or
|
19
|
+
smtp.
|
20
|
+
* Times out a cron job that runs past the configurable timeout.
|
21
|
+
* Easily scales to handle multiple users' cron jobs.
|
22
|
+
* Both the default behavior and the per-cron job behavior are highly
|
23
|
+
configurable.
|
24
|
+
|
25
|
+
== SYNOPSIS:
|
26
|
+
|
27
|
+
Lets walk through an example invocation of +jobmanager+ via cron.
|
28
|
+
|
29
|
+
Here is an example +crontab+ entry that invokes a simplified database
|
30
|
+
backup script via +jobmanager+ every day at 22:00.
|
31
|
+
00 22 * * * jobmanager --job_name nightly_backup "mysql_backup test"
|
32
|
+
|
33
|
+
Here is the example +mysql_backup+ script referenced above:
|
34
|
+
======<tt>examples/mysql_backup.rb</tt>:
|
35
|
+
#!/usr/bin/env ruby
|
36
|
+
|
37
|
+
print "=================================================\n"
|
38
|
+
print "Backing up Database\n"
|
39
|
+
print "=================================================\n"
|
40
|
+
|
41
|
+
database = ARGV[0]
|
42
|
+
|
43
|
+
command = "mysqldump #{database} > /tmp/#{database}.mysql"
|
44
|
+
print command, "\n"
|
45
|
+
|
46
|
+
if (system("#{command}"))
|
47
|
+
print"\nSuccess!\n"
|
48
|
+
else
|
49
|
+
print "\nFailure!\n"
|
50
|
+
exit(1)
|
51
|
+
end
|
52
|
+
|
53
|
+
|
54
|
+
|
55
|
+
Here is the configuration file to be read by +jobmanager+.
|
56
|
+
======<tt>/etc/jobmanager.yaml</tt>:
|
57
|
+
################################################################################
|
58
|
+
# The jobmanager configuration file.
|
59
|
+
#
|
60
|
+
# For a full description of each of the fields, and their default values,
|
61
|
+
# see the CONFIGURATION FILE section in README.txt.
|
62
|
+
################################################################################
|
63
|
+
|
64
|
+
# The number of log files to be kept per job.
|
65
|
+
number_of_job_logs: 5
|
66
|
+
|
67
|
+
# The directory in which the job logs will be kept, to be interpreted
|
68
|
+
# by ERB. Allowed variables: user_name.
|
69
|
+
job_logs_directory: /tmp/logs
|
70
|
+
|
71
|
+
# jobmanager logs operational information (when jobs are launched,
|
72
|
+
# exit, etc). This can be done via syslog or directly to a log file.
|
73
|
+
# Possible values are: syslog, file.
|
74
|
+
central_log_mode: file
|
75
|
+
|
76
|
+
# If the central_log_mode is set to "file", the log file must be
|
77
|
+
# specified. This parameter will be interpreted by ERB. Allowed
|
78
|
+
# variables: user_name.
|
79
|
+
central_log_file: /tmp/logs/jobmanager.log
|
80
|
+
|
81
|
+
# The path to search for the command that jobmanager is invoked with.
|
82
|
+
#command_path: /usr/bin:/usr/designingpatterns/bin
|
83
|
+
|
84
|
+
# Whether jobmanager should print debug trace to the central log file.
|
85
|
+
debug: false
|
86
|
+
|
87
|
+
# The condition upon which condition results should be emailed.
|
88
|
+
# Possible values are: always, never, on_failure.
|
89
|
+
email_condition: always
|
90
|
+
|
91
|
+
# The configuration file for the simpleemail gem. This parameter will
|
92
|
+
# be interpreted by ERB. Allowed variables: user_name. Note: If a
|
93
|
+
# relative path is specified, it will be considered to be relative to
|
94
|
+
# the location of this configuration file.
|
95
|
+
email_settings_file: email_settings.rb
|
96
|
+
|
97
|
+
# The email subject, to be interpreted by ERB.
|
98
|
+
# Allowed variables: job_name, command, host_name, results, user_name.
|
99
|
+
email_subject: "jobmanager results for <%=job_name%> on <%=host_name%> : <%=result%>"
|
100
|
+
|
101
|
+
# The extension of the rotated job log file. If true, the extension
|
102
|
+
# is a date_time. Otherwise, it is a simple count (.1, .2, etc.).
|
103
|
+
date_time_extension: true
|
104
|
+
|
105
|
+
# If date_time_extension is true, this field will be used as the
|
106
|
+
# format for the date/time extension of the rotated job log file
|
107
|
+
# name.
|
108
|
+
date_time_extension_format: '%F_%T'
|
109
|
+
|
110
|
+
# Whether the rotated job log file should be zipped.
|
111
|
+
zip_rotated_log_file: false
|
112
|
+
|
113
|
+
# The timeout (in seconds). If not present, no timeout will be used.
|
114
|
+
timeout: 1800
|
115
|
+
|
116
|
+
# The sender of emails sent from jobmanager. This can also be
|
117
|
+
# specified in the email_settings_file (attribute default_from).
|
118
|
+
#email_from_address: "Sender <sender@gmail.com>"
|
119
|
+
|
120
|
+
# The receiver of emails sent from jobmanager. This can also be
|
121
|
+
# specified in the email_settings_file (attributes default_to).
|
122
|
+
#email_to_address: "Receiver <receiver@gmail.com>"
|
123
|
+
|
124
|
+
|
125
|
+
Here is the email configuration file referenced in +jobmanager.yaml+
|
126
|
+
above. In this example, we choose to email via smtp, although
|
127
|
+
sendmail is also an option. This file is documented further in the
|
128
|
+
{+simpleemail+}[http://simpleemail.rubyforge.org] gem.
|
129
|
+
======<tt>/etc/email_settings.rb</tt>:
|
130
|
+
ActionMailer::Base.delivery_method = :smtp
|
131
|
+
|
132
|
+
ActionMailer::Base.smtp_settings = {
|
133
|
+
:address => 'smtp.gmail.com',
|
134
|
+
:port => 587,
|
135
|
+
:domain => "gmail.com",
|
136
|
+
:authentication => "login",
|
137
|
+
:user_name => "sender_address@gmail.com",
|
138
|
+
:password => "password",
|
139
|
+
:tls => :auto
|
140
|
+
}
|
141
|
+
|
142
|
+
SimpleEmail::Email.default_from = "Sender Address <sender_address@gmail.com>"
|
143
|
+
SimpleEmail::Email.default_to = "Receiver Address <receiver_address@gmail.com>"
|
144
|
+
|
145
|
+
|
146
|
+
Here is the central log file that +jobmanager+ updated.
|
147
|
+
======<tt>/tmp/logs/jobmanager.log</tt>:
|
148
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Command (/usr/bin/mysql_backup test) launched, pid = 2302.
|
149
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Exited successfully
|
150
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Job log rotated to /tmp/logs/nightly_backup.log.2008-07-30_22:00:00.
|
151
|
+
|
152
|
+
|
153
|
+
Here is the rotated job log file.
|
154
|
+
======<tt>/tmp/logs/nightly_backup.log.2008-07-30_22:00:00</tt>:
|
155
|
+
=================================================
|
156
|
+
Backing up Database
|
157
|
+
=================================================
|
158
|
+
mysqldump test > /tmp/test.mysql
|
159
|
+
|
160
|
+
Success!
|
161
|
+
|
162
|
+
Here is the subject and body of the email that was sent upon job completion.
|
163
|
+
|
164
|
+
======<tt>Email Subject</tt>:
|
165
|
+
jobmanager results for nightly_backup on jackfruit : Success
|
166
|
+
|
167
|
+
======<tt>Email Body</tt>:
|
168
|
+
---------------------------------------------------------
|
169
|
+
jobmanager Output:
|
170
|
+
---------------------------------------------------------
|
171
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Command (/usr/bin/mysql_backup test) launched, pid = 2302.
|
172
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Exited successfully
|
173
|
+
I, [2008-07-30T22:00:00 #2301] INFO -- : [janet, nightly_backup] Job log rotated to /tmp/logs/nightly_backup.log.2008-08-01_11:29:47.
|
174
|
+
|
175
|
+
---------------------------------------------------------
|
176
|
+
Job Output:
|
177
|
+
---------------------------------------------------------
|
178
|
+
=================================================
|
179
|
+
Backing up Database
|
180
|
+
=================================================
|
181
|
+
mysqldump test > /tmp/test.mysql
|
182
|
+
|
183
|
+
Success!
|
184
|
+
|
185
|
+
|
186
|
+
|
187
|
+
== CONFIGURATION FILE:
|
188
|
+
|
189
|
+
A configuration file is required for +jobmanager+ to run. By default,
|
190
|
+
+jobmanager+ will expect the configuration file to be located at
|
191
|
+
+/etc/jobmanager.yaml+. If the environment variable
|
192
|
+
+JOBMANAGER_CONFIG_FILE+ is set, +jobmanager+ will instead read the
|
193
|
+
configuration file specified by this variable.
|
194
|
+
|
195
|
+
The configuration file provides the following parameters:
|
196
|
+
|
197
|
+
[+job_logs_directory+: required]
|
198
|
+
The directory in which the job log files will be kept. The
|
199
|
+
+job_logs_directory+ parameter will be interpreted by ERB. The
|
200
|
+
allowed variables are: +user_name+. Specifying the +user_name+
|
201
|
+
variable in the +job_logs_directory+ value can be useful if you
|
202
|
+
want multiple users to share the same configuration file, but have
|
203
|
+
different job logs directories.
|
204
|
+
|
205
|
+
[+central_log_mode+: optional, default: +syslog+]
|
206
|
+
+jobmanager+ logs operational information (when jobs are
|
207
|
+
launched, exit, etc). This can be done via +syslog+ or directly to a
|
208
|
+
log file. Possible values are: <tt>syslog, file</tt>.
|
209
|
+
|
210
|
+
[+central_log_file+: optional]
|
211
|
+
If +central_log_mode+ is set to +file+, this parameter must be set.
|
212
|
+
The +central_log_file+ parameter will be interpreted by ERB. The
|
213
|
+
allowed variables are: +user_name+. Specifying the +user_name+
|
214
|
+
variable in the +central_log_file+ value can be useful if you want
|
215
|
+
multiple users to share the same configuration file, but have
|
216
|
+
different central log files.
|
217
|
+
|
218
|
+
[+number_of_job_logs+: optional, default: +3+]
|
219
|
+
The number of logs to be kept per job.
|
220
|
+
|
221
|
+
[+command_path+: optional, default: <tt>ENV['PATH']</tt> ]
|
222
|
+
The path to be searched for the command that +jobmanager+ is
|
223
|
+
invoked with. If unspecified, the environment path will be used.
|
224
|
+
|
225
|
+
[+debug+: optional, default: +false+]
|
226
|
+
Whether +jobmanager+ should print debug trace to the central log file.
|
227
|
+
|
228
|
+
[+email_condition+: optional, default: +on_failure+]
|
229
|
+
The condition upon which results should be emailed. Possible
|
230
|
+
values are: <tt>always, never, on_failure</tt>.
|
231
|
+
|
232
|
+
[+email_settings_file+: optional, default: +email_settings.rb+]
|
233
|
+
The configuration file for the
|
234
|
+
{+simpleemail+}[http://simpleemail.rubyforge.org] gem. This
|
235
|
+
specifies the delivery method (+smtp+, +sendmail+), as well as
|
236
|
+
delivery method specific settings. Default to/from emails can be
|
237
|
+
specified in here. If the path specified is relative, it is assumed
|
238
|
+
that it is relative to the directory in which the +jobmanager+
|
239
|
+
configuration file is located. If +email_condition+ is set to a
|
240
|
+
value other than +never+, this parameter must specified. The
|
241
|
+
+email_settings_file+ parameter will be interpreted by ERB. The
|
242
|
+
allowed variables are: +user_name+. Specifying the +user_name+
|
243
|
+
variable in the +email_settings_file+ value can be useful if you
|
244
|
+
want multiple users to share the same configuration file, but have
|
245
|
+
separate email settings files.
|
246
|
+
|
247
|
+
|
248
|
+
[+email_subject+, optional, default: <tt>"jobmanager results for <%=job_name%> on <%=host_name%> : <%=result%>"</tt>]
|
249
|
+
The email subject, to be interpreted by ERB. Allowed variables
|
250
|
+
inside the string are: <tt>job_name, command, host_name, results, user_name</tt>.
|
251
|
+
|
252
|
+
[+date_time_extension+, optional, default: +true+]
|
253
|
+
This refers to the extension of the rotated job log file. If
|
254
|
+
+date_time_extension+ is present and set to +true+, the extension
|
255
|
+
represents a date and time. Otherwise, it is a simple count (.1, .2, etc.).
|
256
|
+
|
257
|
+
[+date_time_extension_format+, optional, default: <tt>'%F_%T'</tt>]
|
258
|
+
If +date_time_extension+ is +true+, this parameter will be used as the
|
259
|
+
format for the date and time extension of the rotated job log file.
|
260
|
+
|
261
|
+
[+zip_rotated_log_file+, optional, default: +false+]
|
262
|
+
Whether the rotated job log file should be gzip'ed.
|
263
|
+
|
264
|
+
[+timeout+, optional]
|
265
|
+
The timeout (in seconds). If the job spawned by +jobmanager+ does not
|
266
|
+
exit after +timeout+ seconds, +jobmanager+ will kill the process, and
|
267
|
+
report failure. If +timeout+ is unspecifed, +jobmanager+ will let the
|
268
|
+
process run for an unlimited amount of time (+jobmanager+ will simply
|
269
|
+
wait til it returns).
|
270
|
+
|
271
|
+
[+email_from_address+, optional]
|
272
|
+
The sender of emails sent from +jobmanager+. This can also be
|
273
|
+
specified in the +email_settings_file+ (attribute +default_from+).
|
274
|
+
|
275
|
+
[+email_to_address+, optional]
|
276
|
+
The receiver of emails sent from +jobmanager+. This can also be
|
277
|
+
specified in the +email_settings_file+ (attribute +default_to+).
|
278
|
+
|
279
|
+
== COMMAND LINE ARGUMENTS:
|
280
|
+
|
281
|
+
Usage:
|
282
|
+
Usage: jobmanager.rb [options] <command>
|
283
|
+
|
284
|
+
Notable command line options:
|
285
|
+
[<tt>-j, --job_name</tt>]
|
286
|
+
The name of the job to be run. If this is unspecified, the
|
287
|
+
job name will be set to the basename of the COMMAND specified.
|
288
|
+
The job name is used when logging to the central log file, and
|
289
|
+
in the name of the job log file.
|
290
|
+
|
291
|
+
The remaining command line options are duplicates of the configuration
|
292
|
+
file parameters. In fact, all of the configuration parameters
|
293
|
+
specified in the configuration file can be overriden via +jobmanager+
|
294
|
+
command line options.
|
295
|
+
|
296
|
+
== STARTING JOBMANAGER FROM THE COMMAND LINE VS. CRON:
|
297
|
+
|
298
|
+
The most common use case for +jobmanager+ is to be run from cron, or a
|
299
|
+
cron-like application. However, invoking +jobmanager+ from the
|
300
|
+
command line can be useful for testing, debugging, and one-off runs.
|
301
|
+
As +jobmanager+ is commonly invoked via the cron, +jobmanager+ logs
|
302
|
+
all output to either +syslog+ or a central log file (specified by the
|
303
|
+
configuration parameters +central_log_mode+, +central_log_file+).
|
304
|
+
When +jobmanager+ is run from the command line (or any script whose
|
305
|
+
STDIN, STDOUT, or STDERR is connected to a terminal), these two
|
306
|
+
configuration parameters will be ignored and +jobmanager+'s output
|
307
|
+
will be sent to the terminal.
|
308
|
+
|
309
|
+
== REQUIREMENTS:
|
310
|
+
|
311
|
+
The +jobmanager+ application requires a number of gems: hoe,
|
312
|
+
SyslogLogger, sys-host, configtoolkit, simpleemail, logrotate,
|
313
|
+
assertions, and relative.
|
314
|
+
|
315
|
+
Hoe and assertions are required only for running the tests. Relative is
|
316
|
+
required for running the tests and the examples. The rest of the gems
|
317
|
+
listed above are required for the proper functioning of the
|
318
|
+
+jobmanager+ application. The sys-host gem is required only for the
|
319
|
+
purpose of including the hostname in the email subject line.
|
320
|
+
|
321
|
+
== INSTALL:
|
322
|
+
|
323
|
+
1. <tt>sudo gem install jobmanager</tt>
|
324
|
+
|
325
|
+
2. Set up the +jobmanager+ configuration file (+jobmanager.yaml+).
|
326
|
+
For more detailed instructions, see the CONFIGURATION FILE section
|
327
|
+
above.
|
328
|
+
|
329
|
+
3. Set up the simpleemail configuration file (+email_settings.rb+).
|
330
|
+
Alternately, if you want to disable email, set the configuration
|
331
|
+
parameter +email_condition+ to +never+ in +jobmanager.yaml+. See
|
332
|
+
the {+simpleemail+}[http://simpleemail.rubyforge.org] gem rdoc for
|
333
|
+
more details.
|
334
|
+
|
335
|
+
4. Confirm that the configuration files are valid by invoking
|
336
|
+
+jobmanager+ from the command line with one of your cron jobs or a
|
337
|
+
test command.
|
338
|
+
|
339
|
+
5. Edit your +crontab+ to run new and/or existing cron jobs through
|
340
|
+
+jobmanager+.
|
341
|
+
|
342
|
+
For possible reasons why running +jobmanager+ via the command line may
|
343
|
+
be successful, but running +jobmanager+ with the same configuration
|
344
|
+
via the cron may fail, consult FAQ.txt.
|
345
|
+
|
346
|
+
== PROBLEMS:
|
347
|
+
|
348
|
+
None (known).
|
349
|
+
|
350
|
+
== AUTHORS:
|
351
|
+
=== Designing Patterns
|
352
|
+
* Homepage: http://www.designingpatterns.com
|
353
|
+
* Blogs: http://blogs.designingpatterns.com
|
354
|
+
|
355
|
+
== SUPPORT:
|
356
|
+
Please post questions, concerns, or requests for enhancement to the forums on
|
357
|
+
the project page. Alternatively, direct contact information for
|
358
|
+
Designing Patterns can be found on the project page for this gem.
|
359
|
+
|
360
|
+
== ENHANCEMENTS:
|
361
|
+
Please feel free to contact us with any ideas; we will try our best to
|
362
|
+
enhance the software and respond to user requests. Of course, we are more
|
363
|
+
likely to work on a particular enhancement if we know that there are users
|
364
|
+
who want it. Designing Patterns provides contracting and consulting services,
|
365
|
+
so if there is an enhancement that *must* get done (and in a specified time
|
366
|
+
frame), please inquire about retaining our services!
|
367
|
+
|
368
|
+
== LICENSE:
|
369
|
+
The license text can be found in the +LICENSE+ file at the root of the
|
370
|
+
distribution.
|
371
|
+
|
372
|
+
This package is licensed with an MIT license:
|
373
|
+
|
374
|
+
Copyright (c) 2008 Designing Patterns
|
375
|
+
|
376
|
+
Permission is hereby granted, free of charge, to any person obtaining
|
377
|
+
a copy of this software and associated documentation files (the
|
378
|
+
'Software'), to deal in the Software without restriction, including
|
379
|
+
without limitation the rights to use, copy, modify, merge, publish,
|
380
|
+
distribute, sublicense, and/or sell copies of the Software, and to
|
381
|
+
permit persons to whom the Software is furnished to do so, subject to
|
382
|
+
the following conditions:
|
383
|
+
|
384
|
+
The above copyright notice and this permission notice shall be
|
385
|
+
included in all copies or substantial portions of the Software.
|
386
|
+
|
387
|
+
THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
|
388
|
+
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
|
389
|
+
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
|
390
|
+
IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
|
391
|
+
CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
|
392
|
+
TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
|
393
|
+
SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
|
394
|
+
|
395
|
+
== SHARE AND ENJOY!
|