bigwig 0.2 → 0.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/CHANGELOG +2 -0
- data/Manifest +2 -0
- data/README.markdown +24 -3
- data/Rakefile +1 -1
- data/bigwig.gemspec +8 -8
- data/bigwig.yml +1 -1
- data/bin/bigwig-ping +4 -1
- data/bin/bigwig-push +24 -0
- data/lib/bigwig/pinger.rb +3 -52
- data/lib/bigwig/push.rb +97 -0
- metadata +8 -3
data/CHANGELOG
CHANGED
data/Manifest
CHANGED
@@ -7,10 +7,12 @@ bigwig.yml
|
|
7
7
|
bigwig.yml.example
|
8
8
|
bin/bigwig
|
9
9
|
bin/bigwig-ping
|
10
|
+
bin/bigwig-push
|
10
11
|
lib/bigwig.rb
|
11
12
|
lib/bigwig/internal_plugins/ping_plugin.rb
|
12
13
|
lib/bigwig/job.rb
|
13
14
|
lib/bigwig/pinger.rb
|
14
15
|
lib/bigwig/plugin.rb
|
15
16
|
lib/bigwig/plugins.rb
|
17
|
+
lib/bigwig/push.rb
|
16
18
|
lib/bigwig/runner.rb
|
data/README.markdown
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
Bigwig
|
2
2
|
======
|
3
3
|
|
4
|
-
Bigwig is a daemon process that listens to a RabbitMQ queue and processes messages that it receives.
|
4
|
+
Bigwig is a daemon process that listens to a RabbitMQ queue and processes messages that it receives, along with some simple command-line tools to place messages on the queue.
|
5
5
|
|
6
6
|
Typical Usage
|
7
7
|
-------------
|
@@ -10,7 +10,9 @@ Typical Usage
|
|
10
10
|
* Start an AMQP server
|
11
11
|
* Create a folder of plugins
|
12
12
|
* Create a bigwig configuration file
|
13
|
-
* Start bigwig - `bigwig start -c config -l
|
13
|
+
* Start bigwig - `bigwig start -c config-file -l log-folder`
|
14
|
+
* Ping bigwig - `bigwig-ping -c config-file`
|
15
|
+
* Push a message with data onto the queue - `bigwig-push -m my_message -d '{:key => "value"}' -c config-file`
|
14
16
|
|
15
17
|
Configuration
|
16
18
|
-------------
|
@@ -30,7 +32,7 @@ A typical configuration file looks like this:
|
|
30
32
|
warren_logging: false
|
31
33
|
plugins_folder: /full/path/to/a/folder
|
32
34
|
|
33
|
-
The first six items tell bigwig how to connect to the AMQP server. The next item specifies whether you want warren (the lower-level AMQP processor) to log its output. Lastly, you tell bigwig where to find its plugins.
|
35
|
+
The first six items tell bigwig how to connect to the AMQP server. The next item specifies whether you want warren (the lower-level AMQP processor) to log its output (normally you will want this to be false, as warren can be quite noisy). Lastly, you tell bigwig where to find its plugins.
|
34
36
|
|
35
37
|
Logging
|
36
38
|
-------
|
@@ -79,11 +81,30 @@ For example, a plugin, which we shall call LoggingPlugin, would look like this:
|
|
79
81
|
|
80
82
|
When bigwig receives a message where the `:method` parameter == 'logging' it will then invoke `LoggingPlugin#call`, passing it the `:id` and `:data` from the original message as task_id and args respectively.
|
81
83
|
|
84
|
+
If your plugin needs to know where it is on the filesystem (for example, so that it can shell out to run a script) it can access `BigWig::Plugins.root` which returns the folder reference you specified within the configuration file.
|
85
|
+
|
82
86
|
Pings
|
83
87
|
-----
|
84
88
|
|
85
89
|
There is a plugin built-in to bigwig called PingPlugin, that registers itself under the name "ping". If you place a message onto the queue with :method => 'ping', the PingPlugin responds by writing a message to the log file. This is useful for monitoring bigwig - another system places ping messages onto the queue at regular intervals and we watch to ensure that the log file's update time is changing.
|
86
90
|
|
91
|
+
Command-line Interface
|
92
|
+
----------------------
|
93
|
+
|
94
|
+
There are two command-line scripts that push messages on to the queue.
|
95
|
+
|
96
|
+
The simplest is `bigwig-ping`.
|
97
|
+
|
98
|
+
This pushes a ping message onto the queue - ideal for calling from a cron job. This takes two parameters: `-t` (or `--timeout`) which is the connection timeout in seconds (defaulting to 5 if not supplied) and `-c` (or `--config`) which is the path to a Bigwig configuration file (defaulting to bigwig.yml in the current directory if not supplied).
|
99
|
+
|
100
|
+
bigwig-ping -c /path/to/config -t 10
|
101
|
+
|
102
|
+
There is also `bigwig-push`.
|
103
|
+
|
104
|
+
This pushes an arbitrary message onto the queue - ideal for testing, or just manually poking Bigwig to make something happen. It takes the same `--config` and `--timeout` parameters as bigwig-ping, plus a few more for specifying the message itself. These are `-m` (or `--method`) which is the method name (for selecting which plugin responds), `-i` (or `--id`) which is the optional task id (useful when linking to an external system) and `-d` (or `--data`) for a hash of values that becomes the `:data` parameter.
|
105
|
+
|
106
|
+
bigwig-push -m my_message -d '{:key => "value", :key2 => "something else"}' -i 123 -c /path/to/config -t 10
|
107
|
+
|
87
108
|
License
|
88
109
|
-------
|
89
110
|
|
data/Rakefile
CHANGED
@@ -3,6 +3,6 @@ Echoe.new 'bigwig' do | gem |
|
|
3
3
|
gem.author = 'David Smalley, Caius Durling, Rahoul Baruah'
|
4
4
|
gem.summary = 'A daemon that listens to an AMQP queue and responds to messages by invoking commands from a set of plugins'
|
5
5
|
gem.url = 'http://www.brightbox.co.uk/'
|
6
|
-
gem.runtime_dependencies = ["
|
6
|
+
gem.runtime_dependencies = ["warren >=0.9.0", "daemons >=1.0.10"]
|
7
7
|
gem.install_message = 'Welcome to bigwig. Please set up a configuration file and a plugins folder before starting bigwig...'
|
8
8
|
end
|
data/bigwig.gemspec
CHANGED
@@ -2,16 +2,16 @@
|
|
2
2
|
|
3
3
|
Gem::Specification.new do |s|
|
4
4
|
s.name = %q{bigwig}
|
5
|
-
s.version = "0.
|
5
|
+
s.version = "0.3"
|
6
6
|
|
7
7
|
s.required_rubygems_version = Gem::Requirement.new(">= 1.2") if s.respond_to? :required_rubygems_version=
|
8
8
|
s.authors = ["David Smalley, Caius Durling, Rahoul Baruah"]
|
9
|
-
s.date = %q{2009-
|
9
|
+
s.date = %q{2009-11-05}
|
10
10
|
s.description = %q{A daemon that listens to an AMQP queue and responds to messages by invoking commands from a set of plugins}
|
11
11
|
s.email = %q{}
|
12
|
-
s.executables = ["bigwig", "bigwig-ping"]
|
13
|
-
s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README.markdown", "bin/bigwig", "bin/bigwig-ping", "lib/bigwig.rb", "lib/bigwig/internal_plugins/ping_plugin.rb", "lib/bigwig/job.rb", "lib/bigwig/pinger.rb", "lib/bigwig/plugin.rb", "lib/bigwig/plugins.rb", "lib/bigwig/runner.rb"]
|
14
|
-
s.files = ["CHANGELOG", "LICENSE", "Manifest", "README.markdown", "Rakefile", "bigwig.yml", "bigwig.yml.example", "bin/bigwig", "bin/bigwig-ping", "lib/bigwig.rb", "lib/bigwig/internal_plugins/ping_plugin.rb", "lib/bigwig/job.rb", "lib/bigwig/pinger.rb", "lib/bigwig/plugin.rb", "lib/bigwig/plugins.rb", "lib/bigwig/runner.rb", "bigwig.gemspec"]
|
12
|
+
s.executables = ["bigwig", "bigwig-ping", "bigwig-push"]
|
13
|
+
s.extra_rdoc_files = ["CHANGELOG", "LICENSE", "README.markdown", "bin/bigwig", "bin/bigwig-ping", "bin/bigwig-push", "lib/bigwig.rb", "lib/bigwig/internal_plugins/ping_plugin.rb", "lib/bigwig/job.rb", "lib/bigwig/pinger.rb", "lib/bigwig/plugin.rb", "lib/bigwig/plugins.rb", "lib/bigwig/push.rb", "lib/bigwig/runner.rb"]
|
14
|
+
s.files = ["CHANGELOG", "LICENSE", "Manifest", "README.markdown", "Rakefile", "bigwig.yml", "bigwig.yml.example", "bin/bigwig", "bin/bigwig-ping", "bin/bigwig-push", "lib/bigwig.rb", "lib/bigwig/internal_plugins/ping_plugin.rb", "lib/bigwig/job.rb", "lib/bigwig/pinger.rb", "lib/bigwig/plugin.rb", "lib/bigwig/plugins.rb", "lib/bigwig/push.rb", "lib/bigwig/runner.rb", "bigwig.gemspec"]
|
15
15
|
s.homepage = %q{http://www.brightbox.co.uk/}
|
16
16
|
s.post_install_message = %q{Welcome to bigwig. Please set up a configuration file and a plugins folder before starting bigwig...}
|
17
17
|
s.rdoc_options = ["--line-numbers", "--inline-source", "--title", "Bigwig", "--main", "README.markdown"]
|
@@ -25,14 +25,14 @@ Gem::Specification.new do |s|
|
|
25
25
|
s.specification_version = 3
|
26
26
|
|
27
27
|
if Gem::Version.new(Gem::RubyGemsVersion) >= Gem::Version.new('1.2.0') then
|
28
|
-
s.add_runtime_dependency(%q<
|
28
|
+
s.add_runtime_dependency(%q<warren>, [">= 0.9.0"])
|
29
29
|
s.add_runtime_dependency(%q<daemons>, [">= 1.0.10"])
|
30
30
|
else
|
31
|
-
s.add_dependency(%q<
|
31
|
+
s.add_dependency(%q<warren>, [">= 0.9.0"])
|
32
32
|
s.add_dependency(%q<daemons>, [">= 1.0.10"])
|
33
33
|
end
|
34
34
|
else
|
35
|
-
s.add_dependency(%q<
|
35
|
+
s.add_dependency(%q<warren>, [">= 0.9.0"])
|
36
36
|
s.add_dependency(%q<daemons>, [">= 1.0.10"])
|
37
37
|
end
|
38
38
|
end
|
data/bigwig.yml
CHANGED
data/bin/bigwig-ping
CHANGED
data/bin/bigwig-push
ADDED
@@ -0,0 +1,24 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
require File.join(File.dirname(__FILE__), "..", "lib", "bigwig")
|
3
|
+
require 'optparse'
|
4
|
+
|
5
|
+
options = BigWig::Push.load_options_from ARGV do | parser, options |
|
6
|
+
|
7
|
+
parser.on '-m', '--method METHOD', 'Specify the method name' do | value |
|
8
|
+
options[:method] = value
|
9
|
+
end
|
10
|
+
|
11
|
+
parser.on '-i', '--id TASK_ID', 'Specify the task id' do | value |
|
12
|
+
options[:id] = value
|
13
|
+
end
|
14
|
+
|
15
|
+
parser.on '-d', '--data HASH', 'Specify the message data as a Hash "{:key => \'value\', :key2 => \'other_value\'}"' do | value |
|
16
|
+
options[:data] = eval(value)
|
17
|
+
end
|
18
|
+
|
19
|
+
parser.on '-q', '--queue QUEUE_NAME', 'Specify which queue to put the message onto' do | value |
|
20
|
+
options[:queue] = value
|
21
|
+
end
|
22
|
+
end
|
23
|
+
|
24
|
+
BigWig::Push.new(options).message options[:method], options[:data] || {}, options[:id], options[:queue] || :default
|
data/lib/bigwig/pinger.rb
CHANGED
@@ -1,60 +1,11 @@
|
|
1
|
-
require 'warren'
|
2
|
-
require 'warren/adapters/bunny_adapter'
|
3
|
-
require 'daemons'
|
4
|
-
require 'optparse'
|
5
|
-
require 'ostruct'
|
6
|
-
require 'benchmark'
|
7
|
-
require 'timeout'
|
8
|
-
|
9
1
|
module BigWig
|
10
2
|
class Pinger
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
def initialize(command_line)
|
15
|
-
load_options_from command_line
|
3
|
+
def initialize command_line
|
4
|
+
@push = Push.new command_line
|
16
5
|
end
|
17
6
|
|
18
7
|
def run
|
19
|
-
|
20
|
-
@config = YAML.load(File.open(@options.config))
|
21
|
-
BigWig::connect_using @config
|
22
|
-
|
23
|
-
begin
|
24
|
-
Timeout::timeout(@options.timeout) do
|
25
|
-
begin
|
26
|
-
Warren::Queue.publish :default, :method => 'ping'
|
27
|
-
rescue Exception => ex
|
28
|
-
BigWig::logger.error("...ping failed with #{ex}")
|
29
|
-
exit 2
|
30
|
-
end
|
31
|
-
end
|
32
|
-
rescue Timeout::Error => te
|
33
|
-
BigWig::logger.error("...ping timed out: #{te}")
|
34
|
-
exit 1
|
35
|
-
end
|
36
|
-
|
37
|
-
BigWig::logger.info("...bigwig pinger finished")
|
38
|
-
exit 0
|
39
|
-
end
|
40
|
-
|
41
|
-
protected
|
42
|
-
def load_options_from command_line
|
43
|
-
@options = OpenStruct.new(:config => File.expand_path('./bigwig.yml'), :timeout => 5)
|
44
|
-
opts = OptionParser.new do |opts|
|
45
|
-
opts.banner = "Usage: #{File.basename($0)} [options]"
|
46
|
-
opts.on('-h', '--help', 'Show this message') do
|
47
|
-
puts opts
|
48
|
-
exit 1
|
49
|
-
end
|
50
|
-
opts.on('-c', '--config=file', 'Path to the config file; if not supplied then bigwig looks for bigwig.yml in the current folder') do |conf|
|
51
|
-
@options.config = conf
|
52
|
-
end
|
53
|
-
opts.on('-t', '--timeout=value', 'Timeout value in seconds') do | conf |
|
54
|
-
@options.timeout = conf.to_i
|
55
|
-
end
|
56
|
-
end
|
57
|
-
@args = opts.parse!(command_line)
|
8
|
+
@push.message 'ping'
|
58
9
|
end
|
59
10
|
end
|
60
11
|
end
|
data/lib/bigwig/push.rb
ADDED
@@ -0,0 +1,97 @@
|
|
1
|
+
require 'warren'
|
2
|
+
require 'warren/adapters/bunny_adapter'
|
3
|
+
require 'daemons'
|
4
|
+
require 'optparse'
|
5
|
+
require 'benchmark'
|
6
|
+
require 'timeout'
|
7
|
+
|
8
|
+
module BigWig
|
9
|
+
# pushes a message onto the queue
|
10
|
+
# USAGE:
|
11
|
+
# push = BigWig::Push.new :config => '/path/to/config/file', :timeout => 5
|
12
|
+
# By default, it uses bigwig.yml in the current directory and a timeout of 5 seconds
|
13
|
+
# The configuration file is expected to have the same structure as the BigWig daemon's file (without the plugins folder)
|
14
|
+
class Push
|
15
|
+
attr_reader :config
|
16
|
+
attr_reader :options
|
17
|
+
|
18
|
+
# Create a new BigWig::Push
|
19
|
+
# Options:
|
20
|
+
# :config => '/path/to/config/file'
|
21
|
+
# :timeout => 5
|
22
|
+
#
|
23
|
+
def initialize(overrides = {})
|
24
|
+
@options = {
|
25
|
+
:config => File.expand_path('./bigwig.yml'),
|
26
|
+
:timeout => 5
|
27
|
+
}.merge(overrides)
|
28
|
+
end
|
29
|
+
|
30
|
+
# Push a message on to the queue, passing in a message name, plus an optional data hash, an optional task id and an optional queue name
|
31
|
+
#
|
32
|
+
def message method, data = {}, task_id = nil, queue = :default
|
33
|
+
BigWig::logger.info("Pushing #{method} on to #{queue}")
|
34
|
+
@config = YAML.load(File.open(@options[:config]))
|
35
|
+
BigWig::connect_using @config
|
36
|
+
|
37
|
+
begin
|
38
|
+
Timeout::timeout(@options[:timeout]) do
|
39
|
+
begin
|
40
|
+
Warren::Queue.publish queue, :method => method, :id => task_id, :data => data
|
41
|
+
rescue Exception => ex
|
42
|
+
BigWig::logger.error("...push failed with #{ex}")
|
43
|
+
exit 1
|
44
|
+
end
|
45
|
+
end
|
46
|
+
rescue Timeout::Error => te
|
47
|
+
BigWig::logger.error("...push timed out: #{te}")
|
48
|
+
exit 1
|
49
|
+
end
|
50
|
+
|
51
|
+
BigWig::logger.info("...bigwig push finished")
|
52
|
+
exit 0
|
53
|
+
end
|
54
|
+
|
55
|
+
# Helper method so that calling scripts don't need to parse the Push parameters themselves
|
56
|
+
#
|
57
|
+
# USAGE:
|
58
|
+
# options = BigWig::Push.load_options_from ARGV
|
59
|
+
# BigWig::Push.new(options).message 'whatever'
|
60
|
+
#
|
61
|
+
# If this is being called from a wrapper script (such as bin/bigwig-push) that needs its own parameters you can pass a block to append your own details
|
62
|
+
#
|
63
|
+
# options = BigWig::Push.load_options_from(ARGV) do | parser, options |
|
64
|
+
# parser.on('--xxx', 'Push it good') do
|
65
|
+
# options[:push_it_good] = :push_it_real_good
|
66
|
+
# end
|
67
|
+
# end
|
68
|
+
#
|
69
|
+
def self.load_options_from command_line
|
70
|
+
options = {}
|
71
|
+
|
72
|
+
parser = OptionParser.new do | parser |
|
73
|
+
parser.banner = "Usage: #{File.basename($0)} [options]"
|
74
|
+
|
75
|
+
parser.on('-h', '--help', 'Show this message') do
|
76
|
+
puts parser
|
77
|
+
exit 2
|
78
|
+
end
|
79
|
+
|
80
|
+
parser.on('-c', '--config=file', 'Path to the config file; if not supplied then bigwig looks for bigwig.yml in the current folder') do |conf|
|
81
|
+
options[:config] = conf
|
82
|
+
end
|
83
|
+
|
84
|
+
parser.on('-t', '--timeout=value', 'Timeout value in seconds') do | conf |
|
85
|
+
options[:timeout] = conf.to_i
|
86
|
+
end
|
87
|
+
|
88
|
+
yield(parser, options) if block_given?
|
89
|
+
|
90
|
+
end
|
91
|
+
parser.parse!(command_line)
|
92
|
+
|
93
|
+
return options
|
94
|
+
end
|
95
|
+
|
96
|
+
end
|
97
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: bigwig
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: "0.
|
4
|
+
version: "0.3"
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- David Smalley, Caius Durling, Rahoul Baruah
|
@@ -9,11 +9,11 @@ autorequire:
|
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
11
|
|
12
|
-
date: 2009-
|
12
|
+
date: 2009-11-05 00:00:00 +00:00
|
13
13
|
default_executable:
|
14
14
|
dependencies:
|
15
15
|
- !ruby/object:Gem::Dependency
|
16
|
-
name:
|
16
|
+
name: warren
|
17
17
|
type: :runtime
|
18
18
|
version_requirement:
|
19
19
|
version_requirements: !ruby/object:Gem::Requirement
|
@@ -37,6 +37,7 @@ email: ""
|
|
37
37
|
executables:
|
38
38
|
- bigwig
|
39
39
|
- bigwig-ping
|
40
|
+
- bigwig-push
|
40
41
|
extensions: []
|
41
42
|
|
42
43
|
extra_rdoc_files:
|
@@ -45,12 +46,14 @@ extra_rdoc_files:
|
|
45
46
|
- README.markdown
|
46
47
|
- bin/bigwig
|
47
48
|
- bin/bigwig-ping
|
49
|
+
- bin/bigwig-push
|
48
50
|
- lib/bigwig.rb
|
49
51
|
- lib/bigwig/internal_plugins/ping_plugin.rb
|
50
52
|
- lib/bigwig/job.rb
|
51
53
|
- lib/bigwig/pinger.rb
|
52
54
|
- lib/bigwig/plugin.rb
|
53
55
|
- lib/bigwig/plugins.rb
|
56
|
+
- lib/bigwig/push.rb
|
54
57
|
- lib/bigwig/runner.rb
|
55
58
|
files:
|
56
59
|
- CHANGELOG
|
@@ -62,12 +65,14 @@ files:
|
|
62
65
|
- bigwig.yml.example
|
63
66
|
- bin/bigwig
|
64
67
|
- bin/bigwig-ping
|
68
|
+
- bin/bigwig-push
|
65
69
|
- lib/bigwig.rb
|
66
70
|
- lib/bigwig/internal_plugins/ping_plugin.rb
|
67
71
|
- lib/bigwig/job.rb
|
68
72
|
- lib/bigwig/pinger.rb
|
69
73
|
- lib/bigwig/plugin.rb
|
70
74
|
- lib/bigwig/plugins.rb
|
75
|
+
- lib/bigwig/push.rb
|
71
76
|
- lib/bigwig/runner.rb
|
72
77
|
- bigwig.gemspec
|
73
78
|
has_rdoc: true
|