bunnicula 0.1.1 → 0.2.0

data/README.rdoc

@@ -0,0 +1,119 @@
+== Overview
+Bunnicula is a simple AMQP relay implemented as a ruby daemon (a-la {daemon-kit}[http://github.com/kennethkalmer/daemon-kit]). Similar in intent to {shovel}[http://www.lshift.net/blog/2010/02/01/rabbitmq-shovel-message-relocation-equipment],
+Bunnicula is intended to enable the common messaging scenario where services and applications publish messages to an AMQP broker on the local LAN for speed and reliability which are then subsequently relayed to a remote AMQP instance by a relay process which isn't so irritable as message producers tend to be when it comes to network speed and reliability.
+
+Bunnicula can be configured via configuration file (a Ruby DSL) or, for most common configurations, through command line arguments.
+
+== Install
+
+Install the gem with:
+
+    gem install bunnicula
+          or
+    sudo gem install bunnicula
+
+== Quick Start
+relay messages published to the direct exchange 'my.exchange' on localhost to the direct exchange 'my.exchange' on my.remote.broker
+
+    bunnicula start --target my.remote.broker --relay my.exchange
+
+
+== Motivation
+While LShift maintains a relay called "Shovel" that is tightly integrated with RabbitMQ, the setup and configuration of that plugin required building specific versions of the broker and other acrobatic feats that proved too much for my feeble brain (and appetite for deployment complexity).
+
+== Approach
+Bunnicula uses the AMQP gem (which in turn runs on Event Machine) to create a (optionally durable) queue bound to each configured message exchange. When it receives a message on this exchange, it uses Bunny, a synchronous AMQP library, to immediately publish that message to a target AMQP broker and (optionally) ack the original message once this is successful.
+By default, Bunnicula will re-use the configuration details for the source exchange (exchange name, durability and exchange type) but can be configured to publish to alternate exchanges as desired.
+
+
+== Suck some Blood(y messages) already
+As a daemon-kit daemon, bunnicula understands the following commands
+* run - run as a regular script, i.e. do not daemonize
+* start - start the daemon
+* stop - stop the daemon
+
+Note: when specifying amqp brokers, either on the command line or from the DSL, Bunnicula uses a URI as a connection string.
+So, amqp://usr:pwd@my.broker/my_vhost:12345 will result in Bunnicula connecting to my.broker on port 12345
+with the username 'usr' the password 'pwd' and choosing vhost /my_vhost. All of the options except the host name can be excluded in favor of the
+standard defaults. If you just have a hostname to specify, you don't need the uri syntax, and can just specify the hostname as is, such as "my_source_broker" or "192.168.0.71"
+
+
+== From the command line
+Feel free to type bunnicula --help to get a full list of options that may or may not have adequate documentation of usage.
+
+Relay messages published to a topic exchange named an.exchange.to.replay on amqp://my_source_broker to an identical exchange on a_target_broker:
+
+    bunnicula run --source my_source_broker --target a_target_broker --relay an.exchange.to.relay --type topic --log /var/log/bunnicula
+
+Relay messages published to a topic exchange named a.source.exchange on my_source_broker to a fanout exchange named a.target.exchange on a_target_broker
+
+    bunnicula run --source my_source_broker --target a_target_broker --relay a.source.exchange --type topic --to a.target.exchange --type fanout --log /path/to/my/log
+
+You can specify as many --target flags as you like, each indicating a single target AMQP broker, and you can in turn specify as many --relay flags as you like for each --target.
+Bunnicula currently supports listening to a single source broker per process.
+
+== Configuring Bunnicula via the Ruby DSL
+You can optionally configure bunnicula using a Ruby DSL. If you have a complex configuration or you need a feature not available from the command line (aka message filtering) you can
+select the ruby configuration file you want via the --relay_file flag.
+
+    bunnicula run --relay_file MyRelayFile.rb --log /path/to/my/log
+
+A Relay file will look something like what you see below. In keeping with the vampire rabbit theme evoked by the name of the library, the DSL may be a little colorful for a few of you with your pocket protectors on too tight. For those folks, all the fun has been aliased out of the DSL so your configuration file can look nice and prudish as you please, but you'll need to figure out how from the source.
+
+    Bunnicula.bite {
+
+      #Bunnicula will listen for incoming messages on the following host:
+      victim "amqp://a:b@a-host:12345/tada"
+
+      #This sets up a target broker
+      transfusion_to "amqp://username:password@example.com:12345" do
+
+        #Relay all messages published to "an.exchange.name", using the default type of 'direct', durable=true, ack=true
+        relay "an.exchange.name"
+
+      end
+
+      #A second target broker
+      transfusion_to {
+
+        #Specify connection settings the long way
+        host     "target_server_1"
+        port     12345
+        username "a"
+        password "b"
+        vhost    "tada"
+
+        #Set up a relay with a destination exchange that varies its configuration from the source exchange
+        relay do
+          from "test_source_exchange",      :type=>:topic, :durable=>true, :ack=>true
+          to   "test_destination_exchange", :durable=>true
+        end
+
+        #Set up a group of relays that share a common configuration. All exchange settings specified in this way
+        #will be mirrored on the destination broker.
+        relay  "another_exchange",
+               "and_another_exchange",
+               "and_even_another",
+               :type=>:fanout,
+               :durable=>false,
+               :ack=>true
+      }
+    }
+
+The DSL is pretty self explanatory, but keep in mind you can have only one victim, but as many transfusions (which represent a target server) and relays per transfusion as you please.
+In all cases, the plethora of individual host, port, username etc.. command specifying connection options can be replaced by passing a URI of the structure described previously as an argument
+to the command itself.
+
+== Other Resources
+A wealth of information on Bunnicula can be found {here}[http://www.google.com/images?q=bunnicula]
+
+== Limitations
+* None. Bunnicula is 100% perfect in every way. If you find a problem, you are probably confused and may be high or having a stroke. My first recommendation would be to seek medical attention or check yourself into rehab. If however you insist in persisting in your delusion that Bunnicula has a limitation, or perhaps even more preposterously has made a mistake, you can use the issue tracker at which point I will confirm your insanity and draw on my vast knowledge of shamanistic medicine to recommend an appropriate treatment for your ailment.
+
+== Bugs/Issues
+Please report them {on github}[http://github.com/plasticlizard/bunnicula/issues].
+
+== Todo
+Per-relay message filtering based on a regex or a code block has been implemented but not tested or documented. I don't really have a use case for such a feature yet in my own
+systems. I should probably remove it until I do, but I didn't. So if this is something that would be useful, put it in the issue tracker
+and I'll finish documenting and testing it. If you want to use it like a cowboy, see relay.rb

data/Rakefile

@@ -28,7 +28,6 @@ begin
     s.extra_rdoc_files = ["README.rdoc", "LICENSE.txt"]
     s.files = FileList["[A-Z]*", "{bin,lib,config,libexec,test}/**/*"]
 
-    s.add_dependency('daemon-kit', '0.1.7.12')
     s.add_dependency('bunny', '0.6.0')
     s.add_dependency('amqp', '0.6.7')
 

data/bunnicula.gemspec

@@ -5,21 +5,22 @@
 
 Gem::Specification.new do |s|
   s.name = %q{bunnicula}
-  s.version = "0.1.1"
+  s.version = "0.2.0"
 
   s.required_rubygems_version = Gem::Requirement.new(">= 0") if s.respond_to? :required_rubygems_version=
   s.authors = ["Nathan Stults"]
-  s.date = %q{2010-07-09}
+  s.date = %q{2010-07-12}
   s.default_executable = %q{bunnicula}
   s.description = %q{A very simple relay for moving messages from a local broker to a remote broker}
   s.email = %q{hereiam@sonic.net}
   s.executables = ["bunnicula"]
   s.extra_rdoc_files = [
-    "LICENSE.txt"
+    "LICENSE.txt",
+     "README.rdoc"
   ]
   s.files = [
     "LICENSE.txt",
-     "README",
+     "README.rdoc",
      "Rakefile",
      "Relayfile",
      "bin/bunnicula",
@@ -65,18 +66,15 @@ Gem::Specification.new do |s|
     s.specification_version = 3
 
     if Gem::Version.new(Gem::VERSION) >= Gem::Version.new('1.2.0') then
-      s.add_runtime_dependency(%q<daemon-kit>, ["= 0.1.7.12"])
       s.add_runtime_dependency(%q<bunny>, ["= 0.6.0"])
       s.add_runtime_dependency(%q<amqp>, ["= 0.6.7"])
       s.add_development_dependency(%q<shoulda>, ["= 2.11.1"])
     else
-      s.add_dependency(%q<daemon-kit>, ["= 0.1.7.12"])
       s.add_dependency(%q<bunny>, ["= 0.6.0"])
       s.add_dependency(%q<amqp>, ["= 0.6.7"])
       s.add_dependency(%q<shoulda>, ["= 2.11.1"])
     end
   else
-    s.add_dependency(%q<daemon-kit>, ["= 0.1.7.12"])
     s.add_dependency(%q<bunny>, ["= 0.6.0"])
     s.add_dependency(%q<amqp>, ["= 0.6.7"])
     s.add_dependency(%q<shoulda>, ["= 2.11.1"])

data/config/arguments.rb

@@ -1,70 +1,69 @@
-# Argument handling for your daemon is configured here.
-#
-# You have access to two variables when this file is
-# parsed. The first is +opts+, which is the object yielded from
-# +OptionParser.new+, the second is +@options+ which is a standard
-# Ruby hash that is later accessible through
-# DaemonKit.arguments.options and can be used in your daemon process.
-
-# Here is an example:
-# opts.on('-f', '--foo FOO', 'Set foo') do |foo|
-#  @options[:foo] = foo
-# end
-
-
-opts.on('-c PATH','--config_path PATH') do |config|
-  @options[:configuration_file] = config
-end
-
-opts.on('-s SOURCE','--source SOURCE') do |source_uri|
-  @options[:source_uri] = source_uri
-end
-
-opts.on('-t TARGET','--target TARGET') do |target_uri|
-  @options[:targets] ||= []
-  @options[:targets] << {:target_uri=>target_uri}
-end
-
-opts.on('-r RELAY', '--relay RELAY') do |relay|
-  raise "You specified a RELAY parameter, but did not first specify a TARGET parameter. Please indicate the target AMQP server in the form of -t TARGET or --target TARGET" unless @options[:targets]
-  @options[:targets][-1][:relays] ||= []
-  @options[:targets][-1][:relays] << {:from=>relay}
-end
-
-opts.on('-y TO', '--to TO') do |to|
-  puts to
-  raise "You specified a RELAY_TO parameter, but did not first specify a RELAY parameter. Please indicate the target AMQP server in the form of -r RELAY or --relay RELAY" unless @options[:targets] && @options[:targets][-1][:relays]
-  relay = @options[:targets][-1][:relays][-1]
-  relay[:to] = to
-end
-
-opts.on('-x TYPE', '--type TYPE') do |type|
-  raise "You specified a TYPE parameter, but did not first specify a RELAY parameter. Please indicate the target AMQP server in the form of -r RELAY or --relay RELAY" unless @options[:targets] && @options[:targets][-1][:relays]
-  relay = @options[:targets][-1][:relays][-1]
-  if relay[:to]
-    relay[:to_type] = type.downcase.to_sym
-  else
-    relay[:from_type] = type.downcase.to_sym
-  end
-end
-
-opts.on('-d','--durable') do |durable|
-  raise "You specified a DURABLE parameter, but did not first specify a RELAY parameter. Please indicate the target AMQP server in the form of -r RELAY or --relay RELAY" unless @options[:targets] && @options[:targets][-1][:relays]
-  relay = @options[:targets][-1][:relays][-1]
-  if relay[:to]
-    relay[:to_durable] = durable
-  else
-    relay[:from_durable] = durable
-  end
-end
-
-opts.on('-a','--ack') do |ack|
-  raise "You specified a ACK parameter, but did not first specify a RELAY parameter. Please indicate the target AMQP server in the form of -r RELAY or --relay RELAY" unless @options[:targets] && @options[:targets][-1][:relays]  
-  relay = @options[:targets][-1][:relays][-1]
-  if relay[:to]
-    relay[:to_ack] = ack
-  else
-    relay[:from_ack] = ack
-  end
-end
-
+# Argument handling for your daemon is configured here.
+#
+# You have access to two variables when this file is
+# parsed. The first is +opts+, which is the object yielded from
+# +OptionParser.new+, the second is +@options+ which is a standard
+# Ruby hash that is later accessible through
+# DaemonKit.arguments.options and can be used in your daemon process.
+
+# Here is an example:
+# opts.on('-f', '--foo FOO', 'Set foo') do |foo|
+#  @options[:foo] = foo
+# end
+
+opts.on('--relay_file PATH') do |config|
+  @options[:configuration_file] = config
+end
+
+opts.on('-s SOURCE','--source SOURCE') do |source_uri|
+  @options[:source_uri] = source_uri
+end
+
+opts.on('-t TARGET','--target TARGET') do |target_uri|
+  @options[:targets] ||= []
+  @options[:targets] << {:target_uri=>target_uri}
+end
+
+opts.on('-r RELAY', '--relay RELAY') do |relay|
+  raise "You specified a RELAY parameter, but did not first specify a TARGET parameter. Please indicate the target AMQP server in the form of -t TARGET or --target TARGET" unless @options[:targets]
+  @options[:targets][-1][:relays] ||= []
+  @options[:targets][-1][:relays] << {:from=>relay}
+end
+
+opts.on('-y TO', '--to TO') do |to|
+  puts to
+  raise "You specified a RELAY_TO parameter, but did not first specify a RELAY parameter. Please indicate the target AMQP server in the form of -r RELAY or --relay RELAY" unless @options[:targets] && @options[:targets][-1][:relays]
+  relay = @options[:targets][-1][:relays][-1]
+  relay[:to] = to
+end
+
+opts.on('-x TYPE', '--type TYPE') do |type|
+  raise "You specified a TYPE parameter, but did not first specify a RELAY parameter. Please indicate the target AMQP server in the form of -r RELAY or --relay RELAY" unless @options[:targets] && @options[:targets][-1][:relays]
+  relay = @options[:targets][-1][:relays][-1]
+  if relay[:to]
+    relay[:to_type] = type.downcase.to_sym
+  else
+    relay[:from_type] = type.downcase.to_sym
+  end
+end
+
+opts.on('-d','--durable') do |durable|
+  raise "You specified a DURABLE parameter, but did not first specify a RELAY parameter. Please indicate the target AMQP server in the form of -r RELAY or --relay RELAY" unless @options[:targets] && @options[:targets][-1][:relays]
+  relay = @options[:targets][-1][:relays][-1]
+  if relay[:to]
+    relay[:to_durable] = durable
+  else
+    relay[:from_durable] = durable
+  end
+end
+
+opts.on('-a','--ack') do |ack|
+  raise "You specified a ACK parameter, but did not first specify a RELAY parameter. Please indicate the target AMQP server in the form of -r RELAY or --relay RELAY" unless @options[:targets] && @options[:targets][-1][:relays]  
+  relay = @options[:targets][-1][:relays][-1]
+  if relay[:to]
+    relay[:to_ack] = ack
+  else
+    relay[:from_ack] = ack
+  end
+end
+

data/config/boot.rb

@@ -1,59 +1,60 @@
-# Don't change this file!
-# Configure your daemon in config/environment.rb
-
-DAEMON_ROOT = "#{File.expand_path(File.dirname(__FILE__))}/.." unless defined?( DAEMON_ROOT )
-
-module DaemonKit
-  class << self
-    def boot!
-      unless booted?
-        pick_boot.run
-      end
-    end
-
-    def booted?
-      defined? DaemonKit::Initializer
-    end
-
-    def pick_boot
-      (vendor_kit? ? VendorBoot : GemBoot).new
-    end
-
-    def vendor_kit?
-      File.exists?( "#{DAEMON_ROOT}/vendor/daemon_kit" )
-    end
-  end
-
-  class Boot
-    def run
-      load_initializer
-      DaemonKit::Initializer.run
-    end
-  end
-
-  class VendorBoot < Boot
-    def load_initializer
-      require "#{DAEMON_ROOT}/vendor/daemon_kit/lib/daemon_kit/initializer"
-    end
-  end
-
-  class GemBoot < Boot
-    def load_initializer
-      begin
-        gem 'daemon-kit'
-        require 'daemon_kit/initializer'
-      rescue Gem::LoadError => e
-        msg = <<EOF
-You are missing the daemon-kit gem. Please install the following gems:
-
-* Stable   - sudo gem install daemon-kit
-
-EOF
-        $stderr.puts msg
-        exit 1
-      end
-    end
-  end
-end
-
-DaemonKit.boot!
+# Don't change this file!
+# Configure your daemon in config/environment.rb
+
+DAEMON_ROOT = "#{File.expand_path(File.dirname(__FILE__))}/.." unless defined?( DAEMON_ROOT )
+
+module DaemonKit
+  class << self
+    def boot!
+      unless booted?
+        pick_boot.run
+      end
+    end
+
+    def booted?
+      defined? DaemonKit::Initializer
+    end
+
+    def pick_boot
+      (vendor_kit? ? VendorBoot : GemBoot).new
+    end
+
+    def vendor_kit?
+      File.exists?( "#{DAEMON_ROOT}/vendor/daemon_kit/daemon_kit" )
+    end
+  end
+
+  class Boot
+    def run
+      load_initializer
+      DaemonKit::Arguments.parser_available = true
+      DaemonKit::Initializer.run
+    end
+  end
+
+  class VendorBoot < Boot
+    def load_initializer
+      require "#{DAEMON_ROOT}/vendor/daemon_kit/daemon_kit/lib/daemon_kit/initializer"
+    end
+  end
+
+  class GemBoot < Boot
+    def load_initializer
+      begin
+        gem 'daemon-kit'
+        require 'daemon_kit/initializer'
+      rescue Gem::LoadError => e
+        msg = <<EOF
+You are missing the daemon-kit gem. Please install the following gems:
+
+* Stable   - sudo gem install daemon-kit
+
+EOF
+        $stderr.puts msg
+        exit 1
+      end
+    end
+  end
+end
+
+DaemonKit.boot!

data/config/environment.rb

@@ -1,33 +1,34 @@
-# Be sure to restart your daemon when you modify this file
-
-# Uncomment below to force your daemon into production mode
-ENV['DAEMON_ENV'] ||= 'production'
-
-# Boot up
-require "rubygems"
-
-begin
-  require 'amqp'
-  require 'mq'
-rescue LoadError
-  $stderr.puts "Missing amqp gem. Please run 'gem install amqp'."
-  exit 1
-end
-
-require File.join(File.dirname(__FILE__), 'boot')
-
-DaemonKit::Initializer.run do |config|
-
-  # The name of the daemon as reported by process monitoring tools
-  config.daemon_name = 'bunnicula'
-
-  # Force the daemon to be killed after X seconds from asking it to
-  # config.force_kill_wait = 30
-
-  # Log backraces when a thread/daemon dies (Recommended)
-  # config.backtraces = true
-
-  # Configure the safety net (see DaemonKit::Safety)
-  # config.safety_net.handler = :mail # (or :hoptoad )
-  # config.safety_net.mail.host = 'localhost'
-end
+# Be sure to restart your daemon when you modify this file
+
+# Uncomment below to force your daemon into production mode
+ENV['DAEMON_ENV'] ||= 'production'
+
+# Boot up
+require "rubygems"
+
+begin
+  require 'amqp'
+  require 'mq'
+rescue LoadError
+  $stderr.puts "Missing amqp gem. Please run 'gem install amqp'."
+  exit 1
+end
+
+require File.join(File.dirname(__FILE__), 'boot')
+
+DaemonKit::Arguments.parser_available = true
+DaemonKit::Initializer.run do |config|
+
+  # The name of the daemon as reported by process monitoring tools
+  config.daemon_name = 'bunnicula'
+
+  # Force the daemon to be killed after X seconds from asking it to
+  # config.force_kill_wait = 30
+
+  # Log backraces when a thread/daemon dies (Recommended)
+  # config.backtraces = true
+
+  # Configure the safety net (see DaemonKit::Safety)
+  # config.safety_net.handler = :mail # (or :hoptoad )
+  # config.safety_net.mail.host = 'localhost'
+end

data/lib/bunnicula/version.rb

@@ -1,3 +1,3 @@
 module Bunnicula
-  VERSION = '0.1.1'
+  VERSION = '0.2.0'
 end

data/libexec/bunnicula-daemon.rb

@@ -1,5 +1,4 @@
 # Generated amqp daemon
-
 # Do your post daemonization configuration here
 # At minimum you need just the first line (without the block), or a lot
 # of strange things might start happening...

metadata

@@ -1,13 +1,13 @@
 --- !ruby/object:Gem::Specification 
 name: bunnicula
 version: !ruby/object:Gem::Version 
-  hash: 25
+  hash: 23
   prerelease: false
   segments: 
   - 0
-  - 1
-  - 1
-  version: 0.1.1
+  - 2
+  - 0
+  version: 0.2.0
 platform: ruby
 authors: 
 - Nathan Stults
@@ -15,30 +15,13 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2010-07-09 00:00:00 -07:00
+date: 2010-07-12 00:00:00 -07:00
 default_executable: bunnicula
 dependencies: 
-- !ruby/object:Gem::Dependency 
-  name: daemon-kit
-  prerelease: false
-  requirement: &id001 !ruby/object:Gem::Requirement 
-    none: false
-    requirements: 
-    - - "="
-      - !ruby/object:Gem::Version 
-        hash: 67
-        segments: 
-        - 0
-        - 1
-        - 7
-        - 12
-        version: 0.1.7.12
-  type: :runtime
-  version_requirements: *id001
 - !ruby/object:Gem::Dependency 
   name: bunny
   prerelease: false
-  requirement: &id002 !ruby/object:Gem::Requirement 
+  requirement: &id001 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - "="
@@ -50,11 +33,11 @@ dependencies:
         - 0
         version: 0.6.0
   type: :runtime
-  version_requirements: *id002
+  version_requirements: *id001
 - !ruby/object:Gem::Dependency 
   name: amqp
   prerelease: false
-  requirement: &id003 !ruby/object:Gem::Requirement 
+  requirement: &id002 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - "="
@@ -66,11 +49,11 @@ dependencies:
         - 7
         version: 0.6.7
   type: :runtime
-  version_requirements: *id003
+  version_requirements: *id002
 - !ruby/object:Gem::Dependency 
   name: shoulda
   prerelease: false
-  requirement: &id004 !ruby/object:Gem::Requirement 
+  requirement: &id003 !ruby/object:Gem::Requirement 
     none: false
     requirements: 
     - - "="
@@ -82,7 +65,7 @@ dependencies:
         - 1
         version: 2.11.1
   type: :development
-  version_requirements: *id004
+  version_requirements: *id003
 description: A very simple relay for moving messages from a local broker to a remote broker
 email: hereiam@sonic.net
 executables: 
@@ -91,9 +74,10 @@ extensions: []
 
 extra_rdoc_files: 
 - LICENSE.txt
+- README.rdoc
 files: 
 - LICENSE.txt
-- README
+- README.rdoc
 - Rakefile
 - Relayfile
 - bin/bunnicula

data/README

@@ -1,30 +0,0 @@
-== Overview
-
-== Motivation
-
-== Approach
-
-== Install
-
-Install the gem with:
-
-    gem install bunnicula
-          or
-    sudo gem install bunnicula
-
-
-== An Example
-
-== Limitations
-* Cubicle cannot currently cause child documents to be emitted in the map reduce. This is a pretty big limitation, and will be resolved shortly.
-* Documentation is non-existent. This is being worked on (head that one before?)
-* Test coverage is OK, but the tests could be better organized
-* Code needs to be modularized a bit, main classes are pretty hairy at the moment
-
-
-== Bugs/Issues
-Please report them {on github}[http://github.com/plasticlizard/bunnicula/issues].
-
-== Links
-
-== Todo