dripdrop 0.10.0-java

data/.document

@@ -0,0 +1,5 @@
+README.rdoc
+lib/**/*.rb
+bin/*
+features/**/*.feature
+LICENSE

data/.gitignore

@@ -0,0 +1,31 @@
+## MAC OS
+.DS_Store
+
+## TEXTMATE
+*.tmproj
+tmtags
+
+## EMACS
+*~
+\#*
+.\#*
+
+## VIM
+*.swp
+*.swo
+*.swn
+
+## PROJECT::GENERAL
+coverage
+rdoc
+doc
+pkg
+
+#RBX
+*.rbc
+
+## PROJECT::SPECIFIC
+.rvmrc
+Gemfile.lock
+.bundle
+*.gem

data/Gemfile

@@ -0,0 +1,5 @@
+source "http://rubygems.org"
+
+# Specify your gem's dependencies in dripdrop.gemspec
+gemspec
+

data/LICENSE

@@ -0,0 +1,20 @@
+Copyright (c) 2009 Andrew Cholakian
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,164 @@
+# DripDrop
+
+## INSTALLATION NOTES:
+
+1. This does NOT work with 1.8.7. 1.9.2/RBX/jRuby only.
+2. Build zeromq2 from [git master](https://github.com/zeromq/zeromq2) if you haven't already.
+3. Install with 'gem install dripdrop'.
+4. If on jRuby, http servers will be unavailable unless you have a jruby with cext support, and manually `gem install eventmachine_httpserver`
+5. If on 1.9.x `gem install ffi`
+
+## About
+
+DripDrop is a library aiming to help you write better message passing apps. It's a rich toolchest, currently based on EventMachine, which provides async IO. 
+
+DripDrop helps in these ways:
+
+  1. Normalized Communication Interfaces. All protocols, regardless of their using HTTP, ZeroMQ, or WebSockets, have a unified API, with minor differences only to accomodate the reality of the underlying protocol.
+  2. A set of tools to break your app into composable parts--we call them nodelets in dripdrop--ideally communicating with each other with the aforementioned interfaces..
+  3. A simple, yet powerful messaging class, that lets you control the structure, formatting, and serialization of messages sent between nodelets.
+  4. Tools to break your nodelets off when it comes time to deploy, letting you scale your app by delegating roles and scaling out resources
+
+## Normalized Interfaces
+
+Let's start by looking at the normalized communication interface in a simple app.
+    
+    class MyApp < DripDrop::Node
+      def action #Special method that gets executed on Node#start
+        # Define some sockets, here we create an HTTP server, and
+        # a client to it. :my_hts and :my_htc are custom names
+        # that will be available after definition
+        route :my_server, :http_server, 'http://127.0.0.1:2201'
+        route :my_client, :http_client, 'http://127.0.0.1:2201'
+        
+        # Our http server is a simple time server
+        my_server.on_recv do |message,response|
+          response.send_message(:name => 'time', :body => {'time' => Time.now.to_s})
+        end
+        
+        # Here, we setup a timer, and periodically poll the http server
+        EM::PeriodicTimer.new(1) do
+          # Messages must have a :name. They can optionally have a :body.
+          # Additionally, they can set custom :head properties.
+          my_client.send_message(:name => 'time_request') do |response_message|
+            puts "The time is: #{response_message.body['time']}"
+          end
+        end
+      end
+    end
+     
+    #Start the app and block
+    MyApp.new.start!
+
+What we've done here is use HTTP as a simple messaging protocol. Yes, we've thrown out a good chunk of what HTTP does, but consider this, that exact same code would work if we replaced the top two lines with:
+
+        route :my_server, :zmq_xrep, 'http://127.0.0.1:2201', :bind
+        route :my_client, :zmq_xreq, 'http://127.0.0.1:2201', :connect
+
+That replaces the HTTP server and client with ultra-high performance zeromq sockets. Now, protocols have varying strengths and weaknesses, and ZeroMQ is not HTTP necessarily, for instance, given a :zmq_pub socket, you can only send_messages, but there is no response message, because :zmq_pub is the publishing end of a request/reply pattern. The messaging API attempts to reduce all methods on sockets to the following set:
+
+  * on_recv (sometimes takes a block with |message,response| if it can send a response)
+  * send_message
+  * on_open  (Websockets only)
+  * on_close (Websockets only)
+
+
+## Composable Parts
+
+The tools mentioned above are useful, but if you try and build a larger app you'll quickly find them lacking. The callbacks get tricky, and mixing your logic up in a single #action method becomes messy. That's why we have nodelets in DripDrop. Here's a trivial example.
+
+    class MyApp < DripDrop::Node
+      # This will instantiate a new StatsCollector object, and define the
+      # stats_raw and stats_filtered methods inside it.
+      nodelet :stats_collector, StatsCollector do |nodelet|
+        nodelet.route :stats_raw,      :zmq_pull, 'tcp://127.0.0.1:2301', :bind
+        nodelet.route :stats_filtered, :zmq_push, 'tcp://127.0.0.1:2302', :bind
+      end
+      
+      nodelet :stats_processor, StatsProcessor do |nodelet|
+        nodelet.route :stats_ingress, :zmq_pull, 'tcp://127.0.0.1:2302', :bind
+      end
+
+      # The nodelets method gives you access to all defined nodelets
+      # We created a #run method on each nodelet we call here.
+      nodelets.each {|n| n.run}
+    end
+
+    # You must subclass Nodelet
+    # The method #run here is merely a convention
+    class StatsCollector < DripDrop::Node::Nodelet
+      def run
+        stats_raw.on_recv do |raw_stat_msg|
+          # Custom filtering code could go here...
+          stats_filtered.send_message(raw_stat_msg)
+        end
+      end
+    end
+  
+    class StatsProcessor < DripDrop::Node::Nodelet
+      # Initialize shouldn't be subclassed on a Nodelet, this gets called
+      # After the nodelet is instantiated
+      def configure
+        @name_counts = Hash.new(0)
+      end
+      
+      def run
+        stats_ingress.on_recv do |message|
+          @name_counts[message.name] += 1
+          puts @name_counts.inspect
+        end
+      end
+    end
+    
+    MyApp.new.start!
+
+# Custom Messages
+
+  DripDrop::Message is the parent class of all messages in dripdrop, it's a flexible and freeform way to send data. In more complex apps you'll want to both define custom behaviour on messages, and restrict the data they carry. This is possible by subclassing DripDrop::Message. Before we look at that though, lets see what makes a DripDrop::Message. 
+
+  The simplest DripDrop::Message you could create would look something like this if dumped into JSON:
+    
+    {name: 'msgname', head: {}, body: null}
+
+  In other words, a dripdrop message *must* provide a name, it must also be able to store arbitrary, nested, keys and values in its head, and it may use the body for any data it wishes.
+
+  If you'd like to create your own Message format, simply Subclass DripDrop::Message. If you want to restrict your handlers to using a specific message type, it's easily done by passing in the :message_class option. For instance
+
+    class MyMessageClass < DripDrop::Message
+      # Custom code
+    end
+    class MyApp < DripDrop::Node
+      def action
+        route :myhandler, :zmq_publish,   'tcp://127.0.0.1:2200', :bind,    :message_class => MyMessageClass 
+        route :myhandler, :zmq_subscribe, 'tcp://127.0.0.1:2200', :connect, :message_class => MyMessageClass
+      end
+    end
+
+# Breaking out your nodelets
+  
+One of the core ideas behind dripdrop, is that if your application is composed of a bunch of separate parts, that in production deployment, will run on separate physical servers, it should still be possible for you to develop and test with ease. If you structure your app into separate nodelets, and *only* communicate between them via message passing, you can accomplish this easily. 
+
+While you will have to write your own executable wrappers suitable for your own deployment, one convenenience feature built in is the notion of a +run_list+. By setting the #run_list you can restrict which nodelets actually get initialized. For example:
+
+    class MyApp < DripDrop::Node
+      nodelet :service_one, ServiceOneClass do
+        #nodelet setup
+      end
+      nodelet :service_two, ServiceTwoClass do
+        #nodelet setup
+      end
+    end
+
+    # Only starts :service_two, the setup for :service_one
+    # is skipped as well
+    MyApp.new(:run_list => [:service_two]).start!
+
+#RDocs
+
+RDocs can be found [here](http://www.rdoc.info/github/andrewvc/dripdrop/master/frames). Most of the interesting stuff is in the [Node](http://www.rdoc.info/github/andrewvc/dripdrop/master/DripDrop/Node) and [Message](http://www.rdoc.info/github/andrewvc/dripdrop/master/DripDrop/Message) classes.
+
+#Contributors
+
+* Andrew Cholakian: [andrewvc](http://github.com/andrewvc)
+* John W Higgins: [wishdev](http://github.com/wishdev)
+* Nick Recobra: [oruen](https://github.com/oruen)

data/Rakefile

@@ -0,0 +1,16 @@
+require 'bundler'
+Bundler::GemHelper.install_tasks
+
+require 'rake/rdoctask'
+require 'dripdrop/version'
+Rake::RDocTask.new do |rdoc|
+  rdoc.rdoc_dir = 'rdoc'
+  rdoc.title = "dripdrop #{DripDrop::VERSION}"
+  rdoc.rdoc_files.include('README*')
+  rdoc.rdoc_files.include('lib/**/*.rb')
+end
+
+require 'rspec/core/rake_task'
+RSpec::Core::RakeTask.new(:spec) do |t|
+end
+task :default => :spec

data/dripdrop.gemspec

@@ -0,0 +1,37 @@
+# -*- encoding: utf-8 -*-
+$:.push File.expand_path("../lib", __FILE__)
+require "dripdrop/version"
+
+Gem::Specification.new do |s|
+  s.name        = "dripdrop"
+  s.version     = DripDrop::VERSION
+  s.platform    = RUBY_PLATFORM[/java/] || Gem::Platform::RUBY
+  s.authors     = ["Andrew Cholakian"]
+  s.email       = ["andrew@andrewvc.com"]
+  s.homepage    = "https://github.com/andrewvc/dripdrop"
+  s.summary     = %q{Evented framework for ZeroMQ and EventMachine Apps.}
+  s.description = %q{Evented framework for ZeroMQ and EventMachine Apps.}
+  s.extra_rdoc_files = [
+    "LICENSE",
+    "README.md"
+  ]
+
+  s.rubyforge_project = "dripdrop"
+
+  s.add_dependency "eventmachine", ">= 0.12.10"
+  s.add_dependency "em-websocket", ">= 0"
+  s.add_dependency "em-zeromq", ">= 0.2.1"
+  if s.platform.to_s == "java"
+    s.add_dependency "json", ">= 1.5.1" 
+  else
+    s.add_dependency "yajl-ruby", ">= 0.8.1"
+    s.add_dependency "eventmachine_httpserver", ">= 0.2.1"
+  end
+  s.add_development_dependency "rspec", ">= 2.4.0"
+
+  s.files         = `git ls-files`.split("\n")
+  s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")
+  s.executables   = `git ls-files -- bin/*`.split("\n").map{ |f| File.basename(f) }
+  s.require_paths = ["lib"]
+end
+

data/example/agent_test.rb

@@ -0,0 +1,14 @@
+##
+##TODO: This badly needs to be rewritten
+##
+
+require 'rubygems'
+require 'dripdrop/agent'
+
+agent = DripDrop::Agent.new(ZMQ::PUB,'tcp://127.0.0.1:2900',:connect)
+
+loop do
+  agent.send_message('test', :body => 'hello', :head => {:key => 'value'})
+  puts "SEND"
+  sleep 1
+end

data/example/combined.rb

@@ -0,0 +1,33 @@
+require 'dripdrop'
+Thread.abort_on_exception = true #Always a good idea in multithreaded apps.
+
+# Encapsulates our EM and ZMQ reactors
+DripDrop::Node.new do
+  # Define all our sockets
+  route :stats_pub,      :zmq_publish,   'tcp://127.0.0.1:2200', :bind
+  route :stats_sub1,     :zmq_subscribe, stats_pub.address, :connect
+  route :stats_sub2,     :zmq_subscribe, stats_pub.address, :connect
+  route :http_collector, :http_server,   'http://127.0.0.1:8080'
+  route :http_agent,     :http_client,   http_collector.address
+    
+  stats_sub1.on_recv do |message|
+    puts "Receiver 1: #{message.body}"
+  end
+  stats_sub2.on_recv do |message|
+    puts "Receiver 2: #{message.body}"
+  end
+  
+  i = 0
+  http_collector.on_recv do |message,response|
+    i += 1
+    stats_pub.send_message(message)
+    response.send_message(:name => 'ack', :body => {:seq => i})
+  end
+
+  EM::PeriodicTimer.new(1) do
+    msg = DripDrop::Message.new('http/status', :body => "Success #{i}")
+    http_agent.send_message(msg) do |resp_msg|
+      puts "RESP: #{resp_msg.body['seq']}"
+    end
+  end
+end.start! #Start the reactor and block until complete

data/example/complex/README

@@ -0,0 +1,22 @@
+This example creates an async, evented app that can do the following.
+
+* Broadcast messages to all connected websockets originating from a
+master control server.
+* Proxy requests sent via websocket to HTTP through a master
+control server
+
+It's broken into three parts
+* service.rb: The async core, written in dripdrop/eventmachine/zeromq
+* client.rb:  A Test websocket client
+* service.rb: A test HTTP web-service that could beb used to control messages
+
+To run.
+
+In one terminal (in dripdrop root)
+ruby -I lib/ example/complex/server.rb
+
+In another terminal (Websocket client)
+cd example/complex && ruby client.rb
+
+In a third terminal (Minimal webapp in sinatra)
+cd example/complex && ruby service.rb

data/example/complex/client.rb

@@ -0,0 +1,20 @@
+require 'rubygems'
+require 'websocket'
+require 'dripdrop/message'
+
+Thread.abort_on_exception = true
+
+client = WebSocket.new('ws://127.0.0.1:8080')
+
+Thread.new do
+  while data = client.receive
+    puts data
+  end
+end
+
+i = 0
+while sleep 1
+  i += 1
+  puts '.'
+  client.send(DripDrop::Message.new('Client Broadcast', :body => i).json_encoded)
+end

data/example/complex/server.rb

@@ -0,0 +1,102 @@
+require 'dripdrop'
+Thread.abort_on_exception = true
+
+class ComplexExample < DripDrop::Node
+  def initialize(mode=:all)
+    super()
+    @mode = mode
+  end
+  
+  def action
+    nodelet :ws_listener, WSListener do |n|
+      n.route :ws_listener,  :websocket, 'ws://127.0.0.1:8080'
+      n.route :broadcast_in, :zmq_subscribe, 'tcp://127.0.0.1:2200', :connect
+      n.route :reqs_out,     :zmq_xreq,      'tcp://127.0.0.1:2201', :connect
+    end
+      
+    nodelet :coordinator, Coordinator do |n|
+      n.route :broadcast_out, :zmq_publish, 'tcp://127.0.0.1:2200', :bind
+      n.route :reqs_in,       :zmq_xrep,    'tcp://127.0.0.1:2201', :bind
+      n.route :reqs_htout,    :http_client, 'tcp://127.0.0.1:3000/endpoint'
+    end
+  end
+end
+
+class Coordinator < DripDrop::Node::Nodelet
+  def run
+    proxy_reqs
+    heartbeat
+  end
+
+  def proxy_reqs
+    reqs_in.on_recv do |message, response|
+      puts "Proxying #{message.inspect} to htout"
+      reqs_htout.send_message(message) do |http_response|
+        puts "Received http response #{http_response.inspect} sending back"
+        response.send_message(http_response)
+      end
+    end
+  end
+  
+  def heartbeat
+    EM::PeriodicTimer.new(1) do
+      broadcast_out.send_message :name => 'tick', :body => Time.now.to_s
+    end
+  end
+end
+
+class WSListener < DripDrop::Node::Nodelet
+  def initialize(*args)
+    super
+    @client_channel = EM::Channel.new
+  end
+  
+  def run
+    proxy_websockets
+    broadcast_to_websockets
+  end
+
+  def broadcast_to_websockets
+    # Receives messages from Broadcast Out
+    broadcast_in.on_recv do |message|
+      puts "Broadcast In recv: #{message.inspect}"
+      @client_channel.push(message)
+    end
+  end
+
+  def proxy_websockets
+    sigs_sids = {} #Map connection signatures to subscriber IDs
+     
+    ws.on_open do |conn|
+      puts "WS Connected"
+      conn.send_message(DripDrop::Message.new('test'))
+      
+      sid = @client_channel.subscribe do |message|
+        puts message.inspect
+        conn.send_message(message)
+      end
+       
+      sigs_sids[conn.signature] = sid
+    end
+    ws.on_close do |conn|
+      puts "Closed #{conn.signature}"
+      @client_channel.unsubscribe sigs_sids[conn.signature]
+    end
+    ws.on_error do |reason,conn|
+      puts "Errored #{reason.inspect}, #{conn.signature}"
+      @client_channel.unsubscribe sigs_sids[conn.signature]
+    end
+
+    ws.on_recv do |message,conn|
+      puts "WS Recv #{message.name}"
+      reqs_out.send_message(message) do |resp_message|
+        puts "Recvd resp_message #{resp_message.inspect}, sending back to client"
+        conn.send_message(resp_message)
+      end
+    end
+  end
+end   
+    
+
+puts "Starting..."
+ComplexExample.new.start!