redstorm 0.2.0 → 0.2.1

data/CHANGELOG.md

@@ -10,3 +10,6 @@
 - examples using simple DSL
 - redstorm command usage syntax change
 - more doc in README
+
+# 0.2.1, 11-23-2011
+- gems support in production cluster

data/README.md

@@ -1,12 +1,13 @@
-# RedStorm v0.2.0 - JRuby on Storm
+# RedStorm v0.2.1 - JRuby on Storm
 
 RedStorm provides the JRuby integration for the [Storm][storm] distributed realtime computation system.
 
 ## Changes from 0.1.x
 
-- This release introduces the *simple* DSL. Topology, Spout and Bolt classes can inherit from the SimpleTopoloy, SimpleSpout and SimpleBolt classes which provides a very clean and consise DSL. See [examples/simple](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple).
-- Use the same SimpleTopology class for local development cluster or remote production cluster.
-- The `redstorm` command has a new syntax.
+- this release introduces the *simple* DSL. Topology, Spout and Bolt classes can inherit from the SimpleTopology, SimpleSpout and SimpleBolt classes which provides a very clean and consise DSL. See [examples/simple](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple).
+- use the same SimpleTopology class for local development cluster or remote production cluster.
+- the `redstorm` command has a new syntax.
+- gems support in production cluster
 
 ## Dependencies
 
@@ -27,7 +28,7 @@ $ gem install redstorm
 
 ### Initial setup
 
-Install RedStom dependencies; from your project root directory execute:
+- install RedStom dependencies; from your project root directory execute:
 
 ``` sh
 $ redstorm install
@@ -37,9 +38,21 @@ The `install` command will install all Java jars dependencies using [ruby-maven]
 
 ***DON'T PANIC*** it's Maven. The first time you run `$ redstorm install` Maven will take a few minutes resolving dependencies and in the end will download and install the dependency jar files.
 
-### Run in local mode
+- create a topology class. The *underscore* topology_class_file_name.rb **MUST** correspond to its *CamelCase* class name.
+
+### Gems
+
+Until this is better integrated, you can use **gems** in local mode and on a production cluster:
+
+- **local mode**: simply install your gems the usual way, they will be picked up when run in local mode.
+
+- **production cluster**: install your gem in the `target/gems` folder using:
 
-Create a topology class. The *underscore* topology_class_file_name.rb **MUST** correspond to its *CamelCase* class name.
+```sh
+gem install <the gem> --install-dir target/gems/ --no-ri --no-rdoc
+```
+
+### Run in local mode
 
 ``` sh
 $ redstorm local <path/to/topology_class_file_name.rb>
@@ -81,7 +94,7 @@ $ redstorm local examples/simple/exclamation_topology2.rb
 $ redstorm local examples/simple/word_count_topology.rb
 ```
 
-This next example requires the use of the [Redis Gem](https://github.com/ezmobius/redis-rb) and a [Redis][redis] server runnig on `localhost:6379`
+This next example requires the use of the [Redis Gem](https://github.com/ezmobius/redis-rb) and a [Redis][redis] server running on `localhost:6379`
 
 ``` sh
 $ redstorm local examples/simple/redis_word_count_topology.rb
@@ -105,6 +118,22 @@ $ redstorm jar examples
 $ storm jar ./target/cluster-topology.jar redstorm.TopologyLauncher cluster examples/simple/word_count_topology.rb
 ```
 
+- to run `examples/simple/redis_word_count_topology.rb` you need a [Redis][redis] server running on `localhost:6379` and the Redis gem in `target/gems` using:
+
+```sh
+gem install redis --install-dir target/gems/ --no-ri --no-rdoc
+```
+
+- generate jar and submit:
+
+``` sh
+$ redstorm jar examples
+$ storm jar ./target/cluster-topology.jar redstorm.TopologyLauncher cluster examples/simple/redis_word_count_topology.rb
+```
+
+- using `redis-cli`, push words into the `test` list and watch Storm pick them up
+
+
 Basically you must follow the [Storm instructions](https://github.com/nathanmarz/storm/wiki) to [setup a production cluster](https://github.com/nathanmarz/storm/wiki/Setting-up-a-Storm-cluster) and [submit your topology to the cluster](https://github.com/nathanmarz/storm/wiki/Running-topologies-on-a-production-cluster).
 
 ## DSL usage
@@ -430,6 +459,57 @@ end
 - [SplitSentenceBolt](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/split_sentence_bolt.rb)
 - [WordCountBolt](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/word_count_bolt.rb)
 
+## Development
+
+### Requirements
+
+- JRuby 1.6.5
+- rake gem
+- ruby-maven gem
+- rspec gem
+
+### Contribute
+
+Fork the project, create a branch and submit a pull request.
+
+Some ways you can contribute:
+
+- by reporting bugs using the issue tracker
+- by suggesting new features using the issue tracker
+- by writing or editing documentation
+- by writing specs
+- by writing code
+- by refactoring code
+- ...
+
+### Workflow
+
+- fork project
+- create branch
+- install dependencies in `target/dependencies`
+
+```sh
+$ rake deps
+```
+
+- generate and build Java source into `target/classes`
+
+```sh
+$ rake build
+```
+
+- run topology in local dev cluster
+
+```sh
+$ bin/redstorm local path/to/topology_class.rb
+```
+
+- generate remote cluster topology jar into `target/cluster-topology.jar`, including the `examples/` directory.
+
+```sh
+$ rake jar['examples']
+```
+
 ## Author
 Colin Surprenant, [@colinsurprenant][twitter], [colin.surprenant@needium.com][needium], [colin.surprenant@gmail.com][gmail], [http://github.com/colinsurprenant][github]
 

data/TODO.md

@@ -0,0 +1,4 @@
+# TODO
+
+- expose the log4j logger in Simple{Topology|Bolt|Spout} (Java::org.apache.log4j.Logger.getLogger(...))
+- see if using bundler would provide better gems integration and easier jar packaging

data/examples/simple/redis_word_count_topology.rb

@@ -1,3 +1,4 @@
+require 'rubygems'
 require 'redis'
 require 'thread'
 require 'red_storm'

data/lib/red_storm/version.rb

@@ -1,3 +1,3 @@
 module RedStorm
-  VERSION = '0.2.0'
+  VERSION = '0.2.1'
 end

data/lib/tasks/red_storm.rake

@@ -15,6 +15,7 @@ TARGET_SRC_DIR = "#{TARGET_DIR}/src"
 TARGET_CLASSES_DIR = "#{TARGET_DIR}/classes"  
 TARGET_DEPENDENCY_DIR = "#{TARGET_DIR}/dependency"
 TARGET_DEPENDENCY_UNPACKED_DIR = "#{TARGET_DIR}/dependency-unpacked"
+TARGET_GEMS_DIR = "#{TARGET_DIR}/gems"
 TARGET_CLUSTER_JAR = "#{TARGET_DIR}/cluster-topology.jar"
 
 JAVA_SRC_DIR = "#{RedStorm::REDSTORM_HOME}/src/main"
@@ -42,6 +43,7 @@ task :setup do
   ant.mkdir :dir => TARGET_DIR 
   ant.mkdir :dir => TARGET_CLASSES_DIR 
   ant.mkdir :dir => TARGET_SRC_DIR
+  ant.mkdir :dir => TARGET_GEMS_DIR
   ant.path :id => 'classpath' do  
     fileset :dir => TARGET_DEPENDENCY_DIR  
     fileset :dir => TARGET_CLASSES_DIR  
@@ -60,6 +62,7 @@ task :devjar => [:unpack, :clean_jar] do
   ant.jar :destfile => TARGET_CLUSTER_JAR do
     fileset :dir => TARGET_CLASSES_DIR
     fileset :dir => TARGET_DEPENDENCY_UNPACKED_DIR
+    fileset :dir => TARGET_GEMS_DIR
     fileset :dir => RedStorm::REDSTORM_HOME do
       include :name => "examples/**/*"
     end
@@ -77,9 +80,7 @@ task :jar, [:dir] => [:unpack, :clean_jar] do |t, args|
   ant.jar :destfile => TARGET_CLUSTER_JAR do
     fileset :dir => TARGET_CLASSES_DIR
     fileset :dir => TARGET_DEPENDENCY_UNPACKED_DIR
-    fileset :dir => RedStorm::REDSTORM_HOME do
-      include :name => "examples/**/*"
-    end
+    fileset :dir => TARGET_GEMS_DIR
     fileset :dir => JRUBY_SRC_DIR do
       exclude :name => "tasks/**"
     end

data/pom.xml

@@ -4,7 +4,7 @@
 
   <groupId>redstorm</groupId>
   <artifactId>redstorm</artifactId>
-  <version>0.2.0</version>
+  <version>0.2.1</version>
   <name>RedStorm JRuby on Storm</name>
 
   <properties>

metadata

@@ -2,7 +2,7 @@
 name: redstorm
 version: !ruby/object:Gem::Version 
   prerelease: 
-  version: 0.2.0
+  version: 0.2.1
 platform: ruby
 authors: 
   - Colin Surprenant
@@ -10,7 +10,7 @@ autorequire:
 bindir: bin
 cert_chain: []
 
-date: 2011-11-17 00:00:00 Z
+date: 2011-11-23 00:00:00 Z
 dependencies: 
   - !ruby/object:Gem::Dependency 
     name: rubyforge
@@ -90,6 +90,7 @@ files:
   - README.md
   - CHANGELOG.md
   - LICENSE.md
+  - TODO.md
 homepage: https://github.com/colinsurprenant/redstorm
 licenses: []