redstorm 0.6.6.beta2 → 0.6.6

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
@@ -91,7 +91,10 @@
91
91
  - improved DSL performance by refactoring closures invocation, see [redstorm-benchmark](https://github.com/colinsurprenant/redstorm-benchmark/)
92
92
  - fixed FFI support
93
93
  - exposed topology builder for specs
94
- - [issue #11](https://github.com/colinsurprenant/redstorm/issues/11), [#17](https://github.com/colinsurprenant/redstorm/issues/17) - example specs using the Storm testing API see [redstorm-starter](https://github.com/colinsurprenant/redstorm-starter/)
94
+ - [issue #11](https://github.com/colinsurprenant/redstorm/issues/11), [#17](https://github.com/colinsurprenant/redstorm/issues/17) - example specs using the Storm testing API see [redstorm-starter](https://github.com/colinsurprenant/redsto rm-starter/)
95
95
 
96
96
  # 0.6.6.beta2, 07-20-2013
97
- - [issue #76](https://github.com/colinsurprenant/redstorm/issues/76) - avoid shelling out to storm jar command for cluster submission
97
+ - [issue #76](https://github.com/colinsurprenant/redstorm/issues/76) - avoid shelling out to storm jar command for cluster submission
98
+
99
+ # 0.6.6, 07-25-2013
100
+ - updated example Kafka topology for new dependencies for Storm KafkaSpout
data/README.md CHANGED
@@ -1,4 +1,4 @@
1
- # RedStorm v0.6.6.beta2 - JRuby on Storm
1
+ # RedStorm - JRuby on Storm
2
2
 
3
3
  [![build status](https://secure.travis-ci.org/colinsurprenant/redstorm.png)](http://travis-ci.org/colinsurprenant/redstorm)
4
4
 
@@ -13,11 +13,12 @@ Check also these related projects:
13
13
 
14
14
  ## Documentation
15
15
 
16
+ <!--
16
17
  ---
17
- This is the documentation for the **current WIP version of RedStorm** - the **[latest released Gem is v0.6.5](wiki/RedStorm-Gem-v0.6.5-Documentation)**
18
+ This is the documentation for the **current 0.6.6-beta2 version of RedStorm** - the **[latest released Gem is v0.6.5](https://github.com/colinsurprenant/redstorm/wiki/RedStorm-Gem-v0.6.5-Documentation)**
18
19
 
19
20
  ---
20
-
21
+ -->
21
22
  Chances are new versions of RedStorm will introduce changes that will break compatibility or change the developement workflow. To prevent out-of-sync documentation, per version specific documentation are kept [in the wiki](https://github.com/colinsurprenant/redstorm/wiki) when necessary.
22
23
 
23
24
  ## Dependencies
@@ -29,14 +30,14 @@ Tested on **OSX 10.8.3** and **Ubuntu Linux 12.10** using **Storm 0.9.0-wip16**
29
30
  - RubyGems
30
31
 
31
32
  ``` sh
32
- $ gem install redstorm --pre
33
+ $ gem install redstorm
33
34
  ```
34
35
 
35
36
  - Bundler
36
37
 
37
38
  ``` ruby
38
39
  source "https://rubygems.org"
39
- gem "redstorm", "~> 0.6.6.beta2"
40
+ gem "redstorm", "~> 0.6.6"
40
41
  ```
41
42
 
42
43
  ## Usage
@@ -140,7 +141,7 @@ You can supply custom `storm` and `topology` dependencies by creating `ivy/storm
140
141
 
141
142
  ``` xml
142
143
  <?xml version="1.0"?>
143
- <ivy-module version="2.0">
144
+ <ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven">
144
145
  <info organisation="redstorm" module="storm-deps"/>
145
146
  <dependencies>
146
147
  <dependency org="storm" name="storm" rev="0.9.0-wip16" conf="default" transitive="true" />
@@ -153,10 +154,18 @@ You can supply custom `storm` and `topology` dependencies by creating `ivy/storm
153
154
 
154
155
  ``` xml
155
156
  <?xml version="1.0"?>
156
- <ivy-module version="2.0">
157
+ <ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven">
157
158
  <info organisation="redstorm" module="topology-deps"/>
158
159
  <dependencies>
159
160
  <dependency org="org.jruby" name="jruby-core" rev="1.7.4" conf="default" transitive="true"/>
161
+
162
+ <!-- explicitely specify jffi to also fetch the native jar. make sure to update jffi version matching jruby-core version -->
163
+ <!-- this is the only way I found using Ivy to fetch the native jar -->
164
+ <dependency org="com.github.jnr" name="jffi" rev="1.2.5" conf="default" transitive="true">
165
+ <artifact name="jffi" type="jar" />
166
+ <artifact name="jffi" type="jar" m:classifier="native"/>
167
+ </dependency>
168
+
160
169
  </dependencies>
161
170
  </ivy-module>
162
171
  ```
@@ -172,9 +181,9 @@ The jars repositories can be configured by adding the `ivy/settings.xml` file in
172
181
  <resolvers>
173
182
  <chain name="repositories">
174
183
  <ibiblio name="ibiblio" m2compatible="true"/>
175
- <ibiblio name="maven2" root="http://repo.maven.apache.org/maven2/" m2compatible="true"/>
176
184
  <ibiblio name="sonatype" root="http://repo.maven.apache.org/maven2/" m2compatible="true"/>
177
185
  <ibiblio name="clojars" root="http://clojars.org/repo/" m2compatible="true"/>
186
+ <ibiblio name="conjars" root="http://conjars.org/repo/" m2compatible="true"/>
178
187
  </chain>
179
188
  </resolvers>
180
189
  </ivysettings>
@@ -189,7 +198,7 @@ $ redstorm local <sources_directory_path/topology_class_file_name.rb>
189
198
  note that the topology can also be launched with the following command:
190
199
 
191
200
  ``` sh
192
- $ java -Djruby.compat.version=RUBY1_9 -cp "target/classes:target/dependency/storm/default/*:target/dependency/topology/default/*:<sources_directory_path>" redstorm.TopologyLauncher local <sources_directory_path/topology_class_file_name.rb>
201
+ $ java -cp "target/classes:target/dependency/storm/default/*:target/dependency/topology/default/*:<sources_directory_path>" redstorm.TopologyLauncher local <sources_directory_path/topology_class_file_name.rb>
193
202
  ```
194
203
 
195
204
  **See examples below** to run examples in local mode or on a production cluster.
@@ -1,6 +1,11 @@
1
1
  #!/usr/bin/env ruby
2
2
 
3
- require 'rubygems'
3
+ unless RUBY_PLATFORM =~ /java/
4
+ puts("error: RedStorm requires JRuby. Check http://jruby.org/ or https://github.com/sstephenson/rbenv or https://rvm.io/")
5
+ exit(1)
6
+ end
7
+
8
+ require 'rubygems'
4
9
  require 'rake'
5
10
 
6
11
  begin
@@ -1,7 +1,6 @@
1
- java_import 'storm.kafka.KafkaConfig'
2
1
  java_import 'storm.kafka.SpoutConfig'
3
- java_import 'storm.kafka.StringScheme'
4
2
  java_import 'storm.kafka.KafkaSpout'
3
+ java_import 'storm.kafka.KafkaConfig'
5
4
 
6
5
  require 'red_storm'
7
6
 
@@ -12,28 +11,37 @@ require 'red_storm'
12
11
  # dependencies in the "ivy/topology_dependencies.xml" file in the root of your RedStorm project:
13
12
  #
14
13
  # <?xml version="1.0"?>
15
- # <ivy-module version="2.0">
14
+ # <ivy-module version="2.0" xmlns:m="http://ant.apache.org/ivy/maven">
16
15
  # <info organisation="redstorm" module="topology-deps"/>
17
16
  # <dependencies>
18
17
  # <dependency org="org.jruby" name="jruby-core" rev="1.7.4" conf="default" transitive="true"/>
19
- # <dependency org="org.scala-lang" name="scala-library" rev="2.8.0" conf="default" transitive="false"/>
20
- # <dependency org="storm" name="kafka" rev="0.7.0-incubating" conf="default" transitive="false"/>
21
- # <dependency org="storm" name="storm-kafka" rev="0.8.0-wip4" conf="default" transitive="false"/>
18
+
19
+ # <dependency org="org.scala-lang" name="scala-library" rev="2.9.2" conf="default" transitive="false"/>
20
+ # <dependency org="com.twitter" name="kafka_2.9.2" rev="0.7.0" conf="default" transitive="false"/>
21
+ # <dependency org="storm" name="storm-kafka" rev="0.9.0-wip16a-scala292" conf="default" transitive="true"/>
22
+
23
+ # <!-- explicitely specify jffi to also fetch the native jar. make sure to update jffi version matching jruby-core version -->
24
+ # <!-- this is the only way I found using Ivy to fetch the native jar -->
25
+ # <dependency org="com.github.jnr" name="jffi" rev="1.2.5" conf="default" transitive="true">
26
+ # <artifact name="jffi" type="jar" />
27
+ # <artifact name="jffi" type="jar" m:classifier="native"/>
28
+ # </dependency>
29
+
22
30
  # </dependencies>
23
31
  # </ivy-module>
24
32
 
33
+ class SplitStringBolt < RedStorm::DSL::Bolt
34
+ on_receive {|tuple| tuple[0].split.map{|w| [w]}}
35
+ end
36
+
25
37
  class KafkaTopology < RedStorm::DSL::Topology
38
+
26
39
  spout_config = SpoutConfig.new(
27
40
  KafkaConfig::ZkHosts.new("localhost:2181", "/brokers"),
28
41
  "words", # topic to read from
29
- "/kafkastorm", # Zookeeper root path to store the consumer offsets
30
- "discovery" # Zookeeper consumer id to store the consumer offsets
42
+ "/kafkaspout", # Zookeeper root path to store the consumer offsets
43
+ "someid" # Zookeeper consumer id to store the consumer offsets
31
44
  )
32
- spout_config.scheme = StringScheme.new
33
-
34
- class SplitStringBolt < RedStorm::DSL::Bolt
35
- on_receive {|tuple| tuple[0].split.map{|w| [w]}}
36
- end
37
45
 
38
46
  spout KafkaSpout, [spout_config]
39
47
 
@@ -38,7 +38,7 @@ module RedStorm
38
38
  class RedisWordCountTopology < DSL::Topology
39
39
  spout RedisWordSpout
40
40
 
41
- bolt WordCountBolt, :parallelism => 3 do
41
+ bolt WordCountBolt, :parallelism => 2 do
42
42
  source RedisWordSpout, :fields => ["word"]
43
43
  end
44
44
 
@@ -46,10 +46,10 @@ module RedStorm
46
46
  debug true
47
47
  case env
48
48
  when :local
49
- max_task_parallelism 3
49
+ max_task_parallelism 2
50
50
  when :cluster
51
- max_task_parallelism 5
52
- num_workers 20
51
+ max_task_parallelism 2
52
+ num_workers 2
53
53
  max_spout_pending(1000)
54
54
  end
55
55
  end
@@ -4,9 +4,9 @@
4
4
  <resolvers>
5
5
  <chain name="repositories">
6
6
  <ibiblio name="ibiblio" m2compatible="true"/>
7
- <ibiblio name="maven2" root="http://repo.maven.apache.org/maven2/" m2compatible="true"/>
8
- <ibiblio name="sonatype" root="http://repo.maven.apache.org/maven2/" m2compatible="true"/>
9
- <ibiblio name="clojars" root="http://clojars.org/repo/" m2compatible="true"/>
7
+ <ibiblio name="sonatype" root="http://repo.maven.apache.org/maven2/" m2compatible="true"/>
8
+ <ibiblio name="clojars" root="http://clojars.org/repo/" m2compatible="true"/>
9
+ <ibiblio name="conjars" root="http://conjars.org/repo/" m2compatible="true"/>
10
10
  </chain>
11
11
  </resolvers>
12
12
  </ivysettings>
@@ -1,33 +1,6 @@
1
1
  require 'red_storm/version'
2
2
  require 'red_storm/environment'
3
3
 
4
- CWD = Dir.pwd
5
- TARGET_DIR = "#{CWD}/target"
6
- TARGET_LIB_DIR = "#{TARGET_DIR}/lib"
7
- TARGET_SRC_DIR = "#{TARGET_DIR}/src"
8
- TARGET_GEM_DIR = "#{TARGET_DIR}/gems/gems"
9
- TARGET_SPECS_DIR = "#{TARGET_DIR}/gems/specifications"
10
- TARGET_CLASSES_DIR = "#{TARGET_DIR}/classes"
11
- TARGET_DEPENDENCY_DIR = "#{TARGET_DIR}/dependency"
12
- TARGET_DEPENDENCY_UNPACKED_DIR = "#{TARGET_DIR}/dependency-unpacked"
13
- TARGET_CLUSTER_JAR = "#{TARGET_DIR}/cluster-topology.jar"
14
-
15
- REDSTORM_JAVA_SRC_DIR = "#{RedStorm::REDSTORM_HOME}/src/main"
16
- REDSTORM_LIB_DIR = "#{RedStorm::REDSTORM_HOME}/lib"
17
-
18
- SRC_EXAMPLES = "#{RedStorm::REDSTORM_HOME}/examples"
19
- DST_EXAMPLES = "#{CWD}/examples"
20
-
21
- SRC_IVY_DIR = "#{RedStorm::REDSTORM_HOME}/ivy"
22
- DST_IVY_DIR = "#{CWD}/ivy"
23
- DEFAULT_IVY_SETTINGS = "#{SRC_IVY_DIR}/settings.xml"
24
- CUSTOM_IVY_SETTINGS = "#{DST_IVY_DIR}/settings.xml"
25
- DEFAULT_IVY_STORM_DEPENDENCIES = "#{SRC_IVY_DIR}/storm_dependencies.xml"
26
- CUSTOM_IVY_STORM_DEPENDENCIES = "#{DST_IVY_DIR}/storm_dependencies.xml"
27
- DEFAULT_IVY_TOPOLOGY_DEPENDENCIES = "#{SRC_IVY_DIR}/topology_dependencies.xml"
28
- CUSTOM_IVY_TOPOLOGY_DEPENDENCIES = "#{DST_IVY_DIR}/topology_dependencies.xml"
29
-
30
- DEFAULT_STORM_CONF_FILE = File.expand_path("~/.storm/storm.yaml")
31
4
 
32
5
  module RedStorm
33
6
 
@@ -2,30 +2,57 @@ require 'java'
2
2
  java_import 'java.lang.System'
3
3
 
4
4
  module RedStorm
5
+ CWD = Dir.pwd
5
6
 
6
- LAUNCH_PATH = File.expand_path(File.dirname(__FILE__))
7
- JAR_CONTEXT = !!(LAUNCH_PATH =~ /\.jar!\/red_storm$/)
7
+ launch_path = File.expand_path(File.dirname(__FILE__))
8
+ jar_context = !!(launch_path =~ /\.jar!\/red_storm$/)
8
9
 
9
- if JAR_CONTEXT
10
- BASE_PATH = File.expand_path(LAUNCH_PATH + '/..')
10
+ if jar_context
11
+ BASE_PATH = File.expand_path(launch_path + '/..')
11
12
  REDSTORM_HOME = BASE_PATH
12
- TARGET_PATH = BASE_PATH
13
+ GEM_PATH = "#{REDSTORM_HOME}/gems/"
13
14
  else
14
- BASE_PATH = Dir.pwd
15
- REDSTORM_HOME = File.expand_path(LAUNCH_PATH + '/../..')
16
- TARGET_PATH = "#{BASE_PATH}/target"
15
+ BASE_PATH = CWD
16
+ REDSTORM_HOME = File.expand_path(launch_path + '/../..')
17
+ GEM_PATH = "#{BASE_PATH}/target/gems/"
17
18
  end
18
19
 
19
20
  unless defined?(SPECS_CONTEXT)
20
- GEM_PATH = "#{TARGET_PATH}/gems/"
21
21
  ENV["GEM_PATH"] = GEM_PATH
22
22
  ENV["GEM_HOME"] = GEM_PATH
23
23
  end
24
24
 
25
+ TARGET_DIR = "#{CWD}/target"
26
+ TARGET_LIB_DIR = "#{TARGET_DIR}/lib"
27
+ TARGET_SRC_DIR = "#{TARGET_DIR}/src"
28
+ TARGET_GEM_DIR = "#{TARGET_DIR}/gems/gems"
29
+ TARGET_SPECS_DIR = "#{TARGET_DIR}/gems/specifications"
30
+ TARGET_CLASSES_DIR = "#{TARGET_DIR}/classes"
31
+ TARGET_DEPENDENCY_DIR = "#{TARGET_DIR}/dependency"
32
+ TARGET_DEPENDENCY_UNPACKED_DIR = "#{TARGET_DIR}/dependency-unpacked"
33
+ TARGET_CLUSTER_JAR = "#{TARGET_DIR}/cluster-topology.jar"
34
+
35
+ REDSTORM_JAVA_SRC_DIR = "#{REDSTORM_HOME}/src/main"
36
+ REDSTORM_LIB_DIR = "#{REDSTORM_HOME}/lib"
37
+
38
+ SRC_EXAMPLES = "#{REDSTORM_HOME}/examples"
39
+ DST_EXAMPLES = "#{CWD}/examples"
40
+
41
+ SRC_IVY_DIR = "#{REDSTORM_HOME}/ivy"
42
+ DST_IVY_DIR = "#{CWD}/ivy"
43
+ DEFAULT_IVY_SETTINGS = "#{SRC_IVY_DIR}/settings.xml"
44
+ CUSTOM_IVY_SETTINGS = "#{DST_IVY_DIR}/settings.xml"
45
+ DEFAULT_IVY_STORM_DEPENDENCIES = "#{SRC_IVY_DIR}/storm_dependencies.xml"
46
+ CUSTOM_IVY_STORM_DEPENDENCIES = "#{DST_IVY_DIR}/storm_dependencies.xml"
47
+ DEFAULT_IVY_TOPOLOGY_DEPENDENCIES = "#{SRC_IVY_DIR}/topology_dependencies.xml"
48
+ CUSTOM_IVY_TOPOLOGY_DEPENDENCIES = "#{DST_IVY_DIR}/topology_dependencies.xml"
49
+
50
+ DEFAULT_STORM_CONF_FILE = File.expand_path("~/.storm/storm.yaml")
51
+
25
52
  def current_ruby_mode
26
- RUBY_VERSION =~ /(\d+\.\d+)(\.\d+)*/
27
- raise("unknown Ruby version #{$1}") unless $1 == "1.8" || $1 == "1.9"
28
- $1
53
+ version = RUBY_VERSION[/(\d+\.\d+)(\.\d+)*/, 1]
54
+ raise("unknown Ruby version #{$1}") unless ["1.8", "1.9"].include?(version)
55
+ version
29
56
  end
30
57
 
31
58
  def jruby_mode_token(ruby_version = nil)
@@ -34,7 +61,7 @@ module RedStorm
34
61
  end
35
62
 
36
63
  def java_runtime_version
37
- System.properties["java.runtime.version"].to_s =~ /^(\d+\.\d+).[^\s]+$/ ? $1 : "1.7"
64
+ System.properties["java.runtime.version"].to_s[/^(\d+\.\d+).[^\s]+$/, 1] || raise("unknown java runtime version #{System.properties["java.runtime.version"].to_s}")
38
65
  end
39
66
 
40
67
  module_function :current_ruby_mode, :jruby_mode_token, :java_runtime_version
@@ -1,3 +1,3 @@
1
1
  module RedStorm
2
- VERSION = '0.6.6.beta2'
2
+ VERSION = '0.6.6'
3
3
  end
@@ -1,8 +1,8 @@
1
1
  begin
2
2
  require 'ant'
3
3
  rescue
4
- puts("ERROR: unable to load Ant, make sure Ant is installed, in your PATH and $ANT_HOME is defined properly")
5
- puts("\nerror detail:\n#{$!}")
4
+ puts("error: unable to load Ant, make sure Ant is installed, in your PATH and $ANT_HOME is defined properly")
5
+ puts("\nerror details:\n#{$!}")
6
6
  exit(1)
7
7
  end
8
8
 
@@ -10,7 +10,9 @@ require 'jruby/jrubyc'
10
10
  require 'red_storm/environment'
11
11
  require 'red_storm/application'
12
12
 
13
- INSTALL_IVY_VERSION = "2.3.0"
13
+ module RedStorm
14
+ INSTALL_IVY_VERSION = "2.3.0"
15
+ end
14
16
 
15
17
  task :launch, :env, :storm_conf, :ruby_mode, :class_file do |t, args|
16
18
  # use ruby mode parameter or default to current interpreter version
@@ -20,8 +22,8 @@ task :launch, :env, :storm_conf, :ruby_mode, :class_file do |t, args|
20
22
  when "local"
21
23
  RedStorm::Application.local_storm_command(args[:class_file], args[:ruby_mode])
22
24
  when "cluster"
23
- unless File.exist?(TARGET_CLUSTER_JAR)
24
- puts("error: cluster jar file #{TARGET_CLUSTER_JAR} not found. Generate it using $redstorm jar DIR1 [DIR2, ...]")
25
+ unless File.exist?(RedStorm::TARGET_CLUSTER_JAR)
26
+ puts("error: cluster jar file #{RedStorm::TARGET_CLUSTER_JAR} not found. Generate it using $redstorm jar DIR1 [DIR2, ...]")
25
27
  exit(1)
26
28
  end
27
29
  unless File.exist?(args[:storm_conf])
@@ -38,50 +40,50 @@ task :launch, :env, :storm_conf, :ruby_mode, :class_file do |t, args|
38
40
  end
39
41
 
40
42
  task :clean do
41
- ant.delete 'dir' => TARGET_DIR
43
+ ant.delete 'dir' => RedStorm::TARGET_DIR
42
44
  end
43
45
 
44
46
  task :clean_jar do
45
- ant.delete 'file' => TARGET_CLUSTER_JAR
47
+ ant.delete 'file' => RedStorm::TARGET_CLUSTER_JAR
46
48
  end
47
49
 
48
50
  task :setup do
49
51
  puts("\n--> Setting up target directories")
50
- ant.mkdir 'dir' => TARGET_DIR
51
- ant.mkdir 'dir' => TARGET_CLASSES_DIR
52
- ant.mkdir 'dir' => TARGET_DEPENDENCY_DIR
53
- ant.mkdir 'dir' => TARGET_SRC_DIR
54
- ant.mkdir 'dir' => TARGET_GEM_DIR
55
- ant.mkdir 'dir' => TARGET_SPECS_DIR
52
+ ant.mkdir 'dir' => RedStorm::TARGET_DIR
53
+ ant.mkdir 'dir' => RedStorm::TARGET_CLASSES_DIR
54
+ ant.mkdir 'dir' => RedStorm::TARGET_DEPENDENCY_DIR
55
+ ant.mkdir 'dir' => RedStorm::TARGET_SRC_DIR
56
+ ant.mkdir 'dir' => RedStorm::TARGET_GEM_DIR
57
+ ant.mkdir 'dir' => RedStorm::TARGET_SPECS_DIR
56
58
  ant.path 'id' => 'classpath' do
57
- fileset 'dir' => TARGET_DEPENDENCY_DIR
58
- fileset 'dir' => TARGET_CLASSES_DIR
59
+ fileset 'dir' => RedStorm::TARGET_DEPENDENCY_DIR
60
+ fileset 'dir' => RedStorm::TARGET_CLASSES_DIR
59
61
  end
60
62
  end
61
63
 
62
64
  desc "install dependencies and compile proxy classes"
63
65
  task :install, [:jvm_version] => [:deps, :build] do |t, args|
64
- puts("\nRedStorm install completed. All dependencies installed in #{TARGET_DIR}")
66
+ puts("\nRedStorm install completed. All dependencies installed in #{RedStorm::TARGET_DIR}")
65
67
  end
66
68
 
67
69
  desc "locally install examples"
68
70
  task :examples do
69
- if File.identical?(SRC_EXAMPLES, DST_EXAMPLES)
71
+ if File.identical?(RedStorm::SRC_EXAMPLES, RedStorm::DST_EXAMPLES)
70
72
  STDERR.puts("error: cannot copy examples into itself")
71
73
  exit(1)
72
74
  end
73
- if File.exist?(DST_EXAMPLES)
74
- STDERR.puts("error: directory #{DST_EXAMPLES} already exists")
75
+ if File.exist?(RedStorm::DST_EXAMPLES)
76
+ STDERR.puts("error: directory #{RedStorm::DST_EXAMPLES} already exists")
75
77
  exit(1)
76
78
  end
77
79
 
78
- puts("\n--> Installing examples into #{DST_EXAMPLES}")
79
- FileUtils.mkdir(DST_EXAMPLES)
80
- FileUtils.cp_r(Dir["#{SRC_EXAMPLES}/*"], DST_EXAMPLES)
80
+ puts("\n--> Installing examples into #{RedStorm::DST_EXAMPLES}")
81
+ FileUtils.mkdir(RedStorm::DST_EXAMPLES)
82
+ FileUtils.cp_r(Dir["#{RedStorm::SRC_EXAMPLES}/*"], RedStorm::DST_EXAMPLES)
81
83
  end
82
84
 
83
85
  task :copy_red_storm do
84
- FileUtils.cp_r(REDSTORM_LIB_DIR, TARGET_DIR)
86
+ FileUtils.cp_r(RedStorm::REDSTORM_LIB_DIR, RedStorm::TARGET_DIR)
85
87
  end
86
88
 
87
89
  desc "compile JRuby and Java proxy classes"
@@ -89,22 +91,22 @@ task :build, [:jvm_version] => [:setup, :copy_red_storm] do |t, args|
89
91
  jvm_version = args[:jvm_version].to_s =~ /--(1.\d)/ ? $1 : RedStorm.java_runtime_version
90
92
 
91
93
  # compile the JRuby proxy classes to Java
92
- build_jruby("#{REDSTORM_LIB_DIR}/red_storm/proxy")
94
+ build_jruby("#{RedStorm::REDSTORM_LIB_DIR}/red_storm/proxy")
93
95
 
94
96
  # compile the generated Java proxy classes
95
- build_java_dir("#{TARGET_SRC_DIR}", jvm_version)
97
+ build_java_dir("#{RedStorm::TARGET_SRC_DIR}", jvm_version)
96
98
 
97
99
  # generate the JRuby topology launcher
98
- build_jruby("#{REDSTORM_LIB_DIR}/red_storm/topology_launcher.rb")
100
+ build_jruby("#{RedStorm::REDSTORM_LIB_DIR}/red_storm/topology_launcher.rb")
99
101
 
100
102
  # compile the JRuby proxy classes
101
- build_java_dir("#{REDSTORM_JAVA_SRC_DIR}", jvm_version)
103
+ build_java_dir("#{RedStorm::REDSTORM_JAVA_SRC_DIR}", jvm_version)
102
104
 
103
105
  # compile the JRuby proxy classes
104
- build_java_dir("#{TARGET_SRC_DIR}", jvm_version)
106
+ build_java_dir("#{RedStorm::TARGET_SRC_DIR}", jvm_version)
105
107
  end
106
108
 
107
- desc "package topology gems into #{TARGET_GEM_DIR}"
109
+ desc "package topology gems into #{RedStorm::TARGET_GEM_DIR}"
108
110
  task :bundle, [:groups] => :setup do |t, args|
109
111
  require 'bundler'
110
112
  defaulted_args = {:groups => 'default'}.merge(args.to_hash.delete_if{|k, v| v.to_s.empty?})
@@ -113,9 +115,9 @@ task :bundle, [:groups] => :setup do |t, args|
113
115
  next if spec.name == 'bundler'
114
116
 
115
117
  # try to avoid infinite recursion
116
- next if TARGET_GEM_DIR.start_with?(spec.full_gem_path)
118
+ next if RedStorm::TARGET_GEM_DIR.start_with?(spec.full_gem_path)
117
119
 
118
- destination_path = "#{TARGET_GEM_DIR}/#{spec.full_name}"
120
+ destination_path = "#{RedStorm::TARGET_GEM_DIR}/#{spec.full_name}"
119
121
  next if File.directory?(destination_path)
120
122
 
121
123
  puts("installing gem #{spec.full_name} into #{destination_path}")
@@ -124,7 +126,7 @@ task :bundle, [:groups] => :setup do |t, args|
124
126
  # copy the evaluated gemspec into the specifications/ dir (we
125
127
  # may not have enough info to reconstruct once we delete the
126
128
  # .git directory)
127
- File.open(File.join(TARGET_SPECS_DIR, File.basename(spec.loaded_from)), 'w'){|f| f.write(spec.to_ruby)}
129
+ File.open(File.join(RedStorm::TARGET_SPECS_DIR, File.basename(spec.loaded_from)), 'w'){|f| f.write(spec.to_ruby)}
128
130
  # strip the .git directory from git dependencies, it can be huge
129
131
  FileUtils.rm_rf("#{destination_path}/.git")
130
132
  end
@@ -132,17 +134,17 @@ end
132
134
 
133
135
  namespace :ivy do
134
136
  task :download do
135
- mkdir_p DST_IVY_DIR
137
+ mkdir_p RedStorm::DST_IVY_DIR
136
138
  ant.get({
137
- 'src' => "http://repo1.maven.org/maven2/org/apache/ivy/ivy/#{INSTALL_IVY_VERSION}/ivy-#{INSTALL_IVY_VERSION}.jar",
138
- 'dest' => "#{DST_IVY_DIR}/ivy-#{INSTALL_IVY_VERSION}.jar",
139
+ 'src' => "http://repo1.maven.org/maven2/org/apache/ivy/ivy/#{RedStorm::INSTALL_IVY_VERSION}/ivy-#{RedStorm::INSTALL_IVY_VERSION}.jar",
140
+ 'dest' => "#{RedStorm::DST_IVY_DIR}/ivy-#{RedStorm::INSTALL_IVY_VERSION}.jar",
139
141
  'usetimestamp' => true,
140
142
  })
141
143
  end
142
144
 
143
145
  task :install => :download do
144
146
  ant.path 'id' => 'ivy.lib.path' do
145
- fileset 'dir' => DST_IVY_DIR, 'includes' => '*.jar'
147
+ fileset 'dir' => RedStorm::DST_IVY_DIR, 'includes' => '*.jar'
146
148
  end
147
149
 
148
150
  ant.taskdef({
@@ -154,44 +156,44 @@ namespace :ivy do
154
156
  end
155
157
 
156
158
  task :ivy_config do
157
- ant.configure 'file' => File.exists?(CUSTOM_IVY_SETTINGS) ? CUSTOM_IVY_SETTINGS : DEFAULT_IVY_SETTINGS
159
+ ant.configure 'file' => File.exists?(RedStorm::CUSTOM_IVY_SETTINGS) ? RedStorm::CUSTOM_IVY_SETTINGS : RedStorm::DEFAULT_IVY_SETTINGS
158
160
  end
159
161
 
160
162
  task :storm_deps => ["ivy:install", :ivy_config] do
161
163
  puts("\n--> Installing Storm dependencies")
162
164
 
163
- ant.resolve 'file' => File.exists?(CUSTOM_IVY_STORM_DEPENDENCIES) ? CUSTOM_IVY_STORM_DEPENDENCIES : DEFAULT_IVY_STORM_DEPENDENCIES
164
- ant.retrieve 'pattern' => "#{TARGET_DEPENDENCY_DIR}/storm/[conf]/[artifact](-[classifier])-[revision].[ext]", 'sync' => "true"
165
+ ant.resolve 'file' => File.exists?(RedStorm::CUSTOM_IVY_STORM_DEPENDENCIES) ? RedStorm::CUSTOM_IVY_STORM_DEPENDENCIES : RedStorm::DEFAULT_IVY_STORM_DEPENDENCIES
166
+ ant.retrieve 'pattern' => "#{RedStorm::TARGET_DEPENDENCY_DIR}/storm/[conf]/[artifact](-[classifier])-[revision].[ext]", 'sync' => "true"
165
167
  end
166
168
 
167
169
  task :topology_deps => ["ivy:install", :ivy_config] do
168
170
  puts("\n--> Installing topology dependencies")
169
171
 
170
- ant.resolve 'file' => File.exists?(CUSTOM_IVY_TOPOLOGY_DEPENDENCIES) ? CUSTOM_IVY_TOPOLOGY_DEPENDENCIES : DEFAULT_IVY_TOPOLOGY_DEPENDENCIES
171
- ant.retrieve 'pattern' => "#{TARGET_DEPENDENCY_DIR}/topology/[conf]/[artifact](-[classifier])-[revision].[ext]", 'sync' => "true"
172
+ ant.resolve 'file' => File.exists?(RedStorm::CUSTOM_IVY_TOPOLOGY_DEPENDENCIES) ? RedStorm::CUSTOM_IVY_TOPOLOGY_DEPENDENCIES : RedStorm::DEFAULT_IVY_TOPOLOGY_DEPENDENCIES
173
+ ant.retrieve 'pattern' => "#{RedStorm::TARGET_DEPENDENCY_DIR}/topology/[conf]/[artifact](-[classifier])-[revision].[ext]", 'sync' => "true"
172
174
  end
173
175
 
174
- desc "install storm and topology dependencies in #{TARGET_DEPENDENCY_DIR}"
176
+ desc "install storm and topology dependencies in #{RedStorm::TARGET_DEPENDENCY_DIR}"
175
177
  task :deps => ["ivy:install", :ivy_config, :storm_deps, :topology_deps] do
176
178
  end
177
179
 
178
- desc "generate #{TARGET_CLUSTER_JAR}"
180
+ desc "generate #{RedStorm::TARGET_CLUSTER_JAR}"
179
181
  task :jar, [:include_dir] => [:clean_jar] do |t, args|
180
- puts("\n--> Generating JAR file #{TARGET_CLUSTER_JAR}")
182
+ puts("\n--> Generating JAR file #{RedStorm::TARGET_CLUSTER_JAR}")
181
183
 
182
- ant.jar 'destfile' => TARGET_CLUSTER_JAR do
184
+ ant.jar 'destfile' => RedStorm::TARGET_CLUSTER_JAR do
183
185
  # rejar all topology jars
184
186
  Dir["target/dependency/topology/default/*.jar"].each do |jar|
185
187
  puts("Extracting #{jar}")
186
188
  zipfileset 'src' => jar, 'includes' => "**/*"
187
189
  end
188
- fileset 'dir' => TARGET_DIR do
190
+ fileset 'dir' => RedStorm::TARGET_DIR do
189
191
  include 'name' => "gems/**"
190
192
  end
191
- fileset 'dir' => TARGET_CLASSES_DIR
193
+ fileset 'dir' => RedStorm::TARGET_CLASSES_DIR
192
194
  # red_storm.rb and red_storm/* must be in root of jar so that "require 'red_storm'"
193
195
  # in bolts/spouts works in jar context
194
- fileset 'dir' => TARGET_LIB_DIR do
196
+ fileset 'dir' => RedStorm::TARGET_LIB_DIR do
195
197
  exclude 'name' => "tasks/**"
196
198
  end
197
199
  if args[:include_dir]
@@ -209,7 +211,7 @@ task :jar, [:include_dir] => [:clean_jar] do |t, args|
209
211
  end
210
212
 
211
213
  # include complete source dir tree (note we don't care about potential duplicated resources dir)
212
- fileset 'dir' => CWD do
214
+ fileset 'dir' => RedStorm::CWD do
213
215
  dirs.each{|dir| include 'name' => "#{dir}/**/*"}
214
216
  end
215
217
  end
@@ -217,14 +219,14 @@ task :jar, [:include_dir] => [:clean_jar] do |t, args|
217
219
  attribute 'name' => "Main-Class", 'value' => "redstorm.TopologyLauncher"
218
220
  end
219
221
  end
220
- puts("\nRedStorm generated JAR file #{TARGET_CLUSTER_JAR}")
222
+ puts("\nRedStorm generated JAR file #{RedStorm::TARGET_CLUSTER_JAR}")
221
223
  end
222
224
 
223
225
  def build_java_dir(source_folder, jvm_version)
224
226
  puts("\n--> Compiling Java for JVM #{jvm_version}")
225
227
  ant.javac(
226
228
  'srcdir' => source_folder,
227
- 'destdir' => TARGET_CLASSES_DIR,
229
+ 'destdir' => RedStorm::TARGET_CLASSES_DIR,
228
230
  'classpathref' => 'classpath',
229
231
  'source' => jvm_version,
230
232
  'target' => jvm_version,
@@ -242,13 +244,13 @@ def build_jruby(source_path)
242
244
  puts("\n--> Compiling JRuby")
243
245
  Dir.chdir(RedStorm::REDSTORM_HOME) do
244
246
  argv = []
245
- argv << '-t' << TARGET_SRC_DIR
247
+ argv << '-t' << RedStorm::TARGET_SRC_DIR
246
248
  argv << '--verbose'
247
249
  argv << '--java'
248
- Dir["#{TARGET_DEPENDENCY_DIR}/storm/default/*.jar"].each do |jar|
250
+ Dir["#{RedStorm::TARGET_DEPENDENCY_DIR}/storm/default/*.jar"].each do |jar|
249
251
  argv << '-c' << %("#{jar}")
250
252
  end
251
- argv << '-c' << %("#{TARGET_CLASSES_DIR}")
253
+ argv << '-c' << %("#{RedStorm::TARGET_CLASSES_DIR}")
252
254
  argv << source_path
253
255
  status = JRuby::Compiler::compile_argv(argv)
254
256
  end
metadata CHANGED
@@ -1,15 +1,15 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: redstorm
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.6.6.beta2
5
- prerelease: 6
4
+ version: 0.6.6
5
+ prerelease:
6
6
  platform: ruby
7
7
  authors:
8
8
  - Colin Surprenant
9
9
  autorequire:
10
10
  bindir: bin
11
11
  cert_chain: []
12
- date: 2013-07-20 00:00:00.000000000 Z
12
+ date: 2013-07-25 00:00:00.000000000 Z
13
13
  dependencies:
14
14
  - !ruby/object:Gem::Dependency
15
15
  name: rspec
@@ -131,9 +131,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
131
131
  none: false
132
132
  required_rubygems_version: !ruby/object:Gem::Requirement
133
133
  requirements:
134
- - - '>'
134
+ - - '>='
135
135
  - !ruby/object:Gem::Version
136
- version: 1.3.1
136
+ version: '0'
137
137
  none: false
138
138
  requirements: []
139
139
  rubyforge_project: redstorm