redstorm 0.4.0 → 0.5.0

Sign up to get free protection for your applications and to get access to all the features.
data/CHANGELOG.md CHANGED
@@ -19,4 +19,14 @@
19
19
 
20
20
  # 0.4.0, 02-08-2012
21
21
  - Storm 0.6.2
22
- - JRuby 1.6.6
22
+ - JRuby 1.6.6
23
+
24
+ # 0.5.0, 05-28-2012
25
+ - issue #16, Bundler support for topology gems
26
+ - issue #19, support for multiple dirs in jar
27
+ - issue #20, update to Storm 0.7.1
28
+ - issue #21, proper support for 1.8/1.9 Ruby compatibility mode
29
+ - issue #22, fixed Config class name clash
30
+ - JRuby 1.6.7
31
+ - DSL Support for per bolt/spout configuration and spout activate/deactivate in Storm 0.7.x
32
+ - consistent workflow using the redstorm command in local dev or gem mode
data/README.md CHANGED
@@ -1,16 +1,47 @@
1
- # RedStorm v0.4.0 - JRuby on Storm
1
+ # RedStorm v0.5.0 - JRuby on Storm
2
2
 
3
- RedStorm provides the JRuby integration for the [Storm][storm] distributed realtime computation system.
3
+ [![build status](https://secure.travis-ci.org/colinsurprenant/redstorm.png)](http://travis-ci.org/colinsurprenant/redstorm)
4
+
5
+ RedStorm provides a Ruby DSL using JRuby integration for the [Storm][storm] distributed realtime computation system.
6
+
7
+ ## Documentation
8
+
9
+ Chances are new versions of RedStorm will introduce changes that will break compatibility or change the developement workflow. To prevent out-of-sync documentation, per version specific documentation are kept in the wiki when necessary. This README reflects the current/master developement state.
10
+
11
+ ### v0.4.x gem
12
+
13
+ - [RedStorm Gem v0.4.x Documentation](https://github.com/colinsurprenant/redstorm/wiki/RedStorm-Gem-v0.4.x-Documentation)
4
14
 
5
15
  ## Dependencies
6
16
 
7
- This has been tested on OSX 10.6.8 and Linux 10.04 using Storm 0.6.2 and JRuby 1.6.6
17
+ Tested on OSX 10.6.8 and Linux 10.04 using Storm 0.6.2 and JRuby 1.6.7
18
+
19
+ ## Notes about 1.8/1.9 JRuby compatibility
20
+
21
+ Up until the upcoming JRuby 1.7, JRuby runs in 1.8 Ruby compatibility mode by default. Unless you have a specific need to run topologies in 1.8 mode, you should use 1.9 mode, which will become the default in JRuby. Things are a bit tricky with Storm/RedStorm. There are 3 contexts where the Ruby compatibility mode has to be controlled.
22
+
23
+ - when installing the topology required gems. the installation path embeds the Ruby version
24
+ - when running in local mode or for the submission phase in remote/cluster mode
25
+ - when Storm runs the topology in remote/cluster mode
26
+
27
+ For each of these contexts, 1.9 mode has to be explicitly specified to avoid any problems. All commands/examples below will use the 1.9 compatibility mode. If you want to avoid the explicit --1.9 mode option, using [RVM][rvm] you can compile your JRuby to run in 1.9 mode by default. If you run your topology in remote/cluster mode, you will still need to include some bits of 1.9 options and configuration since in this case JRuby and your topology is run independently by Storm.
8
28
 
9
29
  ## Installation
30
+
31
+ ### Released v0.4.0 gem
10
32
  ``` sh
11
33
  $ gem install redstorm
12
34
  ```
13
35
 
36
+ ### From github master
37
+
38
+ - clone/fork project
39
+
40
+ ``` sh
41
+ $ gem build redstorm.gemspec
42
+ $ gem install redstorm-x.y.z.gem
43
+ ```
44
+
14
45
  ## Usage overview
15
46
 
16
47
  - create a new empty project directory.
@@ -21,34 +52,34 @@ $ gem install redstorm
21
52
 
22
53
  ### Initial setup
23
54
 
24
- - install RedStom dependencies; from your project root directory execute:
55
+ - install RedStom dependencies. From your project root directory execute:
25
56
 
26
- ``` sh
27
- $ redstorm install
28
- ```
57
+ ``` sh
58
+ $ redstorm --1.9 install
59
+ ```
29
60
 
30
- The `install` command will install all Java jars dependencies using [ruby-maven][ruby-maven] in `target/dependency` and generate & compile the Java bindings in `target/classes`
61
+ The `install` command will install all Java jars dependencies using [ruby-maven][ruby-maven] in `target/dependency`, generate & compile the Java bindings in `target/classes` and install gems in `target/gems`.
31
62
 
32
- ***DON'T PANIC*** it's Maven. The first time you run `$ redstorm install` Maven will take a few minutes resolving dependencies and in the end will download and install the dependency jar files.
63
+ ***DON'T PANIC*** it's Maven. The first time you run `$ redstorm --1.9 install` Maven will take a few minutes resolving dependencies and in the end will download and install the dependency jar files.
33
64
 
34
- - create a topology class. The *underscore* topology_class_file_name.rb **MUST** correspond to its *CamelCase* class name.
65
+ - create a topology class in your sources subdirectory. The *underscore* topology_class_file_name.rb **MUST** correspond to its *CamelCase* class name.
35
66
 
36
- ### Gems
67
+ ### Gems in your topology
37
68
 
38
- Until this is better integrated, you can use **gems** in local mode and on a production cluster:
69
+ RedStorm now support [Bundler](http://gembundler.com/) for using gems in your topology. Basically supply a `Gemfile` in the root of your project directory and execute this command to install the gems into the `target/gems` directory. **Note that if you change the Gemfile you must rerun this command**.
39
70
 
40
- - **local mode**: simply install your gems the usual way, they will be picked up when run in local mode.
71
+ ``` sh
72
+ $ redstorm --1.9 gems [--gemfile=GEMFILE]
73
+ ```
41
74
 
42
- - **production cluster**: install your gem in the `target/gems` folder using:
75
+ All `bundle install` command options can be passed as options to `redstorm --1.9 gem` like `--gemfile=GEMFILE` to specify a Gemfile in an alternate path.
43
76
 
44
- ```sh
45
- gem install <the gem> --install-dir target/gems/ --no-ri --no-rdoc
46
- ```
77
+ Basically, the `redstorm --1.9 gems` command installs the *Bundler* and *Rake* gems and all the gems specified in the Gemfile into the `target/gems` directory. The idea is that in order for the topology to run in a Storm cluster, everything, including the fully *installed* gems, must be packaged and self-contained into a single JAR file. This has an important consequence: the gems will not be *installed* on the cluster target machines, they are already *installed* in the JAR file. This could possibly lead to problems if the machine used to *install* the gems is of a different architecture than the cluster target machines **and** some of these gems have *native* C/FFI extensions.
47
78
 
48
79
  ### Run in local mode
49
80
 
50
81
  ``` sh
51
- $ redstorm local <path/to/topology_class_file_name.rb>
82
+ $ redstorm --1.9 local <path/to/topology_class_file_name.rb>
52
83
  ```
53
84
 
54
85
  **See examples below** to run examples in local mode or on a production cluster.
@@ -57,15 +88,17 @@ $ redstorm local <path/to/topology_class_file_name.rb>
57
88
 
58
89
  - generate `target/cluster-topology.jar`. This jar file will include your sources directory plus the required dependencies from the `target/` directory:
59
90
 
60
- ``` sh
61
- $ redstorm jar <sources_directory>
62
- ```
91
+ ``` sh
92
+ $ redstorm --1.9 jar <sources_directory1> <sources_directory2> ...
93
+ ```
63
94
 
64
95
  - submit the cluster topology jar file to the cluster. Assuming you have the Storm distribution installed and the Storm `bin/` directory in your path:
65
96
 
66
- ``` sh
67
- storm jar ./target/cluster-topology.jar redstorm.TopologyLauncher cluster <path/to/topology_class_file_name.rb>
68
- ```
97
+ ``` sh
98
+ storm jar ./target/cluster-topology.jar -Djruby.compat.version=RUBY1_9 redstorm.TopologyLauncher cluster <path/to/topology_class_file_name.rb>
99
+ ```
100
+
101
+ Note the **-Djruby.compat.version=RUBY1_9** parameter.
69
102
 
70
103
  Basically you must follow the [Storm instructions](https://github.com/nathanmarz/storm/wiki) to [setup a production cluster](https://github.com/nathanmarz/storm/wiki/Setting-up-a-Storm-cluster) and [submit your topology to the cluster](https://github.com/nathanmarz/storm/wiki/Running-topologies-on-a-production-cluster).
71
104
 
@@ -77,391 +110,119 @@ Install the [example files](https://github.com/colinsurprenant/redstorm/tree/mas
77
110
  $ redstorm examples
78
111
  ```
79
112
 
80
- All examples using the **simple DSL** are located in `examples/simple`. Examples using the standard Java interface are in `examples/native`.
113
+ All examples using the [simple DSL](https://github.com/colinsurprenant/redstorm/wiki/Ruby-DSL-Documentation) are located in `examples/simple`. Examples using the standard Java interface are in `examples/native`.
81
114
 
82
115
  ### Local mode
83
116
 
84
117
  ``` sh
85
- $ redstorm local examples/simple/exclamation_topology.rb
86
- $ redstorm local examples/simple/exclamation_topology2.rb
87
- $ redstorm local examples/simple/word_count_topology.rb
118
+ $ redstorm --1.9 local examples/simple/exclamation_topology.rb
119
+ $ redstorm --1.9 local examples/simple/exclamation_topology2.rb
120
+ $ redstorm --1.9 local examples/simple/word_count_topology.rb
121
+ ```
122
+
123
+ To run `examples/simple/redis_word_count_topology.rb` you need a [Redis][redis] server running on `localhost:6379`
124
+
125
+ ``` sh
126
+ $ redstorm --1.9 gems --gemfile examples/simple/Gemfile
88
127
  ```
89
128
 
90
- This next example requires the use of the [Redis Gem](https://github.com/ezmobius/redis-rb) and a [Redis][redis] server running on `localhost:6379`
129
+ Run the topology in local mode
91
130
 
92
131
  ``` sh
93
- $ redstorm local examples/simple/redis_word_count_topology.rb
132
+ $ redstorm --1.9 local examples/simple/redis_word_count_topology.rb
94
133
  ```
95
134
 
96
135
  Using `redis-cli`, push words into the `test` list and watch Storm pick them up
97
136
 
98
137
  ### Production cluster
99
138
 
100
- All examples using the **simple DSL** can also run on a productions cluster. The only **native** example compatible with a production cluster is the [ClusterWordCountTopology](https://github.com/colinsurprenant/redstorm/tree/master/examples/native/cluster_word_count_topology.rb)
139
+ All examples using the [simple DSL](https://github.com/colinsurprenant/redstorm/wiki/Ruby-DSL-Documentation) can also run on a productions cluster. The only **native** example compatible with a production cluster is the [ClusterWordCountTopology](https://github.com/colinsurprenant/redstorm/tree/master/examples/native/cluster_word_count_topology.rb)
101
140
 
102
141
  - genererate the `target/cluster-topology.jar` and include the `examples/` directory.
103
142
 
104
- ``` sh
105
- $ redstorm jar examples
106
- ```
143
+ ``` sh
144
+ $ redstorm --1.9 jar examples
145
+ ```
107
146
 
108
147
  - submit the cluster topology jar file to the cluster, assuming you have the Storm distribution installed and the Storm `bin/` directory in your path:
109
148
 
110
- ``` sh
111
- $ storm jar ./target/cluster-topology.jar redstorm.TopologyLauncher cluster examples/simple/word_count_topology.rb
112
- ```
113
-
114
- - to run `examples/simple/redis_word_count_topology.rb` you need a [Redis][redis] server running on `localhost:6379` and the Redis gem in `target/gems` using:
115
-
116
- ```sh
117
- gem install redis --install-dir target/gems/ --no-ri --no-rdoc
118
- ```
149
+ ``` sh
150
+ $ storm jar ./target/cluster-topology.jar -Djruby.compat.version=RUBY1_9 redstorm.TopologyLauncher cluster examples/simple/word_count_topology.rb
151
+ ```
119
152
 
120
- - generate jar and submit:
153
+ Note the **-Djruby.compat.version=RUBY1_9** parameter.
121
154
 
122
- ``` sh
123
- $ redstorm jar examples
124
- $ storm jar ./target/cluster-topology.jar redstorm.TopologyLauncher cluster examples/simple/redis_word_count_topology.rb
125
- ```
155
+ - to run `examples/simple/redis_word_count_topology.rb` you need a [Redis][redis] server running on `localhost:6379`
126
156
 
127
- - using `redis-cli`, push words into the `test` list and watch Storm pick them up
157
+ ``` sh
158
+ $ redstorm --1.9 gems --gemfile examples/simple/Gemfile
159
+ $ redstorm --1.9 jar examples
160
+ $ storm jar ./target/cluster-topology.jar -Djruby.compat.version=RUBY1_9 redstorm.TopologyLauncher cluster examples/simple/redis_word_count_topology.rb
161
+ ```
128
162
 
163
+ - using `redis-cli`, push words into the `test` list and watch Storm pick them up
129
164
 
130
165
  Basically you must follow the [Storm instructions](https://github.com/nathanmarz/storm/wiki) to [setup a production cluster](https://github.com/nathanmarz/storm/wiki/Setting-up-a-Storm-cluster) and [submit your topology to the cluster](https://github.com/nathanmarz/storm/wiki/Running-topologies-on-a-production-cluster).
131
166
 
132
- ## DSL usage
133
-
134
- Your project can be created in a single file containing all spouts, bolts and topology classes or each classes can be in its own file, your choice. There are [many examples](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple) for the *simple* DSL.
135
-
136
- The DSL uses a **callback metaphor** to attach code to the topology/spout/bolt execution contexts using `on_*` DSL constructs (ex.: on_submit, on_send, ...). When using `on_*` you can attach you code in 3 different ways:
137
-
138
- - using a code block
139
-
140
- ```ruby
141
- on_receive (:ack => true, :anchor => true) {|tuple| do_something_with(tuple)}
142
-
143
- on_receive :ack => true, :anchor => true do |tuple|
144
- do_something_with(tuple)
145
- end
146
- ```
147
-
148
- - defining the corresponding method
149
-
150
- ```ruby
151
- on_receive :ack => true, :anchor => true
152
- def on_receive(tuple)
153
- do_something_with(tuple)
154
- end
155
- ```
156
-
157
- - defining an arbitrary method
158
-
159
- ```ruby
160
- on_receive :my_method, :ack => true, :anchor => true
161
- def my_method(tuple)
162
- do_something_with(tuple)
163
- end
164
- ```
165
-
166
- The [example SplitSentenceBolt](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/split_sentence_bolt.rb) shows the 3 different coding style.
167
-
168
- ### Topology DSL
169
-
170
- Normally Storm topology components are assigned and referenced using numeric ids. In the SimpleTopology DSL **ids are optional**. By default the DSL will use the component class name as an implicit symbolic id and bolt source ids can use these implicit ids. The DSL will automatically resolve and assign numeric ids upon topology submission. If two components are of the same class, creating a conflict, then the id can be explicitly defined using either a numeric value, a symbol or a string. Numeric values will be used as-is at topology submission while symbols and strings will be resolved and assigned a numeric id.
171
-
172
- ```ruby
173
- require 'red_storm'
174
-
175
- class MyTopology < RedStorm::SimpleTopology
176
-
177
- spout spout_class, options
178
-
179
- bolt bolt_class, options do
180
- source source_id, grouping
181
- ...
182
- end
183
-
184
- configure topology_name do |env|
185
- config_attribute value
186
- ...
187
- end
188
-
189
- on_submit do |env|
190
- ...
191
- end
192
- end
193
- ```
194
-
195
- #### spout statement
196
-
197
- ```ruby
198
- spout spout_class, options
199
- ```
200
-
201
- - `spout_class` — spout Ruby class
202
- - `options`
203
- - `:id` — spout explicit id (**default** is spout class name)
204
- - `:parallelism` — spout parallelism (**default** is 1)
205
-
206
- #### bolt statement
207
-
208
- ```ruby
209
- bolt bolt_class, options do
210
- source source_id, grouping
211
- ...
212
- end
213
- ```
214
-
215
- - `bolt_class` — bolt Ruby class
216
- - `options`
217
- - `:id` — bolt explicit id (**default** is bolt class name)
218
- - `:parallelism` — bolt parallelism (**default** is 1)
219
- - `source_id` — source id reference. can be the source class name if unique or the explicit id if defined
220
- - `grouping`
221
- - `:fields => ["field", ...]` — fieldsGrouping using fields on the source_id
222
- - `:shuffle` — shuffleGrouping on the source_id
223
- - `:global` — globalGrouping on the source_id
224
- - `:none` — noneGrouping on the source_id
225
- - `:all` — allGrouping on the source_id
226
- - `:direct` — directGrouping on the source_id
227
-
228
- #### configure statement
229
-
230
- ```ruby
231
- configure topology_name do |env|
232
- configuration_field value
233
- ...
234
- end
235
- ```
236
-
237
- The `configure` statement is **required**.
238
-
239
- - `topology_name` — alternate topology name (**default** is topology class name)
240
- - `env` — is set to `:local` or `:cluster` for you to set enviroment specific configurations
241
- - `config_attribute` — the Storm Config attribute name. See Storm for complete list. The attribute name correspond to the Java setter method, without the "set" prefix and the suffix converted from CamelCase to underscore. Ex.: `setMaxTaskParallelism` is `:max_task_parallelism`.
242
- - `:debug`
243
- - `:max_task_parallelism`
244
- - `:num_workers`
245
- - `:max_spout_pending`
246
- - ...
247
-
248
- #### on_submit statement
249
-
250
- ```ruby
251
- on_submit do |env|
252
- ...
253
- end
254
- ```
255
-
256
- The `on_submit` statement is **optional**. Use it to execute code after the topology submission.
257
-
258
- - `env` — is set to `:local` or `:cluster`
259
-
260
- For example, you can use `on_submit` to shutdown the LocalCluster after some time. The LocalCluster instance is available usign the `cluster` method.
261
-
262
- ```ruby
263
- on_submit do |env|
264
- if env == :local
265
- sleep(5)
266
- cluster.shutdown
267
- end
268
- end
269
- ```
270
-
271
- #### Examples
167
+ ## Ruby DSL
272
168
 
273
- - [ExclamationTopology](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/exclamation_topology.rb)
274
- - [ExclamationTopology2](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/exclamation_topology2.rb)
275
- - [WordCountTopology](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/word_count_topology.rb)
276
- - [RedisWordCountTopology](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/redis_word_count_topology.rb)
169
+ [Ruby DSL Documentation](https://github.com/colinsurprenant/redstorm/wiki/Ruby-DSL-Documentation)
277
170
 
278
- ### Spout DSL
171
+ ## RedStorm Development
279
172
 
280
- ```ruby
281
- require 'red_storm'
173
+ It is possible to fork the RedStorm project and run local and remote/cluster topologies directly from the project sources without installing the gem. This is a useful setup when contributing to the project.
282
174
 
283
- class MySpout < RedStorm::SimpleSpout
284
- set spout_attribute => value
285
- ...
286
-
287
- output_fields :field, ...
288
-
289
- on_send options do
290
- ...
291
- end
292
-
293
- on_init do
294
- ...
295
- end
296
-
297
- on_close do
298
- ...
299
- end
300
-
301
- on_ack do |msg_id|
302
- ...
303
- end
304
-
305
- on_fail do |msg_id|
306
- ...
307
- end
308
- end
309
- ```
310
-
311
- #### set statement
312
-
313
- ```ruby
314
- set spout_attribute => value
315
- ```
316
-
317
- The `set` statement is **optional**. Use it to set spout specific attributes.
318
-
319
- - `spout_attributes`
320
- - `:is_distributed` — set to `true` for a distributed spout (**default** is `false`)
321
-
322
- #### output_fields statement
323
-
324
- ```ruby
325
- output_fields :field, ...
326
- ```
327
-
328
- Define the output fields for this spout.
329
-
330
- - `:field` — the field name, can be symbol or string.
331
-
332
- #### on_send statement
333
-
334
- ```ruby
335
- on_send options do
336
- ...
337
- end
338
- ```
339
-
340
- `on_send` relates to the Java spout `nextTuple` method and is called periodically by storm to allow the spout to output a tuple. When using auto-emit (default), the block return value will be auto emited. A single value return will be emited as a single-field tuple. An array of values `[a, b]` will be emited as a multiple-fields tuple. Normally a spout [should only output a single tuple per on_send invocation](https://groups.google.com/forum/#!topic/storm-user/SGwih7vPiDE/discussion).
341
-
342
- - `:options`
343
- - `:emit` — set to `false` to disable auto-emit (**default** is `true`)
344
-
345
- #### on_init statement
346
-
347
- ```ruby
348
- on_init do
349
- ...
350
- end
351
- ```
352
-
353
- `on_init` relates to the Java spout `open` method. When `on_init` is called, the `config`, `context` and `collector` are set to return the Java spout config `Map`, `TopologyContext` and `SpoutOutputCollector`.
354
-
355
- #### on_close statement
356
-
357
- ```ruby
358
- on_close do
359
- ...
360
- end
361
- ```
362
-
363
- `on_close` relates to the Java spout `close` method.
364
-
365
- #### on_ack statement
366
-
367
- ```ruby
368
- on_ack do |msg_id|
369
- ...
370
- end
371
- ```
372
-
373
- `on_ack` relates to the Java spout `ack` method.
374
-
375
- #### on_fail statement
376
-
377
- ```ruby
378
- on_fail do |msg_id|
379
- ...
380
- end
381
- ```
382
-
383
- `on_fail` relates to the Java spout `fail` method.
384
-
385
- #### Examples
386
-
387
- - [RandomSentenceSpout](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/random_sentence_spout.rb)
388
- - [RedisWordSpout](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/redis_word_count_topology.rb)
389
-
390
- ### Bolt DSL
391
-
392
- ```ruby
393
- require 'red_storm'
394
-
395
- class MyBolt < RedStorm::SimpleBolt
396
- output_fields :field, ...
397
-
398
- on_receive options do
399
- ...
400
- end
175
+ ### Requirements
401
176
 
402
- on_init do
403
- ...
404
- end
177
+ - JRuby 1.6.7
405
178
 
406
- on_close do
407
- ...
408
- end
409
- end
410
- ```
179
+ ### Workflow
411
180
 
412
- #### on_receive statement
181
+ - fork project and create branch
182
+ - install required gems
413
183
 
414
- ```ruby
415
- on_receive options do
416
- ...
417
- end
418
- ```
184
+ ```sh
185
+ $ jruby --1.9 -S bundle install
186
+ ```
419
187
 
420
- `on_receive` relates to the Java bolt `execute` method and is called upon tuple reception by Storm. When using auto-emit, the block return value will be auto emited. A single value return will be emited as a single-field tuple. An array of values `[a, b]` will be emited as a multiple-fields tuple. An array of arrays `[[a, b], [c, d]]` will be emited as multiple-fields multiple tuples. When not using auto-emit, the `unanchored_emit(value, ...)` and `anchored_emit(tuple, value, ...)` method can be used to emit a single tuple. When using auto-anchor (disabled by default) the sent tuples will be anchored to the received tuple. When using auto-ack (disabled by default) the received tuple will be ack'ed after emitting the return value. When not using auto-ack, the `ack(tuple)` method can be used to ack the tuple.
188
+ - install dependencies in `target/dependencies`
421
189
 
422
- Note that setting auto-ack and auto-anchor is possible **only** when auto-emit is enabled.
190
+ ```sh
191
+ $ bin/redstorm --1.9 deps
192
+ ```
423
193
 
424
- - `:options`
425
- - `:emit` — set to `false` to disable auto-emit (**default** is `true`)
426
- - `:ack` — set to `true` to enable auto-ack (**default** is `false`)
427
- - `:anchor` — set to `true` to enable auto-anchor (**default** is `false`)
194
+ - generate and build Java source into `target/classes`
428
195
 
429
- #### on_init statement
196
+ ```sh
197
+ $ bin/redstorm --1.9 build
198
+ ```
430
199
 
431
- ```ruby
432
- on_init do
433
- ...
434
- end
435
- ```
200
+ if you modify any of the Java binding code, you need to run this to rebuild the bindings
436
201
 
437
- `on_init` relates to the Java bolt `prepare` method. When `on_init` is called, the `config`, `context` and `collector` are set to return the Java spout config `Map`, `TopologyContext` and `SpoutOutputCollector`.
202
+ - run topology in **local** Storm mode
438
203
 
439
- #### on_close statement
204
+ ```sh
205
+ $ bin/redstorm --1.9 local path/to/topology_class.rb
206
+ ```
440
207
 
441
- ```ruby
442
- on_close do
443
- ...
444
- end
445
- ```
208
+ If you only make changes to your topology code, this is the only step you need to repeat to try your updated code.
446
209
 
447
- `on_close` relates to the Java bolt `cleanup` method.
210
+ - generate remote cluster topology jar into `target/cluster-topology.jar`, including the `mydir/` directory.
448
211
 
449
- #### Examples
212
+ ```sh
213
+ $ bin/redstorm --1.9 jar mydir otherdir1 otherdir2 ...
214
+ ```
450
215
 
451
- - [ExclamationBolt](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/exclamation_bolt.rb)
452
- - [SplitSentenceBolt](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/split_sentence_bolt.rb)
453
- - [WordCountBolt](https://github.com/colinsurprenant/redstorm/tree/master/examples/simple/word_count_bolt.rb)
216
+ - **if you add/change** Gemfile for your topology, install gems in `target/gems`. Alternate gemfile path can be specified using --gemfile=GEMFILE
454
217
 
455
- ## Development
218
+ ```sh
219
+ $ bin/redstorm --1.9 gems [--gemfile=GEMFILE]
220
+ ```
456
221
 
457
- ### Requirements
222
+ **do not forget** to rerurn `bin/redstorm --1.9 jar ...` to pick up these gems, before submitting your topology on a remote cluster.
458
223
 
459
- - JRuby 1.6.6
460
- - rake gem ~> 0.9.2.2
461
- - ruby-maven gem ~> 3.0.3.0.28.5
462
- - rspec gem ~> 2.8.0
463
224
 
464
- ### Contribute
225
+ ### How to Contribute
465
226
 
466
227
  Fork the project, create a branch and submit a pull request.
467
228
 
@@ -475,42 +236,12 @@ Some ways you can contribute:
475
236
  - by refactoring code
476
237
  - ...
477
238
 
478
- ### Workflow
479
-
480
- - fork project
481
- - create branch
482
- - install dependencies in `target/dependencies`
483
-
484
- ```sh
485
- $ rake deps
486
- ```
487
-
488
- - generate and build Java source into `target/classes`
489
-
490
- ```sh
491
- $ rake build
492
- ```
493
-
494
- - run topology in local dev cluster
495
-
496
- ```sh
497
- $ bin/redstorm local path/to/topology_class.rb
498
- ```
499
-
500
- - generate remote cluster topology jar into `target/cluster-topology.jar`, including the `examples/` directory.
501
-
502
- ```sh
503
- $ rake jar['examples']
504
- ```
505
-
506
239
  ## Author
507
- Colin Surprenant, [@colinsurprenant][twitter], [colin.surprenant@needium.com][needium], [colin.surprenant@gmail.com][gmail], [http://github.com/colinsurprenant][github]
240
+ Colin Surprenant, [@colinsurprenant][twitter], [http://github.com/colinsurprenant][github], colin.surprenant@gmail.com, colin.surprenant@needium.com
508
241
 
509
242
  ## License
510
243
  Apache License, Version 2.0. See the LICENSE.md file.
511
244
 
512
- [needium]: colin.surprenant@needium.com
513
- [gmail]: colin.surprenant@gmail.com
514
245
  [twitter]: http://twitter.com/colinsurprenant
515
246
  [github]: http://github.com/colinsurprenant
516
247
  [rvm]: http://beginrescueend.com/