iron_response 0.0.2 → 0.0.3
Sign up to get free protection for your applications and to get access to all the features.
- data/README.md +12 -3
- data/lib/iron_response/version.rb +1 -1
- data/lib/iron_response.rb +14 -3
- data/test/gem_dependency_test.rb +35 -0
- data/test/workers/fcc_filings_counter.rb +12 -0
- metadata +6 -2
data/README.md
CHANGED
@@ -2,7 +2,7 @@
|
|
2
2
|
|
3
3
|
# iron_response
|
4
4
|
|
5
|
-
|
5
|
+
Glues together IronWorker and AWS S3 to provide a response object to remote worker scripts. This allows you to write massively concurrent Ruby programs without worrying about threads.
|
6
6
|
|
7
7
|
```ruby
|
8
8
|
require "iron_response"
|
@@ -50,6 +50,8 @@ end
|
|
50
50
|
|
51
51
|
For many use cases, this is fine. But what if I want to know the result of `do_something`? A simple way to get the result would be for your worker to POST the final result somewhere, then have the client it. This gem simply abstracts that process away, allowing the developer to avoid boilerplate and to keep worker code elegant.
|
52
52
|
|
53
|
+
On top of all this, another benefit to using this gem is that it makes it much easier to test workers.
|
54
|
+
|
53
55
|
Under the hood, `iron_response` uses some functional and meta-programming to capture the final expression of a worker file, convert it to JSON, and then POST it to Amazon S3. When all the workers in an `IronResponse::Batch` have finished, the gem retrieves the file and converts the JSON string back to Ruby.
|
54
56
|
|
55
57
|
This process means there a few important implications:
|
@@ -57,8 +59,6 @@ This process means there a few important implications:
|
|
57
59
|
- Response objects "sent" from workers should be JSON-parseable. This means sticking to basic Ruby objects and data structures such as `String`, `Fixnum`, `Hash`, and `Array`.
|
58
60
|
- Though these objects must be parseable, they can be fairly large. Though I haven't figured out exactly what the limit is, you really are only constrained by IronWorkers RAM and HD limits, and the bandwidth between the client (your computer/server), IronWorker servers, and S3. In an ideal setup, all three of these components are inside of Amazon's cloud, allowing you to get very fast throughput.
|
59
61
|
|
60
|
-
One final gotcha: I haven't yet implemented a clean way to use the `iron_worker_ng` gem to declare dependencies. This should be done soon, though.
|
61
|
-
|
62
62
|
## Usage
|
63
63
|
|
64
64
|
This gem requires a basic understanding of how to use [IronWorker with Ruby](https://github.com/iron-io/iron_worker_ruby_ng).
|
@@ -140,6 +140,15 @@ batch.params_array = Array ("a".."z").map {|i| {letter: i}}
|
|
140
140
|
results = batch.run!
|
141
141
|
```
|
142
142
|
|
143
|
+
If your worker code requires any gems, you can use [`iron_worker_ng`](https://github.com/iron-io/iron_worker_ruby_ng)'s API:
|
144
|
+
|
145
|
+
```ruby
|
146
|
+
batch.code.merge_gem("nokogiri", "< 1.6.0") # decreases remote build time
|
147
|
+
batch.code.merge_gem("ecfs")
|
148
|
+
batch.code.full_remote_build(true)
|
149
|
+
```
|
150
|
+
|
151
|
+
|
143
152
|
## Installation
|
144
153
|
|
145
154
|
Add this line to your application's Gemfile:
|
data/lib/iron_response.rb
CHANGED
@@ -73,10 +73,21 @@ module IronResponse
|
|
73
73
|
JSON.parse(response)
|
74
74
|
end
|
75
75
|
|
76
|
+
def code
|
77
|
+
@code ||= IronWorkerNG::Code::Ruby.new(exec: @worker).tap do |c|
|
78
|
+
c.name = worker_name
|
79
|
+
c.merge_gem("iron_response")
|
80
|
+
c.runtime = "ruby"
|
81
|
+
end
|
82
|
+
|
83
|
+
@code
|
84
|
+
end
|
85
|
+
|
86
|
+
def patch_code!
|
87
|
+
@client.codes.patch(code)
|
88
|
+
end
|
89
|
+
|
76
90
|
def create_code!
|
77
|
-
code = IronWorkerNG::Code::Ruby.new(exec: @worker)
|
78
|
-
code.name(worker_name)
|
79
|
-
code.gem("iron_response")
|
80
91
|
@client.codes.create(code)
|
81
92
|
end
|
82
93
|
end
|
@@ -0,0 +1,35 @@
|
|
1
|
+
require "test_helper"
|
2
|
+
|
3
|
+
class GemDependencyTest < MiniTest::Unit::TestCase
|
4
|
+
def test_synopsis
|
5
|
+
|
6
|
+
config = Configuration.keys
|
7
|
+
batch = IronResponse::Batch.new
|
8
|
+
|
9
|
+
batch.auto_update_worker = true
|
10
|
+
|
11
|
+
batch.config[:iron_io] = config[:iron_io]
|
12
|
+
batch.config[:aws_s3] = config[:aws_s3]
|
13
|
+
batch.worker = "test/workers/fcc_filings_counter.rb"
|
14
|
+
|
15
|
+
batch.code.merge_gem("nokogiri", "< 1.6.0") # keeps the build times low
|
16
|
+
batch.code.merge_gem("ecfs")
|
17
|
+
batch.code.full_remote_build(true)
|
18
|
+
|
19
|
+
batch.params_array = [
|
20
|
+
"12-375", "12-268",
|
21
|
+
"12-238", "12-353",
|
22
|
+
"13-150", "13-5",
|
23
|
+
"10-71"
|
24
|
+
].map { |i| {docket_number: i} }
|
25
|
+
|
26
|
+
results = batch.run!
|
27
|
+
|
28
|
+
total = results.map {|r| r["length"]}.inject(:+)
|
29
|
+
|
30
|
+
p "There are #{total} total filings in these dockets."
|
31
|
+
|
32
|
+
assert_equal Array, results.class
|
33
|
+
assert_equal batch.params_array.length, results.length
|
34
|
+
end
|
35
|
+
end
|
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: iron_response
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.0.
|
4
|
+
version: 0.0.3
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2013-08-
|
12
|
+
date: 2013-08-10 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: iron_worker_ng
|
@@ -107,8 +107,10 @@ files:
|
|
107
107
|
- lib/iron_response.rb
|
108
108
|
- lib/iron_response/version.rb
|
109
109
|
- test/configuration.rb.example
|
110
|
+
- test/gem_dependency_test.rb
|
110
111
|
- test/synopsis_test.rb
|
111
112
|
- test/test_helper.rb
|
113
|
+
- test/workers/fcc_filings_counter.rb
|
112
114
|
- test/workers/is_prime.rb
|
113
115
|
homepage: ''
|
114
116
|
licenses: []
|
@@ -136,7 +138,9 @@ specification_version: 3
|
|
136
138
|
summary: Provides a response object for IronWorker tasks.
|
137
139
|
test_files:
|
138
140
|
- test/configuration.rb.example
|
141
|
+
- test/gem_dependency_test.rb
|
139
142
|
- test/synopsis_test.rb
|
140
143
|
- test/test_helper.rb
|
144
|
+
- test/workers/fcc_filings_counter.rb
|
141
145
|
- test/workers/is_prime.rb
|
142
146
|
has_rdoc:
|