skn_utils 5.4.1 → 5.5.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rspec +2 -0
- data/README.md +173 -54
- data/bin/concurrent_test_block +54 -0
- data/bin/concurrent_test_grouped +45 -0
- data/bin/concurrent_test_procs +45 -0
- data/bin/concurrent_test_wrapped +49 -0
- data/lib/skn_utils.rb +10 -2
- data/lib/skn_utils/concurrent_jobs.rb +96 -0
- data/lib/skn_utils/http_processor.rb +34 -0
- data/lib/skn_utils/job_commands.rb +201 -0
- data/lib/skn_utils/version.rb +2 -2
- data/skn_utils.gemspec +3 -0
- data/spec/lib/skn_utils/concurrent_jobs_spec.rb +277 -0
- data/spec/spec_helper.rb +4 -0
- metadata +39 -2
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 9a78df2df5ed11a61cea3d00b0f3543295e3ce5b
|
4
|
+
data.tar.gz: eb021f9d1f707e8e08411b542b05959061322d25
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 0ee33852c9675b13b64a8beb22fdd26fa6afd09994ee4809e4d2fc84bbc46201f469f9b715f4a7ba2291abadd3f5de36f6d98d257c3cc1f085b29f4f7c316374
|
7
|
+
data.tar.gz: d7ff622a8c1dc13ab5a0438c08b5f9f0e5fc04f32c989bcc8ab32d8d08243cae69c1cd6ce985c0a7871b7784f56a181d6ed6ab8879282e9c139588cbd4112f95
|
data/.rspec
CHANGED
data/README.md
CHANGED
@@ -2,68 +2,81 @@
|
|
2
2
|
[![Gem Version](https://badge.fury.io/rb/skn_utils.svg)](http://badge.fury.io/rb/skn_utils)
|
3
3
|
|
4
4
|
# SknUtils
|
5
|
-
## SknUtils::NestedResult class; dynamic key/value container
|
6
|
-
A Ruby Gem containing a Ruby PORO (Plain Old Ruby Object) that can be instantiated at runtime with an input hash. This library creates
|
7
|
-
an Object with Dot and Hash notational accessors to each key's value. Additional key/value pairs can be added post-create
|
8
|
-
by simply assigning it; `obj.my_new_var = "some value"`
|
9
5
|
|
10
|
-
|
11
|
-
|
12
|
-
|
13
|
-
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
30
|
-
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
|
35
|
-
|
36
|
-
|
37
|
-
|
38
|
-
|
39
|
-
|
40
|
-
|
41
|
-
|
42
|
-
|
43
|
-
## Available Classes
|
44
|
-
* SknSuccess
|
45
|
-
* SknFailure
|
6
|
+
`SknUtils` is a collection of pure-ruby utility classes and modules, with limited
|
7
|
+
dependencies, to augment the development of Ruby applications. Examples of these
|
8
|
+
utilities in action can be found in my related projects `SknServices`, `SknWebApp`,
|
9
|
+
and `SknBase`.
|
10
|
+
|
11
|
+
* The exchange or handoff of values between objects is addressed via the `NestedResults`
|
12
|
+
class which implements dot-notation and nesting over a concurrent hash: A ruby Hash can
|
13
|
+
use any valid ruby object as a key or value.
|
14
|
+
* `NestedResults` is later sub-classed as `Configuration` to provide application level
|
15
|
+
settings using YAML files with a API simular to the RbConfig gem.
|
16
|
+
* Object or method return values can be enclosed in `SknSuccess` or `SknFailure` classes
|
17
|
+
to prevent or minimize nil returns.
|
18
|
+
* Precise microsecond `duration`s, Number to `as_human_size`, and a `catch_exceptions`
|
19
|
+
retry-able feature are implemented on the SknUtils class directly for ease of use.
|
20
|
+
* `Configurable` module extends any class or module with configurable attribute method
|
21
|
+
as needed, with a set of defaults methods which emulate Rails.env, Rails.logger, and
|
22
|
+
Rails.root functionality.
|
23
|
+
* `CoreObjectExtensions` simular to ActiveSupport's, `#present?` and `#blank?` are
|
24
|
+
automatically applied, unless already present, when SknUtils gem is loaded.
|
25
|
+
* `SknRegistry` class is an advanced feature which allows you to manually register
|
26
|
+
classes, procs, or any value with a user-defined `label`. Initialization and dependency injection
|
27
|
+
requirements of service-like classes can be included in this registration process allowing
|
28
|
+
them to be centrally maintained. The `label` can be any valid ruby value; like a
|
29
|
+
classname, symbol, or string as needed
|
30
|
+
* `NullObject`, `NotifierBase`, and `Wrappable` are interesting classes which you might want to explore further.
|
31
|
+
* `ConcurrentJobs` is a feature implemented to allow concurrent/multi-threaded execution of jobs. The included
|
32
|
+
companion classes focus on HTTP GET,PUT,POST, and DELETE jobs as an example of how to use the `ConcurrentJobs` feature.
|
33
|
+
|
34
|
+
All classes and modules have RSpec test coverage (90+) of their originally intended use-cases.
|
35
|
+
|
36
|
+
|
37
|
+
### Available Classes
|
46
38
|
* SknSettings
|
47
39
|
* SknUtils::Configuration
|
48
40
|
* SknUtils::EnvStringHandler
|
49
|
-
* SknRegistry
|
50
|
-
* SknContainer
|
51
41
|
* SknHash
|
52
42
|
* SknUtils::ResultBean
|
53
43
|
* SknUtils::PageControls
|
54
44
|
* SknUtils::NestedResult
|
55
|
-
*
|
56
|
-
*
|
45
|
+
* SknRegistry
|
46
|
+
* SknContainer
|
47
|
+
* SknSuccess
|
48
|
+
* SknFailure
|
57
49
|
* SknUtils::Configurable
|
58
|
-
|
59
|
-
|
50
|
+
* SknUtils::CoreObjectExtensions
|
51
|
+
* SknUtils::NullObject
|
52
|
+
* SknUtils::NotifierBase
|
53
|
+
* SknUtils::Wrappable
|
54
|
+
* SknUtils::ConcurrentJobs
|
55
|
+
* SknUtils::JobWrapper (via ConcurrentJobs)
|
56
|
+
* SknUtils::CommandJSONPost (via JobCommands)
|
57
|
+
* SknUtils::CommandFORMPost
|
58
|
+
* SknUtils::CommandJSONGet
|
59
|
+
* SknUtils::CommandJSONPut
|
60
|
+
* SknUtils::CommandFORMDelete
|
61
|
+
* SknUtils::HttpProcessor
|
62
|
+
|
63
|
+
### Available Class.Methods
|
60
64
|
* SknUtils.catch_exceptions()
|
61
65
|
* SknUtils.as_human_size()
|
62
66
|
* SknUtils.duration(start_time=nil)
|
63
67
|
|
64
68
|
|
65
|
-
|
66
69
|
## History
|
70
|
+
2/24/2019 V5.5.0
|
71
|
+
Added
|
72
|
+
* ConcurrentJobs feature set
|
73
|
+
- Executes (HTTP/any) jobs in parallel
|
74
|
+
|
75
|
+
1/2/2019 V5.4.1
|
76
|
+
Added
|
77
|
+
- Wrappable module for evaluation.
|
78
|
+
- Ruby comment # frozen_string_literal, everywhere
|
79
|
+
|
67
80
|
12/16/2018 V5.4.0
|
68
81
|
Added :duration() utils to SknUtils module:
|
69
82
|
#duration() #=> returns start_time value
|
@@ -154,6 +167,43 @@ There are many more use cases for Ruby's Hash that this gem just makes easier to
|
|
154
167
|
06/2015 V1.5.1 commit #67ef656
|
155
168
|
Last Version to depend on Rails (ActiveModel) for #to_json and #to_xml serialization
|
156
169
|
|
170
|
+
### NestedResult class; dynamic key/value container
|
171
|
+
A class implementing a Ruby PORO (Plain Old Ruby Object) that can be instantiated at runtime with a hash. Creates
|
172
|
+
a nested object with Dot and Hash notational accessors to each key's value. Additional key/value pairs can be added post-create
|
173
|
+
by simply assigning it; `obj.my_new_var = "some value"`
|
174
|
+
|
175
|
+
* Transforms the initialing hash into accessible object instance values, with their keys as method names.
|
176
|
+
* If the key's value is also a hash, it too will become an Object.
|
177
|
+
* if the key's value is a Array of Hashes, or Array of Arrays of Hashes, each hash element of the Arrays will become an Object.
|
178
|
+
* The current key/value (including nested) pairs are returned via #to_hash or #to_json when and if needed.
|
179
|
+
* Best described as dot notation wrapper over a Ruby (Concurrent-Ruby) Hash.
|
180
|
+
|
181
|
+
Ruby's Hash object is already extremely flexible, even more so with the addition of dot-notation. As I work more with Ruby outside of Rails, I'm finding more use cases for the capabilities of this gem. Here are a few examples:
|
182
|
+
|
183
|
+
1. Application settings containers, SknSettings. Loads Yaml file based on `ENV['RACK_ENV']` value, or specified file-key.
|
184
|
+
- Replaces Config and/or RBConfig Gems for yaml based settings
|
185
|
+
1. Substitute for Rails.root, via a little ERB/YAML/Marshal statement in settings.yml file, and a helper class
|
186
|
+
- settings.yml (YAML)
|
187
|
+
- `root: <%= Dir.pwd %>`
|
188
|
+
- enables `SknSettings.root`
|
189
|
+
- `env: !ruby/string:SknUtils::EnvStringHandler <%= ENV.fetch('RACK_ENV', 'development') %>`
|
190
|
+
- enables `SknSettings.env.production?` ...
|
191
|
+
1. Since SknSettings is by necessity a global constant, it can serve as Session Storage to keep system objects; like a ROM-RB instance.
|
192
|
+
1. In-Memory Key-Store, use it to cache active user objects, or active Integration passwords, and/or objects that are not serializable.
|
193
|
+
1. Command registries used to dispatch command requests to proper command handler. see example app [SknBase](https://github.com/skoona/skn_base/blob/master/strategy/services/content/command_handler.rb)
|
194
|
+
```ruby
|
195
|
+
SknSettings.command_handler = {
|
196
|
+
Commands::RetrieveAvailableResources => method(:resources_metadata_service),
|
197
|
+
Commands::RetrieveResourceContent => method(:resource_content_service)
|
198
|
+
}
|
199
|
+
...
|
200
|
+
SknSettings.command_handler[ cmd.class ].call( cmd )
|
201
|
+
-- or --
|
202
|
+
SknSettings.command_handler.key?( cmd.class ) &&
|
203
|
+
cmd.valid? ? SknSettings.command_handler[ cmd.class ].call( cmd ) : command_not_found_action()
|
204
|
+
```
|
205
|
+
There are many more use cases for Ruby's Hash that this gem just makes easier to implement.
|
206
|
+
|
157
207
|
|
158
208
|
## Public Components
|
159
209
|
SknUtils::NestedResult # Primary Key/Value Container with Dot/Hash notiation support.
|
@@ -171,6 +221,14 @@ There are many more use cases for Ruby's Hash that this gem just makes easier to
|
|
171
221
|
SknSuccess # Three attribute value containers for return codes -- #value, #message, #success
|
172
222
|
- Extra #payload method returns value as NestResult if value is_a Hash
|
173
223
|
SknFailure # Three attribute value containers for return codes -- #value, #message, #success
|
224
|
+
SknUtils::ConcurrentJobs # Async/Sync Job executor pool with HTTP support
|
225
|
+
SknUtils::CommandJSONGet # HTTP Get Command class expecting `json` return, located inside `job_commands`
|
226
|
+
SknUtils::CommandJSONPut # HTTP Put Command class expecting `json` return, located inside `job_commands`
|
227
|
+
SknUtils::CommandJSONPost # HTTP Post Command class expecting `json` return, located inside `job_commands`
|
228
|
+
SknUtils::CommandFORMPost # HTTP Post Command class expecting `form` data return, located inside `job_commands`
|
229
|
+
SknUtils::CommandFORMDelete # HTTP Delete Command class expecting `form` data return, located inside `job_commands`
|
230
|
+
SknUtils::HttpProcessor # Command driven HTTP processing object supporting GET,PUT,POST, and DELETE
|
231
|
+
SknUtils::JobWrapper # Outer wrapper class to capture catastrophic exceptions (syntax normally)
|
174
232
|
|
175
233
|
|
176
234
|
## Public Methods: SknUtils::Configurable module
|
@@ -239,8 +297,8 @@ There are many more use cases for Ruby's Hash that this gem just makes easier to
|
|
239
297
|
```
|
240
298
|
|
241
299
|
|
242
|
-
## Public Methods:
|
243
|
-
SknContainer is global constant assigned to an instantiated instance of SknRegistry
|
300
|
+
## Public Methods: SknRegistry ONLY
|
301
|
+
`SknContainer` is global constant assigned to an instantiated instance of `SknRegistry`.
|
244
302
|
Returns the keyed value as the original instance/value or if provided a proc the result of calling that proc.
|
245
303
|
To register a class or object for global retrieval, use the following API. Also review the RSpecs for additional useage info.
|
246
304
|
|
@@ -266,9 +324,69 @@ There are many more use cases for Ruby's Hash that this gem just makes easier to
|
|
266
324
|
same_instance = SknContainer.resolve(:the_instance)
|
267
325
|
|
268
326
|
|
327
|
+
## Public Methods: SknUtils::ConcurrentJobs ONLY
|
328
|
+
`ConcurrentJobs` behaves as a concurrent thread pool by using Concurrent::Promise from the `concurrent-ruby` gem.
|
329
|
+
Enables the definition of Procs, or any callable class, which will be executed in parrallel the available jobs
|
330
|
+
loaded into ConcurrentJobs. Meant to reduce user-sensitive response times when multiple APIs must be invoked.
|
331
|
+
Also review the RSpecs for additional useage info.
|
332
|
+
|
333
|
+
SknUtils::ConcurrentJobs
|
334
|
+
#call(async: true) - Instantiate ConcurrentJobs with Async Workers, false for Sync Workers
|
335
|
+
#register_jobs(cmds, callable) - Array of Command to be executed by single callable
|
336
|
+
#register_job(&block) - Adds callable block to internal worker queue
|
337
|
+
#render_jobs - Collect results from all jobs into a Result object
|
338
|
+
#elapsed_time_string - "0.012 seconds" string showing duration of last #render_jobs call
|
339
|
+
|
340
|
+
SknUtils::Result - Contains individual results from each job executed
|
341
|
+
#success? - Determines if any job failed
|
342
|
+
#messages - Retrieves messages from job results, assumed present when job fails
|
343
|
+
#values - Returns an array of individual results from job executions
|
344
|
+
|
345
|
+
Commands and HttpProcessors are included to demonstrate Job creating patterns. ConcurrentJobs is not restricted
|
346
|
+
to Http calls or the command to command handler pattern. Using the #register_job method you can pass callable BLOCK
|
347
|
+
and it will be executed when #render_jobs is invoked. HttpProcessor is what I needed and triggered me to add this feature.
|
348
|
+
|
349
|
+
Example here:
|
350
|
+
```ruby
|
351
|
+
begin
|
352
|
+
# CommandJSONPost, CommandFORMGet, CommandJSONGet,
|
353
|
+
# CommandJSONPut, CommandFORMDelete
|
354
|
+
commands = [
|
355
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/posts"),
|
356
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/comments"),
|
357
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/todos/1"),
|
358
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/users")
|
359
|
+
]
|
360
|
+
|
361
|
+
# Initialize the queue with Async Workers by default
|
362
|
+
provider = SknUtils::ConcurrentJobs.call
|
363
|
+
|
364
|
+
# Populate WorkQueue
|
365
|
+
provider.register_jobs(commands, SknUtils::HttpProcessor) # mis-spelling these params result in an immediate exception (line 43 below)
|
366
|
+
|
367
|
+
# Execute WorkQueue
|
368
|
+
result = provider.render_jobs
|
369
|
+
|
370
|
+
if result.success?
|
371
|
+
puts "Success: true"
|
372
|
+
puts "Values: #{result.values}"
|
373
|
+
puts "Messages: #{result.messages}"
|
374
|
+
else
|
375
|
+
puts "Success: false - errors: #{result.messages.join(', ')}"
|
376
|
+
puts "Values: #{result.values}"
|
377
|
+
end
|
378
|
+
|
379
|
+
# result.values
|
380
|
+
rescue => e
|
381
|
+
$stderr.puts e.message, e.backtrace
|
382
|
+
end
|
383
|
+
|
384
|
+
```
|
385
|
+
|
386
|
+
|
269
387
|
|
270
388
|
## Public Methods: SknSettings ONLY
|
271
|
-
SknSettings is global constant containing an initialized Object of SknUtils::Configuration using defaults
|
389
|
+
SknSettings is a global constant containing an initialized Object of SknUtils::Configuration using defaults
|
272
390
|
To change the 'development'.yml default please use the following method early or in the case of Rails in 'application.rb
|
273
391
|
#load_config_basename!(config_name) -- Where config_name is the name of yml files stored in the `./config/settings` directory
|
274
392
|
#config_path!(path) -- Where path format is './<dirs>/', default is: './config/'
|
@@ -420,10 +538,11 @@ There are many more use cases for Ruby's Hash that this gem just makes easier to
|
|
420
538
|
## Installation
|
421
539
|
|
422
540
|
runtime prereqs:
|
423
|
-
*
|
424
|
-
*
|
425
|
-
*
|
426
|
-
*
|
541
|
+
* V5+ None
|
542
|
+
* V4+ None
|
543
|
+
* V3+ None
|
544
|
+
* V2+ None
|
545
|
+
* V1+ gem 'active_model', '~> 3.0'
|
427
546
|
|
428
547
|
|
429
548
|
Add this line to your application's Gemfile:
|
@@ -0,0 +1,54 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
#
|
3
|
+
# file: concurrent_test_block
|
4
|
+
#
|
5
|
+
|
6
|
+
require 'bundler/setup'
|
7
|
+
require 'skn_utils'
|
8
|
+
|
9
|
+
|
10
|
+
# ##
|
11
|
+
# MainLine
|
12
|
+
# ##
|
13
|
+
#
|
14
|
+
begin
|
15
|
+
# CommandJSONPost, CommandFORMGet, CommandJSONGet,
|
16
|
+
# CommandJSONPut, CommandFORMDelete
|
17
|
+
commands = [
|
18
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/posts"),
|
19
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/comments"),
|
20
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/todos/1"),
|
21
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/users")
|
22
|
+
]
|
23
|
+
|
24
|
+
# Initialize the queue with Async Workers by default
|
25
|
+
provider = SknUtils::ConcurrentJobs.call
|
26
|
+
|
27
|
+
# Populate WorkQueue
|
28
|
+
commands.each do |command|
|
29
|
+
provider.register_job do
|
30
|
+
begin
|
31
|
+
SknUtils::HttpProcessor.call(command) # mis-spelling these params result in [SknFailure, SknFailure, ...] results
|
32
|
+
rescue => ex
|
33
|
+
$stderr.puts ex
|
34
|
+
SknFailure.(ex.class.name, "#{ex.message}; #{ex.backtrace[0]}")
|
35
|
+
end
|
36
|
+
end
|
37
|
+
end
|
38
|
+
|
39
|
+
# Execute WorkQueue
|
40
|
+
result = provider.render_jobs
|
41
|
+
|
42
|
+
if result.success?
|
43
|
+
puts "Success: true"
|
44
|
+
puts "Values: #{result.values}"
|
45
|
+
puts "Messages: #{result.messages}"
|
46
|
+
else
|
47
|
+
puts "Success: false - errors: #{result.messages.join(', ')}"
|
48
|
+
puts "Values: #{result.values}"
|
49
|
+
end
|
50
|
+
|
51
|
+
# result.values
|
52
|
+
rescue => e
|
53
|
+
$stderr.puts e.message, e.backtrace
|
54
|
+
end
|
@@ -0,0 +1,45 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
#
|
3
|
+
# file: concurrent_test_grouped
|
4
|
+
#
|
5
|
+
|
6
|
+
require 'bundler/setup'
|
7
|
+
require 'skn_utils'
|
8
|
+
|
9
|
+
|
10
|
+
# ##
|
11
|
+
# MainLine
|
12
|
+
# ##
|
13
|
+
#
|
14
|
+
begin
|
15
|
+
# CommandJSONPost, CommandFORMGet, CommandJSONGet,
|
16
|
+
# CommandJSONPut, CommandFORMDelete
|
17
|
+
commands = [
|
18
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/posts"),
|
19
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/comments"),
|
20
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/todos/1"),
|
21
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/users")
|
22
|
+
]
|
23
|
+
|
24
|
+
# Initialize the queue with Async Workers by default
|
25
|
+
provider = SknUtils::ConcurrentJobs.call
|
26
|
+
|
27
|
+
# Populate WorkQueue
|
28
|
+
provider.register_jobs(commands, SknUtils::HttpProcessor) # mis-spelling these params result in an immediate exception (line 43 below)
|
29
|
+
|
30
|
+
# Execute WorkQueue
|
31
|
+
result = provider.render_jobs
|
32
|
+
|
33
|
+
if result.success?
|
34
|
+
puts "Success: true"
|
35
|
+
puts "Values: #{result.values}"
|
36
|
+
puts "Messages: #{result.messages}"
|
37
|
+
else
|
38
|
+
puts "Success: false - errors: #{result.messages.join(', ')}"
|
39
|
+
puts "Values: #{result.values}"
|
40
|
+
end
|
41
|
+
|
42
|
+
# result.values
|
43
|
+
rescue => e
|
44
|
+
$stderr.puts e.message, e.backtrace
|
45
|
+
end
|
@@ -0,0 +1,45 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
#
|
3
|
+
# file: concurrent_test_procs
|
4
|
+
#
|
5
|
+
require 'bundler/setup'
|
6
|
+
require 'skn_utils'
|
7
|
+
|
8
|
+
|
9
|
+
# ##
|
10
|
+
# MainLine
|
11
|
+
# ##
|
12
|
+
#
|
13
|
+
begin
|
14
|
+
# CommandJSONPost, CommandFORMGet, CommandJSONGet,
|
15
|
+
# CommandJSONPut, CommandFORMDelete
|
16
|
+
commands = [
|
17
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/posts"),
|
18
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/comments"),
|
19
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/todos/1"),
|
20
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/users")
|
21
|
+
]
|
22
|
+
|
23
|
+
# Initialize the queue with Async Workers by default
|
24
|
+
provider = SknUtils::ConcurrentJobs.call
|
25
|
+
|
26
|
+
# Populate WorkQueue
|
27
|
+
work_proc = ->(cmd) { SknSuccess.(cmd.uri.request_uri, "Ok") } # mis-spelling these params result in [SknFailure, SknFailure, ...] results
|
28
|
+
provider.register_jobs(commands, work_proc)
|
29
|
+
|
30
|
+
# Execute WorkQueue
|
31
|
+
result = provider.render_jobs
|
32
|
+
|
33
|
+
if result.success?
|
34
|
+
puts "Success: true"
|
35
|
+
puts "Values: #{result.values}"
|
36
|
+
puts "Messages: #{result.messages}"
|
37
|
+
else
|
38
|
+
puts "Success: false - errors: #{result.messages.join(', ')}"
|
39
|
+
puts "Values: #{result.values}"
|
40
|
+
end
|
41
|
+
|
42
|
+
# result.values
|
43
|
+
rescue => e
|
44
|
+
$stderr.puts e.message, e.backtrace
|
45
|
+
end
|
@@ -0,0 +1,49 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
#
|
3
|
+
# file: concurrent_test_wrapped
|
4
|
+
#
|
5
|
+
|
6
|
+
require 'bundler/setup'
|
7
|
+
require 'skn_utils'
|
8
|
+
|
9
|
+
|
10
|
+
# ##
|
11
|
+
# MainLine
|
12
|
+
# ##
|
13
|
+
#
|
14
|
+
begin
|
15
|
+
# CommandJSONPost, CommandFORMGet, CommandJSONGet,
|
16
|
+
# CommandJSONPut, CommandFORMDelete
|
17
|
+
commands = [
|
18
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/posts"),
|
19
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/comments"),
|
20
|
+
SknUtils::CommandJSONGet.call(full_url: "https://jsonplaceholder.typicode.com/todos/1"),
|
21
|
+
SknUtils::CommandJSONGet.call(full_url: "http://jsonplaceholder.typicode.com/users")
|
22
|
+
]
|
23
|
+
|
24
|
+
# Initialize the queue with Async Workers by default
|
25
|
+
provider = SknUtils::ConcurrentJobs.call
|
26
|
+
|
27
|
+
# Populate WorkQueue
|
28
|
+
commands.each do |command|
|
29
|
+
provider.register_job do
|
30
|
+
SknUtils::JobWrapper.call(command, SknUtils::HttpProcessor) # mis-spelling these params result in [nil, nil, ...] results
|
31
|
+
end
|
32
|
+
end
|
33
|
+
|
34
|
+
# Execute WorkQueue
|
35
|
+
result = provider.render_jobs
|
36
|
+
|
37
|
+
if result.success?
|
38
|
+
puts "Success: true"
|
39
|
+
puts "Values: #{result.values}"
|
40
|
+
puts "Messages: #{result.messages}"
|
41
|
+
else
|
42
|
+
puts "Success: false - errors: #{result.messages.join(', ')}"
|
43
|
+
puts "Values: #{result.values}"
|
44
|
+
end
|
45
|
+
|
46
|
+
# result.values
|
47
|
+
rescue => e
|
48
|
+
$stderr.puts e.message, e.backtrace
|
49
|
+
end
|
data/lib/skn_utils.rb
CHANGED
@@ -10,9 +10,12 @@ require 'time'
|
|
10
10
|
require 'concurrent'
|
11
11
|
unless defined?(Rails)
|
12
12
|
begin
|
13
|
+
require "uri"
|
14
|
+
require "net/http"
|
15
|
+
require 'net/https'
|
13
16
|
require 'deep_merge'
|
14
17
|
rescue LoadError => e
|
15
|
-
puts e.message
|
18
|
+
$stderr.puts e.message, e.backtrace
|
16
19
|
end
|
17
20
|
end
|
18
21
|
require 'skn_utils/core_extensions'
|
@@ -29,6 +32,10 @@ require 'skn_utils/configuration'
|
|
29
32
|
require 'skn_utils/configurable'
|
30
33
|
require 'skn_utils/wrappable'
|
31
34
|
|
35
|
+
require "skn_utils/job_commands"
|
36
|
+
require "skn_utils/http_processor"
|
37
|
+
require "skn_utils/concurrent_jobs"
|
38
|
+
|
32
39
|
require 'skn_hash'
|
33
40
|
require 'skn_registry'
|
34
41
|
require 'skn_container'
|
@@ -51,7 +58,7 @@ module SknUtils
|
|
51
58
|
[SknFailure, SknFailure].any? {|o| res.kind_of?(o) } ? res : SknSuccess.( res )
|
52
59
|
|
53
60
|
rescue StandardError, ScriptError => error
|
54
|
-
puts "#{retry_count} - #{error.class.name}:#{error.message}"
|
61
|
+
$stderr.puts "#{retry_count} - #{error.class.name}:#{error.message}"
|
55
62
|
if retry_count <= attempts
|
56
63
|
retry_count+= 1
|
57
64
|
sleep(pause_between)
|
@@ -89,3 +96,4 @@ module SknUtils
|
|
89
96
|
end
|
90
97
|
|
91
98
|
end
|
99
|
+
|
@@ -0,0 +1,96 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
# ##
|
3
|
+
#
|
4
|
+
#
|
5
|
+
# See JobCommands, HttpProcessor, ...
|
6
|
+
# See ./bin/par_test_[block|grouped|wrapped] examples
|
7
|
+
#
|
8
|
+
|
9
|
+
module SknUtils
|
10
|
+
|
11
|
+
class SyncWorker
|
12
|
+
def initialize(&blk)
|
13
|
+
@blk = blk
|
14
|
+
end
|
15
|
+
|
16
|
+
def call
|
17
|
+
@blk.call
|
18
|
+
end
|
19
|
+
end
|
20
|
+
|
21
|
+
class AsyncWorker
|
22
|
+
def initialize(&blk)
|
23
|
+
@blk = Concurrent::Promise.execute(&blk)
|
24
|
+
end
|
25
|
+
|
26
|
+
def call
|
27
|
+
@blk.value
|
28
|
+
end
|
29
|
+
end
|
30
|
+
|
31
|
+
class Result
|
32
|
+
def initialize(merged)
|
33
|
+
@merged = merged
|
34
|
+
end
|
35
|
+
|
36
|
+
def success?
|
37
|
+
@merged.all?(&:success) rescue false
|
38
|
+
end
|
39
|
+
|
40
|
+
def messages
|
41
|
+
@merged.map(&:message)&.compact rescue []
|
42
|
+
end
|
43
|
+
|
44
|
+
def values
|
45
|
+
@merged
|
46
|
+
end
|
47
|
+
end
|
48
|
+
|
49
|
+
class JobWrapper
|
50
|
+
def self.call(command, callable)
|
51
|
+
callable.call(command)
|
52
|
+
rescue => ex
|
53
|
+
SknFailure.(ex.class.name, "#{ex.message}; #{ex.backtrace[0]}")
|
54
|
+
end
|
55
|
+
end
|
56
|
+
|
57
|
+
class ConcurrentJobs
|
58
|
+
attr_reader :elapsed_time_string
|
59
|
+
|
60
|
+
def self.call(async: true)
|
61
|
+
worker = async ? AsyncWorker : SyncWorker
|
62
|
+
new(worker: worker)
|
63
|
+
end
|
64
|
+
|
65
|
+
def initialize(worker:)
|
66
|
+
@worker = worker
|
67
|
+
@workers = []
|
68
|
+
end
|
69
|
+
|
70
|
+
# commands: array of command objects related to callable
|
71
|
+
# callable: callable class or proc, ex:SknUtils::HttpProcessor
|
72
|
+
# callable must return SknSuccess || SknFailure
|
73
|
+
def register_jobs(commands, callable)
|
74
|
+
commands.each do |command|
|
75
|
+
register_job do
|
76
|
+
JobWrapper.call(command,callable)
|
77
|
+
end
|
78
|
+
end
|
79
|
+
end
|
80
|
+
|
81
|
+
def register_job(&blk)
|
82
|
+
@workers << @worker.new(&blk)
|
83
|
+
end
|
84
|
+
|
85
|
+
def render_jobs
|
86
|
+
stime = SknUtils.duration
|
87
|
+
merged = @workers.each_with_object([]) do |worker, acc|
|
88
|
+
acc.push( worker.call )
|
89
|
+
end
|
90
|
+
@elapsed_time_string = SknUtils.duration(stime)
|
91
|
+
Result.new(merged)
|
92
|
+
rescue => e
|
93
|
+
Result.new(merged || [])
|
94
|
+
end
|
95
|
+
end
|
96
|
+
end
|
@@ -0,0 +1,34 @@
|
|
1
|
+
# ##
|
2
|
+
#
|
3
|
+
#
|
4
|
+
module SknUtils
|
5
|
+
|
6
|
+
class HttpProcessor
|
7
|
+
|
8
|
+
def self.call(command)
|
9
|
+
completion = false
|
10
|
+
|
11
|
+
response = Net::HTTP.start( command.uri.host,command.uri.port,
|
12
|
+
use_ssl: command.uri.scheme.eql?("https")
|
13
|
+
) do |http|
|
14
|
+
http.open_timeout = 5 # in seconds, for internal http timeouts
|
15
|
+
http.read_timeout = 15 # in seconds
|
16
|
+
if command.uri.scheme.eql?("https")
|
17
|
+
http.verify_mode = OpenSSL::SSL::VERIFY_NONE
|
18
|
+
end
|
19
|
+
http.request(command.request)
|
20
|
+
end
|
21
|
+
|
22
|
+
if ( response.kind_of?(Net::HTTPClientError) or response.kind_of?(Net::HTTPServerError) )
|
23
|
+
completion = SknFailure.call(response.code, response.message)
|
24
|
+
else
|
25
|
+
payload = command.json? ? JSON.load(response.body) : response.body
|
26
|
+
completion = SknSuccess.call(payload, response.class.name)
|
27
|
+
end
|
28
|
+
|
29
|
+
completion
|
30
|
+
rescue => exception
|
31
|
+
SknFailure.call(command.uri.request_uri, "#{exception.message}; #{exception.backtrace[0]}")
|
32
|
+
end
|
33
|
+
end
|
34
|
+
end
|
@@ -0,0 +1,201 @@
|
|
1
|
+
# ##
|
2
|
+
#
|
3
|
+
# Ref: https://yukimotopress.github.io/http
|
4
|
+
|
5
|
+
module SknUtils
|
6
|
+
|
7
|
+
# #################################################
|
8
|
+
#
|
9
|
+
class CommandJSONPost
|
10
|
+
def self.call(options) # {full_url:,username:,userpass:,payload:}
|
11
|
+
new(options)
|
12
|
+
end
|
13
|
+
|
14
|
+
def json?
|
15
|
+
true
|
16
|
+
end
|
17
|
+
|
18
|
+
def uri
|
19
|
+
@_uri
|
20
|
+
end
|
21
|
+
|
22
|
+
def request
|
23
|
+
req = Net::HTTP::Post.new(uri.path) # Generate HTTPRequest object
|
24
|
+
req.basic_auth(@_username, @_userpass) if credentials?
|
25
|
+
req.content_type = 'application/json'
|
26
|
+
req.body = formatted_data
|
27
|
+
req
|
28
|
+
end
|
29
|
+
|
30
|
+
private
|
31
|
+
|
32
|
+
def initialize(opts={})
|
33
|
+
@_username = opts[:username]
|
34
|
+
@_userpass = opts[:userpass]
|
35
|
+
@_uri = URI.parse( opts[:full_url])
|
36
|
+
@_data = opts[:payload]
|
37
|
+
end
|
38
|
+
|
39
|
+
def formatted_data
|
40
|
+
@_data.respond_to?(:to_json) ? @_data.to_json : @_data
|
41
|
+
end
|
42
|
+
|
43
|
+
def credentials?
|
44
|
+
!(@_username.nil? || @_userpass.nil?)
|
45
|
+
end
|
46
|
+
end
|
47
|
+
|
48
|
+
|
49
|
+
# #################################################
|
50
|
+
#
|
51
|
+
class CommandFORMPost
|
52
|
+
def self.call(options) # {full_url:,username:,userpass:,payload:}
|
53
|
+
new(options)
|
54
|
+
end
|
55
|
+
|
56
|
+
def json?
|
57
|
+
false
|
58
|
+
end
|
59
|
+
|
60
|
+
def uri
|
61
|
+
@_uri
|
62
|
+
end
|
63
|
+
|
64
|
+
def request
|
65
|
+
req = Net::HTTP::Post.new(uri.path) # Generate HTTPRequest object
|
66
|
+
req.basic_auth(@_username, @_userpass) if credentials?
|
67
|
+
req.content_type = 'application/x-www-form-urlencoded'
|
68
|
+
req.set_form_data(formatted_data)
|
69
|
+
req
|
70
|
+
end
|
71
|
+
|
72
|
+
private
|
73
|
+
|
74
|
+
def initialize(opts={})
|
75
|
+
@_username = opts[:username]
|
76
|
+
@_userpass = opts[:userpass]
|
77
|
+
@_uri = URI.parse( opts[:full_url])
|
78
|
+
@_data = opts[:payload]
|
79
|
+
end
|
80
|
+
|
81
|
+
def formatted_data
|
82
|
+
@_data
|
83
|
+
end
|
84
|
+
|
85
|
+
def credentials?
|
86
|
+
!(@_username.nil? || @_userpass.nil?)
|
87
|
+
end
|
88
|
+
end
|
89
|
+
|
90
|
+
|
91
|
+
# #################################################
|
92
|
+
#
|
93
|
+
class CommandJSONGet
|
94
|
+
def self.call(options) # {full_url:,username:,userpass:}
|
95
|
+
new(options)
|
96
|
+
end
|
97
|
+
|
98
|
+
def json?
|
99
|
+
true
|
100
|
+
end
|
101
|
+
|
102
|
+
def uri
|
103
|
+
@_uri
|
104
|
+
end
|
105
|
+
|
106
|
+
def request
|
107
|
+
req = Net::HTTP::Get.new(uri.request_uri)
|
108
|
+
req.basic_auth(@_username, @_userpass) if credentials?
|
109
|
+
req
|
110
|
+
end
|
111
|
+
|
112
|
+
private
|
113
|
+
|
114
|
+
def initialize(opts={})
|
115
|
+
@_username = opts[:username]
|
116
|
+
@_userpass = opts[:userpass]
|
117
|
+
@_uri = URI.parse( opts[:full_url])
|
118
|
+
end
|
119
|
+
|
120
|
+
def credentials?
|
121
|
+
!(@_username.nil? || @_userpass.nil?)
|
122
|
+
end
|
123
|
+
end
|
124
|
+
|
125
|
+
|
126
|
+
# #################################################
|
127
|
+
#
|
128
|
+
class CommandJSONPut
|
129
|
+
def self.call(options) # {full_url:,username:,userpass:,payload:}
|
130
|
+
new(options)
|
131
|
+
end
|
132
|
+
|
133
|
+
def json?
|
134
|
+
true
|
135
|
+
end
|
136
|
+
|
137
|
+
def uri
|
138
|
+
@_uri
|
139
|
+
end
|
140
|
+
|
141
|
+
def request
|
142
|
+
req = Net::HTTP::Put.new(uri.path) # Generate HTTPRequest object
|
143
|
+
req.basic_auth(@_username, @_userpass) if credentials?
|
144
|
+
req.content_type = 'application/json'
|
145
|
+
req.body = formatted_data
|
146
|
+
req
|
147
|
+
end
|
148
|
+
|
149
|
+
private
|
150
|
+
|
151
|
+
def initialize(opts={})
|
152
|
+
@_username = opts[:username]
|
153
|
+
@_userpass = opts[:userpass]
|
154
|
+
@_uri = URI.parse( opts[:full_url])
|
155
|
+
@_data = opts[:payload]
|
156
|
+
end
|
157
|
+
|
158
|
+
def formatted_data
|
159
|
+
@_data.respond_to?(:to_json) ? @_data.to_json : @_data
|
160
|
+
end
|
161
|
+
|
162
|
+
def credentials?
|
163
|
+
!(@_username.nil? || @_userpass.nil?)
|
164
|
+
end
|
165
|
+
end
|
166
|
+
|
167
|
+
|
168
|
+
# #################################################
|
169
|
+
#
|
170
|
+
class CommandFORMDelete
|
171
|
+
def self.call(options) # {full_url:,username:,userpass:}
|
172
|
+
new(options)
|
173
|
+
end
|
174
|
+
|
175
|
+
def json?
|
176
|
+
false
|
177
|
+
end
|
178
|
+
|
179
|
+
def uri
|
180
|
+
@_uri
|
181
|
+
end
|
182
|
+
|
183
|
+
def request
|
184
|
+
req = Net::HTTP::Delete.new(uri.request_uri)
|
185
|
+
req.basic_auth(@_username, @_userpass) if credentials?
|
186
|
+
req
|
187
|
+
end
|
188
|
+
|
189
|
+
private
|
190
|
+
|
191
|
+
def initialize(opts={})
|
192
|
+
@_username = opts[:username]
|
193
|
+
@_userpass = opts[:userpass]
|
194
|
+
@_uri = URI.parse( opts[:full_url])
|
195
|
+
end
|
196
|
+
|
197
|
+
def credentials?
|
198
|
+
!(@_username.nil? || @_userpass.nil?)
|
199
|
+
end
|
200
|
+
end
|
201
|
+
end
|
data/lib/skn_utils/version.rb
CHANGED
data/skn_utils.gemspec
CHANGED
@@ -39,7 +39,10 @@ EOF
|
|
39
39
|
spec.add_development_dependency "rake", "~> 10"
|
40
40
|
spec.add_development_dependency "rspec", '~> 3'
|
41
41
|
spec.add_development_dependency "pry", "~> 0"
|
42
|
+
spec.add_development_dependency "pry-coolline"
|
42
43
|
spec.add_development_dependency "simplecov", "~> 0"
|
43
44
|
spec.add_development_dependency 'benchmark-ips', '~> 2'
|
45
|
+
spec.add_development_dependency 'webmock'
|
46
|
+
|
44
47
|
|
45
48
|
end
|
@@ -0,0 +1,277 @@
|
|
1
|
+
##
|
2
|
+
# spec/lib/skn_utils/as_human_size_spec.rb
|
3
|
+
#
|
4
|
+
|
5
|
+
|
6
|
+
describe SknUtils::ConcurrentJobs, 'Run Multiple Jobs' do
|
7
|
+
|
8
|
+
let(:commands) {
|
9
|
+
[
|
10
|
+
SknUtils::CommandJSONPost.call(full_url: "http://example.com/posts", payload: {one: 1}),
|
11
|
+
SknUtils::CommandFORMPost.call(full_url: "http://example.com/posts", payload: {one: 1}),
|
12
|
+
SknUtils::CommandJSONGet.call(full_url: "http://example.com/posts/1"),
|
13
|
+
SknUtils::CommandJSONPut.call(full_url: "http://example.com/posts", payload: {one: 1}),
|
14
|
+
SknUtils::CommandFORMDelete.call(full_url: "http://example.com/posts/1")
|
15
|
+
]
|
16
|
+
}
|
17
|
+
|
18
|
+
let(:test_proc) {
|
19
|
+
->(cmd) { SknSuccess.(cmd.uri.request_uri, "Ok") }
|
20
|
+
}
|
21
|
+
|
22
|
+
let(:inline_failure_proc) {
|
23
|
+
->(cmd) { SknFailure.(cmd.uri.request_uri, "Failure") }
|
24
|
+
}
|
25
|
+
|
26
|
+
let(:catastrophic_proc) {
|
27
|
+
->(cmd) { SomeUnkownThing.(cmd.uri.request_uri, "Catastrophic") }
|
28
|
+
}
|
29
|
+
|
30
|
+
context "HTTP Requests " do
|
31
|
+
it "Job Commands will provide a valid http request object" do
|
32
|
+
expect(commands.any?(&:request)).to be true
|
33
|
+
end
|
34
|
+
|
35
|
+
it "Performs Http Post Requests" do
|
36
|
+
test_url = "http://jsonplaceholder.typicode.com/users"
|
37
|
+
stub_request(:post, test_url).
|
38
|
+
to_return(status: 200, body: "{\"message\":\"me\"}", headers: {})
|
39
|
+
|
40
|
+
cmd = SknUtils::CommandJSONPost.call(full_url: test_url, payload: {"one" => 1})
|
41
|
+
|
42
|
+
provider = SknUtils::ConcurrentJobs.call
|
43
|
+
provider.register_job do
|
44
|
+
SknUtils::JobWrapper.call(cmd, SknUtils::HttpProcessor)
|
45
|
+
end
|
46
|
+
result = provider.render_jobs
|
47
|
+
|
48
|
+
expect(result).to be_a(SknUtils::Result)
|
49
|
+
expect(result.success?).to be true
|
50
|
+
expect(result.values.size).to eq(1)
|
51
|
+
expect(result.values[0]).to be_a(SknSuccess)
|
52
|
+
expect(result.values[0].value).to be_a(Hash)
|
53
|
+
expect(result.values[0].value["message"]).to eq("me")
|
54
|
+
end
|
55
|
+
|
56
|
+
it "Performs Http Form Post Requests" do
|
57
|
+
test_url = "http://jsonplaceholder.typicode.com/users"
|
58
|
+
stub_request(:post, test_url).
|
59
|
+
to_return(status: 200, body: "message=me", headers: {})
|
60
|
+
|
61
|
+
cmd = SknUtils::CommandFORMPost.call(full_url: test_url, payload: {"one" => 1})
|
62
|
+
|
63
|
+
provider = SknUtils::ConcurrentJobs.call
|
64
|
+
provider.register_job do
|
65
|
+
SknUtils::JobWrapper.call(cmd, SknUtils::HttpProcessor)
|
66
|
+
end
|
67
|
+
result = provider.render_jobs
|
68
|
+
|
69
|
+
expect(result).to be_a(SknUtils::Result)
|
70
|
+
expect(result.success?).to be true
|
71
|
+
expect(result.values.size).to eq(1)
|
72
|
+
expect(result.values[0]).to be_a(SknSuccess)
|
73
|
+
expect(result.values[0].value).to be_a(String)
|
74
|
+
expect(result.values[0].value).to eq("message=me")
|
75
|
+
end
|
76
|
+
|
77
|
+
it "Performs Http Get Requests" do
|
78
|
+
test_url = "http://jsonplaceholder.typicode.com/users"
|
79
|
+
stub_request(:get, test_url).
|
80
|
+
to_return(status: 200, body: "{\"message\":\"me\"}", headers: {})
|
81
|
+
|
82
|
+
cmd = SknUtils::CommandJSONGet.call(full_url: test_url)
|
83
|
+
|
84
|
+
provider = SknUtils::ConcurrentJobs.call
|
85
|
+
provider.register_job do
|
86
|
+
SknUtils::JobWrapper.call(cmd, SknUtils::HttpProcessor)
|
87
|
+
end
|
88
|
+
result = provider.render_jobs
|
89
|
+
|
90
|
+
expect(result).to be_a(SknUtils::Result)
|
91
|
+
expect(result.success?).to be true
|
92
|
+
expect(result.values.size).to eq(1)
|
93
|
+
expect(result.values[0]).to be_a(SknSuccess)
|
94
|
+
expect(result.values[0].value).to be_a(Hash)
|
95
|
+
expect(result.values[0].value["message"]).to eq("me")
|
96
|
+
end
|
97
|
+
|
98
|
+
it "Performs Http Put Requests" do
|
99
|
+
test_url = "http://jsonplaceholder.typicode.com/users"
|
100
|
+
stub_request(:put, test_url).
|
101
|
+
to_return(status: 200, body: "{\"message\":\"me\"}", headers: {})
|
102
|
+
|
103
|
+
cmd = SknUtils::CommandJSONPut.call(full_url: test_url, payload: {"one" => 1})
|
104
|
+
|
105
|
+
provider = SknUtils::ConcurrentJobs.call
|
106
|
+
provider.register_job do
|
107
|
+
SknUtils::JobWrapper.call(cmd, SknUtils::HttpProcessor)
|
108
|
+
end
|
109
|
+
result = provider.render_jobs
|
110
|
+
|
111
|
+
expect(result).to be_a(SknUtils::Result)
|
112
|
+
expect(result.success?).to be true
|
113
|
+
expect(result.values.size).to eq(1)
|
114
|
+
expect(result.values[0]).to be_a(SknSuccess)
|
115
|
+
expect(result.values[0].value).to be_a(Hash)
|
116
|
+
expect(result.values[0].value["message"]).to eq("me")
|
117
|
+
end
|
118
|
+
|
119
|
+
it "Performs Http Form Delete Requests" do
|
120
|
+
test_url = "http://jsonplaceholder.typicode.com/users"
|
121
|
+
stub_request(:delete, test_url).
|
122
|
+
to_return(status: 200, body: "message=me", headers: {})
|
123
|
+
|
124
|
+
cmd = SknUtils::CommandFORMDelete.call(full_url: test_url, payload: {"one" => 1})
|
125
|
+
|
126
|
+
provider = SknUtils::ConcurrentJobs.call
|
127
|
+
provider.register_job do
|
128
|
+
SknUtils::JobWrapper.call(cmd, SknUtils::HttpProcessor)
|
129
|
+
end
|
130
|
+
result = provider.render_jobs
|
131
|
+
|
132
|
+
expect(result).to be_a(SknUtils::Result)
|
133
|
+
expect(result.success?).to be true
|
134
|
+
expect(result.values.size).to eq(1)
|
135
|
+
expect(result.values[0]).to be_a(SknSuccess)
|
136
|
+
expect(result.values[0].value).to be_a(String)
|
137
|
+
expect(result.values[0].value).to eq("message=me")
|
138
|
+
end
|
139
|
+
end
|
140
|
+
|
141
|
+
context "Asynchronous" do
|
142
|
+
it "Runs Jobs" do
|
143
|
+
provider = SknUtils::ConcurrentJobs.call
|
144
|
+
provider.register_jobs(commands, test_proc)
|
145
|
+
result = provider.render_jobs
|
146
|
+
|
147
|
+
expect(result).to be_a(SknUtils::Result)
|
148
|
+
expect(result.success?).to be true
|
149
|
+
expect(result.values.size).to eq(commands.size)
|
150
|
+
expect(result.values[0]).to be_a(SknSuccess)
|
151
|
+
end
|
152
|
+
it "Runs Jobs and handles inline failures" do
|
153
|
+
provider = SknUtils::ConcurrentJobs.call
|
154
|
+
|
155
|
+
provider.register_job do
|
156
|
+
SknUtils::JobWrapper.call(commands[0], test_proc)
|
157
|
+
end
|
158
|
+
provider.register_job do
|
159
|
+
SknUtils::JobWrapper.call(commands[1], inline_failure_proc)
|
160
|
+
end
|
161
|
+
provider.register_job do
|
162
|
+
SknUtils::JobWrapper.call(commands[2], inline_failure_proc)
|
163
|
+
end
|
164
|
+
provider.register_job do
|
165
|
+
SknUtils::JobWrapper.call(commands[3], inline_failure_proc)
|
166
|
+
end
|
167
|
+
provider.register_job do
|
168
|
+
SknUtils::JobWrapper.call(commands[4], test_proc)
|
169
|
+
end
|
170
|
+
|
171
|
+
result = provider.render_jobs
|
172
|
+
|
173
|
+
expect(result).to be_a(SknUtils::Result)
|
174
|
+
expect(result.success?).to be false
|
175
|
+
expect(result.values.size).to eq(commands.size)
|
176
|
+
expect(result.values[1]).to be_a(SknFailure)
|
177
|
+
expect(result.values[1].message).to eq("Failure")
|
178
|
+
end
|
179
|
+
it "Runs Jobs and handles catastrophic failures" do
|
180
|
+
provider = SknUtils::ConcurrentJobs.call
|
181
|
+
provider.register_job do
|
182
|
+
SknUtils::JobWrapper.call(command[0], test_proc)
|
183
|
+
end
|
184
|
+
provider.register_job do # notice `command` vs `commands`
|
185
|
+
SknUtils::JobWrapper.call(command[1], inline_failure_proc)
|
186
|
+
end
|
187
|
+
provider.register_job do
|
188
|
+
SknUtils::JobWrapper.call(command[2], catastrophic_proc)
|
189
|
+
end
|
190
|
+
provider.register_job do
|
191
|
+
SknUtils::JobWrapper.call(commands[3], catastrophic_proc)
|
192
|
+
end
|
193
|
+
provider.register_job do
|
194
|
+
SknUtils::JobWrapper.call(commands[4], test_proc)
|
195
|
+
end
|
196
|
+
|
197
|
+
result = provider.render_jobs
|
198
|
+
|
199
|
+
expect(result).to be_a(SknUtils::Result)
|
200
|
+
expect(result.success?).to be false
|
201
|
+
expect(result.values.size).to eq(commands.size)
|
202
|
+
expect(result.values[3].value).to eq("NameError")
|
203
|
+
expect(result.values[3]).to be_a(SknFailure)
|
204
|
+
expect(result.values[3]).to be_a(SknFailure)
|
205
|
+
expect(result.values[0]).to be nil
|
206
|
+
end
|
207
|
+
end
|
208
|
+
|
209
|
+
context "Synchronous" do
|
210
|
+
it "Runs Jobs" do
|
211
|
+
provider = SknUtils::ConcurrentJobs.call(async: false)
|
212
|
+
provider.register_jobs(commands, test_proc)
|
213
|
+
result = provider.render_jobs
|
214
|
+
|
215
|
+
expect(result).to be_a(SknUtils::Result)
|
216
|
+
expect(result.success?).to be true
|
217
|
+
expect(result.values.size).to eq(commands.size)
|
218
|
+
expect(result.values[0]).to be_a(SknSuccess)
|
219
|
+
end
|
220
|
+
it "Runs Jobs and handles inline failures" do
|
221
|
+
provider = SknUtils::ConcurrentJobs.call
|
222
|
+
|
223
|
+
provider.register_job do
|
224
|
+
SknUtils::JobWrapper.call(commands[0], test_proc)
|
225
|
+
end
|
226
|
+
provider.register_job do
|
227
|
+
SknUtils::JobWrapper.call(commands[1], inline_failure_proc)
|
228
|
+
end
|
229
|
+
provider.register_job do
|
230
|
+
SknUtils::JobWrapper.call(commands[2], inline_failure_proc)
|
231
|
+
end
|
232
|
+
provider.register_job do
|
233
|
+
SknUtils::JobWrapper.call(commands[3], inline_failure_proc)
|
234
|
+
end
|
235
|
+
provider.register_job do
|
236
|
+
SknUtils::JobWrapper.call(commands[4], test_proc)
|
237
|
+
end
|
238
|
+
|
239
|
+
result = provider.render_jobs
|
240
|
+
|
241
|
+
expect(result).to be_a(SknUtils::Result)
|
242
|
+
expect(result.success?).to be false
|
243
|
+
expect(result.values.size).to eq(commands.size)
|
244
|
+
expect(result.values[1]).to be_a(SknFailure)
|
245
|
+
expect(result.values[1].message).to eq("Failure")
|
246
|
+
end
|
247
|
+
it "Runs Jobs and handles catastrophic failures" do
|
248
|
+
provider = SknUtils::ConcurrentJobs.call
|
249
|
+
provider.register_job do
|
250
|
+
SknUtils::JobWrapper.call(commands[0], test_proc)
|
251
|
+
end
|
252
|
+
provider.register_job do
|
253
|
+
SknUtils::JobWrapper.call(commands[1], inline_failure_proc)
|
254
|
+
end
|
255
|
+
provider.register_job do
|
256
|
+
SknUtils::JobWrapper.call(commands[2], catastrophic_proc)
|
257
|
+
end
|
258
|
+
provider.register_job do
|
259
|
+
SknUtils::JobWrapper.call(commands[3], catastrophic_proc)
|
260
|
+
end
|
261
|
+
provider.register_job do
|
262
|
+
SknUtils::JobWrapper.call(commands[4], test_proc)
|
263
|
+
end
|
264
|
+
|
265
|
+
result = provider.render_jobs
|
266
|
+
|
267
|
+
expect(result).to be_a(SknUtils::Result)
|
268
|
+
expect(result.success?).to be false
|
269
|
+
expect(result.values.size).to eq(commands.size)
|
270
|
+
expect(result.values[3].value).to eq("NameError")
|
271
|
+
expect(result.values[2]).to be_a(SknFailure)
|
272
|
+
expect(result.values[1]).to be_a(SknFailure)
|
273
|
+
expect(result.values[0]).to be_a(SknSuccess)
|
274
|
+
end
|
275
|
+
end
|
276
|
+
|
277
|
+
end
|
data/spec/spec_helper.rb
CHANGED
@@ -15,6 +15,8 @@ end
|
|
15
15
|
require 'skn_utils'
|
16
16
|
require 'rspec'
|
17
17
|
|
18
|
+
require 'webmock/rspec'
|
19
|
+
|
18
20
|
# See http://rubydoc.info/gems/rspec-core/RSpec/Core/Configuration
|
19
21
|
RSpec.configure do |config|
|
20
22
|
Kernel.srand config.seed
|
@@ -30,6 +32,8 @@ RSpec.configure do |config|
|
|
30
32
|
# config.disable_monkey_patching! # -- breaks rspec runtime
|
31
33
|
config.warnings = true
|
32
34
|
|
35
|
+
config.include WebMock::API
|
36
|
+
|
33
37
|
if config.files_to_run.one?
|
34
38
|
config.formatter = :documentation
|
35
39
|
else
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: skn_utils
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 5.
|
4
|
+
version: 5.5.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- James Scott Jr
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date: 2019-
|
11
|
+
date: 2019-02-25 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: deep_merge
|
@@ -108,6 +108,20 @@ dependencies:
|
|
108
108
|
- - "~>"
|
109
109
|
- !ruby/object:Gem::Version
|
110
110
|
version: '0'
|
111
|
+
- !ruby/object:Gem::Dependency
|
112
|
+
name: pry-coolline
|
113
|
+
requirement: !ruby/object:Gem::Requirement
|
114
|
+
requirements:
|
115
|
+
- - ">="
|
116
|
+
- !ruby/object:Gem::Version
|
117
|
+
version: '0'
|
118
|
+
type: :development
|
119
|
+
prerelease: false
|
120
|
+
version_requirements: !ruby/object:Gem::Requirement
|
121
|
+
requirements:
|
122
|
+
- - ">="
|
123
|
+
- !ruby/object:Gem::Version
|
124
|
+
version: '0'
|
111
125
|
- !ruby/object:Gem::Dependency
|
112
126
|
name: simplecov
|
113
127
|
requirement: !ruby/object:Gem::Requirement
|
@@ -136,6 +150,20 @@ dependencies:
|
|
136
150
|
- - "~>"
|
137
151
|
- !ruby/object:Gem::Version
|
138
152
|
version: '2'
|
153
|
+
- !ruby/object:Gem::Dependency
|
154
|
+
name: webmock
|
155
|
+
requirement: !ruby/object:Gem::Requirement
|
156
|
+
requirements:
|
157
|
+
- - ">="
|
158
|
+
- !ruby/object:Gem::Version
|
159
|
+
version: '0'
|
160
|
+
type: :development
|
161
|
+
prerelease: false
|
162
|
+
version_requirements: !ruby/object:Gem::Requirement
|
163
|
+
requirements:
|
164
|
+
- - ">="
|
165
|
+
- !ruby/object:Gem::Version
|
166
|
+
version: '0'
|
139
167
|
description: "The intent of the NestedResult class is to be a container for data values
|
140
168
|
composed of key/value pairs, \nwith easy access to its contents, and on-demand transformation
|
141
169
|
back to the hash (#to_hash).\n\nReview the RSpec tests, and or review the README
|
@@ -156,6 +184,10 @@ files:
|
|
156
184
|
- Rakefile
|
157
185
|
- _config.yml
|
158
186
|
- bin/bench_nested_result.rb
|
187
|
+
- bin/concurrent_test_block
|
188
|
+
- bin/concurrent_test_grouped
|
189
|
+
- bin/concurrent_test_procs
|
190
|
+
- bin/concurrent_test_wrapped
|
159
191
|
- bin/configs/settings.yml
|
160
192
|
- bin/configs/settings/development.yml
|
161
193
|
- bin/configs/settings/production.yml
|
@@ -172,11 +204,14 @@ files:
|
|
172
204
|
- lib/skn_settings.rb
|
173
205
|
- lib/skn_success.rb
|
174
206
|
- lib/skn_utils.rb
|
207
|
+
- lib/skn_utils/concurrent_jobs.rb
|
175
208
|
- lib/skn_utils/configurable.rb
|
176
209
|
- lib/skn_utils/configuration.rb
|
177
210
|
- lib/skn_utils/core_extensions.rb
|
178
211
|
- lib/skn_utils/dotted_hash.rb
|
179
212
|
- lib/skn_utils/env_string_handler.rb
|
213
|
+
- lib/skn_utils/http_processor.rb
|
214
|
+
- lib/skn_utils/job_commands.rb
|
180
215
|
- lib/skn_utils/nested_result.rb
|
181
216
|
- lib/skn_utils/notifier_base.rb
|
182
217
|
- lib/skn_utils/null_object.rb
|
@@ -196,6 +231,7 @@ files:
|
|
196
231
|
- spec/lib/skn_settings_spec.rb
|
197
232
|
- spec/lib/skn_utils/as_human_size_spec.rb
|
198
233
|
- spec/lib/skn_utils/catch_exceptions_spec.rb
|
234
|
+
- spec/lib/skn_utils/concurrent_jobs_spec.rb
|
199
235
|
- spec/lib/skn_utils/configurable_spec.rb
|
200
236
|
- spec/lib/skn_utils/container_spec.rb
|
201
237
|
- spec/lib/skn_utils/nested_result_spec.rb
|
@@ -246,6 +282,7 @@ test_files:
|
|
246
282
|
- spec/lib/skn_settings_spec.rb
|
247
283
|
- spec/lib/skn_utils/as_human_size_spec.rb
|
248
284
|
- spec/lib/skn_utils/catch_exceptions_spec.rb
|
285
|
+
- spec/lib/skn_utils/concurrent_jobs_spec.rb
|
249
286
|
- spec/lib/skn_utils/configurable_spec.rb
|
250
287
|
- spec/lib/skn_utils/container_spec.rb
|
251
288
|
- spec/lib/skn_utils/nested_result_spec.rb
|