pdi 1.0.0.pre.alpha.1 → 2.1.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.rubocop.yml +3 -3
- data/.ruby-version +1 -1
- data/.travis.yml +3 -4
- data/CHANGELOG.md +25 -0
- data/README.md +19 -7
- data/lib/pdi.rb +2 -0
- data/lib/pdi/executor.rb +48 -12
- data/lib/pdi/executor/result.rb +1 -1
- data/lib/pdi/executor/status.rb +2 -3
- data/lib/pdi/spoon.rb +43 -14
- data/lib/pdi/spoon/kitchen_error.rb +2 -0
- data/lib/pdi/spoon/options.rb +0 -11
- data/lib/pdi/spoon/pan_error.rb +2 -0
- data/lib/pdi/version.rb +1 -1
- data/pdi.gemspec +2 -2
- data/spec/mocks/spoon/sleep.sh +10 -0
- data/spec/pdi/executor_spec.rb +64 -0
- data/spec/pdi/spoon/kitchen_error_spec.rb +42 -0
- data/spec/pdi/spoon/{error_spec.rb → pan_error_spec.rb} +0 -0
- data/spec/pdi/spoon_spec.rb +40 -11
- metadata +14 -7
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 7aad69dc7048e9bc7b4c017d42b2343bca49c98718d1ba3d44981bb4434562f4
|
4
|
+
data.tar.gz: 5405f5f819e2ae116de4831d5a609b32cbf0d08b9117992f3126ca0eefc39e67
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 2e3fbfc5cb28c486153acda733620c63c21ccb7a7521cb8534de0c153364ff01d12f0af0fcd1a495eeaa8c146430faf002f3c9bd06d1e1ce6a9b35df12af6f76
|
7
|
+
data.tar.gz: c220cf297f8b8a3377fc1f6287e236efdb1549867680bdf6e52a3e9b905e6e717e4420df093d1be537dcb768540a8040e23bac3eaea12aa49cfbb1282c2ef8ff
|
data/.rubocop.yml
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
|
1
|
+
Layout/LineLength:
|
2
2
|
Max: 100
|
3
3
|
Exclude:
|
4
4
|
- pdi.gemspec
|
@@ -13,10 +13,10 @@ Metrics/BlockLength:
|
|
13
13
|
- define
|
14
14
|
|
15
15
|
Metrics/MethodLength:
|
16
|
-
Max:
|
16
|
+
Max: 17
|
17
17
|
|
18
18
|
AllCops:
|
19
|
-
TargetRubyVersion: 2.
|
19
|
+
TargetRubyVersion: 2.5
|
20
20
|
|
21
21
|
Metrics/AbcSize:
|
22
22
|
Max: 16
|
data/.ruby-version
CHANGED
@@ -1 +1 @@
|
|
1
|
-
2.6.
|
1
|
+
2.6.6
|
data/.travis.yml
CHANGED
@@ -4,10 +4,9 @@ env:
|
|
4
4
|
language: ruby
|
5
5
|
rvm:
|
6
6
|
# Build on the latest stable of all supported Rubies (https://www.ruby-lang.org/en/downloads/):
|
7
|
-
- 2.
|
8
|
-
- 2.
|
9
|
-
- 2.
|
10
|
-
- 2.6.5
|
7
|
+
- 2.5.8
|
8
|
+
- 2.6.6
|
9
|
+
- 2.7.1
|
11
10
|
cache: bundler
|
12
11
|
before_script:
|
13
12
|
- curl -L https://codeclimate.com/downloads/test-reporter/test-reporter-latest-linux-amd64 > ./cc-test-reporter
|
data/CHANGELOG.md
CHANGED
@@ -0,0 +1,25 @@
|
|
1
|
+
# 2.1.0 (June 3rd, 2020)
|
2
|
+
|
3
|
+
Enhancements:
|
4
|
+
|
5
|
+
* Support for streamed command output.
|
6
|
+
|
7
|
+
# 2.0.0 (May 11th, 2020)
|
8
|
+
|
9
|
+
Breaking Changes:
|
10
|
+
|
11
|
+
* Standard error and output have been combined into one stream (out). Early feedback indicated that reading both at the same time was preferable.
|
12
|
+
|
13
|
+
Enhancements:
|
14
|
+
|
15
|
+
* Added optional `timeout_in_seconds` argument to Pdi::Spoon#initialize.
|
16
|
+
|
17
|
+
# 1.0.1 (February 19th, 2020)
|
18
|
+
|
19
|
+
Fixes:
|
20
|
+
|
21
|
+
* It now properly calls kitchen for jobs (it was previously calling pan.)
|
22
|
+
|
23
|
+
# 1.0.0 (February 19th, 2020)
|
24
|
+
|
25
|
+
Initial release.
|
data/README.md
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
#
|
1
|
+
# PDI
|
2
2
|
|
3
3
|
---
|
4
4
|
|
@@ -36,17 +36,21 @@ Pull Requests are welcome for:
|
|
36
36
|
|
37
37
|
## Examples
|
38
38
|
|
39
|
-
All examples assume
|
39
|
+
All examples assume PDI has been installed to your home directory: `~/data-integration`.
|
40
40
|
|
41
41
|
### Creating a Spoon Instance
|
42
42
|
|
43
|
-
`Pdi::Spoon` is the common interface you will use when interacting with
|
43
|
+
`Pdi::Spoon` is the common interface you will use when interacting with PDI. It will use Pan and Kitchen for executing Spoon commands.
|
44
44
|
|
45
45
|
```ruby
|
46
|
-
spoon = Pdi::Spoon.new(dir:
|
46
|
+
spoon = Pdi::Spoon.new(dir: '~/data-integration')
|
47
47
|
```
|
48
48
|
|
49
|
-
|
49
|
+
Notes:
|
50
|
+
|
51
|
+
* You can also override the names of the scripts using the `kitchen` and `pan` constructor keyword arguments. The defaults are `kitchen.sh` and `pan.sh`, respectively.
|
52
|
+
* For other command line arguments that are not supported first-class in the Options objects below you can utilize the `args` argument when instantiating a `Spoon` instance.
|
53
|
+
* Another optional argument is `timeout_in_seconds`. It is set to `nil` by default which means there is no timeout. If set it will ensure the sub-process runs within a given window. If it times out the sub-process will be terminated and a Timeout::Error will be raised.
|
50
54
|
|
51
55
|
### Executing a Job/Transformation
|
52
56
|
|
@@ -66,12 +70,20 @@ result = spoon.run(options)
|
|
66
70
|
|
67
71
|
`Spoon#run` will return:
|
68
72
|
|
69
|
-
* `Pdi::
|
73
|
+
* `Pdi::Executor::Result` upon a successful run.
|
70
74
|
* If a non-zero exit code was returned then a `Pdi::Spoon::PanError` or `Pdi::Spoon::KitchenError` will be raised.
|
71
75
|
|
72
76
|
You can access the raw command line results by tapping into the execution attribute of the result or error object.
|
73
77
|
|
74
|
-
Note: Not all options are currently supported. See
|
78
|
+
Note: Not all options are currently supported. See PDI's official references for [Pan](https://help.pentaho.com/Documentation/6.1/0L0/0Y0/070/000) and [Kitchen](https://help.pentaho.com/Documentation/6.1/0L0/0Y0/070/010) to see all options.
|
79
|
+
|
80
|
+
### Output
|
81
|
+
|
82
|
+
There are two ways to see the output of a `Pdi::Spoon` run. First, the output is available when a run completes through `Pdi::Executor::Result#out`. It is also possible to get the output throughout the run by passing a block to run. For example:
|
83
|
+
|
84
|
+
````ruby
|
85
|
+
spoon.run(options) { |output| print output }
|
86
|
+
````
|
75
87
|
|
76
88
|
## Contributing
|
77
89
|
|
data/lib/pdi.rb
CHANGED
data/lib/pdi/executor.rb
CHANGED
@@ -13,20 +13,56 @@ module Pdi
|
|
13
13
|
# This class is the library's "metal" layer, the one which actually makes the system call and
|
14
14
|
# interacts with the operating system (through Ruby's standard library.)
|
15
15
|
class Executor
|
16
|
-
|
16
|
+
attr_reader :timeout_in_seconds
|
17
|
+
|
18
|
+
def initialize(timeout_in_seconds: nil)
|
19
|
+
@timeout_in_seconds = timeout_in_seconds
|
20
|
+
|
21
|
+
freeze
|
22
|
+
end
|
23
|
+
|
24
|
+
def run(args, &streaming_reader)
|
17
25
|
args = Array(args).map(&:to_s)
|
18
26
|
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
27
|
+
IO.popen(args, err: %i[child out]) do |io|
|
28
|
+
io_read = read(io, &streaming_reader)
|
29
|
+
io.close
|
30
|
+
status = $CHILD_STATUS
|
31
|
+
|
32
|
+
Result.new(
|
33
|
+
args: args,
|
34
|
+
status: {
|
35
|
+
code: status.exitstatus,
|
36
|
+
out: io_read,
|
37
|
+
pid: status.pid
|
38
|
+
}
|
39
|
+
)
|
40
|
+
rescue Timeout::Error => e
|
41
|
+
Process.kill(9, io.pid)
|
42
|
+
raise e
|
43
|
+
end
|
44
|
+
end
|
45
|
+
|
46
|
+
private
|
47
|
+
|
48
|
+
def read(io, &streaming_reader)
|
49
|
+
if timeout_in_seconds
|
50
|
+
Timeout.timeout(timeout_in_seconds) { streamed_read(io, &streaming_reader) }
|
51
|
+
else
|
52
|
+
streamed_read(io, &streaming_reader)
|
53
|
+
end
|
54
|
+
end
|
55
|
+
|
56
|
+
def streamed_read(io, &streaming_reader)
|
57
|
+
return io.read unless block_given?
|
58
|
+
|
59
|
+
read = ''
|
60
|
+
io.each_line do |line_read|
|
61
|
+
streaming_reader.call(line_read)
|
62
|
+
read += line_read
|
63
|
+
end
|
64
|
+
|
65
|
+
read
|
30
66
|
end
|
31
67
|
end
|
32
68
|
end
|
data/lib/pdi/executor/result.rb
CHANGED
data/lib/pdi/executor/status.rb
CHANGED
@@ -13,12 +13,11 @@ module Pdi
|
|
13
13
|
class Status
|
14
14
|
acts_as_hashable
|
15
15
|
|
16
|
-
attr_reader :code, :out, :
|
16
|
+
attr_reader :code, :out, :pid
|
17
17
|
|
18
|
-
def initialize(code:, out: '',
|
18
|
+
def initialize(code:, out: '', pid:)
|
19
19
|
@code = code
|
20
20
|
@out = out
|
21
|
-
@err = err
|
22
21
|
@pid = pid
|
23
22
|
|
24
23
|
freeze
|
data/lib/pdi/spoon.rb
CHANGED
@@ -7,33 +7,41 @@
|
|
7
7
|
# LICENSE file in the root directory of this source tree.
|
8
8
|
#
|
9
9
|
|
10
|
+
require_relative 'spoon/kitchen_error'
|
10
11
|
require_relative 'spoon/options'
|
12
|
+
require_relative 'spoon/pan_error'
|
11
13
|
require_relative 'spoon/parser'
|
12
14
|
require_relative 'spoon/result'
|
13
15
|
|
14
16
|
module Pdi
|
15
|
-
# This class is the main wrapper for
|
17
|
+
# This class is the main wrapper for PDI's pan and kitchen scripts.
|
16
18
|
class Spoon
|
17
19
|
DEFAULT_KITCHEN = 'kitchen.sh'
|
18
20
|
DEFAULT_PAN = 'pan.sh'
|
19
21
|
|
20
|
-
|
22
|
+
TYPES_TO_ERRORS = {
|
23
|
+
Options::Type::JOB => KitchenError,
|
24
|
+
Options::Type::TRANSFORMATION => PanError
|
25
|
+
}.freeze
|
26
|
+
|
27
|
+
attr_reader :args, :dir, :executor, :kitchen, :pan
|
21
28
|
|
22
29
|
def initialize(
|
23
30
|
args: [],
|
24
31
|
dir:,
|
25
32
|
kitchen: DEFAULT_KITCHEN,
|
26
|
-
pan: DEFAULT_PAN
|
33
|
+
pan: DEFAULT_PAN,
|
34
|
+
timeout_in_seconds: nil
|
27
35
|
)
|
28
36
|
assert_required(:dir, dir)
|
29
37
|
assert_required(:kitchen, kitchen)
|
30
38
|
assert_required(:pan, pan)
|
31
39
|
|
32
40
|
@args = Array(args)
|
33
|
-
@dir = dir.to_s
|
41
|
+
@dir = File.expand_path(dir.to_s)
|
34
42
|
@kitchen = kitchen.to_s
|
35
43
|
@pan = pan.to_s
|
36
|
-
@executor = Executor.new
|
44
|
+
@executor = Executor.new(timeout_in_seconds: timeout_in_seconds)
|
37
45
|
@parser = Parser.new
|
38
46
|
|
39
47
|
freeze
|
@@ -52,20 +60,41 @@ module Pdi
|
|
52
60
|
Result.new(result, version_line)
|
53
61
|
end
|
54
62
|
|
55
|
-
# Returns
|
56
|
-
# a PanError (transformation) or KitchenError (job).
|
57
|
-
|
58
|
-
|
59
|
-
|
60
|
-
|
61
|
-
|
62
|
-
|
63
|
+
# Returns a <tt>Pdi::Executor::Result</tt> instance when PDI returns error
|
64
|
+
# code 0 or else raises a PanError (transformation) or KitchenError (job).
|
65
|
+
#
|
66
|
+
# An optional block may be passed in so that the output of a run is
|
67
|
+
# available in a streaming manner during the run. This block takes one
|
68
|
+
# parameter which is the current chunk of output and is called repeatedly
|
69
|
+
# throughout the run.
|
70
|
+
def run(options, &streaming_reader)
|
71
|
+
options = Options.make(options)
|
72
|
+
all_args = run_args(options)
|
73
|
+
|
74
|
+
executor.run(all_args, &streaming_reader).tap do |result|
|
75
|
+
raise(error_constant(options), result) if result.code != 0
|
63
76
|
end
|
64
77
|
end
|
65
78
|
|
66
79
|
private
|
67
80
|
|
68
|
-
attr_reader :
|
81
|
+
attr_reader :parser
|
82
|
+
|
83
|
+
def error_constant(options)
|
84
|
+
TYPES_TO_ERRORS.fetch(options.type)
|
85
|
+
end
|
86
|
+
|
87
|
+
def run_args(options)
|
88
|
+
[script_path(options.type)] + args + options.to_args
|
89
|
+
end
|
90
|
+
|
91
|
+
def script_path(options_type)
|
92
|
+
if options_type == Options::Type::JOB
|
93
|
+
kitchen_path
|
94
|
+
elsif options_type == Options::Type::TRANSFORMATION
|
95
|
+
pan_path
|
96
|
+
end
|
97
|
+
end
|
69
98
|
|
70
99
|
def kitchen_path
|
71
100
|
File.join(dir, kitchen)
|
@@ -11,6 +11,8 @@ module Pdi
|
|
11
11
|
class Spoon
|
12
12
|
# This class subclasses Ruby's StandardError and provides a mapping to custom Kitchen
|
13
13
|
# specific error codes to messages.
|
14
|
+
# You can find the list of errors in Pentaho's documentation site, for example:
|
15
|
+
# https://help.pentaho.com/Documentation/8.0/Products/Data_Integration/Command_Line_Tools
|
14
16
|
class KitchenError < StandardError
|
15
17
|
MESSAGES = {
|
16
18
|
'1' => 'Errors occurred during processing',
|
data/lib/pdi/spoon/options.rb
CHANGED
@@ -7,10 +7,8 @@
|
|
7
7
|
# LICENSE file in the root directory of this source tree.
|
8
8
|
#
|
9
9
|
|
10
|
-
require_relative 'kitchen_error'
|
11
10
|
require_relative 'options/level'
|
12
11
|
require_relative 'options/param'
|
13
|
-
require_relative 'pan_error'
|
14
12
|
|
15
13
|
module Pdi
|
16
14
|
class Spoon
|
@@ -23,11 +21,6 @@ module Pdi
|
|
23
21
|
TRANSFORMATION = :transformation
|
24
22
|
end
|
25
23
|
|
26
|
-
TYPES_TO_ERRORS = {
|
27
|
-
Type::JOB => KitchenError,
|
28
|
-
Type::TRANSFORMATION => PanError
|
29
|
-
}.freeze
|
30
|
-
|
31
24
|
TYPES_TO_KEYS = {
|
32
25
|
Type::JOB => Arg::Key::JOB,
|
33
26
|
Type::TRANSFORMATION => Arg::Key::TRANS
|
@@ -63,10 +56,6 @@ module Pdi
|
|
63
56
|
base_args + param_args
|
64
57
|
end
|
65
58
|
|
66
|
-
def error_constant
|
67
|
-
TYPES_TO_ERRORS.fetch(type)
|
68
|
-
end
|
69
|
-
|
70
59
|
private
|
71
60
|
|
72
61
|
def key
|
data/lib/pdi/spoon/pan_error.rb
CHANGED
@@ -11,6 +11,8 @@ module Pdi
|
|
11
11
|
class Spoon
|
12
12
|
# This class subclasses Ruby's StandardError and provides a mapping to custom Pan
|
13
13
|
# specific error codes to messages.
|
14
|
+
# You can find the list of errors in Pentaho's documentation site, for example:
|
15
|
+
# https://help.pentaho.com/Documentation/8.0/Products/Data_Integration/Command_Line_Tools
|
14
16
|
class PanError < StandardError
|
15
17
|
MESSAGES = {
|
16
18
|
'1' => 'Errors occurred during processing',
|
data/lib/pdi/version.rb
CHANGED
data/pdi.gemspec
CHANGED
@@ -11,7 +11,7 @@ Gem::Specification.new do |s|
|
|
11
11
|
Pentaho Data Integration allows for the creation of ETL transformation and jobs. This library allows those ETL tasks to be executed from Ruby.
|
12
12
|
DESCRIPTION
|
13
13
|
|
14
|
-
s.authors = ['Matthew Ruggio']
|
14
|
+
s.authors = ['Matthew Ruggio', 'Ryan Gerry']
|
15
15
|
s.email = ['mruggio@bluemarblepayroll.com']
|
16
16
|
s.files = `git ls-files`.split("\n")
|
17
17
|
s.test_files = `git ls-files -- {test,spec,features}/*`.split("\n")
|
@@ -19,7 +19,7 @@ Gem::Specification.new do |s|
|
|
19
19
|
s.homepage = 'https://github.com/bluemarblepayroll/pdi'
|
20
20
|
s.license = 'MIT'
|
21
21
|
|
22
|
-
s.required_ruby_version = '>= 2.
|
22
|
+
s.required_ruby_version = '>= 2.5.8'
|
23
23
|
|
24
24
|
s.add_dependency('acts_as_hashable', '~>1')
|
25
25
|
|
@@ -0,0 +1,64 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
#
|
4
|
+
# Copyright (c) 2018-present, Blue Marble Payroll, LLC
|
5
|
+
#
|
6
|
+
# This source code is licensed under the MIT license found in the
|
7
|
+
# LICENSE file in the root directory of this source tree.
|
8
|
+
#
|
9
|
+
|
10
|
+
require 'spec_helper'
|
11
|
+
|
12
|
+
describe Pdi::Executor do
|
13
|
+
let(:script) { File.join('spec', 'mocks', 'spoon', 'sleep.sh') }
|
14
|
+
let(:one_second) { 1 }
|
15
|
+
let(:return_code) { 33 }
|
16
|
+
let(:script_output) { "std_out\nerr_out\nafter_sleep\n" }
|
17
|
+
|
18
|
+
describe '#run' do
|
19
|
+
context 'with a timeout' do
|
20
|
+
# do not make these too high, bugs could cause the entire script to still be executed without
|
21
|
+
# killing it so these high values could draw out results.
|
22
|
+
let(:thirty_seconds) { 30 }
|
23
|
+
|
24
|
+
subject { described_class.new(timeout_in_seconds: one_second) }
|
25
|
+
|
26
|
+
# This will run a script that will take 30 seconds to process, but by limiting the
|
27
|
+
# timeout using the #run argument, it should raise an error after one second.
|
28
|
+
it 'times out and kills process after 1 second' do
|
29
|
+
args = [script, thirty_seconds]
|
30
|
+
|
31
|
+
expect { subject.run(args) }.to raise_error(Timeout::Error)
|
32
|
+
end
|
33
|
+
end
|
34
|
+
|
35
|
+
context 'without a timeout' do
|
36
|
+
it 'returns right exit status as code' do
|
37
|
+
args = [script, one_second, return_code]
|
38
|
+
|
39
|
+
result = subject.run(args)
|
40
|
+
|
41
|
+
expect(result.code).to eq(return_code)
|
42
|
+
end
|
43
|
+
|
44
|
+
it 'returns right standard output and error as out' do
|
45
|
+
args = [script, one_second, return_code]
|
46
|
+
|
47
|
+
result = subject.run(args)
|
48
|
+
|
49
|
+
expect(result.out).to eq script_output
|
50
|
+
end
|
51
|
+
|
52
|
+
describe 'streaming output' do
|
53
|
+
it 'accepts a block and passes in the Pentaho output and populates result output' do
|
54
|
+
found_output = ''
|
55
|
+
|
56
|
+
result = subject.run([script, 0, return_code]) { |output| found_output += output }
|
57
|
+
|
58
|
+
expect(found_output).to eq script_output
|
59
|
+
expect(result.out).to eq script_output
|
60
|
+
end
|
61
|
+
end
|
62
|
+
end
|
63
|
+
end
|
64
|
+
end
|
@@ -0,0 +1,42 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
#
|
4
|
+
# Copyright (c) 2018-present, Blue Marble Payroll, LLC
|
5
|
+
#
|
6
|
+
# This source code is licensed under the MIT license found in the
|
7
|
+
# LICENSE file in the root directory of this source tree.
|
8
|
+
#
|
9
|
+
|
10
|
+
require 'spec_helper'
|
11
|
+
|
12
|
+
describe Pdi::Spoon::KitchenError do
|
13
|
+
describe 'initialization' do
|
14
|
+
[1, 2, 7, 8, 9].each do |code|
|
15
|
+
specify "code #{code} should have message" do
|
16
|
+
result = Pdi::Executor::Result.new(
|
17
|
+
args: [],
|
18
|
+
status: {
|
19
|
+
code: code,
|
20
|
+
pid: 123
|
21
|
+
}
|
22
|
+
)
|
23
|
+
|
24
|
+
expect(described_class.new(result).message).not_to eq('Unknown')
|
25
|
+
end
|
26
|
+
end
|
27
|
+
|
28
|
+
[-1, 0, 3, 4, 5, 6, 10, 11].each do |code|
|
29
|
+
specify "code #{code} should not have message" do
|
30
|
+
result = Pdi::Executor::Result.new(
|
31
|
+
args: [],
|
32
|
+
status: {
|
33
|
+
code: code,
|
34
|
+
pid: 123
|
35
|
+
}
|
36
|
+
)
|
37
|
+
|
38
|
+
expect(described_class.new(result).message).to eq('Unknown')
|
39
|
+
end
|
40
|
+
end
|
41
|
+
end
|
42
|
+
end
|
File without changes
|
data/spec/pdi/spoon_spec.rb
CHANGED
@@ -14,7 +14,19 @@ describe Pdi::Spoon do
|
|
14
14
|
let(:mocks_dir) { %w[spec mocks spoon] }
|
15
15
|
let(:dir) { File.join(*mocks_dir) }
|
16
16
|
|
17
|
+
describe '#initialize' do
|
18
|
+
it 'sets Executor#timeout_in_seconds' do
|
19
|
+
timeout_in_seconds = 987
|
20
|
+
subject = described_class.new(dir: dir, timeout_in_seconds: timeout_in_seconds)
|
21
|
+
executor = subject.executor
|
22
|
+
|
23
|
+
expect(executor.timeout_in_seconds).to eq(timeout_in_seconds)
|
24
|
+
end
|
25
|
+
end
|
26
|
+
|
17
27
|
describe '#run' do
|
28
|
+
let(:script_output) { "output to stdout\noutput to sterr\n" }
|
29
|
+
|
18
30
|
context 'transformations' do
|
19
31
|
let(:options) do
|
20
32
|
{
|
@@ -24,21 +36,31 @@ describe Pdi::Spoon do
|
|
24
36
|
}
|
25
37
|
end
|
26
38
|
|
39
|
+
it 'will call pan script' do
|
40
|
+
subject = described_class.new(args: 0, dir: dir, pan: script)
|
41
|
+
|
42
|
+
result = subject.run(options)
|
43
|
+
|
44
|
+
expect(result.args.first).to include(script)
|
45
|
+
end
|
46
|
+
|
27
47
|
context 'when code is 0' do
|
28
|
-
|
29
|
-
subject = described_class.new(
|
30
|
-
args: 0,
|
31
|
-
dir: dir,
|
32
|
-
kitchen: script,
|
33
|
-
pan: script
|
34
|
-
)
|
48
|
+
let(:subject) { described_class.new(args: 0, dir: dir, kitchen: script, pan: script) }
|
35
49
|
|
50
|
+
it 'returns correct stdout, stderr and code' do
|
36
51
|
result = subject.run(options)
|
37
52
|
|
38
|
-
expect(result.out).to
|
39
|
-
expect(result.err).to eq("output to sterr\n")
|
53
|
+
expect(result.out).to eq script_output
|
40
54
|
expect(result.code).to eq(0)
|
41
55
|
end
|
56
|
+
|
57
|
+
it 'allows for streaming output when passed a block' do
|
58
|
+
found_output = ''
|
59
|
+
|
60
|
+
subject.run(options) { |output| found_output += output }
|
61
|
+
|
62
|
+
expect(found_output).to eq script_output
|
63
|
+
end
|
42
64
|
end
|
43
65
|
|
44
66
|
[1, 2, 3, 7, 8, 9].each do |code|
|
@@ -70,6 +92,14 @@ describe Pdi::Spoon do
|
|
70
92
|
}
|
71
93
|
end
|
72
94
|
|
95
|
+
it 'will call kitchen script' do
|
96
|
+
subject = described_class.new(args: 0, dir: dir, kitchen: script)
|
97
|
+
|
98
|
+
result = subject.run(options)
|
99
|
+
|
100
|
+
expect(result.args.first).to include(script)
|
101
|
+
end
|
102
|
+
|
73
103
|
context 'when code is 0' do
|
74
104
|
it 'returns correct stdout, stderr and code' do
|
75
105
|
subject = described_class.new(
|
@@ -81,8 +111,7 @@ describe Pdi::Spoon do
|
|
81
111
|
|
82
112
|
result = subject.run(options)
|
83
113
|
|
84
|
-
expect(result.out).to eq("output to stdout\n")
|
85
|
-
expect(result.err).to eq("output to sterr\n")
|
114
|
+
expect(result.out).to eq("output to stdout\noutput to sterr\n")
|
86
115
|
expect(result.code).to eq(0)
|
87
116
|
end
|
88
117
|
end
|
metadata
CHANGED
@@ -1,14 +1,15 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pdi
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.0
|
4
|
+
version: 2.1.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matthew Ruggio
|
8
|
+
- Ryan Gerry
|
8
9
|
autorequire:
|
9
10
|
bindir: bin
|
10
11
|
cert_chain: []
|
11
|
-
date: 2020-
|
12
|
+
date: 2020-06-09 00:00:00.000000000 Z
|
12
13
|
dependencies:
|
13
14
|
- !ruby/object:Gem::Dependency
|
14
15
|
name: acts_as_hashable
|
@@ -160,11 +161,14 @@ files:
|
|
160
161
|
- lib/pdi/version.rb
|
161
162
|
- pdi.gemspec
|
162
163
|
- spec/mocks/spoon/return_code.sh
|
164
|
+
- spec/mocks/spoon/sleep.sh
|
163
165
|
- spec/mocks/spoon/version.sh
|
164
|
-
- spec/pdi/
|
166
|
+
- spec/pdi/executor_spec.rb
|
167
|
+
- spec/pdi/spoon/kitchen_error_spec.rb
|
165
168
|
- spec/pdi/spoon/options/arg_spec.rb
|
166
169
|
- spec/pdi/spoon/options/param_spec.rb
|
167
170
|
- spec/pdi/spoon/options_spec.rb
|
171
|
+
- spec/pdi/spoon/pan_error_spec.rb
|
168
172
|
- spec/pdi/spoon_spec.rb
|
169
173
|
- spec/spec_helper.rb
|
170
174
|
homepage: https://github.com/bluemarblepayroll/pdi
|
@@ -179,12 +183,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
179
183
|
requirements:
|
180
184
|
- - ">="
|
181
185
|
- !ruby/object:Gem::Version
|
182
|
-
version: 2.
|
186
|
+
version: 2.5.8
|
183
187
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
184
188
|
requirements:
|
185
|
-
- - "
|
189
|
+
- - ">="
|
186
190
|
- !ruby/object:Gem::Version
|
187
|
-
version:
|
191
|
+
version: '0'
|
188
192
|
requirements: []
|
189
193
|
rubygems_version: 3.0.3
|
190
194
|
signing_key:
|
@@ -192,10 +196,13 @@ specification_version: 4
|
|
192
196
|
summary: Ruby wrapper for invoking Pentaho Data Integration
|
193
197
|
test_files:
|
194
198
|
- spec/mocks/spoon/return_code.sh
|
199
|
+
- spec/mocks/spoon/sleep.sh
|
195
200
|
- spec/mocks/spoon/version.sh
|
196
|
-
- spec/pdi/
|
201
|
+
- spec/pdi/executor_spec.rb
|
202
|
+
- spec/pdi/spoon/kitchen_error_spec.rb
|
197
203
|
- spec/pdi/spoon/options/arg_spec.rb
|
198
204
|
- spec/pdi/spoon/options/param_spec.rb
|
199
205
|
- spec/pdi/spoon/options_spec.rb
|
206
|
+
- spec/pdi/spoon/pan_error_spec.rb
|
200
207
|
- spec/pdi/spoon_spec.rb
|
201
208
|
- spec/spec_helper.rb
|