burner 1.4.0.pre.alpha → 1.6.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/CHANGELOG.md +12 -1
- data/README.md +2 -1
- data/lib/burner/jobs.rb +1 -0
- data/lib/burner/library.rb +1 -0
- data/lib/burner/library/collection/zip.rb +53 -0
- data/lib/burner/library/io/write.rb +28 -1
- data/lib/burner/version.rb +1 -1
- metadata +5 -4
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA256:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: aa6bd0e3067707c53c82cd82816d3320ebe687ba071fa6bbbf97707035b5f46f
|
4
|
+
data.tar.gz: d104558bc6dcd8e4dd7186e7f5365294d3e421ff3e7fc13fe28765d3f7a69b7d
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 47ff0ccaba50a39709a13b84f6727876a480dd8f4c6c420a9e374396db30fa9c74e64d62a32c4daf452320eca6266b662f3710d7f86262fbc9507391752a5d9c
|
7
|
+
data.tar.gz: d0900276a49941b883fddd34f1fdd78d126ca076e790cf53b7d72b94a7d24892142013710a53a7b314a270a1853ffcb8e51731868c5c7c47461322e38fd24315
|
data/CHANGELOG.md
CHANGED
@@ -1,4 +1,15 @@
|
|
1
|
-
|
1
|
+
|
2
|
+
# 1.6.0 (December 22nd, 2020)
|
3
|
+
|
4
|
+
Additions:
|
5
|
+
|
6
|
+
* b/io/write now provides an optional `supress_side_effect` option.
|
7
|
+
# 1.5.0 (December 21st, 2020)
|
8
|
+
|
9
|
+
Added Jobs:
|
10
|
+
|
11
|
+
* b/collection/zip
|
12
|
+
# 1.4.0 (December 17th, 2020)
|
2
13
|
|
3
14
|
Additions:
|
4
15
|
|
data/README.md
CHANGED
@@ -227,6 +227,7 @@ This library only ships with very basic, rudimentary jobs that are meant to just
|
|
227
227
|
* **b/collection/unpivot** [pivot_set, register]: Take an array of objects and unpivot specific sets of keys into rows. Under the hood it uses [HashMath's Unpivot class](https://github.com/bluemarblepayroll/hash_math#unpivot-hash-key-coalescence-and-row-extrapolation).
|
228
228
|
* **b/collection/validate** [invalid_register, join_char, message_key, register, separator, validations]: Take an array of objects, run it through each declared validator, and split the objects into two registers. The valid objects will be split into the current register while the invalid ones will go into the invalid_register as declared. Optional arguments, join_char and message_key, help determine the compiled error messages. The separator option can be utilized to use dot-notation for validating keys. See each validation's options by viewing their classes within the `lib/modeling/validations` directory.
|
229
229
|
* **b/collection/values** [include_keys, register]: Take an array of objects and call `#values` on each object. If include_keys is true (it is false by default), then call `#keys` on the first object and inject that as a "header" object.
|
230
|
+
* **b/collection/zip** [base_register, register, with_register]: Combines `base_register` and `with_register`s' data to form one single array in `register`. It will combine each element, positionally in each array to form the final array. For example: ['hello', 'bugs'] + ['world', 'bunny'] => [['hello', 'world'], ['bugs', 'bunny']]
|
230
231
|
|
231
232
|
#### Compression
|
232
233
|
|
@@ -245,7 +246,7 @@ By default all jobs will use the `Burner::Disks::Local` disk for its persistence
|
|
245
246
|
* **b/io/exist** [disk, path, short_circuit]: Check to see if a file exists. The path parameter can be interpolated using `Payload#params`. If short_circuit was set to true (defaults to false) and the file does not exist then the pipeline will be short-circuited.
|
246
247
|
* **b/io/read** [binary, disk, path, register]: Read in a local file. The path parameter can be interpolated using `Payload#params`. If the contents are binary, pass in `binary: true` to open it up in binary+read mode.
|
247
248
|
* **b/io/row_reader** [data_key, disk, ignore_blank_path, ignore_file_not_found, path_key, register, separator]: Iterates over an array of objects, extracts a filepath from a key in each object, and attempts to load the file's content for each record. The file's content will be stored at the specified data_key. By default missing paths or files will be treated as hard errors. If you wish to ignore these then pass in true for ignore_blank_path and/or ignore_file_not_found.
|
248
|
-
* **b/io/write** [binary, disk, path, register]: Write to a local file. The path parameter can be interpolated using `Payload#params`. If the contents are binary, pass in `binary: true` to open it up in binary+write mode.
|
249
|
+
* **b/io/write** [binary, disk, path, register, supress_side_effect]: Write to a local file. The path parameter can be interpolated using `Payload#params`. If the contents are binary, pass in `binary: true` to open it up in binary+write mode. By default, written files are also logged as WrittenFile instances to the Payload#side_effects array. You can pass in supress_side_effect: true to disable this behavior.
|
249
250
|
|
250
251
|
#### Serialization
|
251
252
|
|
data/lib/burner/jobs.rb
CHANGED
@@ -34,6 +34,7 @@ module Burner
|
|
34
34
|
register 'b/collection/unpivot', Library::Collection::Unpivot
|
35
35
|
register 'b/collection/values', Library::Collection::Values
|
36
36
|
register 'b/collection/validate', Library::Collection::Validate
|
37
|
+
register 'b/collection/zip', Library::Collection::Zip
|
37
38
|
|
38
39
|
register 'b/compress/row_reader', Library::Compress::RowReader
|
39
40
|
|
data/lib/burner/library.rb
CHANGED
@@ -25,6 +25,7 @@ require_relative 'library/collection/transform'
|
|
25
25
|
require_relative 'library/collection/unpivot'
|
26
26
|
require_relative 'library/collection/validate'
|
27
27
|
require_relative 'library/collection/values'
|
28
|
+
require_relative 'library/collection/zip'
|
28
29
|
|
29
30
|
require_relative 'library/compress/row_reader'
|
30
31
|
|
@@ -0,0 +1,53 @@
|
|
1
|
+
# frozen_string_literal: true
|
2
|
+
|
3
|
+
#
|
4
|
+
# Copyright (c) 2020-present, Blue Marble Payroll, LLC
|
5
|
+
#
|
6
|
+
# This source code is licensed under the MIT license found in the
|
7
|
+
# LICENSE file in the root directory of this source tree.
|
8
|
+
#
|
9
|
+
|
10
|
+
module Burner
|
11
|
+
module Library
|
12
|
+
module Collection
|
13
|
+
# This job can take two arrays and coalesces them by index. For example:
|
14
|
+
#
|
15
|
+
# input:
|
16
|
+
# base_register: [ 'hello', 'bugs' ]
|
17
|
+
# with_register: [ 'world', 'bunny' ]
|
18
|
+
# output:
|
19
|
+
# register: [ ['hello', 'world'], ['bugs', 'bunny'] ]
|
20
|
+
#
|
21
|
+
# Expected Payload[base_register] input: array of objects.
|
22
|
+
# Expected Payload[with_register] input: array of objects.
|
23
|
+
# Payload[register] output: An array of two-dimensional arrays.
|
24
|
+
class Zip < JobWithRegister
|
25
|
+
attr_reader :base_register, :with_register
|
26
|
+
|
27
|
+
def initialize(
|
28
|
+
name:,
|
29
|
+
with_register:,
|
30
|
+
base_register: DEFAULT_REGISTER,
|
31
|
+
register: DEFAULT_REGISTER
|
32
|
+
)
|
33
|
+
super(name: name, register: register)
|
34
|
+
|
35
|
+
@base_register = base_register.to_s
|
36
|
+
@with_register = with_register.to_s
|
37
|
+
|
38
|
+
freeze
|
39
|
+
end
|
40
|
+
|
41
|
+
def perform(output, payload)
|
42
|
+
base_data = array(payload[base_register])
|
43
|
+
with_data = array(payload[with_register])
|
44
|
+
|
45
|
+
output.detail("Combining register: #{base_register} (#{base_data.length} record(s))")
|
46
|
+
output.detail("With register: #{with_register} (#{with_data.length} record(s))")
|
47
|
+
|
48
|
+
payload[register] = base_data.zip(with_data)
|
49
|
+
end
|
50
|
+
end
|
51
|
+
end
|
52
|
+
end
|
53
|
+
end
|
@@ -12,11 +12,34 @@ require_relative 'open_file_base'
|
|
12
12
|
module Burner
|
13
13
|
module Library
|
14
14
|
module IO
|
15
|
-
# Write value to disk.
|
15
|
+
# Write value to disk. By default, written files are also logged as WrittenFile
|
16
|
+
# instances to the Payload#side_effects array. You can pass in
|
17
|
+
# supress_side_effect: true to disable this behavior.
|
16
18
|
#
|
17
19
|
# Expected Payload[register] input: anything.
|
18
20
|
# Payload[register] output: whatever was passed in.
|
19
21
|
class Write < OpenFileBase
|
22
|
+
attr_reader :supress_side_effect
|
23
|
+
|
24
|
+
def initialize(
|
25
|
+
name:,
|
26
|
+
path:,
|
27
|
+
binary: false,
|
28
|
+
disk: {},
|
29
|
+
register: DEFAULT_REGISTER,
|
30
|
+
supress_side_effect: false
|
31
|
+
)
|
32
|
+
@supress_side_effect = supress_side_effect || false
|
33
|
+
|
34
|
+
super(
|
35
|
+
binary: binary,
|
36
|
+
disk: disk,
|
37
|
+
name: name,
|
38
|
+
path: path,
|
39
|
+
register: register
|
40
|
+
)
|
41
|
+
end
|
42
|
+
|
20
43
|
def perform(output, payload)
|
21
44
|
logical_filename = job_string_template(path, output, payload)
|
22
45
|
physical_filename = nil
|
@@ -29,6 +52,10 @@ module Burner
|
|
29
52
|
|
30
53
|
output.detail("Wrote to: #{physical_filename}")
|
31
54
|
|
55
|
+
return if supress_side_effect
|
56
|
+
|
57
|
+
output.detail("Saving to side effects: #{logical_filename}")
|
58
|
+
|
32
59
|
side_effect = SideEffects::WrittenFile.new(
|
33
60
|
logical_filename: logical_filename,
|
34
61
|
physical_filename: physical_filename,
|
data/lib/burner/version.rb
CHANGED
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: burner
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 1.
|
4
|
+
version: 1.6.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Matthew Ruggio
|
8
8
|
autorequire:
|
9
9
|
bindir: exe
|
10
10
|
cert_chain: []
|
11
|
-
date: 2020-12-
|
11
|
+
date: 2020-12-22 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: acts_as_hashable
|
@@ -252,6 +252,7 @@ files:
|
|
252
252
|
- lib/burner/library/collection/unpivot.rb
|
253
253
|
- lib/burner/library/collection/validate.rb
|
254
254
|
- lib/burner/library/collection/values.rb
|
255
|
+
- lib/burner/library/collection/zip.rb
|
255
256
|
- lib/burner/library/compress/row_reader.rb
|
256
257
|
- lib/burner/library/deserialize/csv.rb
|
257
258
|
- lib/burner/library/deserialize/json.rb
|
@@ -309,9 +310,9 @@ required_ruby_version: !ruby/object:Gem::Requirement
|
|
309
310
|
version: '2.5'
|
310
311
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
311
312
|
requirements:
|
312
|
-
- - "
|
313
|
+
- - ">="
|
313
314
|
- !ruby/object:Gem::Version
|
314
|
-
version:
|
315
|
+
version: '0'
|
315
316
|
requirements: []
|
316
317
|
rubygems_version: 3.0.3
|
317
318
|
signing_key:
|