kiba 0.6.1 → 1.0.0
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/.travis.yml +5 -2
- data/Changes.md +5 -2
- data/README.md +12 -3
- data/appveyor.yml +18 -0
- data/lib/kiba/runner.rb +1 -1
- data/lib/kiba/version.rb +1 -1
- data/test/test_runner.rb +12 -0
- metadata +4 -3
checksums.yaml
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
---
|
2
2
|
SHA1:
|
3
|
-
metadata.gz:
|
4
|
-
data.tar.gz:
|
3
|
+
metadata.gz: 622b10fe7f524f66152c10ab8618d6a661831776
|
4
|
+
data.tar.gz: 7410d79e2dec5bb48e9f1079e94c87260226d66c
|
5
5
|
SHA512:
|
6
|
-
metadata.gz:
|
7
|
-
data.tar.gz:
|
6
|
+
metadata.gz: 5fe1c537d31ccc49446316f40c76630f724774431a3914019b870ce4cbdc3d34ed7eda4b66b622855d7d6972f435a0b8f831567f9b53b159e2836257b766671c
|
7
|
+
data.tar.gz: 60c2e8929bdd350308c5f377b0a6c989c3473f147903a90dab7ab9b4b58f6ec06cdc038094c5bca2ebd593defe2f0e7994752bdef325bd4ccbd7e04ef27f86a9
|
data/.travis.yml
CHANGED
data/Changes.md
CHANGED
data/README.md
CHANGED
@@ -1,13 +1,22 @@
|
|
1
|
+
**Foreword - if you need help**: please [ask your question with tag kiba-etl on StackOverflow](http://stackoverflow.com/questions/ask?tags=kiba-etl) so that other can benefit from your contribution! I monitor this specific tag and will reply to you.
|
2
|
+
|
1
3
|
Writing reliable, concise, well-tested & maintainable data-processing code is tricky.
|
2
4
|
|
3
5
|
Kiba lets you define and run such high-quality ETL ([Extract-Transform-Load](http://en.wikipedia.org/wiki/Extract,_transform,_load)) jobs, using Ruby (see [supported versions](#supported-ruby-versions)).
|
4
6
|
|
5
|
-
Learn more on the [Kiba blog](http://thibautbarrere.com):
|
7
|
+
Learn more on the [Kiba blog](http://thibautbarrere.com) and on [StackOverflow](http://stackoverflow.com/questions/tagged/kiba-etl):
|
6
8
|
|
9
|
+
* [Live Coding Session - Processing data with Kiba ETL](http://thibautbarrere.com/2015/11/09/video-processing-data-with-kiba-etl/)
|
7
10
|
* [Rubyists - are you doing ETL unknowningly?](http://thibautbarrere.com/2015/03/25/rubyists-are-you-doing-etl-unknowingly/)
|
8
11
|
* [How to write solid data processing code](http://thibautbarrere.com/2015/04/05/how-to-write-solid-data-processing-code/)
|
9
12
|
* [How to reformat CSV files with Kiba](http://thibautbarrere.com/2015/06/04/how-to-reformat-csv-files-with-kiba/) (in-depth, hands-on tutorial)
|
10
13
|
* [How to explode multivalued attributes with Kiba ETL?](http://thibautbarrere.com/2015/06/25/how-to-explode-multivalued-attributes-with-kiba/)
|
14
|
+
* [Common techniques to compute aggregates with Kiba](https://stackoverflow.com/questions/31145715/how-to-do-a-aggregation-transformation-in-a-kiba-etl-script-kiba-gem)
|
15
|
+
* [How to run Kiba in a Rails environment?](http://thibautbarrere.com/2015/09/26/how-to-run-kiba-in-a-rails-environment/)
|
16
|
+
|
17
|
+
**Consulting services**: if your organization needs to leverage data processing to solve a given business problem, I'm available to help you out via consulting sessions. [More information](http://thibautbarrere.com/hire-me/).
|
18
|
+
|
19
|
+
**Kiba Pro**: I'm working on a Pro version ([read more here](https://github.com/thbar/kiba/issues/20)) which will provide more advanced features and built-in goodies in exchange for a yearly subscription. This will also make sure I can support Kiba for the many years to come. [Chime in](https://github.com/thbar/kiba/issues/20) if your company is interested!
|
11
20
|
|
12
21
|
[![Gem Version](https://badge.fury.io/rb/kiba.svg)](http://badge.fury.io/rb/kiba)
|
13
22
|
[![Build Status](https://travis-ci.org/thbar/kiba.svg?branch=master)](https://travis-ci.org/thbar/kiba) [![Code Climate](https://codeclimate.com/github/thbar/kiba/badges/gpa.svg)](https://codeclimate.com/github/thbar/kiba) [![Dependency Status](https://gemnasium.com/thbar/kiba.svg)](https://gemnasium.com/thbar/kiba)
|
@@ -175,7 +184,7 @@ Like the block form, it can return `nil` to dismiss the row. The class form allo
|
|
175
184
|
Like sources, destinations are classes that you are providing. Destinations must implement:
|
176
185
|
- a constructor (to which Kiba will pass the provided arguments in the DSL)
|
177
186
|
- a `write(row)` method that will be called for each non-dismissed row
|
178
|
-
-
|
187
|
+
- an optional `close` method that will be called, if present, at the end of the processing (useful to tear down resources such as connections)
|
179
188
|
|
180
189
|
Here is an example destination:
|
181
190
|
|
@@ -256,7 +265,7 @@ Make sure to subscribe to my [Ruby ETL blog](http://thibautbarrere.com) where I'
|
|
256
265
|
|
257
266
|
## Supported Ruby versions
|
258
267
|
|
259
|
-
Kiba currently supports Ruby 2.0
|
268
|
+
Kiba currently supports Ruby 2.0+ and JRuby (with its default 1.9 syntax). See [test matrix](https://travis-ci.org/thbar/kiba).
|
260
269
|
|
261
270
|
## History & Credits
|
262
271
|
|
data/appveyor.yml
ADDED
@@ -0,0 +1,18 @@
|
|
1
|
+
version: '{build}'
|
2
|
+
|
3
|
+
skip_tags: true
|
4
|
+
|
5
|
+
environment:
|
6
|
+
matrix:
|
7
|
+
- ruby_version: "21"
|
8
|
+
- ruby_version: "21-x64"
|
9
|
+
|
10
|
+
install:
|
11
|
+
- SET PATH=C:\Ruby%ruby_version%\bin;%PATH%
|
12
|
+
- gem install bundler --no-document -v 1.10.5
|
13
|
+
- bundle install --retry=3
|
14
|
+
|
15
|
+
test_script:
|
16
|
+
- bundle exec rake
|
17
|
+
|
18
|
+
build: off
|
data/lib/kiba/runner.rb
CHANGED
data/lib/kiba/version.rb
CHANGED
data/test/test_runner.rb
CHANGED
@@ -72,4 +72,16 @@ class TestRunner < Kiba::Test
|
|
72
72
|
|
73
73
|
assert_equal [:pre_processor_executed, :source_instantiated], calls
|
74
74
|
end
|
75
|
+
|
76
|
+
def test_no_error_raised_if_destination_close_not_implemented
|
77
|
+
# NOTE: this fake destination does not implement `close`
|
78
|
+
destination_instance = MiniTest::Mock.new
|
79
|
+
|
80
|
+
mock_destination_class = MiniTest::Mock.new
|
81
|
+
mock_destination_class.expect(:new, destination_instance)
|
82
|
+
|
83
|
+
control = Kiba::Control.new
|
84
|
+
control.destinations << { klass: mock_destination_class }
|
85
|
+
Kiba.run(control)
|
86
|
+
end
|
75
87
|
end
|
metadata
CHANGED
@@ -1,14 +1,14 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: kiba
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 1.0.0
|
5
5
|
platform: ruby
|
6
6
|
authors:
|
7
7
|
- Thibaut Barrère
|
8
8
|
autorequire:
|
9
9
|
bindir: bin
|
10
10
|
cert_chain: []
|
11
|
-
date:
|
11
|
+
date: 2016-12-01 00:00:00.000000000 Z
|
12
12
|
dependencies:
|
13
13
|
- !ruby/object:Gem::Dependency
|
14
14
|
name: rake
|
@@ -66,6 +66,7 @@ files:
|
|
66
66
|
- Gemfile
|
67
67
|
- README.md
|
68
68
|
- Rakefile
|
69
|
+
- appveyor.yml
|
69
70
|
- bin/kiba
|
70
71
|
- kiba.gemspec
|
71
72
|
- lib/kiba.rb
|
@@ -108,7 +109,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
108
109
|
version: '0'
|
109
110
|
requirements: []
|
110
111
|
rubyforge_project:
|
111
|
-
rubygems_version: 2.4.
|
112
|
+
rubygems_version: 2.4.8
|
112
113
|
signing_key:
|
113
114
|
specification_version: 4
|
114
115
|
summary: Lightweight ETL for Ruby
|