conformist 0.2.0 → 0.2.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.travis.yml +3 -2
- data/CHANGELOG.md +6 -1
- data/README.md +26 -6
- data/conformist.gemspec +3 -2
- data/lib/conformist.rb +0 -5
- data/lib/conformist/column.rb +16 -3
- data/lib/conformist/schema.rb +8 -3
- data/lib/conformist/version.rb +1 -1
- data/test/helper.rb +1 -0
- data/test/unit/conformist/builder_test.rb +3 -3
- data/test/unit/conformist/column_test.rb +2 -2
- data/test/unit/conformist/hash_struct_test.rb +3 -3
- data/test/unit/conformist/schema_test.rb +1 -1
- data/test/unit/conformist_test.rb +1 -1
- data/test/unit/integration_test.rb +13 -2
- metadata +30 -27
checksums.yaml
ADDED
@@ -0,0 +1,7 @@
|
|
1
|
+
---
|
2
|
+
SHA1:
|
3
|
+
metadata.gz: e2571df3b1d38f03302084d750cf10d2a8d1db75
|
4
|
+
data.tar.gz: d0637c6991ff0a7caca1fbf5ad19cfa03ffcc0bd
|
5
|
+
SHA512:
|
6
|
+
metadata.gz: 429e5ccf39a435cb89efe02a1be91cc20e2b46ed23c76f512ff6263f5d7a9b9a2b39319d9a8ae8d445a0177b1a5d41df0c87c11c4b5772826461ec1d39e5ab00
|
7
|
+
data.tar.gz: fdb0a7140d3618ca81f864cbba957c49d29c716498f555ac273da568f5ac6d6f07a5e561240c86bf7dba893c1e5ce74bc242c195c4747dc6b4f60528e278298d
|
data/.travis.yml
CHANGED
data/CHANGELOG.md
CHANGED
@@ -1,5 +1,10 @@
|
|
1
1
|
# CHANGELOG
|
2
2
|
|
3
|
+
## 0.2.1 / 2015-01-31
|
4
|
+
|
5
|
+
* Add support for named columns. (@rywall)
|
6
|
+
* Dropped support for Ruby 1.8.7, 1.9.2.
|
7
|
+
|
3
8
|
## 0.2.0 / 2012-08-18
|
4
9
|
|
5
10
|
* `Conformist::Schema#confrom` takes option to skip first row.
|
@@ -7,7 +12,7 @@
|
|
7
12
|
|
8
13
|
## 0.1.3 / 2012-02-09
|
9
14
|
|
10
|
-
* Column indexes are implicitly incremented when the index argument is omitted. Implicit indexing is all or nothing.
|
15
|
+
* Column indexes are implicitly incremented when the index argument is omitted. Implicit indexing is all or nothing. (@coop)
|
11
16
|
|
12
17
|
## 0.1.2 / 2012-01-19
|
13
18
|
|
data/README.md
CHANGED
@@ -1,6 +1,7 @@
|
|
1
1
|
# Conformist
|
2
2
|
|
3
3
|
[](http://travis-ci.org/tatey/conformist)
|
4
|
+
[](https://codeclimate.com/github/tatey/conformist)
|
4
5
|
|
5
6
|
Bend CSVs to your will with declarative schemas. Map one or many columns, preprocess cells and lazily enumerate. Declarative schemas are easier to understand, quicker to setup and independent of I/O. Use [CSV](http://www.ruby-doc.org/stdlib-1.9.3/libdoc/csv/rdoc/CSV.html) (Formally [FasterCSV](https://rubygems.org/gems/fastercsv)), [Spreadsheet](https://rubygems.org/gems/spreadsheet) or any array of array-like data structure.
|
6
7
|
|
@@ -42,7 +43,7 @@ Only insert the transmitters with the name "Mount Cooth-tha" using ActiveRecord
|
|
42
43
|
transmitters = schema.conform(csv).select do |transmitter|
|
43
44
|
transmitter.name == 'Mount Coot-tha'
|
44
45
|
end
|
45
|
-
|
46
|
+
transmitters.each do |transmitter|
|
46
47
|
Transmitter.create! transmitter.attributes
|
47
48
|
end
|
48
49
|
```
|
@@ -186,6 +187,20 @@ the first row is the header row and irrelevant for enumeration.
|
|
186
187
|
schema.conform CSV.open('~/file_with_headers.csv'), :skip_first => true
|
187
188
|
```
|
188
189
|
|
190
|
+
#### Named Columns
|
191
|
+
|
192
|
+
Strings can be used as column indexes instead of integers. These strings will be matched
|
193
|
+
against the first row to determine the appropriate numerical index.
|
194
|
+
|
195
|
+
``` ruby
|
196
|
+
citizen = Conformist.new do
|
197
|
+
column :email, 'EM'
|
198
|
+
column :name, 'FN', 'LN'
|
199
|
+
end
|
200
|
+
|
201
|
+
citizen.conform [['FN', 'LN', 'EM'], ['Tate', 'Johnson', 'tate@tatey.com']], :skip_first => true
|
202
|
+
```
|
203
|
+
|
189
204
|
#### Enumerator
|
190
205
|
|
191
206
|
`#conform` is lazy, returning an [Enumerator](http://www.ruby-doc.org/core-1.9.3/Enumerator.html). Input is not parsed until you call `#each`, `#map` or any method defined in [Enumerable](http://www.ruby-doc.org/core-1.9.3/Enumerable.html). That means schemas can be assigned now and evaluated later. `#each` has the lowest memory footprint because it does not build a collection.
|
@@ -331,9 +346,8 @@ See CHANGELOG.md for a full list of changes.
|
|
331
346
|
|
332
347
|
## Compatibility
|
333
348
|
|
334
|
-
* MRI 1.9.
|
335
|
-
*
|
336
|
-
* JRuby 1.6.5
|
349
|
+
* MRI 2.2.0, 2.1.0, 2.0.0, 1.9.3
|
350
|
+
* JRuby
|
337
351
|
|
338
352
|
## Dependencies
|
339
353
|
|
@@ -344,8 +358,14 @@ No explicit dependencies, although `CSV` and `Spreadsheet` are commonly used.
|
|
344
358
|
1. Fork
|
345
359
|
2. Install dependancies by running `$ bundle install`
|
346
360
|
3. Write tests and code
|
347
|
-
4. Make sure the tests pass by running `$ bundle exec rake`
|
348
|
-
5. Push and
|
361
|
+
4. Make sure the tests pass locally by running `$ bundle exec rake`
|
362
|
+
5. Push to GitHub and make sure continuous integration tests pass at
|
363
|
+
https://travis-ci.org/tatey/conformist/pull_requests
|
364
|
+
5. Send a pull request on GitHub
|
365
|
+
|
366
|
+
Please do not increment the version number in `lib/conformist/version.rb`.
|
367
|
+
The version number will be incremented by the maintainer after the patch
|
368
|
+
is accepted.
|
349
369
|
|
350
370
|
## Motivation
|
351
371
|
|
data/conformist.gemspec
CHANGED
@@ -14,9 +14,10 @@ Gem::Specification.new do |s|
|
|
14
14
|
|
15
15
|
s.rubyforge_project = "conformist"
|
16
16
|
|
17
|
-
s.required_ruby_version = '>= 1.
|
17
|
+
s.required_ruby_version = '>= 1.9.3'
|
18
18
|
|
19
|
-
s.add_development_dependency 'minitest'
|
19
|
+
s.add_development_dependency 'minitest', '>= 5.4.0'
|
20
|
+
s.add_development_dependency 'pry'
|
20
21
|
s.add_development_dependency 'rake'
|
21
22
|
s.add_development_dependency 'spreadsheet'
|
22
23
|
|
data/lib/conformist.rb
CHANGED
data/lib/conformist/column.rb
CHANGED
@@ -1,13 +1,26 @@
|
|
1
1
|
module Conformist
|
2
2
|
class Column
|
3
|
-
attr_accessor :name, :indexes, :preprocessor
|
3
|
+
attr_accessor :name, :sources, :indexes, :preprocessor
|
4
4
|
|
5
|
-
def initialize name, *
|
5
|
+
def initialize name, *sources, &preprocessor
|
6
6
|
self.name = name
|
7
|
-
self.
|
7
|
+
self.sources = sources
|
8
|
+
self.indexes = sources
|
8
9
|
self.preprocessor = preprocessor
|
9
10
|
end
|
10
11
|
|
12
|
+
def calculate_indices!(headers)
|
13
|
+
headers = Array(headers).collect {|header| header.to_s.downcase.squeeze.strip }
|
14
|
+
|
15
|
+
self.indexes = sources.collect do |source|
|
16
|
+
if source.is_a?(String)
|
17
|
+
headers.index(source.downcase)
|
18
|
+
else
|
19
|
+
source
|
20
|
+
end
|
21
|
+
end.compact
|
22
|
+
end
|
23
|
+
|
11
24
|
def values_in enumerable
|
12
25
|
values = Array(enumerable).values_at(*indexes).map do |value|
|
13
26
|
if value.respond_to? :strip
|
data/lib/conformist/schema.rb
CHANGED
@@ -1,4 +1,4 @@
|
|
1
|
-
module Conformist
|
1
|
+
module Conformist
|
2
2
|
module Schema
|
3
3
|
def self.included base
|
4
4
|
base.send :include, InstanceExtensions
|
@@ -24,7 +24,7 @@ module Conformist
|
|
24
24
|
self.columns = super_schema.columns.dup
|
25
25
|
end
|
26
26
|
if block
|
27
|
-
instance_eval &block
|
27
|
+
instance_eval &block
|
28
28
|
end
|
29
29
|
end
|
30
30
|
end
|
@@ -53,9 +53,14 @@ module Conformist
|
|
53
53
|
|
54
54
|
def conform enumerables, options = {}
|
55
55
|
options = options.dup
|
56
|
+
|
56
57
|
Enumerator.new do |yielder|
|
57
58
|
enumerables.each do |enumerable|
|
58
|
-
|
59
|
+
if options.delete :skip_first
|
60
|
+
columns.each {|column| column.calculate_indices!(enumerable) }
|
61
|
+
next
|
62
|
+
end
|
63
|
+
|
59
64
|
yielder.yield builder.call(self, enumerable)
|
60
65
|
end
|
61
66
|
end
|
data/lib/conformist/version.rb
CHANGED
data/test/helper.rb
CHANGED
@@ -1,11 +1,11 @@
|
|
1
1
|
require 'helper'
|
2
2
|
|
3
|
-
class Conformist::BuilderTest <
|
3
|
+
class Conformist::BuilderTest < Minitest::Test
|
4
4
|
def test_call
|
5
|
-
column =
|
5
|
+
column = Minitest::Mock.new
|
6
6
|
column.expect :name, :a
|
7
7
|
column.expect :values_in, [1], [Array]
|
8
|
-
definition =
|
8
|
+
definition = Minitest::Mock.new
|
9
9
|
definition.expect :columns, [column]
|
10
10
|
assert_equal HashStruct.new({:a => [1]}), Builder.call(definition, [])
|
11
11
|
end
|
@@ -1,6 +1,6 @@
|
|
1
1
|
require 'helper'
|
2
2
|
|
3
|
-
class Conformist::ColumnTest <
|
3
|
+
class Conformist::ColumnTest < Minitest::Test
|
4
4
|
def stub_row
|
5
5
|
('a'..'d').to_a
|
6
6
|
end
|
@@ -41,7 +41,7 @@ class Conformist::ColumnTest < MiniTest::Unit::TestCase
|
|
41
41
|
end
|
42
42
|
|
43
43
|
def test_array
|
44
|
-
mock =
|
44
|
+
mock = Minitest::Mock.new
|
45
45
|
mock.expect :to_a, ['a']
|
46
46
|
column = Column.new :foo, 0
|
47
47
|
assert_equal 'a', column.values_in(['a'])
|
@@ -1,6 +1,6 @@
|
|
1
1
|
require 'helper'
|
2
2
|
|
3
|
-
class Conformist::HashStructTest <
|
3
|
+
class Conformist::HashStructTest < Minitest::Test
|
4
4
|
def test_initialize
|
5
5
|
assert_equal({:a => 1}, HashStruct.new({:a => 1}).attributes)
|
6
6
|
assert_empty HashStruct.new.attributes
|
@@ -17,9 +17,9 @@ class Conformist::HashStructTest < MiniTest::Unit::TestCase
|
|
17
17
|
def test_equality
|
18
18
|
hash1 = HashStruct.new :a => 1
|
19
19
|
hash2 = HashStruct.new :a => 1
|
20
|
-
hash3 =
|
20
|
+
hash3 = Minitest::Mock.new
|
21
21
|
hash3.expect :attributes, {:a => 1}
|
22
|
-
hash3.expect :class,
|
22
|
+
hash3.expect :class, Minitest::Mock
|
23
23
|
assert_equal hash1, hash2
|
24
24
|
refute_equal hash1, hash3
|
25
25
|
end
|
@@ -1,6 +1,6 @@
|
|
1
1
|
require 'helper'
|
2
2
|
|
3
|
-
class Conformist::SchemaTest <
|
3
|
+
class Conformist::SchemaTest < Minitest::Test
|
4
4
|
def test_initialize_with_instance
|
5
5
|
parent = Class.new { include Schema }.new.tap { |d| d.columns = [0] }
|
6
6
|
child1 = Class.new { include Schema }.new(parent).tap { |d| d.columns << 1 }
|
@@ -5,7 +5,7 @@ require 'schemas/citizens'
|
|
5
5
|
require 'schemas/fcc'
|
6
6
|
require 'spreadsheet'
|
7
7
|
|
8
|
-
class IntegrationTest <
|
8
|
+
class IntegrationTest < Minitest::Test
|
9
9
|
def fixture filename
|
10
10
|
File.expand_path "../../fixtures/#{filename}", __FILE__
|
11
11
|
end
|
@@ -81,7 +81,7 @@ class IntegrationTest < MiniTest::Unit::TestCase
|
|
81
81
|
column :capital, 2
|
82
82
|
end
|
83
83
|
child = Conformist.new parent do
|
84
|
-
column :country do
|
84
|
+
column :country do
|
85
85
|
'Australia'
|
86
86
|
end
|
87
87
|
end
|
@@ -89,4 +89,15 @@ class IntegrationTest < MiniTest::Unit::TestCase
|
|
89
89
|
last = enumerable.to_a.last
|
90
90
|
assert_equal HashStruct.new(:state => 'QLD, Queensland', :capital => 'Brisbane', :country => 'Australia'), last
|
91
91
|
end
|
92
|
+
|
93
|
+
def test_named_columns
|
94
|
+
schema = Conformist.new do
|
95
|
+
column :name, 'Name'
|
96
|
+
column :age, 'Age'
|
97
|
+
column :gender, 'Gender'
|
98
|
+
end
|
99
|
+
enumerable = schema.conform open_csv('citizens.csv'), :skip_first => true
|
100
|
+
first = enumerable.to_a.first
|
101
|
+
assert_equal HashStruct.new(:name => 'Aaron', :age => '21', :gender => 'Male'), first
|
102
|
+
end
|
92
103
|
end
|
metadata
CHANGED
@@ -1,62 +1,69 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: conformist
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.2.
|
5
|
-
prerelease:
|
4
|
+
version: 0.2.1
|
6
5
|
platform: ruby
|
7
6
|
authors:
|
8
7
|
- Tate Johnson
|
9
8
|
autorequire:
|
10
9
|
bindir: bin
|
11
10
|
cert_chain: []
|
12
|
-
date:
|
11
|
+
date: 2015-01-31 00:00:00.000000000 Z
|
13
12
|
dependencies:
|
14
13
|
- !ruby/object:Gem::Dependency
|
15
14
|
name: minitest
|
16
15
|
requirement: !ruby/object:Gem::Requirement
|
17
|
-
none: false
|
18
16
|
requirements:
|
19
|
-
- -
|
17
|
+
- - ">="
|
18
|
+
- !ruby/object:Gem::Version
|
19
|
+
version: 5.4.0
|
20
|
+
type: :development
|
21
|
+
prerelease: false
|
22
|
+
version_requirements: !ruby/object:Gem::Requirement
|
23
|
+
requirements:
|
24
|
+
- - ">="
|
25
|
+
- !ruby/object:Gem::Version
|
26
|
+
version: 5.4.0
|
27
|
+
- !ruby/object:Gem::Dependency
|
28
|
+
name: pry
|
29
|
+
requirement: !ruby/object:Gem::Requirement
|
30
|
+
requirements:
|
31
|
+
- - ">="
|
20
32
|
- !ruby/object:Gem::Version
|
21
33
|
version: '0'
|
22
34
|
type: :development
|
23
35
|
prerelease: false
|
24
36
|
version_requirements: !ruby/object:Gem::Requirement
|
25
|
-
none: false
|
26
37
|
requirements:
|
27
|
-
- -
|
38
|
+
- - ">="
|
28
39
|
- !ruby/object:Gem::Version
|
29
40
|
version: '0'
|
30
41
|
- !ruby/object:Gem::Dependency
|
31
42
|
name: rake
|
32
43
|
requirement: !ruby/object:Gem::Requirement
|
33
|
-
none: false
|
34
44
|
requirements:
|
35
|
-
- -
|
45
|
+
- - ">="
|
36
46
|
- !ruby/object:Gem::Version
|
37
47
|
version: '0'
|
38
48
|
type: :development
|
39
49
|
prerelease: false
|
40
50
|
version_requirements: !ruby/object:Gem::Requirement
|
41
|
-
none: false
|
42
51
|
requirements:
|
43
|
-
- -
|
52
|
+
- - ">="
|
44
53
|
- !ruby/object:Gem::Version
|
45
54
|
version: '0'
|
46
55
|
- !ruby/object:Gem::Dependency
|
47
56
|
name: spreadsheet
|
48
57
|
requirement: !ruby/object:Gem::Requirement
|
49
|
-
none: false
|
50
58
|
requirements:
|
51
|
-
- -
|
59
|
+
- - ">="
|
52
60
|
- !ruby/object:Gem::Version
|
53
61
|
version: '0'
|
54
62
|
type: :development
|
55
63
|
prerelease: false
|
56
64
|
version_requirements: !ruby/object:Gem::Requirement
|
57
|
-
none: false
|
58
65
|
requirements:
|
59
|
-
- -
|
66
|
+
- - ">="
|
60
67
|
- !ruby/object:Gem::Version
|
61
68
|
version: '0'
|
62
69
|
description: Bend CSVs to your will with declarative schemas.
|
@@ -66,8 +73,8 @@ executables: []
|
|
66
73
|
extensions: []
|
67
74
|
extra_rdoc_files: []
|
68
75
|
files:
|
69
|
-
- .gitignore
|
70
|
-
- .travis.yml
|
76
|
+
- ".gitignore"
|
77
|
+
- ".travis.yml"
|
71
78
|
- CHANGELOG.md
|
72
79
|
- Gemfile
|
73
80
|
- LICENSE
|
@@ -96,30 +103,26 @@ files:
|
|
96
103
|
- test/unit/integration_test.rb
|
97
104
|
homepage: https://github.com/tatey/conformist
|
98
105
|
licenses: []
|
106
|
+
metadata: {}
|
99
107
|
post_install_message:
|
100
108
|
rdoc_options: []
|
101
109
|
require_paths:
|
102
110
|
- lib
|
103
111
|
required_ruby_version: !ruby/object:Gem::Requirement
|
104
|
-
none: false
|
105
112
|
requirements:
|
106
|
-
- -
|
113
|
+
- - ">="
|
107
114
|
- !ruby/object:Gem::Version
|
108
|
-
version: 1.
|
115
|
+
version: 1.9.3
|
109
116
|
required_rubygems_version: !ruby/object:Gem::Requirement
|
110
|
-
none: false
|
111
117
|
requirements:
|
112
|
-
- -
|
118
|
+
- - ">="
|
113
119
|
- !ruby/object:Gem::Version
|
114
120
|
version: '0'
|
115
|
-
segments:
|
116
|
-
- 0
|
117
|
-
hash: -2869076359774702419
|
118
121
|
requirements: []
|
119
122
|
rubyforge_project: conformist
|
120
|
-
rubygems_version:
|
123
|
+
rubygems_version: 2.2.2
|
121
124
|
signing_key:
|
122
|
-
specification_version:
|
125
|
+
specification_version: 4
|
123
126
|
summary: Bend CSVs to your will with declarative schemas.
|
124
127
|
test_files:
|
125
128
|
- test/fixtures/acma.csv
|