rgarner-csv-mapper 0.5.1

Sign up to get free protection for your applications and to get access to all the features.
data/.gitignore ADDED
@@ -0,0 +1,4 @@
1
+ pkg/*
2
+ doc/*
3
+ .idea/*
4
+ *.gemspec
data/History.txt ADDED
@@ -0,0 +1,25 @@
1
+ == 0.5.0 2010-05-12
2
+ * Parsing performance is now approximately 8x-10x faster when not specifying a custom map_to class.
3
+ * Default class mapping is now Struct instead of OpenStruct
4
+ * Recommended usage is now "CsvMapper.import(...)"
5
+ * Fixes recurring problem with using CsvMapper in rake tasks
6
+ * Keeps everything a little cleaner
7
+ * #map transformations now prefer blocks over lambdas or symbols.
8
+ * Converted from using newgem to jeweler for managing the library itself.
9
+
10
+ == 0.0.4 2009-08-05
11
+ * Merged contributions from Jeffrey Chupp - http://semanticart.com
12
+ * Added support for "Automagical Attribute Discovery"
13
+ * Added Ruby 1.9 compatibility
14
+
15
+ == 0.0.3 2008-12-22
16
+ * Fixed specs to work with RSpec 1.1.9 and later where Modules aren't auto included
17
+
18
+ == 0.0.2 2008-12-15
19
+
20
+ * Added #stop_at_row method to RowMap
21
+
22
+ == 0.0.1 2008-12-05
23
+
24
+ * 1 major enhancement:
25
+ * Initial release
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ (The MIT License)
2
+
3
+ Copyright (c) 2009 Luke Pillow
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ 'Software'), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED 'AS IS', WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.
19
+ IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY
20
+ CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT,
21
+ TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION WITH THE
22
+ SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.rdoc ADDED
@@ -0,0 +1,74 @@
1
+ = csv-mapper
2
+
3
+ == DESCRIPTION:
4
+
5
+ CsvMapper is a small library intended to simplify the common steps involved with importing CSV files to a usable form in Ruby. CsvMapper is compatible with recent 1.8 versions of Ruby as well as Ruby 1.9+
6
+
7
+ == EXAMPLES:
8
+
9
+ The following example will import a CSV file to an Array of Struct[http://www.ruby-doc.org/core/classes/Struct.html] instances.
10
+
11
+ ==== Example CSV File Structure
12
+
13
+ First Name,Last Name,Age
14
+ John,Doe,27
15
+ Jane,Doe,26
16
+ Bat,Man,52
17
+ ...etc...
18
+
19
+ ==== Simple Usage Example
20
+ results = CsvMapper.import('/path/to/file.csv') do
21
+ start_at_row 1
22
+ [first_name, last_name, age]
23
+ end
24
+
25
+ results.first.first_name # John
26
+ results.first.last_name # Doe
27
+ results.first.age # 27
28
+
29
+ ==== Automagical Attribute Discovery Example
30
+ results = CsvMapper.import('/path/to/file.csv') do
31
+ read_attributes_from_file
32
+ end
33
+
34
+ results.first.first_name # John
35
+ results.first.last_name # Doe
36
+ results.first.age # 27
37
+
38
+ ==== Import to ActiveRecord Example
39
+ Although CsvMapper has no dependency on ActiveRecord; it's easy to import a CSV file to ActiveRecord models and save them.
40
+
41
+ # Define an ActiveRecord model
42
+ class Person < ActiveRecord::Base; end
43
+
44
+ results = CsvMapper.import('/path/to/file.csv') do
45
+ map_to Person # Map to the Person ActiveRecord class (defined above) instead of the default Struct.
46
+ after_row lambda{|row, person| person.save } # Call this lambda and save each record after it's parsed.
47
+
48
+ start_at_row 1
49
+ [first_name, last_name, age]
50
+ end
51
+
52
+ See CsvMapper for a more detailed description
53
+
54
+ == REQUIREMENTS:
55
+
56
+ FasterCSV[http://fastercsv.rubyforge.org/] on pre 1.9 versions of Ruby
57
+
58
+ == INSTALL:
59
+
60
+ * sudo gem install csv-mapper
61
+
62
+ == Note on Patches/Pull Requests
63
+
64
+ * Fork the project.
65
+ * Make your feature addition or bug fix.
66
+ * Add tests for it. This is important so I don't break it in a
67
+ future version unintentionally.
68
+ * Commit, do not mess with rakefile, version, or history.
69
+ (if you want to have your own version, that is fine but bump version in a commit by itself I can ignore when I pull)
70
+ * Send me a pull request. Bonus points for topic branches.
71
+
72
+ == Copyright
73
+
74
+ Copyright (c) 2009 Luke Pillow. See LICENSE for details.
data/Rakefile ADDED
@@ -0,0 +1,47 @@
1
+ require 'rubygems'
2
+ require 'rake'
3
+
4
+ begin
5
+ require 'jeweler'
6
+ Jeweler::Tasks.new do |gem|
7
+ gem.name = "rgarner-csv-mapper"
8
+ gem.summary = %Q{rgarner-CsvMapper is a fork of a small library intended to simplify the common steps involved with importing CSV files to a usable form in Ruby. It has support for null column names. When this is merged, this gem will be removed.}
9
+ gem.description = %Q{CSV Mapper makes it easy to import data from CSV files directly to a collection of any type of Ruby object. The simplest way to create mappings is declare the names of the attributes in the order corresponding to the CSV file column order.}
10
+ gem.email = "rgarner@zephyros-systems.co.uk"
11
+ gem.homepage = "http://github.com/rgarner/csv-mapper"
12
+ gem.authors = ["Luke Pillow", "Russell Garner"]
13
+ gem.add_development_dependency "rspec", ">= 1.2.9"
14
+ gem.add_dependency "fastercsv"
15
+ gem.extra_rdoc_files << "History.txt"
16
+ # gem is a Gem::Specification... see http://www.rubygems.org/read/chapter/20 for additional settings
17
+ end
18
+ Jeweler::GemcutterTasks.new
19
+ rescue LoadError
20
+ puts "Jeweler (or a dependency) not available. Install it with: gem install jeweler"
21
+ end
22
+
23
+ require 'spec/rake/spectask'
24
+ Spec::Rake::SpecTask.new(:spec) do |spec|
25
+ spec.libs << 'lib' << 'spec'
26
+ spec.spec_files = FileList['spec/**/*_spec.rb']
27
+ end
28
+
29
+ Spec::Rake::SpecTask.new(:rcov) do |spec|
30
+ spec.libs << 'lib' << 'spec'
31
+ spec.pattern = 'spec/**/*_spec.rb'
32
+ spec.rcov = true
33
+ end
34
+
35
+ task :spec => :check_dependencies
36
+
37
+ task :default => :spec
38
+
39
+ require 'rake/rdoctask'
40
+ Rake::RDocTask.new do |rdoc|
41
+ version = File.exist?('VERSION') ? File.read('VERSION') : ""
42
+
43
+ rdoc.rdoc_dir = 'rdoc'
44
+ rdoc.title = "csv-mapper #{version}"
45
+ rdoc.rdoc_files.include('README*')
46
+ rdoc.rdoc_files.include('lib/**/*.rb')
47
+ end
data/VERSION ADDED
@@ -0,0 +1 @@
1
+ 0.5.1
@@ -0,0 +1,55 @@
1
+ module CsvMapper
2
+ # A CsvMapper::AttributeMap contains the instructions to parse a value from a CSV row and to know the
3
+ # name of the attribute it is targeting.
4
+ class AttributeMap
5
+ attr_reader :name, :index
6
+
7
+ # Creates a new instance using the provided attribute +name+, CSV row +index+, and evaluation +map_context+
8
+ def initialize(name, index, map_context)
9
+ @name, @index, @map_context = name, index, map_context
10
+ end
11
+
12
+ # Set the index that this map is targeting.
13
+ #
14
+ # Returns this AttributeMap for chainability
15
+ def at(index)
16
+ @index = index
17
+ self
18
+ end
19
+
20
+ # Provide a lambda or the symbol name of a method on this map's evaluation context to be used when parsing
21
+ # the value from a CSV row.
22
+ # Both the lambda or the method provided should accept a single +row+ parameter
23
+ #
24
+ # Returns this AttributeMap for chainability
25
+ def map(transform=nil, &block_transform)
26
+ @transformer = block_transform || transform
27
+ self
28
+ end
29
+
30
+ # Given a CSV row, return the value at this AttributeMap's index using any provided map transforms (see map)
31
+ def parse(csv_row)
32
+ @transformer ? parse_transform(csv_row) : raw_value(csv_row)
33
+ end
34
+
35
+ # Access the raw value of the CSV row without any map transforms applied.
36
+ def raw_value(csv_row)
37
+ csv_row[self.index]
38
+ end
39
+
40
+ private
41
+
42
+ def parse_transform(csv_row)
43
+ if @transformer.is_a? Symbol
44
+ transform_name = @transformer
45
+ @transformer = lambda{|row, index| @map_context.send(transform_name, row, index) }
46
+ end
47
+
48
+ if @transformer.arity == 1
49
+ @transformer.call(csv_row)
50
+ else
51
+ @transformer.call(csv_row, @index)
52
+ end
53
+ end
54
+ end
55
+ end
@@ -0,0 +1,184 @@
1
+ require 'csv-mapper/attribute_map'
2
+
3
+ module CsvMapper
4
+ # CsvMapper::RowMap provides a simple, DSL-like interface for constructing mappings.
5
+ # A CsvMapper::RowMap provides the main functionality of the library. It will mostly be used indirectly through the CsvMapper API,
6
+ # but may be useful to use directly for the dynamic CSV mappings.
7
+ class RowMap
8
+ #Start with a 'blank slate'
9
+ instance_methods.each { |m| undef_method m unless m =~ /^__||instance_eval/ }
10
+
11
+ Infinity = 1.0/0
12
+ attr_reader :mapped_attributes
13
+
14
+ # Create a new instance with access to an evaluation context
15
+ def initialize(context, csv_data = nil, &map_block)
16
+ @context = context
17
+ @csv_data = csv_data
18
+ @before_filters = []
19
+ @after_filters = []
20
+ @parser_options = {}
21
+ @start_at_row = 0
22
+ @stop_at_row = Infinity
23
+ @delimited_by = FasterCSV::DEFAULT_OPTIONS[:col_sep]
24
+ @mapped_attributes = []
25
+
26
+ self.instance_eval(&map_block) if block_given?
27
+ end
28
+
29
+ # Each row of a CSV is parsed and mapped to a new instance of a Ruby class; Struct by default.
30
+ # Use this method to change the what class each row is mapped to.
31
+ # The given class must respond to a parameter-less #new and all attribute mappings defined.
32
+ # Providing a hash of defaults will ensure that each resulting object will have the providing name and attribute values
33
+ # unless overridden by a mapping
34
+ def map_to(klass, defaults={})
35
+ @map_to_klass = klass
36
+
37
+ defaults.each do |name, value|
38
+ self.add_attribute(name, -99).map lambda{|row, index| value}
39
+ end
40
+ end
41
+
42
+ # Allow us to read the first line of a csv file to automatically generate the attribute names.
43
+ # Spaces are replaced with underscores and non-word characters are removed.
44
+ #
45
+ # Keep in mind that there is potential for overlap in using this (i.e. you have a field named
46
+ # files+ and one named files- and they both get named 'files').
47
+ #
48
+ # You can specify aliases to rename fields to prevent conflicts and/or improve readability and compatibility.
49
+ #
50
+ # i.e. read_attributes_from_file('files+' => 'files_plus', 'files-' => 'files_minus)
51
+ def read_attributes_from_file aliases = {}
52
+ attributes = FasterCSV.new(@csv_data, @parser_options).readline
53
+ @start_at_row = [ @start_at_row, 1 ].max
54
+ @csv_data.rewind
55
+ unnamed_number = 1
56
+ attributes.each_with_index do |name, index|
57
+ if name.nil?
58
+ name = "_field_#{unnamed_number}"
59
+ unnamed_number += 1
60
+ end
61
+ name.strip!
62
+ use_name = aliases[name] || name.gsub(/\s+/, '_').gsub(/[\W]+/, '').downcase
63
+ add_attribute use_name, index
64
+ end
65
+ end
66
+
67
+ # Specify a hash of FasterCSV options to be used for CSV parsing
68
+ #
69
+ # Can be anything FasterCSV::new()[http://fastercsv.rubyforge.org/classes/FasterCSV.html#M000018] accepts
70
+ def parser_options(opts=nil)
71
+ @parser_options = opts if opts
72
+ @parser_options.merge :col_sep => @delimited_by
73
+ end
74
+
75
+ # Convenience method to 'move' the cursor skipping the current index.
76
+ def _SKIP_
77
+ self.move_cursor
78
+ end
79
+
80
+ # Specify the CSV column delimiter. Defaults to comma.
81
+ def delimited_by(delimiter=nil)
82
+ @delimited_by = delimiter if delimiter
83
+ @delimited_by
84
+ end
85
+
86
+ # Declare what row to begin parsing the CSV.
87
+ # This is useful for skipping headers and such.
88
+ def start_at_row(row_number=nil)
89
+ @start_at_row = row_number if row_number
90
+ @start_at_row
91
+ end
92
+
93
+ # Declare the last row to be parsed in a CSV.
94
+ def stop_at_row(row_number=nil)
95
+ @stop_at_row = row_number if row_number
96
+ @stop_at_row
97
+ end
98
+
99
+ # Declare method name symbols and/or lambdas to be executed before each row.
100
+ # Each method or lambda must accept to parameters: +csv_row+, +target_object+
101
+ # Methods names should refer to methods available within the RowMap's provided context
102
+ def before_row(*befores)
103
+ self.add_filters(@before_filters, *befores)
104
+ end
105
+
106
+ # Declare method name symbols and/or lambdas to be executed before each row.
107
+ # Each method or lambda must accept to parameters: +csv_row+, +target_object+
108
+ # Methods names should refer to methods available within the RowMap's provided context
109
+ def after_row(*afters)
110
+ self.add_filters(@after_filters, *afters)
111
+ end
112
+
113
+ # Add a new attribute to this map. Mostly used internally, but is useful for dynamic map creation.
114
+ # returns the newly created CsvMapper::AttributeMap
115
+ def add_attribute(name, index=nil)
116
+ attr_mapping = CsvMapper::AttributeMap.new(name.to_sym, index, @context)
117
+ self.mapped_attributes << attr_mapping
118
+ attr_mapping
119
+ end
120
+
121
+ # The current cursor location
122
+ def cursor # :nodoc:
123
+ @cursor ||= 0
124
+ end
125
+
126
+ # Move the cursor relative to it's current position
127
+ def move_cursor(positions=1) # :nodoc:
128
+ self.cursor += positions
129
+ end
130
+
131
+ # Given a CSV row return an instance of an object defined by this mapping
132
+ def parse(csv_row)
133
+ target = self.map_to_class.new
134
+ @before_filters.each {|filter| filter.call(csv_row, target) }
135
+
136
+ self.mapped_attributes.each do |attr_map|
137
+ target.send("#{attr_map.name}=", attr_map.parse(csv_row))
138
+ end
139
+
140
+ @after_filters.each {|filter| filter.call(csv_row, target) }
141
+
142
+ return target
143
+ end
144
+
145
+ protected # :nodoc:
146
+
147
+ # The Hacktastic "magic"
148
+ # Used to dynamically create CsvMapper::AttributeMaps based on unknown method calls that
149
+ # should represent the names of mapped attributes.
150
+ #
151
+ # An optional first argument is used to move this maps cursor position and as the index of the
152
+ # new AttributeMap
153
+ def method_missing(name, *args) # :nodoc:
154
+
155
+ if index = args[0]
156
+ self.move_cursor(index - self.cursor)
157
+ else
158
+ index = self.cursor
159
+ self.move_cursor
160
+ end
161
+
162
+ add_attribute(name, index)
163
+ end
164
+
165
+ def add_filters(to_hook, *filters) # :nodoc:
166
+ (to_hook << filters.collect do |filter|
167
+ filter.is_a?(Symbol) ? lambda{|row, target| @context.send(filter, row, target)} : filter
168
+ end).flatten!
169
+ end
170
+
171
+ def map_to_class # :nodoc:
172
+ unless @map_to_klass
173
+ attrs = mapped_attributes.collect {|attr_map| attr_map.name}
174
+ @map_to_klass = Struct.new(nil, *attrs)
175
+ end
176
+
177
+ @map_to_klass
178
+ end
179
+
180
+ def cursor=(value) # :nodoc:
181
+ @cursor=value
182
+ end
183
+ end
184
+ end
data/lib/csv-mapper.rb ADDED
@@ -0,0 +1,122 @@
1
+ dir = File.dirname(__FILE__)
2
+ $LOAD_PATH.unshift dir unless $LOAD_PATH.include?(dir)
3
+
4
+ require 'rubygems'
5
+
6
+ # the following is slightly modified from Gregory Brown's
7
+ # solution on the Ruport Blaag:
8
+ # http://ruport.blogspot.com/2008/03/fastercsv-api-shim-for-19.html
9
+ if RUBY_VERSION > "1.9"
10
+ require "csv"
11
+ unless defined? FCSV
12
+ class Object
13
+ FasterCSV = CSV
14
+ alias_method :FasterCSV, :CSV
15
+ end
16
+ end
17
+ else
18
+ require "fastercsv"
19
+ end
20
+
21
+ # This module provides the main interface for importing CSV files & data to mapped Ruby objects.
22
+ # = Usage
23
+ # Including CsvMapper will provide two methods:
24
+ # - +import+
25
+ # - +map_csv+
26
+ #
27
+ # See csv-mapper.rb[link:files/lib/csv-mapper_rb.html] for method docs.
28
+ #
29
+ # === Import From File
30
+ # results = import('/path/to/file.csv') do
31
+ # # declare mapping here
32
+ # end
33
+ #
34
+ # === Import From String or IO
35
+ # results = import(csv_data, :type => :io) do
36
+ # # declare mapping here
37
+ # end
38
+ #
39
+ # === Mapping
40
+ # Mappings are built inside blocks. All three of CsvMapper's main API methods accept a block containing a mapping.
41
+ # Maps are defined by using +map_to+, +start_at_row+, +before_row+, and +after_row+ (methods on CsvMapper::RowMap) and
42
+ # by defining your own mapping attributes.
43
+ # A mapping block uses an internal cursor to keep track of the order the mapping attributes are declared and use that order to
44
+ # know the corresponding CSV column index to associate with the attribute.
45
+ #
46
+ # ===== The Basics
47
+ # * +map_to+ - Override the default Struct target. Accepts a class and an optional hash of default attribute names and values.
48
+ # * +start_at_row+ - Specify what row to begin parsing at. Use this to skip headers.
49
+ # * +before_row+ - Accepts an Array of method name symbols or lambdas to be invoked before parsing each row.
50
+ # * +after_row+ - Accepts an Array of method name symbols or lambdas to be invoked after parsing each row.
51
+ # * +delimited_by+ - Accepts a character to be used to delimit columns. Use this to specify pipe-delimited files.
52
+ # * <tt>\_SKIP_</tt> - Use as a placehold to skip a CSV column index.
53
+ # * +parser_options+ - Accepts a hash of FasterCSV options. Can be anything FasterCSV::new()[http://fastercsv.rubyforge.org/classes/FasterCSV.html#M000018] understands
54
+ #
55
+ # ===== Attribute Mappings
56
+ # Attribute mappings are created by using the name of the attribute to be mapped to.
57
+ # The order in which attribute mappings are declared determines the index of the corresponding CSV row.
58
+ # All mappings begin at the 0th index of the CSV row.
59
+ # foo # maps the 0th CSV row position value to the value of the 'foo' attribute on the target object.
60
+ # bar # maps the 1st row position to 'bar'
61
+ # This could also be a nice one liner for easy CSV format conversion
62
+ # [foo, bar] # creates the same attribute maps as above.
63
+ # The mapping index may be specifically declared in two additional ways:
64
+ # foo(2) # maps the 2nd CSV row position value to 'foo' and moves the cursor to 3
65
+ # bar # maps the 3rd CSV row position to 'bar' due to the current cursor position
66
+ # baz.at(0) # maps the 0th CSV row position to 'baz' but only increments the cursor 1 position to 4
67
+ # Each attribute mapping may be configured to parse the record using a lambda or a method name
68
+ # foo.map lambda{|row| row[2].strip } # maps the 2nd row position value with leading and trailing whitespace removed to 'foo'.
69
+ # bar.map :clean_bar # maps the result of the clean_bar method to 'bar'. clean_bar must accept the row as a parameter.
70
+ # Attribute mapping declarations and "modifiers" may be chained
71
+ # foo.at(4).map :some_transform
72
+ #
73
+ # === Create Reusable Mappings
74
+ # The +import+ method accepts an instance of RowMap as an optional mapping parameter.
75
+ # The easiest way to create an instance of a RowMap is by using +map_csv+.
76
+ # a_row_map = map_csv do
77
+ # # declare mapping here
78
+ # end
79
+ # Then you can reuse the mapping
80
+ # results = import(some_string, :type => :io, :map => a_row_map)
81
+ # other_results = import('/path/to/file.csv', :map => a_row_map)
82
+ #
83
+ module CsvMapper
84
+
85
+ # Create a new RowMap instance from the definition in the given block.
86
+ def map_csv(&map_block)
87
+ CsvMapper::RowMap.new(self, &map_block)
88
+ end
89
+
90
+ # Load CSV data and map the values according to the definition in the given block.
91
+ # Accepts either a file path, String, or IO as +data+. Defaults to file path.
92
+ #
93
+ # The following +options+ may be used:
94
+ # <tt>:type</tt>:: defaults to <tt>:file_path</tt>. Use <tt>:io</tt> to specify data as String or IO.
95
+ # <tt>:map</tt>:: Specify an instance of a RowMap to take presidence over a given block defintion.
96
+ #
97
+ def import(data, options={}, &map_block)
98
+ csv_data = options[:type] == :io ? data : File.new(data, 'r')
99
+
100
+ config = { :type => :file_path,
101
+ :map => map_csv_with_data(csv_data, &map_block) }.merge!(options)
102
+
103
+ map = config[:map]
104
+
105
+ results = []
106
+ FasterCSV.new(csv_data, map.parser_options ).each_with_index do |row, i|
107
+ results << map.parse(row) if i >= map.start_at_row && i <= map.stop_at_row
108
+ end
109
+
110
+ results
111
+ end
112
+
113
+ protected
114
+ # Create a new RowMap instance from the definition in the given block and pass the csv_data.
115
+ def map_csv_with_data(csv_data, &map_block) # :nodoc:
116
+ CsvMapper::RowMap.new(self, csv_data, &map_block)
117
+ end
118
+
119
+ extend self
120
+ end
121
+
122
+ require 'csv-mapper/row_map'
@@ -0,0 +1,65 @@
1
+ require File.dirname(__FILE__) + '/../spec_helper.rb'
2
+
3
+ describe CsvMapper::AttributeMap do
4
+
5
+ class TestContext
6
+ def transform_it(row, index)
7
+ :transform_it_success
8
+ end
9
+ end
10
+
11
+ before(:each) do
12
+ @row_attr = CsvMapper::AttributeMap.new('foo', 1, TestContext.new)
13
+ @csv_row = ['first_name', 'last_name']
14
+ end
15
+
16
+ it "should map a destination attribute name" do
17
+ @row_attr.name.should == 'foo'
18
+ end
19
+
20
+ it "should map a CSV column index" do
21
+ @row_attr.index.should be(1)
22
+ end
23
+
24
+ it "should map a transformation between the CSV value and destination value and chain method calls" do
25
+ @row_attr.map(:named_transform).should be(@row_attr)
26
+ end
27
+
28
+ it "should provide ability to set the index and chain method calls" do
29
+ @row_attr.at(9).should be(@row_attr)
30
+ @row_attr.index.should be(9)
31
+ end
32
+
33
+ it "should parse values" do
34
+ @row_attr.parse(@csv_row).should == @csv_row[1]
35
+ end
36
+
37
+ it "should parse values using a mapped lambda transformers" do
38
+ @row_attr.map( lambda{|row, index| :success } )
39
+ @row_attr.parse(@csv_row).should == :success
40
+ end
41
+
42
+ it "should parse values using a mapped lambda transformer that only accepts the row" do
43
+ @row_attr.map( lambda{|row| :success } )
44
+ @row_attr.parse(@csv_row).should == :success
45
+ end
46
+
47
+ it "should parse values using a mapped block transformers" do
48
+ @row_attr.map {|row, index| :success }
49
+ @row_attr.parse(@csv_row).should == :success
50
+ end
51
+
52
+ it "should parse values using a mapped block transformer that only accepts the row" do
53
+ @row_attr.map {|row, index| :success }
54
+ @row_attr.parse(@csv_row).should == :success
55
+ end
56
+
57
+ it "should parse values using a named method on the context" do
58
+ @row_attr.map(:transform_it).parse(@csv_row).should == :transform_it_success
59
+ end
60
+
61
+ it "should provide access to the raw value" do
62
+ @row_attr.raw_value(@csv_row).should be(@csv_row[@row_attr.index])
63
+ end
64
+
65
+ end
@@ -0,0 +1,133 @@
1
+ require File.dirname(__FILE__) + '/../spec_helper.rb'
2
+
3
+ describe CsvMapper::RowMap do
4
+
5
+ class TestMapToClass
6
+ attr_accessor :foo, :bar, :baz
7
+ end
8
+
9
+ class TestMapContext
10
+ def transform(row, index)
11
+ :transform_success
12
+ end
13
+
14
+ def change_name(row, target)
15
+ row[0] = :changed_name
16
+ end
17
+ end
18
+
19
+ before(:each) do
20
+ @row_map = CsvMapper::RowMap.new(TestMapContext.new)
21
+ @csv_row = ['first_name', 'last_name']
22
+ end
23
+
24
+ it "should parse a CSV row" do
25
+ @row_map.parse(@csv_row).should_not be_nil
26
+ end
27
+
28
+ it "should map to a Struct by default" do
29
+ @row_map.parse(@csv_row).should be_kind_of(Struct)
30
+ end
31
+
32
+ it "should parse a CSV row returning the mapped result" do
33
+ @row_map.fname
34
+ @row_map.lname
35
+
36
+ result = @row_map.parse(@csv_row)
37
+ result.fname.should == @csv_row[0]
38
+ result.lname.should == @csv_row[1]
39
+ end
40
+
41
+ it "should map to a ruby class with optional default attribute values" do
42
+ @row_map.map_to TestMapToClass, :baz => :default_baz
43
+
44
+ @row_map.foo
45
+ @row_map.bar
46
+
47
+ (result = @row_map.parse(@csv_row)).should be_instance_of(TestMapToClass)
48
+ result.foo.should == @csv_row[0]
49
+ result.bar.should == @csv_row[1]
50
+ result.baz.should == :default_baz
51
+ end
52
+
53
+ it "should define Infinity" do
54
+ CsvMapper::RowMap::Infinity.should == 1.0/0
55
+ end
56
+
57
+ it "should start at the specified CSV row" do
58
+ @row_map.start_at_row.should == 0
59
+ @row_map.start_at_row(1)
60
+ @row_map.start_at_row.should == 1
61
+ end
62
+
63
+ it "should stop at the specified row" do
64
+ @row_map.stop_at_row.should be(CsvMapper::RowMap::Infinity)
65
+ @row_map.stop_at_row(6)
66
+ @row_map.stop_at_row.should == 6
67
+ end
68
+
69
+ it "should allow before row processing" do
70
+ @row_map.before_row :change_name, lambda{|row, target| row[1] = 'bar'}
71
+
72
+ @row_map.first_name
73
+ @row_map.foo
74
+
75
+ result = @row_map.parse(@csv_row)
76
+ result.first_name.should == :changed_name
77
+ result.foo.should == 'bar'
78
+ end
79
+
80
+ it "should allow after row processing" do
81
+ filter_var = nil
82
+ @row_map.after_row lambda{|row, target| filter_var = :woot}
83
+
84
+ @row_map.parse(@csv_row)
85
+ filter_var.should == :woot
86
+ end
87
+
88
+ it "should have a moveable cursor" do
89
+ @row_map.cursor.should be(0)
90
+ @row_map.move_cursor
91
+ @row_map.cursor.should be(1)
92
+ @row_map.move_cursor 3
93
+ @row_map.cursor.should be(4)
94
+ end
95
+
96
+ it "should skip indexes" do
97
+ pre_cursor = @row_map.cursor
98
+ @row_map._SKIP_
99
+ @row_map.cursor.should be(pre_cursor + 1)
100
+ end
101
+
102
+ it "should accept FasterCSV parser options" do
103
+ @row_map.parser_options :row_sep => :auto
104
+ @row_map.parser_options[:row_sep].should == :auto
105
+ end
106
+
107
+ it "should have a configurable the column delimiter" do
108
+ @row_map.delimited_by '|'
109
+ @row_map.delimited_by.should == '|'
110
+ end
111
+
112
+ it "should maintain a collection of attribute mappings" do
113
+ @row_map.mapped_attributes.should be_kind_of(Enumerable)
114
+ end
115
+
116
+ it "should lazy initialize attribute maps and move the cursor" do
117
+ pre_cursor = @row_map.cursor
118
+ (attr_map = @row_map.first_name).should be_instance_of(CsvMapper::AttributeMap)
119
+ attr_map.index.should be(pre_cursor)
120
+ @row_map.cursor.should be(pre_cursor + 1)
121
+ end
122
+
123
+ it "should lazy initialize attribute maps with optional cursor position" do
124
+ pre_cursor = @row_map.cursor
125
+ @row_map.last_name(1).index.should be(1)
126
+ @row_map.cursor.should be(1)
127
+ end
128
+
129
+ it "should share its context with its mappings" do
130
+ @row_map.first_name.map(:transform)
131
+ @row_map.parse(@csv_row).first_name.should == :transform_success
132
+ end
133
+ end
@@ -0,0 +1,181 @@
1
+ require File.dirname(__FILE__) + '/spec_helper.rb'
2
+
3
+ describe CsvMapper do
4
+ describe "included" do
5
+ before(:each) do
6
+ @mapped_klass = Class.new do
7
+ include CsvMapper
8
+ def upcase_name(row, index)
9
+ row[index].upcase
10
+ end
11
+ end
12
+ @mapped = @mapped_klass.new
13
+ end
14
+
15
+ it "should allow the creation of CSV mappings" do
16
+ mapping = @mapped.map_csv do
17
+ start_at_row 2
18
+ end
19
+
20
+ mapping.should be_instance_of(CsvMapper::RowMap)
21
+ mapping.start_at_row.should == 2
22
+ end
23
+
24
+ it "should import a CSV IO" do
25
+ io = 'foo,bar,00,01'
26
+ results = @mapped.import(io, :type => :io) do
27
+ first
28
+ second
29
+ end
30
+
31
+ results.should be_kind_of(Enumerable)
32
+ results.should have(1).things
33
+ results[0].first.should == 'foo'
34
+ results[0].second.should == 'bar'
35
+ end
36
+
37
+ it "should import a CSV File IO" do
38
+ results = @mapped.import(File.dirname(__FILE__) + '/test.csv') do
39
+ start_at_row 1
40
+ [first_name, last_name, age]
41
+ end
42
+
43
+ results.size.should == 3
44
+ end
45
+
46
+ it "should stop importing at a specified row" do
47
+ results = @mapped.import(File.dirname(__FILE__) + '/test.csv') do
48
+ start_at_row 1
49
+ stop_at_row 2
50
+ [first_name, last_name, age]
51
+ end
52
+
53
+ results.size.should == 2
54
+ end
55
+
56
+ it "should be able to read attributes from a csv file" do
57
+ results = @mapped.import(File.dirname(__FILE__) + '/test.csv') do
58
+ # we'll alias age here just as an example
59
+ read_attributes_from_file('Age' => 'number_of_years_old')
60
+ end
61
+ results[1].first_name.should == 'Jane'
62
+ results[1].last_name.should == 'Doe'
63
+ results[1].number_of_years_old.should == '26'
64
+ end
65
+
66
+ it "should import non-comma delimited files" do
67
+ piped_io = 'foo|bar|00|01'
68
+
69
+ results = @mapped.import(piped_io, :type => :io) do
70
+ delimited_by '|'
71
+ [first, second]
72
+ end
73
+
74
+ results.should have(1).things
75
+ results[0].first.should == 'foo'
76
+ results[0].second.should == 'bar'
77
+ end
78
+
79
+ it "should allow named tranformation mappings" do
80
+ def upcase_name(row)
81
+ row[0].upcase
82
+ end
83
+
84
+ results = @mapped.import(File.dirname(__FILE__) + '/test.csv') do
85
+ start_at_row 1
86
+
87
+ first_name.map :upcase_name
88
+ end
89
+
90
+ results[0].first_name.should == 'JOHN'
91
+ end
92
+ end
93
+
94
+ describe "extended" do
95
+ it "should allow the creation of CSV mappings" do
96
+ mapping = CsvMapper.map_csv do
97
+ start_at_row 2
98
+ end
99
+
100
+ mapping.should be_instance_of(CsvMapper::RowMap)
101
+ mapping.start_at_row.should == 2
102
+ end
103
+
104
+ it "should import a CSV IO" do
105
+ io = 'foo,bar,00,01'
106
+ results = CsvMapper.import(io, :type => :io) do
107
+ first
108
+ second
109
+ end
110
+
111
+ results.should be_kind_of(Enumerable)
112
+ results.should have(1).things
113
+ results[0].first.should == 'foo'
114
+ results[0].second.should == 'bar'
115
+ end
116
+
117
+ it "should import a CSV File IO" do
118
+ results = CsvMapper.import(File.dirname(__FILE__) + '/test.csv') do
119
+ start_at_row 1
120
+ [first_name, last_name, age]
121
+ end
122
+
123
+ results.size.should == 3
124
+ end
125
+
126
+ it "should stop importing at a specified row" do
127
+ results = CsvMapper.import(File.dirname(__FILE__) + '/test.csv') do
128
+ start_at_row 1
129
+ stop_at_row 2
130
+ [first_name, last_name, age]
131
+ end
132
+
133
+ results.size.should == 2
134
+ end
135
+
136
+ it "should be able to read attributes from a csv file" do
137
+ results = CsvMapper.import(File.dirname(__FILE__) + '/test.csv') do
138
+ # we'll alias age here just as an example
139
+ read_attributes_from_file('Age' => 'number_of_years_old')
140
+ end
141
+ results[1].first_name.should == 'Jane'
142
+ results[1].last_name.should == 'Doe'
143
+ results[1].number_of_years_old.should == '26'
144
+ end
145
+
146
+ it "should be able to assign default column names when column names are null" do
147
+ results = CsvMapper.import(File.dirname(__FILE__) + '/test_with_empty_column_names.csv') do
148
+ read_attributes_from_file
149
+ end
150
+
151
+ results[1]._field_1.should == 'unnamed_value'
152
+ end
153
+
154
+ it "should import non-comma delimited files" do
155
+ piped_io = 'foo|bar|00|01'
156
+
157
+ results = CsvMapper.import(piped_io, :type => :io) do
158
+ delimited_by '|'
159
+ [first, second]
160
+ end
161
+
162
+ results.should have(1).things
163
+ results[0].first.should == 'foo'
164
+ results[0].second.should == 'bar'
165
+ end
166
+
167
+ it "should not allow transformation mappings" do
168
+ def upcase_name(row)
169
+ row[0].upcase
170
+ end
171
+
172
+ (lambda do
173
+ results = CsvMapper.import(File.dirname(__FILE__) + '/test.csv') do
174
+ start_at_row 1
175
+
176
+ first_name.map :upcase_name
177
+ end
178
+ end).should raise_error(Exception)
179
+ end
180
+ end
181
+ end
data/spec/spec.opts ADDED
@@ -0,0 +1 @@
1
+ --color
@@ -0,0 +1,9 @@
1
+ $LOAD_PATH.unshift(File.dirname(__FILE__))
2
+ $LOAD_PATH.unshift(File.join(File.dirname(__FILE__), '..', 'lib'))
3
+ require 'csv-mapper'
4
+ require 'spec'
5
+ require 'spec/autorun'
6
+
7
+ Spec::Runner.configure do |config|
8
+
9
+ end
data/spec/test.csv ADDED
@@ -0,0 +1,4 @@
1
+ First Name, Last Name, Age
2
+ John,Doe,27
3
+ Jane,Doe,26
4
+ Bat,Man,52
@@ -0,0 +1,4 @@
1
+ First Name, Last Name, Age,,,
2
+ John,Doe,27,,,
3
+ Jane,Doe,26,unnamed_value,,
4
+ Bat,Man,52,,,
metadata ADDED
@@ -0,0 +1,117 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: rgarner-csv-mapper
3
+ version: !ruby/object:Gem::Version
4
+ hash: 9
5
+ prerelease: false
6
+ segments:
7
+ - 0
8
+ - 5
9
+ - 1
10
+ version: 0.5.1
11
+ platform: ruby
12
+ authors:
13
+ - Luke Pillow
14
+ - Russell Garner
15
+ autorequire:
16
+ bindir: bin
17
+ cert_chain: []
18
+
19
+ date: 2010-08-25 00:00:00 +01:00
20
+ default_executable:
21
+ dependencies:
22
+ - !ruby/object:Gem::Dependency
23
+ name: rspec
24
+ prerelease: false
25
+ requirement: &id001 !ruby/object:Gem::Requirement
26
+ none: false
27
+ requirements:
28
+ - - ">="
29
+ - !ruby/object:Gem::Version
30
+ hash: 13
31
+ segments:
32
+ - 1
33
+ - 2
34
+ - 9
35
+ version: 1.2.9
36
+ type: :development
37
+ version_requirements: *id001
38
+ - !ruby/object:Gem::Dependency
39
+ name: fastercsv
40
+ prerelease: false
41
+ requirement: &id002 !ruby/object:Gem::Requirement
42
+ none: false
43
+ requirements:
44
+ - - ">="
45
+ - !ruby/object:Gem::Version
46
+ hash: 3
47
+ segments:
48
+ - 0
49
+ version: "0"
50
+ type: :runtime
51
+ version_requirements: *id002
52
+ description: CSV Mapper makes it easy to import data from CSV files directly to a collection of any type of Ruby object. The simplest way to create mappings is declare the names of the attributes in the order corresponding to the CSV file column order.
53
+ email: rgarner@zephyros-systems.co.uk
54
+ executables: []
55
+
56
+ extensions: []
57
+
58
+ extra_rdoc_files:
59
+ - History.txt
60
+ - LICENSE
61
+ - README.rdoc
62
+ files:
63
+ - .gitignore
64
+ - History.txt
65
+ - LICENSE
66
+ - README.rdoc
67
+ - Rakefile
68
+ - VERSION
69
+ - lib/csv-mapper.rb
70
+ - lib/csv-mapper/attribute_map.rb
71
+ - lib/csv-mapper/row_map.rb
72
+ - spec/csv-mapper/attribute_map_spec.rb
73
+ - spec/csv-mapper/row_map_spec.rb
74
+ - spec/csv-mapper_spec.rb
75
+ - spec/spec.opts
76
+ - spec/spec_helper.rb
77
+ - spec/test.csv
78
+ - spec/test_with_empty_column_names.csv
79
+ has_rdoc: true
80
+ homepage: http://github.com/rgarner/csv-mapper
81
+ licenses: []
82
+
83
+ post_install_message:
84
+ rdoc_options:
85
+ - --charset=UTF-8
86
+ require_paths:
87
+ - lib
88
+ required_ruby_version: !ruby/object:Gem::Requirement
89
+ none: false
90
+ requirements:
91
+ - - ">="
92
+ - !ruby/object:Gem::Version
93
+ hash: 3
94
+ segments:
95
+ - 0
96
+ version: "0"
97
+ required_rubygems_version: !ruby/object:Gem::Requirement
98
+ none: false
99
+ requirements:
100
+ - - ">="
101
+ - !ruby/object:Gem::Version
102
+ hash: 3
103
+ segments:
104
+ - 0
105
+ version: "0"
106
+ requirements: []
107
+
108
+ rubyforge_project:
109
+ rubygems_version: 1.3.7
110
+ signing_key:
111
+ specification_version: 3
112
+ summary: rgarner-CsvMapper is a fork of a small library intended to simplify the common steps involved with importing CSV files to a usable form in Ruby. It has support for null column names. When this is merged, this gem will be removed.
113
+ test_files:
114
+ - spec/csv-mapper/attribute_map_spec.rb
115
+ - spec/csv-mapper/row_map_spec.rb
116
+ - spec/csv-mapper_spec.rb
117
+ - spec/spec_helper.rb