sycsvpro 0.1.12 → 0.1.13

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    sycsvpro (0.1.12)
+    sycsvpro (0.1.13)
       gli (= 2.9.0)
       timeleap (~> 0.0.1)
 

data/README.md

@@ -7,6 +7,7 @@ Processing of csv files. *sycsvpro* offers following functions
 * extract rows and columns from a file
 * remove duplicate lines from a file where duplicates are identified by key
   columns (since version 0.1.11)
+  add unique to command line interface (since version 0.1.12)
 * collect values of rows and assign them to categories
 * map column values to new values
 * allocate column values to a key column (since version 0.0.4)
@@ -22,6 +23,7 @@ Processing of csv files. *sycsvpro* offers following functions
   version 0.1.4)
 * join two file based on a joint column value (since version 0.1.7)
 * merge files based on common headline columns (since version 0.1.10)
+* transpose (swapping) rows and columns (since version 0.1.13)
 
 To get help type
 
@@ -108,7 +110,7 @@ Collect all product rows (2, 3 and 4) to the category product
 
 Map
 ---
-Map the product names to new names
+Map the product names to new names. Consider columns 2-4 only for mapping
 
 The mapping file (mapping) uses the result from the collect command above
 
@@ -127,6 +129,35 @@ The mapping file (mapping) uses the result from the collect command above
 
     $ sycsvpro -f in.csv -o out.csv map mapping -c 2-4
 
+Transpose
+---------
+Swap rows and columns of revenue.csv to out.csv
+
+    $ sycsvpro -f revenue.csv -o out.csv transpose
+    
+    2010;50;100;2000
+    2011;100;50;250
+    2012;150;10;300
+    2013;100;1000;3000
+    2014;200;20;20
+    customer;hello;indix;chiro
+
+To use only columns 2013 and 2014 you can specify a the columns to transpose
+
+    $ sycsvpro -f revenue.csv -o out.csv transpose -c 3-5
+
+    2013;100;1000;3000
+    2014;200;20;20
+    customer;hello;indix;chiro
+
+To filter for hello only
+
+    $ sycsvpor -f revenue.csv -o out.csv transpose -c 3-5 -r 0,1
+
+    2013;100
+    2014;200
+    customer;hello
+
 Allocate
 --------
 Allocate all the machine types to the customer
@@ -196,7 +227,7 @@ Process arithmetic operations on the contract count and create a target column
 and a sum which is added at the end of the result file
 
     $ sycsvpro -f in.csv -o out.csv calc -r 2-20 -h *,target 
-               -c 6:*2,7:target=c6*10
+               -c 6:*2,7:c6*10
 
     $ cat out.csv
     customer;machine;control;drive;motor;date;contract;target
@@ -210,6 +241,20 @@ and a sum which is added at the end of the result file
 In the sum row non-numbers in the colums are converted to 0. Therefore column 0
 is summed up to 0 as all strings are converted to 0.
 
+Write only columns 0, 6 and 7 by specifying write columns
+
+    $ sycsvpro -f in.csv -o out.csv calc -r 2-20 -h "customer,contract,target"
+                                         -c 6:*2,7:c6*10
+                                         -w 0,6-7
+    $ cat out.csv
+    customer;contract;target
+    hello;2;20
+    hello;2;20
+    indix;2;20
+    chiro;2;20
+    chiro;2;20
+    0;10;100
+
 Join
 ----
 Join the machine and contract file with columns from the customer address file
@@ -250,6 +295,7 @@ Merge files machine_count.csv and revenue.csv based on the year columns.
 This will create the out.csv
 
 ```
+$ cat out.csv
 ;2010;2013;2014
 hello;1;0;0
 indix;1;0;0
@@ -266,6 +312,7 @@ Sort rows on specified columns as an example sort rows based on customer
 
     $ sycsvpro -f in.csv -o out.csv sort -r 2-20 -c s:0,d:5
     
+    $cat out.csv
     customer;machine;control;drive;motor;date;contract;target
     hello;h2;con123;dri130;mot110;1.02.3012;1
     hello;h1;con123;dri120;mot100;1.01.3013;1
@@ -406,8 +453,8 @@ row are added on top of the sorted file
 * `sycsvpro -f infile analyze` now lists the columns with sample data
 * Add `params` method to *Dsl* that retrieves the params provided in the execute
 command: `sycsvpro execute script.rb method infile param1 param2`
-* Add `clean_up` to *Dsl* that takes files to be deleted after the script has
-run: `clean_up(%w{file1 file2})`
+* Add `clean\_up` to *Dsl* that takes files to be deleted after the script has
+run: `clean\_up(%w{file1 file2})`
 
 Version 0.1.4
 -------------
@@ -465,7 +512,7 @@ Version 0.1.7
   This will join infile.csv with source.csv based on the join columns (j "1=3").
   From source.csv columns 2 and 4 (-c "2,4") will be inserted at column
   positions 1 and 3 (-p "1,3"). The header will be used from the infile.csv
-  (-h "*") supplemented by the columns A and B (-i "A,B") that will also be
+  (-h "\*") supplemented by the columns A and B (-i "A,B") that will also be
   positioned at column 1 and 3 (-p "1,3").
 
 Version 0.1.8
@@ -474,8 +521,9 @@ Version 0.1.8
 
 Version 0.1.9
 -------------
-* When creating columns dynamically they are in arbitrary sequence. You can now
-  provide a switch `sort: "2"` which will sort the header from column 2 on.
+* When creating columns dynamically in count they are in arbitrary sequence. 
+  You can now provide a switch `sort: "2"` which will sort the header from 
+  column 2 on.
 
 Version 0.1.10
 --------------
@@ -488,6 +536,27 @@ Version 0.1.11
 * Unique removes duplicate lines from the infile. Duplicate lines are identified
   by key columns
 
+Version 0.1.12
+--------------
+* Add unique to sycsvpro command line interface
+
+Version 0.1.13
+--------------
+* Optimize Mapper by only considering columns provided for mapping which should
+  increase performance
+* match\_boolean\_filter? in Filter now also processes strings with single
+  quotes inside
+* Tranposer tranposes rows and columns that is make columns rows and vice versa
+* Calculator can now have colons inside the operation
+     sycsvpro -f in.csv -o out.csv -c "122:+[1,3,5].inject(:+)"
+  Previously the operation would have been cut after inject(
+* A write flag in Calculator specifies which colons to add to the result.
+* Calculator introduced a switch 'final\_header' which indicates the header
+  provided should not be filtered in regard to a provided 'write' flag but 
+  written to the result file as is
+* Merger now doesn't require a key column that is files can be merged without
+  key columns.
+
 Installation
 ============
 [![Gem Version](https://badge.fury.io/rb/sycsvpro.png)](http://badge.fury.io/rb/sycsvpro)

data/bin/sycsvpro

@@ -589,6 +589,27 @@ command :map do |c|
   end
 end
 
+desc 'Transposes rows and columns'
+command :transpose do |c|
+  c.desc 'Rows to consider'
+  c.arg_name 'ROW1,ROW2,ROW10-ROW30,45-EOF,REGEXP'
+  c.flag [:r, :row], :must_match => row_regex 
+
+  c.desc 'Columns to consider for mapping'
+  c.arg_name 'COL1,COL2,COL10-COL30'
+  c.flag [:c, :col], :must_match => /\d+(?:,\d+|-\d+)*/
+ 
+  c.action do |global_options,options,args|
+    print "Transpose..."
+    transpose = Sycsvpro::Transposer.new(infile:  global_options[:f], 
+                                         outfile: global_options[:o],
+                                         rows:    options[:r], 
+                                         cols:    options[:c])
+    transpose.execute
+    puts "done"
+  end
+end
+
 desc 'Process operations on columns. Optionally add a sum row for columns with'+
      'number values'
 command :calc do |c|
@@ -600,6 +621,11 @@ command :calc do |c|
     default_value '*'
     c.flag [:h, :header], :must_match => /^[*|\w ]+(?:,[\w ]+)*/
 
+    c.desc 'Indicates whether the provided header is final. That is if columns'+
+           ' to be written to the outfile are selected by the write flag then '+
+           'the header should left untouched and written as is'
+    c.switch [:f, :final], :default_value => false
+
     c.desc 'Rows to consider for calculations'
     c.arg_name 'ROW1,ROW2-ROW10,45-EOF,REGEXP'
     c.flag [:r, :row], :must_match => row_regex
@@ -610,6 +636,10 @@ command :calc do |c|
     c.arg_name "COL1:*2,COL2:-C3,COL3:*2+(4+C5)"
     c.flag [:c, :col], :must_match => /^\d+:.+/
 
+    c.desc 'Columns to be written to the result file'
+    c.arg_name "COL1,COL2-COL5"
+    c.flag [:w, :write], :must_match => /\d+(?:,\d+|-\d+)*/
+
     c.desc 'Date format of date columns'
     c.arg_name '%d.%m.%Y|%Y-%m-%d|...'
     c.flag [:df]
@@ -622,13 +652,15 @@ command :calc do |c|
     help_now! "You need to provide the column flag" if options[:c].nil?
 
     print "Calculating..."
-    calculator = Sycsvpro::Calculator.new(infile:  global_options[:f], 
-                                          outfile: global_options[:o],
-                                          header:  options[:h], 
-                                          rows:    options[:r], 
-                                          cols:    options[:c],
-                                          sum:     options[:s], 
-                                          df:      options[:df])
+    calculator = Sycsvpro::Calculator.new(infile:       global_options[:f], 
+                                          outfile:      global_options[:o],
+                                          header:       options[:h], 
+                                          final_header: options[:f],
+                                          rows:         options[:r], 
+                                          cols:         options[:c],
+                                          write:        options[:w],
+                                          sum:          options[:s], 
+                                          df:           options[:df])
     calculator.execute 
     puts "done"
   end

data/lib/sycsvpro/calculator.rb

@@ -58,8 +58,13 @@ module Sycsvpro
     attr_reader :formulae
     # header of the outfile
     attr_reader :header
+    # indicates whether this header is final and should not be filtered in
+    # respect to the columns defined by write
+    attr_reader :final_header
     # filter that is used for columns
     attr_reader :columns
+    # selected columns to be written to outfile
+    attr_reader :write
     # if true add a sum row at the bottom of the out file
     attr_reader :add_sum_row
 
@@ -67,29 +72,36 @@ module Sycsvpro
     # can be supplemented with additional column names that are generated due 
     # to an arithmetic operation that creates new columns
     # :call-seq:
-    #   Sycsvpro::Calculator.new(infile:  "in.csv",
-    #                            outfile: "out.csv",
-    #                            df:      "%d.%m.%Y",
-    #                            rows:    "1,2,BEGINn3>20END",
-    #                            header:  "*,Count",
-    #                            cols:    "4:c1+c2*2",
-    #                            sum:     true).execute
+    #   Sycsvpro::Calculator.new(infile:       "in.csv",
+    #                            outfile:      "out.csv",
+    #                            df:           "%d.%m.%Y",
+    #                            rows:         "1,2,BEGINn3>20END",
+    #                            header:       "*,Count",
+    #                            final_header: false,
+    #                            cols:         "4:c1+c2*2",
+    #                            write:        "1,3-5",
+    #                            sum:          true).execute
     # infile:: File that contains the rows to be operated on
     # outfile:: Result of the operations
     # df:: Date format
     # rows:: Row filter that indicates which rows to consider
     # header:: Header of the columns
+    # final_header:: Indicates that if write filters columns the header should
+    # not be filtered when written
     # cols:: Operations on the column values
+    # write:: Columns that are written to the outfile
     # sum:: Indicate whether to add a sum row
     def initialize(options={})
-      @infile      = options[:infile]
-      @outfile     = options[:outfile]
-      @date_format = options[:df] || "%Y-%m-%d"
-      @row_filter  = RowFilter.new(options[:rows], df: options[:df])
-      @header      = Header.new(options[:header])
-      @sum_row     = []
-      @add_sum_row = options[:sum] || false
-      @formulae    = {}
+      @infile       = options[:infile]
+      @outfile      = options[:outfile]
+      @date_format  = options[:df] || "%Y-%m-%d"
+      @row_filter   = RowFilter.new(options[:rows], df: options[:df])
+      @write_filter = ColumnFilter.new(options[:write], df: options[:df])
+      @header       = Header.new(options[:header])
+      @final_header = options[:final_header]
+      @sum_row      = []
+      @add_sum_row  = options[:sum]
+      @formulae     = {}
       create_calculator(options[:cols])
     end
 
@@ -112,7 +124,8 @@ module Sycsvpro
 
           unless processed_header
             header_row = header.process(line.chomp)
-            out.puts header_row unless header_row.empty?
+            header_row = @write_filter.process(header_row) unless @final_header
+            out.puts header_row unless header_row.nil? or header_row.empty?
             processed_header = true
             next
           end
@@ -123,7 +136,7 @@ module Sycsvpro
           formulae.each do |col, formula|
             @columns[col.to_i] = eval(formula)
           end
-          out.puts @columns.join(';')
+          out.puts @write_filter.process(@columns.join(';'))
 
           @columns.each_with_index do |column, index|
             column = 0 unless column.to_s =~ /^[\d\.,]*$/
@@ -137,7 +150,7 @@ module Sycsvpro
 
         end
 
-        out.puts @sum_row.join(';') if add_sum_row
+        out.puts @write_filter.process(@sum_row.join(';')) if add_sum_row
 
       end
     end
@@ -154,7 +167,7 @@ module Sycsvpro
       # column 1 + 1 c[4] = c[1] + 1
       def create_calculator(code)
         code.split(/,(?=\d+:)/).each do |operation|
-          col, term = operation.split(':')
+          col, term = operation.split(':', 2)
           term = "c#{col}#{term}" if term =~ /^[+\-*\/%]/
           formulae[col] = term
         end

data/lib/sycsvpro/dsl.rb

@@ -76,8 +76,9 @@ module Dsl
     end
   end
 
-  # Remove leading and trailing " and spaces as well as reducing more than 2 spaces between words
-  # from csv values. Replac ; with , from values as ; is used as value separator
+  # Remove leading and trailing " and spaces as well as reducing more than 2 
+  # spaces between words from csv values. Replace ; with , from values as ; 
+  # is used as value separator
   def unstring(line)
     line = str2utf8(line)
     line.scan(/(?<=^"|;")[^"]+(?=;)+[^"]*|;+[^"](?=";|"$)/).each do |value|

data/lib/sycsvpro/filter.rb

@@ -71,7 +71,7 @@ module Sycsvpro
         when 'n'
           values[c[2].to_i].empty? ? '0' : values[c[2].to_i]
         when 's'
-          "'#{values[c[2].to_i]}'"
+          "\"#{values[c[2].to_i]}\""
         when 'd'
           begin
             Date.strptime(values[c[2].to_i], date_format)

data/lib/sycsvpro/mapper.rb

@@ -2,8 +2,33 @@
 module Sycsvpro
 
   # Map values to new values described in a mapping file
+  #
+  # in.csv
+  #
+  # | ID  | Name |
+  # | --- | ---- |
+  # | 1   | Hank |
+  # | 2   | Jane |
+  #
+  # mapping
+  #
+  # 1:01
+  # 2:02
+  #
+  # Sycsvpro::Mapping.new(infile:  "in.csv",
+  #                       outfile: "out.csv",
+  #                       mapping: "mapping",
+  #                       cols:    "0").execute
+  # out.csv
+  #
+  # | ID  | Name |
+  # | --- | ---- |
+  # | 01  | Hank |
+  # | 02  | Jane |
   class Mapper
 
+    include Dsl
+
     # infile contains the data that is operated on
     attr_reader :infile
     # outfile is the file where the result is written to
@@ -12,15 +37,29 @@ module Sycsvpro
     attr_reader :mapper
     # filter that is used for rows
     attr_reader :row_filter
-    # filter that is used for columns
+    # filter that contains columns that are considered for mappings
     attr_reader :col_filter
 
     # Creates new mapper
+    # :call-seq:
+    #   Sycsvpro::Mapper.new(infile: "in.csv",
+    #                        outfile: "out.csv",
+    #                        mapping: "mapping.csv",
+    #                        rows:    "1,3-5",
+    #                        cols:    "3,4-7"
+    #                        df:      "%Y-%m-%d").execute
+    #
+    # infile:: File that contains columns to be mapped
+    # outfile:: File that contains the mapping result after execute
+    # mapping:: File that contains the mappings. Mappings are separated by ':'
+    # rows:: Rows to consider for mappings
+    # cols:: Columns that should be mapped
+    # df:: Date format for row filter if rows are filtered on date values
     def initialize(options={})
       @infile = options[:infile]
       @outfile = options[:outfile]
-      @row_filter = RowFilter.new(options[:row_filter], df: options[:df])
-      @col_filter = ColumnFilter.new(options[:col_filter], df: options[:df])
+      @row_filter = RowFilter.new(options[:rows], df: options[:df])
+      @col_filter = init_col_filter(options[:cols], @infile)
       @mapper = {}
       init_mapper(options[:mapping])
     end
@@ -29,25 +68,49 @@ module Sycsvpro
     def execute
       File.open(outfile, 'w') do |out|
         File.new(infile, 'r').each_with_index do |line, index|
-          result = col_filter.process(row_filter.process(line, row: index))
+          result = row_filter.process(line, row: index)
           next if result.chomp.empty? or result.nil?
-          mapper.each do |from, to|
-            result = result.chomp.gsub(/(?<=^|;)#{from}(?=;|$)/, to)
+          result += ' ' if result =~ /;$/
+          cols = result.split(';')
+          @col_filter.each do |key|
+            substitute = mapper[cols[key]]
+            cols[key] = substitute if substitute
           end
-          out.puts result
+          out.puts cols.join(';').strip
         end
       end
     end
 
     private
 
-      # Initializes the mappings
+      # Initializes the mappings. A mapping consists of the value to be mapped
+      # to another value. The values are spearated by colons ':'
+      # Example:
+      #     source_value:mapping_value
       def init_mapper(file)
         File.new(file, 'r').each_line do |line|
-          from, to = line.chomp.split(':')
+          from, to = unstring(line).split(':')
           mapper[from] = to
         end
       end
+
+      # Initialize the col_filter that contains columns to be considered for
+      # mapping. If no columns are provided, that is being empty, a filter with
+      # all columns is returned 
+      def init_col_filter(columns, source)
+        if columns.nil?
+          File.open(source, 'r').each do |line| 
+            line = unstring(line)
+            next if line.empty?
+            line += ' ' if line =~ /;$/
+            size = line.split(';').size
+            columns = "0-#{size-1}"
+            break
+          end
+        end
+        ColumnFilter.new(columns).filter.flatten
+      end
+
   end
 
 end

data/lib/sycsvpro/merger.rb

@@ -69,21 +69,25 @@ module Sycsvpro
     # source_header:: pattern for each header of the source file to determine
     # the column. The pattern is a regex without the enclosing slashes '/'
     # key:: first column value from the source file that is used as first
-    # column in the target file
+    # column in the target file. The key is optional.
     def initialize(options = {})
       @outfile       = options[:outfile]
       @header_cols   = options[:header].split(',')
       @source_header = options[:source_header].split(',')
-      @key           = options[:key].split(',')
+      @key           = options[:key] ? options[:key].split(',') : []
+      @has_key       = !@key.empty?
       @files         = options[:files].split(',')
+      if @source_header.count != @files.count
+        raise "file count has to be equal to source_header count"
+      end
     end
 
     # Merges the files based on the provided parameters
     def execute
       File.open(outfile, 'w') do |out|
-        out.puts ";#{header_cols.join(';')}"
+        out.puts "#{';' unless @key.empty?}#{header_cols.join(';')}"
         files.each do |file|
-          @current_key = @key.shift
+          @current_key = create_current_key
           @current_source_header = @source_header.shift
           processed_header = false
           File.open(file).each_with_index do |line, index|
@@ -110,16 +114,25 @@ module Sycsvpro
           columns[i] = c.scan(Regexp.new(@current_source_header)).flatten[0]
         end
 
-        @file_header = [@current_key.to_i]
+        @file_header = @current_key ? [@current_key.to_i] : []
+
         header_cols.each do |h|
           @file_header << columns.index(h) 
         end 
+
         @file_header.compact!
       end
 
+      # create the current key dependent on the value returns a number or nil
+      def create_current_key
+        key = @key.shift
+        key.nil? || key.strip.empty? ? nil : key
+      end
+
       # create a line filtered by the file_header
       def create_line(columns)
-        columns.values_at(*@file_header).join(';')
+        empty_col = ';' if @has_key && @current_key.nil?
+        "#{empty_col}#{columns.values_at(*@file_header).join(';')}"
       end
 
   end

data/lib/sycsvpro/transposer.rb

@@ -0,0 +1,77 @@
+# Operating csv files
+module Sycsvpro
+
+  # Tranposes rows to columns and vice versa
+  # 
+  # Example
+  #
+  # infile.csv
+  # | Year | SP | RP | Total | SP-O | RP-O | O   |
+  # | ---- | -- | -- | ----- | ---- | ---- | --- |
+  # |      | 10 | 20 | 30    | 100  | 40   | 140 |
+  # | 2008 |  5 | 10 | 15    |  10  | 20   |  10 |
+  # | 2009 |  2 |  5 |  5    |  20  | 10   |  30 |
+  # | 2010 |  3 |  5 | 10    |  70  | 10   | 100 |
+  # 
+  # outfile.csv
+  # | Year  |     | 2008 | 2009 | 2010 |
+  # | ----- | --- | ---- | ---- | ---- |
+  # | SP    |  10 |    5 |    5 |    3 |
+  # | RP    |  20 |   10 |   10 |    5 |
+  # | Total |  30 |   15 |   15 |   10 |
+  # | SP-O  | 100 |   10 |   10 |   70 |
+  # | RP-O  |  40 |   20 |   20 |   10 |
+  # | O     | 140 |   10 |   30 |  100 |
+  #
+  class Transposer
+
+    include Dsl
+
+    # infile contains the data that is operated on
+    attr_reader :infile
+    # outfile is the file where the result is written to
+    attr_reader :outfile
+    # filter that is used for rows
+    attr_reader :row_filter
+    # filter that is used for columns
+    attr_reader :col_filter
+ 
+    # Create a new Transpose
+    # :call-seq:
+    #   Sycsvpro::Transpose(infile:  "infile.csv",
+    #                       outfile: "outfile.csv",
+    #                       rows:    "0,3-5",
+    #                       cols:    "1,3").execute
+    def initialize(options = {})
+      @infile  = options[:infile]
+      @outfile = options[:outfile]
+      @row_filter = RowFilter.new(options[:rows])
+      @col_filter = ColumnFilter.new(options[:cols])
+    end
+
+    # Executes the transpose by reading the infile and writing the result to 
+    # the outfile
+    def execute
+      transpose = {}
+
+      File.open(@infile).each_with_index do |line, index|
+        line = unstring(line)
+        next if line.empty?
+
+        result = @col_filter.process(@row_filter.process(line, row: index))
+        next if result.nil?
+
+        result.split(';').each_with_index do |col, index|
+          transpose[index] ||= []
+          transpose[index] << col
+        end
+      end
+
+      File.open(@outfile, 'w') do |out|
+        transpose.values.each { |value| out.puts value.join(';') }
+      end
+    end
+
+  end
+
+end

data/lib/sycsvpro/version.rb

@@ -1,5 +1,5 @@
 # Operating csv files
 module Sycsvpro
   # Version number of sycsvpro
-  VERSION = '0.1.12'
+  VERSION = '0.1.13'
 end

data/lib/sycsvpro.rb

@@ -17,3 +17,4 @@ require 'sycsvpro/table.rb'
 require 'sycsvpro/join.rb'
 require 'sycsvpro/merger.rb'
 require 'sycsvpro/unique.rb'
+require 'sycsvpro/transposer.rb'

data/spec/sycsvpro/calculator_spec.rb

@@ -12,6 +12,96 @@ module Sycsvpro
       @out_file = File.join(File.dirname(__FILE__), "files/machines_out.csv")
     end
 
+    it "should ignore colons within calculation expression" do
+      cols   = "3:+[c1,c2].inject(:+),4:c2*3"
+      header = "*,times"
+      
+      calculator = Calculator.new(infile: @in_number_file,
+                                  outfile: @out_file,
+                                  header:  header,
+                                  cols:    cols)
+
+      calculator.execute
+
+      result = [ "customer;before;between;after;times",
+                 "Fink;2;3;6;9",
+                 "Haas;3;1;10;3",
+                 "Gent;4;4;12;12",
+                 "Rank;5;4;10;12" ]
+
+      rows = 0
+
+      File.open(@out_file).each_with_index do |line, index|
+        line.chomp.should eq result[index]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
+    it "should save only specified columns" do
+      cols   = "3:+[c1,c2].inject(:+),4:c3*3"
+      write  = "0,3-4"
+      header = "customer;sum;times"
+      
+      calculator = Calculator.new(infile:       @in_number_file,
+                                  outfile:      @out_file,
+                                  header:       header,
+                                  final_header: true,
+                                  write:        write,
+                                  cols:         cols,
+                                  sum:          true)
+
+      calculator.execute
+
+      result = [ "customer;sum;times",
+                 "Fink;6;18",
+                 "Haas;10;30",
+                 "Gent;12;36",
+                 "Rank;10;30",
+                 "0;38;114" ]
+
+      rows = 0
+
+      File.open(@out_file).each_with_index do |line, index|
+        line.chomp.should eq result[index]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
+    it "should save only specified columns" do
+      cols   = "3:+[c1,c2].inject(:+),4:c3*3"
+      write  = "0,3-4"
+      header = "*,times"
+      
+      calculator = Calculator.new(infile: @in_number_file,
+                                  outfile: @out_file,
+                                  header:  header,
+                                  write:   write,
+                                  cols:    cols,
+                                  sum:     true)
+
+      calculator.execute
+
+      result = [ "customer;after;times",
+                 "Fink;6;18",
+                 "Haas;10;30",
+                 "Gent;12;36",
+                 "Rank;10;30",
+                 "0;38;114" ]
+
+      rows = 0
+
+      File.open(@out_file).each_with_index do |line, index|
+        line.chomp.should eq result[index]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
     it "should operate on existing row" do
       rows = "2-8"
       cols = "3:*3,4:*4+1"

data/spec/sycsvpro/mapper_spec.rb

@@ -6,12 +6,16 @@ module Sycsvpro
 
     before do
       @in_file  = File.join(File.dirname(__FILE__), "files/in.csv")
+      @in_file5 = File.join(File.dirname(__FILE__), "files/in5.csv")
       @out_file = File.join(File.dirname(__FILE__), "files/out.csv")
       @mappings = File.join(File.dirname(__FILE__), "files/mappings")
     end
 
-    it "should map values to new values" do
-      mapper = Mapper.new(infile: @in_file, outfile: @out_file, mapping: @mappings)
+    it "should map values to new values in all columns" do
+      mapper = Mapper.new(infile:  @in_file, 
+                          outfile: @out_file, 
+                          rows:    "0-7",
+                          mapping: @mappings)
 
       mapper.execute
 
@@ -30,6 +34,60 @@ module Sycsvpro
 
     end
 
+    it "should map values to new values on specified columns only" do
+      mapper = Mapper.new(infile:  @in_file,
+                          outfile: @out_file,
+                          rows:    "0-7",
+                          cols:    "4",
+                          mapping: @mappings).execute
+
+      result = [ "customer;contract-number;expires-on;machine;product1;product2",
+                 "Fink;1234;20.12.2015;f1;control123;dri222",
+                 "Haas;3322;1.10.2011;h1;control332;dri111",
+                 "Gent;4323;1.3.2014;g1;control123;dri111",
+                 "Fink;1234;30.12.2016;f2;control333;dri321",
+                 "Rank;3232;1.5.2013;r1;control332;dri321",
+                 "Klig;4432;;k1;control332;dri222",
+                 "fink;1234;;f3;control332;dri321" ]
+
+      rows = 0
+
+      File.open(@out_file).each_with_index do |line, index|
+        line.chomp.should eq result[index]
+        rows += 1
+      end
+
+      rows.should eq result.size
+
+    end
+
+    it "should map values to new values where last column is empty" do
+      mapper = Mapper.new(infile:  @in_file5,
+                          outfile: @out_file,
+                          cols:    "5",
+                          mapping: @mappings).execute
+
+      result = [ "customer;contract-number;expires-on;machine;product1;product2",
+                 "Fink;1234;20.12.2015;f1;con123;drive222",
+                 "Haas;3322;1.10.2011;h1;con332;drive111",
+                 "Gent;4323;1.3.2014;g1;con123;drive111",
+                 "Fink;1234;30.12.2016;f2;con333;drive321",
+                 "Rank;3232;1.5.2013;r1;con332;drive321",
+                 "Klig;4432;;k1;con332;drive222",
+                 "fink;1234;;f3;con332;drive321",
+                 "zink;8839;8.8.2018;z3;con332;" ]
+
+      rows = 0
+
+      File.open(@out_file).each_with_index do |line, index|
+        line.chomp.should eq result[index]
+        rows += 1
+      end
+
+      rows.should eq result.size
+
+    end
+
   end
 
 end

data/spec/sycsvpro/merger_spec.rb

@@ -7,6 +7,8 @@ module Sycsvpro
     before do
       @file1   = File.join(File.dirname(__FILE__), "files/merge1.csv")
       @file2   = File.join(File.dirname(__FILE__), "files/merge2.csv")
+      @file3   = File.join(File.dirname(__FILE__), "files/merge3.csv")
+      @file4   = File.join(File.dirname(__FILE__), "files/merge4.csv")
       @outfile = File.join(File.dirname(__FILE__), "files/merged.csv")
     end
 
@@ -100,6 +102,97 @@ module Sycsvpro
       rows.should eq result.size
     end
 
+    it "should merge two files without key columns" do
+      header = "2010,2011,2012,2014"
+      source_header = "(\\d{4}),(\\d{4})"
+
+      Sycsvpro::Merger.new(outfile:       @outfile,
+                           files:         "#{@file4},#{@file3}",
+                           header:        header,
+                           source_header: source_header).execute
+
+      result = [ "2010;2011;2012;2014",
+                 "20;30;40;60",
+                 "30;40;50;70",
+                 "40;50;60;80",
+                 "50;60;70;90",
+                 "m1;m2;m3",
+                 "n1;n2;n3",
+                 "o1;;o3", ]
+
+      rows = 0
+
+      File.open(@outfile).each_with_index do |row, index|
+        row.chomp.should eq result[index]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
+    it "should merge two files key columns in one file only" do
+      header = "2010,2011,2012,2014"
+      key = "0"
+      source_header = "(\\d{4}),(\\d{4})"
+
+      Sycsvpro::Merger.new(outfile:       @outfile,
+                           files:         "#{@file1},#{@file3}",
+                           header:        header,
+                           key:           key,
+                           source_header: source_header).execute
+
+      result = [ ";2010;2011;2012;2014",
+                 "SP;20;30;40;60",
+                 "RP;30;40;50;70",
+                 "MP;40;50;60;80",
+                 "NP;50;60;70;90",
+                 ";m1;m2;m3",
+                 ";n1;n2;n3",
+                 ";o1;;o3", ]
+
+      rows = 0
+
+      File.open(@outfile).each_with_index do |row, index|
+        row.chomp.should eq result[index]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
+    it "should merge two files key columns in two files of three only" do
+      header = "2010,2011,2012,2014"
+      key = "0, ,0"
+      source_header = "(\\d{4}),(\\d{4}),(\\d{4})"
+
+      Sycsvpro::Merger.new(outfile:       @outfile,
+                           files:         "#{@file1},#{@file3},#{@file2}",
+                           header:        header,
+                           key:           key,
+                           source_header: source_header).execute
+
+      result = [ ";2010;2011;2012;2014",
+                 "SP;20;30;40;60",
+                 "RP;30;40;50;70",
+                 "MP;40;50;60;80",
+                 "NP;50;60;70;90",
+                 ";m1;m2;m3",
+                 ";n1;n2;n3",
+                 ";o1;;o3",
+                 "M;m1;m2;m3",
+                 "N;n1;n2;n3",
+                 "O;o1;;o3" ]
+
+      rows = 0
+
+      File.open(@outfile).each_with_index do |row, index|
+        row.chomp.should eq result[index]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
   end
 
 end

data/spec/sycsvpro/transposer_spec.rb

@@ -0,0 +1,76 @@
+require 'sycsvpro/transposer'
+
+module Sycsvpro
+
+  describe Transposer do
+
+    before do
+      @infile  = File.join(File.dirname(__FILE__), 'files/in6.csv')
+      @outfile = File.join(File.dirname(__FILE__), 'files/out.csv')
+    end
+
+    it "should transpose (change rows to columns) complete file" do
+      Sycsvpro::Transposer.new(infile:  @infile,
+                               outfile: @outfile).execute
+
+      result = [ "Year;;2008;2009;2010",
+                 "SP;10;5;2;3",
+                 "RP;20;10;5;5",
+                 "Total;30;15;5;10",
+                 "SP-O;100;10;20;70",
+                 "RP-O;40;20;10;10",
+                 "O;140;10;30;100" ]
+
+      rows = 0
+
+      File.open(@outfile).each_with_index do |line, i|
+        line.chomp.should eq result[i]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
+    it "should transpose selected columns" do
+      Sycsvpro::Transposer.new(infile:  @infile,
+                               outfile: @outfile,
+                               cols:    "0-2").execute
+
+      result = [ "Year;;2008;2009;2010",
+                 "SP;10;5;2;3",
+                 "RP;20;10;5;5" ]
+
+      rows = 0
+
+      File.open(@outfile).each_with_index do |line, i|
+        line.chomp.should eq result[i]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
+    it "should transpose selected rows and columns" do
+      Sycsvpro::Transposer.new(infile:  @infile,
+                               outfile: @outfile,
+                               rows:    "0,2-4",
+                               cols:    "0-2").execute
+
+      result = [ "Year;2008;2009;2010",
+                 "SP;5;2;3",
+                 "RP;10;5;5" ]
+
+      rows = 0
+
+      File.open(@outfile).each_with_index do |line, i|
+        line.chomp.should eq result[i]
+        rows += 1
+      end
+
+      rows.should eq result.size
+    end
+
+
+  end
+
+end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sycsvpro
 version: !ruby/object:Gem::Version
-  version: 0.1.12
+  version: 0.1.13
   prerelease: 
 platform: ruby
 authors:
@@ -9,7 +9,7 @@ authors:
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2014-07-09 00:00:00.000000000 Z
+date: 2014-07-14 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -151,6 +151,7 @@ files:
 - lib/sycsvpro/script_list.rb
 - lib/sycsvpro/sorter.rb
 - lib/sycsvpro/table.rb
+- lib/sycsvpro/transposer.rb
 - lib/sycsvpro/unique.rb
 - lib/sycsvpro/version.rb
 - spec/sycsvpro/aggregator_spec.rb
@@ -175,6 +176,7 @@ files:
 - spec/sycsvpro/script_list_spec.rb
 - spec/sycsvpro/sorter_spec.rb
 - spec/sycsvpro/table_spec.rb
+- spec/sycsvpro/transposer_spec.rb
 - spec/sycsvpro/unique_spec.rb
 - sycsvpro.gemspec
 - sycsvpro.rdoc