sycsvpro 0.1.4 → 0.1.7

data/Gemfile.lock

@@ -1,7 +1,7 @@
 PATH
   remote: .
   specs:
-    sycsvpro (0.1.4)
+    sycsvpro (0.1.7)
       gli (= 2.9.0)
       timeleap (~> 0.0.1)
 

data/README.md

@@ -16,21 +16,32 @@ Processing of csv files. *sycsvpro* offers following functions
 * create or edit a Ruby script
 * list scripts available optionally with methods (since version 0.0.7)
 * execute a Ruby script file that operates a csv file
-* create a table from a source file with dynamically create columns (since version 0.1.4)
+* create a table from a source file with dynamically create columns (since 
+  version 0.1.4)
+* join two file based on a joint column value (since version 0.1.7)
 
 To get help type
 
     $ sycsvpro -h
     
-In the following examples we assume the following file
+In the following examples we assume the following files 'machines.csv' and
+'region.csv'
 
 ```
-customer;machine;control;drive;motor;date;contract
-hello;h1;con123;dri120;mot100;1.01.3013;1
-hello;h2;con123;dri130;mot110;1.02.3012;1
-indix;i1;con456;dri130;mot090;5.11.3013;1
-chiro;c1;con333;dri110;mot100;1.10.3011;1
-chiro;c2;con331;dri100;mot130;3.05.3010;1
+customer;machine;control;drive;motor;date;contract;price;c-id
+hello;h1;con123;dri120;mot100;1.01.3013;1;2.5;123
+hello;h2;con123;dri130;mot110;1.02.3012;1;12.1;123
+indix;i1;con456;dri130;mot090;5.11.3013;1;23.24;345
+chiro;c1;con333;dri110;mot100;1.10.3011;1;122.15;456
+chiro;c2;con331;dri100;mot130;3.05.3010;1;25.3;456
+```
+
+```
+region;country;c-id
+R1;DE,123
+R2;AT;234
+R3;US;345
+R4;CA;456
 ```
 
 Analyze
@@ -114,7 +125,8 @@ Count all customers (key column) in rows 2 to 20 that have machines that start
 with *h* and have a contract valid beginning after 1.1.2000. Add a sum row with
 title Total at column 1
 
-    $ sycsvpro -f in.csv -o out.csv count -r 2-20 -k 0:customer -c 1:/^h/,5:">1.1.2000" --df "%d.%m.%Y" -s "Total:1"
+    $ sycsvpro -f in.csv -o out.csv count -r 2-20 -k 0:customer 
+               -c 1:/^h/,5:">1.1.2000" --df "%d.%m.%Y" -s "Total:1"
 
 The result in file out.csv is
 
@@ -143,12 +155,30 @@ The aggregation result in out.csv is
     indix;1
     chiro;2
 
+Table
+-----
+Analyze the contract revenue per customer and per year
+
+    $ sycsvpro -f in.csv -o out.csv table 
+               -h "Customer,c5=~/\\.(\\d{4})/"
+               -k c1
+               -c "c5=~/\\.\\d{4})/:+n1" 
+
+The table result will be in out.csv
+
+    $ cat out.csv
+      Customer;3013;3012;3011;3010
+      hello;2.5;12.1;0;0
+      indix;23.24;0;0;0
+      chiro;0;0;122.15;25.3
+
 Calc
 ----
 Process arithmetic operations on the contract count and create a target column 
 and a sum which is added at the end of the result file
 
-    $ sycsvpro -f in.csv -o out.csv calc -r 2-20 -h *,target -c 6:*2,7:target=c6*10
+    $ sycsvpro -f in.csv -o out.csv calc -r 2-20 -h *,target 
+               -c 6:*2,7:target=c6*10
 
     $ cat out.csv
     customer;machine;control;drive;motor;date;contract;target
@@ -162,6 +192,26 @@ and a sum which is added at the end of the result file
 In the sum row non-numbers in the colums are converted to 0. Therefore column 0
 is summed up to 0 as all strings are converted to 0.
 
+Join
+----
+Join the machine and contract file with columns from the customer address file
+
+    $ sycsvpro -f in.csv -o out.csv join address.csv -c 0,1 
+                                                     -p 2,1
+                                                     -i "COUNTRY,REGION"
+                                                     -j "3=8"
+
+This will create the result
+
+```
+customer;COUNTRY;REGION;machine;control;drive;motor;date;contract;price;c-id
+hello;DE;R1;h1;con123;dri120;mot100;1.01.3013;1;2.5;123
+hello;DE;R1;h2;con123;dri130;mot110;1.02.3012;1;12.1;123
+indix;US;R3i1;con456;dri130;mot090;5.11.3013;1;23.24;345
+chiro;CA;R4;c1;con333;dri110;mot100;1.10.3011;1;122.15;456
+chiro;CA;R4;c2;con331;dri100;mot130;3.05.3010;1;25.3;456
+```
+
 Sort
 ----
 Sort rows on specified columns as an example sort rows based on customer 
@@ -198,7 +248,7 @@ the name script.rb and a method call_me
 List
 ----
 List the scripts, insert-file or all scripts available in the scripts directory
-which is also displayed
+which is also displayed. Comments before methods are also displayed
 
     script directory: ~/.syc/sycsvpro/scripts
     $ sycsvpro list -m
@@ -257,6 +307,9 @@ end with _column_ or _columns_ dependent if a value or an array should be
 returned. You can find the *rows* and *write_to* methods at 
 _lib/sycsvpro/dsl.rb_.
 
+Examples for scripts using sycsvpro can be found at 
+[sugaryourcoffee/sycsvpro-scripts](https://github.com/sugaryourcoffee/sycsvpro-scripts)
+
 Working with sycsvpro
 =====================
 
@@ -316,19 +369,58 @@ Version 0.1.4
     * Associate values to multi keys
     * Create values based on arithmetic operations of source table data
   Example
-    `sycsvpro -f in.csv -o out.csv table -h "c4,c5,c0=~/\\.(\\d{4})/" \
-                                         -k "c4,c5" \
-                                         -c "c0=~/\\.(\\d{4})/:+n1"`
-  h:: the header is created from the source table header of column 4 and 5.
-      Another header column is created dynamicall based on the year part of a
-      date in column 0
-  k:: the key is based on source table of column 4 and 5
-  c:: the column operation is in the form HeaderName:Operation. In this case the
-      HeaderName is dynamically determined based on column 0 and added the value
-      of column 1 to this column that is associated to the key
+      `sycsvpro -f in.csv -o out.csv table -h "c4,c5,c0=~/\\.(\\d{4})/" 
+                                           -k "c4,c5" 
+                                           -c "c0=~/\\.(\\d{4})/:+n1"`
+  + h the header is created from the source table header of column 4 and 5.
+      Another header column is created dynamicall based on the year part of 
+      a date in column 0
+  + k the key is based on source table of column 4 and 5
+  + c the column operation is in the form HeaderName:Operation. In this case
+      the HeaderName is dynamically determined based on column 0 and added
+      the value of column 1 to this column that is associated to the key
 
   c4, n4, d4 are string, number and date values respectively
 
+Version 0.1.5
+-------------
+* Add a sum row after the heading or at the end of file like so
+      `sycsvpro -f in.csv -o out.csv table -h   "c4,c5,c0=~/\\.(\\d{4})/" 
+                                           -k   "c4,c5" 
+                                           -c   "c0=~/\\.(\\d{4})/:+n1"
+                                           -s   "c0=~/\\.(\\d{4})/"`
+  This will sum up the dynamically created column.
+
+Version 0.1.6
+-------------
+* Commas within columns expression are now ignored while splitting columns of
+  table columns
+* Table takes a number format now with `--nf DE` which will convert numbers
+  from DE locale like 1.000,00 to 1000.00
+* Table uses a precision for numbers. Default is 2. Can be assigned with `pr: 2`
+
+Version 0.1.7
+-------------
+* Calc can now be used not to only do arithmetic operations on columns but also
+  string operations. Ultimately any valid Ruby command can be used to process a
+  column value
+      `sycsvpro -f customer.csv -o customer-number.csv calc 
+                            -h "Customer_ID,Customer,Country" 
+                            -r "1-eof" 
+                            -c "2:s0.scan(/^([A-Z]+)\\//).flatten[0],
+                                0:s0.scan(/(?<=\\/)(.*)$/).flatten[0],1:s1"
+* Join is a new class that joins to tables based on a joint column value
+      `sycsvpro -f infile.csv -o outfile.csv join source.csv -c "2,4"
+                                                             -j "1=3"
+                                                             -p "1,3"
+                                                             -h "*"
+                                                             -i "A,B"`
+  This will join infile.csv with source.csv based on the join columns (j "1=3").
+  From source.csv columns 2 and 4 (-c "2,4") will be inserted at column
+  positions 1 and 3 (-p "1,3"). The header will be used from the infile.csv
+  (-h "*") supplemented by the columns A and B (-i "A,B") that will also be
+  positioned at column 1 and 3 (-p "1,3").
+
 Installation
 ============
 [![Gem Version](https://badge.fury.io/rb/sycsvpro.png)](http://badge.fury.io/rb/sycsvpro)

data/bin/sycsvpro

@@ -89,12 +89,13 @@ command :extract do |c|
   end
 end
 
-desc 'Collect values of specified rows and columns from the file and group them in categories'
+desc 'Collect values of specified rows and columns from the file and group '+
+     'them in categories'
 command :collect do |c|
 
   c.desc 'Rows to consider for collection'
   c.arg_name 'ROW1,ROW2,ROW10-ROW30,45-EOF,REGEXP'
-  c.flag [:r, :row], :must_match => row_regex #/\d+(?:,\d+|-\d+|-eof|,\/.*\/)*|\/.*\/(?:,\/.*\/|\d+)*/i
+  c.flag [:r, :row], :must_match => row_regex
 
   c.desc 'Columns to collect values from'
   c.arg_name 'CATEGORY1:COL1,COL2,COL10-COL30+CATEGORY2:COL3-COL9'
@@ -120,7 +121,7 @@ desc 'Allocate specified columns from the file to a key value'
 command :allocate do |c|
   c.desc 'Rows to consider'
   c.arg_name '1,2,10-30,45-EOF,REGEXP'
-  c.flag [:r, :row], :must_match => row_regex #/\d+(?:,\d+|-\d+|-eof|,\/.*\/)*|\/.*\/(?:,\/.*\/|\d+)*/i
+  c.flag [:r, :row], :must_match => row_regex 
 
   c.desc 'Key to allocate columns to'
   c.arg_name '0'
@@ -147,7 +148,8 @@ command :allocate do |c|
   end
 end
 
-desc 'Creates a script/insert file or opens a script/insert file for editing if it exists'
+desc 'Creates a script/insert file or opens a script/insert file for editing '+
+     'if it exists'
 command :edit do |c|
   c.desc 'Name of the script/insert file'
   c.arg_name 'SCRIPT_NAME.rb|INSERT_NAME.ins'
@@ -159,12 +161,14 @@ command :edit do |c|
 
   c.action do |global_options,options,args|
     script_creator = Sycsvpro::ScriptCreator.new(dir: sycsvpro_directory, 
-                                                 script: options[:s], method: options[:m])  
+                                                 script: options[:s], 
+                                                 method: options[:m])  
     system "vi #{script_creator.script_file}"
   end
 end
 
-desc 'Lists script or insert files in the scripts directory with optionally listing methods of script files'
+desc 'Lists script or insert files in the scripts directory with optionally '+
+     'listing methods of script files'
 command :list do |c|
   c.desc 'Type of script (Ruby, insert or all files)'
   c.default_value 'script'
@@ -235,7 +239,7 @@ command :count do |c|
 
   c.desc 'Rows to consider'
   c.arg_name '1,2,10-30,45-EOF,REGEXP'
-  c.flag [:r, :row], :must_match => row_regex #/\d+(?:,\d+|-\d+|-eof|,\/.*\/)*|\/.*\/(?:,\/.*\/|\d+)*/i
+  c.flag [:r, :row], :must_match => row_regex
 
   c.desc 'Columns to count where columns 2 and 3 are counted conditionally'
   c.arg_name '1,2:<14.2.2014,10-30,3:>10'
@@ -274,7 +278,7 @@ command :aggregate do |c|
 
   c.desc 'Rows to consider'
   c.arg_name '1,2,10-30,45-EOF,REGEXP'
-  c.flag [:r, :row], :must_match => row_regex #/\d+(?:,\d+|-\d+|-eof|,\/.*\/)*|\/.*\/(?:,\/.*\/|\d+)*/i
+  c.flag [:r, :row], :must_match => row_regex
 
   c.desc 'Columns to count'
   c.arg_name '1,2-4'
@@ -311,18 +315,27 @@ command :table do |c|
   c.arg_name '1,2,10-30,45-EOF,REGEXP'
   c.flag [:r, :row], :must_match => row_regex
 
-  c.desc 'Header can be defined by Words (Year), references to source header (c1) and dynamically created header values (c1+c2,c0=~/\\.(\\d{4})/)'
+  c.desc 'Header can be defined by Words (Year), references to source header '+
+         '(c1) and dynamically created header values (c1+c2,c0=~/\\.(\\d{4})/)'
   c.arg_name "COL_A,c6,c2+c4,c0=~/\\.(\\d{4})/"
   c.flag [:h, :header]
 
-  c.desc 'Key to that the other columns are associated to. A key can be created dynamically'
+  c.desc 'Key to that the other columns are associated to. A key can be '+
+         'created dynamically'
   c.arg_name "c0=~/\\.(\\d{4})/,c6"
   c.flag [:k, :key]
 
-  c.desc 'Columns to be associated to the key. Columns are identified by the column name. The operation to create the column value is separated by a colon (:) from the column name'
+  c.desc 'Columns to be associated to the key. Columns are identified by the '+
+         'column name. The operation to create the column value is separated '+
+         'by a colon (:) from the column name'
   c.arg_name "c0=~/\\.(\\d{4})/:+n1,Value:+n2"
   c.flag [:c, :col]
 
+  c.desc 'Adds a sum row after the heading or at the end of the file for col '+
+         'values'
+  c.arg_name "TOP|EOF:c0=~/\\.(\\d{4})/,Value"
+  c.flag [:s, :sum]
+
   c.desc 'Format of date values'
   c.arg_name '%d.%m.%Y|%m/%d/%Y|...'
   c.flag [:df]
@@ -341,20 +354,76 @@ command :table do |c|
                                 rows:    options[:r], 
                                 header:  options[:h],
                                 key:     options[:k],
-                                cols:    options[:c]) 
+                                cols:    options[:c],
+                                sum:     options[:s]) 
     table.execute
     puts "done"
   end
 
 end
 
+desc 'Join two files based on a joint column value'
+arg_name 'SOURCE_FILE'
+command :join do |c|
+  c.desc 'Rows to consider'
+  c.arg_name '1,2,10-30,45-EOF,REGEXP'
+  c.flag [:r, :row], :must_match => row_regex 
+  
+  c.desc 'Columns to merge into the infile'
+  c.arg_name '1,5,7'
+  c.flag [:c, :cols], :must_match => /^\d+(?:,\d+)*/
+
+  c.desc 'The position at which column position to insert the columns within '+
+         'the infile. The sequence of the position is assigned to the columns '+
+         'to be inserted'
+  c.arg_name '5,1'
+  c.flag [:p, :pos], :must_match => /^\d+(?:,\d+)*/
+
+  c.desc 'The join columns in the source file, which contains the columns to '+
+         'be inserted into the infile'
+  c.arg_name '2=1'
+  c.flag [:j, :join], :must_match => /^\d+=\d+$/
+
+  c.desc 'Indicates whether the infile headerless'
+  c.default_value false
+  c.switch [:headerless]
+
+  c.desc 'Header columns of the infile'
+  c.arg_name '*,COL1,COL2'
+  c.default_value '*'
+  c.flag [:h, :header]
+
+  c.desc 'Header columns to be used for the inserted columns from the source '+
+         'file. The position (-p 5,1) determines where to insert the header '+
+         'columns'
+  c.arg_name 'INS_COL1,INS_COL2'
+  c.flag [:i, :insert]
+
+  c.action do |global_options,options,args|
+    join = Sycsvpro::Join.new(infile:        global_options[:f], 
+                              outfile:       global_options[:o],
+                              source:        args[0],
+                              rows:          options[:r], 
+                              cols:          options[:c], 
+                              pos:           options[:p],
+                              joins:         options[:j],
+                              headerless:    options[:headerless], 
+                              header:        options[:h],
+                              insert_header: options[:i])
+    print 'Joining...'
+    join.execute
+    print 'done'
+  end
+end
+
 desc 'Sort rows based on column values'
 command :sort do |c|
   c.desc 'Rows to consider'
   c.arg_name '1,2,10-30,45-EOF,REGEXP'
-  c.flag [:r, :row], :must_match => row_regex #/\d+(?:,\d+|-\d+|-eof|,\/.*\/)*|\/.*\/(?:,\/.*\/|\d+)*/i
+  c.flag [:r, :row], :must_match => row_regex 
 
-  c.desc 'Columns to sort based on a type (n = number, s = string, d = date) and its value'
+  c.desc 'Columns to sort based on a type (n = number, s = string, d = date) '+
+         'and its value'
   c.arg_name 'n:1,s:2-5,d:7'
   c.flag [:c, :col], :must_match => /[d|n|s]:\d+(?:-\d+|,[d|n|s]:\d+)*/
 
@@ -443,29 +512,33 @@ command :map do |c|
   end
 end
 
-desc 'Process math operations on columns. Optionally add a sum row'
+desc 'Process operations on columns. Optionally add a sum row for columns with'+
+     'number values'
 command :calc do |c|
     c.desc 'The first non-empty column is considered the header. '+
-           'If additional columns are created then *,COL1,COL2 will create the additional header '+
-           'columns COL1 and COL2'
-    c.arg_name '*,COL2,COL2'
+           'If additional columns are created then *,COL1,COL2 will create '+
+           'the additional header columns COL1 and COL2. It is also possible '+
+           'to specify different header columns like COL1,COL2,COL3'
+    c.arg_name '*,COL2,COL2|COL1,COL2,COL3'
     default_value '*'
-    c.flag [:h, :header], :must_match => /\*(?:,\w+)*/
+    c.flag [:h, :header], :must_match => /^[*|\w ]+(?:,[\w ]+)*/
 
     c.desc 'Rows to consider for calculations'
     c.arg_name 'ROW1,ROW2-ROW10,45-EOF,REGEXP'
-    c.flag [:r, :row], :must_match => row_regex #/\d+(?:,\d+|-\d+|-eof|,\/.*\/)*|\/.*\/(?:,\/.*\/|\d+)*/i
+    c.flag [:r, :row], :must_match => row_regex
 
-    c.desc 'Column to do calculations on'
-    c.arg_name 'COL1:*2,COL2:-C3,COL3:*2+(4+C5),COL6:NEW_COL=C1+5'
-    #c.flag [:c, :col], :must_match => /\d+:(?:[\*\/\+\-]|\w+=[\d|(]*)[\*\/\+\-\dc()]*(?:,\d+:(?:[\*\/\+\-]|\w+=[\d|(]*)[\*\/\+\-\dc()]*)*/
-    c.flag [:c, :col], :must_match => /\d+:(?:[\*\+\-\/\d\w=\[\],\.:()]*)/
+    c.desc 'Column to do operations on. s0 = String in column 0, c1 = number '+
+           'in column 1 and d2 = date in column 2. Examples: 2:c1+1,3:s0,'+
+           '4:s0.scan(/(\\d+)\//).flatten[0]'
+    c.arg_name "COL1:*2,COL2:-C3,COL3:*2+(4+C5)"
+    c.flag [:c, :col], :must_match => /^\d+:.+/
 
     c.desc 'Date format of date columns'
     c.arg_name '%d.%m.%Y|%Y-%m-%d|...'
     c.flag [:df]
 
-    c.desc 'Indicate to add a sum row'
+    c.desc 'Indicate to add a sum row at end of file. Will sum up values with '+
+           'numbers. Columns with non-number values will be set to 0.'
     c.switch [:s, :sum]
 
   c.action do |global_options,options,args|

data/lib/sycsvpro/calculator.rb

@@ -6,9 +6,42 @@ require 'date'
 # Operating csv files
 module Sycsvpro
 
-  # Processes arithmetic operations on columns of a csv file. A column value has to be a number.
-  # Possible operations are +, -, * and /. It is also possible to use values of columns as an
-  # operator like c1*2 will multiply the value of column 1 with 2.
+  # Processes operations on columns of a csv file. 
+  #
+  # A column value has to be a number in case of arithmetical operations. 
+  #
+  # Possible operations are +, -, *, /, % and **. 
+  #
+  # It is possible to use values of columns as an operator like multiply 
+  # column 1 of the csv file with 2 and assign it to column 4 of the result 
+  # file: c1*2
+  #
+  # Other values might be dates or strings.
+  #
+  # d1:: date value in column 1
+  # s2:: string value in column 2
+  # c3:: number value in column 3
+  #
+  # To assign a string from column 1 of the csv file to column 3 of the 
+  # resulting file you can do like so: 3:s1
+  #
+  # You can also use Ruby expressions to assign values: 0:[d1,d2,d3].min - This
+  # will assign the least date value from columns 1, 2 and 3 to column 0.
+  #
+  # Note: If you assign a value to column 1 and subsequently are using column 1
+  # in other assignments then column 1 will have the result of a previous
+  # operation.
+  #
+  # Example:
+  # Having a row "CA/123456" and you want to have 123456 in column 0
+  # of the resulting csv file and CA in column 2. If you conduct following
+  # operations it will fail
+  #     1:s0.scan(/\/(.+)/).flatten[0]   -> 123456 
+  #     2:s0.scan(/([A-Z]+)/).flatten[0] -> nil
+  # To achieve the required result you have to change the operational sequence
+  # like so
+  #     2:s0.scan(/([A-Z]+)/).flatten[0] -> CA
+  #     1.so.scan(/\/(.+)/).flatten[0]   -> 123456
   class Calculator
 
     include Dsl
@@ -30,18 +63,24 @@ module Sycsvpro
     # if true add a sum row at the bottom of the out file
     attr_reader :add_sum_row
 
-    # Creates a new Calculator. Options expects :infile, :outfile, :rows and 
-    # :columns. Optionally a header can be provided. The header can be 
-    # supplemented with additional column names that are generated due to a 
-    # arithmetic operation that creates new columns
+    # Creates a new Calculator. Optionally a header can be provided. The header 
+    # can be supplemented with additional column names that are generated due 
+    # to an arithmetic operation that creates new columns
     # :call-seq:
     #   Sycsvpro::Calculator.new(infile:  "in.csv",
     #                            outfile: "out.csv",
     #                            df:      "%d.%m.%Y",
     #                            rows:    "1,2,BEGINn3>20END",
     #                            header:  "*,Count",
-    #                            cols:    "4:Count=c1+c2*2",
+    #                            cols:    "4:c1+c2*2",
     #                            sum:     true).execute
+    # infile:: File that contains the rows to be operated on
+    # outfile:: Result of the operations
+    # df:: Date format
+    # rows:: Row filter that indicates which rows to consider
+    # header:: Header of the columns
+    # cols:: Operations on the column values
+    # sum:: Indicate whether to add a sum row
     def initialize(options={})
       @infile      = options[:infile]
       @outfile     = options[:outfile]
@@ -59,6 +98,7 @@ module Sycsvpro
     def method_missing(id, *args, &block)
       return to_number(columns[$1.to_i]) if id =~ /c(\d+)/
       return to_date(columns[$1.to_i])   if id =~ /d(\d+)/
+      return columns[$1.to_i]            if id =~ /s(\d+)/
       super
     end
 
@@ -68,7 +108,7 @@ module Sycsvpro
 
       File.open(outfile, 'w') do |out|
         File.open(infile).each_with_index do |line, index|
-          next if line.chomp.empty?
+          next if line.chomp.empty? || unstring(line).chomp.split(';').empty?
 
           unless processed_header
             header_row = header.process(line.chomp)
@@ -115,7 +155,7 @@ module Sycsvpro
       def create_calculator(code)
         code.split(/,(?=\d+:)/).each do |operation|
           col, term = operation.split(':')
-          term = "c#{col}#{term}" unless term =~ /^c\d+|^\[/
+          term = "c#{col}#{term}" if term =~ /^[+\-*\/%]/
           formulae[col] = term
         end
       end

data/lib/sycsvpro/dsl.rb

@@ -2,6 +2,12 @@ require_relative 'row_filter'
 
 # Methods to be used in customer specific script files
 module Dsl
+  
+  # Splits comma separated strings that contain commas within the value. Such
+  # values have to be enclosed between BEGIN and END
+  # Example:
+  #     Year,c1+c2,c1=~/[A-Z]{1,2}/,Month
+  COMMA_SPLITTER_REGEX = /(?<=,|^)(BEGIN.*?END|\/.*?\/|.*?)(?=,|$)/i
 
   # read arguments provided at invocation
   # :call-seq:
@@ -85,6 +91,12 @@ module Dsl
     str.encode('UTF-8', 'binary', invalid: :replace, undef: :replace, replace: '')
   end
 
+  # Retrieves the values scanned by a COMMA_SPLITTER_REGEX
+  def split_by_comma_regex(values)
+    values.scan(COMMA_SPLITTER_REGEX).flatten.each.
+      collect { |h| h.gsub(/BEGIN|END/, "") }
+  end
+
   private
 
     # Assigns values to keys that are used in rows and yielded to the block

data/lib/sycsvpro/header.rb

@@ -11,14 +11,16 @@ module Sycsvpro
 
     # Header columns
     attr_reader :header_cols
+    # Columns that will be inserted into the header at the defined positions
+    attr_reader :insert_cols
+    # Positions where to insert the insert_cols
+    attr_reader :positions
 
     # Create a new header
-    def initialize(header)
-      unless header.nil? or header.empty?
-        @header_cols = header.split(',')
-      else
-        @header_cols = []
-      end
+    def initialize(header, options = {})
+      @header_cols = split_by_comma_regex(header || "")
+      @insert_cols = (options[:insert] || "").split(',')
+      @positions   = options[:pos] || []
     end
 
     def method_missing(id, *args, &block)
@@ -28,7 +30,7 @@ module Sycsvpro
 
     # Returns the header
     def process(line, values = true)
-      return "" if @header_cols.empty?
+      return "" if @header_cols.empty? && @insert_cols.empty?
       header_patterns = {}
       @row_cols = unstring(line).split(';')
       if @header_cols[0] == '*'
@@ -52,13 +54,14 @@ module Sycsvpro
           end
         end
       end
+      insert_header_cols
       header_patterns.each { |i,h| @header_cols.insert(i,h) }
       to_s
     end
 
     # Returns @header_cols without pattern
     def clear_header_cols
-      @header_cols.flatten.select { |col| col !~ /^c\d+[=~+]{1,2}/ }
+      @header_cols.select { |col| col !~ /^c\d+[=~+]{1,2}/ }
     end
 
     # Returns the index of the column
@@ -66,11 +69,24 @@ module Sycsvpro
       clear_header_cols.index(value)
     end
 
+    # Returns the value of column number
+    def value_of(column)
+      clear_header_cols[column]
+    end
+
     # Returns the header
     def to_s
       clear_header_cols.join(';')
     end
 
+    private
+
+      def insert_header_cols
+        @header_cols.flatten!
+        positions.sort.each { |p| header_cols.insert(p, "") }
+        positions.each_with_index { |p,i| header_cols[p] = insert_cols[i] }
+      end
+
   end
 
 end