sycsvpro 0.1.2 → 0.1.3
Sign up to get free protection for your applications and to get access to all the features.
- data/Gemfile.lock +3 -1
- data/README.md +21 -8
- data/README.rdoc +3 -2
- data/bin/sycsvpro +17 -6
- data/lib/sycsvpro/aggregator.rb +16 -5
- data/lib/sycsvpro/calculator.rb +19 -5
- data/lib/sycsvpro/counter.rb +17 -7
- data/lib/sycsvpro/dsl.rb +43 -0
- data/lib/sycsvpro/filter.rb +0 -7
- data/lib/sycsvpro/sorter.rb +28 -8
- data/lib/sycsvpro/version.rb +1 -1
- data/spec/sycsvpro/sorter_spec.rb +50 -0
- data/sycsvpro.gemspec +1 -0
- metadata +18 -66
- data/html/Dsl.html +0 -277
- data/html/Object.html +0 -187
- data/html/README_rdoc.html +0 -187
- data/html/Sycsvpro/Aggregator.html +0 -482
- data/html/Sycsvpro/Allocator.html +0 -293
- data/html/Sycsvpro/Analyzer.html +0 -239
- data/html/Sycsvpro/Calculator.html +0 -407
- data/html/Sycsvpro/Collector.html +0 -281
- data/html/Sycsvpro/ColumnFilter.html +0 -170
- data/html/Sycsvpro/ColumnTypeFilter.html +0 -191
- data/html/Sycsvpro/Counter.html +0 -492
- data/html/Sycsvpro/Extractor.html +0 -269
- data/html/Sycsvpro/Filter.html +0 -411
- data/html/Sycsvpro/Header.html +0 -239
- data/html/Sycsvpro/Inserter.html +0 -275
- data/html/Sycsvpro/Mapper.html +0 -288
- data/html/Sycsvpro/Profiler.html +0 -234
- data/html/Sycsvpro/RowFilter.html +0 -175
- data/html/Sycsvpro/ScriptCreator.html +0 -235
- data/html/Sycsvpro/ScriptList.html +0 -281
- data/html/Sycsvpro/Sorter.html +0 -346
- data/html/Sycsvpro.html +0 -155
- data/html/created.rid +0 -24
- data/html/fonts/Lato-Light.ttf +0 -0
- data/html/fonts/Lato-LightItalic.ttf +0 -0
- data/html/fonts/Lato-Regular.ttf +0 -0
- data/html/fonts/Lato-RegularItalic.ttf +0 -0
- data/html/fonts/SourceCodePro-Bold.ttf +0 -0
- data/html/fonts/SourceCodePro-Regular.ttf +0 -0
- data/html/fonts.css +0 -167
- data/html/images/add.png +0 -0
- data/html/images/arrow_up.png +0 -0
- data/html/images/brick.png +0 -0
- data/html/images/brick_link.png +0 -0
- data/html/images/bug.png +0 -0
- data/html/images/bullet_black.png +0 -0
- data/html/images/bullet_toggle_minus.png +0 -0
- data/html/images/bullet_toggle_plus.png +0 -0
- data/html/images/date.png +0 -0
- data/html/images/delete.png +0 -0
- data/html/images/find.png +0 -0
- data/html/images/loadingAnimation.gif +0 -0
- data/html/images/macFFBgHack.png +0 -0
- data/html/images/package.png +0 -0
- data/html/images/page_green.png +0 -0
- data/html/images/page_white_text.png +0 -0
- data/html/images/page_white_width.png +0 -0
- data/html/images/plugin.png +0 -0
- data/html/images/ruby.png +0 -0
- data/html/images/tag_blue.png +0 -0
- data/html/images/tag_green.png +0 -0
- data/html/images/transparent.png +0 -0
- data/html/images/wrench.png +0 -0
- data/html/images/wrench_orange.png +0 -0
- data/html/images/zoom.png +0 -0
- data/html/index.html +0 -225
- data/html/js/darkfish.js +0 -140
- data/html/js/jquery.js +0 -18
- data/html/js/navigation.js +0 -142
- data/html/js/search.js +0 -109
- data/html/js/search_index.js +0 -1
- data/html/js/searcher.js +0 -228
- data/html/rdoc.css +0 -580
- data/html/table_of_contents.html +0 -347
data/Gemfile.lock
CHANGED
@@ -1,8 +1,9 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
sycsvpro (0.1.
|
4
|
+
sycsvpro (0.1.3)
|
5
5
|
gli (= 2.9.0)
|
6
|
+
timeleap (~> 0.0.1)
|
6
7
|
|
7
8
|
GEM
|
8
9
|
remote: https://rubygems.org/
|
@@ -39,6 +40,7 @@ GEM
|
|
39
40
|
rspec-expectations (2.14.4)
|
40
41
|
diff-lcs (>= 1.1.3, < 2.0)
|
41
42
|
rspec-mocks (2.14.4)
|
43
|
+
timeleap (0.0.1)
|
42
44
|
|
43
45
|
PLATFORMS
|
44
46
|
ruby
|
data/README.md
CHANGED
@@ -280,21 +280,34 @@ Release notes
|
|
280
280
|
|
281
281
|
Version 0.1.2
|
282
282
|
-------------
|
283
|
-
* Now it is possible to have , in the filter as non separating values.
|
284
|
-
now define filter like 1-2,4,/[56789]{2,}/,10
|
283
|
+
* Now it is possible to have comma ',' in the filter as non separating values.
|
284
|
+
You can now define a filter like 1-2,4,/[56789]{2,}/,10
|
285
285
|
* Filtering rows on boolean expression based on values contained in columns.
|
286
286
|
The boolean expression has to be enclosed between BEGIN and END
|
287
|
+
|
287
288
|
Example:
|
288
|
-
|
289
|
-
s0 - string in column 0
|
290
|
-
n1 - number in column 1
|
291
|
-
d2 - date in column 2
|
292
|
-
* ``list`` shows the directory of the script file and has the
|
293
|
-
show all scripts, that is _insert files_ and _Ruby files_
|
289
|
+
+ ``-r BEGINs0=='Ruby'&&n1<1||d2==Date.new(2014,6,17)END``
|
290
|
+
+ s0 - string in column 0
|
291
|
+
+ n1 - number in column 1
|
292
|
+
+ d2 - date in column 2
|
293
|
+
* ``list`` shows the directory of the script file (`dir: true`) and has the
|
294
|
+
flag *all* to show all scripts, that is _insert files_ and _Ruby files_
|
294
295
|
* When counting columns with *count* the column headers are sorted
|
295
296
|
alphabetically. No it is possible to set ``sort: false`` to keep the column
|
296
297
|
headers in the sequence they are specified
|
297
298
|
|
299
|
+
Version 0.1.3
|
300
|
+
-------------
|
301
|
+
* In counter `sort: false` didn't work with column filters like `cols: "1,2"`.
|
302
|
+
Now all filters work
|
303
|
+
* Sorter now accepts a start row where to start sorting. Rows before the start
|
304
|
+
row are added on top of the sorted file
|
305
|
+
* `sycsvpro -f infile analyze` now lists the columns with sample data
|
306
|
+
* Add `params` method to *Dsl* that retrieves the params provided in the execute
|
307
|
+
command: `sycsvpro execute script.rb method infile param1 param2`
|
308
|
+
* Add `clean_up` to *Dsl* that takes files to be deleted after the script has
|
309
|
+
run: `clean_up(%w{file1 file2})`
|
310
|
+
|
298
311
|
Installation
|
299
312
|
============
|
300
313
|
[![Gem Version](https://badge.fury.io/rb/sycsvpro.png)](http://badge.fury.io/rb/sycsvpro)
|
data/README.rdoc
CHANGED
@@ -4,7 +4,8 @@ Author:: Pierre Sugar (mailto:pierre@sugaryourcoffee.de)
|
|
4
4
|
Copyright:: Copyright (c) 2014 by Pierre Sugar
|
5
5
|
License:: Distributed uder the MIT license, see LICENSE in the source distro
|
6
6
|
|
7
|
-
The application provides an interface for analyzing, cleaning and operating on
|
7
|
+
The application provides an interface for analyzing, cleaning and operating on
|
8
|
+
csv files
|
8
9
|
|
9
10
|
{<img src="https://badge.fury.io/rb/sycsvpro.png" alt="Gem Version" />}[http://badge.fury.io/rb/sycsvpro]
|
10
11
|
|
@@ -41,5 +42,5 @@ Test files are in
|
|
41
42
|
|
42
43
|
spec/sycsvpro/files
|
43
44
|
|
44
|
-
:include:sycsvpro.rdoc
|
45
|
+
:include: sycsvpro.rdoc
|
45
46
|
|
data/bin/sycsvpro
CHANGED
@@ -53,11 +53,13 @@ command :analyze do |c|
|
|
53
53
|
puts "Analysis of #{global_options[:f]}"
|
54
54
|
puts "#{result.col_count} columns: #{result.cols}"
|
55
55
|
puts "#{result.row_count} rows"
|
56
|
+
puts; puts "Row sample data:"
|
57
|
+
puts "#{result.sample_row}"
|
58
|
+
puts; puts "Column index: Column name | Column sample value"
|
59
|
+
sample_cols = result.sample_row.split(';')
|
56
60
|
result.cols.each_with_index do |col, index|
|
57
|
-
puts "#{index}: #{col}"
|
61
|
+
puts "#{index}: #{col} | #{sample_cols[index]}"
|
58
62
|
end
|
59
|
-
puts "Row sample data:"
|
60
|
-
puts "#{result.sample_row}"
|
61
63
|
end
|
62
64
|
end
|
63
65
|
|
@@ -316,6 +318,10 @@ command :sort do |c|
|
|
316
318
|
c.default_value '%Y-%m-%d'
|
317
319
|
c.flag [:df]
|
318
320
|
|
321
|
+
c.desc 'First row to sort, row numbers are 0 based'
|
322
|
+
c.arg_name 'ROW_NUMBER'
|
323
|
+
c.flag [:start]
|
324
|
+
|
319
325
|
c.desc 'File doesn\'t contain a header'
|
320
326
|
c.switch [:h, :headerless]
|
321
327
|
|
@@ -323,9 +329,14 @@ command :sort do |c|
|
|
323
329
|
c.switch [:d, :desc]
|
324
330
|
|
325
331
|
c.action do |global_options,options,args|
|
326
|
-
sorter = Sycsvpro::Sorter.new(infile: global_options[:f],
|
327
|
-
|
328
|
-
|
332
|
+
sorter = Sycsvpro::Sorter.new(infile: global_options[:f],
|
333
|
+
outfile: global_options[:o],
|
334
|
+
rows: options[:r],
|
335
|
+
cols: options[:c],
|
336
|
+
df: options[:df],
|
337
|
+
start: options[:start],
|
338
|
+
headerless: options[:h],
|
339
|
+
desc: options[:d])
|
329
340
|
print 'Sorting...'
|
330
341
|
sorter.execute
|
331
342
|
print 'done'
|
data/lib/sycsvpro/aggregator.rb
CHANGED
@@ -5,7 +5,8 @@ require_relative 'dsl'
|
|
5
5
|
# Operating csv files
|
6
6
|
module Sycsvpro
|
7
7
|
|
8
|
-
# An Aggregator counts specified row values and adds a sum to the end of
|
8
|
+
# An Aggregator counts specified row values and adds a sum to the end of
|
9
|
+
# the row
|
9
10
|
class Aggregator
|
10
11
|
|
11
12
|
include Dsl
|
@@ -35,8 +36,16 @@ module Sycsvpro
|
|
35
36
|
# sums of the column values
|
36
37
|
attr_reader :sums
|
37
38
|
|
38
|
-
# Creates a new aggregator. Takes as attributes infile, outfile, key, rows,
|
39
|
-
# and indicator whether to add a sum row
|
39
|
+
# Creates a new aggregator. Takes as attributes infile, outfile, key, rows,
|
40
|
+
# cols, date-format and indicator whether to add a sum row
|
41
|
+
# :call-seq:
|
42
|
+
# Sycsvpro::Aggregator.new(infile: "in.csv",
|
43
|
+
# outfile: "out.csv",
|
44
|
+
# headerless: false,
|
45
|
+
# rows: "1,2-4,/\S/"
|
46
|
+
# cols: "0,5",
|
47
|
+
# df: "%d.%m.%Y",
|
48
|
+
# sum: "Total:1,Items").execute
|
40
49
|
def initialize(options={})
|
41
50
|
@infile = options[:infile]
|
42
51
|
@outfile = options[:outfile]
|
@@ -55,7 +64,8 @@ module Sycsvpro
|
|
55
64
|
write_result
|
56
65
|
end
|
57
66
|
|
58
|
-
# Process the aggregation of the key values
|
67
|
+
# Process the aggregation of the key values. The result will be written to
|
68
|
+
# _outfile_
|
59
69
|
def process_aggregation
|
60
70
|
File.new(infile).each_with_index do |line, index|
|
61
71
|
result = col_filter.process(row_filter.process(line.chomp, row: index))
|
@@ -92,7 +102,8 @@ module Sycsvpro
|
|
92
102
|
|
93
103
|
private
|
94
104
|
|
95
|
-
# Initializes the sum row title an positions as well as the sum column
|
105
|
+
# Initializes the sum row title an positions as well as the sum column
|
106
|
+
# title and position
|
96
107
|
def init_sum_scheme(sum_scheme)
|
97
108
|
row_scheme, col_scheme = sum_scheme.split(',') unless sum_scheme.nil?
|
98
109
|
|
data/lib/sycsvpro/calculator.rb
CHANGED
@@ -30,9 +30,18 @@ module Sycsvpro
|
|
30
30
|
# if true add a sum row at the bottom of the out file
|
31
31
|
attr_reader :add_sum_row
|
32
32
|
|
33
|
-
# Creates a new Calculator. Options expects :infile, :outfile, :rows and
|
34
|
-
# a header can be provided. The header can be
|
35
|
-
# are generated due to a
|
33
|
+
# Creates a new Calculator. Options expects :infile, :outfile, :rows and
|
34
|
+
# :columns. Optionally a header can be provided. The header can be
|
35
|
+
# supplemented with additional column names that are generated due to a
|
36
|
+
# arithmetic operation that creates new columns
|
37
|
+
# :call-seq:
|
38
|
+
# Sycsvpro::Calculator.new(infile: "in.csv",
|
39
|
+
# outfile: "out.csv",
|
40
|
+
# df: "%d.%m.%Y",
|
41
|
+
# rows: "1,2,BEGINn3>20END",
|
42
|
+
# header: "*,Count",
|
43
|
+
# cols: "4:Count=c1+c2*2",
|
44
|
+
# sum: true).execute
|
36
45
|
def initialize(options={})
|
37
46
|
@infile = options[:infile]
|
38
47
|
@outfile = options[:outfile]
|
@@ -46,12 +55,14 @@ module Sycsvpro
|
|
46
55
|
end
|
47
56
|
|
48
57
|
# Retrieves the values from a row as the result of a arithmetic operation
|
58
|
+
# with #eval
|
49
59
|
def method_missing(id, *args, &block)
|
50
60
|
return to_number(columns[$1.to_i]) if id =~ /c(\d+)/
|
51
61
|
return to_date(columns[$1.to_i]) if id =~ /d(\d+)/
|
62
|
+
super
|
52
63
|
end
|
53
64
|
|
54
|
-
# Executes the calculator
|
65
|
+
# Executes the calculator and writes the result to the _outfile_
|
55
66
|
def execute
|
56
67
|
processed_header = false
|
57
68
|
|
@@ -75,6 +86,8 @@ module Sycsvpro
|
|
75
86
|
out.puts @columns.join(';')
|
76
87
|
|
77
88
|
@columns.each_with_index do |column, index|
|
89
|
+
column = 0 unless column.to_s =~ /^[\d\.,]*$/
|
90
|
+
|
78
91
|
if @sum_row[index]
|
79
92
|
@sum_row[index] += to_number column
|
80
93
|
else
|
@@ -107,7 +120,8 @@ module Sycsvpro
|
|
107
120
|
end
|
108
121
|
end
|
109
122
|
|
110
|
-
# Casts a string to an integer or float depending whether the value has a
|
123
|
+
# Casts a string to an integer or float depending whether the value has a
|
124
|
+
# decimal point
|
111
125
|
def to_number(value)
|
112
126
|
return value.to_i unless value =~ /\./
|
113
127
|
return value.to_f if value =~ /\./
|
data/lib/sycsvpro/counter.rb
CHANGED
@@ -76,16 +76,13 @@ module Sycsvpro
|
|
76
76
|
end
|
77
77
|
end
|
78
78
|
end
|
79
|
-
unless sum_col_title.nil?
|
80
|
-
heading << sum_col_title
|
81
|
-
sums[sum_col_title] = sums.values.inject(:+)
|
82
|
-
end
|
83
79
|
end
|
84
80
|
|
85
|
-
|
81
|
+
# Writes the count results
|
86
82
|
def write_result
|
87
83
|
sum_line = [sum_row_title] + [''] * (key_titles.size - 1)
|
88
|
-
headline = heading_sort ? heading.sort :
|
84
|
+
headline = heading_sort ? heading.sort : original_pivot_sequence_heading
|
85
|
+
headline << add_sum_col unless sum_col_title.nil?
|
89
86
|
headline.each do |h|
|
90
87
|
sum_line << sums[h]
|
91
88
|
end
|
@@ -107,7 +104,8 @@ module Sycsvpro
|
|
107
104
|
|
108
105
|
private
|
109
106
|
|
110
|
-
# Initializes the sum row title an positions as well as the
|
107
|
+
# Initializes the sum row title an positions as well as the sum column
|
108
|
+
# title
|
111
109
|
def init_sum_scheme(sum_scheme)
|
112
110
|
|
113
111
|
return if sum_scheme.nil?
|
@@ -140,6 +138,18 @@ module Sycsvpro
|
|
140
138
|
|
141
139
|
end
|
142
140
|
|
141
|
+
# Arrange heading in the original sequence regarding conditional column
|
142
|
+
# filters
|
143
|
+
def original_pivot_sequence_heading
|
144
|
+
(heading.sort - col_filter.pivot.keys << col_filter.pivot.keys).flatten
|
145
|
+
end
|
146
|
+
|
147
|
+
# Add a sum column to the end of the heading
|
148
|
+
def add_sum_col
|
149
|
+
sums[sum_col_title] = sums.values.inject(:+)
|
150
|
+
sum_col_title
|
151
|
+
end
|
152
|
+
|
143
153
|
end
|
144
154
|
|
145
155
|
end
|
data/lib/sycsvpro/dsl.rb
CHANGED
@@ -3,6 +3,49 @@ require_relative 'row_filter'
|
|
3
3
|
# Methods to be used in customer specific script files
|
4
4
|
module Dsl
|
5
5
|
|
6
|
+
# read arguments provided at invocation
|
7
|
+
# :call-seq:
|
8
|
+
# params => infile, Result, other_params
|
9
|
+
#
|
10
|
+
# Result methods are #cols, #col_count, #row_count, #sample_row
|
11
|
+
def params
|
12
|
+
|
13
|
+
script = ARGV.shift
|
14
|
+
method = ARGV.shift
|
15
|
+
infile = ARGV.shift
|
16
|
+
|
17
|
+
if infile.nil?
|
18
|
+
STDERR.puts "You must provide an input file"
|
19
|
+
exit -1
|
20
|
+
elsif !File.exists? infile
|
21
|
+
STDERR.puts "#{infile} does not exist. You must provide a valid input file"
|
22
|
+
exit -1
|
23
|
+
end
|
24
|
+
|
25
|
+
if ARGV.empty?
|
26
|
+
print "#{method}(#{infile})"
|
27
|
+
else
|
28
|
+
print "#{method}(#{infile}, #{ARGV.join(', ')})"
|
29
|
+
end
|
30
|
+
|
31
|
+
puts; print "Analyzing #{infile}..."
|
32
|
+
|
33
|
+
result = Sycsvpro::Analyzer.new(infile).result
|
34
|
+
puts; print "> #{result.col_count} cols | #{result.row_count} rows"
|
35
|
+
|
36
|
+
[infile, result, ARGV].flatten
|
37
|
+
|
38
|
+
end
|
39
|
+
|
40
|
+
# Delete obsolete files
|
41
|
+
# :call-seq:
|
42
|
+
# clean_up(%w{ file1 file2 }) -> nil
|
43
|
+
def clean_up(files)
|
44
|
+
puts; print "Cleaning up directory..."
|
45
|
+
|
46
|
+
files.each { |file| File.delete(file) }
|
47
|
+
end
|
48
|
+
|
6
49
|
# Retrieves rows and columns from the file and returns them to the block provided by the caller
|
7
50
|
def rows(options={})
|
8
51
|
infile = File.expand_path(options[:infile])
|
data/lib/sycsvpro/filter.rb
CHANGED
@@ -66,11 +66,7 @@ module Sycsvpro
|
|
66
66
|
return false if boolean_filter.empty? or values.empty?
|
67
67
|
expression = boolean_filter
|
68
68
|
columns = expression.scan(/(([nsd])(\d+))([<!=~>]{1,2})(.*?)(?:[\|&]{2}|$)/)
|
69
|
-
# STDERR.puts "expr = #{expression.inspect}"
|
70
|
-
# STDERR.puts "vals = #{values.inspect}"
|
71
|
-
# STDERR.puts "cols = #{columns.inspect}"
|
72
69
|
columns.each do |c|
|
73
|
-
# STDERR.puts "val = #{values[c[2].to_i].inspect}"
|
74
70
|
value = case c[1]
|
75
71
|
when 'n'
|
76
72
|
values[c[2].to_i].empty? ? '0' : values[c[2].to_i]
|
@@ -93,9 +89,7 @@ module Sycsvpro
|
|
93
89
|
end
|
94
90
|
end
|
95
91
|
expression = expression.gsub(c[0], value)
|
96
|
-
# STDERR.puts "val2 = #{value}"
|
97
92
|
end
|
98
|
-
# STDERR.puts "exp = #{expression.inspect}"
|
99
93
|
eval(expression)
|
100
94
|
end
|
101
95
|
|
@@ -124,7 +118,6 @@ module Sycsvpro
|
|
124
118
|
# Creates a filter based on the provided rows and columns select criteria
|
125
119
|
def create_filter(values)
|
126
120
|
values.scan(/(?<=,|^)(BEGIN.*?END|\/.*?\/|.*?)(?=,|$)/i).flatten.each do |value|
|
127
|
-
# STDERR.puts "value = #{value}"
|
128
121
|
send(value)
|
129
122
|
end unless values.nil?
|
130
123
|
end
|
data/lib/sycsvpro/sorter.rb
CHANGED
@@ -20,17 +20,33 @@ module Sycsvpro
|
|
20
20
|
attr_reader :col_type_filter
|
21
21
|
# sorted rows
|
22
22
|
attr_reader :sorted_rows
|
23
|
-
# file doesn't contain a header
|
23
|
+
# file doesn't contain a header. If not headerless then empty rows from
|
24
|
+
# beginning of file are discarted and first non empty row is considered as
|
25
|
+
# header. Subsequent rows will be sorted and added in the resulting file
|
26
|
+
# after the header
|
24
27
|
attr_reader :headerless
|
28
|
+
# First row to sort. Will skip rows 0 to start - 1 and add them to top of
|
29
|
+
# file. Rows from start on will be sorted.
|
30
|
+
attr_reader :start
|
25
31
|
# sort order descending or ascending
|
26
32
|
attr_reader :desc
|
27
33
|
|
28
|
-
# Creates a Sorter and takes as options infile, outfile, rows, cols
|
29
|
-
# date format for the date columns to sort (optional).
|
34
|
+
# Creates a Sorter and takes as options infile, outfile, rows, cols
|
35
|
+
# including types and a date format for the date columns to sort (optional).
|
36
|
+
# :call-seq:
|
37
|
+
# Sycsvrpo::Sorter.new(infile: "infile.csv",
|
38
|
+
# outfile: "outfile.csv",
|
39
|
+
# rows: "1,2-5,12-30",
|
40
|
+
# cols: "n:1,s:3",
|
41
|
+
# headerless: true,
|
42
|
+
# df: "%d.%m.%Y",
|
43
|
+
# start: "2").execute
|
44
|
+
# The sorted infile will saved to outfile
|
30
45
|
def initialize(options={})
|
31
46
|
@infile = options[:infile]
|
32
47
|
@outfile = options[:outfile]
|
33
48
|
@headerless = options[:headerless] || false
|
49
|
+
@start = options[:start]
|
34
50
|
@desc = options[:desc] || false
|
35
51
|
@row_filter = RowFilter.new(options[:rows], df: options[:df])
|
36
52
|
@col_type_filter = ColumnTypeFilter.new(options[:cols], df: options[:df])
|
@@ -41,9 +57,15 @@ module Sycsvpro
|
|
41
57
|
def execute
|
42
58
|
rows = File.readlines(infile)
|
43
59
|
|
60
|
+
skipped_rows = []
|
61
|
+
|
44
62
|
unless headerless
|
45
|
-
|
46
|
-
|
63
|
+
skipped_rows[0] = ""
|
64
|
+
skipped_rows[0] = rows.shift while skipped_rows[0].chomp.strip.empty?
|
65
|
+
end
|
66
|
+
|
67
|
+
if start
|
68
|
+
(0...start.to_i).each { |row| skipped_rows << rows.shift }
|
47
69
|
end
|
48
70
|
|
49
71
|
rows.each_with_index do |line, index|
|
@@ -53,9 +75,7 @@ module Sycsvpro
|
|
53
75
|
end
|
54
76
|
|
55
77
|
File.open(outfile, 'w') do |out|
|
56
|
-
|
57
|
-
out.puts header
|
58
|
-
end
|
78
|
+
skipped_rows.each { |row| out.puts unstring(row) }
|
59
79
|
|
60
80
|
if desc
|
61
81
|
sorted_rows.compact.sort.reverse.each do |row|
|
data/lib/sycsvpro/version.rb
CHANGED
@@ -6,6 +6,7 @@ module Sycsvpro
|
|
6
6
|
|
7
7
|
before do
|
8
8
|
@in_file = File.join(File.dirname(__FILE__), "files/in.csv")
|
9
|
+
@in_start = File.join(File.dirname(__FILE__), "files/in_start.csv")
|
9
10
|
@in_2_file = File.join(File.dirname(__FILE__), "files/in2.csv")
|
10
11
|
@in_3_file = File.join(File.dirname(__FILE__), "files/in3.csv")
|
11
12
|
@out_file = File.join(File.dirname(__FILE__), "files/out.csv")
|
@@ -80,6 +81,55 @@ module Sycsvpro
|
|
80
81
|
end
|
81
82
|
end
|
82
83
|
|
84
|
+
it "should start sorting at specified row with headerless false" do
|
85
|
+
rows = "0-7"
|
86
|
+
cols = "s:3-5,s:0"
|
87
|
+
df = "%d.%m.%Y"
|
88
|
+
|
89
|
+
sorter = Sorter.new(infile: @in_start, outfile: @out_file, rows: rows,
|
90
|
+
cols: cols, df: df, start: "1")
|
91
|
+
|
92
|
+
sorter.execute
|
93
|
+
|
94
|
+
result = [ "customer;contract-number;expires-on;machine;product1;product2",
|
95
|
+
"Total;0;0;0;0;0",
|
96
|
+
"Fink;1234;20.12.2015;f1;con123;dri222",
|
97
|
+
"Fink;1234;30.12.2016;f2;con333;dri321",
|
98
|
+
"fink;1234;;f3;con332;dri321",
|
99
|
+
"Gent;4323;1.3.2014;g1;con123;dri111",
|
100
|
+
"Haas;3322;1.10.2011;h1;con332;dri111",
|
101
|
+
"Klig;4432;;k1;con332;dri222",
|
102
|
+
"Rank;3232;1.5.2013;r1;con332;dri321" ]
|
103
|
+
|
104
|
+
File.open(@out_file).each_with_index do |line, index|
|
105
|
+
line.chomp.should eq result[index]
|
106
|
+
end
|
107
|
+
end
|
108
|
+
|
109
|
+
it "should start sorting at specified row with header true" do
|
110
|
+
rows = "0-7"
|
111
|
+
cols = "s:3-5,s:0"
|
112
|
+
df = "%d.%m.%Y"
|
113
|
+
|
114
|
+
sorter = Sorter.new(infile: @in_start, outfile: @out_file, rows: rows,
|
115
|
+
cols: cols, df: df, headerless: true, start: "2")
|
116
|
+
|
117
|
+
sorter.execute
|
118
|
+
|
119
|
+
result = [ "customer;contract-number;expires-on;machine;product1;product2",
|
120
|
+
"Total;0;0;0;0;0",
|
121
|
+
"Fink;1234;20.12.2015;f1;con123;dri222",
|
122
|
+
"Fink;1234;30.12.2016;f2;con333;dri321",
|
123
|
+
"fink;1234;;f3;con332;dri321",
|
124
|
+
"Gent;4323;1.3.2014;g1;con123;dri111",
|
125
|
+
"Haas;3322;1.10.2011;h1;con332;dri111",
|
126
|
+
"Klig;4432;;k1;con332;dri222",
|
127
|
+
"Rank;3232;1.5.2013;r1;con332;dri321" ]
|
128
|
+
|
129
|
+
File.open(@out_file).each_with_index do |line, index|
|
130
|
+
line.chomp.should eq result[index]
|
131
|
+
end
|
132
|
+
end
|
83
133
|
|
84
134
|
it "should sort a date column" do
|
85
135
|
rows = "1-7"
|
data/sycsvpro.gemspec
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: sycsvpro
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.1.
|
4
|
+
version: 0.1.3
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -9,7 +9,7 @@ authors:
|
|
9
9
|
autorequire:
|
10
10
|
bindir: bin
|
11
11
|
cert_chain: []
|
12
|
-
date: 2014-06-
|
12
|
+
date: 2014-06-21 00:00:00.000000000 Z
|
13
13
|
dependencies:
|
14
14
|
- !ruby/object:Gem::Dependency
|
15
15
|
name: rake
|
@@ -75,6 +75,22 @@ dependencies:
|
|
75
75
|
- - ! '>='
|
76
76
|
- !ruby/object:Gem::Version
|
77
77
|
version: '0'
|
78
|
+
- !ruby/object:Gem::Dependency
|
79
|
+
name: timeleap
|
80
|
+
requirement: !ruby/object:Gem::Requirement
|
81
|
+
none: false
|
82
|
+
requirements:
|
83
|
+
- - ~>
|
84
|
+
- !ruby/object:Gem::Version
|
85
|
+
version: 0.0.1
|
86
|
+
type: :runtime
|
87
|
+
prerelease: false
|
88
|
+
version_requirements: !ruby/object:Gem::Requirement
|
89
|
+
none: false
|
90
|
+
requirements:
|
91
|
+
- - ~>
|
92
|
+
- !ruby/object:Gem::Version
|
93
|
+
version: 0.0.1
|
78
94
|
- !ruby/object:Gem::Dependency
|
79
95
|
name: gli
|
80
96
|
requirement: !ruby/object:Gem::Requirement
|
@@ -112,70 +128,6 @@ files:
|
|
112
128
|
- features/step_definitions/sycsvpro_steps.rb
|
113
129
|
- features/support/env.rb
|
114
130
|
- features/sycsvpro.feature
|
115
|
-
- html/Dsl.html
|
116
|
-
- html/Object.html
|
117
|
-
- html/README_rdoc.html
|
118
|
-
- html/Sycsvpro.html
|
119
|
-
- html/Sycsvpro/Aggregator.html
|
120
|
-
- html/Sycsvpro/Allocator.html
|
121
|
-
- html/Sycsvpro/Analyzer.html
|
122
|
-
- html/Sycsvpro/Calculator.html
|
123
|
-
- html/Sycsvpro/Collector.html
|
124
|
-
- html/Sycsvpro/ColumnFilter.html
|
125
|
-
- html/Sycsvpro/ColumnTypeFilter.html
|
126
|
-
- html/Sycsvpro/Counter.html
|
127
|
-
- html/Sycsvpro/Extractor.html
|
128
|
-
- html/Sycsvpro/Filter.html
|
129
|
-
- html/Sycsvpro/Header.html
|
130
|
-
- html/Sycsvpro/Inserter.html
|
131
|
-
- html/Sycsvpro/Mapper.html
|
132
|
-
- html/Sycsvpro/Profiler.html
|
133
|
-
- html/Sycsvpro/RowFilter.html
|
134
|
-
- html/Sycsvpro/ScriptCreator.html
|
135
|
-
- html/Sycsvpro/ScriptList.html
|
136
|
-
- html/Sycsvpro/Sorter.html
|
137
|
-
- html/created.rid
|
138
|
-
- html/fonts.css
|
139
|
-
- html/fonts/Lato-Light.ttf
|
140
|
-
- html/fonts/Lato-LightItalic.ttf
|
141
|
-
- html/fonts/Lato-Regular.ttf
|
142
|
-
- html/fonts/Lato-RegularItalic.ttf
|
143
|
-
- html/fonts/SourceCodePro-Bold.ttf
|
144
|
-
- html/fonts/SourceCodePro-Regular.ttf
|
145
|
-
- html/images/add.png
|
146
|
-
- html/images/arrow_up.png
|
147
|
-
- html/images/brick.png
|
148
|
-
- html/images/brick_link.png
|
149
|
-
- html/images/bug.png
|
150
|
-
- html/images/bullet_black.png
|
151
|
-
- html/images/bullet_toggle_minus.png
|
152
|
-
- html/images/bullet_toggle_plus.png
|
153
|
-
- html/images/date.png
|
154
|
-
- html/images/delete.png
|
155
|
-
- html/images/find.png
|
156
|
-
- html/images/loadingAnimation.gif
|
157
|
-
- html/images/macFFBgHack.png
|
158
|
-
- html/images/package.png
|
159
|
-
- html/images/page_green.png
|
160
|
-
- html/images/page_white_text.png
|
161
|
-
- html/images/page_white_width.png
|
162
|
-
- html/images/plugin.png
|
163
|
-
- html/images/ruby.png
|
164
|
-
- html/images/tag_blue.png
|
165
|
-
- html/images/tag_green.png
|
166
|
-
- html/images/transparent.png
|
167
|
-
- html/images/wrench.png
|
168
|
-
- html/images/wrench_orange.png
|
169
|
-
- html/images/zoom.png
|
170
|
-
- html/index.html
|
171
|
-
- html/js/darkfish.js
|
172
|
-
- html/js/jquery.js
|
173
|
-
- html/js/navigation.js
|
174
|
-
- html/js/search.js
|
175
|
-
- html/js/search_index.js
|
176
|
-
- html/js/searcher.js
|
177
|
-
- html/rdoc.css
|
178
|
-
- html/table_of_contents.html
|
179
131
|
- lib/sycsvpro.rb
|
180
132
|
- lib/sycsvpro/aggregator.rb
|
181
133
|
- lib/sycsvpro/allocator.rb
|