data_list_converter 0.3.10 → 0.4.4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
-SHA1:
-  metadata.gz: 85c9410c6b8c64a1af696f42485e098f43f432c2
-  data.tar.gz: 930b7bb74e9d441e183451d4329e639c06aa3cdf
+SHA256:
+  metadata.gz: f65479d343c6345e3dbaf9762d3dcd8e6e0fc353bfc8441cc1dd3e7d9385aaba
+  data.tar.gz: fc2be9587bfafe4a5da8558e10b49fd3a85b514e0c79057bb8abe61b5bb5f958
 SHA512:
-  metadata.gz: b53cb6e80ac57b256b74100421b12d14c9e1f57bee172807bb0f6437efe27e2c2c8915705d52b0e1ff01855188a6e4a3ac274138f7623cbec648a187ba349631
-  data.tar.gz: c87df4a93536be36f7aec7d88c3c93325b39fe03ab09414d4d1533024569e6d550f46cfff724df48bd39612bd6b52727473acbe20b1fdd273b92194b4f9286fc
+  metadata.gz: 9faf4b96e80b3bc6146ead5b2cbbdc6b707ccd472489fbbaab88723d57e1e4865037d11adeec061b9162cdacdc4ba9c8942a70acf1b617a153d2603fb3201196
+  data.tar.gz: '090045d9d7b897e899bbdaa53135854a98f66b1a3a4e698df7ba24ecfeb0c05c0ebf5a7527576af67bae4d997611625398b1d0efbe2a31e70f9d3fdb81b07c49'

data/README.md

@@ -22,61 +22,73 @@ DataListConverter.convert(:xls_file, :multi_sheet_item_data, {filename: 'result.
 You can also add filter to this process:
 
 ```ruby
-filters = [
-  # filter with default options
-  :limit,
-  # filter with options
-  {limit: {size: 2}},
-  # multiple filters
-  [{limit: {size: 12}},
-   {count: {size: 4}}],
-]
-filters.map do |filter|
-  data = [{name: "james"}] * 10
-  DataListConverter.convert(:item_data, :table_data, data, table_iterator: {filter: filter})
-end
+data = (1..20).map{|i| {name: "user-#{i}", age: i+20}}
+# filter with default options (limit 10)
+DataListConverter.convert(:item_data, :table_data, data, item_iterator: {filter: :limit})
+# filter with options
+DataListConverter.convert(:item_data, :table_data, data, item_iterator: {filter: {limit: {size: 2}}})
+# multiple filters
+DataListConverter.convert(:item_data, :table_data, data, item_iterator: {filter: [{limit: {size: 12}}, {count: {size: 4}}]})
 ```
 
-Please read [the source code](https://github.com/halida/data_list_converter/blob/master/lib/data_list_converter/) for more information,
-also you can check [test examples](https://github.com/halida/data_list_converter/blob/master/test/).
-
 ## Data Types
 
+Default data types:
+
 - **item_data** like: `[{name: 'James', age: '22'}, ...]`, keys should be symbol.
 - **item_iterator** iterator for item_data, used like: iter.call{|item| out << item}
 - **table_data** like: `[["name", "age"], ["James", "22"], ["Bob", "33"], ...]`
 - **table_iterator** iterator for table_data
-- **csv_file** file in csv format
-- **xls_file** file in excel format, should install `spreadsheet` gem, and `require 'data_list_converter/types/xls_file'`
-- **xlsx_file** file in excel xml format, should install `rubyXL` gem, and `require 'data_list_converter/types/xlsx_file'`
 - **multi_sheet** Contains several data with sheets:
     - **multi_sheet_table_iterator**: like: `{sheet1: table_iterator1, sheet2: table_iterator2}`
     - **multi_sheet_table_data**: like: `{sheet1: [['name', 'age'], ...], sheet2: ...}`
     - **multi_sheet_item_iterator**: like: `{sheet1: item_iterator1, sheet2: item_iterator2}`
     - **multi_sheet_item_data**: like: `{sheet1: [{name: 'James', age: 32}], sheet2: ...}`
-- **records** ActiveRecord records, usage: `DataListConverter.convert(:records, :item_data, Users.where(condition), item_iterator: {columns: [:name, :age]})`
 
+Plugin data types, should required first by `require 'data_list_converter/types/#{type}'`
+
+- **csv_file** file in csv format
+- **xls_file** file in excel format, should install `spreadsheet` gem first
+- **xlsx_file** file in excel xml format, should install `rubyXL` gem first
+- **records** ActiveRecord records
+
+Please check [test examples](https://github.com/halida/data_list_converter/blob/master/test/types_test.rb) to see how to use those types.
 
 ## Filters
 
-**item_iterator/table_iterator limit**: limit item_iterator result counts, usage: `DataListConverter.convert(:item_data, :table_data, item_data, item_iterator: {filter: {limit: {size: 2}}})`
+**item_iterator/table_iterator limit**: limit item_iterator result counts, usage: `DataListConverter.convert(:item_data, :table_data, item_data, item_iterator: {filter: {limit: {size: 2}}})`, default limit size is 10.
 
-**item_iterator count**: count item_iterator items, usage: `DataListConverter.convert(:xls_file, :item_data, {filename: 'result.xls'}, item_iterator: {filter: {count: {size: 10}}})`, it will print current item counts every `size`.
+**item_iterator count**: count item_iterator items, usage: `DataListConverter.convert(:xls_file, :item_data, {filename: 'result.xls'}, item_iterator: {filter: {count: {size: 10}}})`, it will print current item counts every `size`, please [see here](https://github.com/halida/data_list_converter/blob/master/lib/data_list_converter/filters/count.rb) for more options.
 
-Please check more [test examples](https://github.com/halida/data_list_converter/blob/master/test/filters_test.rb)
+Please see [test examples](https://github.com/halida/data_list_converter/blob/master/test/filters_test.rb) to learn how to use filter.
+
+## helpers
+
+- **types**: Get current valid types.
+- **routes**: Get current valid routes.
+- **file_types**: Get current file types, which is the types has `_file` suffix.
+- **get_file_format**: Get file type by filename, which compare file extension with `file_types`. `DataListConverter.get_file_format('xxx.xls') == :xls_file`
+- **save_to_file**: Save data to file, it will use `get_file_format` to find proper file format.  `DataListConverter.save_to_file(filename, data, data_format=:item_data)`
+- **load_from_file**: Get data from file, it will use `get_file_format` to find proper file format. `DataListConverter.load_from_file(filename, data_format=:item_data)`
+- **unify_item_data_keys**: Sometimes in the `item_data` list, each data keys don't exactly same, so use this function to fix it, example: `DataListConverter.unify_item_data_keys([{a: 12}, {b: 11}]) == [{a: 12, b: nil}, {a: nil, b: 11}]`.
+- **flatten**: Flatten multi level item_data list into one level, example: `DataListConverter.flatten({a: {b: 12}, c: {d: {e: 11}}}) == {:"a:b"=>12, :"c:d:e"=>11}`, can change seperator: `DataListConverter.flatten(data, '_')`, set max level: `DataListConverter.flatten(data, '_', 2)`
 
 ## Extend
 
 You can add your own data types and filters, example:
 
 ```ruby
-DataListConverter.register_converter(:records, :item_iterator) do |records, options|
-  columns = options[:columns]
+DataListConverter.register_converter(:records, :item_iterator) do |input, options|
+  query = self.parameter(input, :query, :input)
+  columns = self.parameter(input, :columns, :input)
+  display = input[:display] || columns
+
   lambda { |&block|
-    records.find_each do |record|
-      item = columns.map do |column|
-        [column.first.to_sym, record.send(column[1])]
-      end.to_h
+    query.pluck(*columns).each do |data|
+      item = {}
+      data.each_with_index do |d, i|
+        item[display[i]] = d
+      end
       block.call(item)
     end
   }
@@ -94,14 +106,3 @@ DataListConverter.register_filter(:item_iterator, :limit) do |proc, options|
   }
 end
 ```
-
-
-## Todo
-
-- check type exists
-- route not found, list all routes
-- register can bind method again
-- better API
-- better error message
-- load file/save file with auto format by extname
-- report error when cannot find type

data/changelog.md

@@ -1,3 +1,10 @@
+## 0.4
+
+Add parameter check, type check, change record type API.
+
+- 0.4.1 add max_level to flatten helper
+- 0.4.2 fix value cannot save if value is symbol
+
 ## 0.3
 
 Refactor.

data/data_list_converter.gemspec

@@ -17,6 +17,10 @@ Gem::Specification.new do |s|
   s.add_development_dependency 'spreadsheet', '~> 1.0'
   s.add_development_dependency 'rubyXL', '~> 3.3'
   s.add_development_dependency 'pry', '~> 0.10.1'
+  s.add_development_dependency 'sqlite3', '~> 1.3'
+  s.add_development_dependency 'activerecord', '~> 4.2'
+  s.add_development_dependency 'xlsxtream'
+  s.add_development_dependency 'creek'
 
   s.files         = `git ls-files`.split("\n")
   s.test_files    = `git ls-files -- {test,spec,features}/*`.split("\n")

data/lib/data_list_converter/base.rb

@@ -21,6 +21,7 @@ class DataListConverter
       puts "#{Time.now.strftime('%Y-%m-%d %H:%M:%S')}\t#{msg}"
     end
 
+    # register_converter(:item_data, :item_iterator){ |data, options| ... }
     def register_converter(from_type, to_type, &block)
       @route_map = nil # clear cache
       CONVERTERS[[from_type, to_type]] = block
@@ -44,7 +45,7 @@ class DataListConverter
     def convert(from_type, to_type, from_value, options={})
       methods = []
       add_filter = lambda { |type|
-        filters = (options[type] || {}).delete(:filter)
+        filters = (options[type] || {})[:filter]
         return unless filters
         methods += normalize_filters(type, filters)
       }
@@ -57,6 +58,7 @@ class DataListConverter
         from_type, to_type = route[i], route[i+1]
         method = CONVERTERS[[from_type, to_type]]
         raise "cannot find converter #{from_type} -> #{to_type}" unless method
+        self.log "#{from_type} -> #{to_type} options: #{options[to_type]}"
         methods.push([method, options[to_type] || {}])
         add_filter.call(to_type)
       end
@@ -91,7 +93,11 @@ class DataListConverter
     # If we want to convert between types, like: convert item_data into csv_file,
     # we need find all the intermidate data type, like: [:item_data, :item_iterator, :table_iterator, :csv_file]
     def find_route(from_type, to_type)
+      [from_type, to_type].each do |type|
+        raise Exception, "cannot find type: #{type}" unless self.types.include?(type)
+      end
       raise Exception, "from_type should not equal to to_type: #{from_type}" if from_type == to_type
+
       # map wide search
       checked = Set.new
       checking = Set.new([from_type])
@@ -124,7 +130,10 @@ class DataListConverter
           checking += Set.new(next_nodes) - checked
         end
       end
-      raise Exception, "Route not found: #{from_type} -> #{to_type}"
+
+      log = ["Route not found: #{from_type} -> #{to_type}", "Current routes:"]
+      log += self.routes.map{|from, to| "#{from} -> #{to}"}
+      raise Exception, log.join("\n")
     end
 
     # convert adjacency list into quick lookup hash

data/lib/data_list_converter/filters/count.rb

@@ -5,6 +5,7 @@ class DataListConverter
     msg_format = options[:msg] || "on %{count}"
     total_format = options[:total] || "total: %{total}"
     out = options[:out] || STDOUT
+
     total = 1
     lambda { |&block|
       proc.call do |item|

data/lib/data_list_converter/helper.rb

@@ -1,14 +1,14 @@
 class DataListConverter
-  def self.save_to_file(filename, data, data_format=:item_data)
+  def self.save_to_file(filename, data, data_format=:item_data, options={})
     file_format = self.get_file_format(filename)
-    DataListConverter.convert(data_format, file_format, data,
-                              file_format => {filename: filename})
+    options[file_format] = {filename: filename}
+    DataListConverter.convert(data_format, file_format, data, options)
   end
 
-  def self.load_from_file(filename, data_format=:item_data)
+  def self.load_from_file(filename, data_format=:item_data, options={})
     file_format = self.get_file_format(filename)
-    DataListConverter.convert(file_format, data_format,
-                              {filename: filename})
+    options[:filename] = filename
+    DataListConverter.convert(file_format, data_format, options)
   end
 
   def self.get_file_format(filename)
@@ -23,6 +23,10 @@ class DataListConverter
     CONVERTERS.keys.flatten.uniq.sort
   end
 
+  def self.routes
+    CONVERTERS.keys
+  end
+
   def self.file_types
     matcher = /(.*)_file$/
     DataListConverter.types.select do |type|
@@ -51,22 +55,27 @@ class DataListConverter
   #   {a: {b: 12}, c: {d: {e: 11}}}
   #   =>
   #   {:"a:b"=>12, :"c:d:e"=>11}
-  def self.flatten(data, sep=':')
+  def self.flatten(data, sep=':', max_level=nil)
     out = {}
-    recursive_flatten(out, data, nil, sep)
+    recursive_flatten(out, data, nil, sep, 1, max_level)
     out
   end
 
-  def self.recursive_flatten(out, data, header, sep)
+  def self.recursive_flatten(out, data, header, sep, level, max_level)
     data.each do |k, v|
       k = header ? :"#{header}#{sep}#{k}" : k
-      case v
-      when Hash
-        recursive_flatten(out, v, k, sep)
+      if v.kind_of?(Hash) and (!max_level or level <= max_level)
+        recursive_flatten(out, v, k, sep, level+1, max_level)
       else
         out[k] = v
       end
     end
   end
 
+  def self.parameter(data, key, type)
+    raise Exception, "`#{type}` should be hash, not `#{data.class}`: #{data}" unless data.kind_of?(Hash)
+    raise Exception, "Need `#{key}` for `#{type}`, current: #{data}" if not data.has_key?(key)
+    data.fetch(key)
+  end
+
 end

data/lib/data_list_converter/types/csv_file.rb

@@ -3,7 +3,8 @@ require 'csv'
 class DataListConverter
   self.register_converter(:csv_file, :table_iterator) do |input, options|
     lambda { |&block|
-      CSV.open(input[:filename]) do |csv|
+      filename = self.parameter(input, :filename, :input)
+      CSV.open(filename) do |csv|
         csv.each do |row|
           block.call(row)
         end
@@ -12,7 +13,8 @@ class DataListConverter
   end
 
   self.register_converter(:table_iterator, :csv_file) do |proc, options|
-    CSV.open(options[:filename], 'wb', force_quotes: true) do |csv|
+    filename = self.parameter(options, :filename, :csv_file)
+    CSV.open(filename, 'wb', force_quotes: true) do |csv|
       proc.call do |row|
         csv << row
       end

data/lib/data_list_converter/types/fast_xlsx_file.rb

@@ -0,0 +1,58 @@
+require 'xlsxtream'
+require 'creek'
+
+class DataListConverter
+
+  self.register_converter(:xlsx_file, :table_iterator) do |input, options|
+    lambda { |&block|
+      filename = self.parameter(input, :filename, :input)
+
+      creek = Creek::Book.new filename
+      sheet = creek.sheets[input[:sheet] || 0]
+      sheet.rows.each do |row|
+        block.call(row.values)
+      end
+    }
+  end
+
+  self.register_converter(:xlsx_file, :multi_sheet_table_iterator) do |input, options|
+    filename = self.parameter(input, :filename, :input)
+
+    creek = Creek::Book.new filename
+    creek.sheets.map do |sheet|
+      iterator = lambda { |&block|
+        sheet.rows.each do |row|
+          block.call(row.values)
+        end
+      }
+      [sheet.name.to_sym, iterator]
+    end.to_h
+  end
+
+  self.register_converter(:table_iterator, :xlsx_file) do |proc, options|
+    filename = self.parameter(options, :filename, :xlsx_file)
+    Xlsxtream::Workbook.open(filename) do |xlsx|
+      xlsx.write_worksheet (options[:sheet] || "Sheet1") do |sheet|
+        proc.call do |row|
+          sheet << row
+        end
+      end
+    end
+    filename
+  end
+
+  self.register_converter(:multi_sheet_table_iterator, :xlsx_file) do |data, options|
+    filename = self.parameter(options, :filename, :xlsx_file)
+    Xlsxtream::Workbook.open(filename) do |xlsx|
+      data.each do |name, table_iterator|
+        xlsx.write_worksheet(name.to_s) do |sheet|
+          table_iterator.call do |row|
+            sheet << row
+          end
+        end
+      end
+    end
+    filename
+  end
+
+end

data/lib/data_list_converter/types/marshal.rb

@@ -1,13 +1,15 @@
 class DataListConverter
 
   def self.marshal_file_to_data(input, options=nil)
-    File.open(input[:filename]) do |f|
+    filename = self.parameter(input, :filename, :input)
+    File.open(filename) do |f|
       Marshal.load(f)
     end
   end
 
   def self.marshal_data_to_file(data, options)
-    File.open(options[:filename], 'w+') do |f|
+    filename = self.parameter(options, :filename, :marshal)
+    File.open(filename, 'w+') do |f|
       Marshal.dump(data, f)
     end
     options[:filename]

data/lib/data_list_converter/types/multi_sheet.rb

@@ -19,8 +19,9 @@ class DataListConverter
       :"multi_sheet_#{from_type}",
       :"multi_sheet_#{to_type}",
     ) do |data, options|
+      self.log("multi_sheet #{from_type} -> #{to_type} with options: #{options}")
       data.map do |sheet, from_data|
-        to_data = self.convert(from_type, to_type, from_data)
+        to_data = self.convert(from_type, to_type, from_data, options)
         [sheet, to_data]
       end.to_h
     end

data/lib/data_list_converter/types/records.rb

@@ -1,13 +1,20 @@
 # ActiveRecords
 class DataListConverter
 
-  self.register_converter(:records, :item_iterator) do |records, options|
-    columns = options[:columns]
+  # columns = [:table_column1, :table_column2, ...]
+  # display = [:display_name1, :display_name2, ...]
+  # convert(:records, :item_data, query: query, columns: columns, display: display)
+  self.register_converter(:records, :item_iterator) do |input, options|
+    query = self.parameter(input, :query, :input)
+    columns = self.parameter(input, :columns, :input)
+    display = input[:display] || columns
+
     lambda { |&block|
-      records.find_each do |record|
-        item = columns.map do |column|
-          [column[0].to_sym, record.send(column[1])]
-        end.to_h
+      query.pluck(*columns).each do |data|
+        item = {}
+        data.each_with_index do |d, i|
+          item[display[i]] = d
+        end
         block.call(item)
       end
     }

data/lib/data_list_converter/types/xls_file.rb

@@ -5,7 +5,8 @@ class DataListConverter
 
   self.register_converter(:xls_file, :table_iterator) do |input, options|
     lambda { |&block|
-      book = Spreadsheet.open(input[:filename])
+      filename = self.parameter(input, :filename, :input)
+      book = Spreadsheet.open(filename)
       sheet = book.worksheet input[:sheet] || 0
       sheet.each do |row|
         block.call(row.to_a)
@@ -18,11 +19,13 @@ class DataListConverter
     sheet = book.create_worksheet(name: (options[:sheet] || "Sheet1"))
     i = 0
     proc.call do |row|
+      row = row.map(&:to_s)
       sheet.row(i).push *row
       i += 1
     end
-    book.write(options[:filename])
-    options[:filename]
+    filename = self.parameter(options, :filename, :xls_file)
+    book.write(filename)
+    filename
   end
 
   self.register_converter(:multi_sheet_table_iterator, :xls_file) do |data, options|
@@ -31,17 +34,19 @@ class DataListConverter
       sheet = book.create_worksheet(name: name.to_s)
       i = 0
       table_iterator.call do |row|
+        row = row.map(&:to_s)
         sheet.row(i).concat(row)
         i += 1
       end
     end
-    filename = options[:filename]
+    filename = self.parameter(options, :filename, :xls_file)
     book.write(filename)
     filename
   end
 
-  self.register_converter(:xls_file, :multi_sheet_table_iterator) do |data, options|
-    book = Spreadsheet.open(data[:filename])
+  self.register_converter(:xls_file, :multi_sheet_table_iterator) do |input, options|
+    filename = self.parameter(input, :filename, :input)
+    book = Spreadsheet.open(filename)
     book.worksheets.map do |sheet|
       iterator = lambda { |&block|
         sheet.each do |row| 

data/lib/data_list_converter/types/xlsx_file.rb

@@ -5,7 +5,8 @@ class DataListConverter
 
   self.register_converter(:xlsx_file, :table_iterator) do |input, options|
     lambda { |&block|
-      book = RubyXL::Parser.parse(input[:filename])
+      filename = self.parameter(input, :filename, :input)
+      book = RubyXL::Parser.parse(filename)
       sheet = book.worksheets[input[:sheet] || 0]
       sheet.each do |row|
         next unless row
@@ -22,11 +23,11 @@ class DataListConverter
     i = 0
     proc.call do |row|
       row.each_with_index do |v, j|
-        sheet.add_cell(i, j, v)
+        sheet.add_cell(i, j, v.to_s)
       end
       i += 1
     end
-    filename = options[:filename]
+    filename = self.parameter(options, :filename, :xlsx_file)
     book.write(filename)
     filename
   end
@@ -41,27 +42,28 @@ class DataListConverter
         row.each_with_index do |v, j|
           if v.kind_of?(Hash)
             # custom cell format
-            cell = sheet.add_cell(i, j, v[:text])
+            cell = sheet.add_cell(i, j, v[:text].to_s)
             v.each do |k, v|
               next if k == :text
               cell.send(k, v)
             end
             cell.change_fill(v[:fill_color]) if v[:fill_color]
           else
-            cell = sheet.add_cell(i, j, v)
+            cell = sheet.add_cell(i, j, v.to_s)
           end
 
         end
         i += 1
       end
     end
-    filename = options[:filename]
+    filename = self.parameter(options, :filename, :xlsx_file)
     book.write(filename)
     filename
   end
 
-  self.register_converter(:xlsx_file, :multi_sheet_table_iterator) do |data, options|
-    book = RubyXL::Parser.parse(data[:filename])
+  self.register_converter(:xlsx_file, :multi_sheet_table_iterator) do |input, options|
+    filename = self.parameter(input, :filename, :input)
+    book = RubyXL::Parser.parse(filename)
     book.worksheets.map do |sheet|
       iterator = lambda { |&block|
         sheet.each do |row|

data/lib/data_list_converter/version.rb

@@ -1,3 +1,3 @@
 class DataListConverter
-  VERSION = "0.3.10".freeze
+  VERSION = "0.4.4".freeze
 end

data/test/base_test.rb

@@ -40,6 +40,12 @@ describe DataListConverter do
       string.string.must_equal ".4\n.8\n.12\n"
     end
 
+    it 'test data format exists' do
+      -> {
+        DataListConverter.convert(:item_data, :ppp, {a: 12})
+      }.must_raise Exception
+    end
+
   end
 
 end

data/test/filters_test.rb

@@ -26,12 +26,30 @@ describe DataListConverter do
       string = StringIO.new
       filter = {count: {size: 4000,
                         out: string,
-                        msg: "%{percent}%"}}
+                        msg: "%{percent}%%"}}
       result = DataListConverter.convert(
         :item_iterator, :table_data, iter,
         item_iterator: {filter: filter})
       string.string.split("\n").must_equal ['total: 10000', '40.0%', '80.0%']
     end
   end
+
+  describe :remove_debug do
+    specify do
+      filter = {remove_debug: true}
+      item_data = [{name: "james", debug: "", a: 12}] * 2
+      result = DataListConverter.convert(
+        :item_data, :table_data, item_data,
+        table_iterator: {filter: filter})
+      result.must_equal [["name"], ["james"], ["james"]]
+
+      # check on multi_sheet
+      item_data = {a: [{name: "james", debug: "", a: 12}] * 2, b: [{name: 'cc', debug: "", b: 3}]*3}
+      result = DataListConverter.convert(
+        :multi_sheet_item_data, :multi_sheet_table_data, item_data,
+        multi_sheet_table_iterator: {table_iterator: {filter: {remove_debug: true}}})
+      result.must_equal(a: [["name"], ["james"], ["james"]], b: [["name"], ["cc"], ["cc"], ["cc"]])
+    end
+  end
   
 end

data/test/helper_test.rb

@@ -79,6 +79,12 @@ describe DataListConverter do
       # change sep
       data = {a: {b: 12}, c: {d: {e: 11}}}
       @c.flatten(data, '_').must_equal({:"a_b"=>12, :"c_d_e"=>11})
+
+      # set max level
+      data = {a: {b: 12}, c: {d: {e: {f: 11}}}}
+      @c.flatten(data, ':', 1).must_equal({:"a:b"=>12, :"c:d"=>{e: {f: 11}}})
+      data = {a: {b: 12}, c: {d: {e: {f: 11}}}}
+      @c.flatten(data, ':', 2).must_equal({:"a:b"=>12, :"c:d:e"=>{f: 11}})
     end
   end
 end

data/test/types_test.rb

@@ -79,6 +79,27 @@ describe DataListConverter do
     end
   end
 
+  require 'data_list_converter/types/fast_xlsx_file'
+
+  describe :xlsx_file do
+    specify do
+      filename = 'test.xlsx'
+      begin
+        @c.convert(:item_data, :xlsx_file, ITEM_DATA, xlsx_file: {filename: filename})
+        @c.convert(:xlsx_file, :item_data, {filename: filename}).must_equal ITEM_DATA
+
+        @c.convert(:multi_sheet_table_data, :xlsx_file, MULTI_SHEET_TABLE_DATA,
+                   xlsx_file: {filename: filename})
+        @c.convert(:xlsx_file, :multi_sheet_table_data,
+                   {filename: filename},
+                  ).must_equal(MULTI_SHEET_TABLE_DATA)
+      ensure
+        FileUtils.rm_f(filename)
+      end
+    end
+
+  end
+
   describe :marshal do
     specify do
       filename = 'test.marshal'
@@ -99,5 +120,36 @@ describe DataListConverter do
       end
     end
   end
+
+  describe :records do
+    specify do
+      require 'sqlite3'
+      require 'active_record'
+
+      ActiveRecord::Base.establish_connection(
+        adapter: 'sqlite3',
+        database: ':memory:'
+      )
+
+      ActiveRecord::Schema.define do
+        create_table :users, force: true do |t|
+          t.string :name
+          t.integer :age
+        end
+      end
+
+      class User < ActiveRecord::Base; end
+      (1..10).each{ |i| User.create(name: "user-#{i}", age: i+20) }
+
+      query = User.where("age > 25 and age < 27")
+      columns = [:name, :age]
+      @c.convert(:records, :item_data, query: query, columns: columns).
+        must_equal([{name: "user-6", age: 26}])
+
+      display = [:n, :a]
+      @c.convert(:records, :item_data, query: query, columns: columns, display: display).
+        must_equal([{n: "user-6", a: 26}])
+    end
+  end
   
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: data_list_converter
 version: !ruby/object:Gem::Version
-  version: 0.3.10
+  version: 0.4.4
 platform: ruby
 authors:
 - linjunhalida
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2016-06-02 00:00:00.000000000 Z
+date: 2020-12-30 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rake
@@ -80,6 +80,62 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: 0.10.1
+- !ruby/object:Gem::Dependency
+  name: sqlite3
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1.3'
+- !ruby/object:Gem::Dependency
+  name: activerecord
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.2'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '4.2'
+- !ruby/object:Gem::Dependency
+  name: xlsxtream
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+- !ruby/object:Gem::Dependency
+  name: creek
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description: Data List Converter is a tool to convert data between different formats.
 email:
 - linjunhalida@gmail.com
@@ -102,6 +158,7 @@ files:
 - lib/data_list_converter/helper.rb
 - lib/data_list_converter/types/basic.rb
 - lib/data_list_converter/types/csv_file.rb
+- lib/data_list_converter/types/fast_xlsx_file.rb
 - lib/data_list_converter/types/marshal.rb
 - lib/data_list_converter/types/multi_sheet.rb
 - lib/data_list_converter/types/records.rb
@@ -132,14 +189,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.4.8
+rubygems_version: 3.0.8
 signing_key: 
 specification_version: 4
 summary: convert data between different formats
-test_files:
-- test/base_test.rb
-- test/filters_test.rb
-- test/helper_test.rb
-- test/testdata.rb
-- test/types_test.rb
+test_files: []