xsv 0.3.15 → 1.0.0.pre.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 84691bdf67411e4a1e2d28c23a04e6656285247820d65c3124382261b0aba27e
-  data.tar.gz: 2ae8332a7e5e857039b729abee1478f063663e29b4eb8b9f45a4ae3ac2632916
+  metadata.gz: 2c7de4a72e380ca950dd3f076353c6e275d67314491216708ab2abe28ca1c3b2
+  data.tar.gz: 583a01b780ee1b95f9558d28f8700b885874822c133d5c17b4f3f2e94e0bb6e3
 SHA512:
-  metadata.gz: 666a4b190de6a64d676fc5dd66d5d836255b7c5b8114f21f53ce57b9d09deaa1cfd4f644c31018a48619449d5962dd86b6bdd41355861462275601fc7486c386
-  data.tar.gz: 3c6c957f46e31dac95f7c77f533ab43c52db1733f1d309b7a6e6bdc7e503da72963a3fe4f80a8269d46db436977bbc37c0b425203c2d75c8a6ef1be889714eb3
+  metadata.gz: 6a39ec94b6c3b34ce9367e9e4e2790bae830924b60a10938b54cc7c5f40749d2ebef29bfeaa9d7f9587c19c9cafdd6de9335b6aa8a8f9775dbff8632a867177d
+  data.tar.gz: 12967f46a655720dd2262ddaf1acd357d24a846e0160715e8652b116fa991a766250e55cd77ac69e85c315065e1d13458d04684f421524fc3978358c940a5868

data/.travis.yml

@@ -4,10 +4,7 @@ language: ruby
 cache: bundler
 rvm:
   - 2.5.8
-  - 2.7.1
-env:
-  - "rubyzip=1.3.0"
-  - "rubyzip=2.2.0"
-  - "ox=2.9"
-  - "ox=2.13.2"
+  - 3.0
+  - truffleruby
+  - jruby
 before_install: gem install bundler

data/CHANGELOG.md

@@ -1,5 +1,30 @@
 # Xsv Changelog
 
+## 1.0.0.pre.2 2021-01-22
+
+- Reduce allocations in XML parser
+- Return strings with the correct encoding
+- Handle XML entities
+
+## 1.0.0.pre 2021-01-18
+
+-  Switch to a minimalistic XML parser in native Ruby (#21)
+-  Ruby 3.0 compatibility
+-  Various internal cleanup and optimization
+-  API is backwards compatible with 0.3.x
+
+## 0.3.18 2020-09-30
+
+-  Improve inline string support (#18)
+
+## 0.3.17 2020-07-03
+
+- Fix parsing of empty worksheets (#17)
+
+## 0.3.16 2020-06-03
+
+- Support complex numbers (#16)
+
 ## 0.3.15 2020-06-02
 
 - Fix issue with workbooks that don't contain shared strings (#15)

data/README.md

@@ -3,7 +3,7 @@
 [![Travis CI](https://img.shields.io/travis/martijn/xsv/master)](https://travis-ci.org/martijn/xsv)
 [![Yard Docs](http://img.shields.io/badge/yard-docs-blue.svg)](https://rubydoc.info/github/martijn/xsv)
 
-Xsv is a fast, lightweight parser for Office Open XML spreadsheet files
+Xsv is a fast, lightweight, pure Ruby parser for Office Open XML spreadsheet files
 (commonly known as Excel or .xlsx files). It strives to be minimal in the
 sense that it provides nothing a CSV reader wouldn't, meaning it only
 deals with minimal formatting and cannot create or modify documents.
@@ -11,8 +11,9 @@ deals with minimal formatting and cannot create or modify documents.
 Xsv is designed for worksheets with a single table of data, optionally
 with a header row. It only casts values to basic Ruby types (integer, float,
 date and time) and does not deal with most formatting or more advanced
-functionality. The goal is to allow for fast parsing of large worksheets with
-minimal RAM and CPU consumption.
+functionality. It strives for fast processing of large worksheets with
+minimal RAM and CPU consumption and has been in production use since the earliest
+versions.
 
 Xsv stands for 'Excel Separated Values', because Excel just gets in the way.
 
@@ -32,11 +33,14 @@ Or install it yourself as:
 
     $ gem install xsv
 
-Xsv targets ruby ~> 2.6 and depends on `rubyzip` and `ox`.
+Xsv targets ruby >= 2.5 and has a just single dependency, `rubyzip`. It has been
+tested successfully with MRI, JRuby, and TruffleRuby. Due to the lack of
+native extensions should work well in multi-threaded environments or in Ractor
+when that becomes stable.
 
 ## Usage
 
-Xsv has two modes of operation. By default it returns an array for
+Xsv has two modes of operation. By default, it returns an array for
 each row in the sheet:
 
 ```ruby
@@ -75,15 +79,15 @@ end
 sheet[1] # => {"header1" => "value1", "header2" => "value2"}
 ```
 
-Be aware that hash mode will lead to unpredictable results if you have multiple
-columns with the same name!
+Be aware that hash mode will lead to unpredictable results if the worksheet
+has multiple columns with the same header.
 
-`Xsv::Workbook.open` accepts a filename, or a IO or String containing a workbook.
+`Xsv::Workbook.open` accepts a filename, or an IO or String containing a workbook.
 
 `Xsv::Sheet` implements `Enumerable` so you can call methods like `#first`,
-`#filter`/`#select` and `#map` on it.
+`#filter`/`#select`, and `#map` on it.
 
-The sheets could be accessed by index or by name:
+The sheets can be accessed by index or by name:
 
 ```ruby
 x = Xsv::Workbook.open("sheet.xlsx")
@@ -93,7 +97,7 @@ sheet = x.sheets[0] # gets sheet by index
 sheet = x.sheets_by_name('Name').first # gets sheet by name
 ```
 
-To get all the workbook's sheets names:
+To get all the sheets names:
 
 ```ruby
 sheet_names = x.sheets.map(&:name)
@@ -123,6 +127,16 @@ After checking out the repo, run `bin/setup` to install dependencies. Then, run
 
 To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
 
+## Performance and Benchmarks
+
+Xsv is faster and more memory efficient than other gems because of two things: it only _reads values_ from Excel files and it's based on a SAX-based parser instead of a DOM-based parser. If you want to read some background on this, check out my blog post on
+[Efficient XML parsing in Ruby](https://storck.io/posts/efficient-xml-parsing-in-ruby/).
+
+Jamie Schembri did a shootout of Xsv against various other Excel reading gems comparing parsing speed, memory usage, and allocations.
+Check our his blog post: [Faster Excel parsing in Ruby](https://blog.schembri.me/post/faster-excel-parsing-in-ruby/).
+
+Pre-1.0, Xsv used a native extension for XML parsing, which was faster than the native Ruby one (on MRI). But even with the native Ruby version generally Xsv still outperforms other Ruby parsing gems.
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/martijn/xsv.

data/lib/xsv.rb

@@ -1,17 +1,18 @@
 # frozen_string_literal: true
-require "date"
-require "ox"
 
-require "xsv/helpers"
-require "xsv/relationships_handler"
-require "xsv/shared_strings_parser"
-require "xsv/sheet"
-require "xsv/sheet_bounds_handler"
-require "xsv/sheet_rows_handler"
-require "xsv/sheets_ids_handler"
-require "xsv/styles_handler"
-require "xsv/version"
-require "xsv/workbook"
+require 'date'
+
+require 'xsv/helpers'
+require 'xsv/sax_parser'
+require 'xsv/relationships_handler'
+require 'xsv/shared_strings_parser'
+require 'xsv/sheet'
+require 'xsv/sheet_bounds_handler'
+require 'xsv/sheet_rows_handler'
+require 'xsv/sheets_ids_handler'
+require 'xsv/styles_handler'
+require 'xsv/version'
+require 'xsv/workbook'
 
 # XSV is a fast, lightweight parser for Office Open XML spreadsheet files
 # (commonly known as Excel or .xlsx files). It strives to be minimal in the

data/lib/xsv/helpers.rb

@@ -1,52 +1,54 @@
 # frozen_string_literal: true
+
 module Xsv
   module Helpers
     # The default OOXML Spreadheet number formats according to the ECMA standard
     # User formats are appended from index 174 onward
     BUILT_IN_NUMBER_FORMATS = {
-      1 => "0",
-      2 => "0.00",
-      3 => "#, ##0",
-      4 => "#, ##0.00",
-      5 => "$#, ##0_);($#, ##0)",
-      6 => "$#, ##0_);[Red]($#, ##0)",
-      7 => "$#, ##0.00_);($#, ##0.00)",
-      8 => "$#, ##0.00_);[Red]($#, ##0.00)",
-      9 => "0%",
-      10 => "0.00%",
-      11 => "0.00E+00",
-      12 => "# ?/?",
-      13 => "# ??/??",
-      14 => "m/d/yyyy",
-      15 => "d-mmm-yy",
-      16 => "d-mmm",
-      17 => "mmm-yy",
-      18 => "h:mm AM/PM",
-      19 => "h:mm:ss AM/PM",
-      20 => "h:mm",
-      21 => "h:mm:ss",
-      22 => "m/d/yyyy h:mm",
-      37 => "#, ##0_);(#, ##0)",
-      38 => "#, ##0_);[Red](#, ##0)",
-      39 => "#, ##0.00_);(#, ##0.00)",
-      40 => "#, ##0.00_);[Red](#, ##0.00)",
-      45 => "mm:ss",
-      46 => "[h]:mm:ss",
-      47 => "mm:ss.0",
-      48 => "##0.0E+0",
-      49 => "@",
+      1 => '0',
+      2 => '0.00',
+      3 => '#, ##0',
+      4 => '#, ##0.00',
+      5 => '$#, ##0_);($#, ##0)',
+      6 => '$#, ##0_);[Red]($#, ##0)',
+      7 => '$#, ##0.00_);($#, ##0.00)',
+      8 => '$#, ##0.00_);[Red]($#, ##0.00)',
+      9 => '0%',
+      10 => '0.00%',
+      11 => '0.00E+00',
+      12 => '# ?/?',
+      13 => '# ??/??',
+      14 => 'm/d/yyyy',
+      15 => 'd-mmm-yy',
+      16 => 'd-mmm',
+      17 => 'mmm-yy',
+      18 => 'h:mm AM/PM',
+      19 => 'h:mm:ss AM/PM',
+      20 => 'h:mm',
+      21 => 'h:mm:ss',
+      22 => 'm/d/yyyy h:mm',
+      37 => '#, ##0_);(#, ##0)',
+      38 => '#, ##0_);[Red](#, ##0)',
+      39 => '#, ##0.00_);(#, ##0.00)',
+      40 => '#, ##0.00_);[Red](#, ##0.00)',
+      45 => 'mm:ss',
+      46 => '[h]:mm:ss',
+      47 => 'mm:ss.0',
+      48 => '##0.0E+0',
+      49 => '@'
     }.freeze
 
-    MINUTE = 60.freeze
-    HOUR = 3600.freeze
-    A_CODEPOINT = "A".ord.freeze
+    MINUTE = 60
+    HOUR = 3600
+    A_CODEPOINT = 'A'.ord.freeze
     # The epoch for all dates in OOXML Spreadsheet documents
     EPOCH = Date.new(1899, 12, 30).freeze
 
     # Return the index number for the given Excel column name (i.e. "A1" => 0)
     def column_index(col)
       col.each_codepoint.reduce(0) do |sum, n|
-        break sum - 1 if n < A_CODEPOINT  # reached a number
+        break sum - 1 if n < A_CODEPOINT # reached a number
+
         sum * 26 + (n - A_CODEPOINT + 1)
       end
     end
@@ -59,9 +61,7 @@ module Xsv
     # Return a time as a string for the given Excel time value
     def parse_time(number)
       # Disregard date part
-      if number > 0
-        number = number - number.truncate
-      end
+      number -= number.truncate if number.positive?
 
       base = number * 24
 
@@ -70,11 +70,11 @@ module Xsv
 
       # Compensate for rounding errors
       if minutes >= 60
-        hours = hours + (minutes / 60)
+        hours += (minutes / 60)
         minutes = minutes % 60
       end
 
-      "%02d:%02d" % [hours, minutes]
+      format('%02d:%02d', hours, minutes)
     end
 
     # Returns a time including a date as a {Time} object
@@ -92,8 +92,10 @@ module Xsv
 
     # Returns a number as either Integer or Float
     def parse_number(string)
-      if string.include? "."
+      if string.include? '.'
         string.to_f
+      elsif string.include? 'E'
+        Complex(string).to_f
       else
         string.to_i
       end
@@ -103,11 +105,11 @@ module Xsv
     def parse_number_format(number, format)
       number = parse_number(number) if number.is_a?(String)
 
-      if is_datetime_format?(format)
+      if datetime_format?(format)
         parse_datetime(number)
-      elsif is_date_format?(format)
+      elsif date_format?(format)
         parse_date(number)
-      elsif is_time_format?(format)
+      elsif time_format?(format)
         parse_time(number)
       else
         number
@@ -115,20 +117,22 @@ module Xsv
     end
 
     # Tests if the given format string includes both date and time
-    def is_datetime_format?(format)
-      is_date_format?(format) && is_time_format?(format)
+    def datetime_format?(format)
+      date_format?(format) && time_format?(format)
     end
 
     # Tests if the given format string is a date
-    def is_date_format?(format)
+    def date_format?(format)
       return false if format.nil?
+
       # If it contains at least 2 sequences of d's, m's or y's it's a date!
       format.scan(/[dmy]+/).length > 1
     end
 
     # Tests if the given format string is a time
-    def is_time_format?(format)
+    def time_format?(format)
       return false if format.nil?
+
       # If it contains at least 2 sequences of h's, m's or s's it's a time!
       format.scan(/[hms]+/).length > 1
     end

data/lib/xsv/relationships_handler.rb

@@ -1,40 +1,23 @@
 # frozen_string_literal: true
+
 module Xsv
   # RelationshipsHandler parses the "xl/_rels/workbook.xml.rels" file to get the existing relationships.
   # This is used internally  when opening a workbook.
-  class RelationshipsHandler < Ox::Sax
+  class RelationshipsHandler < SaxParser
     def self.get_relations(io)
       relations = []
-      handler = new do |relation|
-        relations << relation
-      end
 
-      Ox.sax_parse(handler, io.read)
-      return relations
-    end
+      new { |relation| relations << relation }.parse(io)
 
-    # Ox::Sax implementation
+      relations
+    end
 
     def initialize(&block)
       @block = block
-      @relationship = {}
-    end
-
-    def start_element(name)
-      @relationship = {} if name == :Relationship
-    end
-
-    def attr(name, value)
-      case name
-      when :Id, :Type, :Target
-        @relationship[name] = value
-      end
     end
 
-    def end_element(name)
-      return unless name == :Relationship
-
-      @block.call(@relationship)
+    def start_element(name, attrs)
+      @block.call(attrs.slice(:Id, :Type, :Target)) if name == 'Relationship'
     end
   end
 end

data/lib/xsv/sax_parser.rb

@@ -0,0 +1,88 @@
+# frozen_string_literal: true
+
+module Xsv
+  class SaxParser
+    ATTR_REGEX = /((\S+)="(.*?)")/m.freeze
+
+    def parse(io)
+      state = :look_start
+      if io.is_a?(String)
+        pbuf = io.dup
+        eof_reached = true
+        must_read = false
+      else
+        pbuf = String.new(capacity: 8192)
+        eof_reached = false
+        must_read = true
+      end
+
+      loop do
+        if must_read
+          begin
+            pbuf << io.sysread(2048)
+          rescue EOFError, TypeError
+            # EOFError is thrown by IO, rubyzip returns nil from sysread on EOF
+            eof_reached = true
+          end
+
+          must_read = false
+        end
+
+        if state == :look_start
+          if (o = pbuf.index('<'))
+            chars = pbuf.slice!(0, o + 1).chop!.force_encoding('utf-8')
+
+            if respond_to?(:characters) && !chars.empty?
+              if chars.index('&')
+                chars.gsub!('&amp;', %q{&})
+                chars.gsub!('&apos;', %q{'})
+                chars.gsub!('&gt;', %q{>})
+                chars.gsub!('&lt;', %q{<})
+                chars.gsub!('&quot;', %q{"})
+              end
+              characters(chars)
+            end
+
+            state = :look_end
+          elsif eof_reached
+            # Discard anything after the last tag in the document
+            break
+          else
+            # Continue loop to read more data into the buffer
+            must_read = true
+            next
+          end
+        end
+
+        if state == :look_end
+          if (o = pbuf.index('>'))
+            if (s = pbuf.index(' ')) && s < o
+              tag_name = pbuf.slice!(0, s + 1).chop!
+              args = pbuf.slice!(0, o - s)
+            else
+              tag_name = pbuf.slice!(0, o + 1).chop!
+              args = nil
+            end
+
+            if tag_name.start_with?('/')
+              end_element(tag_name[1..-1]) if respond_to?(:end_element)
+            else
+              if args.nil?
+                start_element(tag_name, nil)
+              else
+                start_element(tag_name, args.scan(ATTR_REGEX).each_with_object({}) { |m, h| h[m[1].to_sym] = m[2] })
+                end_element(tag_name) if args.end_with?('/') && respond_to?(:end_element)
+              end
+            end
+
+            state = :look_start
+          elsif eof_reached
+            raise 'Malformed XML document, looking for end of tag beyond EOF'
+          else
+            must_read = true
+          end
+        end
+      end
+    end
+  end
+end

data/lib/xsv/shared_strings_parser.rb

@@ -1,13 +1,13 @@
 # frozen_string_literal: true
+
 module Xsv
   # Interpret the sharedStrings.xml file from the workbook
   # This is used internally when opening a sheet.
-  class SharedStringsParser < Ox::Sax
+  class SharedStringsParser < SaxParser
     def self.parse(io)
       strings = []
-      handler = new { |s| strings << s }
-      Ox.sax_parse(handler, io.read, skip: :skip_none)
-      return strings
+      new { |s| strings << s }.parse(io)
+      strings
     end
 
     def initialize(&block)
@@ -15,24 +15,24 @@ module Xsv
       @state = nil
     end
 
-    def start_element(name)
+    def start_element(name, _attrs)
       case name
-      when :si
-        @current_string = ""
-      when :t
+      when 'si'
+        @current_string = ''
+      when 't'
         @state = name
       end
     end
 
-    def text(value)
-      @current_string += value if @state == :t
+    def characters(value)
+      @current_string += value if @state == 't'
     end
 
     def end_element(name)
       case name
-      when :si
+      when 'si'
         @block.call(@current_string)
-      when :t
+      when 't'
         @state = nil
       end
     end

data/lib/xsv/sheet.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module Xsv
   # Sheet represents a single worksheet from a workbook and is normally accessed through {Workbook#sheets}
   #
@@ -39,14 +40,14 @@ module Xsv
       @headers = []
       @mode = :array
       @row_skip = 0
-      @hidden = ids[:state] == "hidden"
+      @hidden = ids[:state] == 'hidden'
 
       @last_row, @column_count = SheetBoundsHandler.get_bounds(@io, @workbook)
     end
 
     # @return [String]
     def inspect
-      "#<#{self.class.name}:#{self.object_id}>"
+      "#<#{self.class.name}:#{object_id}>"
     end
 
     # Returns true if the worksheet is hidden
@@ -60,15 +61,7 @@ module Xsv
 
       handler = SheetRowsHandler.new(@mode, empty_row, @workbook, @row_skip, @last_row, &block)
 
-      # For smaller sheets, memory performance is a lot better if Ox is
-      # handed a string. For larger sheets this leads to awful performance.
-      # This is probably caused by either something in SheetRowsHandler or
-      # the interaction between Zip::InputStream and Ox
-      if @size > 100_000_000
-        Ox.sax_parse(handler, @io)
-      else
-        Ox.sax_parse(handler, @io.read)
-      end
+      handler.parse(@io)
 
       true
     end
@@ -82,7 +75,7 @@ module Xsv
         return row if i == number
       end
 
-      return empty_row
+      empty_row
     end
 
     # Load headers in the top row of the worksheet. After parsing of headers
@@ -112,7 +105,7 @@ module Xsv
       elsif @mode == :hash
         @mode = :array
         headers.tap { @mode = :hash }
-      end
+      end || []
     end
 
     def empty_row

data/lib/xsv/sheet_bounds_handler.rb

@@ -1,9 +1,10 @@
 # frozen_string_literal: true
+
 module Xsv
   # SheetBoundsHandler scans a sheet looking for the outer bounds of the content within.
   # This is used internally when opening a sheet to deal with worksheets that do not
   # have a correct dimension tag.
-  class SheetBoundsHandler < Ox::Sax
+  class SheetBoundsHandler < SaxParser
     include Xsv::Helpers
 
     def self.get_bounds(sheet, workbook)
@@ -18,12 +19,11 @@ module Xsv
       end
 
       sheet.rewind
-      Ox.sax_parse(handler, sheet.read)
 
-      return rows, cols
-    end
+      handler.parse(sheet)
 
-    # Ox::Sax implementation
+      [rows, cols]
+    end
 
     def initialize(trim_empty_rows, &block)
       @block = block
@@ -35,36 +35,22 @@ module Xsv
       @trim_empty_rows = trim_empty_rows
     end
 
-    def start_element(name)
+    def start_element(name, attrs)
       case name
-      when :c
+      when 'c'
         @state = name
-        @cell = nil
-      when :v
+        @cell = attrs[:r]
+      when 'v'
         col = column_index(@cell)
         @maxColumn = col if col > @maxColumn
         @maxRow = @row if @row > @maxRow
-      when :row
+      when 'row'
         @state = name
-        @row = nil
-      when :dimension
+        @row = attrs[:r].to_i
+      when 'dimension'
         @state = name
-      end
-    end
-
-    def end_element(name)
-      if name == :sheetData
-        @block.call(@maxRow, @maxColumn)
-      end
-    end
 
-    def attr(name, value)
-      if @state == :c && name == :r
-        @cell = value
-      elsif @state == :row && name == :r
-        @row = value.to_i
-      elsif @state == :dimension && name == :ref
-        _firstCell, lastCell = value.split(":")
+        _firstCell, lastCell = attrs[:ref].split(':')
 
         if lastCell
           @maxColumn = column_index(lastCell)
@@ -75,5 +61,9 @@ module Xsv
         end
       end
     end
+
+    def end_element(name)
+      @block.call(@maxRow, @maxColumn) if name == 'sheetData'
+    end
   end
 end

data/lib/xsv/sheet_rows_handler.rb

@@ -1,98 +1,58 @@
 # frozen_string_literal: true
+
 module Xsv
   # This is the core worksheet parser, implemented as an Ox::Sax handler. This is
   # used internally to enumerate rows.
-  class SheetRowsHandler < Ox::Sax
+  class SheetRowsHandler < SaxParser
     include Xsv::Helpers
 
-    def format_cell
-      return nil if @current_value.empty?
-
-      case @current_cell[:t]
-      when "s"
-        @workbook.shared_strings[@current_value.to_i]
-      when "str", "inlineStr"
-        @current_value.dup
-      when "e" # N/A
-        nil
-      when nil, "n"
-        if @current_cell[:s]
-          style = @workbook.xfs[@current_cell[:s].to_i]
-          numFmt = @workbook.numFmts[style[:numFmtId].to_i]
-
-          parse_number_format(@current_value, numFmt)
-        else
-          parse_number(@current_value)
-        end
-      when "b"
-        @current_value == "1"
-      else
-        raise Xsv::Error, "Encountered unknown column type #{@current_cell[:t]}"
-      end
-    end
-
-    # Ox::Sax implementation below
-
     def initialize(mode, empty_row, workbook, row_skip, last_row, &block)
-      @block = block
-
-      # :sheetData
-      # :row
-      # :c
-      # :v
-      @state = nil
-
       @mode = mode
       @empty_row = empty_row
       @workbook = workbook
       @row_skip = row_skip
+      @last_row = last_row - @row_skip
+      @block = block
+
+      @state = nil
+
       @row_index = 0
       @current_row = {}
       @current_row_attrs = {}
       @current_cell = {}
       @current_value = String.new
-      @last_row = last_row
 
-      if @mode == :hash
-        @headers = @empty_row.keys
-      end
+      @headers = @empty_row.keys if @mode == :hash
     end
 
-    def start_element(name)
+    def start_element(name, attrs)
       case name
-      when :c
+      when 'c'
         @state = name
-        @current_cell.clear
+        @current_cell = attrs
         @current_value.clear
-      when :v
+      when 'v', 'is'
         @state = name
-      when :row
+      when 'row'
         @state = name
         @current_row = @empty_row.dup
-        @current_row_attrs.clear
+        @current_row_attrs = attrs
+      when 't'
+        @state = nil unless @state == 'is'
       else
         @state = nil
       end
     end
 
-    def text(value)
-      if @state == :v
-        @current_value << value
-      end
-    end
-
-    def attr(name, value)
-      case @state
-      when :c
-        @current_cell[name] = value
-      when :row
-        @current_row_attrs[name] = value
-      end
+    def characters(value)
+      @current_value << value if @state == 'v' || @state == 'is'
     end
 
     def end_element(name)
       case name
-      when :c
+      when 'v'
+        @state = nil
+      when 'c'
         col_index = column_index(@current_cell[:r])
 
         case @mode
@@ -101,28 +61,54 @@ module Xsv
         when :hash
           @current_row[@headers[col_index]] = format_cell
         end
-      when :row
-        @real_row_number = @current_row_attrs[:r].to_i
-        @adjusted_row_number = @real_row_number - @row_skip
+      when 'row'
+        real_row_number = @current_row_attrs[:r].to_i
+        adjusted_row_number = real_row_number - @row_skip
 
-        if @real_row_number <= @row_skip
-          return
-        end
+        return if real_row_number <= @row_skip
 
         @row_index += 1
 
         # Skip first row if we're in hash mode
-        return if @adjusted_row_number == 1 && @mode == :hash
+        return if adjusted_row_number == 1 && @mode == :hash
 
         # Pad empty rows
-        while @row_index < @adjusted_row_number
+        while @row_index < adjusted_row_number
           @block.call(@empty_row)
           @row_index += 1
           next
         end
 
         # Do not return empty trailing rows
-        @block.call(@current_row) unless @row_index > @last_row - @row_skip
+        @block.call(@current_row) unless @row_index > @last_row
+      end
+    end
+
+    private
+
+    def format_cell
+      return nil if @current_value.empty?
+
+      case @current_cell[:t]
+      when 's'
+        @workbook.shared_strings[@current_value.to_i]
+      when 'str', 'inlineStr'
+        @current_value.strip
+      when 'e' # N/A
+        nil
+      when nil, 'n'
+        if @current_cell[:s]
+          style = @workbook.xfs[@current_cell[:s].to_i]
+          numFmt = @workbook.numFmts[style[:numFmtId].to_i]
+
+          parse_number_format(@current_value, numFmt)
+        else
+          parse_number(@current_value)
+        end
+      when 'b'
+        @current_value == '1'
+      else
+        raise Xsv::Error, "Encountered unknown column type #{@current_cell[:t]}"
       end
     end
   end

data/lib/xsv/sheets_ids_handler.rb

@@ -1,56 +1,23 @@
 # frozen_string_literal: true
+
 module Xsv
   # SheetsIdsHandler interprets the relevant parts of workbook.xml
   # This is used internally to get the sheets ids, relationship_ids, and names when opening a workbook.
-  class SheetsIdsHandler < Ox::Sax
+  class SheetsIdsHandler < SaxParser
     def self.get_sheets_ids(io)
       sheets_ids = []
-      handler = new do |sheet_ids|
-        sheets_ids << sheet_ids
-      end
 
-      Ox.sax_parse(handler, io.read)
-      return sheets_ids
-    end
+      new { |sheet_ids| sheets_ids << sheet_ids }.parse(io)
 
-    # Ox::Sax implementation
+      sheets_ids
+    end
 
     def initialize(&block)
       @block = block
-      @parsing = false
-    end
-
-    def start_element(name)
-      if name == :sheets
-        @parsing = true
-        return
-      end
-
-      return unless name == :sheet
-
-      @sheet_ids = {}
-    end
-
-    def attr(name, value)
-      return unless @parsing
-
-      case name
-      when :name, :sheetId, :state
-        @sheet_ids[name] = value
-      when :'r:id'
-        @sheet_ids[:r_id] = value
-      end
     end
 
-    def end_element(name)
-      if name == :sheets
-        @parsing = false
-        return
-      end
-
-      return unless name == :sheet
-
-      @block.call(@sheet_ids)
+    def start_element(name, attrs)
+      @block.call(attrs.slice(:name, :sheetId, :state, :'r:id')) if name == 'sheet'
     end
   end
 end

data/lib/xsv/styles_handler.rb

@@ -1,59 +1,43 @@
 # frozen_string_literal: true
+
 module Xsv
   # StylesHandler interprets the relevant parts of styles.xml
   # This is used internally when opening a sheet.
-  class StylesHandler < Ox::Sax
-    def self.get_styles(io, numFmts)
-      @xfs = nil
-      @numFmts = nil
-      handler = new(numFmts) do |xfs, numFmts|
+  class StylesHandler < SaxParser
+    def self.get_styles(io)
+      handler = new(Xsv::Helpers::BUILT_IN_NUMBER_FORMATS.dup) do |xfs, numFmts|
         @xfs = xfs
         @numFmts = numFmts
       end
 
-      Ox.sax_parse(handler, io.read)
-      return @xfs, @numFmts
-    end
+      handler.parse(io)
 
-    # Ox::Sax implementation
+      [@xfs, @numFmts]
+    end
 
     def initialize(numFmts, &block)
       @block = block
       @state = nil
       @xfs = []
       @numFmts = numFmts
-
-      @xf = {}
-      @numFmt = {}
     end
 
-    def start_element(name)
+    def start_element(name, attrs)
       case name
-      when :cellXfs, :numFmts
-        @state = name
-      when :xf
-        @xf = {}
-      when :numFmt
-        @numFmt = {}
-      end
-    end
-
-    def attr(name, value)
-      case @state
-      when :cellXfs
-        @xf[name] = value
-      when :numFmts
-        @numFmt[name] = value
+      when 'cellXfs'
+        @state = 'cellXfs'
+      when 'xf'
+        @xfs << attrs if @state == 'cellXfs'
+      when 'numFmt'
+        @numFmts[attrs[:numFmtId].to_i] = attrs[:formatCode]
       end
     end
 
     def end_element(name)
-      if @state == :cellXfs && name == :xf
-        @xfs << @xf
-      elsif @state == :numFmts && name == :numFmt
-        @numFmts[@numFmt[:numFmtId].to_i] = @numFmt[:formatCode]
-      elsif name == :styleSheet
+      if name == 'styleSheet'
         @block.call(@xfs, @numFmts)
+      elsif name == 'cellXfs'
+        @state = nil
       end
     end
   end

data/lib/xsv/version.rb

@@ -1,4 +1,5 @@
 # frozen_string_literal: true
+
 module Xsv
-  VERSION = "0.3.15"
+  VERSION = '1.0.0.pre.2'
 end

data/lib/xsv/workbook.rb

@@ -1,11 +1,11 @@
 # frozen_string_literal: true
-require "zip"
+
+require 'zip'
 
 module Xsv
   # An OOXML Spreadsheet document is called a Workbook. A Workbook consists of
   # multiple Sheets that are available in the array that's accessible through {#sheets}
   class Workbook
-
     # Access the Sheet objects contained in the workbook
     # @return [Array<Sheet>]
     attr_reader :sheets
@@ -15,13 +15,13 @@ module Xsv
     # Open the workbook of the given filename, string or buffer. For additional
     # options see {.initialize}
     def self.open(data, **kws)
-      if data.is_a?(IO) || data.respond_to?(:read) # is it a buffer?
-        @workbook = self.new(Zip::File.open_buffer(data), **kws)
-      elsif data.start_with?("PK\x03\x04") # is it a string containing a filename?
-        @workbook = self.new(Zip::File.open_buffer(data), **kws)
-      else # must be a filename
-        @workbook = self.new(Zip::File.open(data), **kws)
-      end
+      @workbook = if data.is_a?(IO) || data.respond_to?(:read) # is it a buffer?
+                    new(Zip::File.open_buffer(data), **kws)
+                  elsif data.start_with?("PK\x03\x04") # is it a string containing a file?
+                    new(Zip::File.open_buffer(data), **kws)
+                  else # must be a filename
+                    new(Zip::File.open(data), **kws)
+                  end
     end
 
     # Open a workbook from an instance of {Zip::File}. Generally it's recommended
@@ -36,19 +36,16 @@ module Xsv
       @trim_empty_rows = trim_empty_rows
 
       @sheets = []
-      @xfs = []
-      @numFmts = Xsv::Helpers::BUILT_IN_NUMBER_FORMATS.dup
-
-      fetch_shared_strings
-      fetch_styles
-      fetch_sheets_ids
-      fetch_relationships
-      fetch_sheets
+      @xfs, @numFmts = fetch_styles
+      @sheet_ids = fetch_sheet_ids
+      @relationships = fetch_relationships
+      @shared_strings = fetch_shared_strings
+      @sheets = fetch_sheets
     end
 
     # @return [String]
     def inspect
-      "#<#{self.class.name}:#{self.object_id}>"
+      "#<#{self.class.name}:#{object_id}>"
     end
 
     # Close the handle to the workbook file and leave all resources for the GC to collect
@@ -60,7 +57,7 @@ module Xsv
       @numFmts = nil
       @relationships = nil
       @shared_strings = nil
-      @sheets_ids = nil
+      @sheet_ids = nil
 
       true
     end
@@ -75,42 +72,44 @@ module Xsv
     private
 
     def fetch_shared_strings
-      handle = @zip.glob("xl/sharedStrings.xml").first
+      handle = @zip.glob('xl/sharedStrings.xml').first
       return if handle.nil?
 
       stream = handle.get_input_stream
-      @shared_strings = SharedStringsParser.parse(stream)
-
-      stream.close
+      SharedStringsParser.parse(stream)
+    ensure
+      stream.close if stream
     end
 
     def fetch_styles
-      stream = @zip.glob("xl/styles.xml").first.get_input_stream
+      stream = @zip.glob('xl/styles.xml').first.get_input_stream
 
-      @xfs, @numFmts = StylesHandler.get_styles(stream, @numFmts)
+      StylesHandler.get_styles(stream)
+    ensure
+      stream.close
     end
 
     def fetch_sheets
-      @zip.glob("xl/worksheets/sheet*.xml").sort do |a, b|
+      @zip.glob('xl/worksheets/sheet*.xml').sort do |a, b|
         a.name[/\d+/].to_i <=> b.name[/\d+/].to_i
-      end.each do |entry|
-        rel = @relationships.detect { |r| entry.name.end_with?(r[:Target]) && r[:Type].end_with?("worksheet") }
-        sheet_ids = @sheets_ids.detect { |i| i[:r_id] == rel[:Id] }
-        @sheets << Xsv::Sheet.new(self, entry.get_input_stream, entry.size, sheet_ids)
+      end.map do |entry|
+        rel = @relationships.detect { |r| entry.name.end_with?(r[:Target]) && r[:Type].end_with?('worksheet') }
+        sheet_ids = @sheet_ids.detect { |i| i[:"r:id"] == rel[:Id] }
+        Xsv::Sheet.new(self, entry.get_input_stream, entry.size, sheet_ids)
       end
     end
 
-    def fetch_sheets_ids
-      stream = @zip.glob("xl/workbook.xml").first.get_input_stream
-      @sheets_ids = SheetsIdsHandler.get_sheets_ids(stream)
-
+    def fetch_sheet_ids
+      stream = @zip.glob('xl/workbook.xml').first.get_input_stream
+      SheetsIdsHandler.get_sheets_ids(stream)
+    ensure
       stream.close
     end
 
     def fetch_relationships
-      stream = @zip.glob("xl/_rels/workbook.xml.rels").first.get_input_stream
-      @relationships = RelationshipsHandler.get_relations(stream)
-
+      stream = @zip.glob('xl/_rels/workbook.xml.rels').first.get_input_stream
+      RelationshipsHandler.get_relations(stream)
+    ensure
       stream.close
     end
   end

data/xsv.gemspec

@@ -36,12 +36,11 @@ Gem::Specification.new do |spec|
   spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
   spec.require_paths = ["lib"]
 
-  spec.required_ruby_version = "~> 2.5"
+  spec.required_ruby_version = ">= 2.5"
 
   spec.add_dependency "rubyzip", ">= 1.3", "< 3"
-  spec.add_dependency "ox", ">= 2.9"
 
   spec.add_development_dependency "bundler", "< 3"
   spec.add_development_dependency "rake", "~> 13.0"
-  spec.add_development_dependency "minitest", "~> 5.0"
+  spec.add_development_dependency "minitest", "~> 5.14.2"
 end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: xsv
 version: !ruby/object:Gem::Version
-  version: 0.3.15
+  version: 1.0.0.pre.2
 platform: ruby
 authors:
 - Martijn Storck
-autorequire: 
+autorequire:
 bindir: exe
 cert_chain: []
-date: 2020-06-02 00:00:00.000000000 Z
+date: 2021-01-22 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rubyzip
@@ -30,20 +30,6 @@ dependencies:
     - - "<"
       - !ruby/object:Gem::Version
         version: '3'
-- !ruby/object:Gem::Dependency
-  name: ox
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '2.9'
-  type: :runtime
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - ">="
-      - !ruby/object:Gem::Version
-        version: '2.9'
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -78,14 +64,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '5.0'
+        version: 5.14.2
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '5.0'
+        version: 5.14.2
 description: |2
                               Xsv is a fast, lightweight parser for Office Open XML spreadsheet files
                               (commonly known as Excel or .xlsx files). It strives to be minimal in the
@@ -109,6 +95,7 @@ files:
 - lib/xsv.rb
 - lib/xsv/helpers.rb
 - lib/xsv/relationships_handler.rb
+- lib/xsv/sax_parser.rb
 - lib/xsv/shared_strings_parser.rb
 - lib/xsv/sheet.rb
 - lib/xsv/sheet_bounds_handler.rb
@@ -125,23 +112,23 @@ metadata:
   homepage_uri: https://github.com/martijn/xsv
   source_code_uri: https://github.com/martijn/xsv
   changelog_uri: https://github.com/martijn/xsv/CHANGELOG.md
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
 required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
-  - - "~>"
+  - - ">="
     - !ruby/object:Gem::Version
       version: '2.5'
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
-  - - ">="
+  - - ">"
     - !ruby/object:Gem::Version
-      version: '0'
+      version: 1.3.1
 requirements: []
-rubygems_version: 3.1.2
-signing_key: 
+rubygems_version: 3.2.3
+signing_key:
 specification_version: 4
 summary: A fast and lightweiggt xlsx parser that provides nothing a CSV parser wouldn't
 test_files: []