calx 0.1.0-x86_64-linux

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 811dfc464f10bee639f21aa1909980c35ea6def130e1d66bed88e26ac191d9f2
+  data.tar.gz: e92ae5d0e5e0d6a73b56947627ac0894b3adba9efc591200f9c0a0a293eafe1a
+SHA512:
+  metadata.gz: 45242529912a2e7ece7591931ac867c783f9d6ff359931ec56d75101e24b007c65fc6cc2f229064a5b79d32278ce1ed20958c400e483428e1ee77fac40c04116
+  data.tar.gz: 799e40aad7093263ec494a5606aa3c5d7732ec6dca35700d310746c71c4e9624f488bd2b464e808c6662cd7a16786a2bffa6f38e74492a19acb7c00720dca245

data/CHANGELOG.md

@@ -0,0 +1,7 @@
+# Changelog
+
+## [Unreleased]
+
+## [0.1.0] - 2026-11-05
+
+- First release: `Calx.each_row`, `Calx.read` and `Calx.sheet_names` for CSV, TSV, xlsx, xlsm, xlsb, xls and ods files.

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Zac
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,145 @@
+# calx
+
+[![CI](https://github.com/7a6163/calx/actions/workflows/main.yml/badge.svg)](https://github.com/7a6163/calx/actions/workflows/main.yml)
+[![codecov](https://codecov.io/gh/7a6163/calx/graph/badge.svg)](https://codecov.io/gh/7a6163/calx)
+
+Read CSV, TSV and spreadsheets (xlsx, xlsm, xlsb, xls, ods) row by row from Ruby with a Rust engine ([csv](https://github.com/BurntSushi/rust-csv) and [calamine](https://github.com/tafia/calamine), via [magnus](https://github.com/matsadler/magnus)). The CSV reader is 4–16× faster than Ruby's `CSV`, and the xlsx reader 10–27× faster than simple_xlsx_reader, roo and creek ([benchmarks](#performance)).
+
+It only reads. To write xlsx files, see its sibling [fast_xlsx](https://github.com/7a6163/fast_xlsx).
+
+> **Status: 0.x.** The API may still change before 1.0.
+
+## Installation
+
+```bash
+bundle add calx
+```
+
+Requires CRuby 3.3 or later. Installing from source needs a Rust toolchain.
+
+## Usage
+
+```ruby
+require "calx"
+
+# One Array per row; the format comes from the file extension.
+Calx.each_row("sales.csv") { |row| p row }    # ["North", "120", "9.5"]
+Calx.each_row("sales.xlsx") { |row| p row }   # ["North", 120, 9.5, #<Date 2026-07-03>]
+
+rows = Calx.read("sales.xlsx", sheet: "July") # every row at once
+Calx.sheet_names("sales.xlsx")                # ["June", "July"]
+Calx.each_row("big.csv").first(10)            # an Enumerator without a block
+```
+
+Options:
+
+| Option | For | |
+|---|---|---|
+| `format:` | all | `:csv`, `:tsv`, `:xlsx`, `:xlsm`, `:xlsb`, `:xls` or `:ods`, for files whose extension doesn't tell (uploads saved as temp files) |
+| `sheet:` | spreadsheets | the sheet's index (from 0) or name; the first sheet by default |
+| `col_sep:` | CSV | a one-byte separator; `","` (`"\t"` for `.tsv`) by default |
+| `encoding:` | CSV | the file's encoding, such as `"Big5"` or `"Shift_JIS"`; UTF-8 by default |
+
+`path` can be a String or anything with `#to_path` (Pathname, File, Tempfile, Rails' `UploadedFile`).
+
+### What you get back
+
+- **CSV**: every cell is a String, as with Ruby's `CSV`. A UTF-8 byte order mark is skipped. Rows can have different lengths. A file that isn't valid UTF-8 raises `Calx::Error`, with a hint to pass `encoding:`.
+- **Spreadsheets**: String, Integer, Float, `true`/`false`, `Date`, `Time` or `nil`.
+  - Whole numbers come back as Integer (xlsx stores `42` and `42.0` the same way).
+  - A date with no time of day is a `Date`; one with a time is a local `Time` (Excel stores no time zone).
+  - Durations (`[h]:mm` formats) are the Float number of days Excel stores; in ods files they are ISO 8601 Strings such as `"PT10H10M10S"`.
+  - Formulas give the value Excel last saved. Error cells give their text, such as `"#N/A"`.
+  - Rows count from the sheet's first row and `row[0]` is always column A, even when the data starts lower: rows above it are `[]` and cells left of it `nil`. Every other row is padded with `nil` to the width of the used range.
+
+### Errors
+
+`TypeError` for an argument of the wrong type, `ArgumentError` for a value that isn't allowed (an unknown format, sheet name or encoding), `IndexError` for a sheet index past the last sheet, `Errno::ENOENT` for a missing file, and `Calx::Error` for a file that can't be read.
+
+### Limits
+
+- Read only: no styles, comments or merged-cell ranges, and no editing an existing file.
+- A spreadsheet's sheet is loaded into memory whole before its rows are yielded. A CSV file is read as it goes, except with `encoding:`, which reads the whole file to transcode it.
+
+## Performance
+
+200,000 rows read with each library, Ruby 3.4.11 with YJIT, Apple Silicon (arm64). Best of 5 runs for CSV, 3 for xlsx; every run must read all 200,000 rows. Lower is better.
+
+**CSV, 10 columns of numbers and short strings** (12.5 MB)
+
+| Library | Method | Time | vs calx |
+|---|---|---:|---:|
+| **calx** 0.1.0 | `Calx.each_row` | **51 ms** | 1.0× |
+| **calx** 0.1.0 | `Calx.read` | **49 ms** | 1.0× |
+| csv 3.3.7 | `CSV.foreach` | 231 ms | 4.5× slower |
+| csv 3.3.7 | `CSV.read` | 327 ms | 6.4× slower |
+
+**CSV, 10 columns with quotes, commas, newlines and CJK** (26.2 MB)
+
+| Library | Method | Time | vs calx |
+|---|---|---:|---:|
+| **calx** 0.1.0 | `Calx.each_row` | **64 ms** | 1.0× |
+| **calx** 0.1.0 | `Calx.read` | **64 ms** | 1.0× |
+| csv 3.3.7 | `CSV.foreach` | 865 ms | 13.4× slower |
+| csv 3.3.7 | `CSV.read` | 1,012 ms | 15.7× slower |
+
+**xlsx, 5 columns: integer, string, float, date, string** (6.0 MB, written by fast_xlsx)
+
+| Library | Method | Time | vs calx |
+|---|---|---:|---:|
+| **calx** 0.1.0 | `Calx.each_row` | **222 ms** | 1.0× |
+| fastsheet 0.2.0 (GitHub) | `Sheet.new(path).rows` | 281 ms | 1.3× slower |
+| simple_xlsx_reader 5.1.0 | `rows.each` | 2,411 ms | 10.8× slower |
+| roo 4.0.0 | `each_row_streaming` | 5,117 ms | 23.0× slower |
+| creek 2.6.3 | `simple_rows` | 6,014 ms | 27.0× slower |
+
+Ruby's `CSV` slows down most on quoted fields, which real exports and uploads usually have. The libraries don't return quite the same thing: roo yields cell objects and creek yields a Hash per row, while calx and simple_xlsx_reader yield Arrays of values.
+
+fastsheet also uses calamine, so it is close on speed, but it builds every row before returning any and its values differ: whole numbers come back as Float (`1.0`), dates as a `Time` at midnight UTC (so `2026-01-02 08:00 +0800` in Taipei, and the previous day west of UTC), and error cells as `nil`. It reads the first sheet only and no CSV. Its current version is not on RubyGems (0.1.1 there is from 2017), so the benchmark includes it only when it is installed from [GitHub](https://github.com/dmk/fastsheet).
+
+Most of calx's time goes into making the Ruby Strings and Arrays: parsing alone is about 5× faster again. Times move by 10–30% between runs, so treat the ratios as rough. Run it yourself:
+
+```bash
+bundle exec rake compile
+BUNDLE_GEMFILE=bench/Gemfile bundle install
+BUNDLE_GEMFILE=bench/Gemfile bundle exec ruby --yjit bench/read.rb [rows]
+```
+
+## Development
+
+```bash
+bin/setup
+bundle exec rake           # compile + test + rubocop
+bundle exec rbs -r date -I sig validate
+cargo clippy --manifest-path ext/calx/Cargo.toml --all-targets -- -D warnings
+```
+
+All behaviour is tested from Ruby (`test/`); there are no Rust unit tests.
+
+Coverage of the Rust extension while the Ruby tests run (needs [cargo-llvm-cov](https://github.com/taiki-e/cargo-llvm-cov) and `rustup component add llvm-tools-preview`). CI uploads it to Codecov.
+
+```bash
+rm -rf tmp lib/calx/calx.bundle         # force an instrumented rebuild
+eval "$(cargo llvm-cov show-env --sh)"
+bundle exec rake compile test
+cargo llvm-cov report --release                     # or --lcov / --html
+```
+
+Rebuild in a clean shell afterwards (`rm -rf tmp && bundle exec rake compile`) so the everyday build is not instrumented.
+
+Mutation tests ([cargo-mutants](https://mutants.rs)) change the Rust source one small edit at a time and check that a Ruby test fails for each change. `ext/calx/tests/ruby_suite.rs` connects the two: it rebuilds the extension and runs the Ruby suite, so `cargo test` covers the Ruby tests too. Mutants that can't change behaviour are excluded in `.cargo/mutants.toml`.
+
+```bash
+cargo install cargo-mutants
+cargo mutants --file ext/calx/src/lib.rs --jobs 4   # ~1.5 minutes; results in mutants.out/
+```
+
+Mutants listed in `mutants.out/missed.txt` point at behaviour no test checks. The `Mutation tests` workflow runs this weekly and on demand. Dependabot opens weekly update PRs for the gems, crates and GitHub Actions.
+
+The xls, xlsb, ods and some xlsx files in `test/fixtures` come from calamine's test suite (MIT).
+
+Releasing: bump `lib/calx/version.rb`, update the CHANGELOG, and push a `vX.Y.Z` tag. The `Build gems` workflow builds the platform gems and publishes them to RubyGems.
+
+## License
+
+MIT

data/Rakefile

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "minitest/test_task"
+
+Minitest::TestTask.create
+
+require "rubocop/rake_task"
+
+RuboCop::RakeTask.new
+
+require "rb_sys/extensiontask"
+
+task build: :compile
+
+GEMSPEC = Gem::Specification.load("calx.gemspec")
+
+RbSys::ExtensionTask.new("calx", GEMSPEC) do |ext|
+  ext.lib_dir = "lib/calx"
+end
+
+task default: %i[compile test rubocop]

data/lib/calx/version.rb

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+module Calx
+  # The gem version.
+  VERSION = "0.1.0"
+end

data/lib/calx.rb

@@ -0,0 +1,141 @@
+# frozen_string_literal: true
+
+require "date"
+require "time"
+require_relative "calx/version"
+
+# Reads CSV, TSV and spreadsheets (xlsx, xlsm, xlsb, xls, ods) row by row
+# with a Rust core (the csv and calamine crates).
+#
+# Errors: TypeError for an argument of the wrong type, ArgumentError for a
+# value that isn't allowed, IndexError for a sheet index past the last sheet,
+# Errno::ENOENT for a missing file, and Calx::Error for a file that
+# can't be read.
+module Calx
+  # Raised when a file can't be read: corrupt, the wrong format, or (CSV) not
+  # in the encoding given.
+  class Error < StandardError; end
+
+  # The native extension looks up Calx::Error, so load it after Error is defined.
+  # Precompiled gems ship one binary per Ruby version (calx/3.4/calx);
+  # a gem compiled on install has a single calx/calx.
+  begin
+    require "calx/#{RUBY_VERSION[/\d+\.\d+/]}/calx"
+  rescue LoadError
+    require "calx/calx"
+  end
+
+  # File extensions and the format each one is read as.
+  FORMATS = {
+    ".csv" => :csv, ".tsv" => :tsv,
+    ".xlsx" => :xlsx, ".xlsm" => :xlsm, ".xlsb" => :xlsb, ".xls" => :xls, ".ods" => :ods
+  }.freeze
+
+  class << self
+    # Yields each row of a sheet as an Array.
+    #
+    # CSV and TSV cells are Strings. Spreadsheet cells are String, Integer,
+    # Float, true/false, Date (a date with no time of day), Time (local time:
+    # Excel stores no zone) or nil (empty); error cells are their text, such as
+    # "#N/A", and formulas give the value Excel last saved. Row 0 is the
+    # sheet's first row and row[0] its column A, even when the data starts lower.
+    #
+    # @param path [String, #to_path] the file (a Pathname, File or Tempfile works)
+    # @param format [Symbol, nil] one of the values of {FORMATS}; by default
+    #   taken from the file extension
+    # @param sheet [Integer, String, nil] spreadsheets only: the sheet's index
+    #   (from 0) or name; the first sheet by default
+    # @param col_sep [String, nil] CSV only: the one-byte separator; "," for
+    #   CSV and "\t" for TSV by default
+    # @param encoding [String, Encoding, nil] CSV only: the file's encoding,
+    #   such as "Big5" or "Shift_JIS"; UTF-8 by default. A UTF-8 byte order
+    #   mark is skipped.
+    # @yieldparam row [Array]
+    # @return [nil, Enumerator] an Enumerator when no block is given
+    def each_row(path, format: nil, sheet: nil, col_sep: nil, encoding: nil, &block)
+      return enum_for(__method__, path, format:, sheet:, col_sep:, encoding:) unless block
+
+      path = file_path(path)
+      case (format = format_of(path, format))
+      when :csv, :tsv
+        raise ArgumentError, "sheet: is only for spreadsheets" if sheet
+
+        each_csv_row(path, format, col_sep, encoding, &block)
+      else
+        raise ArgumentError, "col_sep: and encoding: are only for CSV files" if col_sep || encoding
+
+        index, name = sheet_ref(sheet)
+        _each_workbook_row(path, format.to_s, index, name, &block)
+      end
+      nil
+    end
+
+    # Every row of a sheet; takes the options of {each_row}.
+    # @return [Array<Array>]
+    def read(path, **)
+      each_row(path, **).to_a
+    end
+
+    # The names of a spreadsheet's sheets, in order.
+    # @param path [String, #to_path]
+    # @param format [Symbol, nil] as in {each_row}
+    # @return [Array<String>]
+    def sheet_names(path, format: nil)
+      path = file_path(path)
+      format = format_of(path, format)
+      raise ArgumentError, "a #{format.upcase} file has no sheets" if %i[csv tsv].include?(format)
+
+      _sheet_names(path, format.to_s)
+    end
+
+    private
+
+    def file_path(path)
+      path = path.to_path if path.respond_to?(:to_path)
+      raise TypeError, "path must be a String or respond to #to_path, not #{path.class}" unless path.is_a?(String)
+      raise Errno::ENOENT, path unless File.file?(path)
+
+      path
+    end
+
+    def format_of(path, format)
+      return FORMATS.fetch(File.extname(path).downcase) { raise ArgumentError, unknown_format(path) } unless format
+      unless FORMATS.value?(format)
+        raise ArgumentError,
+              "unknown format #{format.inspect} (#{FORMATS.values.join(", ")})"
+      end
+
+      format
+    end
+
+    def unknown_format(path)
+      "can't tell the format of #{path.inspect} from its extension; pass format: (#{FORMATS.values.join(", ")})"
+    end
+
+    def sheet_ref(sheet)
+      case sheet
+      when nil then [nil, nil]
+      when String then [nil, sheet]
+      when Integer
+        raise IndexError, "sheet index #{sheet} is negative" if sheet.negative?
+
+        [sheet, nil]
+      else raise TypeError, "sheet must be an Integer or String, not #{sheet.class}"
+      end
+    end
+
+    def each_csv_row(path, format, col_sep, encoding, &)
+      col_sep ||= format == :tsv ? "\t" : ","
+      raise TypeError, "col_sep must be a String, not #{col_sep.class}" unless col_sep.is_a?(String)
+      raise ArgumentError, "col_sep must be one byte, not #{col_sep.inspect}" unless col_sep.bytesize == 1
+
+      # Transcoded in Ruby, then parsed as UTF-8: the whole file is read into memory.
+      if encoding
+        encoding = Encoding.find(encoding) # ArgumentError for an unknown name
+        bom = "BOM|" if encoding.name.start_with?("UTF-")
+        data = File.read(path, encoding: "#{bom}#{encoding}:UTF-8")
+      end
+      _each_csv_row(path, data, col_sep.ord, &)
+    end
+  end
+end

data/sig/calx.rbs

@@ -0,0 +1,21 @@
+module Calx
+  VERSION: String
+  FORMATS: Hash[String, Symbol]
+
+  class Error < StandardError
+  end
+
+  type path = String | _ToPath
+  type cell = String | Integer | Float | bool | Date | Time | nil
+
+  interface _ToPath
+    def to_path: () -> String
+  end
+
+  def self.each_row: (path path, ?format: Symbol?, ?sheet: (Integer | String)?, ?col_sep: String?, ?encoding: (String | Encoding)?) { (Array[cell] row) -> void } -> nil
+                   | (path path, ?format: Symbol?, ?sheet: (Integer | String)?, ?col_sep: String?, ?encoding: (String | Encoding)?) -> Enumerator[Array[cell], nil]
+
+  def self.read: (path path, ?format: Symbol?, ?sheet: (Integer | String)?, ?col_sep: String?, ?encoding: (String | Encoding)?) -> Array[Array[cell]]
+
+  def self.sheet_names: (path path, ?format: Symbol?) -> Array[String]
+end

metadata

@@ -0,0 +1,62 @@
+--- !ruby/object:Gem::Specification
+name: calx
+version: !ruby/object:Gem::Version
+  version: 0.1.0
+platform: x86_64-linux
+authors:
+- Zac
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2026-11-05 00:00:00.000000000 Z
+dependencies: []
+description: Reads CSV, TSV, xlsx, xlsm, xlsb, xls and ods files row by row through
+  a native Rust extension (the csv and calamine crates), several times faster than
+  Ruby's CSV.
+email:
+- 579103+7a6163@users.noreply.github.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- CHANGELOG.md
+- LICENSE.txt
+- README.md
+- Rakefile
+- lib/calx.rb
+- lib/calx/3.3/calx.so
+- lib/calx/3.4/calx.so
+- lib/calx/4.0/calx.so
+- lib/calx/version.rb
+- sig/calx.rbs
+homepage: https://github.com/7a6163/calx
+licenses:
+- MIT
+metadata:
+  homepage_uri: https://github.com/7a6163/calx
+  source_code_uri: https://github.com/7a6163/calx
+  changelog_uri: https://github.com/7a6163/calx/blob/main/CHANGELOG.md
+  rubygems_mfa_required: 'true'
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '3.3'
+  - - "<"
+    - !ruby/object:Gem::Version
+      version: 4.1.dev
+required_rubygems_version: !ruby/object:Gem::Requirement
+  requirements:
+  - - ">="
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubygems_version: 3.5.23
+signing_key: 
+specification_version: 4
+summary: Fast CSV and spreadsheet reader for Ruby with a Rust engine
+test_files: []