calcsheet-native 0.1.0

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: 24c9944f46380b2013bd15f70dbe021f4ce3405957be54ee7051296dbf8f92a7
+  data.tar.gz: a2c957b36974b768d4c1bdf51c982d1c208ce36ca9e566afbf74e68b181eba94
+SHA512:
+  metadata.gz: 27bec1c0bb46629cd1caff21c105b2022ccbe21f393dc1a95a37c8eb342a103ffd5dfd04a0933e320b440aab4bcf2565324fa14f57a49c367fa522a355fcff25
+  data.tar.gz: 8504e11364610d90744222fd790e8eae2f3e61276140fd2877ca646da5095b2d7df5ee69713c1bb3408d52800648fcd22098453e0d0f329f61e8022d202d3c87

data/CHANGELOG.md

@@ -0,0 +1,36 @@
+# Changelog
+
+All notable changes to this gem are documented in this file.
+
+The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/), and this gem adheres to
+[Semantic Versioning](https://semver.org/spec/v2.0.0.html).
+
+## [Unreleased]
+
+## [0.1.0] - 2026-10-28
+
+First release, for calcsheet 0.2 and 0.3 (protocol 1), with precompiled gems for `x86_64-linux`, `aarch64-linux`,
+`x86_64-linux-musl`, `aarch64-linux-musl`, `arm64-darwin`, `x86_64-darwin` and `x64-mingw-ucrt` (Ruby 3.3, 3.4 and
+4.0) beside the source gem.
+
+### Added
+
+- The gem skeleton: a Rust extension built with magnus and rb_sys, `Calcsheet::Native::PROTOCOL` 1 and
+  `Calcsheet::Native.backends`.
+- The native CSV backend: a Rust tokenizer that accepts and refuses what the stdlib CSV parser does, run without the
+  GVL, under calcsheet's own CSV reader (dialect, encodings, numbering, checks and conversion). About twice as fast
+  as the Ruby backend on large files, with the same memory.
+- The native XLSX backend: the XML of each sheet is read in Rust, without the GVL, for a plain subset of XML and the
+  values calcsheet accepts; anything else is read again by calcsheet's own parser, so results and errors are always
+  the Ruby backend's. The archive, its limits, the workbook, styles and shared strings stay calcsheet's. About 1.7
+  times as fast as the Ruby backend, with the same memory.
+- The native ODS backend, built the same way: the XML of each table is read in Rust, the Scanner that finds tables
+  and rows, the archive and the cell styles stay calcsheet's. About twice as fast as the Ruby backend, with the same
+  memory.
+- XLSX shared strings read in Rust, in one table shared by the sheets, and a faster XML lexer: with YJIT, XLSX
+  reads in 8.5 s per million rows instead of 10.3 s, ODS in 12.3 s instead of 13.6 s.
+- An ODS read finds its sheet without listing every sheet (calcsheet's ADR 013): with YJIT, 9.1 s per million rows
+  instead of 12.3 s, and about 40 MB of peak memory instead of 100.
+
+[Unreleased]: https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet-native/-/compare/v0.1.0...develop
+[0.1.0]: https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet-native/-/tags/v0.1.0

data/Cargo.lock

@@ -0,0 +1,283 @@
+# This file is automatically @generated by Cargo.
+# It is not intended for manual editing.
+version = 4
+
+[[package]]
+name = "aho-corasick"
+version = "1.1.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
+dependencies = [
+ "memchr",
+]
+
+[[package]]
+name = "bindgen"
+version = "0.72.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "993776b509cfb49c750f11b8f07a46fa23e0a1386ffc01fb1e7d343efc387895"
+dependencies = [
+ "bitflags",
+ "cexpr",
+ "clang-sys",
+ "itertools",
+ "proc-macro2",
+ "quote",
+ "regex",
+ "rustc-hash",
+ "shlex",
+ "syn",
+]
+
+[[package]]
+name = "bitflags"
+version = "2.13.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
+
+[[package]]
+name = "calcsheet_native"
+version = "0.1.0"
+dependencies = [
+ "magnus",
+ "memchr",
+ "rb-sys",
+]
+
+[[package]]
+name = "cexpr"
+version = "0.6.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6fac387a98bb7c37292057cffc56d62ecb629900026402633ae9160df93a8766"
+dependencies = [
+ "nom",
+]
+
+[[package]]
+name = "cfg-if"
+version = "1.0.5"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "4e7648175b45a9a48536d676f68d918270699102aa8dab5496df06904c914600"
+
+[[package]]
+name = "clang-sys"
+version = "1.9.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "157a8ba7b480713b56f4c09fd13fc3e0a22a5dfab8097ba61cbc5feef950788a"
+dependencies = [
+ "glob",
+ "libc",
+ "libloading",
+]
+
+[[package]]
+name = "either"
+version = "1.19.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0e9c71c2167ca323c882b99918929403426e2373ea17242ff5653e0d5e1058be"
+
+[[package]]
+name = "glob"
+version = "0.3.4"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "e4eba85ea1d0a966a983acd07deee566e67395d2d96b6fb39e62b5a833f1eb0b"
+
+[[package]]
+name = "itertools"
+version = "0.13.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "413ee7dfc52ee1a4949ceeb7dbc8a33f2d6c088194d9f922fb8318faf1f01186"
+dependencies = [
+ "either",
+]
+
+[[package]]
+name = "lazy_static"
+version = "1.5.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "20870f649af7073d53e38067b2a84312175d56ea15217e1b15bc83506ec50afb"
+
+[[package]]
+name = "libc"
+version = "0.2.190"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ce5d3ddc6d3fa000eb1536d85e147bfe31aacaba692ed6a876f95cb7c855be78"
+
+[[package]]
+name = "libloading"
+version = "0.8.9"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d7c4b02199fee7c5d21a5ae7d8cfa79a6ef5bb2fc834d6e9058e89c825efdc55"
+dependencies = [
+ "cfg-if",
+ "windows-link",
+]
+
+[[package]]
+name = "magnus"
+version = "0.9.2"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "b9068710e00d21be762edca59b9c4c8946dbdb29820d4f76ad559afea07982c9"
+dependencies = [
+ "magnus-macros",
+ "rb-sys",
+ "rb-sys-env",
+ "seq-macro",
+]
+
+[[package]]
+name = "magnus-macros"
+version = "0.9.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "9dafde2eeb66f090e0f326112323eb8324b12cb32b5d1210e541d3df6b0ce77e"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "syn",
+]
+
+[[package]]
+name = "memchr"
+version = "2.8.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
+
+[[package]]
+name = "minimal-lexical"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "68354c5c6bd36d73ff3feceb05efa59b6acb7626617f4962be322a825e61f79a"
+
+[[package]]
+name = "nom"
+version = "7.1.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d273983c5a657a70a3e8f2a01329822f3b8c8172b73826411a55751e404a0a4a"
+dependencies = [
+ "memchr",
+ "minimal-lexical",
+]
+
+[[package]]
+name = "proc-macro2"
+version = "1.0.107"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
+dependencies = [
+ "unicode-ident",
+]
+
+[[package]]
+name = "quote"
+version = "1.0.47"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
+dependencies = [
+ "proc-macro2",
+]
+
+[[package]]
+name = "rb-sys"
+version = "0.9.130"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "02faf625bb10ba893e3ae620f19c9fb1b5f8fcae0fe4eb86bb3f2230fad75edb"
+dependencies = [
+ "rb-sys-build",
+]
+
+[[package]]
+name = "rb-sys-build"
+version = "0.9.130"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "05be6f9c86fe5482808162826f6c047d093b3670582296c770de1d94f8066694"
+dependencies = [
+ "bindgen",
+ "lazy_static",
+ "proc-macro2",
+ "quote",
+ "regex",
+ "shell-words",
+ "syn",
+]
+
+[[package]]
+name = "rb-sys-env"
+version = "0.2.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "cca7ad6a7e21e72151d56fe2495a259b5670e204c3adac41ee7ef676ea08117a"
+
+[[package]]
+name = "regex"
+version = "1.13.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f020237b6c8eed93db2e2cb53c00c60a8e1bc73da7d073199a1180401450218d"
+dependencies = [
+ "aho-corasick",
+ "memchr",
+ "regex-automata",
+ "regex-syntax",
+]
+
+[[package]]
+name = "regex-automata"
+version = "0.4.18"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
+dependencies = [
+ "aho-corasick",
+ "memchr",
+ "regex-syntax",
+]
+
+[[package]]
+name = "regex-syntax"
+version = "0.8.11"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
+
+[[package]]
+name = "rustc-hash"
+version = "2.1.3"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "6b1e7f9a428571be2dc5bc0505c13fb6bf936822b894ec87abf8a08a4e51742d"
+
+[[package]]
+name = "seq-macro"
+version = "0.3.6"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "1bc711410fbe7399f390ca1c3b60ad0f53f80e95c5eb935e52268a0e2cd49acc"
+
+[[package]]
+name = "shell-words"
+version = "1.1.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "dc6fe69c597f9c37bfeeeeeb33da3530379845f10be461a66d16d03eca2ded77"
+
+[[package]]
+name = "shlex"
+version = "1.3.0"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "0fda2ff0d084019ba4d7c6f371c95d8fd75ce3524c3cb8fb653a3023f6323e64"
+
+[[package]]
+name = "syn"
+version = "2.0.119"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
+dependencies = [
+ "proc-macro2",
+ "quote",
+ "unicode-ident",
+]
+
+[[package]]
+name = "unicode-ident"
+version = "1.0.26"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "d245f478577f809a851594d02313b640fb437e0bb33866753cff937863096954"
+
+[[package]]
+name = "windows-link"
+version = "0.2.1"
+source = "registry+https://github.com/rust-lang/crates.io-index"
+checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"

data/Cargo.toml

@@ -0,0 +1,7 @@
+[workspace]
+resolver = "2"
+members = ["ext/calcsheet_native"]
+
+[profile.release]
+codegen-units = 1
+lto = "thin"

data/LICENSE.txt

@@ -0,0 +1,21 @@
+The MIT License (MIT)
+
+Copyright (c) 2026 Demetra Opinioni.net Srl
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in
+all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
+THE SOFTWARE.

data/README.md

@@ -0,0 +1,65 @@
+# calcsheet-native
+
+Native reader backends for [calcsheet](https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet): a Rust
+core reads CSV, XLSX and ODS (inflating, tokenizing, limits) without the GVL and hands raw cells to calcsheet, which
+converts them exactly as its Ruby backends do. With the same file and the same options both backends must give the
+same rows, cell errors and exceptions; the only allowed differences are listed in
+[docs/parity-exceptions.md](docs/parity-exceptions.md).
+
+## Status
+
+Released on rubygems.org. CSV, XLSX and ODS are read natively. Precompiled gems exist for `x86_64-linux`,
+`aarch64-linux`, `x86_64-linux-musl`, `aarch64-linux-musl`, `arm64-darwin`, `x86_64-darwin` and `x64-mingw-ucrt`, each
+for Ruby 3.3, 3.4 and 4.0; any other platform installs the source gem, which needs the Rust toolchain and libclang
+(see Development).
+
+Seconds per million rows of 11 columns (Ruby 4.0, x86_64; XLSX and ODS measured on 100,000 rows), end to end
+with the values converted; the peak memory is the same with both backends:
+
+| File | Ruby | Ruby + YJIT | Native | Native + YJIT |
+|---|---|---|---|---|
+| CSV | 7.5 s | 3.9 s | 3.7 s | 2.0 s |
+| XLSX | 27.8 s | 16.9 s | 12.0 s | 8.5 s |
+| ODS | 35.7 s | 17.7 s | 14.8 s | 9.1 s |
+
+With YJIT, the conversion of the cells in Ruby (calcsheet's ADR 011) is 30 to 45% of the native time. Two reads in
+two threads take 1.7 to 2 times as long as one.
+
+## Usage
+
+Add it next to calcsheet; nothing else changes in the application:
+
+```ruby
+gem "calcsheet", "~> 0.3"
+gem "calcsheet-native"
+```
+
+It needs calcsheet 0.2 or 0.3 (protocol 1). calcsheet uses it under `backend: :auto` (the default) for the formats it reads, and falls back to the Ruby backends,
+with one warning, when it is installed but unusable.
+
+## Development
+
+Requirements: Ruby 3.3 or later, the Rust toolchain of `rust-toolchain.toml` (rustup installs it) and libclang
+(Debian: `libclang-dev`), which rb_sys uses to generate the Ruby bindings.
+
+```sh
+bundle install
+bundle exec rake compile
+bundle exec rake          # compile, specs, calcsheet's suite on the native backends, RuboCop
+cargo fmt --check && cargo clippy --all-targets -- -D warnings
+```
+
+The Gemfile takes calcsheet from the tag of its release, whose suite `rake parity` runs (the published gem leaves
+it out).
+
+A release builds the precompiled gems and the source gem into `pkg/` with Docker, from a checkout of its tag, then
+pushes each of them (rubygems.org asks for the security key of every push):
+
+```sh
+bin/build-native-gems
+for gem in pkg/*.gem; do gem push "$gem"; done
+```
+
+## License
+
+MIT, see [LICENSE.txt](LICENSE.txt).

data/ext/calcsheet_native/Cargo.toml

@@ -0,0 +1,14 @@
+[package]
+name = "calcsheet_native"
+version = "0.1.0"
+edition = "2024"
+license = "MIT"
+publish = false
+
+[lib]
+crate-type = ["cdylib"]
+
+[dependencies]
+magnus = "=0.9.2"
+memchr = "=2.8.3"
+rb-sys = "=0.9.130"

data/ext/calcsheet_native/extconf.rb

@@ -0,0 +1,6 @@
+# frozen_string_literal: true
+
+require "mkmf"
+require "rb_sys/mkmf"
+
+create_rust_makefile("calcsheet/native/calcsheet_native")