calcsheet 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 79bc1c89fd78d1d3350d4018f828341811fbeb50ca15bcdf57bebf4673196d1a
-  data.tar.gz: a32050d33976568d5a40ba3667bc4abce2a6c5bc28b6e7dd1ea49ecc63bc7bca
+  metadata.gz: 291f855f2646898ede484b6dbaca57d4cac890387ece27c70352fbb463770057
+  data.tar.gz: 4cb2e6174412b74611168b5da6daf2844fcd8f33ae5c1d88b6b4f528b777267a
 SHA512:
-  metadata.gz: 5a2c9a9420a2f5869519ce1eaa10dd7463ab21e63ad45de2d2e948fe54af0f66bda5a9ce424e06b17b3adcf01c0bad941b8dce7a529fe486559696ee2669ccdc
-  data.tar.gz: 06acdf306e97bd8a9305e2b5a537e1d119a6e3c47257831abacdba9a5fe52cd752316f87e7e05b74fbf9cc5e12c1ae10b83d9400b51b7319de68b5d02a3831d5
+  metadata.gz: c10916111f1bd9d947e7964642b7e6db53e6af1e5acd98e43c8c98067c6212a4a272d6e0a753b3cd33d5f91403f858e3b93108d7e15a8d11e11a20959dfefdfe
+  data.tar.gz: b6ee874c89945474f843bed12c44b68a84ad8e56d81bc58528a783e513fa34009c53f419340aa37c2d0d5c006f553a05abdfd22baa1145b2564715e3fe77042b

data/CHANGELOG.md

@@ -7,6 +7,46 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 
 ## [Unreleased]
 
+## [0.2.0] - 2026-10-22
+
+### Added
+
+- The `backend:` option (`:auto`, the default, `:ruby` or `:native`; also in `Calcsheet.configure`) and
+  `Book#backend`, a hook for the optional calcsheet-native gem, which is not released yet. `:auto` uses it when it is
+  installed and compatible, and otherwise reads with the Ruby backends (one warning when it is installed but
+  unusable); `:native` raises the new `Calcsheet::BackendUnavailableError` instead.
+- `CALCSHEET_BACKEND` runs the suite against a selected backend.
+- Styled writing for XLSX and ODS (ADR 015): `book.style(name, ...)` (font, colours, alignment, wrap, borders, number
+  formats such as `"0.0 %"`) applied with `sheet.add(row, style:)`; `Calcsheet::Formula` values, translated to
+  OpenFormula in ODS and written as text in CSV; `sheet.merge("A1:C1")`; sheets without `columns:` whose values take
+  the type of their class, with `widths:`; `auto_width: true`, which measures the columns, keeping the rows in a
+  temporary file until the sheet ends. The writer backend contract takes the new arguments as optional ones, so a
+  backend written for the previous contract keeps working for typed sheets.
+
+### Changed
+
+- The CSV, XLSX and ODS readers share one conversion of raw cells into values, with no change of behavior.
+- ADR 011 supersedes ADR 007: the native backend also reads CSV, converts in Ruby, and must give the same results as
+  the Ruby backends.
+- A read lists the sheets of an ODS only as far as it selects them: the default sheet no longer costs a scan of the
+  whole `content.xml`, about a quarter of the read. A damaged table, a bad sheet name or too many sheets after the
+  sheets read raise only when a read or `book.sheets` reaches them; the new open option `precheck_sheets: true`
+  lists every sheet first, as before (ADR 013). The backend contract has the optional `#each_sheet`.
+- CSV records are read by calcsheet's own CsvRecords instead of the stdlib parser when the sample shows the line
+  ending: the same records and errors, about a third faster (ADR 014).
+- A zip stream runs a minor GC every 2 MiB it inflates, unless one ran meanwhile, so that its garbage no longer
+  grows the peak memory of a reader that allocates little else (calcsheet-native, byte scanners) by tens of MB.
+
+### Fixed
+
+- A whitespace-only cell at the end of a row is kept as text with `types: :string` and native values, as it already
+  was inside the row; declared types that read whitespace as `nil` still trim it. In header mode it was `nil`, without
+  a header it was dropped (`"x,y, "` gave `["x", "y"]`).
+- `skip_blank: false` yields the blank records at the end of a CSV, as it does those between records. XLSX and ODS
+  sheets still end with their last non-blank row.
+- The backend contract (and so the calcsheet-native protocol 1, not released yet) trims rows this way and has the
+  optional `#end_row(sheet_index)`, which the CSV reader answers.
+
 ## [0.1.0] - 2026-10-21
 
 First release.
@@ -58,5 +98,6 @@ First release.
 - Report vulnerabilities as described in
   [SECURITY.md](https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet/-/blob/main/SECURITY.md).
 
-[Unreleased]: https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet/-/compare/v0.1.0...develop
+[Unreleased]: https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet/-/compare/v0.2.0...develop
+[0.2.0]: https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet/-/compare/v0.1.0...v0.2.0
 [0.1.0]: https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet/-/tags/v0.1.0

data/README.md

@@ -20,7 +20,7 @@ memory, not gigabytes. It is built for imports and exports of large lists:
 
 - [Installation](#installation)
 - [Reading](#reading)
-- [Writing](#writing)
+- [Writing](#writing) ([styles, formulas and merges](#styles-formulas-and-merges))
 - [Types](#types)
 - [Row numbers](#row-numbers)
 - [Errors](#errors)
@@ -31,6 +31,7 @@ memory, not gigabytes. It is built for imports and exports of large lists:
 - [Time zones](#time-zones)
 - [Configuration](#configuration)
 - [Options reference](#options-reference)
+- [Native backend](#native-backend)
 - [Concurrency](#concurrency)
 - [Performance](#performance)
 - [Compatibility](#compatibility)
@@ -44,7 +45,7 @@ Calcsheet will be published on [rubygems.org](https://rubygems.org/gems/calcshee
 release. Once it is released, add it to your Gemfile:
 
 ```rb
-gem "calcsheet", "~> 0.1"
+gem "calcsheet", "~> 0.2"
 ```
 
 The runtime dependencies are `ox` (a SAX parser with a C extension), `rubyzip`, `zip_kit`, `bigdecimal` and `csv`.
@@ -158,7 +159,7 @@ opened: `max_file_size`, `max_uncompressed_size`, `max_compression_ratio`, `comp
 | `types:` | `:native` | `:string` reads every column without a declared type as text. |
 | `on_error:` | `:collect` | `:collect` attaches cell errors to the row; `:raise` raises the first one as `Calcsheet::CellError`. |
 | `start_at:` | `nil` | Resume an interrupted job: a row number, or `{ sheet:, row: }` when reading several sheets. |
-| `skip_blank:` | `true` | Rows without a non-blank cell are not yielded; row numbers keep the gaps. |
+| `skip_blank:` | `true` | Rows without a non-blank cell are not yielded; row numbers keep the gaps. `false` yields them without values: between non-blank rows in every format, and also at the end of a CSV, whose blank records are records of the file. |
 | `include_hidden:` | `false` | `sheet: :all` includes the hidden sheets. A hidden sheet is otherwise read only when named or indexed. |
 | `required_headers:` | `nil` | Header names that must be present (ignoring case and whitespace), checked before the first row. |
 
@@ -175,8 +176,9 @@ Each `Calcsheet::Row` has:
   which `row["D"]` keeps meaning);
 - `errors` (an Array of `Calcsheet::CellError`) and `valid?`.
 
-Without a schema, `values` are as long as the row with trailing blank cells trimmed; in header mode they are padded
-with `nil` to the width of the header, and cells beyond the header are kept.
+Without a schema, `values` end with the last non-blank cell or, after it, the last whitespace-only text kept as text
+(see `required:` below); empty cells and whitespace that reads as `nil` are trimmed at the end. In header mode the
+values are padded with `nil` to the width of the header, and cells beyond the header are kept.
 
 ```ruby
 require "calcsheet"
@@ -347,6 +349,59 @@ writing them if the file must hold rounded values. Numbers with more than 15 sig
 given: Calcsheet reads integers and numbers of more than 17 significant digits back exactly, rounds fractions of 16
 or 17 to 15, and Excel keeps only 15.
 
+### Styles, formulas and merges
+
+```ruby
+require "calcsheet"
+
+Calcsheet.write("report.xlsx") do |book|
+  book.style(:title, bold: true, size: 14, align: :center)
+  book.style(:head, bold: true, fill: "DDEEFF", wrap: true, valign: :top,
+                    border: { style: :thin, color: "000000", edges: %i[top bottom] })
+  book.style(:percent, format: "0.0 %")
+
+  book.sheet("Answers", auto_width: true) do |sheet|        # no columns: a free sheet
+    sheet.merge("A1:C1")                                     # before the row it starts on
+    sheet.add(["Answers to question 1", nil, nil], style: :title)
+    sheet.add(["Answer", "Count", "Share"], style: :head)
+    sheet << ["Yes", 30, Calcsheet::Formula.new("B3/B5")]
+    sheet << ["No", 10, Calcsheet::Formula.new("B4/B5")]
+    sheet.add(["Total", Calcsheet::Formula.new("SUM(B3:B4)"), 1], style: [nil, nil, :percent])
+  end
+end
+
+Calcsheet.open("report.xlsx") do |book|
+  book.rows(header: false).map(&:values).last   # => ["Total", nil, 1]
+end
+```
+
+`book.style(name, **attributes)` registers a style for every sheet of the book: `bold:`, `italic:`, `size:` (points),
+`color:` and `fill:` (`"RRGGBB"`), `align:` (`:left`, `:center`, `:right`), `valign:` (`:top`, `:center`,
+`:bottom`), `wrap:`, `border:` (`{ style:, color:, edges: }`, `style` one of `:thin`, `:medium`, `:thick`,
+`:dashed`, `:dotted`, `:double`, `edges` some of `:top`, `:bottom`, `:left`, `:right`; all four by default) and
+`format:`, a number format both formats can show: `"0"`, `"0.00"`, `"#,##0.00"`, percentages such as `"0%"` or
+`"0.0 %"` (0.25 shows as 25.0 %), dates and times from `yyyy`, `mm`, `dd`, `hh`, `ss` (`"dd/mm/yyyy"`, `"hh:mm"`),
+or `"@"` for text. An unknown attribute or value raises `ArgumentError`. `sheet.add(row, style:)` writes a row as
+`<<` does, with the name of a style for every cell or an Array of names (or `nil`) by column; a style given for a
+column beyond the values styles an empty cell (the border of a merged title). XLSX and ODS apply styles, CSV ignores
+them.
+
+- **Free sheets.** `book.sheet(name)` without `columns:` writes rows of any length (Arrays only, no header row), each
+  value with the type of its class: `String`, `Integer`, `BigDecimal`, `Date`, `Time`, `true`, `false`; a `Float` or
+  a `DateTime` raises `WriteError`, as in a typed column. `widths: [12, nil, 30]` sets their column widths.
+- **Formulas.** `Calcsheet::Formula.new("SUM(B3:B4)")` (a leading `=` is optional) writes a formula in any sheet and
+  column, in the syntax of Excel: ODS gets it translated to OpenFormula (`of:=SUM([.B3:.B4])`, `;` between
+  arguments), CSV writes the text `=SUM(B3:B4)`. No result is stored: Excel and LibreOffice compute it when they open
+  the file, and calcsheet reads such a cell back as `nil`.
+- **Merges.** `sheet.merge("A1:C1")` merges a range, rows counted from the first of the sheet (the header included).
+  ODS writes a merge on its first cell, so it must be declared before that row is written; the other cells it covers
+  must be empty (`WriteError` otherwise), and a merge may not overlap another. CSV ignores merges.
+- **`auto_width: true`** gives each column without a `width:` the width of its longest value, headers included, in
+  characters scaled by the style's font size and weight (a rough measure, as Excel's own autofit is). Formulas and
+  the first cell of a merge across columns are not measured; `sheet.add(row, measure: false)` leaves out a row such
+  as a long note. The widths come before the rows in both formats, so the rows of an `auto_width` sheet wait in a
+  temporary file until the sheet ends.
+
 ### To an IO
 
 ```ruby
@@ -581,7 +636,10 @@ but `max_rows` applies when the file is opened: give it to `Calcsheet.open` or t
   alone; an application can silence the warning with `Zip.warn_invalid_date = false` (after `require "zip"`).
 - `max_uncompressed_size` is cumulative because an ODS keeps every sheet in one `content.xml`, inflated again from
   its start for each sheet read, sampled or merge-checked: however many passes a crafted file causes, one book
-  never inflates more than the budget. `max_sheets` bounds those passes.
+  never inflates more than the budget. `max_sheets` bounds those passes. A read lists the sheets of an ODS only as
+  far as it selects them (the default sheet costs the scan up to it, not of the whole file), so a damaged table, a
+  bad sheet name or too many sheets after the sheets read raise only when a read or `book.sheets` reaches them;
+  `precheck_sheets: true` lists every sheet first, as `book.sheets` does.
 - Verbose, repetitive XML (ODS exports especially) deflates beyond 100:1, so an entry may always inflate up to
   `compression_ratio_floor`; only beyond it does the ratio apply. Memory stays flat either way: entries are
   streamed. The XLSX shared strings are the exception: they are kept in memory, 8 bytes per string beyond their
@@ -709,6 +767,9 @@ Writing:
 
 - The header row is bold and frozen (XLSX `pane`, ODS `settings.xml`) when `header: true`. Columns get their
   `width:`; ODS widths are rounded to whole characters, from 1 to 255.
+- Styles (see [styles, formulas and merges](#styles-formulas-and-merges)) are XLSX cell formats with their fonts,
+  fills and borders, and ODS common cell styles `Calcsheet1`, `Calcsheet2`... in `styles.xml`. A workbook with
+  formulas asks Excel to compute them when it opens (`fullCalcOnLoad`).
 - Number formats: `scale:` decimals (`0.00`), dates `yyyy-mm-dd`, datetimes `yyyy-mm-dd hh:mm:ss`. `scale:` is
   display only: the full value is stored.
 - XLSX text is written as inline strings (no shared strings table) and every sheet is streamed as its rows arrive,
@@ -808,7 +869,9 @@ word and format lists are Arrays of non-empty Strings.
 | `compression_ratio_floor` | `536_870_912` (512 MiB) | open | Bytes any zip entry may inflate to whatever its ratio, except the XLSX shared strings (a fixed 16 MiB); `nil` or `0` for none. |
 | `max_zip_entries` | `10_000` | open | Entries of a zip archive; always a positive Integer. |
 | `max_sheets` | `1_000` | open | Sheets of an XLSX or ODS workbook; `nil` for no limit. |
+| `precheck_sheets` | `false` | open | ODS: list every sheet, scanning the whole `content.xml`, before a read starts, so that a damaged table, a bad sheet name or too many sheets after the one read raise before the first row. `false` scans only as far as the sheets the read selects. |
 | `max_cell_bytes` | `1_048_576` (1 MiB) | open | Bytes of the text of one cell; `nil` for no limit, which also lifts the bound on what the XML parser buffers. See [limits](#limits). |
+| `backend` | `:auto` | open | The reader: `:auto` uses calcsheet-native when it is installed and compatible, else the Ruby backends; `:ruby` always the Ruby backends; `:native` calcsheet-native, or raises `Calcsheet::BackendUnavailableError`. `book.backend` tells which read. |
 | `decimals` | `:bigdecimal` | read | `:bigdecimal` or `:float`: `:float` returns native fractional numbers, and fractional durations, as `Float`. |
 | `time_zone` | `nil` | read, write | `nil` (UTC), `"UTC"` or `"Z"`, a UTC offset such as `"+01:00"` or `"+0100"`, or an object responding to `local` (and, for writing, to `utc_to_local` or `utc_offset`): see [time zones](#time-zones). |
 | `decimal_separator` | `"."` | read, write CSV | `"."` or `","`: text read as `:decimal` or `:integer`; CSV output. |
@@ -831,6 +894,50 @@ word and format lists are Arrays of non-empty Strings.
 | `excel_it` | `false` | write CSV | The preset of `;`, BOM, decimal comma and `%d/%m/%Y` dates. |
 | `sanitize_sheet_names` | `false` | write | Repair invalid sheet names and number duplicates instead of raising. |
 
+## Native backend
+
+Calcsheet reads with its Ruby backends (Ox, rubyzip), which hold Ruby's global VM lock. `calcsheet-native` is an
+optional, separate gem with a Rust core that tokenizes CSV, XLSX and ODS without the lock and hands the raw cells
+back to Calcsheet, which converts them exactly as it does for the Ruby backends. Calcsheet never depends on it and
+never needs a Rust toolchain.
+
+`calcsheet-native` is not released yet. Once it is, it is installed by adding it next to Calcsheet in the Gemfile;
+nothing in your code changes:
+
+```rb
+gem "calcsheet", "~> 0.2"
+gem "calcsheet-native"
+```
+
+It is planned as precompiled gems for `x86_64-linux` and `aarch64-linux` (glibc), `arm64-darwin` and `x86_64-darwin`,
+for Ruby 3.3, 3.4 and 4.0; not for musl (Alpine) or Windows. On any other platform Calcsheet keeps reading with its
+Ruby backends.
+
+**Same results.** With the same file and the same options the native backend gives the same result as the Ruby
+backend: the same values with the same classes, row numbers, sheet names, cell errors and limits. A file the Ruby
+backend refuses, it refuses too, with the same exception class, attributes and location. The only differences allowed
+are those listed, each approved, in the documentation of `calcsheet-native`; a difference found otherwise is a defect.
+
+The `backend` option (see the [options reference](#options-reference)) chooses the reader:
+
+```ruby
+Calcsheet.open("contacts.xlsx", backend: :ruby) { |book| book.backend } # => :ruby
+```
+
+```rb
+Calcsheet.open("contacts.xlsx") { |book| book.backend }  # :native with calcsheet-native installed, else :ruby
+Calcsheet.open("contacts.xlsx", backend: :native)        # calcsheet-native, or Calcsheet::BackendUnavailableError
+```
+
+- `:auto` (the default) uses `calcsheet-native` when it is installed and compatible. When it is installed but cannot
+  be used (a protocol version this Calcsheet does not support, an incomplete or failing gem), Calcsheet prints one
+  warning to standard error and reads with the Ruby backends. When it is not installed, it is silent.
+- `:native` raises `Calcsheet::BackendUnavailableError` instead of falling back when `calcsheet-native` is not
+  installed, when it is installed but cannot be used (as above), and when your application registered its own reader
+  for that format in `Calcsheet.backends`.
+- `book.backend` is `:ruby` or `:native`: the reader that this book uses, for logs and support.
+- Writing is always done by Calcsheet's Ruby writers.
+
 ## Concurrency
 
 Call `Calcsheet.configure` (or `Calcsheet.config`) once at boot, before threads read or write: the configuration is
@@ -839,7 +946,7 @@ snapshot of the options, and books are independent; a single book must not be sh
 
 Ox parses while holding Ruby's global VM lock, so two large reads in two threads of one process take turns instead
 of running in parallel (measured: 66 s for two reads that take 27 s each alone). Process large files in separate
-worker processes, or with one worker thread per process, until a native backend that releases the lock exists.
+worker processes, or with one worker thread per process, until `calcsheet-native`, which releases the lock, is available (see [native backend](#native-backend)).
 
 ## Performance
 
@@ -871,10 +978,10 @@ decides.
 
 - XLS (Excel 97–2003): the only Ruby reader is GPL-licensed. Save the file as XLSX, ODS or CSV.
 - Random access to cells (`sheet["B7"]`) and editing existing files while keeping their formatting.
-- Writing formulas, rich formatting, charts, conditional formats, data validation, images.
+- Writing rich text, charts, conditional formats, data validation, images; formulas with a stored result.
 - Merged reading of sheets without a header row.
 
-Possible future work: a faster native reader as a separate, optional gem (see [ADR 007](https://gitlab.opinioni.net/demetra-developers/rubygems/calcsheet/-/blob/main/docs/adr/007-optional-native-backend.md)); random access to rows
+Possible future work: random access to rows
 (`book.row(900_000)`) by inflating a sheet once into a temporary file and indexing the byte offset of each row
 (about 8 MB per million rows), then seeking and parsing one row; editing an uploaded file while keeping everything
 else (copying the untouched zip entries, rewriting only the changed cells with new strings inline, marking formulas
@@ -887,12 +994,19 @@ appears.
 bundle exec rspec                      # the suite; slow and external-tool cases are excluded
 bundle exec rspec --tag performance    # 1M-row reads and writes with their time and memory bounds
 bundle exec rspec --tag libreoffice    # written files opened by LibreOffice headless (needs soffice)
+CALCSHEET_BACKEND=native CALCSHEET_FAKE_NATIVE=1 COVERAGE=false bundle exec rspec  # the suite on a stand-in backend
 bundle exec rubocop
 bundle exec rake                       # the default task: the suite, then RuboCop
 bundle exec yard stats --list-undoc    # the public API left undocumented (options in .yardopts)
 bundle exec yard                       # the API documentation, in tmp/yard
 ```
 
+- `CALCSHEET_BACKEND` (`ruby`, the default, or `native`) chooses the reader backend the suite runs against. Under
+  `native` every example runs with `backend: :native`, the examples of Ruby internals (tagged `ruby_internals:` with
+  a reason, or in the specs of the readers' parts) are excluded, and an example fails when a csv, xlsx or ods book
+  that should have been read by calcsheet-native was read by a Ruby backend. `CALCSHEET_FAKE_NATIVE=1` installs the
+  stand-in of `spec/support/fake_native.rb` (backends that subclass the Ruby readers); without it the
+  calcsheet-native gem must be installed, and the same suite then verifies its backends.
 - Every fixture is synthetic (neutral names, `example.com` addresses); `spec/fixtures/README.md` says how each one
   was produced. Large files are generated by the specs that need them and never committed.
 - The code comments are YARD in RDoc markup (`+code+`). The public API is what `.yardopts` shows; implementation
@@ -903,6 +1017,7 @@ bundle exec yard                       # the API documentation, in tmp/yard
   - `rspec` on Ruby 3.3, 3.4 and 4.0, with a JUnit test report and a Cobertura coverage report that merge requests
     show; it fails below 99% line or 98.5% branch coverage (checked only when `CI` is set, so focused local runs
     are not);
+  - `rspec: fake native`, the suite with `CALCSHEET_BACKEND=native CALCSHEET_FAKE_NATIVE=1`;
   - `rubocop`;
   - `gem_build`, which checks the files of the built gem, and `gem_smoke`, which installs it on Ruby 3.3 outside the
     bundle, with the locked runtime dependencies, and writes and reads a file in each format;

data/lib/calcsheet/book.rb

@@ -1,6 +1,7 @@
 # frozen_string_literal: true
 
 require_relative "reading"
+require_relative "sheet_list"
 
 module Calcsheet
   # An opened workbook, returned by Calcsheet.open. Every read method takes the read options
@@ -18,26 +19,32 @@ module Calcsheet
     attr_reader :format
     # @return [Options]
     attr_reader :options
+    # @return [Symbol] +:native+ when calcsheet-native reads the book, +:ruby+ otherwise (calcsheet's own reader, or
+    #   one an application registered in Calcsheet.backends): for logs and support
+    attr_reader :backend
 
     # @api private
-    # @param backend [Object] a reader backend (see Reading)
+    # @param reader [Object] a reader backend (see Reading)
     # @param source [Source] closed with the book
-    def initialize(backend, format:, options:, source:)
-      @backend = backend
+    # @param backend [Symbol] +:ruby+ or +:native+, the origin of +reader+
+    def initialize(reader, format:, options:, source:, backend:)
+      @reader = reader
       @format = format
       @options = options
       @source = source
+      @backend = backend
       @closed = false
     end
 
     # @return [Array<SheetInfo>] every sheet in workbook order, hidden ones included (a CSV file has one, named after
     #   the file); read once and kept
     # @raise [IOError] after #close
-    # @raise [MalformedFileError] an ODS whose content.xml cannot be scanned (ODS lists its sheets at the first call)
+    # @raise [MalformedFileError] an ODS whose content.xml cannot be scanned (ODS lists its sheets at the first call,
+    #   scanning the whole content.xml; a read lists them only as far as it selects, unless +precheck_sheets+)
     # @raise [TooManySheetsError] an ODS with more sheets than +max_sheets+
     def sheets
       check_open
-      @sheets ||= @backend.sheets.dup.freeze
+      @sheets ||= @reader.sheets.dup.freeze
     end
 
     # The header and the first data rows of each selected sheet, with native values, reading only the beginning of
@@ -54,7 +61,7 @@ module Calcsheet
     # @raise [IOError] after #close
     # @raise [Calcsheet::Error] reading the sheets (MalformedFileError, ZipBombError...)
     def sample(rows: nil, sheets: :all, include_hidden: false, **overrides)
-      reading = Reading.new(@backend, self.sheets, @options, overrides, sheet: sheets, include_hidden: include_hidden)
+      reading = Reading.new(@reader, selectable_sheets, @options, overrides, sheet: sheets, include_hidden: include_hidden)
       rows ||= reading.options.sample_rows
       raise ArgumentError, "rows: must be a non-negative Integer, got #{rows.inspect}" unless rows.is_a?(Integer) && rows >= 0
 
@@ -72,7 +79,7 @@ module Calcsheet
     # @raise [IOError] after #close
     # @raise [Calcsheet::Error] reading the header rows (MalformedFileError, ZipBombError...)
     def merge_check(sheets: :all, include_hidden: false)
-      Reading.new(@backend, self.sheets, @options, sheet: sheets, include_hidden: include_hidden).merge_check
+      Reading.new(@reader, selectable_sheets, @options, sheet: sheets, include_hidden: include_hidden).merge_check
     end
 
     # @param read [Hash] read options (Reading::KEYS: +sheet:+, +header:+, +columns:+, +types:+, +on_error:+,
@@ -139,7 +146,7 @@ module Calcsheet
 
       @closed = true
       begin
-        @backend.close
+        @reader.close
       ensure
         @source.close
       end
@@ -154,7 +161,16 @@ module Calcsheet
 
     def reading(read)
       settings, overrides = Reading.split(read)
-      Reading.new(@backend, sheets, @options, overrides, **settings).tap { _1.open_check = method(:check_open) }
+      Reading.new(@reader, selectable_sheets, @options, overrides, **settings).tap { _1.open_check = method(:check_open) }
+    end
+
+    # The sheets a read selects from: all of them (#sheets) once listed, with +precheck_sheets+, or for a backend
+    # that lists them at no cost; else a SheetList, which finds them only as far as the selection reads (ODS).
+    def selectable_sheets
+      check_open
+      return sheets if @sheets || @options.precheck_sheets || !@reader.respond_to?(:each_sheet)
+
+      @sheet_list ||= SheetList.new(@reader)
     end
 
     def check_open

data/lib/calcsheet/cell_range.rb

@@ -0,0 +1,54 @@
+# frozen_string_literal: true
+
+require_relative "column_letter"
+require_relative "grid"
+
+module Calcsheet
+  # A rectangle of cells, from its top-left to its bottom-right cell: rows counted from 1, columns from 0. Parsed
+  # from an A1 reference such as "A1:B2" (WriteSheet#merge).
+  #
+  # @api private
+  CellRange = Data.define(:first_row, :first_column, :last_row, :last_column) do
+    # @param reference [String] "A1:B2", case-insensitive, its corners in any order
+    # @return [CellRange]
+    # @raise [ArgumentError] for another text, or a cell beyond the grid
+    def self.parse(reference)
+      match = reference.is_a?(String) && CellRange::REFERENCE.match(reference)
+      raise ArgumentError, "a cell range is written like \"A1:B2\", got #{reference.inspect}" unless match
+
+      columns = [ match[1], match[3] ].map { ColumnLetter.to_index(_1) }
+      rows = [ match[2], match[4] ].map(&:to_i)
+      if columns.max > Grid::LAST_COLUMN_INDEX || rows.max > Grid::MAX_ROWS || rows.min < 1
+        raise ArgumentError, "cell range #{reference.inspect} is beyond the grid (A1:#{Grid::LAST_COLUMN}#{Grid::MAX_ROWS})"
+      end
+
+      new(first_row: rows.min, first_column: columns.min, last_row: rows.max, last_column: columns.max)
+    end
+
+    # @return [Integer] the cells it covers
+    def size
+      (last_row - first_row + 1) * (last_column - first_column + 1)
+    end
+
+    # @return [Boolean] whether it shares a cell with +other+
+    def overlap?(other)
+      first_row <= other.last_row && other.first_row <= last_row &&
+        first_column <= other.last_column && other.first_column <= last_column
+    end
+
+    # @return [Boolean] whether the cell at +row+ (from 1) and +column+ (from 0) is inside
+    def cover?(row, column)
+      row.between?(first_row, last_row) && column.between?(first_column, last_column)
+    end
+
+    # @return [String] "A1:B2"
+    def to_s
+      "#{ColumnLetter.from_index(first_column)}#{first_row}:#{ColumnLetter.from_index(last_column)}#{last_row}"
+    end
+  end
+
+  class CellRange
+    # Two cells: column letters (at most three) and a row number without leading zero.
+    REFERENCE = /\A([A-Za-z]{1,3})([1-9]\d{0,6}):([A-Za-z]{1,3})([1-9]\d{0,6})\z/
+  end
+end

data/lib/calcsheet/configuration.rb

@@ -29,11 +29,17 @@ module Calcsheet
       max_zip_entries: 10_000,
       # The sheets an XLSX or ODS workbook may have; each one can cost a pass over an ODS content.xml.
       max_sheets: 1_000,
+      # ODS: list every sheet (scanning the whole content.xml) before the first row of a read, so that a damaged
+      # table, a bad sheet name or too many sheets after the one read raise before it starts; false scans only as far
+      # as the sheets the read selects.
+      precheck_sheets: false,
       # The bytes of the text of one cell (an XLSX or ODS string; a CSV field once CSV::Parser has built it, so not what
       # parsing it costs, see ADR 008), and the bytes six times this plus 64 KiB between two "<" of an XLSX or ODS
       # part, which bound what the XML parser buffers. The text of a number or a date is at most
       # Cast::MAX_NUMBER_BYTES whatever this is. nil for no limit.
       max_cell_bytes: 1_048_576,
+      # The reader: :auto (calcsheet-native when it is installed and compatible, else Ruby), :ruby or :native.
+      backend: :auto,
       # :bigdecimal or :float: the class of a native fractional number (and of a fractional duration).
       decimals: :bigdecimal,
       time_zone: nil,
@@ -83,8 +89,12 @@ module Calcsheet
     #   @return [Integer] the entries of a zip archive
     # @!attribute [rw] max_sheets
     #   @return [Integer, nil] the sheets of an XLSX or ODS workbook; nil for no limit
+    # @!attribute [rw] precheck_sheets
+    #   @return [Boolean] ODS: list every sheet before a read starts, checking the whole content.xml
     # @!attribute [rw] max_cell_bytes
     #   @return [Integer, nil] the bytes of the text of one cell; nil for no limit
+    # @!attribute [rw] backend
+    #   @return [Symbol] +:auto+, +:ruby+ or +:native+: the reader of the books opened (see Book#backend)
     # @!attribute [rw] decimals
     #   @return [Symbol] +:bigdecimal+ or +:float+, the class of native fractional numbers and durations
     # @!attribute [rw] time_zone

data/lib/calcsheet/display_format.rb

@@ -0,0 +1,75 @@
+# frozen_string_literal: true
+
+module Calcsheet
+  # The number format of a written style (+format:+ of WriteBook#style): how XLSX and ODS show a value, which is
+  # stored unchanged. A small grammar that both formats can express:
+  #
+  # - a number: +"0"+, +"0.00"+ (decimals), +"#,##0"+ or +"#,##0.00"+ (thousands grouped);
+  # - a percentage: a number followed by +"%"+ or +" %"+ (+"0.0 %"+): the value 0.25 shows as 25.0 %;
+  # - a date, a time or both: +yyyy+, +mm+ (month, or minutes after +hh+ or before +ss+), +dd+, +hh+, +ss+,
+  #   separated by "-", "/", ".", ":" or a space (+"dd/mm/yyyy"+, +"yyyy-mm-dd hh:mm:ss"+, +"hh:mm"+);
+  # - +"@"+, text.
+  #
+  # @attr_reader code [String] the code as given, frozen
+  # @attr_reader kind [Symbol] +:number+, +:percent+, +:date+, +:datetime+, +:time+ or +:text+
+  # @attr_reader decimals [Integer] the decimals of a number or a percentage
+  # @attr_reader grouping [Boolean] the thousands of a number or a percentage are grouped
+  # @attr_reader parts [Array<Array(Symbol, String)>] the fields of a date or time in order: +[:year, "yyyy"]+,
+  #   +[:month, "mm"]+, +[:day, "dd"]+, +[:hours, "hh"]+, +[:minutes, "mm"]+, +[:seconds, "ss"]+ or +[:text, "-"]+
+  DisplayFormat = Data.define(:code, :kind, :decimals, :grouping, :parts) do
+    # @param code [String]
+    # @return [DisplayFormat]
+    # @raise [ArgumentError] for a code outside the grammar
+    def self.parse(code)
+      raise ArgumentError, "format: must be a String, got #{code.inspect}" unless code.is_a?(String)
+      return new(code: code.dup.freeze, kind: :text, decimals: 0, grouping: false, parts: []) if code == "@"
+
+      if (match = DisplayFormat::NUMBER.match(code))
+        decimals = match[2] ? match[2].size - 1 : 0
+        kind = match[3] ? :percent : :number
+        return new(code: code.dup.freeze, kind: kind, decimals: decimals, grouping: !match[1].nil?, parts: [])
+      end
+
+      parts = DisplayFormat.temporal_parts(code) or
+        raise ArgumentError, "format: #{code.inspect} is not one of the supported number, percentage, date or time formats"
+      fields = parts.map(&:first) - [ :text ]
+      date = fields.intersect?(%i[year month day])
+      time = fields.intersect?(%i[hours minutes seconds])
+      kind = date && time ? :datetime : (date ? :date : :time)
+      new(code: code.dup.freeze, kind: kind, decimals: 0, grouping: false, parts: parts.freeze)
+    end
+
+    # @api private
+    # The fields of a date or time code, "mm" read as minutes after "hh" or before "ss"; nil for another code.
+    def self.temporal_parts(code)
+      tokens = code.scan(DisplayFormat::TEMPORAL_TOKEN)
+      return if tokens.join != code || tokens.none? { _1.size > 1 }
+
+      parts = tokens.map do |token|
+        case token
+        when "yyyy" then [ :year, token ]
+        when "mm" then [ :month, token ]
+        when "dd" then [ :day, token ]
+        when "hh" then [ :hours, token ]
+        when "ss" then [ :seconds, token ]
+        else [ :text, token ]
+        end
+      end
+      parts.each_with_index do |(field, _), index|
+        next unless field == :month
+
+        before = parts[0...index].reverse.find { _1[0] != :text }
+        after = parts[(index + 1)..].find { _1[0] != :text }
+        parts[index] = [ :minutes, "mm" ] if before&.first == :hours || after&.first == :seconds
+      end
+      parts
+    end
+  end
+
+  class DisplayFormat
+    # A number or a percentage: optional grouping, digits, optional decimals, optional "%" or " %".
+    NUMBER = /\A(#,##)?0(\.0+)?( ?%)?\z/
+    # A field or a separator of a date or time code.
+    TEMPORAL_TOKEN = /yyyy|mm|dd|hh|ss|[-\/.: ]/
+  end
+end

data/lib/calcsheet/errors.rb

@@ -27,6 +27,11 @@ module Calcsheet
   # another compression method, or a format without a backend.
   class UnsupportedFormatError < Error; end
 
+  # +backend: :native+ was asked for and calcsheet-native cannot read the book: it is not installed, its protocol is
+  # not one this calcsheet supports, it does not read the format, or Calcsheet.backends holds an application's own
+  # reader for the format. Raised by Calcsheet.open (before the source is opened when +format:+ is given).
+  class BackendUnavailableError < Error; end
+
   # The file is damaged. The parser error is kept as +cause+.
   class MalformedFileError < Error
     # @return [String, nil] the sheet name, when the damage is in a sheet

data/lib/calcsheet/formula.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+require_relative "column"
+
+module Calcsheet
+  # A formula to write in a cell, in the syntax of Excel: +Formula.new("SUM(B2:B5)")+ (a leading "=" is dropped).
+  # XLSX writes it as is, ODS translates its references and argument separators to OpenFormula, CSV writes it as the
+  # text "=SUM(B2:B5)". No value is cached with it: Excel and LibreOffice compute it when they open the file, and
+  # calcsheet reads such a cell back as nil.
+  #
+  # @attr_reader text [String] the formula without its leading "=", frozen UTF-8
+  Formula = Data.define(:text) do
+    # @param text [String]
+    # @raise [ArgumentError] unless +text+ is a non-empty String convertible to UTF-8, of at most MAX_LENGTH
+    #   characters
+    def initialize(text:)
+      unless text.is_a?(String) && (utf8 = Column.utf8(text))
+        raise ArgumentError, "formula: expected a String convertible to UTF-8, got #{text.inspect}"
+      end
+
+      utf8 = utf8.delete_prefix("=")
+      raise ArgumentError, "formula: the formula is empty" if utf8.strip.empty?
+      if utf8.length > Formula::MAX_LENGTH
+        raise ArgumentError, "formula: #{utf8.length} characters, more than the #{Formula::MAX_LENGTH} of Excel"
+      end
+
+      super(text: utf8.freeze)
+    end
+
+    # @return [String] the formula as a spreadsheet shows it, with its "="
+    def to_s
+      "=#{text}"
+    end
+  end
+
+  class Formula
+    # The longest formula Excel accepts, in characters.
+    MAX_LENGTH = 8_192
+  end
+end

data/lib/calcsheet/limited_inflater.rb

@@ -107,6 +107,8 @@ module Calcsheet
 
     # The compressed bytes read and inflated at a time.
     INPUT_CHUNK_SIZE = 4 * 1024
+    # The inflated bytes between two minor GCs (see #collect_garbage).
+    GC_BYTES = 2 * 1_048_576
     # The fewest consumed bytes at the front of the buffer that #take drops: it drops them once they are at least
     # this many and at least half of the buffer. A buffer consumed in full is reset whatever its size.
     COMPACT_THRESHOLD = 64 * 1024
@@ -140,6 +142,8 @@ module Calcsheet
       @offset = 0
       @inflated = 0
       @crc = 0
+      @next_collection = GC_BYTES
+      @gc_count = nil
       @finished = false
       @failure = nil
       @crc_mismatch = false
@@ -291,6 +295,18 @@ module Calcsheet
       @tokens&.check(piece)
       @crc = Zlib.crc32(piece, @crc)
       @buffer << piece
+      collect_garbage if @inflated >= @next_collection
+    end
+
+    # Each piece Zlib hands out, and each slice a read hands out, is garbage once copied: about twice the bytes
+    # inflated. A reader that allocates little else (a byte scanner, calcsheet-native) lets the GC wait for its malloc
+    # threshold, so the peak memory grows by tens of megabytes. A minor GC every GC_BYTES inflated, unless one ran
+    # meanwhile, keeps it flat at no measurable cost: young garbage is cheap to collect.
+    def collect_garbage
+      @next_collection = @inflated + GC_BYTES
+      count = GC.count
+      GC.start(full_mark: false) if count == @gc_count
+      @gc_count = GC.count
     end
 
     def check_ratio