xlsxrb 0.1.7 → 0.1.9

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (37) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +26 -2
  3. data/README.md +125 -36
  4. data/Rakefile +36 -7
  5. data/benchmark.rb +32 -4
  6. data/docs/ARCHITECTURE.md +32 -30
  7. data/docs/DEVELOPMENT.md +11 -3
  8. data/docs/QUALITY_ASSURANCE.md +3 -4
  9. data/docs/SPEC_SOURCES.md +7 -1
  10. data/docs/visual/VisualGallery.md +157 -157
  11. data/docs/wasm/ruby.wasm +0 -0
  12. data/lib/ruby_lsp/xlsxrb/addon.rb +1 -1
  13. data/lib/ruby_lsp/xlsxrb/completion_listener.rb +11 -13
  14. data/lib/xlsxrb/elements/coordinate_access.rb +101 -0
  15. data/lib/xlsxrb/elements/row.rb +2 -2
  16. data/lib/xlsxrb/elements/workbook.rb +20 -2
  17. data/lib/xlsxrb/elements/worksheet.rb +94 -124
  18. data/lib/xlsxrb/elements.rb +1 -0
  19. data/lib/xlsxrb/ooxml/cfb.rb +490 -0
  20. data/lib/xlsxrb/ooxml/crypto/agile.rb +285 -0
  21. data/lib/xlsxrb/ooxml/crypto/standard.rb +152 -0
  22. data/lib/xlsxrb/ooxml/crypto.rb +60 -0
  23. data/lib/xlsxrb/ooxml/worksheet_parser.rb +21 -15
  24. data/lib/xlsxrb/stream_row.rb +181 -0
  25. data/lib/xlsxrb/version.rb +1 -1
  26. data/lib/xlsxrb.rb +271 -122
  27. data/sig/generated/xlsxrb/elements/coordinate_access.rbs +70 -0
  28. data/sig/generated/xlsxrb/elements/worksheet.rbs +119 -14
  29. data/sig/generated/xlsxrb/ooxml/cfb.rbs +121 -0
  30. data/sig/generated/xlsxrb/ooxml/crypto/agile.rbs +46 -0
  31. data/sig/generated/xlsxrb/ooxml/crypto/standard.rbs +28 -0
  32. data/sig/generated/xlsxrb/ooxml/crypto.rbs +17 -0
  33. data/sig/generated/xlsxrb/ooxml/worksheet_parser.rbs +2 -0
  34. data/sig/generated/xlsxrb/stream_row.rbs +127 -0
  35. data/sig/generated/xlsxrb.rbs +135 -71
  36. data/vendor/sdk_runner/Program.cs +3 -1
  37. metadata +13 -1
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 305368e1e5925e21411db3a34effb2aa2fb59328bf0aca626762657a7c2291c4
4
- data.tar.gz: 6878a769b5ac0aca48a301f7cfbc632f263ff4befb96157c16f4f58bc1ef627e
3
+ metadata.gz: 9930be8e04f9dee4e380c4789398da0b6a95b0c54d2a157265a2d1ee14340132
4
+ data.tar.gz: '086f0abccab70ef69d1e885b9f8d0ec69a77e81e3067fa718764c24c0f37a13f'
5
5
  SHA512:
6
- metadata.gz: a3b70b346fc5481c3242b45ada1135e0a13da6cbf9fd7db2f3a8f084c5560041a0f4187b2b8142da93fd88f12b769f50f6dd4abe75612c6b1fdb0856604721d8
7
- data.tar.gz: 5e18aa8c063522115e1320c74d550402cd9020088239b4d3061de2736a50dcab42487e20f1cb7d1e291f8fe31461251ca62703b299d2bb41d83ad1d1c03c79c6
6
+ metadata.gz: 2097ccb3e8b3e5344e5d55c459db6b06c298b116074bcc41a06fafa034eda67bf0c46d7eaa58184aebc970812cfa91f68388c4a6b4e7d7069a45114507aa9e83
7
+ data.tar.gz: d95b39449b15b2eec1415d1ae65e398cc2c006adefd19b54340727fc4ee2135322040a73bf0b3d2b0ca8f6d171153e88b8f64badb53b7fb841e9d7b8a87afd55
data/CHANGELOG.md CHANGED
@@ -1,5 +1,29 @@
1
- ## [Unreleased]
2
- - No unreleased changes.
1
+ ## [0.1.9] - 2026-08-18
2
+
3
+ ### Added
4
+ - **Password Protection & Document Encryption ([MS-OFFCRYPTO] / [MS-CFB])**: Full native Pure-Ruby support for reading, writing, and modifying password-protected Excel spreadsheets without any external C-extension dependencies.
5
+ - **Standard Encryption**: AES-128-ECB and SHA-1 Key Derivation with CryptoAPI 50,000-spin hashing, fully interoperable across Microsoft Excel, LibreOffice, and Google Sheets.
6
+ - **Agile Encryption**: Modern AES-256-CBC, PBKDF2/SHA-512, and HMAC-SHA512 data integrity verification.
7
+ - **Compound File Binary (CFB) Engine**: Pure-Ruby reader and writer for OLE structured storage containers with Mini Stream, FAT/MiniFAT sectors, and Red-Black tree directory management.
8
+ - **Transparent Public API Integration**: Added `password:` and `encryption_mode:` arguments to `Xlsxrb.read`, `Xlsxrb.write`, and `Xlsxrb.modify`.
9
+ - **Security & Threat Model Hardening**:
10
+ - Constant-time hash verification via `OpenSSL.secure_compare` to prevent timing attacks (CWE-208).
11
+ - CSPRNG-backed salt, IV, and session key generation via `SecureRandom` (CWE-330).
12
+ - Robust DoS defense: spinCount limit ($\le 10\text{M}$), CFB circular sector chain loop detection in directory/FAT parsing, and `total_size` bounds validation (CWE-400, CWE-835).
13
+ - Strict exception hierarchy (`EncryptedFileError`, `InvalidPasswordError`, `DecryptionError`).
14
+ - **Cross-Platform & Interoperability Validation**: Bidirectional validation with Microsoft .NET OpenXML SDK and LibreOffice Calc.
15
+ - **WebAssembly (ruby.wasm) Support**: Pre-packaged `docs/wasm/ruby.wasm` updated with document encryption support for browser playground.
16
+
17
+ ## [0.1.8] - 2026-08-18
18
+
19
+ ### Changed
20
+ - **Unified Symmetric Entrypoints**: Consolidated reading into `Xlsxrb.read` (supporting file path, IO, and raw binary string) and writing into `Xlsxrb.write` (supporting streaming blocks or in-memory Workbooks). Removed legacy `open`, `foreach`, and `generate` methods.
21
+ - **Streaming-First Defaults**: `Xlsxrb.read` yields and returns lightweight `StreamSheet` instances with $O(1)$ constant-memory consumption by default.
22
+ - **Explicit In-Memory Materialization (`#load`)**: Stripped accidental random-access memory traps from `StreamSheet`; introduced explicit `StreamSheet#load` / `Workbook#load` (inspired by ActiveRecord Relations) to transition from lazy streaming to in-memory `Elements::Worksheet` / `Elements::Workbook`.
23
+ - **`CoordinateAccess` Module**: Extracted coordinate lookup methods (`[]`, `cell_value`, `row_at`, `first_row`, `last_row`, `cells`, `cells_hash`) into a dedicated `Xlsxrb::Elements::CoordinateAccess` mixin module included in `Elements::Worksheet`.
24
+
25
+ ### Added
26
+ - **Default Cell Streaming (`Xlsxrb::StreamRow`)**: Enabled streaming along both row and cell dimensions via `row.each_cell` and `sheet.each_cell`, parsing cells on-demand to handle sheets with thousands of columns in $O(1)$ constant memory.
3
27
 
4
28
  ## [0.1.7] - 2026-08-16
5
29
 
data/README.md CHANGED
@@ -6,16 +6,18 @@ A Ruby library for reading and writing XLSX files with streaming support.
6
6
 
7
7
  The Ruby ecosystem already has great XLSX libraries. Each is well-designed for its purpose:
8
8
 
9
- | Library | Read | Write | Model | Write String Storage | Rich Formatting |
10
- | -------------------------------------------------- | ---- | ----- | --------------------- | -------------------- | --------------- |
11
- | [roo](https://rubygems.org/gems/roo) | ✅ | ❌ | Streaming | N/A (Read-only) | ⚠️ (Formulas, Basic styles) |
12
- | [creek](https://rubygems.org/gems/creek) | ✅ | ❌ | Streaming | N/A (Read-only) | ❌ (Raw cell values) |
13
- | [xsv](https://rubygems.org/gems/xsv) | ✅ | ❌ | Streaming | N/A (Read-only) | ❌ (Fast plain text) |
14
- | [caxlsx / axlsx](https://rubygems.org/gems/caxlsx) | | | In-Memory | Inline (opt: SST) | (Charts, Styles) |
15
- | [xlsxtream](https://rubygems.org/gems/xlsxtream) | ❌ | ✅ | Streaming | Inline (opt: SST) | (Plain data only) |
16
- | [fast_excel](https://rubygems.org/gems/fast_excel) | ❌ | ✅ | Streaming (C Ext) | SST (opt: Inline) | ⚠️ (Basic styles) |
17
- | [rubyXL](https://rubygems.org/gems/rubyXL) | | ✅ | In-Memory | Inline / Direct | (DOM editing) |
18
- | **[xlsxrb](https://github.com/niku/xlsxrb)** | | ✅ | **Streaming / In-Memory** | **SST** | **(Full Features)** |
9
+ | Library | Read | Write | Model | Write String Storage | Rich Formatting | Password Encryption |
10
+ | ---------------------------------------------------------- | ---- | ----- | --------------------- | -------------------- | --------------- | :-----------------: |
11
+ | [roo](https://rubygems.org/gems/roo) | ✅ | ❌ | Streaming | N/A (Read-only) | ⚠️ (Formulas, Basic styles) | ❌ |
12
+ | [creek](https://rubygems.org/gems/creek) | ✅ | ❌ | Streaming | N/A (Read-only) | ❌ (Raw cell values) | ❌ |
13
+ | [xsv](https://rubygems.org/gems/xsv) | ✅ | ❌ | Streaming | N/A (Read-only) | ❌ (Fast plain text) | ❌ |
14
+ | [simple_xlsx_reader](https://rubygems.org/gems/simple_xlsx_reader) | | | Streaming | N/A (Read-only) | (Plain data & types) | ❌ |
15
+ | [caxlsx / axlsx](https://rubygems.org/gems/caxlsx) | ❌ | ✅ | In-Memory | Inline (opt: SST) | (Charts, Styles) | ❌ |
16
+ | [write_xlsx](https://rubygems.org/gems/write_xlsx) | ❌ | ✅ | In-Memory | SST | (Charts, Styles) | ❌ |
17
+ | [xlsxtream](https://rubygems.org/gems/xlsxtream) | | ✅ | Streaming | Inline (opt: SST) | (Plain data only) | ❌ |
18
+ | [fast_excel](https://rubygems.org/gems/fast_excel) | | ✅ | Streaming (C Ext) | SST (opt: Inline) | ⚠️ (Basic styles) | ❌ |
19
+ | [rubyXL](https://rubygems.org/gems/rubyXL) | ✅ | ✅ | In-Memory | Inline / Direct | ✅ (DOM editing) | ❌ |
20
+ | **[xlsxrb](https://github.com/niku/xlsxrb)** | ✅ | ✅ | **Streaming / In-Memory** | **SST** | ✅ **(Full Features)** | ✅ **(Standard & Agile)** |
19
21
 
20
22
  Each of these libraries makes deliberate tradeoffs, and they do so thoughtfully:
21
23
  * **Memory & Execution Model (Streaming vs In-Memory)**: Streaming libraries write or read rows sequentially on-the-fly to maintain a constant, low-memory footprint regardless of row count. In-memory libraries build complete document object trees, offering flexible random access and cell updates at the cost of high RAM usage on large sheets.
@@ -78,12 +80,12 @@ For visual demonstrations of various features, check the [Visual Examples Galler
78
80
  ### Quick Start: Streaming (Recommended)
79
81
 
80
82
  #### Streaming Write
81
- Generate large files efficiently by writing data directly to the file stream:
83
+ Generate large files efficiently with $O(1)$ constant memory by writing data directly to the stream:
82
84
  ```ruby
83
85
  require "xlsxrb"
84
86
 
85
- Xlsxrb.generate("large_output.xlsx") do |stream_writer|
86
- stream_writer.sheet("Sales Data") do |sheet|
87
+ Xlsxrb.write("large_output.xlsx") do |writer|
88
+ writer.sheet("Sales Data") do |sheet|
87
89
  sheet.row(["Date", "Amount", "Status"])
88
90
  sheet.row([Date.today, 100, true])
89
91
  sheet.column(0, width: 15.5)
@@ -92,15 +94,63 @@ end
92
94
  ```
93
95
 
94
96
  #### Streaming Read
95
- Read rows one at a time without loading the entire file into memory:
97
+ Read rows and cells lazily one at a time with $O(1)$ constant memory (even for wide sheets with thousands of columns):
96
98
  ```ruby
97
99
  require "xlsxrb"
98
100
 
99
- Xlsxrb.foreach("large_file.xlsx") do |sheet|
101
+ # Stream row-by-row and cell-by-cell (O(1) memory)
102
+ Xlsxrb.read("large_file.xlsx") do |sheet|
100
103
  sheet.each_row do |row|
101
- puts "Row #{row.index}: #{row.cells.map(&:value).join(', ')}"
104
+ row.each_cell do |cell|
105
+ puts "#{cell.ref}: #{cell.value}"
106
+ end
102
107
  end
103
108
  end
109
+
110
+ # Or stream all cells across the sheet directly
111
+ Xlsxrb.read("large_file.xlsx") do |sheet|
112
+ sheet.each_cell do |cell|
113
+ puts "#{cell.ref} = #{cell.value}"
114
+ end
115
+ end
116
+ ```
117
+
118
+ ### Ruby-Idiomatic Core APIs
119
+
120
+ `xlsxrb` provides clean, standard Ruby interfaces (`Enumerable`, `Row#to_a`, `sheet["A1"]`) that feel natural to every Ruby developer without learning complex library-specific APIs:
121
+
122
+ #### Reading Spreadsheets
123
+ ```ruby
124
+ require "xlsxrb"
125
+
126
+ # 1. Read from file path, IO, or raw binary string (O(1) constant memory streaming)
127
+ workbook = Xlsxrb.read("data.xlsx")
128
+ sheet = workbook.sheets.first
129
+
130
+ # 2. Extract sheet data into 2D array of values via standard Enumerable
131
+ matrix = sheet.map(&:to_a) # => [["Name", "Score"], ["Alice", 100], ["Bob", 95]]
132
+
133
+ # 3. Explicitly load into memory for coordinate random access (e.g. sheet["A1"])
134
+ doc_sheet = sheet.load
135
+ doc_sheet["A1"] # => #<Xlsxrb::Elements::Cell value="Name" ...>
136
+ doc_sheet["A1"].value # => "Name"
137
+ ```
138
+
139
+ #### Writing & In-Memory Export (Rails & Mailers)
140
+ ```ruby
141
+ # Build workbook
142
+ wb = Xlsxrb.build do |b|
143
+ b.sheet("Report") do |s|
144
+ s.row(["Metric", "Value"])
145
+ s.row(["Users", 1000])
146
+ end
147
+ end
148
+
149
+ # Save directly to file:
150
+ Xlsxrb.write("report.xlsx", wb)
151
+
152
+ # Or export to binary string (ideal for Rails send_data & ActionMailer):
153
+ binary_data = Xlsxrb.write(wb)
104
154
  ```
105
155
 
106
156
  ### In-Memory Building & Modifying
@@ -147,6 +197,43 @@ Xlsxrb.build do |builder|
147
197
  end
148
198
  ```
149
199
 
200
+ ### Password Protection & Document Encryption ([MS-OFFCRYPTO])
201
+
202
+ `xlsxrb` natively supports reading and writing password-protected Excel files with zero external C-extension dependencies, fully compliant with **[MS-OFFCRYPTO]** (Standard Encryption & Agile Encryption) and **[MS-CFB]** (Compound File Binary Format):
203
+
204
+ #### Writing Password-Protected Spreadsheets
205
+ ```ruby
206
+ # 1. Streaming write with password protection
207
+ Xlsxrb.write("confidential.xlsx", password: "SecretPassword123") do |writer|
208
+ writer.sheet("Financials") do |sheet|
209
+ sheet.row(["Account", "Balance"])
210
+ sheet.row(["Assets", 5_000_000])
211
+ end
212
+ end
213
+
214
+ # 2. In-Memory write with password protection
215
+ wb = Xlsxrb.build do |b|
216
+ b.sheet("Private") { |s| s.row(["Confidential", 100]) }
217
+ end
218
+ encrypted_binary = Xlsxrb.write(wb, password: "SecretPassword123")
219
+ ```
220
+
221
+ #### Reading Password-Protected Spreadsheets
222
+ ```ruby
223
+ # 1. Read directly with password (O(1) constant memory streaming)
224
+ Xlsxrb.read("confidential.xlsx", password: "SecretPassword123") do |sheet|
225
+ sheet.each_row { |row| puts row.cells.map(&:value) }
226
+ end
227
+
228
+ # 2. In-Memory load with password
229
+ workbook = Xlsxrb.read("confidential.xlsx", password: "SecretPassword123").load
230
+ puts workbook.sheets.first["A1"].value
231
+
232
+ # Attempting to read without password or with invalid password raises clear exceptions:
233
+ # Xlsxrb::EncryptedFileError -> Missing password
234
+ # Xlsxrb::InvalidPasswordError -> Incorrect password
235
+ ```
236
+
150
237
  ### IDE Autocompletion & Ruby LSP Support
151
238
 
152
239
  `xlsxrb` bundles a native **Ruby LSP Add-on** (`RubyLsp::Xlsxrb::Addon`) and full **RBS signatures**, enabling zero-configuration method autocompletion and rich Markdown documentation in VS Code and other LSP-enabled editors.
@@ -154,8 +241,8 @@ end
154
241
  Whether you use standard descriptive block variable names (`|stream_writer|`, `|sheet|`, `|workbook|`) or short names (`|wb|`, `|s|`), your editor will automatically provide complete method suggestions and parameter hints:
155
242
 
156
243
  ```ruby
157
- Xlsxrb.generate("output.xlsx") do |stream_writer| # or |wb|
158
- stream_writer.sheet("Data") do |sheet| # or |s|
244
+ Xlsxrb.write("output.xlsx") do |writer| # or |wb|
245
+ writer.sheet("Data") do |sheet| # or |s|
159
246
  sheet.row(["Product", "Price"], styles: :bold)
160
247
  sheet.auto_filter("A1:B100")
161
248
  end
@@ -182,25 +269,27 @@ The following benchmarks measure the time, peak memory, and GC count required to
182
269
 
183
270
  | Library | Model | Write String Storage | Time (Median) | Time (Mean) | Peak Memory | GC Count |
184
271
  | ---------------------- | ----------- | -------------------- | ------------- | ----------- | ----------- | -------- |
185
- | xlsxtream 3.1.0 | Streaming | Inline String | 1.23 s | 1.25 s | 18.1 MB | 1061.0 |
186
- | xlsxrb (Streaming) | Streaming | SST (Shared) | 1.62 s | 1.62 s | 94.5 MB | 39.0 |
187
- | fast_excel 0.5.0 (C) | Streaming | SST (Shared) | 2.03 s | 2.03 s | 147.9 MB | 263.0 |
188
- | xlsxrb (In-Memory) | In-Memory | SST (Shared) | 4.06 s | 4.06 s | 280.3 MB | 32.0 |
189
- | caxlsx 4.5.0 | In-Memory | Inline String | 5.36 s | 5.35 s | 188.7 MB | 23.0 |
190
- | rubyXL 3.4.38 | In-Memory | Inline String | 38.02 s | 38.03 s | 2166.0 MB | 104.0 |
272
+ | xlsxtream 3.1.0 | Streaming | Inline String | 1.19 s | 1.20 s | 18.2 MB | 1072.0 |
273
+ | xlsxrb (Streaming) | Streaming | SST (Shared) | 1.73 s | 1.65 s | 94.4 MB | 39.0 |
274
+ | fast_excel 0.5.0 (C) | Streaming | SST (Shared) | 1.89 s | 1.89 s | 148.2 MB | 245.0 |
275
+ | xlsxrb (In-Memory) | In-Memory | SST (Shared) | 3.84 s | 3.83 s | 278.3 MB | 32.0 |
276
+ | write_xlsx 1.15.0 | In-Memory | SST (Shared) | 4.32 s | 4.34 s | 201.2 MB | 33.0 |
277
+ | caxlsx 4.5.0 | In-Memory | Inline String | 5.15 s | 5.12 s | 188.6 MB | 23.0 |
278
+ | rubyXL 3.4.38 | In-Memory | Inline String | 38.81 s | 37.82 s | 2186.8 MB | 103.0 |
191
279
 
192
- > **Note**: All libraries are evaluated in their **default, out-of-the-box configuration**. Under the same Microsoft Excel-standard Shared String Table (SST) architecture, Pure Ruby `xlsxrb` (Streaming: 1.62s, In-Memory: 4.06s) writes 1,000,000 cells faster than the C-extension `fast_excel` (2.03s) and in-memory gems like `caxlsx` (5.36s).
280
+ > **Note**: All libraries are evaluated in their **default, out-of-the-box configuration**. Under the same Microsoft Excel-standard Shared String Table (SST) architecture, Pure Ruby `xlsxrb` (Streaming: 1.73s, In-Memory: 3.84s) writes 1,000,000 cells faster than the C-extension `fast_excel` (1.89s) and in-memory gems like `write_xlsx` (4.32s) and `caxlsx` (5.15s).
193
281
 
194
282
  ### Read Performance (1,000,000 cells)
195
283
 
196
- | Library | Model | Time (Median) | Time (Mean) | Peak Memory | GC Count |
197
- | ---------------------- | ----------- | ------------- | ----------- | ----------- | -------- |
198
- | xlsxrb (Streaming) | Streaming | 3.38 s | 3.37 s | 90.6 MB | 40.0 |
199
- | xlsxrb (In-Memory) | In-Memory | 5.75 s | 5.94 s | 250.9 MB | 55.0 |
200
- | creek 2.6.3 | Streaming | 7.90 s | 7.93 s | 835.7 MB | 481.0 |
201
- | roo 3.0.0 | Streaming | 10.34 s | 10.28 s | 139.0 MB | 107.0 |
202
- | xsv 1.4.1 | Streaming | 16.39 s | 17.28 s | 75.4 MB | 2215.0 |
203
- | rubyXL 3.4.38 | In-Memory | 35.45 s | 35.38 s | 2281.3 MB | 146.0 |
284
+ | Library | Model | Time (Median) | Time (Mean) | Peak Memory | GC Count |
285
+ | ------------------------ | ----------- | ------------- | ----------- | ----------- | -------- |
286
+ | xlsxrb (Streaming) | Streaming | 3.17 s | 3.22 s | 91.4 MB | 43.0 |
287
+ | simple_xlsx_reader 5.1.0 | Streaming | 4.48 s | 4.45 s | 38.5 MB | 1669.0 |
288
+ | xlsxrb (In-Memory) | In-Memory | 5.71 s | 5.89 s | 224.8 MB | 63.0 |
289
+ | creek 2.6.3 | Streaming | 8.14 s | 8.02 s | 834.6 MB | 477.0 |
290
+ | xsv 1.4.1 | Streaming | 14.61 s | 14.50 s | 76.1 MB | 2224.0 |
291
+ | roo 3.0.0 | Streaming | 15.69 s | 13.36 s | 119.7 MB | 441.0 |
292
+ | rubyXL 3.4.38 | In-Memory | 37.13 s | 40.35 s | 2537.6 MB | 146.0 |
204
293
 
205
294
  ### Running the Benchmarks Locally (Reproducibility)
206
295
 
@@ -224,7 +313,7 @@ As an extra layer of "defense in depth", `xlsxrb` configures the workbook to **n
224
313
  If you absolutely need external links to update automatically, you can explicitly override this (though **it is highly discouraged due to security risks**):
225
314
 
226
315
  ```ruby
227
- Xlsxrb.generate("file.xlsx") do |wb|
316
+ Xlsxrb.write("file.xlsx") do |wb|
228
317
  # WARNING: Enabling this can expose users to malicious external reference vulnerabilities!
229
318
  wb.workbook_property(:update_links, "always")
230
319
  # ...
@@ -237,7 +326,7 @@ To support reliability, compliance with the ECMA-376 specification, and consiste
237
326
 
238
327
  ### Multi-Tier Testing Strategy
239
328
  * **Round-Trip Testing**: Unit tests verify that every generated sheet can be reliably parsed back by the reader with identical content and styling.
240
- * **Contract Consistency**: Ensures semantic output consistency between the Streaming (`Xlsxrb.generate`) and In-Memory (`Xlsxrb.build`) APIs.
329
+ * **Contract Consistency**: Ensures semantic output consistency between the Streaming (`Xlsxrb.write`) and In-Memory (`Xlsxrb.build`) APIs.
241
330
  * **Property-Based Testing (PBT)**: Automatically generates random data to catch edge cases (e.g., huge numbers, special characters) preventing unexpected crashes.
242
331
  * **Concurrency Validation**: Thread and Ractor safety checks to guarantee no global variable pollution during parallel execution.
243
332
  * **Security & DoS Protection**: Hardened against malicious files, including memory exhaustion (ZIP Bombs) and infinite parsing loops.
@@ -247,7 +336,7 @@ To support reliability, compliance with the ECMA-376 specification, and consiste
247
336
  * **Visual Regression Testing (VRT)**: Spreadsheets are rendered via a headless LibreOffice Calc engine and compared pixel-by-pixel against visual baselines to catch subtle rendering regressions.
248
337
 
249
338
  ### Performance & Types
250
- * **Continuous Benchmarking**: Memory usage and processing speeds are profiled in CI on large datasets to prevent performance regressions and OOM leaks.
339
+ * **Continuous Performance & Memory Validation**: Memory retention and processing throughput are profiled in CI on large datasets to prevent regressions and OOM leaks.
251
340
  * **Runtime Type Validation**: Strong dynamic typing using `RBS::Test` to ensure the library's types are perfectly sound at runtime.
252
341
 
253
342
  For a comprehensive breakdown of our QA matrix, see [docs/QUALITY_ASSURANCE.md](docs/QUALITY_ASSURANCE.md). For details on running tests locally, see [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md).
data/Rakefile CHANGED
@@ -7,9 +7,19 @@ require "fileutils"
7
7
  require "open3"
8
8
  require "tmpdir"
9
9
 
10
+ def dotnet_available?
11
+ system("which dotnet > /dev/null 2>&1")
12
+ end
13
+
10
14
  desc "Build the Open XML SDK runner"
11
15
  task :build_sdk_runner do
12
- sh "dotnet build vendor/sdk_runner/sdk_runner.csproj -c Release"
16
+ if dotnet_available?
17
+ sh "dotnet build vendor/sdk_runner/sdk_runner.csproj -c Release"
18
+ elsif File.exist?(sdk_runner_dll)
19
+ puts "dotnet not found in PATH, but pre-built sdk_runner.dll exists. Skipping build."
20
+ else
21
+ warn "dotnet command not found and sdk_runner.dll is missing. Cannot build SDK runner."
22
+ end
13
23
  end
14
24
 
15
25
  def reader_fixture_dir
@@ -38,10 +48,13 @@ rescue ArgumentError
38
48
  end
39
49
 
40
50
  desc "Ensure SDK-generated reader fixtures exist"
41
- task ensure_reader_fixtures: :build_sdk_runner do
51
+ task :ensure_reader_fixtures do
42
52
  missing_specs = reader_fixture_specs.reject { |_scenario_name, fixture_path| File.exist?(fixture_path) }
43
53
  next if missing_specs.empty?
44
54
 
55
+ Rake::Task[:build_sdk_runner].invoke
56
+ raise "Cannot generate #{missing_specs.size} missing reader fixture(s) because dotnet is not installed." unless dotnet_available?
57
+
45
58
  FileUtils.mkdir_p(reader_fixture_dir)
46
59
 
47
60
  queue = Queue.new
@@ -77,16 +90,14 @@ task ensure_reader_fixtures: :build_sdk_runner do
77
90
  raise failures.pop
78
91
  end
79
92
 
80
- Rake::TestTask.new(:test) do |t|
93
+ Rake::TestTask.new(test: :ensure_reader_fixtures) do |t|
81
94
  t.libs << "test"
82
95
  t.libs << "lib"
83
- t.test_files = FileList["test/**/*_test.rb"]
84
- workers = ENV.fetch("TEST_WORKERS", Etc.nprocessors)
96
+ t.test_files = FileList["test/xlsxrb/**/*_test.rb", "test/*_test.rb", "test/contract/**/*_test.rb"]
97
+ workers = ENV.fetch("TEST_WORKERS", [Etc.nprocessors, 4].min)
85
98
  t.options = "--parallel --n-workers=#{workers}"
86
99
  end
87
100
 
88
- task test: %i[build_sdk_runner ensure_reader_fixtures]
89
-
90
101
  namespace :test do
91
102
  desc "Run tests with runtime type checking enabled (RBS_TEST=1)"
92
103
  task :rbs do
@@ -110,6 +121,24 @@ namespace :test do
110
121
  t.options = "--parallel --n-workers=#{workers}"
111
122
  end
112
123
 
124
+ desc "Run property-based tests (PBT)"
125
+ Rake::TestTask.new(:pbt) do |t|
126
+ t.libs << "test"
127
+ t.libs << "lib"
128
+ t.test_files = FileList["test/pbt/**/*_test.rb"]
129
+ workers = ENV.fetch("TEST_WORKERS", Etc.nprocessors)
130
+ t.options = "--parallel --n-workers=#{workers}"
131
+ end
132
+
133
+ desc "Run memory and performance tests"
134
+ Rake::TestTask.new(:performance) do |t|
135
+ t.libs << "test"
136
+ t.libs << "lib"
137
+ t.test_files = FileList["test/performance/**/*_test.rb"]
138
+ end
139
+ desc "Alias for test:performance"
140
+ task perf: :performance
141
+
113
142
  Rake::TestTask.new(:e2e) do |t|
114
143
  t.libs << "test"
115
144
  t.libs << "lib"
data/benchmark.rb CHANGED
@@ -16,6 +16,8 @@ gemfile(true) do
16
16
  gem "roo", "3.0.0"
17
17
  gem "creek", "2.6.3"
18
18
  gem "xsv", "1.4.1"
19
+ gem "write_xlsx", "1.15.0"
20
+ gem "simple_xlsx_reader", "5.1.0"
19
21
  end
20
22
 
21
23
  RUNS = (ENV["RUNS"] || "3").to_i
@@ -28,10 +30,12 @@ AVAILABLE_GEMS = {
28
30
  "xlsxtream" => true,
29
31
  "fast_excel" => true,
30
32
  "caxlsx" => true,
33
+ "write_xlsx" => true,
31
34
  "rubyXL" => true,
32
35
  "creek" => true,
33
36
  "roo" => true,
34
- "xsv" => true
37
+ "xsv" => true,
38
+ "simple_xlsx_reader" => true
35
39
  }
36
40
 
37
41
  puts "=" * 80
@@ -120,6 +124,16 @@ RUNNER_SCRIPT = <<~'RUBY'
120
124
  end
121
125
  p.serialize(filename)
122
126
  end
127
+ when ["write_xlsx", "write"]
128
+ require "write_xlsx"
129
+ measure do
130
+ wb = WriteXLSX.new(filename)
131
+ sheet = wb.add_worksheet("Data")
132
+ rows.times do |r|
133
+ sheet.write_row(r, 0, generate_row(r, cols))
134
+ end
135
+ wb.close
136
+ end
123
137
  when ["rubyXL", "write"]
124
138
  require "rubyXL"
125
139
  measure do
@@ -137,7 +151,7 @@ RUNNER_SCRIPT = <<~'RUBY'
137
151
  when ["xlsxrb_stream", "write"]
138
152
  require_relative "lib/xlsxrb"
139
153
  measure do
140
- Xlsxrb.generate(filename) do |wb|
154
+ Xlsxrb.write(filename) do |wb|
141
155
  wb.sheet("Data") do |sheet|
142
156
  rows.times do |r|
143
157
  sheet.row(generate_row(r, cols))
@@ -161,7 +175,7 @@ RUNNER_SCRIPT = <<~'RUBY'
161
175
  require_relative "lib/xlsxrb"
162
176
  measure do
163
177
  count = 0
164
- Xlsxrb.foreach(filename) do |sheet|
178
+ Xlsxrb.read(filename) do |sheet|
165
179
  sheet.each do |row|
166
180
  row.cells.each do |cell|
167
181
  _val = cell.value
@@ -222,6 +236,19 @@ RUNNER_SCRIPT = <<~'RUBY'
222
236
  end
223
237
  end
224
238
  end
239
+ when ["simple_xlsx_reader", "read"]
240
+ require "simple_xlsx_reader"
241
+ measure do
242
+ doc = SimpleXlsxReader.open(filename)
243
+ count = 0
244
+ doc.sheets.each do |sheet|
245
+ sheet.rows.each do |row|
246
+ row.each do |_val|
247
+ count += 1
248
+ end
249
+ end
250
+ end
251
+ end
225
252
  when ["rubyXL", "read"]
226
253
  require "rubyXL"
227
254
  measure do
@@ -281,7 +308,6 @@ def run_benchmark_series(name, lib, mode, rows, cols, filename, runs)
281
308
  end
282
309
  end
283
310
  puts
284
-
285
311
  return nil if results.empty?
286
312
 
287
313
  times = results.map { |r| r[:time] }.sort
@@ -314,6 +340,7 @@ write_targets = [
314
340
  ["xlsxrb (Streaming)", "xlsxrb_stream", "Streaming", "SST (Shared)"],
315
341
  ["fast_excel 0.5.0 (C)", "fast_excel", "Streaming", "SST (Shared)"],
316
342
  ["caxlsx 4.5.0", "caxlsx", "In-Memory", "Inline String"],
343
+ ["write_xlsx 1.15.0", "write_xlsx", "In-Memory", "SST (Shared)"],
317
344
  ["xlsxrb (In-Memory)", "xlsxrb_inmemory", "In-Memory", "SST (Shared)"],
318
345
  ["rubyXL 3.4.38", "rubyXL", "In-Memory", "Inline String"]
319
346
  ]
@@ -337,6 +364,7 @@ read_targets = [
337
364
  ["xlsxrb (In-Memory)", "xlsxrb_inmemory", "In-Memory"],
338
365
  ["creek 2.6.3", "creek", "Streaming"],
339
366
  ["roo 3.0.0", "roo", "Streaming"],
367
+ ["simple_xlsx_reader 5.1.0", "simple_xlsx_reader", "Streaming"],
340
368
  ["xsv 1.4.1", "xsv", "Streaming"],
341
369
  ["rubyXL 3.4.38", "rubyXL", "In-Memory"]
342
370
  ]
data/docs/ARCHITECTURE.md CHANGED
@@ -29,7 +29,7 @@ As a strict rule, **we do not accept dynamic method definitions using `method_mi
29
29
  Even in cases where proxy patterns (e.g. `WorksheetProxy`) or OOXML builder mappings (e.g. `ChartBuilder`, `SeriesBuilder`) would traditionally benefit from dynamic delegation to avoid boilerplate, we explicitly generate and write out those delegations in the source code.
30
30
 
31
31
  ### 1. **`Xlsxrb` Module is the ONLY Entrypoint:**
32
- The `Xlsxrb` module provides the top-level methods: `generate`, `build`, `read`, `foreach`, and `modify`. Users should **never** instantiate internal classes (like `Xlsxrb::Ooxml::WorkbookWriter`) directly.
32
+ The `Xlsxrb` module provides the top-level methods: `read`, `write`, `build`, and `modify`. Users should **never** instantiate internal classes (like `Xlsxrb::Ooxml::WorkbookWriter`) directly.
33
33
 
34
34
  2. **The `@api public` Contract (SemVer Guarantee):**
35
35
  Any module, class, or method tagged with `# @api public` in its YARD documentation is guaranteed to follow Semantic Versioning.
@@ -38,7 +38,7 @@ Even in cases where proxy patterns (e.g. `WorksheetProxy`) or OOXML builder mapp
38
38
  - Major versions (1.x -> 2.x) are the only time breaking changes to `@api public` components are permitted.
39
39
 
40
40
  3. **Block-Yielded Objects are Public APIs:**
41
- All builder objects yielded into blocks (e.g., `writer` in `Xlsxrb.generate { |writer| }`, `sheet` in `writer.sheet { |sheet| }`, `chart` in `sheet.chart { |chart| }`) are explicitly marked as `@api public`. Their exposed methods constitute the DSL and are strictly protected by the SemVer contract.
41
+ All builder objects yielded into blocks (e.g., `writer` in `Xlsxrb.write { |writer| }`, `sheet` in `writer.sheet { |sheet| }`, `chart` in `sheet.chart { |chart| }`) are explicitly marked as `@api public`. Their exposed methods constitute the DSL and are strictly protected by the SemVer contract.
42
42
 
43
43
  ---
44
44
 
@@ -46,10 +46,11 @@ Even in cases where proxy patterns (e.g. `WorksheetProxy`) or OOXML builder mapp
46
46
 
47
47
  ```
48
48
  lib/
49
- xlsxrb.rb # Facade: Xlsxrb.read / .write / .foreach / .generate / .build
49
+ xlsxrb.rb # Facade: Xlsxrb.read / .write / .build / .modify
50
50
  xlsxrb/
51
51
  version.rb # Xlsxrb::VERSION
52
52
  elements.rb # Requires for Elements layer
53
+ stream_row.rb # Lazy/streaming row and cell reader (O(1) memory)
53
54
  ooxml.rb # Requires for Ooxml layer
54
55
  ooxml/ # Layer 1 – Low-level OOXML
55
56
  reader.rb # Xlsxrb::Ooxml::Reader (core reading logic)
@@ -94,19 +95,20 @@ This layer directly handles ZIP extraction, XML parsing (via SAX), and XML gener
94
95
  * **`Xlsxrb::Ooxml::XmlBuilder`**: Emits well-formed XML strings via `<<` to a writable IO, supporting streaming generation without building a DOM.
95
96
  * **Part-specific parsers/writers**: `WorksheetParser`, `SharedStringsParser`, `StylesParser`, `WorkbookParser`, etc., each encapsulating the SAX event handling for one OpenXML part.
96
97
 
97
- ### 2. High-Level Domain Model (The "Elements" Layer)
98
+ ### 2. High-Level Domain Model (The "Elements" Layer) & Streaming Row Layer
98
99
 
99
- **Namespace:** `Xlsxrb::Elements`
100
+ **Namespace:** `Xlsxrb::Elements` and `Xlsxrb::StreamRow`
100
101
 
101
102
  **Responsibility:**
102
103
  This layer provides idiomatic, easy-to-use Ruby objects representing Excel concepts. It utilizes Ruby 3.2+ `Data` classes for immutability and precise structural definition. All domain models are encapsulated here to keep the top-level namespace clean.
103
104
 
104
- **Core Objects (`Data` classes):**
105
- * **`Xlsxrb::Elements::Workbook`**: Represents the entire file structure. Contains `sheets` (Array of Worksheet), shared styles metadata, and `unmapped_data`.
106
- * **`Xlsxrb::Elements::Worksheet`**: Represents a single sheet. Contains `name`, `rows` (Array of Row), `columns` (Array of Column), and sheet-level properties.
107
- * **`Xlsxrb::Elements::Row`**: Represents one row. Contains `index` (0-based), `cells` (Array of Cell), and row-level attributes (height, hidden, etc.).
108
- * **`Xlsxrb::Elements::Column`**: Represents column formatting. Contains `index` (0-based), `width`, and column-level attributes.
109
- * **`Xlsxrb::Elements::Cell`**: Represents a single cell. Contains `row_index`, `column_index` (both 0-based), `value` (Ruby native type), `formula`, `style`, and `unmapped_data`.
105
+ **Core Objects:**
106
+ * **`Xlsxrb::Elements::Workbook`**: Represents the entire file structure (`Data` class). Contains `sheets` (Array of Worksheet), shared styles metadata, and `unmapped_data`.
107
+ * **`Xlsxrb::Elements::Worksheet`**: Represents a single sheet (`Data` class). Contains `name`, `rows` (Array of Row), `columns` (Array of Column), and sheet-level properties.
108
+ * **`Xlsxrb::Elements::Row`**: Represents one in-memory row (`Data` class). Contains `index` (0-based), `cells` (Array of Cell), and row-level attributes.
109
+ * **`Xlsxrb::StreamRow`**: Represents a streaming row with lazy cell parsing. Provides `row.each_cell` / `row.each` for $O(1)$ constant memory streaming, caching cells on-demand if indexed or converted to an array.
110
+ * **`Xlsxrb::Elements::Column`**: Represents column formatting (`Data` class). Contains `index` (0-based), `width`, and column-level attributes.
111
+ * **`Xlsxrb::Elements::Cell`**: Represents a single cell (`Data` class). Contains `row_index`, `column_index` (both 0-based), `value` (Ruby native type), `formula`, `style`, and `unmapped_data`.
110
112
 
111
113
  **Design Principles:**
112
114
  * **Zero-based Indexing:** To maintain consistency with Ruby's core language (Arrays/Enumerable), all indices (rows, columns, and worksheets) are **0-based**. For Excel-style coordination, use string references like `cell("A1")`.
@@ -115,17 +117,19 @@ This layer provides idiomatic, easy-to-use Ruby objects representing Excel conce
115
117
 
116
118
  ### 3. The Facade / Entrypoint Layer
117
119
 
118
- **Namespace:** `Xlsxrb` module methods
120
+ **Namespace:** `Xlsxrb`
119
121
 
120
122
  **Responsibility:**
121
- Acts as the primary bridge, offering both In-Memory and Streaming APIs.
123
+ Acts as the primary bridge, offering symmetric In-Memory and Streaming APIs.
122
124
 
123
125
  | Method | Type | Description |
124
126
  | :--- | :--- | :--- |
125
- | **`Xlsxrb.read(source)`** | In-Memory | Loads the entire file into a `Workbook` object. |
126
- | **`Xlsxrb.foreach(source, **options)`** | **Streaming** | Yields each `Row` one by one. Ideal for large files. |
127
- | **`Xlsxrb.write(target, workbook)`** | In-Memory | Saves a `Workbook` object to a file or IO. |
128
- | **`Xlsxrb.generate(target, &block)`** | **Streaming** | Provides a DSL to stream data directly to a file/IO. |
127
+ | **`Xlsxrb.read(source, &block)`** | **Streaming** | Streams sheets (`StreamSheet`) and rows (`StreamRow`) with $O(1)$ constant memory. |
128
+ | **`Xlsxrb.write(target, &block)`** | **Streaming** | Streams rows directly to file/IO with minimal memory. |
129
+ | **`Xlsxrb.read(source)`** | In-Memory | Loads from file path, IO, or raw binary string into a `Workbook`. |
130
+ | **`Xlsxrb.write(target, wb)`** / **`Xlsxrb.write(wb)`** | In-Memory | Saves `Workbook` to file/IO, or returns raw binary string (single argument). |
131
+ | **`Xlsxrb.build(&block)`** | In-Memory | Builds an immutable `Workbook` using DSL. |
132
+ | **`Xlsxrb.modify(source, target, &block)`** | In-Memory | Updates cells/sheets of an existing workbook. |
129
133
 
130
134
  ### Facade Expansion Policy
131
135
 
@@ -134,7 +138,7 @@ The long-term API goal is that **all spreadsheet features implemented in the low
134
138
  This applies to both:
135
139
 
136
140
  * **In-Memory DSL** (`Xlsxrb.build` -> `WorkbookBuilder` / `WorksheetBuilder`)
137
- * **Streaming DSL** (`Xlsxrb.generate` -> `StreamWriter`)
141
+ * **Streaming DSL** (`Xlsxrb.write` -> `StreamWriter`)
138
142
 
139
143
  The Facade should not expose only a hand-picked subset forever. If a feature is stable and supported in the low-level writer, the default expectation is that it should eventually gain a high-level entry point.
140
144
 
@@ -193,7 +197,7 @@ If a low-level feature is workbook-scoped, do not force it into a worksheet-only
193
197
 
194
198
  For each feature, choose a single primary Facade shape and reuse it across modes:
195
199
 
196
- * `Xlsxrb.build` and `Xlsxrb.generate` should feel structurally similar
200
+ * `Xlsxrb.build` and `Xlsxrb.write` should feel structurally similar
197
201
  * streaming and in-memory APIs may differ internally, but the surface API should remain as close as possible
198
202
  * differences are acceptable only when memory or ordering constraints make them unavoidable
199
203
 
@@ -291,28 +295,26 @@ Ooxml::WorkbookWriter ── writes workbook.xml, styles.xml, sharedStrings
291
295
  Ooxml::ZipWriter ── writes ZIP output ──► target (path / IO)
292
296
  ```
293
297
 
294
- ### `Xlsxrb.foreach(source, **options)` — Streaming Read
298
+ ### `Xlsxrb.read(source, &block)` — Streaming Read
295
299
 
296
300
  ```
297
- source (path / IO)
301
+ source (path / IO / binary string)
298
302
 
299
303
 
300
304
  Ooxml::ZipReader ── locates xl/sharedStrings.xml, xl/worksheets/sheetN.xml
301
305
 
302
306
  ▼ (SAX parse SST first — kept in memory as a flat Array of strings)
303
307
 
304
- ▼ (then SAX stream worksheet)
305
- Ooxml::WorksheetParser ── on each </row> event:
306
- 1. build Elements::Row with resolved cell values
307
- │ 2. yield Row to caller's block
308
- │ 3. discard Row (GC eligible)
308
+ ▼ (then SAX stream worksheet with StreamRow lazy cell scanner)
309
+ Ooxml::WorksheetParser ── yields StreamRow to caller's block
310
+
309
311
 
310
- caller's block receives Elements::Row, processes, moves on
312
+ caller's block receives StreamRow, streams cells via each_cell with O(1) memory
311
313
  ```
312
314
 
313
- Key memory invariant: only **one Row** (plus the shared-string table) is alive at any time.
315
+ Key memory invariant: only **one Row / Cell** (plus the shared-string table) is parsed at any time.
314
316
 
315
- ### `Xlsxrb.generate(target, &block)` — Streaming Write
317
+ ### `Xlsxrb.write(target, &block)` — Streaming Write
316
318
 
317
319
  ```
318
320
  caller's block
@@ -418,7 +420,7 @@ Every new high-level DSL feature must satisfy the following quality rules before
418
420
  If a feature is intended to exist in both writing modes, tests must cover:
419
421
 
420
422
  * `Xlsxrb.build` / `Xlsxrb.write`
421
- * `Xlsxrb.generate`
423
+ * `Xlsxrb.write`
422
424
 
423
425
  If a feature can only exist in one mode for a technical reason, that restriction must be documented explicitly in code comments and user-facing docs.
424
426