xlsxrb 0.1.2 โ 0.1.3
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/README.md +57 -264
- data/Rakefile +137 -5
- data/benchmark.rb +23 -14
- data/docs/DEVELOPMENT.md +67 -0
- data/docs/coi-serviceworker.js +82 -0
- data/docs/office_thread.js +77 -0
- data/docs/preview.html +719 -0
- data/docs/visual/VisualGallery.md +0 -158
- data/docs/wasm/wasm_doc_helper.css +256 -94
- data/docs/wasm/wasm_doc_helper.js +261 -142
- data/docs/zeta.js +1107 -0
- data/lib/xlsxrb/ooxml/worksheet_parser.rb +178 -24
- data/lib/xlsxrb/ooxml.rb +1 -1
- data/lib/xlsxrb/version.rb +1 -1
- metadata +6 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 985967acd75137e7175ca5ed93493f4fcc001696499e1afdfe0c9d4e5fbaad97
|
|
4
|
+
data.tar.gz: f59bddd81cc312ecda261ef6875daffdfeff6571650298aa0cd56956318db07a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: c88eb2065c70cf4d0fab24508fc168601483fdade7e81593a186272d7fd11de72f9898f178ff591f466d4d3abd14c885c6193238572256a3a08eee56073355d1
|
|
7
|
+
data.tar.gz: a7c80be810fb05b948680492fb917f38f362460f1c095962fc090ed3d399317f83466c47f8b7ad3229c8a3bfa013a6ed23daaa174ef5d9278ef7ccf3ab36f6d8
|
data/README.md
CHANGED
|
@@ -18,14 +18,18 @@ The Ruby ecosystem already has great XLSX libraries. Each is well-designed for i
|
|
|
18
18
|
|
|
19
19
|
Each of these libraries makes deliberate tradeoffs, and they do so thoughtfully. Some focus exclusively on highly efficient reading or writing by streaming data, while others provide a rich API for complex, in-memory document modifications.
|
|
20
20
|
|
|
21
|
-
|
|
21
|
+
Traditionally, attempting to build a "complete package" that offers both reading and writing, rich features, high performance, strict compatibility, and comprehensive documentation presents an inherent open-source challenge: the cumulative maintenance overhead often exceeds the capacity of individual human maintainers.
|
|
22
|
+
|
|
23
|
+
`xlsxrb` is born from a different premise. We believe that Advanced Agentic AI (AI Coders) can help manage this maintenance demand. By utilizing AI agents to automate rigorous E2E testing, visual regression testing, specification compliance checks, and documentation updates, we can reconcile these competing engineering requirements. This allows us to build and continuously maintain a feature-rich, high-performance, and deeply compatible "all-in-one" XLSX library that remains sustainable for the long run.
|
|
22
24
|
|
|
23
25
|
### Design Principles
|
|
24
26
|
|
|
25
|
-
-
|
|
26
|
-
-
|
|
27
|
-
-
|
|
28
|
-
-
|
|
27
|
+
- Minimal Dependencies (Zero Core Logic Dependencies): This library avoids heavy third-party XLSX/XML/ZIP gems, building all core parsing and writing features purely on the Ruby standard library and bundled gems (`zlib`, `rexml`, etc.). The only runtime dependency is `opentelemetry-api`, which provides zero-overhead observability. If you do not configure an OpenTelemetry SDK in your application, it acts as a lightweight no-op, keeping the runtime footprint extremely small.
|
|
28
|
+
- Streaming Support: Both reading and writing are designed to handle large files efficiently by streaming data, keeping memory usage low and predictable.
|
|
29
|
+
- Memory-Efficient XML Parsing: For reading operations, the library uses a custom byte-level streaming parser for worksheet rows (with targeted SAX parsing where appropriate) instead of DOM-based parsing, so entire XML documents are never loaded into memory. This enables true streaming capability for large spreadsheets.
|
|
30
|
+
- Strict Microsoft Excel & OpenXML Interoperability: It is designed to closely follow the Microsoft Office implementation of the ISO 29500 standard. We ensure absolute bidirectional compatibility (both reading and writing) with Microsoft Excel by continuously validating files against the official Microsoft [Open XML SDK](https://github.com/dotnet/Open-XML-SDK).
|
|
31
|
+
- AI-Agent Assisted Maintenance (Managing the Engineering Tradeoff): Building a library that is specification-compliant, rich in features, highly compatible, well-documented, and extremely fast typically presents a substantial maintenance challenge. `xlsxrb` addresses this inherent constraint by leveraging Advanced Agentic AI (AI Coders) to automate testing, feature expansion, and compatibility verification. This AI-assisted development process supports the project's long-term sustainability and high software quality.
|
|
32
|
+
- Modern Ruby 4.0+: Built for the future with Ruby 4.0 or higher.
|
|
29
33
|
|
|
30
34
|
## Installation
|
|
31
35
|
|
|
@@ -43,7 +47,9 @@ On Ruby 4+, some components used by `xlsxrb` and its test suite are shipped as b
|
|
|
43
47
|
|
|
44
48
|
## Interactive Playground (WebAssembly)
|
|
45
49
|
|
|
46
|
-
You can try `xlsxrb` directly in your browser without installing anything!
|
|
50
|
+
You can try `xlsxrb` directly in your browser without installing anything!
|
|
51
|
+
|
|
52
|
+
[๐ Try the Live Demo / Interactive Playground](https://niku.github.io/xlsxrb/docs/visual/VisualGallery_md.html)
|
|
47
53
|
|
|
48
54
|
We have integrated an interactive WebAssembly-powered playground into our RDoc documentation. You can edit the code examples, run them in the browser sandbox, and download the generated `.xlsx` spreadsheets immediately.
|
|
49
55
|
|
|
@@ -56,37 +62,18 @@ To launch the playground locally:
|
|
|
56
62
|
```bash
|
|
57
63
|
bundle exec rake doc:preview
|
|
58
64
|
```
|
|
59
|
-
3. Open [http://localhost:8000](http://localhost:8000) in your browser and click the
|
|
65
|
+
3. Open [http://localhost:8000](http://localhost:8000) in your browser, hover over any code block, and click the "Live Preview" or "Download XLSX" buttons!
|
|
60
66
|
|
|
61
67
|
## Usage
|
|
62
68
|
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
`xlsxrb` offers two different approaches to reading and writing XLSX files: **Streaming** and **In-Memory**.
|
|
69
|
+
`xlsxrb` supports both low-memory Streaming (recommended for large files) and full In-Memory document manipulation (for random-access cell modifications or updating existing sheets).
|
|
66
70
|
|
|
67
|
-
|
|
71
|
+
For visual demonstrations of various features, check the [Visual Examples Gallery](docs/visual/VisualGallery.md).
|
|
68
72
|
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
### 1. Streaming (Recommended)
|
|
72
|
-
|
|
73
|
-
#### Streaming Read
|
|
74
|
-
|
|
75
|
-
Read rows one at a time without loading the entire file into memory:
|
|
76
|
-
|
|
77
|
-
```ruby
|
|
78
|
-
require "xlsxrb"
|
|
79
|
-
|
|
80
|
-
# Yields Xlsxrb::Elements::Row objects for the first sheet
|
|
81
|
-
Xlsxrb.foreach("large_file.xlsx", sheet: 0) do |row|
|
|
82
|
-
puts "Row #{row.index}: #{row.cells.map(&:value).join(', ')}"
|
|
83
|
-
end
|
|
84
|
-
```
|
|
73
|
+
### Quick Start: Streaming (Recommended)
|
|
85
74
|
|
|
86
75
|
#### Streaming Write
|
|
87
|
-
|
|
88
76
|
Generate large files efficiently by writing data directly to the file stream:
|
|
89
|
-
|
|
90
77
|
```ruby
|
|
91
78
|
require "xlsxrb"
|
|
92
79
|
|
|
@@ -99,275 +86,81 @@ Xlsxrb.generate("large_output.xlsx") do |writer|
|
|
|
99
86
|
end
|
|
100
87
|
```
|
|
101
88
|
|
|
102
|
-
|
|
103
|
-
|
|
104
|
-
### 2. In-Memory
|
|
105
|
-
|
|
106
|
-
#### Reading into memory
|
|
107
|
-
|
|
108
|
-
Parse the entire workbook for random access by cell reference:
|
|
109
|
-
|
|
110
|
-
```ruby
|
|
111
|
-
require "xlsxrb"
|
|
112
|
-
|
|
113
|
-
workbook = Xlsxrb.read("example.xlsx")
|
|
114
|
-
sheet = workbook.sheets.first
|
|
115
|
-
|
|
116
|
-
# Random access by cell reference
|
|
117
|
-
puts "Value at C10: #{sheet.cell_value("C10")}"
|
|
118
|
-
```
|
|
119
|
-
|
|
120
|
-
#### Writing from memory
|
|
121
|
-
|
|
122
|
-
Construct a workbook hierarchy in memory using the DSL and write it:
|
|
123
|
-
|
|
124
|
-
```ruby
|
|
125
|
-
require "xlsxrb"
|
|
126
|
-
|
|
127
|
-
workbook = Xlsxrb.build do |w|
|
|
128
|
-
w.add_sheet("My Sheet") do |s|
|
|
129
|
-
s.add_row(["Hello", "World"])
|
|
130
|
-
end
|
|
131
|
-
end
|
|
132
|
-
|
|
133
|
-
Xlsxrb.write("output.xlsx", workbook)
|
|
134
|
-
```
|
|
135
|
-
|
|
136
|
-
#### Modifying an existing workbook
|
|
137
|
-
|
|
138
|
-
Read an existing file, make changes, and write it back:
|
|
139
|
-
|
|
89
|
+
#### Streaming Read
|
|
90
|
+
Read rows one at a time without loading the entire file into memory:
|
|
140
91
|
```ruby
|
|
141
92
|
require "xlsxrb"
|
|
142
93
|
|
|
143
|
-
Xlsxrb.
|
|
144
|
-
|
|
145
|
-
row0 = sheet.row_at(0)
|
|
146
|
-
|
|
147
|
-
new_cell = Xlsxrb::Elements::Cell.new(row_index: 0, column_index: 1, value: "Updated")
|
|
148
|
-
new_row = row0.with(cells: row0.cells.map { |c| c.column_index == 1 ? new_cell : c })
|
|
149
|
-
new_sheet = sheet.with(rows: sheet.rows.map { |r| r.index == 0 ? new_row : r })
|
|
150
|
-
|
|
151
|
-
wb.with(sheets: wb.sheets.map.with_index { |s, i| i == 0 ? new_sheet : s })
|
|
152
|
-
end
|
|
153
|
-
```
|
|
154
|
-
|
|
155
|
-
_For details on adding charts or shapes in-memory, see the [Visual Examples Gallery](docs/visual/VisualGallery.md)._
|
|
156
|
-
|
|
157
|
-
## Specification
|
|
158
|
-
|
|
159
|
-
## Sheet Features
|
|
160
|
-
|
|
161
|
-
All sheet-level features work identically in both the streaming (`Xlsxrb.generate`) and in-memory (`Xlsxrb.build`) APIs. Detailed code examples, runnable scripts, and side-by-side visual rendering results (screenshots) for all sheet-level features are available in the [Visual Gallery (docs/visual/VisualGallery.md)](docs/visual/VisualGallery.md).
|
|
162
|
-
|
|
163
|
-
- **[Formulas](docs/visual/VisualGallery.md#cell-formulas)** - Write standard or Excel formulas with automatic evaluation.
|
|
164
|
-
- **[Hyperlinks](docs/visual/VisualGallery.md#basic-data)** - Attach web URLs or jump to internal worksheet cell locations.
|
|
165
|
-
- **[Auto Filter](docs/visual/VisualGallery.md#interactive-autofilter)** - Add interactive filter drop-down menus to column ranges.
|
|
166
|
-
- **[Data Validation](docs/visual/VisualGallery.md#interactive-validation-list)** - Restrict allowed cell values (lists, range limits, dates, times, custom expressions) with custom warning popups.
|
|
167
|
-
- **[Conditional Formatting](docs/visual/VisualGallery.md#cf-begins-with)** - Highlight cells automatically using rule comparisons, color scales, data bars, icon sets, or formula expressions.
|
|
168
|
-
- **[Tables](docs/visual/VisualGallery.md#borders)** - Wrap cell ranges in structured tables with column headers and preset visual styles.
|
|
169
|
-
- **[Pivot Tables](docs/visual/VisualGallery.md#pivot-tables)** - Summarize data ranges dynamically with row/column grouping and subtotals.
|
|
170
|
-
- **[Comments](docs/visual/VisualGallery.md#interactive-comments)** - Attach hover-activated popup notes to specific cells.
|
|
171
|
-
- **[Merge Cells](docs/visual/VisualGallery.md#merge-freeze)** - Combine rectangular cell ranges into single styled cells.
|
|
172
|
-
- **[Freeze & Split Panes](docs/visual/VisualGallery.md#merge-freeze)** - Freeze top rows or left columns (or split viewport by pixel offsets) to keep headers visible.
|
|
173
|
-
- **[Page margins & Print Setup](docs/visual/VisualGallery.md#page-setup)** - Set portrait/landscape orientation, margins, paper sizes, scaling (fit-to-page), and odd/even headers & footers.
|
|
174
|
-
- **[Print Options](docs/visual/VisualGallery.md#page-grid-lines-print)** - Toggle gridlines, row/column headings visibility when printing, and centering.
|
|
175
|
-
- **[Sheet Protection](docs/visual/VisualGallery.md#sheet-protection)** - Lock sheet structures, formatting, or objects (optionally with SHA-512 hashed passwords).
|
|
176
|
-
- **[Page Breaks](docs/visual/VisualGallery.md#page-setup)** - Insert manual horizontal or vertical page breaks.
|
|
177
|
-
- **[Images](docs/visual/VisualGallery.md#embedded-images)** - Embed and anchor floating images (PNG, JPEG) with custom dimensions.
|
|
178
|
-
- **[Sparklines](docs/visual/VisualGallery.md#sparkline-column)** - Insert inline column or line trend charts into single cells.
|
|
179
|
-
- **[Shapes & Drawings](docs/visual/VisualGallery.md#shapes)** - Draw rectangles, ellipses, and other presets with solid fills, lines, and custom text.
|
|
180
|
-
- **[Sheet Properties and View Settings](docs/visual/VisualGallery.md#sheet-tab-colors)** - Customize per-sheet properties (tab colors, hide gridlines, default zoom levels).
|
|
181
|
-
|
|
182
|
-
## Workbook Features
|
|
183
|
-
|
|
184
|
-
Workbook-level methods are called directly on the writer/builder object (not inside `add_sheet`).
|
|
185
|
-
|
|
186
|
-
### Defined Names, Print Area, and Print Titles
|
|
187
|
-
|
|
188
|
-
```ruby
|
|
189
|
-
Xlsxrb.generate("named.xlsx") do |w|
|
|
190
|
-
w.add_sheet("Data") do |s|
|
|
191
|
-
s.add_row(["Name", "Score"])
|
|
192
|
-
s.add_row(["Alice", 95])
|
|
193
|
-
end
|
|
194
|
-
|
|
195
|
-
# General named range
|
|
196
|
-
w.add_defined_name("TaxRate", "0.1", hidden: false)
|
|
197
|
-
|
|
198
|
-
# Print area for the sheet named "Data"
|
|
199
|
-
w.set_print_area("A1:B10", sheet: "Data")
|
|
200
|
-
|
|
201
|
-
# Repeat first row and first column when printing
|
|
202
|
-
w.set_print_titles(rows: "1:1", cols: "A:A", sheet: "Data")
|
|
203
|
-
end
|
|
204
|
-
```
|
|
205
|
-
|
|
206
|
-
### Workbook Protection
|
|
207
|
-
|
|
208
|
-
Lock the workbook structure (prevents adding/removing/renaming sheets).
|
|
209
|
-
|
|
210
|
-
```ruby
|
|
211
|
-
Xlsxrb.generate("locked_workbook.xlsx") do |w|
|
|
212
|
-
w.add_sheet("Sheet1") { |s| s.add_row(["Data"]) }
|
|
213
|
-
|
|
214
|
-
w.set_workbook_protection(lock_structure: true, lock_windows: false)
|
|
94
|
+
Xlsxrb.foreach("large_file.xlsx", sheet: 0) do |row|
|
|
95
|
+
puts "Row #{row.index}: #{row.cells.map(&:value).join(', ')}"
|
|
215
96
|
end
|
|
216
97
|
```
|
|
217
98
|
|
|
218
|
-
|
|
99
|
+
*(For In-Memory document building, cell modifications, or template updating, please refer to the detailed RDoc API documentation).*
|
|
219
100
|
|
|
220
|
-
|
|
101
|
+
## Feature Support & ECMA-376 Compliance
|
|
221
102
|
|
|
222
|
-
|
|
223
|
-
Xlsxrb.generate("with_props.xlsx") do |w|
|
|
224
|
-
w.add_sheet("Sheet1") { |s| s.add_row(["Hello"]) }
|
|
225
|
-
|
|
226
|
-
# Core properties (Dublin Core โ visible in File > Info)
|
|
227
|
-
w.set_core_property(:title, "Quarterly Report")
|
|
228
|
-
w.set_core_property(:subject, "Sales data Q1 2026")
|
|
229
|
-
w.set_core_property(:creator, "Alice")
|
|
230
|
-
w.set_core_property(:keywords, "sales, report, 2026")
|
|
231
|
-
w.set_core_property(:description, "Auto-generated by xlsxrb")
|
|
232
|
-
|
|
233
|
-
# App properties (application-level metadata)
|
|
234
|
-
w.set_app_property(:application, "MyApp/2.0")
|
|
235
|
-
w.set_app_property(:company, "Acme Corp")
|
|
236
|
-
|
|
237
|
-
# Custom properties (arbitrary key/value pairs)
|
|
238
|
-
w.add_custom_property("ReportVersion", "42", type: :integer)
|
|
239
|
-
w.add_custom_property("ApprovedBy", "Bob", type: :string)
|
|
240
|
-
w.add_custom_property("Published", true, type: :bool)
|
|
241
|
-
end
|
|
242
|
-
```
|
|
103
|
+
`xlsxrb` is designed for full interoperability and strict compliance with the ECMA-376 (Office Open XML) Transitional specification. It supports nearly all major spreadsheet features required for business reports:
|
|
243
104
|
|
|
244
|
-
|
|
105
|
+
* Cells & Layout: Formulas, Hyperlinks, Merge Cells, Freeze & Split Panes, Page Setup (margins, headers/footers, scaling, gridlines).
|
|
106
|
+
* Data & Controls: Auto Filters, Data Validations (dropdowns, range limits), Sheet Protection.
|
|
107
|
+
* Formatting & Styling: Rich Text, Cell Tables, Conditional Formatting (color scales, data bars, icon sets).
|
|
108
|
+
* Graphics & Charts: Embedded Images, Shapes & Drawings, Sparklines, Charts (Line, Bar, Pie, Area, Radar, Scatter).
|
|
109
|
+
* Workbook Level: Defined Names, Print Areas, Workbook Protection, and Document Metadata (core, app, custom properties).
|
|
245
110
|
|
|
246
111
|
For detailed specification references and policies, see [SPEC_SOURCES.md](docs/SPEC_SOURCES.md).
|
|
247
112
|
|
|
248
113
|
## Benchmarks
|
|
249
114
|
|
|
250
|
-
The following benchmarks measure the time and memory required to process
|
|
251
|
-
|
|
252
|
-
### Write Performance (100,000 cells)
|
|
253
|
-
|
|
254
|
-
| Library | Time | Peak Memory | GC Count |
|
|
255
|
-
| ---------------------- | ------ | ----------- | -------- |
|
|
256
|
-
| xlsxtream (Streaming) | 0.15 s | 65.0 MB | 4.4 |
|
|
257
|
-
| fast_excel (Streaming) | 0.28 s | 64.4 MB | 2.0 |
|
|
258
|
-
| xlsxrb (Streaming) | 0.34 s | 64.4 MB | 19.0 |
|
|
259
|
-
| caxlsx (In-Memory) | 1.19 s | 73.8 MB | 5.0 |
|
|
260
|
-
| xlsxrb (In-Memory) | 3.06 s | 132.4 MB | 34.0 |
|
|
261
|
-
| rubyXL (In-Memory) | 5.45 s | 274.8 MB | 33.0 |
|
|
262
|
-
|
|
263
|
-
### Read Performance (100,000 cells)
|
|
264
|
-
|
|
265
|
-
| Library | Time | Peak Memory | GC Count |
|
|
266
|
-
| ------------------ | ------ | ----------- | -------- |
|
|
267
|
-
| xlsxrb (Streaming) | 1.02 s | 67.4 MB | 192.0 |
|
|
268
|
-
| creek (Streaming) | 1.32 s | 160.4 MB | 396.8 |
|
|
269
|
-
| roo (Streaming) | 1.33 s | 87.4 MB | 31.0 |
|
|
270
|
-
| xlsxrb (In-Memory) | 1.82 s | 153.9 MB | 20.0 |
|
|
271
|
-
| xsv (Streaming) | 3.51 s | 92.1 MB | 98.0 |
|
|
272
|
-
| rubyXL (In-Memory) | 4.95 s | 265.9 MB | 38.0 |
|
|
115
|
+
The following benchmarks measure the time and memory required to process a 1,000,000 cells (100,000 rows ร 10 columns) spreadsheet, demonstrating `xlsxrb`'s memory consumption and processing times.
|
|
273
116
|
|
|
274
117
|
### Write Performance (1,000,000 cells)
|
|
275
118
|
|
|
276
119
|
| Library | Time | Peak Memory | GC Count |
|
|
277
120
|
| ---------------------- | ------- | ----------- | -------- |
|
|
278
|
-
| xlsxtream (Streaming) | 0.
|
|
279
|
-
| fast_excel (Streaming) |
|
|
280
|
-
|
|
|
281
|
-
|
|
|
282
|
-
| xlsxrb (In-Memory) |
|
|
283
|
-
| rubyXL (In-Memory) |
|
|
121
|
+
| xlsxtream (Streaming) | 0.12 s | 64.9 MB | 9.0 |
|
|
122
|
+
| fast_excel (Streaming) | 1.34 s | 64.5 MB | 28.0 |
|
|
123
|
+
| caxlsx (In-Memory) | 2.64 s | 142.7 MB | 16.0 |
|
|
124
|
+
| xlsxrb (Streaming) | 3.32 s | 216.9 MB | 65.0 |
|
|
125
|
+
| xlsxrb (In-Memory) | 6.23 s | 431.2 MB | 68.0 |
|
|
126
|
+
| rubyXL (In-Memory) | 37.16 s | 2105.9 MB | 90.0 |
|
|
284
127
|
|
|
285
128
|
### Read Performance (1,000,000 cells)
|
|
286
129
|
|
|
287
130
|
| Library | Time | Peak Memory | GC Count |
|
|
288
131
|
| ------------------ | ------- | ----------- | -------- |
|
|
289
|
-
| xlsxrb (Streaming) |
|
|
290
|
-
|
|
|
291
|
-
|
|
|
292
|
-
|
|
|
293
|
-
|
|
|
294
|
-
| rubyXL (In-Memory) |
|
|
295
|
-
|
|
296
|
-
For reference, the following specification files from the Ecma International website are located in the `vendor/docs/` directory:
|
|
297
|
-
|
|
298
|
-
- `vendor/docs/ECMA-376-Part1/Ecma Office Open XML Part 1 - Fundamentals And Markup Language Reference.pdf`: Part 1 - Fundamentals And Markup Language Reference
|
|
299
|
-
- `vendor/docs/ECMA-376-Part2/ECMA-376-2_5th_edition_december_2021.pdf`: Part 2 - Open Packaging Conventions
|
|
300
|
-
- `vendor/docs/ECMA-376-Part3/ECMA-376-3_5th_edition_december_2015.pdf`: Part 3 - Markup Compatibility and Extensibility
|
|
301
|
-
- `vendor/docs/ECMA-376-Part4/Ecma Office Open XML Part 4 - Transitional Migration Features.pdf`: Part 4 - Transitional Migration Features
|
|
302
|
-
|
|
303
|
-
## Testing Strategy
|
|
304
|
-
|
|
305
|
-
To ensure high quality and strict compliance with the ECMA-376 specification while maintaining a fast development loop, we employ a 4-tier testing strategy:
|
|
306
|
-
|
|
307
|
-
1. **Unit Tests (Fast & No Dependencies):**
|
|
308
|
-
- Verify individual components (Writer, Reader, Packaging) using only the Ruby standard library and bundled gems.
|
|
309
|
-
- **Round-Trip Testing:** Ensure that files generated by the `xlsxrb` Writer can be seamlessly and accurately parsed back by the `xlsxrb` Reader.
|
|
310
|
-
- Run via: `bundle exec rake test:unit`
|
|
311
|
-
|
|
312
|
-
2. **Contract Tests (API Consistency):**
|
|
313
|
-
- Execute the same data scenario through both the Streaming (`Xlsxrb.generate`) and In-Memory (`Xlsxrb.build` + `Xlsxrb.write`) APIs.
|
|
314
|
-
- Verify that both APIs produce semantically identical OOXML output.
|
|
315
|
-
- Run via: `bundle exec rake test:contract`
|
|
316
|
-
|
|
317
|
-
3. **Interoperability Testing (E2E):**
|
|
318
|
-
- Utilize the official **[Open XML SDK](https://github.com/dotnet/Open-XML-SDK)** for robust, two-way verification:
|
|
319
|
-
- **Writer Validation:** Files generated by `xlsxrb` are read and validated using the Open XML SDK validator.
|
|
320
|
-
- **Reader Validation:** Complex, real-world XLSX files generated by the SDK are parsed and verified by the `xlsxrb` reader.
|
|
321
|
-
- Requires `.NET SDK`.
|
|
322
|
-
- Run via: `bundle exec rake test:e2e`
|
|
323
|
-
|
|
324
|
-
4. **Visual Examples & VRT (Visual Regression Testing):**
|
|
325
|
-
- **Visual Gallery:** Automatically compiles example DSL scripts under `examples/visual/` into the [Visual Examples Gallery](docs/visual/VisualGallery.md).
|
|
326
|
-
- **Visual Regression Testing:** Renders generated spreadsheets to PNGs using a headless LibreOffice Calc engine, and compares them against baselines to catch rendering errors or layout regressions.
|
|
327
|
-
- Requires `libreoffice-calc`, `poppler-utils` (`pdftoppm`), and `imagemagick`.
|
|
328
|
-
- Run via: `bundle exec rake test:visual`
|
|
329
|
-
|
|
330
|
-
**Note on Test Environments:**
|
|
331
|
-
All necessary dependencies (.NET SDK, LibreOffice, etc.) are pre-configured in the repository's **Dev Container** setup for a consistent local and CI experience.
|
|
332
|
-
|
|
333
|
-
## Development
|
|
334
|
-
|
|
335
|
-
This project is designed to be developed using [Dev Containers](https://containers.dev/). The provided `.devcontainer` configuration includes all necessary tools (Ruby 4.0, and the .NET SDK for E2E testing) to ensure a consistent development environment.
|
|
132
|
+
| xlsxrb (Streaming) | 5.38 s | 101.8 MB | 1429.0 |
|
|
133
|
+
| creek (Streaming) | 7.05 s | 706.2 MB | 3985.0 |
|
|
134
|
+
| roo (Streaming) | 7.24 s | 128.1 MB | 279.0 |
|
|
135
|
+
| xlsxrb (In-Memory) | 8.62 s | 996.1 MB | 28.0 |
|
|
136
|
+
| xsv (Streaming) | 14.41 s | 93.5 MB | 998.0 |
|
|
137
|
+
| rubyXL (In-Memory) | 24.58 s | 1856.8 MB | 127.0 |
|
|
336
138
|
|
|
337
|
-
|
|
139
|
+
### Running the Benchmarks Locally
|
|
338
140
|
|
|
339
|
-
|
|
141
|
+
The benchmark data is gathered using the bundled [benchmark.rb](file:///workspaces/xlsxrb/benchmark.rb) script, which runs each library's code in isolated subprocesses to ensure accurate memory and GC measurements.
|
|
340
142
|
|
|
341
|
-
|
|
143
|
+
To run the benchmark locally for 1,000,000 cells (100,000 rows):
|
|
144
|
+
```bash
|
|
145
|
+
ruby benchmark.rb 100000
|
|
146
|
+
```
|
|
342
147
|
|
|
343
|
-
|
|
148
|
+
## Testing & Quality Assurance
|
|
344
149
|
|
|
345
|
-
|
|
346
|
-
2. **Writer Unit Tests:** Write unit tests for the Writer component targeting this feature.
|
|
347
|
-
3. **Writer Implementation:** Implement the Writer functionality.
|
|
348
|
-
4. **Run Writer Tests:** Execute the Writer unit tests. If they fail, return to step 3.
|
|
349
|
-
5. **Writer E2E & Validation:** Test the Writer's generated XLSX file using the Open XML SDK. This includes structural validation using `OpenXmlValidator`. If the test or validation fails, return to step 2.
|
|
350
|
-
6. **Reader Unit Tests:** Write unit tests for the Reader component. **Crucially, include round-trip tests** to ensure the Reader can perfectly parse the output of your Writer.
|
|
351
|
-
7. **Reader Implementation:** Implement the Reader functionality.
|
|
352
|
-
8. **Run Reader Tests:** Execute the Reader unit tests. If they fail, return to step 6 or 7. If the round-trip test reveals a structural flaw in the Writer's output, return all the way back to step 2.
|
|
353
|
-
9. **Reader E2E:** Verify that the Reader can successfully parse a valid XLSX file generated by the Open XML SDK that includes the new feature. If it fails, return to step 6 or 7.
|
|
354
|
-
10. **Full Test Suite:** Run the entire test suite (`rake test`). If any tests fail, trace back to the appropriate step.
|
|
355
|
-
11. **Commit:** Commit the changes. The commit message must clearly describe the specific feature implemented in this cycle.
|
|
356
|
-
12. **Next Feature:** Proceed to the next feature and return to step 1.
|
|
150
|
+
To support reliability, compliance with the ECMA-376 specification, and consistent updates, `xlsxrb` is backed by a 4-tier testing strategy:
|
|
357
151
|
|
|
358
|
-
|
|
152
|
+
1. Round-Trip Testing: Unit tests verify that every generated sheet can be reliably parsed back by the reader with identical content and styling.
|
|
153
|
+
2. Contract Consistency: The library ensures semantic output consistency between the Streaming (`Xlsxrb.generate`) and In-Memory (`Xlsxrb.build`) APIs.
|
|
154
|
+
3. Official Open XML SDK Validation (E2E): Every feature is validated against the official Microsoft Open XML SDK. Generated spreadsheets are structurally checked to prevent file corruption warnings.
|
|
155
|
+
4. Visual Regression Testing (VRT): To guarantee rendering correctness, generated XLSX files are rendered via a headless LibreOffice Calc engine, and compared pixel-by-pixel against visual baselines.
|
|
359
156
|
|
|
360
|
-
|
|
157
|
+
For details on running the tests locally or within our pre-configured Dev Container, see [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md).
|
|
361
158
|
|
|
362
|
-
|
|
159
|
+
## Development
|
|
363
160
|
|
|
364
|
-
|
|
161
|
+
We welcome contributions! The project is configured with a ready-to-use Dev Container to streamline local environment setup.
|
|
365
162
|
|
|
366
|
-
|
|
367
|
-
2. No new XML element or branch is introduced.
|
|
368
|
-
3. Unit tests and round-trip tests fully cover the new behaviour.
|
|
369
|
-
4. `rake test` passes with Open XML SDK validation included.
|
|
370
|
-
5. The commit message explicitly names the existing E2E scenario that provides coverage and states why a new scenario adds no value.
|
|
163
|
+
For contribution guidelines, E2E testing policies, and the step-by-step development workflow, please refer to [docs/DEVELOPMENT.md](docs/DEVELOPMENT.md).
|
|
371
164
|
|
|
372
165
|
To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
|
|
373
166
|
|
data/Rakefile
CHANGED
|
@@ -291,15 +291,119 @@ task :wasm do
|
|
|
291
291
|
end
|
|
292
292
|
end
|
|
293
293
|
|
|
294
|
+
desc "Fetch required external assets for offline usage"
|
|
295
|
+
task :fetch_assets do
|
|
296
|
+
require 'open-uri'
|
|
297
|
+
require 'fileutils'
|
|
298
|
+
|
|
299
|
+
def download_file(url, dest)
|
|
300
|
+
# Check if the file exists and is not a tiny placeholder/error document
|
|
301
|
+
return if File.exist?(dest) && File.size(dest) > 1024
|
|
302
|
+
|
|
303
|
+
puts "Downloading #{url} to #{dest}..."
|
|
304
|
+
FileUtils.mkdir_p(File.dirname(dest))
|
|
305
|
+
|
|
306
|
+
require 'net/http'
|
|
307
|
+
uri = URI.parse(url)
|
|
308
|
+
temp_dest = dest + ".tmp"
|
|
309
|
+
|
|
310
|
+
begin
|
|
311
|
+
Net::HTTP.start(uri.host, uri.port, use_ssl: uri.scheme == 'https', open_timeout: 15, read_timeout: 90) do |http|
|
|
312
|
+
request = Net::HTTP::Get.new(uri)
|
|
313
|
+
# Specify User-Agent to bypass scraping prevention on CDNs and act as a normal browser
|
|
314
|
+
request['User-Agent'] = "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36"
|
|
315
|
+
# Force raw (identity) encoding to prevent receiving Brotli-compressed (.br) data,
|
|
316
|
+
# which Ruby's Net::HTTP cannot decode automatically, leading to corrupted Wasm files.
|
|
317
|
+
request['Accept-Encoding'] = 'identity'
|
|
318
|
+
|
|
319
|
+
http.request(request) do |response|
|
|
320
|
+
if response.code.to_i != 200
|
|
321
|
+
raise "HTTP error #{response.code}: #{response.message}"
|
|
322
|
+
end
|
|
323
|
+
|
|
324
|
+
File.open(temp_dest, "wb") do |output|
|
|
325
|
+
response.read_body do |chunk|
|
|
326
|
+
output.write(chunk)
|
|
327
|
+
end
|
|
328
|
+
end
|
|
329
|
+
end
|
|
330
|
+
end
|
|
331
|
+
# Atomic rename to prevent leaving incomplete files on failure
|
|
332
|
+
File.rename(temp_dest, dest)
|
|
333
|
+
puts "Downloaded successfully."
|
|
334
|
+
rescue => e
|
|
335
|
+
puts "Failed to download #{url}: #{e.message}"
|
|
336
|
+
File.delete(temp_dest) if File.exist?(temp_dest)
|
|
337
|
+
File.delete(dest) if File.exist?(dest)
|
|
338
|
+
raise "Required asset download failed. Build aborted."
|
|
339
|
+
end
|
|
340
|
+
end
|
|
341
|
+
|
|
342
|
+
def fetch_google_fonts
|
|
343
|
+
css_dest = "docs/fonts/fonts.css"
|
|
344
|
+
return if File.exist?(css_dest)
|
|
345
|
+
|
|
346
|
+
puts "Fetching and localizing Google Fonts..."
|
|
347
|
+
font_url = "https://fonts.googleapis.com/css2?family=Inter:wght@300;400;600;700&family=JetBrains+Mono:wght@400;500&display=swap"
|
|
348
|
+
|
|
349
|
+
css_content = nil
|
|
350
|
+
begin
|
|
351
|
+
# Specify Chrome User-Agent to ensure Google Fonts returns modern and lightweight .woff2 formats
|
|
352
|
+
# instead of legacy formats (like .ttf or .eot) designed for older browsers
|
|
353
|
+
URI.open(font_url, "User-Agent" => "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/120.0.0.0 Safari/537.36") do |f|
|
|
354
|
+
css_content = f.read
|
|
355
|
+
end
|
|
356
|
+
rescue => e
|
|
357
|
+
puts "Failed to fetch Google Fonts CSS: #{e.message}"
|
|
358
|
+
return
|
|
359
|
+
end
|
|
360
|
+
|
|
361
|
+
urls = css_content.scan(/url\((https:\/\/fonts\.gstatic\.com\/[^\)]+)\)/).flatten
|
|
362
|
+
|
|
363
|
+
urls.uniq.each do |url|
|
|
364
|
+
filename = url.split("/").last
|
|
365
|
+
local_path = "docs/fonts/#{filename}"
|
|
366
|
+
download_file(url, local_path)
|
|
367
|
+
css_content.gsub!(url, filename)
|
|
368
|
+
end
|
|
369
|
+
|
|
370
|
+
FileUtils.mkdir_p("docs/fonts")
|
|
371
|
+
File.write(css_dest, css_content)
|
|
372
|
+
puts "Google Fonts localized successfully."
|
|
373
|
+
end
|
|
374
|
+
|
|
375
|
+
# Fetch Ruby WASI JS
|
|
376
|
+
download_file(
|
|
377
|
+
"https://cdn.jsdelivr.net/npm/@ruby/wasm-wasi@2.9.3-2.9.4/dist/browser.umd.js",
|
|
378
|
+
"docs/wasm/browser.umd.js"
|
|
379
|
+
)
|
|
380
|
+
|
|
381
|
+
# Fetch ZetaOffice Wasm
|
|
382
|
+
base_zeta_url = "https://cdn.zetaoffice.net/zetaoffice_latest/"
|
|
383
|
+
%w[soffice.js soffice.wasm soffice.data soffice.data.js.metadata qtloader.js].each do |file|
|
|
384
|
+
download_file(base_zeta_url + file, "docs/zetaoffice/#{file}")
|
|
385
|
+
end
|
|
386
|
+
|
|
387
|
+
# Fetch and localize Google Fonts
|
|
388
|
+
fetch_google_fonts
|
|
389
|
+
end
|
|
390
|
+
|
|
294
391
|
desc "Generate RDoc documentation including Visual Gallery"
|
|
295
|
-
task doc: :wasm do
|
|
392
|
+
task doc: [:wasm, :fetch_assets] do
|
|
296
393
|
FileUtils.rm_rf("doc")
|
|
297
|
-
|
|
394
|
+
|
|
395
|
+
# RDoc ใณใใณใใๅฎ่ก (--exclude ใๆๅฎใใฆ docs ้
ไธใฎใใฌใใฅใผ็จใขใปใใใฎ่ชคใใผในใๅ้ฟ)
|
|
396
|
+
sh "bundle exec rdoc --op doc" \
|
|
397
|
+
" --exclude 'docs/coi-serviceworker\\.js'" \
|
|
398
|
+
" --exclude 'docs/zeta\\.js'" \
|
|
399
|
+
" --exclude 'docs/office_thread\\.js'" \
|
|
400
|
+
" --exclude 'docs/preview\\.html'" \
|
|
401
|
+
" --title 'xlsxrb Documentation' --main README.md README.md \"docs/visual/VisualGallery.md\" lib/"
|
|
298
402
|
|
|
299
403
|
# Copy visual gallery images and files so they are available in RDoc output
|
|
300
404
|
FileUtils.mkdir_p("doc/test/visual/baselines")
|
|
301
|
-
FileUtils.cp_r("test/visual/baselines", "doc/test/visual")
|
|
302
405
|
FileUtils.mkdir_p("doc/test/visual/support/illustrations")
|
|
406
|
+
FileUtils.cp_r(Dir.glob("test/visual/baselines/*"), "doc/test/visual/baselines")
|
|
303
407
|
FileUtils.cp_r(Dir.glob("test/visual/support/illustrations/*.png"), "doc/test/visual/support/illustrations")
|
|
304
408
|
|
|
305
409
|
FileUtils.mkdir_p("doc/docs/visual/files")
|
|
@@ -312,21 +416,35 @@ task doc: :wasm do
|
|
|
312
416
|
FileUtils.mkdir_p("doc/css")
|
|
313
417
|
FileUtils.mkdir_p("doc/js")
|
|
314
418
|
FileUtils.mkdir_p("doc/wasm")
|
|
419
|
+
FileUtils.mkdir_p("doc/zetaoffice")
|
|
420
|
+
FileUtils.mkdir_p("doc/fonts")
|
|
421
|
+
|
|
315
422
|
FileUtils.cp("docs/wasm/wasm_doc_helper.js", "doc/js/wasm_doc_helper.js")
|
|
316
423
|
FileUtils.cp("docs/wasm/wasm_doc_helper.css", "doc/css/wasm_doc_helper.css")
|
|
317
424
|
FileUtils.cp("docs/wasm/ruby.wasm", "doc/wasm/ruby.wasm")
|
|
425
|
+
FileUtils.cp("docs/wasm/browser.umd.js", "doc/wasm/browser.umd.js")
|
|
426
|
+
FileUtils.cp_r(Dir.glob("docs/zetaoffice/*"), "doc/zetaoffice")
|
|
427
|
+
FileUtils.cp_r(Dir.glob("docs/fonts/*"), "doc/fonts")
|
|
428
|
+
|
|
429
|
+
# Copy LibreOffice Wasm Preview assets to doc directory
|
|
430
|
+
FileUtils.cp("docs/preview.html", "doc/preview.html")
|
|
431
|
+
FileUtils.cp("docs/coi-serviceworker.js", "doc/coi-serviceworker.js")
|
|
432
|
+
FileUtils.cp("docs/zeta.js", "doc/zeta.js")
|
|
433
|
+
FileUtils.cp("docs/office_thread.js", "doc/office_thread.js")
|
|
318
434
|
|
|
319
435
|
# Inject stylesheet and javascript loading tags to all generated HTML docs
|
|
320
436
|
Dir.glob("doc/**/*.html").each do |html_path|
|
|
437
|
+
next if File.basename(html_path) == "preview.html"
|
|
321
438
|
html_content = File.read(html_path)
|
|
322
439
|
depth = html_path.sub(%r{\Adoc/}, "").count("/")
|
|
323
440
|
rel_prefix = "../" * depth
|
|
324
441
|
|
|
442
|
+
coi_tag = %Q{<script src="#{rel_prefix}coi-serviceworker.js"></script>}
|
|
325
443
|
js_tag = %Q{<script src="#{rel_prefix}js/wasm_doc_helper.js" defer></script>}
|
|
326
444
|
css_tag = %Q{<link href="#{rel_prefix}css/wasm_doc_helper.css" rel="stylesheet">}
|
|
327
445
|
|
|
328
446
|
if html_content.include?("<body")
|
|
329
|
-
modified = html_content.sub("<body", "#{js_tag}\n#{css_tag}\n<body")
|
|
447
|
+
modified = html_content.sub("<body", "#{coi_tag}\n#{js_tag}\n#{css_tag}\n<body")
|
|
330
448
|
File.write(html_path, modified)
|
|
331
449
|
end
|
|
332
450
|
end
|
|
@@ -346,7 +464,21 @@ namespace :doc do
|
|
|
346
464
|
Port: port,
|
|
347
465
|
DocumentRoot: File.expand_path("doc", __dir__),
|
|
348
466
|
Logger: WEBrick::Log.new(nil, WEBrick::BasicLog::WARN),
|
|
349
|
-
AccessLog: []
|
|
467
|
+
AccessLog: [],
|
|
468
|
+
# If the file is a Brotli-compressed Wasm/Data asset (checked via magic bytes),
|
|
469
|
+
# dynamically inject 'Content-Encoding: br' header so the browser decompresses it natively.
|
|
470
|
+
RequestCallback: ->(req, res) {
|
|
471
|
+
if req.path.end_with?(".wasm") || req.path.end_with?(".data")
|
|
472
|
+
# Resolve physical file path from req.path manually since res.filename is nil at this stage
|
|
473
|
+
local_path = File.join(File.expand_path("doc", __dir__), req.path)
|
|
474
|
+
if File.exist?(local_path)
|
|
475
|
+
first_4 = File.binread(local_path, 4) rescue nil
|
|
476
|
+
if first_4 != "\x00asm"
|
|
477
|
+
res['Content-Encoding'] = 'br'
|
|
478
|
+
end
|
|
479
|
+
end
|
|
480
|
+
end
|
|
481
|
+
}
|
|
350
482
|
)
|
|
351
483
|
|
|
352
484
|
puts "=================================================="
|