xlsxrb 0.1.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.devcontainer/Dockerfile +64 -0
- data/.devcontainer/devcontainer.json +17 -0
- data/CHANGELOG.md +12 -0
- data/CODE_OF_CONDUCT.md +10 -0
- data/LICENSE.txt +21 -0
- data/README.md +384 -0
- data/Rakefile +290 -0
- data/benchmark.rb +390 -0
- data/docs/ARCHITECTURE.md +488 -0
- data/docs/SPEC_SOURCES.md +42 -0
- data/docs/visual/VisualGallery.md +4684 -0
- data/docs/wasm/wasm_doc_helper.css +189 -0
- data/docs/wasm/wasm_doc_helper.js +320 -0
- data/lib/xlsxrb/elements/cell.rb +77 -0
- data/lib/xlsxrb/elements/column.rb +28 -0
- data/lib/xlsxrb/elements/row.rb +47 -0
- data/lib/xlsxrb/elements/types.rb +36 -0
- data/lib/xlsxrb/elements/workbook.rb +47 -0
- data/lib/xlsxrb/elements/worksheet.rb +50 -0
- data/lib/xlsxrb/elements.rb +15 -0
- data/lib/xlsxrb/ooxml/reader.rb +8048 -0
- data/lib/xlsxrb/ooxml/shared_strings_parser.rb +75 -0
- data/lib/xlsxrb/ooxml/styles_parser.rb +194 -0
- data/lib/xlsxrb/ooxml/utils.rb +122 -0
- data/lib/xlsxrb/ooxml/workbook_parser.rb +78 -0
- data/lib/xlsxrb/ooxml/workbook_writer.rb +1000 -0
- data/lib/xlsxrb/ooxml/worksheet_parser.rb +455 -0
- data/lib/xlsxrb/ooxml/worksheet_writer.rb +1008 -0
- data/lib/xlsxrb/ooxml/writer.rb +5450 -0
- data/lib/xlsxrb/ooxml/xml_builder.rb +113 -0
- data/lib/xlsxrb/ooxml/xml_parser.rb +68 -0
- data/lib/xlsxrb/ooxml/zip_generator.rb +162 -0
- data/lib/xlsxrb/ooxml/zip_reader.rb +158 -0
- data/lib/xlsxrb/ooxml/zip_writer.rb +213 -0
- data/lib/xlsxrb/ooxml.rb +22 -0
- data/lib/xlsxrb/style_builder.rb +241 -0
- data/lib/xlsxrb/version.rb +5 -0
- data/lib/xlsxrb.rb +1427 -0
- data/measure_memory.rb +42 -0
- data/sig/xlsxrb.rbs +23 -0
- data/vendor/sdk_runner/Program.cs +91 -0
- data/vendor/sdk_runner/sdk_runner.csproj +13 -0
- metadata +144 -0
|
@@ -0,0 +1,488 @@
|
|
|
1
|
+
# Xlsxrb Architecture
|
|
2
|
+
|
|
3
|
+
Xlsxrb uses a multi-layered architecture to separate low-level OpenXML specification details from a high-level, idiomatic Ruby API. This separation of concerns ensures the library is both robust against the complex OpenXML spec and user-friendly for Ruby developers.
|
|
4
|
+
|
|
5
|
+
## Dependency Constraints
|
|
6
|
+
|
|
7
|
+
Xlsxrb uses **only** the Ruby standard library and Bundled Gems:
|
|
8
|
+
|
|
9
|
+
| Library | Purpose |
|
|
10
|
+
| :--- | :--- |
|
|
11
|
+
| `rexml` (bundled gem) | SAX2-based XML parsing and DOM-based XML generation |
|
|
12
|
+
| `zlib` (stdlib) | ZIP deflate/inflate compression |
|
|
13
|
+
| `stringio` (stdlib) | In-memory IO for streaming |
|
|
14
|
+
| `date` (stdlib) | Excel serial-number ↔ `Date` / `Time` conversion |
|
|
15
|
+
| `openssl`, `securerandom` (stdlib) | Password hashing for sheet/workbook protection |
|
|
16
|
+
|
|
17
|
+
**No third-party gems** (e.g. Nokogiri, rubyzip) are permitted as runtime dependencies.
|
|
18
|
+
|
|
19
|
+
---
|
|
20
|
+
|
|
21
|
+
## Directory Structure
|
|
22
|
+
|
|
23
|
+
```
|
|
24
|
+
lib/
|
|
25
|
+
xlsxrb.rb # Facade: Xlsxrb.read / .write / .foreach / .generate / .build
|
|
26
|
+
xlsxrb/
|
|
27
|
+
version.rb # Xlsxrb::VERSION
|
|
28
|
+
elements.rb # Requires for Elements layer
|
|
29
|
+
ooxml.rb # Requires for Ooxml layer
|
|
30
|
+
ooxml/ # Layer 1 – Low-level OOXML
|
|
31
|
+
reader.rb # Xlsxrb::Ooxml::Reader (core reading logic)
|
|
32
|
+
writer.rb # Xlsxrb::Ooxml::Writer (core writing logic)
|
|
33
|
+
zip_generator.rb # Xlsxrb::Ooxml::ZipGenerator
|
|
34
|
+
utils.rb # Xlsxrb::Ooxml::Utils (date/time/hash helpers)
|
|
35
|
+
zip_reader.rb # Xlsxrb::Ooxml::ZipReader
|
|
36
|
+
zip_writer.rb # Xlsxrb::Ooxml::ZipWriter
|
|
37
|
+
xml_parser.rb # Xlsxrb::Ooxml::XmlParser
|
|
38
|
+
xml_builder.rb # Xlsxrb::Ooxml::XmlBuilder
|
|
39
|
+
shared_strings_parser.rb # Streaming SST reader
|
|
40
|
+
styles_parser.rb # styles.xml reader
|
|
41
|
+
worksheet_parser.rb # Streaming sheetN.xml reader
|
|
42
|
+
worksheet_writer.rb # Streaming sheetN.xml writer
|
|
43
|
+
workbook_parser.rb # workbook.xml reader
|
|
44
|
+
workbook_writer.rb # workbook.xml / styles / SST writer
|
|
45
|
+
elements/ # Layer 2 – Domain model
|
|
46
|
+
types.rb # Xlsxrb::Elements::Formula, CellError, RichText
|
|
47
|
+
cell.rb # Xlsxrb::Elements::Cell
|
|
48
|
+
row.rb # Xlsxrb::Elements::Row
|
|
49
|
+
column.rb # Xlsxrb::Elements::Column
|
|
50
|
+
worksheet.rb # Xlsxrb::Elements::Worksheet
|
|
51
|
+
workbook.rb # Xlsxrb::Elements::Workbook
|
|
52
|
+
```
|
|
53
|
+
|
|
54
|
+
---
|
|
55
|
+
|
|
56
|
+
## Core Architecture
|
|
57
|
+
|
|
58
|
+
The library is structured into three distinct layers:
|
|
59
|
+
|
|
60
|
+
### 1. Low-Level Infrastructure (The "OOXML" Layer)
|
|
61
|
+
|
|
62
|
+
**Namespace:** `Xlsxrb::Ooxml`
|
|
63
|
+
|
|
64
|
+
**Responsibility:**
|
|
65
|
+
This layer directly handles ZIP extraction, XML parsing (via SAX), and XML generation. It adheres strictly to the ECMA-376 OpenXML specification.
|
|
66
|
+
|
|
67
|
+
* **`Xlsxrb::Ooxml::ZipReader`**: Reads a `.xlsx` ZIP archive entry-by-entry. Accepts a file path or `IO` object. Yields `(entry_name, io)` pairs without loading the entire archive into memory.
|
|
68
|
+
* **`Xlsxrb::Ooxml::ZipWriter`**: Streams ZIP local-file-headers and a central directory to a file path or `IO`. Each entry is compressed with `Zlib::Deflate` in a single pass.
|
|
69
|
+
* **`Xlsxrb::Ooxml::XmlParser`**: Thin wrapper around `REXML::Parsers::SAX2Parser`. Converts SAX2 events into a Hash/Array tree. Unknown elements are collected as opaque `{ tag:, attrs:, children: }` hashes (see *unmapped_data* below).
|
|
70
|
+
* **`Xlsxrb::Ooxml::XmlBuilder`**: Emits well-formed XML strings via `<<` to a writable IO, supporting streaming generation without building a DOM.
|
|
71
|
+
* **Part-specific parsers/writers**: `WorksheetParser`, `SharedStringsParser`, `StylesParser`, `WorkbookParser`, etc., each encapsulating the SAX event handling for one OpenXML part.
|
|
72
|
+
|
|
73
|
+
### 2. High-Level Domain Model (The "Elements" Layer)
|
|
74
|
+
|
|
75
|
+
**Namespace:** `Xlsxrb::Elements`
|
|
76
|
+
|
|
77
|
+
**Responsibility:**
|
|
78
|
+
This layer provides idiomatic, easy-to-use Ruby objects representing Excel concepts. It utilizes Ruby 3.2+ `Data` classes for immutability and precise structural definition. All domain models are encapsulated here to keep the top-level namespace clean.
|
|
79
|
+
|
|
80
|
+
**Core Objects (`Data` classes):**
|
|
81
|
+
* **`Xlsxrb::Elements::Workbook`**: Represents the entire file structure. Contains `sheets` (Array of Worksheet), shared styles metadata, and `unmapped_data`.
|
|
82
|
+
* **`Xlsxrb::Elements::Worksheet`**: Represents a single sheet. Contains `name`, `rows` (Array of Row), `columns` (Array of Column), and sheet-level properties.
|
|
83
|
+
* **`Xlsxrb::Elements::Row`**: Represents one row. Contains `index` (0-based), `cells` (Array of Cell), and row-level attributes (height, hidden, etc.).
|
|
84
|
+
* **`Xlsxrb::Elements::Column`**: Represents column formatting. Contains `index` (0-based), `width`, and column-level attributes.
|
|
85
|
+
* **`Xlsxrb::Elements::Cell`**: Represents a single cell. Contains `row_index`, `column_index` (both 0-based), `value` (Ruby native type), `formula`, `style`, and `unmapped_data`.
|
|
86
|
+
|
|
87
|
+
**Design Principles:**
|
|
88
|
+
* **Zero-based Indexing:** To maintain consistency with Ruby's core language (Arrays/Enumerable), all indices (rows, columns, and worksheets) are **0-based**. For Excel-style coordination, use string references like `cell("A1")`.
|
|
89
|
+
* **Fail-safe Design (Lazy Validation):** Exceptions are not raised during XML parsing. Each class has an `errors` property (Array of String) and a `valid?` method that returns `errors.empty?`.
|
|
90
|
+
* **Forward Compatibility:** All classes have an `unmapped_data` property (Hash) to ensure that any unknown XML attributes or elements are retained, preserving file integrity during round-trips.
|
|
91
|
+
|
|
92
|
+
### 3. The Facade / Entrypoint Layer
|
|
93
|
+
|
|
94
|
+
**Namespace:** `Xlsxrb` module methods
|
|
95
|
+
|
|
96
|
+
**Responsibility:**
|
|
97
|
+
Acts as the primary bridge, offering both In-Memory and Streaming APIs.
|
|
98
|
+
|
|
99
|
+
| Method | Type | Description |
|
|
100
|
+
| :--- | :--- | :--- |
|
|
101
|
+
| **`Xlsxrb.read(source)`** | In-Memory | Loads the entire file into a `Workbook` object. |
|
|
102
|
+
| **`Xlsxrb.foreach(source, **options)`** | **Streaming** | Yields each `Row` one by one. Ideal for large files. |
|
|
103
|
+
| **`Xlsxrb.write(target, workbook)`** | In-Memory | Saves a `Workbook` object to a file or IO. |
|
|
104
|
+
| **`Xlsxrb.generate(target, &block)`** | **Streaming** | Provides a DSL to stream data directly to a file/IO. |
|
|
105
|
+
|
|
106
|
+
### Facade Expansion Policy
|
|
107
|
+
|
|
108
|
+
The long-term API goal is that **all spreadsheet features implemented in the low-level `Ooxml::Writer` layer should be available through the high-level Facade DSL** as well.
|
|
109
|
+
|
|
110
|
+
This applies to both:
|
|
111
|
+
|
|
112
|
+
* **In-Memory DSL** (`Xlsxrb.build` -> `WorkbookBuilder` / `WorksheetBuilder`)
|
|
113
|
+
* **Streaming DSL** (`Xlsxrb.generate` -> `StreamWriter`)
|
|
114
|
+
|
|
115
|
+
The Facade should not expose only a hand-picked subset forever. If a feature is stable and supported in the low-level writer, the default expectation is that it should eventually gain a high-level entry point.
|
|
116
|
+
|
|
117
|
+
### Facade DSL Conventions
|
|
118
|
+
|
|
119
|
+
When adding a new high-level feature, follow these API rules unless there is a clear technical reason not to.
|
|
120
|
+
|
|
121
|
+
#### 1. Support both a concise options form and a block form
|
|
122
|
+
|
|
123
|
+
Each DSL feature should prefer the same dual entry style now used by chart and style configuration:
|
|
124
|
+
|
|
125
|
+
* **Options form** for short, common cases
|
|
126
|
+
* **Block form** for larger or nested configuration
|
|
127
|
+
|
|
128
|
+
Examples:
|
|
129
|
+
|
|
130
|
+
```ruby
|
|
131
|
+
s.add_style("header", bold: true, size: 14, font_color: "FFFF0000")
|
|
132
|
+
|
|
133
|
+
s.add_style("header") do |style|
|
|
134
|
+
style.bold.size(14).font_color("FFFF0000")
|
|
135
|
+
end
|
|
136
|
+
|
|
137
|
+
w.add_chart(type: :bar, title: "Sales", series: [{ cat_ref: "A1:A3", val_ref: "B1:B3" }])
|
|
138
|
+
|
|
139
|
+
w.add_chart do |chart|
|
|
140
|
+
chart.type :bar
|
|
141
|
+
chart.title "Sales"
|
|
142
|
+
chart.series(cat_ref: "A1:A3", val_ref: "B1:B3")
|
|
143
|
+
end
|
|
144
|
+
```
|
|
145
|
+
|
|
146
|
+
The options form should stay compact. The block form should become the preferred place for nested or verbose configuration.
|
|
147
|
+
|
|
148
|
+
#### 2. Keep naming consistent
|
|
149
|
+
|
|
150
|
+
Use naming by intent:
|
|
151
|
+
|
|
152
|
+
* `add_*` for adding a new object or definition
|
|
153
|
+
* `set_*` for mutating a single property or replacing a single setting
|
|
154
|
+
* builder methods inside a block should use the domain name directly where possible (`title`, `series`, `bold`, `fill_color`, etc.)
|
|
155
|
+
|
|
156
|
+
Do not introduce one-off verbs for similar concepts unless the low-level feature truly behaves differently.
|
|
157
|
+
|
|
158
|
+
#### 3. Preserve scope boundaries
|
|
159
|
+
|
|
160
|
+
Each feature should appear in the builder scope that matches its OOXML ownership:
|
|
161
|
+
|
|
162
|
+
* **Workbook scope**: workbook-wide metadata, protection, named ranges, shared resources
|
|
163
|
+
* **Worksheet scope**: tables, charts, panes, filters, print settings, validations, comments, shapes
|
|
164
|
+
* **Row / Cell / Range scope**: formatting or behavior tied to a specific row, cell, or range
|
|
165
|
+
|
|
166
|
+
If a low-level feature is workbook-scoped, do not force it into a worksheet-only API just because it is convenient.
|
|
167
|
+
|
|
168
|
+
#### 4. Prefer one canonical high-level shape
|
|
169
|
+
|
|
170
|
+
For each feature, choose a single primary Facade shape and reuse it across modes:
|
|
171
|
+
|
|
172
|
+
* `Xlsxrb.build` and `Xlsxrb.generate` should feel structurally similar
|
|
173
|
+
* streaming and in-memory APIs may differ internally, but the surface API should remain as close as possible
|
|
174
|
+
* differences are acceptable only when memory or ordering constraints make them unavoidable
|
|
175
|
+
|
|
176
|
+
#### 5. Keep an escape hatch for advanced cases
|
|
177
|
+
|
|
178
|
+
The high-level DSL should cover common and intermediate use cases directly. But it does not need to mirror every obscure OOXML knob one-for-one on day one.
|
|
179
|
+
|
|
180
|
+
When a feature has a long tail of advanced attributes:
|
|
181
|
+
|
|
182
|
+
* support the common attributes first in the builder DSL
|
|
183
|
+
* allow advanced options to pass through as keyword arguments or nested hashes
|
|
184
|
+
* avoid blocking implementation progress until every low-level flag has a bespoke DSL method
|
|
185
|
+
|
|
186
|
+
#### 6. Never break existing call sites to add symmetry
|
|
187
|
+
|
|
188
|
+
High-level API growth must be backward compatible:
|
|
189
|
+
|
|
190
|
+
* adding a block form must not remove the options form
|
|
191
|
+
* adding an options form must not remove the block form
|
|
192
|
+
* existing examples and tests should continue to pass unchanged
|
|
193
|
+
|
|
194
|
+
Symmetry is valuable, but compatibility is mandatory.
|
|
195
|
+
|
|
196
|
+
### Facade Rollout Strategy
|
|
197
|
+
|
|
198
|
+
When promoting a low-level feature into the high-level DSL, use this order of operations:
|
|
199
|
+
|
|
200
|
+
1. Identify the owning scope (`WorkbookBuilder`, `WorksheetBuilder`, `StreamWriter`, or a nested builder)
|
|
201
|
+
2. Add the options form
|
|
202
|
+
3. Add the block form if the shape becomes nested or verbose
|
|
203
|
+
4. Ensure the streaming and in-memory APIs expose the same concept with the same names whenever possible
|
|
204
|
+
5. Document the shortest example and the richer builder example together
|
|
205
|
+
|
|
206
|
+
This keeps the public API coherent as coverage grows.
|
|
207
|
+
|
|
208
|
+
### High-Priority Features For Facade Promotion
|
|
209
|
+
|
|
210
|
+
The following low-level features are especially strong candidates for high-level exposure because they are common, composable, and fit the chart/style pattern well:
|
|
211
|
+
|
|
212
|
+
* hyperlinks
|
|
213
|
+
* auto filters and sort state
|
|
214
|
+
* data validation
|
|
215
|
+
* conditional formatting
|
|
216
|
+
* tables
|
|
217
|
+
* comments
|
|
218
|
+
* freeze/split panes and selection
|
|
219
|
+
* page setup, margins, header/footer, print options
|
|
220
|
+
* workbook and sheet protection
|
|
221
|
+
* defined names, print area, print titles
|
|
222
|
+
* shapes and images
|
|
223
|
+
* pivot tables
|
|
224
|
+
* document properties
|
|
225
|
+
|
|
226
|
+
These should be treated as backlog for Facade parity, not as permanently low-level-only features.
|
|
227
|
+
|
|
228
|
+
---
|
|
229
|
+
|
|
230
|
+
## Data-Flow & Lifecycle
|
|
231
|
+
|
|
232
|
+
### `Xlsxrb.read(source)` — In-Memory Read
|
|
233
|
+
|
|
234
|
+
```
|
|
235
|
+
source (path / IO)
|
|
236
|
+
│
|
|
237
|
+
▼
|
|
238
|
+
Ooxml::ZipReader ── extracts ZIP entries ──► raw bytes per part
|
|
239
|
+
│
|
|
240
|
+
▼
|
|
241
|
+
Ooxml::SharedStringsParser ── SAX parse xl/sharedStrings.xml ──► string table (Array)
|
|
242
|
+
Ooxml::StylesParser ── SAX parse xl/styles.xml ──► styles hash
|
|
243
|
+
Ooxml::WorkbookParser ── SAX parse xl/workbook.xml ──► sheet list
|
|
244
|
+
│
|
|
245
|
+
▼ (for each sheet)
|
|
246
|
+
Ooxml::WorksheetParser ── SAX parse xl/worksheets/sheetN.xml ──►
|
|
247
|
+
│ yields (row_index, cells_array, row_attrs, unmapped)
|
|
248
|
+
▼
|
|
249
|
+
Elements::Cell / Row / Column / Worksheet
|
|
250
|
+
│
|
|
251
|
+
▼
|
|
252
|
+
Elements::Workbook ◄── assembled from all worksheets
|
|
253
|
+
```
|
|
254
|
+
|
|
255
|
+
### `Xlsxrb.write(target, workbook)` — In-Memory Write
|
|
256
|
+
|
|
257
|
+
```
|
|
258
|
+
Elements::Workbook
|
|
259
|
+
│
|
|
260
|
+
▼ (for each worksheet)
|
|
261
|
+
Ooxml::WorksheetWriter ── converts Row/Cell → XML fragments ──►
|
|
262
|
+
│ streams into Ooxml::ZipWriter entry
|
|
263
|
+
▼
|
|
264
|
+
Ooxml::WorkbookWriter ── writes workbook.xml, styles.xml, sharedStrings.xml,
|
|
265
|
+
│ [Content_Types].xml, .rels
|
|
266
|
+
▼
|
|
267
|
+
Ooxml::ZipWriter ── writes ZIP output ──► target (path / IO)
|
|
268
|
+
```
|
|
269
|
+
|
|
270
|
+
### `Xlsxrb.foreach(source, **options)` — Streaming Read
|
|
271
|
+
|
|
272
|
+
```
|
|
273
|
+
source (path / IO)
|
|
274
|
+
│
|
|
275
|
+
▼
|
|
276
|
+
Ooxml::ZipReader ── locates xl/sharedStrings.xml, xl/worksheets/sheetN.xml
|
|
277
|
+
│
|
|
278
|
+
▼ (SAX parse SST first — kept in memory as a flat Array of strings)
|
|
279
|
+
│
|
|
280
|
+
▼ (then SAX stream worksheet)
|
|
281
|
+
Ooxml::WorksheetParser ── on each </row> event:
|
|
282
|
+
│ 1. build Elements::Row with resolved cell values
|
|
283
|
+
│ 2. yield Row to caller's block
|
|
284
|
+
│ 3. discard Row (GC eligible)
|
|
285
|
+
▼
|
|
286
|
+
caller's block receives Elements::Row, processes, moves on
|
|
287
|
+
```
|
|
288
|
+
|
|
289
|
+
Key memory invariant: only **one Row** (plus the shared-string table) is alive at any time.
|
|
290
|
+
|
|
291
|
+
### `Xlsxrb.generate(target, &block)` — Streaming Write
|
|
292
|
+
|
|
293
|
+
```
|
|
294
|
+
caller's block
|
|
295
|
+
│
|
|
296
|
+
▼ block receives a StreamWriter context object
|
|
297
|
+
│ context.add_row([val1, val2, ...])
|
|
298
|
+
│
|
|
299
|
+
▼
|
|
300
|
+
Ooxml::WorksheetWriter ── converts array → <row><c>…</c></row> XML
|
|
301
|
+
│ writes directly to ZipWriter entry stream
|
|
302
|
+
▼
|
|
303
|
+
Ooxml::ZipWriter ── compresses & writes to target
|
|
304
|
+
```
|
|
305
|
+
|
|
306
|
+
Key memory invariant: rows are written and flushed immediately; no row Array accumulates.
|
|
307
|
+
|
|
308
|
+
---
|
|
309
|
+
|
|
310
|
+
## Streaming Internals
|
|
311
|
+
|
|
312
|
+
### Unified Event-Based Streaming (`Ooxml::Event`)
|
|
313
|
+
|
|
314
|
+
To unify parsing across both streaming and in-memory paths, the OOXML layer utilizes a unified event-based parsing model. Individual parsers (such as `WorksheetParser` and `SharedStringsParser`) implement a streaming `each_event` method that emits a sequence of `Xlsxrb::Ooxml::Event` objects.
|
|
315
|
+
|
|
316
|
+
An event contains:
|
|
317
|
+
- `type`: a Symbol representing the event type (e.g., `:row_start`, `:cell`, `:row_end`, `:column`, `:hyperlink`, `:sst_item`).
|
|
318
|
+
- `args`: an Array containing the event data arguments.
|
|
319
|
+
- `source`: a Hash providing context for error reporting (e.g., `{ part: "xl/worksheets/sheet1.xml", row: 0, cell: "A1" }`).
|
|
320
|
+
|
|
321
|
+
#### Event Vocabulary:
|
|
322
|
+
1. **Worksheet Events:**
|
|
323
|
+
- `:row_start` - `args: [row_index, attrs]`
|
|
324
|
+
- `:cell` - `args: [ref, type, style_index, value, formula]`
|
|
325
|
+
- `:row_end` - `args: []`
|
|
326
|
+
- `:column` - `args: [min, max, width, hidden, custom_width, outline_level]`
|
|
327
|
+
- `:hyperlink` - `args: [ref, rid, display, tooltip, location]`
|
|
328
|
+
2. **Shared Strings (SST) Events:**
|
|
329
|
+
- `:sst_item` - `args: [string_value]`
|
|
330
|
+
|
|
331
|
+
The streaming parser `each_row` (or `parse`) consumes the event stream and folds it into raw row hashes, maintaining a minimal state machine and constant memory footprint.
|
|
332
|
+
|
|
333
|
+
### ZIP Streaming
|
|
334
|
+
|
|
335
|
+
`Ooxml::ZipReader` scans local file headers sequentially using `Zlib::Inflate`. It does **not** seek to the central directory — this allows reading from non-seekable IO (pipes, HTTP streams).
|
|
336
|
+
|
|
337
|
+
`Ooxml::ZipWriter` writes local file headers immediately, accumulates a central directory index in memory (entry names + offsets only), and writes the central directory + EOCD at `#close`.
|
|
338
|
+
|
|
339
|
+
---
|
|
340
|
+
|
|
341
|
+
## `unmapped_data` & Forward-Compatibility
|
|
342
|
+
|
|
343
|
+
When the Ooxml layer encounters an XML element or attribute not in its recognized set:
|
|
344
|
+
|
|
345
|
+
1. **Capture**: The element is stored as a Hash `{ tag: String, attrs: Hash, children: Array, text: String? }`.
|
|
346
|
+
2. **Attach**: The Hash is pushed onto the nearest recognized parent's `unmapped_children` array.
|
|
347
|
+
3. **Surface**: The Elements layer receives these as the `unmapped_data` field — a Hash keyed by parent-context (e.g. `{ row: [...], cell: [...], worksheet: [...] }`).
|
|
348
|
+
4. **Restore**: During write-back (`Ooxml::WorksheetWriter`), `unmapped_data` entries are re-serialized to XML in their original order using `XmlBuilder`, preserving any future spec extensions or vendor-specific markup.
|
|
349
|
+
|
|
350
|
+
This ensures that reading then writing an XLSX file does not silently discard unknown content.
|
|
351
|
+
|
|
352
|
+
---
|
|
353
|
+
|
|
354
|
+
## Error Handling & Validation Boundaries
|
|
355
|
+
|
|
356
|
+
### Ooxml Layer (parse-time)
|
|
357
|
+
|
|
358
|
+
* **Never raises** on unexpected XML content. Unrecognized elements → `unmapped_data`. Malformed attribute values → stored as-is (raw strings).
|
|
359
|
+
* **Raises** only on structural corruption that prevents further parsing (e.g., truncated ZIP, invalid UTF-8, ZIP local header CRC mismatch).
|
|
360
|
+
|
|
361
|
+
### Elements Layer (model-time)
|
|
362
|
+
|
|
363
|
+
* Each `Data` class exposes `errors` (frozen Array of String) and `valid?` (`errors.empty?`).
|
|
364
|
+
* Validation is performed at construction time:
|
|
365
|
+
- `Cell`: value type check, column/row index range
|
|
366
|
+
- `Row`: index ≥ 0, cells array consistency
|
|
367
|
+
- `Worksheet`: name present, unique row indices
|
|
368
|
+
- `Workbook`: at least one sheet, unique sheet names
|
|
369
|
+
* Invalid objects **are still created** — the caller decides how to handle `valid? == false`.
|
|
370
|
+
|
|
371
|
+
### Facade Layer
|
|
372
|
+
|
|
373
|
+
* `Xlsxrb.read` / `.foreach`: propagate Ooxml-layer structural exceptions. Content-level issues appear in `errors` on returned objects.
|
|
374
|
+
* `Xlsxrb.write` / `.generate`: validate the `Workbook` / row data at the boundary and raise `Xlsxrb::Error` for fatal issues (e.g., nil target path). Non-fatal issues (e.g., value truncation) are silently handled.
|
|
375
|
+
|
|
376
|
+
---
|
|
377
|
+
|
|
378
|
+
## Benefits of this Approach
|
|
379
|
+
|
|
380
|
+
* **Rubyish Interface:** Methods like `foreach` and `generate` follow Ruby's standard library conventions (e.g., `CSV.foreach`).
|
|
381
|
+
* **Clean Namespace:** Users only interact with the `Xlsxrb` module. Internal models are safely isolated within `Elements`.
|
|
382
|
+
* **Safety & LSP Support:** `Data` objects provide clear property definitions for editor autocomplete.
|
|
383
|
+
* **Constant Memory Streaming:** Both read and write paths support row-at-a-time processing suitable for millions of rows.
|
|
384
|
+
* **Future-Proofing:** The `unmapped_data` mechanism and layered design accommodate future features without rewriting the underlying XML logic.
|
|
385
|
+
|
|
386
|
+
---
|
|
387
|
+
|
|
388
|
+
## Facade Quality Gates
|
|
389
|
+
|
|
390
|
+
Every new high-level DSL feature must satisfy the following quality rules before it is considered complete.
|
|
391
|
+
|
|
392
|
+
### 1. Both API paths must be covered
|
|
393
|
+
|
|
394
|
+
If a feature is intended to exist in both writing modes, tests must cover:
|
|
395
|
+
|
|
396
|
+
* `Xlsxrb.build` / `Xlsxrb.write`
|
|
397
|
+
* `Xlsxrb.generate`
|
|
398
|
+
|
|
399
|
+
If a feature can only exist in one mode for a technical reason, that restriction must be documented explicitly in code comments and user-facing docs.
|
|
400
|
+
|
|
401
|
+
### 2. Both entry forms must be covered when both are supported
|
|
402
|
+
|
|
403
|
+
If a feature exposes both:
|
|
404
|
+
|
|
405
|
+
* an options form
|
|
406
|
+
* a block form
|
|
407
|
+
|
|
408
|
+
then Facade tests should exercise both forms at least once.
|
|
409
|
+
|
|
410
|
+
### 3. Facade tests are mandatory
|
|
411
|
+
|
|
412
|
+
Add or extend tests in `test/facade_test.rb` so the feature is validated at the public API level.
|
|
413
|
+
|
|
414
|
+
These tests should verify:
|
|
415
|
+
|
|
416
|
+
* the feature can be declared through the high-level DSL
|
|
417
|
+
* the generated file can be read back
|
|
418
|
+
* the semantic result is present in the parsed workbook or reader output
|
|
419
|
+
|
|
420
|
+
### 4. Contract tests are preferred when structure parity matters
|
|
421
|
+
|
|
422
|
+
If the feature should produce equivalent OOXML across streaming and in-memory modes, add or extend `test/contract_test.rb`.
|
|
423
|
+
|
|
424
|
+
This is especially important for:
|
|
425
|
+
|
|
426
|
+
* shared workbook/worksheet structures
|
|
427
|
+
* range-based features
|
|
428
|
+
* settings that should serialize identically regardless of API path
|
|
429
|
+
|
|
430
|
+
### 5. E2E coverage is required for new structural output
|
|
431
|
+
|
|
432
|
+
If a feature introduces new XML elements, attributes, relationships, or package parts, add an interoperability or E2E test.
|
|
433
|
+
|
|
434
|
+
At minimum, verify one of:
|
|
435
|
+
|
|
436
|
+
* Open XML SDK validation passes
|
|
437
|
+
* the generated XML parts contain the expected structure and are accepted by the reader
|
|
438
|
+
|
|
439
|
+
### 6. Documentation must ship with the feature
|
|
440
|
+
|
|
441
|
+
Every new high-level feature should update user-facing docs with:
|
|
442
|
+
|
|
443
|
+
* one short example
|
|
444
|
+
* one richer example if the feature has a block form or nested configuration
|
|
445
|
+
* any important streaming vs in-memory limitation
|
|
446
|
+
|
|
447
|
+
### 7. Keep surface area smaller than implementation detail
|
|
448
|
+
|
|
449
|
+
Do not promote every low-level flag into a top-level public method immediately.
|
|
450
|
+
|
|
451
|
+
Prefer this order:
|
|
452
|
+
|
|
453
|
+
1. common user-facing options
|
|
454
|
+
2. nested builder methods for grouped concepts
|
|
455
|
+
3. advanced keyword passthrough for rare flags
|
|
456
|
+
|
|
457
|
+
This keeps the DSL readable while still allowing high feature coverage.
|
|
458
|
+
|
|
459
|
+
### 8. Backward compatibility is a release gate
|
|
460
|
+
|
|
461
|
+
A feature is not complete if it improves symmetry but breaks older call sites, examples, or tests.
|
|
462
|
+
|
|
463
|
+
Backward compatibility must be verified before merging any Facade DSL expansion.
|
|
464
|
+
|
|
465
|
+
---
|
|
466
|
+
|
|
467
|
+
## Testing Strategy
|
|
468
|
+
|
|
469
|
+
To ensure library robustness and consistency across execution paths, we organize tests into four distinct layers:
|
|
470
|
+
|
|
471
|
+
1. **Unit Tests (`test/xlsxrb/`):**
|
|
472
|
+
- Focus on isolated components (such as parsers and writers) without external system dependencies.
|
|
473
|
+
- Includes **Round-trip testing** to verify that generated XML can be successfully parsed back by the reader.
|
|
474
|
+
- Run via: `bundle exec rake test:unit`
|
|
475
|
+
|
|
476
|
+
2. **Contract Tests (`test/contract/`):**
|
|
477
|
+
- Ensures semantic parity between the Streaming and In-Memory API paths.
|
|
478
|
+
- Operates by executing identical data scenarios on both APIs and asserting that they serialize to equivalent structures.
|
|
479
|
+
- Run via: `bundle exec rake test:contract`
|
|
480
|
+
|
|
481
|
+
3. **Interop (E2E) Tests (`test/e2e/`):**
|
|
482
|
+
- Exercises real-world interoperability by validating generated files using the official .NET-based **Open XML SDK** validator, and reading spreadsheets dynamically created by the SDK.
|
|
483
|
+
- Run via: `bundle exec rake test:e2e`
|
|
484
|
+
|
|
485
|
+
4. **Visual Examples & VRT (`test/visual/`):**
|
|
486
|
+
- **Living Documentation:** Compiles visual DSL scripts under `examples/visual/` into the [Visual Examples Gallery](visual/README.md).
|
|
487
|
+
- **Visual Regression Testing:** Renders the generated spreadsheets into PNG files using headless LibreOffice Calc, and calculates pixel differences against reference baselines using ImageMagick.
|
|
488
|
+
- Run via: `bundle exec rake test:visual`
|
|
@@ -0,0 +1,42 @@
|
|
|
1
|
+
# Specification Reference Policy
|
|
2
|
+
|
|
3
|
+
This document defines the specification reference policy for `xlsxrb`.
|
|
4
|
+
|
|
5
|
+
## Canonical Specification (Normative)
|
|
6
|
+
The primary reference for this library is **ECMA-376 (Office Open XML File Formats)**.
|
|
7
|
+
Specifically, `xlsxrb` targets the **Transitional** version of the specification (Part 4) to ensure maximum compatibility with existing applications such as Microsoft Excel, LibreOffice, and Google Sheets.
|
|
8
|
+
|
|
9
|
+
Local copies of the ECMA-376 specifications are located in the `vendor/docs/` directory:
|
|
10
|
+
- [Part 1: Fundamentals And Markup Language Reference](file:///workspaces/xlsxrb/vendor/docs/ECMA-376-Part1/Ecma%20Office%20Open%20XML%20Part%201%20-%20Fundamentals%20And%20Markup%20Language%20Reference.pdf)
|
|
11
|
+
- [Part 2: Open Packaging Conventions](file:///workspaces/xlsxrb/vendor/docs/ECMA-376-Part2/ECMA-376-2_5th_edition_december_2021.pdf)
|
|
12
|
+
- [Part 3: Markup Compatibility and Extensibility](file:///workspaces/xlsxrb/vendor/docs/ECMA-376-Part3/Ecma%20Office%20Open%20XML%20Part%203%20-%20Markup%20Compatibility%20and%20Extensibility.pdf)
|
|
13
|
+
- [Part 4: Transitional Migration Features](file:///workspaces/xlsxrb/vendor/docs/ECMA-376-Part4/Ecma%20Office%20Open%20XML%20Part%204%20-%20Transitional%20Migration%20Features.pdf)
|
|
14
|
+
|
|
15
|
+
## Supplementary Specifications (Excel Real-world Behavior)
|
|
16
|
+
To address gaps between the official ECMA standard and actual implementations in Microsoft Excel, the following Microsoft Open Specifications are used. Note that these files are not bundled in the repository to avoid licensing/redistribution issues; instead, they are referenced via online links and versioned here:
|
|
17
|
+
|
|
18
|
+
1. **[[MS-XLSX]: Excel Extensions to OOXML SpreadsheetML Structure](https://learn.microsoft.com/en-us/openspecs/office_standards/ms-xlsx/)**
|
|
19
|
+
- **Role:** Explains Excel-specific extensions, default attributes, and schema extensions.
|
|
20
|
+
- **Referenced Version:** July 2024 / Version 12.0 (or current release).
|
|
21
|
+
2. **[[MS-OI29500]: Office Implementation Information for ISO/IEC 29500](https://learn.microsoft.com/en-us/openspecs/office_standards/ms-oi29500/)**
|
|
22
|
+
- **Role:** Identifies how Excel actually reads/writes files, including deviations and compatibility behaviors.
|
|
23
|
+
- **Referenced Version:** July 2024 / Version 12.0 (or current release).
|
|
24
|
+
3. **[[MS-OFFCRYPTO]: Office Document Cryptography Structure](https://learn.microsoft.com/en-us/openspecs/office_standards/ms-offcrypto/)**
|
|
25
|
+
- **Role:** Provides details on encryption, passwords, and hashing algorithms used for document/sheet protection.
|
|
26
|
+
- **Referenced Version:** July 2024 / Version 12.0 (or current release).
|
|
27
|
+
|
|
28
|
+
### ISO/IEC 29500 Note
|
|
29
|
+
ISO/IEC 29500 is contents-wise equivalent to ECMA-376. However, ISO/IEC 29500 requires paid purchase in general, whereas ECMA-376 is freely available. Therefore, we primarily cite ECMA-376 sections.
|
|
30
|
+
|
|
31
|
+
## Source Attribution Policy
|
|
32
|
+
When implementing features or fixing bugs that depend on specific behaviors defined in these specifications, developers should:
|
|
33
|
+
1. Reference the specific part, section, or page number in code comments (e.g., `# Reference: ECMA-376 Part 1, Section 18.3.1.73`).
|
|
34
|
+
2. Update the mapping table below to maintain a central registry of implemented standard behaviors.
|
|
35
|
+
|
|
36
|
+
## Implementation Specification Mapping Table
|
|
37
|
+
| Feature | Primary Standard | Specification Section | Notes |
|
|
38
|
+
| :--- | :--- | :--- | :--- |
|
|
39
|
+
| Cell Value Types | ECMA-376 Part 1 | §18.18.11 (ST_CellType) | Defines cell data types (boolean, number, string, formula, etc.) |
|
|
40
|
+
| Shared Strings | ECMA-376 Part 1 | §18.4 (Shared String Table) | Handling of `<sst>` and `<si>` for cell value reuse |
|
|
41
|
+
| Styles & Formatting | ECMA-376 Part 1 | §18.8 (Styles) | Cell style XF indexes, font, fill, border mappings |
|
|
42
|
+
| Hyperlinks | ECMA-376 Part 1 | §18.3.1.48 (hyperlink) | Worksheet hyperlinks referencing external URLs or internal targets |
|