xlsxrb 0.1.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (44) hide show
  1. checksums.yaml +7 -0
  2. data/.devcontainer/Dockerfile +64 -0
  3. data/.devcontainer/devcontainer.json +17 -0
  4. data/CHANGELOG.md +12 -0
  5. data/CODE_OF_CONDUCT.md +10 -0
  6. data/LICENSE.txt +21 -0
  7. data/README.md +384 -0
  8. data/Rakefile +290 -0
  9. data/benchmark.rb +390 -0
  10. data/docs/ARCHITECTURE.md +488 -0
  11. data/docs/SPEC_SOURCES.md +42 -0
  12. data/docs/visual/VisualGallery.md +4684 -0
  13. data/docs/wasm/wasm_doc_helper.css +189 -0
  14. data/docs/wasm/wasm_doc_helper.js +320 -0
  15. data/lib/xlsxrb/elements/cell.rb +77 -0
  16. data/lib/xlsxrb/elements/column.rb +28 -0
  17. data/lib/xlsxrb/elements/row.rb +47 -0
  18. data/lib/xlsxrb/elements/types.rb +36 -0
  19. data/lib/xlsxrb/elements/workbook.rb +47 -0
  20. data/lib/xlsxrb/elements/worksheet.rb +50 -0
  21. data/lib/xlsxrb/elements.rb +15 -0
  22. data/lib/xlsxrb/ooxml/reader.rb +8048 -0
  23. data/lib/xlsxrb/ooxml/shared_strings_parser.rb +75 -0
  24. data/lib/xlsxrb/ooxml/styles_parser.rb +194 -0
  25. data/lib/xlsxrb/ooxml/utils.rb +122 -0
  26. data/lib/xlsxrb/ooxml/workbook_parser.rb +78 -0
  27. data/lib/xlsxrb/ooxml/workbook_writer.rb +1000 -0
  28. data/lib/xlsxrb/ooxml/worksheet_parser.rb +455 -0
  29. data/lib/xlsxrb/ooxml/worksheet_writer.rb +1008 -0
  30. data/lib/xlsxrb/ooxml/writer.rb +5450 -0
  31. data/lib/xlsxrb/ooxml/xml_builder.rb +113 -0
  32. data/lib/xlsxrb/ooxml/xml_parser.rb +68 -0
  33. data/lib/xlsxrb/ooxml/zip_generator.rb +162 -0
  34. data/lib/xlsxrb/ooxml/zip_reader.rb +158 -0
  35. data/lib/xlsxrb/ooxml/zip_writer.rb +213 -0
  36. data/lib/xlsxrb/ooxml.rb +22 -0
  37. data/lib/xlsxrb/style_builder.rb +241 -0
  38. data/lib/xlsxrb/version.rb +5 -0
  39. data/lib/xlsxrb.rb +1427 -0
  40. data/measure_memory.rb +42 -0
  41. data/sig/xlsxrb.rbs +23 -0
  42. data/vendor/sdk_runner/Program.cs +91 -0
  43. data/vendor/sdk_runner/sdk_runner.csproj +13 -0
  44. metadata +144 -0
@@ -0,0 +1,488 @@
1
+ # Xlsxrb Architecture
2
+
3
+ Xlsxrb uses a multi-layered architecture to separate low-level OpenXML specification details from a high-level, idiomatic Ruby API. This separation of concerns ensures the library is both robust against the complex OpenXML spec and user-friendly for Ruby developers.
4
+
5
+ ## Dependency Constraints
6
+
7
+ Xlsxrb uses **only** the Ruby standard library and Bundled Gems:
8
+
9
+ | Library | Purpose |
10
+ | :--- | :--- |
11
+ | `rexml` (bundled gem) | SAX2-based XML parsing and DOM-based XML generation |
12
+ | `zlib` (stdlib) | ZIP deflate/inflate compression |
13
+ | `stringio` (stdlib) | In-memory IO for streaming |
14
+ | `date` (stdlib) | Excel serial-number ↔ `Date` / `Time` conversion |
15
+ | `openssl`, `securerandom` (stdlib) | Password hashing for sheet/workbook protection |
16
+
17
+ **No third-party gems** (e.g. Nokogiri, rubyzip) are permitted as runtime dependencies.
18
+
19
+ ---
20
+
21
+ ## Directory Structure
22
+
23
+ ```
24
+ lib/
25
+ xlsxrb.rb # Facade: Xlsxrb.read / .write / .foreach / .generate / .build
26
+ xlsxrb/
27
+ version.rb # Xlsxrb::VERSION
28
+ elements.rb # Requires for Elements layer
29
+ ooxml.rb # Requires for Ooxml layer
30
+ ooxml/ # Layer 1 – Low-level OOXML
31
+ reader.rb # Xlsxrb::Ooxml::Reader (core reading logic)
32
+ writer.rb # Xlsxrb::Ooxml::Writer (core writing logic)
33
+ zip_generator.rb # Xlsxrb::Ooxml::ZipGenerator
34
+ utils.rb # Xlsxrb::Ooxml::Utils (date/time/hash helpers)
35
+ zip_reader.rb # Xlsxrb::Ooxml::ZipReader
36
+ zip_writer.rb # Xlsxrb::Ooxml::ZipWriter
37
+ xml_parser.rb # Xlsxrb::Ooxml::XmlParser
38
+ xml_builder.rb # Xlsxrb::Ooxml::XmlBuilder
39
+ shared_strings_parser.rb # Streaming SST reader
40
+ styles_parser.rb # styles.xml reader
41
+ worksheet_parser.rb # Streaming sheetN.xml reader
42
+ worksheet_writer.rb # Streaming sheetN.xml writer
43
+ workbook_parser.rb # workbook.xml reader
44
+ workbook_writer.rb # workbook.xml / styles / SST writer
45
+ elements/ # Layer 2 – Domain model
46
+ types.rb # Xlsxrb::Elements::Formula, CellError, RichText
47
+ cell.rb # Xlsxrb::Elements::Cell
48
+ row.rb # Xlsxrb::Elements::Row
49
+ column.rb # Xlsxrb::Elements::Column
50
+ worksheet.rb # Xlsxrb::Elements::Worksheet
51
+ workbook.rb # Xlsxrb::Elements::Workbook
52
+ ```
53
+
54
+ ---
55
+
56
+ ## Core Architecture
57
+
58
+ The library is structured into three distinct layers:
59
+
60
+ ### 1. Low-Level Infrastructure (The "OOXML" Layer)
61
+
62
+ **Namespace:** `Xlsxrb::Ooxml`
63
+
64
+ **Responsibility:**
65
+ This layer directly handles ZIP extraction, XML parsing (via SAX), and XML generation. It adheres strictly to the ECMA-376 OpenXML specification.
66
+
67
+ * **`Xlsxrb::Ooxml::ZipReader`**: Reads a `.xlsx` ZIP archive entry-by-entry. Accepts a file path or `IO` object. Yields `(entry_name, io)` pairs without loading the entire archive into memory.
68
+ * **`Xlsxrb::Ooxml::ZipWriter`**: Streams ZIP local-file-headers and a central directory to a file path or `IO`. Each entry is compressed with `Zlib::Deflate` in a single pass.
69
+ * **`Xlsxrb::Ooxml::XmlParser`**: Thin wrapper around `REXML::Parsers::SAX2Parser`. Converts SAX2 events into a Hash/Array tree. Unknown elements are collected as opaque `{ tag:, attrs:, children: }` hashes (see *unmapped_data* below).
70
+ * **`Xlsxrb::Ooxml::XmlBuilder`**: Emits well-formed XML strings via `<<` to a writable IO, supporting streaming generation without building a DOM.
71
+ * **Part-specific parsers/writers**: `WorksheetParser`, `SharedStringsParser`, `StylesParser`, `WorkbookParser`, etc., each encapsulating the SAX event handling for one OpenXML part.
72
+
73
+ ### 2. High-Level Domain Model (The "Elements" Layer)
74
+
75
+ **Namespace:** `Xlsxrb::Elements`
76
+
77
+ **Responsibility:**
78
+ This layer provides idiomatic, easy-to-use Ruby objects representing Excel concepts. It utilizes Ruby 3.2+ `Data` classes for immutability and precise structural definition. All domain models are encapsulated here to keep the top-level namespace clean.
79
+
80
+ **Core Objects (`Data` classes):**
81
+ * **`Xlsxrb::Elements::Workbook`**: Represents the entire file structure. Contains `sheets` (Array of Worksheet), shared styles metadata, and `unmapped_data`.
82
+ * **`Xlsxrb::Elements::Worksheet`**: Represents a single sheet. Contains `name`, `rows` (Array of Row), `columns` (Array of Column), and sheet-level properties.
83
+ * **`Xlsxrb::Elements::Row`**: Represents one row. Contains `index` (0-based), `cells` (Array of Cell), and row-level attributes (height, hidden, etc.).
84
+ * **`Xlsxrb::Elements::Column`**: Represents column formatting. Contains `index` (0-based), `width`, and column-level attributes.
85
+ * **`Xlsxrb::Elements::Cell`**: Represents a single cell. Contains `row_index`, `column_index` (both 0-based), `value` (Ruby native type), `formula`, `style`, and `unmapped_data`.
86
+
87
+ **Design Principles:**
88
+ * **Zero-based Indexing:** To maintain consistency with Ruby's core language (Arrays/Enumerable), all indices (rows, columns, and worksheets) are **0-based**. For Excel-style coordination, use string references like `cell("A1")`.
89
+ * **Fail-safe Design (Lazy Validation):** Exceptions are not raised during XML parsing. Each class has an `errors` property (Array of String) and a `valid?` method that returns `errors.empty?`.
90
+ * **Forward Compatibility:** All classes have an `unmapped_data` property (Hash) to ensure that any unknown XML attributes or elements are retained, preserving file integrity during round-trips.
91
+
92
+ ### 3. The Facade / Entrypoint Layer
93
+
94
+ **Namespace:** `Xlsxrb` module methods
95
+
96
+ **Responsibility:**
97
+ Acts as the primary bridge, offering both In-Memory and Streaming APIs.
98
+
99
+ | Method | Type | Description |
100
+ | :--- | :--- | :--- |
101
+ | **`Xlsxrb.read(source)`** | In-Memory | Loads the entire file into a `Workbook` object. |
102
+ | **`Xlsxrb.foreach(source, **options)`** | **Streaming** | Yields each `Row` one by one. Ideal for large files. |
103
+ | **`Xlsxrb.write(target, workbook)`** | In-Memory | Saves a `Workbook` object to a file or IO. |
104
+ | **`Xlsxrb.generate(target, &block)`** | **Streaming** | Provides a DSL to stream data directly to a file/IO. |
105
+
106
+ ### Facade Expansion Policy
107
+
108
+ The long-term API goal is that **all spreadsheet features implemented in the low-level `Ooxml::Writer` layer should be available through the high-level Facade DSL** as well.
109
+
110
+ This applies to both:
111
+
112
+ * **In-Memory DSL** (`Xlsxrb.build` -> `WorkbookBuilder` / `WorksheetBuilder`)
113
+ * **Streaming DSL** (`Xlsxrb.generate` -> `StreamWriter`)
114
+
115
+ The Facade should not expose only a hand-picked subset forever. If a feature is stable and supported in the low-level writer, the default expectation is that it should eventually gain a high-level entry point.
116
+
117
+ ### Facade DSL Conventions
118
+
119
+ When adding a new high-level feature, follow these API rules unless there is a clear technical reason not to.
120
+
121
+ #### 1. Support both a concise options form and a block form
122
+
123
+ Each DSL feature should prefer the same dual entry style now used by chart and style configuration:
124
+
125
+ * **Options form** for short, common cases
126
+ * **Block form** for larger or nested configuration
127
+
128
+ Examples:
129
+
130
+ ```ruby
131
+ s.add_style("header", bold: true, size: 14, font_color: "FFFF0000")
132
+
133
+ s.add_style("header") do |style|
134
+ style.bold.size(14).font_color("FFFF0000")
135
+ end
136
+
137
+ w.add_chart(type: :bar, title: "Sales", series: [{ cat_ref: "A1:A3", val_ref: "B1:B3" }])
138
+
139
+ w.add_chart do |chart|
140
+ chart.type :bar
141
+ chart.title "Sales"
142
+ chart.series(cat_ref: "A1:A3", val_ref: "B1:B3")
143
+ end
144
+ ```
145
+
146
+ The options form should stay compact. The block form should become the preferred place for nested or verbose configuration.
147
+
148
+ #### 2. Keep naming consistent
149
+
150
+ Use naming by intent:
151
+
152
+ * `add_*` for adding a new object or definition
153
+ * `set_*` for mutating a single property or replacing a single setting
154
+ * builder methods inside a block should use the domain name directly where possible (`title`, `series`, `bold`, `fill_color`, etc.)
155
+
156
+ Do not introduce one-off verbs for similar concepts unless the low-level feature truly behaves differently.
157
+
158
+ #### 3. Preserve scope boundaries
159
+
160
+ Each feature should appear in the builder scope that matches its OOXML ownership:
161
+
162
+ * **Workbook scope**: workbook-wide metadata, protection, named ranges, shared resources
163
+ * **Worksheet scope**: tables, charts, panes, filters, print settings, validations, comments, shapes
164
+ * **Row / Cell / Range scope**: formatting or behavior tied to a specific row, cell, or range
165
+
166
+ If a low-level feature is workbook-scoped, do not force it into a worksheet-only API just because it is convenient.
167
+
168
+ #### 4. Prefer one canonical high-level shape
169
+
170
+ For each feature, choose a single primary Facade shape and reuse it across modes:
171
+
172
+ * `Xlsxrb.build` and `Xlsxrb.generate` should feel structurally similar
173
+ * streaming and in-memory APIs may differ internally, but the surface API should remain as close as possible
174
+ * differences are acceptable only when memory or ordering constraints make them unavoidable
175
+
176
+ #### 5. Keep an escape hatch for advanced cases
177
+
178
+ The high-level DSL should cover common and intermediate use cases directly. But it does not need to mirror every obscure OOXML knob one-for-one on day one.
179
+
180
+ When a feature has a long tail of advanced attributes:
181
+
182
+ * support the common attributes first in the builder DSL
183
+ * allow advanced options to pass through as keyword arguments or nested hashes
184
+ * avoid blocking implementation progress until every low-level flag has a bespoke DSL method
185
+
186
+ #### 6. Never break existing call sites to add symmetry
187
+
188
+ High-level API growth must be backward compatible:
189
+
190
+ * adding a block form must not remove the options form
191
+ * adding an options form must not remove the block form
192
+ * existing examples and tests should continue to pass unchanged
193
+
194
+ Symmetry is valuable, but compatibility is mandatory.
195
+
196
+ ### Facade Rollout Strategy
197
+
198
+ When promoting a low-level feature into the high-level DSL, use this order of operations:
199
+
200
+ 1. Identify the owning scope (`WorkbookBuilder`, `WorksheetBuilder`, `StreamWriter`, or a nested builder)
201
+ 2. Add the options form
202
+ 3. Add the block form if the shape becomes nested or verbose
203
+ 4. Ensure the streaming and in-memory APIs expose the same concept with the same names whenever possible
204
+ 5. Document the shortest example and the richer builder example together
205
+
206
+ This keeps the public API coherent as coverage grows.
207
+
208
+ ### High-Priority Features For Facade Promotion
209
+
210
+ The following low-level features are especially strong candidates for high-level exposure because they are common, composable, and fit the chart/style pattern well:
211
+
212
+ * hyperlinks
213
+ * auto filters and sort state
214
+ * data validation
215
+ * conditional formatting
216
+ * tables
217
+ * comments
218
+ * freeze/split panes and selection
219
+ * page setup, margins, header/footer, print options
220
+ * workbook and sheet protection
221
+ * defined names, print area, print titles
222
+ * shapes and images
223
+ * pivot tables
224
+ * document properties
225
+
226
+ These should be treated as backlog for Facade parity, not as permanently low-level-only features.
227
+
228
+ ---
229
+
230
+ ## Data-Flow & Lifecycle
231
+
232
+ ### `Xlsxrb.read(source)` — In-Memory Read
233
+
234
+ ```
235
+ source (path / IO)
236
+
237
+
238
+ Ooxml::ZipReader ── extracts ZIP entries ──► raw bytes per part
239
+
240
+
241
+ Ooxml::SharedStringsParser ── SAX parse xl/sharedStrings.xml ──► string table (Array)
242
+ Ooxml::StylesParser ── SAX parse xl/styles.xml ──► styles hash
243
+ Ooxml::WorkbookParser ── SAX parse xl/workbook.xml ──► sheet list
244
+
245
+ ▼ (for each sheet)
246
+ Ooxml::WorksheetParser ── SAX parse xl/worksheets/sheetN.xml ──►
247
+ │ yields (row_index, cells_array, row_attrs, unmapped)
248
+
249
+ Elements::Cell / Row / Column / Worksheet
250
+
251
+
252
+ Elements::Workbook ◄── assembled from all worksheets
253
+ ```
254
+
255
+ ### `Xlsxrb.write(target, workbook)` — In-Memory Write
256
+
257
+ ```
258
+ Elements::Workbook
259
+
260
+ ▼ (for each worksheet)
261
+ Ooxml::WorksheetWriter ── converts Row/Cell → XML fragments ──►
262
+ │ streams into Ooxml::ZipWriter entry
263
+
264
+ Ooxml::WorkbookWriter ── writes workbook.xml, styles.xml, sharedStrings.xml,
265
+ │ [Content_Types].xml, .rels
266
+
267
+ Ooxml::ZipWriter ── writes ZIP output ──► target (path / IO)
268
+ ```
269
+
270
+ ### `Xlsxrb.foreach(source, **options)` — Streaming Read
271
+
272
+ ```
273
+ source (path / IO)
274
+
275
+
276
+ Ooxml::ZipReader ── locates xl/sharedStrings.xml, xl/worksheets/sheetN.xml
277
+
278
+ ▼ (SAX parse SST first — kept in memory as a flat Array of strings)
279
+
280
+ ▼ (then SAX stream worksheet)
281
+ Ooxml::WorksheetParser ── on each </row> event:
282
+ │ 1. build Elements::Row with resolved cell values
283
+ │ 2. yield Row to caller's block
284
+ │ 3. discard Row (GC eligible)
285
+
286
+ caller's block receives Elements::Row, processes, moves on
287
+ ```
288
+
289
+ Key memory invariant: only **one Row** (plus the shared-string table) is alive at any time.
290
+
291
+ ### `Xlsxrb.generate(target, &block)` — Streaming Write
292
+
293
+ ```
294
+ caller's block
295
+
296
+ ▼ block receives a StreamWriter context object
297
+ │ context.add_row([val1, val2, ...])
298
+
299
+
300
+ Ooxml::WorksheetWriter ── converts array → <row><c>…</c></row> XML
301
+ │ writes directly to ZipWriter entry stream
302
+
303
+ Ooxml::ZipWriter ── compresses & writes to target
304
+ ```
305
+
306
+ Key memory invariant: rows are written and flushed immediately; no row Array accumulates.
307
+
308
+ ---
309
+
310
+ ## Streaming Internals
311
+
312
+ ### Unified Event-Based Streaming (`Ooxml::Event`)
313
+
314
+ To unify parsing across both streaming and in-memory paths, the OOXML layer utilizes a unified event-based parsing model. Individual parsers (such as `WorksheetParser` and `SharedStringsParser`) implement a streaming `each_event` method that emits a sequence of `Xlsxrb::Ooxml::Event` objects.
315
+
316
+ An event contains:
317
+ - `type`: a Symbol representing the event type (e.g., `:row_start`, `:cell`, `:row_end`, `:column`, `:hyperlink`, `:sst_item`).
318
+ - `args`: an Array containing the event data arguments.
319
+ - `source`: a Hash providing context for error reporting (e.g., `{ part: "xl/worksheets/sheet1.xml", row: 0, cell: "A1" }`).
320
+
321
+ #### Event Vocabulary:
322
+ 1. **Worksheet Events:**
323
+ - `:row_start` - `args: [row_index, attrs]`
324
+ - `:cell` - `args: [ref, type, style_index, value, formula]`
325
+ - `:row_end` - `args: []`
326
+ - `:column` - `args: [min, max, width, hidden, custom_width, outline_level]`
327
+ - `:hyperlink` - `args: [ref, rid, display, tooltip, location]`
328
+ 2. **Shared Strings (SST) Events:**
329
+ - `:sst_item` - `args: [string_value]`
330
+
331
+ The streaming parser `each_row` (or `parse`) consumes the event stream and folds it into raw row hashes, maintaining a minimal state machine and constant memory footprint.
332
+
333
+ ### ZIP Streaming
334
+
335
+ `Ooxml::ZipReader` scans local file headers sequentially using `Zlib::Inflate`. It does **not** seek to the central directory — this allows reading from non-seekable IO (pipes, HTTP streams).
336
+
337
+ `Ooxml::ZipWriter` writes local file headers immediately, accumulates a central directory index in memory (entry names + offsets only), and writes the central directory + EOCD at `#close`.
338
+
339
+ ---
340
+
341
+ ## `unmapped_data` & Forward-Compatibility
342
+
343
+ When the Ooxml layer encounters an XML element or attribute not in its recognized set:
344
+
345
+ 1. **Capture**: The element is stored as a Hash `{ tag: String, attrs: Hash, children: Array, text: String? }`.
346
+ 2. **Attach**: The Hash is pushed onto the nearest recognized parent's `unmapped_children` array.
347
+ 3. **Surface**: The Elements layer receives these as the `unmapped_data` field — a Hash keyed by parent-context (e.g. `{ row: [...], cell: [...], worksheet: [...] }`).
348
+ 4. **Restore**: During write-back (`Ooxml::WorksheetWriter`), `unmapped_data` entries are re-serialized to XML in their original order using `XmlBuilder`, preserving any future spec extensions or vendor-specific markup.
349
+
350
+ This ensures that reading then writing an XLSX file does not silently discard unknown content.
351
+
352
+ ---
353
+
354
+ ## Error Handling & Validation Boundaries
355
+
356
+ ### Ooxml Layer (parse-time)
357
+
358
+ * **Never raises** on unexpected XML content. Unrecognized elements → `unmapped_data`. Malformed attribute values → stored as-is (raw strings).
359
+ * **Raises** only on structural corruption that prevents further parsing (e.g., truncated ZIP, invalid UTF-8, ZIP local header CRC mismatch).
360
+
361
+ ### Elements Layer (model-time)
362
+
363
+ * Each `Data` class exposes `errors` (frozen Array of String) and `valid?` (`errors.empty?`).
364
+ * Validation is performed at construction time:
365
+ - `Cell`: value type check, column/row index range
366
+ - `Row`: index ≥ 0, cells array consistency
367
+ - `Worksheet`: name present, unique row indices
368
+ - `Workbook`: at least one sheet, unique sheet names
369
+ * Invalid objects **are still created** — the caller decides how to handle `valid? == false`.
370
+
371
+ ### Facade Layer
372
+
373
+ * `Xlsxrb.read` / `.foreach`: propagate Ooxml-layer structural exceptions. Content-level issues appear in `errors` on returned objects.
374
+ * `Xlsxrb.write` / `.generate`: validate the `Workbook` / row data at the boundary and raise `Xlsxrb::Error` for fatal issues (e.g., nil target path). Non-fatal issues (e.g., value truncation) are silently handled.
375
+
376
+ ---
377
+
378
+ ## Benefits of this Approach
379
+
380
+ * **Rubyish Interface:** Methods like `foreach` and `generate` follow Ruby's standard library conventions (e.g., `CSV.foreach`).
381
+ * **Clean Namespace:** Users only interact with the `Xlsxrb` module. Internal models are safely isolated within `Elements`.
382
+ * **Safety & LSP Support:** `Data` objects provide clear property definitions for editor autocomplete.
383
+ * **Constant Memory Streaming:** Both read and write paths support row-at-a-time processing suitable for millions of rows.
384
+ * **Future-Proofing:** The `unmapped_data` mechanism and layered design accommodate future features without rewriting the underlying XML logic.
385
+
386
+ ---
387
+
388
+ ## Facade Quality Gates
389
+
390
+ Every new high-level DSL feature must satisfy the following quality rules before it is considered complete.
391
+
392
+ ### 1. Both API paths must be covered
393
+
394
+ If a feature is intended to exist in both writing modes, tests must cover:
395
+
396
+ * `Xlsxrb.build` / `Xlsxrb.write`
397
+ * `Xlsxrb.generate`
398
+
399
+ If a feature can only exist in one mode for a technical reason, that restriction must be documented explicitly in code comments and user-facing docs.
400
+
401
+ ### 2. Both entry forms must be covered when both are supported
402
+
403
+ If a feature exposes both:
404
+
405
+ * an options form
406
+ * a block form
407
+
408
+ then Facade tests should exercise both forms at least once.
409
+
410
+ ### 3. Facade tests are mandatory
411
+
412
+ Add or extend tests in `test/facade_test.rb` so the feature is validated at the public API level.
413
+
414
+ These tests should verify:
415
+
416
+ * the feature can be declared through the high-level DSL
417
+ * the generated file can be read back
418
+ * the semantic result is present in the parsed workbook or reader output
419
+
420
+ ### 4. Contract tests are preferred when structure parity matters
421
+
422
+ If the feature should produce equivalent OOXML across streaming and in-memory modes, add or extend `test/contract_test.rb`.
423
+
424
+ This is especially important for:
425
+
426
+ * shared workbook/worksheet structures
427
+ * range-based features
428
+ * settings that should serialize identically regardless of API path
429
+
430
+ ### 5. E2E coverage is required for new structural output
431
+
432
+ If a feature introduces new XML elements, attributes, relationships, or package parts, add an interoperability or E2E test.
433
+
434
+ At minimum, verify one of:
435
+
436
+ * Open XML SDK validation passes
437
+ * the generated XML parts contain the expected structure and are accepted by the reader
438
+
439
+ ### 6. Documentation must ship with the feature
440
+
441
+ Every new high-level feature should update user-facing docs with:
442
+
443
+ * one short example
444
+ * one richer example if the feature has a block form or nested configuration
445
+ * any important streaming vs in-memory limitation
446
+
447
+ ### 7. Keep surface area smaller than implementation detail
448
+
449
+ Do not promote every low-level flag into a top-level public method immediately.
450
+
451
+ Prefer this order:
452
+
453
+ 1. common user-facing options
454
+ 2. nested builder methods for grouped concepts
455
+ 3. advanced keyword passthrough for rare flags
456
+
457
+ This keeps the DSL readable while still allowing high feature coverage.
458
+
459
+ ### 8. Backward compatibility is a release gate
460
+
461
+ A feature is not complete if it improves symmetry but breaks older call sites, examples, or tests.
462
+
463
+ Backward compatibility must be verified before merging any Facade DSL expansion.
464
+
465
+ ---
466
+
467
+ ## Testing Strategy
468
+
469
+ To ensure library robustness and consistency across execution paths, we organize tests into four distinct layers:
470
+
471
+ 1. **Unit Tests (`test/xlsxrb/`):**
472
+ - Focus on isolated components (such as parsers and writers) without external system dependencies.
473
+ - Includes **Round-trip testing** to verify that generated XML can be successfully parsed back by the reader.
474
+ - Run via: `bundle exec rake test:unit`
475
+
476
+ 2. **Contract Tests (`test/contract/`):**
477
+ - Ensures semantic parity between the Streaming and In-Memory API paths.
478
+ - Operates by executing identical data scenarios on both APIs and asserting that they serialize to equivalent structures.
479
+ - Run via: `bundle exec rake test:contract`
480
+
481
+ 3. **Interop (E2E) Tests (`test/e2e/`):**
482
+ - Exercises real-world interoperability by validating generated files using the official .NET-based **Open XML SDK** validator, and reading spreadsheets dynamically created by the SDK.
483
+ - Run via: `bundle exec rake test:e2e`
484
+
485
+ 4. **Visual Examples & VRT (`test/visual/`):**
486
+ - **Living Documentation:** Compiles visual DSL scripts under `examples/visual/` into the [Visual Examples Gallery](visual/README.md).
487
+ - **Visual Regression Testing:** Renders the generated spreadsheets into PNG files using headless LibreOffice Calc, and calculates pixel differences against reference baselines using ImageMagick.
488
+ - Run via: `bundle exec rake test:visual`
@@ -0,0 +1,42 @@
1
+ # Specification Reference Policy
2
+
3
+ This document defines the specification reference policy for `xlsxrb`.
4
+
5
+ ## Canonical Specification (Normative)
6
+ The primary reference for this library is **ECMA-376 (Office Open XML File Formats)**.
7
+ Specifically, `xlsxrb` targets the **Transitional** version of the specification (Part 4) to ensure maximum compatibility with existing applications such as Microsoft Excel, LibreOffice, and Google Sheets.
8
+
9
+ Local copies of the ECMA-376 specifications are located in the `vendor/docs/` directory:
10
+ - [Part 1: Fundamentals And Markup Language Reference](file:///workspaces/xlsxrb/vendor/docs/ECMA-376-Part1/Ecma%20Office%20Open%20XML%20Part%201%20-%20Fundamentals%20And%20Markup%20Language%20Reference.pdf)
11
+ - [Part 2: Open Packaging Conventions](file:///workspaces/xlsxrb/vendor/docs/ECMA-376-Part2/ECMA-376-2_5th_edition_december_2021.pdf)
12
+ - [Part 3: Markup Compatibility and Extensibility](file:///workspaces/xlsxrb/vendor/docs/ECMA-376-Part3/Ecma%20Office%20Open%20XML%20Part%203%20-%20Markup%20Compatibility%20and%20Extensibility.pdf)
13
+ - [Part 4: Transitional Migration Features](file:///workspaces/xlsxrb/vendor/docs/ECMA-376-Part4/Ecma%20Office%20Open%20XML%20Part%204%20-%20Transitional%20Migration%20Features.pdf)
14
+
15
+ ## Supplementary Specifications (Excel Real-world Behavior)
16
+ To address gaps between the official ECMA standard and actual implementations in Microsoft Excel, the following Microsoft Open Specifications are used. Note that these files are not bundled in the repository to avoid licensing/redistribution issues; instead, they are referenced via online links and versioned here:
17
+
18
+ 1. **[[MS-XLSX]: Excel Extensions to OOXML SpreadsheetML Structure](https://learn.microsoft.com/en-us/openspecs/office_standards/ms-xlsx/)**
19
+ - **Role:** Explains Excel-specific extensions, default attributes, and schema extensions.
20
+ - **Referenced Version:** July 2024 / Version 12.0 (or current release).
21
+ 2. **[[MS-OI29500]: Office Implementation Information for ISO/IEC 29500](https://learn.microsoft.com/en-us/openspecs/office_standards/ms-oi29500/)**
22
+ - **Role:** Identifies how Excel actually reads/writes files, including deviations and compatibility behaviors.
23
+ - **Referenced Version:** July 2024 / Version 12.0 (or current release).
24
+ 3. **[[MS-OFFCRYPTO]: Office Document Cryptography Structure](https://learn.microsoft.com/en-us/openspecs/office_standards/ms-offcrypto/)**
25
+ - **Role:** Provides details on encryption, passwords, and hashing algorithms used for document/sheet protection.
26
+ - **Referenced Version:** July 2024 / Version 12.0 (or current release).
27
+
28
+ ### ISO/IEC 29500 Note
29
+ ISO/IEC 29500 is contents-wise equivalent to ECMA-376. However, ISO/IEC 29500 requires paid purchase in general, whereas ECMA-376 is freely available. Therefore, we primarily cite ECMA-376 sections.
30
+
31
+ ## Source Attribution Policy
32
+ When implementing features or fixing bugs that depend on specific behaviors defined in these specifications, developers should:
33
+ 1. Reference the specific part, section, or page number in code comments (e.g., `# Reference: ECMA-376 Part 1, Section 18.3.1.73`).
34
+ 2. Update the mapping table below to maintain a central registry of implemented standard behaviors.
35
+
36
+ ## Implementation Specification Mapping Table
37
+ | Feature | Primary Standard | Specification Section | Notes |
38
+ | :--- | :--- | :--- | :--- |
39
+ | Cell Value Types | ECMA-376 Part 1 | §18.18.11 (ST_CellType) | Defines cell data types (boolean, number, string, formula, etc.) |
40
+ | Shared Strings | ECMA-376 Part 1 | §18.4 (Shared String Table) | Handling of `<sst>` and `<si>` for cell value reuse |
41
+ | Styles & Formatting | ECMA-376 Part 1 | §18.8 (Styles) | Cell style XF indexes, font, fill, border mappings |
42
+ | Hyperlinks | ECMA-376 Part 1 | §18.3.1.48 (hyperlink) | Worksheet hyperlinks referencing external URLs or internal targets |