serialbench 0.1.0 → 0.1.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
data/README.adoc CHANGED
@@ -1,31 +1,282 @@
1
- = Serialbench: Comprehensive serialization benchmarking suite for Ruby
1
+ = Serialbench: Ruby serialization library performance benchmarker
2
2
 
3
3
  image:https://img.shields.io/gem/v/serialbench.svg["Gem Version", link="https://rubygems.org/gems/serialbench"]
4
- image:https://github.com/example/serialbench/actions/workflows/rake.yml/badge.svg["Build Status", link="https://github.com/example/serialbench/actions/workflows/rake.yml"]
5
- image:https://img.shields.io/github/issues-pr-raw/example/serialbench.svg["Pull Requests", link="https://github.com/example/serialbench/pulls"]
6
- image:https://img.shields.io/github/commits-since/example/serialbench/latest.svg["Commits since latest",link="https://github.com/example/serialbench/releases"]
4
+ image:https://github.com/metanorma/serialbench/actions/workflows/ci.yml/badge.svg["Build Status", link="https://github.com/metanorma/serialbench/actions/workflows/ci.yml"]
5
+ image:https://github.com/metanorma/serialbench/actions/workflows/benchmark.yml/badge.svg["Benchmark Status", link="https://github.com/metanorma/serialbench/actions/workflows/benchmark.yml"]
6
+ image:https://img.shields.io/github/issues-pr-raw/metanorma/serialbench.svg["Pull Requests", link="https://github.com/metanorma/serialbench/pulls"]
7
7
 
8
- == Purpose
8
+ == Overview
9
9
 
10
- Serialbench is a comprehensive benchmarking suite that evaluates the performance of popular Ruby serialization libraries across multiple formats and dimensions including parsing speed, generation speed, memory usage, and feature completeness.
10
+ Serialbench is a comprehensive benchmarking suite that evaluates the performance of popular Ruby serialization libraries across multiple formats. It provides detailed performance comparisons and analysis to help developers make informed decisions when choosing serialization libraries for their Ruby applications.
11
11
 
12
- This tool helps developers make informed decisions when choosing serialization libraries for their Ruby applications by providing detailed performance comparisons and analysis across XML, JSON, and TOML formats.
12
+ **Supported Formats**: XML, JSON, YAML, TOML, and more
13
13
 
14
- === Tested XML libraries
14
+ **Key Metrics**: Parsing speed, generation speed, memory usage, streaming capabilities, and feature completeness
15
15
 
16
- ==== Core XML Libraries
17
- * **Ox** - High-performance XML parser optimized for speed and low memory usage
18
- * **Nokogiri** - Feature-rich XML/HTML parser with XPath support and comprehensive DOM manipulation
19
- * **LibXML** - Ruby bindings for the libxml2 C library with excellent performance characteristics
20
- * **Oga** - Pure Ruby XML parser with XPath support and streaming capabilities
21
- * **REXML** - Ruby's built-in XML parser with streaming support (reference implementation)
16
+ **Docker Support**: Multi-Ruby version benchmarking with automated result aggregation and GitHub Pages generation
22
17
 
23
- ==== Additional Format Support
24
- * **JSON** - Ruby's built-in JSON parser (for comparison baseline)
25
- * **Oj** - High-performance JSON parser with streaming support (for comparison baseline)
26
- * **YAJL** - Yet Another JSON Library with streaming support (for comparison baseline)
27
- * **TOML-RB** - Ruby TOML parser (for comparison baseline)
28
- * **Tomlib** - Fast TOML parser (for comparison baseline)
18
+ == Supported serialization libraries
19
+
20
+ [cols="1,3,1,4", options="header"]
21
+ |===
22
+ | Format | Name | Version | Description
23
+
24
+ | XML
25
+ | https://github.com/ohler55/ox[Ox]
26
+ | v2.14.23
27
+ | C extension XML parser
28
+
29
+ | XML
30
+ | https://github.com/xml4r/libxml-ruby[LibXML]
31
+ | v4.1.2
32
+ | Ruby bindings for libxml2
33
+
34
+ | XML
35
+ | https://github.com/sparklemotion/nokogiri[Nokogiri]
36
+ | v1.18.8
37
+ | XML/HTML parser with XPath and CSS selectors
38
+
39
+ | XML
40
+ | https://github.com/YorickPeterse/oga[Oga]
41
+ | v3.4
42
+ | Pure Ruby XML parser with XPath support
43
+
44
+ | XML
45
+ | https://github.com/ruby/rexml[REXML]
46
+ | v3.4.1
47
+ | Ruby's standard library XML parser
48
+
49
+ | JSON
50
+ | https://github.com/ohler55/oj[Oj]
51
+ | v3.16.11
52
+ | JSON parser with multiple parsing modes
53
+
54
+ | JSON
55
+ | https://github.com/brianmario/yajl-ruby[YAJL]
56
+ | v1.4.3
57
+ | JSON library with streaming capabilities
58
+
59
+ | JSON
60
+ | https://github.com/flori/json[JSON]
61
+ | v2.12.2
62
+ | Ruby's standard library JSON parser
63
+
64
+ | YAML
65
+ | https://github.com/ruby/psych[Psych]
66
+ | v5.1.2
67
+ | Ruby's standard library YAML parser
68
+
69
+ | YAML
70
+ | https://github.com/ruby/syck[Syck]
71
+ | v1.5.1.1
72
+ | Legacy YAML parser
73
+
74
+ | TOML
75
+ | https://github.com/fbernier/tomlib[Tomlib]
76
+ | v0.7.3
77
+ | TOML parser implemented in C
78
+
79
+ | TOML
80
+ | https://github.com/emancu/toml-rb[TOML-RB]
81
+ | v2.2.0
82
+ | Pure Ruby TOML parser
83
+ |===
84
+
85
+ == Benchmark report JSON format
86
+
87
+ === Single Ruby version results
88
+
89
+ The benchmark results are saved as JSON files with the following structure:
90
+
91
+ [source,json]
92
+ ----
93
+ {
94
+ "environment": {
95
+ "ruby_version": "3.3.8",
96
+ "ruby_platform": "aarch64-linux",
97
+ "serializer_versions": {
98
+ "rexml": "3.4.1",
99
+ "ox": "2.14.23",
100
+ "nokogiri": "1.18.8",
101
+ "oga": "3.4",
102
+ "libxml": "4.1.2",
103
+ "json": "2.12.2",
104
+ "oj": "3.16.11",
105
+ "yajl": "1.4.3",
106
+ "psych": "5.1.2",
107
+ "syck": "1.5.1.1",
108
+ "toml-rb": "2.2.0",
109
+ "tomlib": "0.7.3"
110
+ },
111
+ "timestamp": "2025-06-07T09:00:25+00:00"
112
+ },
113
+ "parsing": {
114
+ "small": {
115
+ "xml": {
116
+ "rexml": {
117
+ "time_per_iterations": 0.002514416000053643,
118
+ "time_per_iteration": 0.00012572080000268216,
119
+ "iterations_per_second": 7954.133285650949,
120
+ "iterations_count": 20
121
+ },
122
+ "ox": {
123
+ "time_per_iterations": 0.00005258399994545471,
124
+ "time_per_iteration": 0.0000026291999972727353,
125
+ "iterations_per_second": 380343.83121759404,
126
+ "iterations_count": 20
127
+ }
128
+ },
129
+ "json": {
130
+ "json": {
131
+ "time_per_iterations": 0.000029707999942729657,
132
+ "time_per_iteration": 0.000001485399997136483,
133
+ "iterations_per_second": 673219.3361571126,
134
+ "iterations_count": 20
135
+ },
136
+ "oj": {
137
+ "time_per_iterations": 0.00003158300000905001,
138
+ "time_per_iteration": 0.0000015791500004525006,
139
+ "iterations_per_second": 633252.0658034089,
140
+ "iterations_count": 20
141
+ }
142
+ }
143
+ },
144
+ "medium": {
145
+ "xml": { /* ... */ },
146
+ "json": { /* ... */ }
147
+ },
148
+ "large": {
149
+ "xml": { /* ... */ },
150
+ "json": { /* ... */ }
151
+ }
152
+ },
153
+ "ruby_version": "3.3.8",
154
+ "ruby_platform": "aarch64-linux",
155
+ "timestamp": "2025-06-07T09:01:02+00:00"
156
+ }
157
+ ----
158
+
159
+ === Multi-Ruby version merged results
160
+
161
+ When results from multiple Ruby versions are merged, the structure becomes:
162
+
163
+ [source,json]
164
+ ----
165
+ {
166
+ "environments": {
167
+ "3_3_8_aarch64_linux": {
168
+ "ruby_version": "3.3.8",
169
+ "ruby_platform": "aarch64-linux",
170
+ "source_file": "docker-results/ruby-3.3/data/results.json",
171
+ "timestamp": "2025-06-07T09:01:02+00:00",
172
+ "environment": {
173
+ "ruby_version": "3.3.8",
174
+ "ruby_platform": "aarch64-linux",
175
+ "serializer_versions": { /* ... */ },
176
+ "timestamp": "2025-06-07T09:00:25+00:00"
177
+ }
178
+ },
179
+ "3_4_4_aarch64_linux": {
180
+ "ruby_version": "3.4.4",
181
+ "ruby_platform": "aarch64-linux",
182
+ /* ... */
183
+ }
184
+ },
185
+ "combined_results": {
186
+ "parsing": {
187
+ "small": {
188
+ "xml": {
189
+ "rexml": {
190
+ "3_3_8_aarch64_linux": {
191
+ "time_per_iterations": 0.002514416000053643,
192
+ "time_per_iteration": 0.00012572080000268216,
193
+ "iterations_per_second": 7954.133285650949,
194
+ "iterations_count": 20
195
+ },
196
+ "3_4_4_aarch64_linux": {
197
+ "time_per_iterations": 0.0025308329999234047,
198
+ "time_per_iteration": 0.00012654164999617023,
199
+ "iterations_per_second": 7902.536437846866,
200
+ "iterations_count": 20
201
+ }
202
+ }
203
+ }
204
+ }
205
+ }
206
+ },
207
+ "metadata": {
208
+ "merged_at": "2025-06-07T17:01:48+08:00",
209
+ "ruby_versions": ["3.3.8", "3.4.4"],
210
+ "platforms": ["aarch64-linux"]
211
+ }
212
+ }
213
+ ----
214
+
215
+ === JSON schema specification
216
+
217
+ The benchmark results follow this schema:
218
+
219
+ [source,json]
220
+ ----
221
+ {
222
+ "$schema": "http://json-schema.org/draft-07/schema#",
223
+ "title": "Serialbench Results",
224
+ "type": "object",
225
+ "properties": {
226
+ "environment": {
227
+ "type": "object",
228
+ "properties": {
229
+ "ruby_version": { "type": "string" },
230
+ "ruby_platform": { "type": "string" },
231
+ "serializer_versions": {
232
+ "type": "object",
233
+ "additionalProperties": { "type": "string" }
234
+ },
235
+ "timestamp": { "type": "string", "format": "date-time" }
236
+ },
237
+ "required": ["ruby_version", "ruby_platform", "serializer_versions", "timestamp"]
238
+ },
239
+ "parsing": {
240
+ "type": "object",
241
+ "properties": {
242
+ "small": { "$ref": "#/definitions/sizeResults" },
243
+ "medium": { "$ref": "#/definitions/sizeResults" },
244
+ "large": { "$ref": "#/definitions/sizeResults" }
245
+ }
246
+ },
247
+ "ruby_version": { "type": "string" },
248
+ "ruby_platform": { "type": "string" },
249
+ "timestamp": { "type": "string", "format": "date-time" }
250
+ },
251
+ "definitions": {
252
+ "sizeResults": {
253
+ "type": "object",
254
+ "properties": {
255
+ "xml": { "$ref": "#/definitions/formatResults" },
256
+ "json": { "$ref": "#/definitions/formatResults" },
257
+ "yaml": { "$ref": "#/definitions/formatResults" },
258
+ "toml": { "$ref": "#/definitions/formatResults" }
259
+ }
260
+ },
261
+ "formatResults": {
262
+ "type": "object",
263
+ "additionalProperties": {
264
+ "$ref": "#/definitions/serializerResults"
265
+ }
266
+ },
267
+ "serializerResults": {
268
+ "type": "object",
269
+ "properties": {
270
+ "time_per_iterations": { "type": "number" },
271
+ "time_per_iteration": { "type": "number" },
272
+ "iterations_per_second": { "type": "number" },
273
+ "iterations_count": { "type": "integer" }
274
+ },
275
+ "required": ["time_per_iterations", "time_per_iteration", "iterations_per_second", "iterations_count"]
276
+ }
277
+ }
278
+ }
279
+ ----
29
280
 
30
281
  == Installation
31
282
 
@@ -38,14 +289,14 @@ gem 'serialbench'
38
289
 
39
290
  And then execute:
40
291
 
41
- [source,shell]
292
+ [source]
42
293
  ----
43
294
  $ bundle install
44
295
  ----
45
296
 
46
297
  Or install it yourself as:
47
298
 
48
- [source,shell]
299
+ [source]
49
300
  ----
50
301
  $ gem install serialbench
51
302
  ----
@@ -54,7 +305,8 @@ $ gem install serialbench
54
305
 
55
306
  To run benchmarks for all supported XML libraries, install the following gems:
56
307
 
57
- [source,shell]
308
+
309
+ [source]
58
310
  ----
59
311
  # Core XML libraries
60
312
  $ gem install ox nokogiri libxml-ruby oga
@@ -66,20 +318,25 @@ $ gem install oj toml-rb
66
318
  $ gem install memory_profiler
67
319
  ----
68
320
 
69
- NOTE: REXML and JSON are included with Ruby and require no additional installation.
321
+
322
+ NOTE: REXML, JSON, and Psych (YAML) are included with Ruby and require no additional installation.
70
323
 
71
324
  === Library-specific installation notes
72
325
 
73
326
  ==== Ox
327
+
74
328
  High-performance C extension requiring compilation:
75
- [source,shell]
329
+
330
+ [source]
76
331
  ----
77
332
  $ gem install ox
78
333
  ----
79
334
 
80
335
  ==== Nokogiri
336
+
81
337
  May require system dependencies on some platforms:
82
- [source,shell]
338
+
339
+ [source]
83
340
  ----
84
341
  # macOS with Homebrew
85
342
  $ brew install libxml2 libxslt
@@ -91,8 +348,10 @@ $ gem install nokogiri
91
348
  ----
92
349
 
93
350
  ==== LibXML
351
+
94
352
  Ruby bindings for libxml2:
95
- [source,shell]
353
+
354
+ [source]
96
355
  ----
97
356
  # macOS with Homebrew
98
357
  $ brew install libxml2
@@ -104,77 +363,531 @@ $ gem install libxml-ruby
104
363
  ----
105
364
 
106
365
  ==== Oga
366
+
107
367
  Pure Ruby implementation with no system dependencies:
108
- [source,shell]
368
+
369
+ [source]
109
370
  ----
110
371
  $ gem install oga
111
372
  ----
112
373
 
113
- == Usage
374
+ == Docker containers and cross-platform benchmarking
114
375
 
115
- === Command line interface
376
+ Serialbench provides pre-built Docker containers and supports cross-platform benchmarking on Windows, macOS, and Linux. Our containers are automatically built and published to GitHub Container Registry for easy access.
377
+
378
+ === Published Docker containers
379
+
380
+ We publish multi-architecture Docker containers for all supported Ruby versions:
381
+
382
+ **Registry**: `ghcr.io/metanorma/serialbench`
383
+
384
+ **Available tags**:
385
+ * `ghcr.io/metanorma/serialbench:main-ruby-3.1` - Ruby 3.1 (linux/amd64, linux/arm64)
386
+ * `ghcr.io/metanorma/serialbench:main-ruby-3.2` - Ruby 3.2 (linux/amd64, linux/arm64)
387
+ * `ghcr.io/metanorma/serialbench:main-ruby-3.3` - Ruby 3.3 (linux/amd64, linux/arm64)
388
+ * `ghcr.io/metanorma/serialbench:main-ruby-3.4` - Ruby 3.4 (linux/amd64, linux/arm64)
389
+
390
+ **Container features**:
391
+ * All serialization libraries pre-installed (XML, JSON, YAML, TOML)
392
+ * Multi-architecture support (Intel x64 and ARM64)
393
+ * Optimized for CI/CD and local development
394
+ * Automatic updates on code changes
395
+
396
+ === Running benchmarks on your computer
116
397
 
117
- Run the complete XML benchmark suite:
398
+ ==== Windows (PowerShell/Command Prompt)
118
399
 
119
- [source,shell]
400
+ [source,powershell]
120
401
  ----
121
- $ serialbench benchmark
402
+ # Pull and run latest Ruby 3.3 container
403
+ docker pull ghcr.io/metanorma/serialbench:main-ruby-3.3
404
+
405
+ # Create results directory
406
+ mkdir results
407
+
408
+ # Run benchmarks with volume mounting
409
+ docker run --rm -v ${PWD}/results:/app/results ghcr.io/metanorma/serialbench:main-ruby-3.3
410
+
411
+ # View results
412
+ dir results
122
413
  ----
123
414
 
124
- Run XML-only benchmarks:
415
+ ==== macOS (Terminal)
125
416
 
126
- [source,shell]
417
+ [source,bash]
127
418
  ----
128
- $ serialbench benchmark --formats xml
419
+ # Pull and run latest Ruby 3.3 container
420
+ docker pull ghcr.io/metanorma/serialbench:main-ruby-3.3
421
+
422
+ # Create results directory
423
+ mkdir -p results
424
+
425
+ # Run benchmarks with volume mounting
426
+ docker run --rm -v $(pwd)/results:/app/results ghcr.io/metanorma/serialbench:main-ruby-3.3
427
+
428
+ # View results
429
+ ls -la results/
129
430
  ----
130
431
 
131
- Run benchmarks with comparison formats:
432
+ ==== Ubuntu/Linux (Terminal)
132
433
 
133
- [source,shell]
434
+ [source,bash]
134
435
  ----
135
- $ serialbench benchmark --formats xml json
136
- $ serialbench benchmark --formats xml json toml
436
+ # Pull and run latest Ruby 3.3 container
437
+ docker pull ghcr.io/metanorma/serialbench:main-ruby-3.3
438
+
439
+ # Create results directory
440
+ mkdir -p results
441
+
442
+ # Run benchmarks with volume mounting
443
+ docker run --rm -v $(pwd)/results:/app/results ghcr.io/metanorma/serialbench:main-ruby-3.3
444
+
445
+ # View results
446
+ ls -la results/
137
447
  ----
138
448
 
139
- Run only DOM parsing benchmarks:
449
+ === Multi-Ruby version comparison
450
+
451
+ Run benchmarks across all supported Ruby versions:
452
+
453
+ ==== Windows (PowerShell)
140
454
 
141
- [source,shell]
455
+ [source,powershell]
142
456
  ----
143
- $ serialbench benchmark --parsing-only
457
+ # Create results directories
458
+ $versions = @("3.1", "3.2", "3.3", "3.4")
459
+ foreach ($version in $versions) {
460
+ mkdir "results-ruby-$version" -Force
461
+ docker pull "ghcr.io/metanorma/serialbench:main-ruby-$version"
462
+ docker run --rm -v "${PWD}/results-ruby-${version}:/app/results" "ghcr.io/metanorma/serialbench:main-ruby-$version"
463
+ }
464
+
465
+ # View all results
466
+ dir results-ruby-*
144
467
  ----
145
468
 
146
- Run only XML generation benchmarks:
469
+ ==== macOS/Linux (Bash)
147
470
 
148
- [source,shell]
471
+ [source,bash]
149
472
  ----
150
- $ serialbench benchmark --generation-only
473
+ # Run benchmarks for all Ruby versions
474
+ for version in 3.1 3.2 3.3 3.4; do
475
+ echo "Running benchmarks for Ruby $version..."
476
+ mkdir -p "results-ruby-$version"
477
+ docker pull "ghcr.io/metanorma/serialbench:main-ruby-$version"
478
+ docker run --rm \
479
+ -v "$(pwd)/results-ruby-$version:/app/results" \
480
+ "ghcr.io/metanorma/serialbench:main-ruby-$version"
481
+ done
482
+
483
+ # View all results
484
+ ls -la results-ruby-*/
151
485
  ----
152
486
 
153
- Run only streaming/SAX parsing benchmarks:
487
+ === Custom benchmark configuration
488
+
489
+ Run benchmarks with custom parameters:
154
490
 
155
- [source,shell]
491
+ [source,bash]
156
492
  ----
157
- $ serialbench benchmark --streaming-only
493
+ # Run specific formats only
494
+ docker run --rm \
495
+ -v $(pwd)/results:/app/results \
496
+ ghcr.io/metanorma/serialbench:main-ruby-3.3 \
497
+ bundle exec serialbench benchmark --formats xml json --iterations 10
498
+
499
+ # Run with memory profiling
500
+ docker run --rm \
501
+ -v $(pwd)/results:/app/results \
502
+ ghcr.io/metanorma/serialbench:main-ruby-3.3 \
503
+ bundle exec serialbench benchmark --memory-profiling
504
+
505
+ # List available serializers
506
+ docker run --rm ghcr.io/metanorma/serialbench:main-ruby-3.3 \
507
+ bundle exec serialbench list
158
508
  ----
159
509
 
160
- Output results in JSON format only:
510
+ === Quick start with Docker
161
511
 
162
- [source,shell]
512
+ ==== Prerequisites
513
+
514
+ * Docker installed and running
515
+ * Command line access (PowerShell, Terminal, or Bash)
516
+
517
+ ==== Running multi-Ruby benchmarks
518
+
519
+ [source]
163
520
  ----
164
- $ serialbench benchmark --output-format json
521
+ # From the project root directory (if you have the source)
522
+ $ ./docker/run-benchmarks.sh
523
+
524
+ # Or using published containers directly
525
+ $ docker pull ghcr.io/metanorma/serialbench:main-ruby-3.3
526
+ $ docker run --rm -v $(pwd)/results:/app/results ghcr.io/metanorma/serialbench:main-ruby-3.3
527
+ ----
528
+
529
+ This will:
530
+
531
+ . **Pull pre-built containers** from GitHub Container Registry
532
+ . **Run comprehensive benchmarks** in isolated environments
533
+ . **Generate detailed reports** with performance comparisons
534
+ . **Output results** to your local `results/` directory
535
+
536
+ ==== Results structure
537
+
538
+ Results are organized in `docker-results/`:
539
+
540
+ [source]
541
+ ----
542
+ docker-results/
543
+ ├── ruby-3.0/ # Ruby 3.0 individual results
544
+ │ ├── benchmark.log # Execution log
545
+ │ ├── data/
546
+ │ │ ├── results.json # Raw benchmark data
547
+ │ │ └── results.yaml # YAML format results
548
+ │ ├── reports/
549
+ │ │ └── benchmark_report.html
550
+ │ └── assets/
551
+ ├── ruby-3.1/ # Ruby 3.1 individual results
552
+ ├── ruby-3.2/ # Ruby 3.2 individual results
553
+ ├── ruby-3.3/ # Ruby 3.3 individual results
554
+ ├── ruby-3.4/ # Ruby 3.4 individual results
555
+ ├── merged/ # Aggregated cross-version results
556
+ │ └── merged_results.json # Combined performance data
557
+ └── docs/ # GitHub Pages ready output
558
+ ├── index.html # Interactive comparison report
559
+ ├── styles.css # Report styling
560
+ └── merged_results.json # Data for interactive charts
561
+ ----
562
+
563
+ === Manual Docker usage
564
+
565
+ ==== Build image for specific Ruby version
566
+
567
+ [source]
568
+ ----
569
+ $ docker build \
570
+ --build-arg RUBY_VERSION=3.3 \
571
+ -t serialbench:ruby-3.3 \
572
+ -f docker/Dockerfile.benchmark \
573
+ .
574
+ ----
575
+
576
+ ==== Run benchmarks in container
577
+
578
+ [source]
579
+ ----
580
+ # Create results directory
581
+ $ mkdir -p results
582
+
583
+ # Run benchmarks with volume mounting
584
+ $ docker run \
585
+ --rm \
586
+ -v $(pwd)/results:/app/results \
587
+ serialbench:ruby-3.3
588
+ ----
589
+
590
+ ==== Custom configuration
591
+
592
+ [source]
593
+ ----
594
+ # Use custom config file
595
+ $ docker run \
596
+ --rm \
597
+ -v $(pwd)/results:/app/results \
598
+ -v $(pwd)/config:/app/config \
599
+ serialbench:ruby-3.3 \
600
+ bundle exec serialbench benchmark --config config/ci.yml
601
+ ----
602
+
603
+ === Supported Ruby versions
604
+
605
+ The Docker setup supports the following Ruby versions:
606
+
607
+ * **Ruby 3.0** - Stable release with good performance baseline
608
+ * **Ruby 3.1** - Improved performance and new features
609
+ * **Ruby 3.2** - Enhanced YJIT and memory optimizations
610
+ * **Ruby 3.3** - Latest stable with performance improvements
611
+ * **Ruby 3.4** - Current development version
612
+
613
+ Each version includes all supported serialization libraries:
614
+
615
+ * **XML**: REXML (built-in), Ox, Nokogiri, Oga, LibXML
616
+ * **JSON**: JSON (built-in), Oj, YAJL
617
+ * **YAML**: Psych (built-in), Syck
618
+ * **TOML**: TOML-RB, Tomlib
619
+
620
+ === Environment variables
621
+
622
+ The Docker images support these environment variables:
623
+
624
+ * `BUNDLE_PATH` - Bundle installation path
625
+ * `BUNDLE_BIN` - Bundle binary path
626
+ * `PATH` - System PATH including bundle binaries
627
+ * `RUBY_VERSION` - Ruby version for build-time configuration
628
+
629
+ === Result aggregation and GitHub Pages
630
+
631
+ ==== Merging multi-version results
632
+
633
+ The Docker workflow automatically merges results from all Ruby versions:
634
+
635
+ [source]
636
+ ----
637
+ # Manual result merging
638
+ $ serialbench merge_results \
639
+ docker-results/ruby-3.0 \
640
+ docker-results/ruby-3.1 \
641
+ docker-results/ruby-3.2 \
642
+ docker-results/ruby-3.3 \
643
+ docker-results/ruby-3.4 \
644
+ docker-results/merged
645
+ ----
646
+
647
+ ==== GitHub Pages generation
648
+
649
+ Generate interactive HTML reports ready for GitHub Pages deployment:
650
+
651
+ [source]
652
+ ----
653
+ # Generate GitHub Pages from multi-version results
654
+ $ serialbench github_pages \
655
+ docker-results/ruby-3.0 \
656
+ docker-results/ruby-3.1 \
657
+ docker-results/ruby-3.2 \
658
+ docker-results/ruby-3.3 \
659
+ docker-results/ruby-3.4 \
660
+ docker-results/docs
165
661
  ----
166
662
 
167
- List available XML parsers:
663
+ The generated GitHub Pages include:
168
664
 
169
- [source,shell]
665
+ * **Interactive Performance Charts**: Compare serializers across Ruby versions
666
+ * **Multi-Version Analysis**: See how performance changes between Ruby releases
667
+ * **Environment Details**: Ruby versions, platforms, and library versions
668
+ * **Responsive Design**: Works on desktop and mobile devices
669
+ * **Direct Deployment**: Ready for GitHub Pages, Netlify, or any static hosting
670
+
671
+ ==== Deploying to GitHub Pages
672
+
673
+ . **Commit the generated files**:
674
+ +
675
+ [source]
676
+ ----
677
+ $ git add docker-results/docs/
678
+ $ git commit -m "Add multi-Ruby benchmark results"
679
+ $ git push origin main
680
+ ----
681
+
682
+ . **Enable GitHub Pages** in repository settings:
683
+ .. Go to Settings → Pages
684
+ .. Set source to "Deploy from a branch"
685
+ .. Select branch containing the `docs/` folder
686
+ .. Set folder to `/docker-results/docs`
687
+
688
+ . **Access your results** at: `https://yourusername.github.io/yourrepo/`
689
+
690
+ === Troubleshooting Docker issues
691
+
692
+ ==== Build failures
693
+
694
+ Check build logs for specific Ruby versions:
695
+
696
+ [source]
697
+ ----
698
+ $ cat docker-results/build-ruby-3.3.log
699
+ ----
700
+
701
+ Common build issues:
702
+
703
+ * **Missing system dependencies**: Ensure libxml2-dev and libxslt1-dev are available
704
+ * **Network timeouts**: Retry the build or use a different network
705
+ * **Disk space**: Ensure sufficient disk space for multiple Ruby images
706
+
707
+ ==== Runtime failures
708
+
709
+ Check benchmark execution logs:
710
+
711
+ [source]
712
+ ----
713
+ $ cat docker-results/ruby-3.3/benchmark.log
714
+ ----
715
+
716
+ Common runtime issues:
717
+
718
+ * **Memory constraints**: Increase Docker memory allocation
719
+ * **Timeout issues**: Some benchmarks may take longer on slower systems
720
+ * **Permission errors**: Ensure proper volume mounting permissions
721
+
722
+ ==== Docker system issues
723
+
724
+ Verify Docker is running properly:
725
+
726
+ [source]
727
+ ----
728
+ $ docker info
729
+ $ docker system df # Check disk usage
730
+ $ docker system prune # Clean up unused resources
731
+ ----
732
+
733
+ Clean up Serialbench Docker resources:
734
+
735
+ [source]
736
+ ----
737
+ # Remove all Serialbench images
738
+ $ docker rmi $(docker images serialbench -q)
739
+
740
+ # Remove all containers
741
+ $ docker container prune
742
+ ----
743
+
744
+ === Customization options
745
+
746
+ ==== Adding Ruby versions
747
+
748
+ Edit the `RUBY_VERSIONS` array in `docker/run-benchmarks.sh`:
749
+
750
+ [source,bash]
751
+ ----
752
+ RUBY_VERSIONS=("3.0" "3.1" "3.2" "3.3" "3.4" "head")
753
+ ----
754
+
755
+ ==== Custom benchmark configuration
756
+
757
+ Create custom config files in the `config/` directory:
758
+
759
+ [source,yaml]
760
+ ----
761
+ # config/custom.yml
762
+ formats:
763
+ - xml
764
+ - json
765
+ iterations: 50
766
+ warmup: 5
767
+ data_sizes:
768
+ - small
769
+ - medium
770
+ ----
771
+
772
+ Reference the custom config in the run script:
773
+
774
+ [source,bash]
775
+ ----
776
+ # In docker/run-benchmarks.sh
777
+ CONFIG_FILE="config/custom.yml"
778
+ ----
779
+
780
+ ==== Output directory customization
781
+
782
+ Change the output directory in the run script:
783
+
784
+ [source,bash]
785
+ ----
786
+ # In docker/run-benchmarks.sh
787
+ OUTPUT_DIR="my-benchmark-results"
788
+ ----
789
+
790
+ === Integration with CI/CD
791
+
792
+ ==== GitHub Actions integration
793
+
794
+ The Docker setup integrates seamlessly with GitHub Actions:
795
+
796
+ [source,yaml]
797
+ ----
798
+ # .github/workflows/benchmark.yml
799
+ name: Multi-Ruby Benchmarks
800
+
801
+ on:
802
+ schedule:
803
+ - cron: '0 2 * * 0' # Weekly on Sunday at 2 AM
804
+ workflow_dispatch:
805
+
806
+ permissions:
807
+ contents: read
808
+ pages: write
809
+ id-token: write
810
+
811
+ concurrency:
812
+ group: "pages"
813
+ cancel-in-progress: false
814
+
815
+ jobs:
816
+ benchmark:
817
+ runs-on: ubuntu-latest
818
+ steps:
819
+ - name: Checkout repository
820
+ uses: actions/checkout@v4
821
+
822
+ - name: Setup Docker Buildx
823
+ uses: docker/setup-buildx-action@v3
824
+
825
+ - name: Run Docker Benchmarks
826
+ run: ./docker/run-benchmarks.sh
827
+
828
+ - name: Upload Results
829
+ uses: actions/upload-artifact@v4
830
+ with:
831
+ name: benchmark-results
832
+ path: docker-results/
833
+
834
+ - name: Setup Pages
835
+ uses: actions/configure-pages@v4
836
+
837
+ - name: Upload Pages artifact
838
+ uses: actions/upload-pages-artifact@v3
839
+ with:
840
+ path: ./docker-results/docs
841
+
842
+ deploy:
843
+ environment:
844
+ name: github-pages
845
+ url: ${{ steps.deployment.outputs.page_url }}
846
+ runs-on: ubuntu-latest
847
+ needs: benchmark
848
+ steps:
849
+ - name: Deploy to GitHub Pages
850
+ id: deployment
851
+ uses: actions/deploy-pages@v4
852
+ ----
853
+
854
+ ==== Performance considerations
855
+
856
+ * **Parallel builds**: Docker builds can run in parallel for faster execution
857
+ * **Build caching**: Subsequent runs use cached layers for faster builds
858
+ * **Memory profiling**: Enabled by default but can be disabled for faster runs
859
+ * **Result compression**: Large result files can be compressed for storage
860
+
861
+ === Security considerations
862
+
863
+ * **Containers run with minimal privileges**: No root access required
864
+ * **No network access during benchmarks**: Isolated execution environment
865
+ * **Volume mounting**: Results written only to specified mounted volumes
866
+ * **Image scanning**: Regular security updates for base Ruby images
867
+
868
+ == Usage
869
+
870
+ === Command line interface
871
+
872
+ ==== Basic usage
873
+
874
+ Run benchmarks for all available formats:
875
+
876
+ [source]
877
+ ----
878
+ $ serialbench benchmark
879
+ ----
880
+
881
+ List all available serializers:
882
+
883
+ [source]
170
884
  ----
171
885
  $ serialbench list
172
- $ serialbench list --format xml
173
886
  ----
174
887
 
175
888
  Show help information:
176
889
 
177
- [source,shell]
890
+ [source]
178
891
  ----
179
892
  $ serialbench help
180
893
  $ serialbench help benchmark
@@ -182,47 +895,142 @@ $ serialbench help benchmark
182
895
 
183
896
  Show version:
184
897
 
185
- [source,shell]
898
+ [source]
186
899
  ----
187
900
  $ serialbench version
188
901
  ----
189
902
 
190
- === XML-specific benchmark options
903
+ ==== Format-specific benchmarks
904
+
905
+ ===== XML benchmarks
191
906
 
192
- Run benchmarks for specific XML libraries only:
907
+ Run all XML library benchmarks:
193
908
 
194
- [source,shell]
909
+ [source]
910
+ ----
911
+ $ serialbench benchmark --formats xml
912
+ ----
913
+
914
+ Test specific XML libraries:
915
+
916
+ [source]
195
917
  ----
196
918
  $ serialbench benchmark --formats xml --parsers ox,nokogiri
197
- $ serialbench benchmark --formats xml --parsers rexml,oga
919
+ $ serialbench benchmark --formats xml --parsers rexml,oga,libxml
920
+ ----
921
+
922
+ XML-only parsing performance:
923
+
924
+ [source]
925
+ ----
926
+ $ serialbench benchmark --formats xml --parsing-only
927
+ ----
928
+
929
+ XML generation benchmarks:
930
+
931
+ [source]
932
+ ----
933
+ $ serialbench benchmark --formats xml --generation-only
934
+ ----
935
+
936
+ XML streaming/SAX parsing:
937
+
938
+ [source]
939
+ ----
940
+ $ serialbench benchmark --formats xml --streaming-only
198
941
  ----
199
942
 
200
- Run memory-intensive benchmarks:
943
+ ===== JSON benchmarks
944
+
945
+ Run all JSON library benchmarks:
201
946
 
202
- [source,shell]
947
+ [source]
203
948
  ----
204
- $ serialbench benchmark --formats xml --memory-profiling
949
+ $ serialbench benchmark --formats json
205
950
  ----
206
951
 
207
- Generate detailed XML processing reports:
952
+ Test specific JSON libraries:
208
953
 
209
- [source,shell]
954
+ [source]
210
955
  ----
211
- $ serialbench benchmark --formats xml --detailed-reports
956
+ $ serialbench benchmark --formats json --parsers oj,json
957
+ $ serialbench benchmark --formats json --parsers yajl,oj
212
958
  ----
213
959
 
214
- === Multi-Ruby Version Comparison
960
+ ===== TOML benchmarks
961
+
962
+ Run all TOML library benchmarks:
963
+
964
+ [source]
965
+ ----
966
+ $ serialbench benchmark --formats toml
967
+ ----
968
+
969
+ Test specific TOML libraries:
970
+
971
+ [source]
972
+ ----
973
+ $ serialbench benchmark --formats toml --parsers tomlib,toml-rb
974
+ ----
975
+
976
+ ==== Cross-format comparisons
977
+
978
+ Compare XML vs JSON performance:
979
+
980
+ [source]
981
+ ----
982
+ $ serialbench benchmark --formats xml json
983
+ ----
984
+
985
+ Compare all supported formats:
986
+
987
+ [source]
988
+ ----
989
+ $ serialbench benchmark --formats xml json toml
990
+ ----
991
+
992
+ ==== Advanced options
993
+
994
+ Memory profiling across formats:
995
+
996
+ [source]
997
+ ----
998
+ $ serialbench benchmark --memory-profiling
999
+ ----
1000
+
1001
+ Generate detailed reports:
1002
+
1003
+ [source]
1004
+ ----
1005
+ $ serialbench benchmark --detailed-reports
1006
+ ----
1007
+
1008
+ Output results in JSON format:
1009
+
1010
+ [source]
1011
+ ----
1012
+ $ serialbench benchmark --output-format json
1013
+ ----
1014
+
1015
+ Custom data sizes and iterations:
1016
+
1017
+ [source]
1018
+ ----
1019
+ $ serialbench benchmark --data-sizes small,medium --iterations 100
1020
+ ----
1021
+
1022
+ === Multi-Ruby version comparison
215
1023
 
216
1024
  Merge benchmark results from multiple Ruby versions:
217
1025
 
218
- [source,shell]
1026
+ [source]
219
1027
  ----
220
1028
  $ serialbench merge_results ruby-3.0/results ruby-3.1/results ruby-3.2/results merged_output/
221
1029
  ----
222
1030
 
223
1031
  Generate GitHub Pages HTML from multiple benchmark runs:
224
1032
 
225
- [source,shell]
1033
+ [source]
226
1034
  ----
227
1035
  $ serialbench github_pages ruby-3.0/results ruby-3.1/results ruby-3.2/results docs/
228
1036
  ----
@@ -234,6 +1042,70 @@ This creates an interactive HTML report with:
234
1042
  * **Environment details**: Ruby versions, platforms, and serializer versions
235
1043
  * **GitHub Pages ready**: Deploy directly to GitHub Pages for public sharing
236
1044
 
1045
+ === Cross-platform performance analysis
1046
+
1047
+ Analyze performance data from multiple benchmark runs across different platforms and Ruby versions:
1048
+
1049
+ [source]
1050
+ ----
1051
+ $ serialbench analyze_performance artifacts/benchmark-results-*/ performance_analysis.json
1052
+ ----
1053
+
1054
+ This command:
1055
+
1056
+ * **Processes multiple result directories** from different platforms and Ruby versions
1057
+ * **Extracts platform and Ruby version** information from directory names
1058
+ * **Generates comprehensive JSON** with detailed performance metrics
1059
+ * **Handles both parsing and generation** benchmark results
1060
+ * **Provides summary statistics** about processed data
1061
+
1062
+ Generate platform comparison reports:
1063
+
1064
+ [source]
1065
+ ----
1066
+ $ serialbench platform_comparison performance_analysis.json platform_comparison.json
1067
+ ----
1068
+
1069
+ This creates a JSON report with:
1070
+
1071
+ * **Cross-platform statistics**: Average, min, max performance by platform
1072
+ * **Format-specific analysis**: Performance breakdown by serialization format
1073
+ * **Operation comparison**: Separate analysis for parsing vs generation
1074
+ * **Sample counts**: Number of data points for statistical confidence
1075
+ * **Standard deviation**: Statistical variance for performance consistency
1076
+
1077
+ ==== Example workflow for cross-platform analysis
1078
+
1079
+ [source]
1080
+ ----
1081
+ # 1. Run benchmarks on different platforms (or use CI artifacts)
1082
+ $ serialbench benchmark --formats xml json yaml toml
1083
+
1084
+ # 2. Collect results from multiple platforms/Ruby versions
1085
+ $ mkdir analysis
1086
+ $ cp -r platform1-ruby3.2/results analysis/benchmark-results-ubuntu-ruby-3.2
1087
+ $ cp -r platform2-ruby3.3/results analysis/benchmark-results-macos-ruby-3.3
1088
+ $ cp -r platform3-ruby3.4/results analysis/benchmark-results-windows-ruby-3.4
1089
+
1090
+ # 3. Generate performance analysis
1091
+ $ serialbench analyze_performance analysis/benchmark-results-*/ performance_analysis.json
1092
+
1093
+ # 4. Create platform comparison report
1094
+ $ serialbench platform_comparison performance_analysis.json platform_comparison.json
1095
+
1096
+ # 5. View results
1097
+ $ cat performance_analysis.json | jq '.summary'
1098
+ $ cat performance_analysis.json | jq '.platforms'
1099
+ $ cat platform_comparison.json | jq '.platforms'
1100
+ ----
1101
+
1102
+ The analysis commands are particularly useful for:
1103
+
1104
+ * **CI/CD integration**: Automated cross-platform performance tracking
1105
+ * **Performance regression detection**: Compare results across builds
1106
+ * **Platform optimization**: Identify platform-specific performance characteristics
1107
+ * **Ruby version migration**: Analyze performance impact of Ruby upgrades
1108
+
237
1109
  === Programmatic usage
238
1110
 
239
1111
  ==== Basic benchmark execution
@@ -341,11 +1213,14 @@ Tests how quickly libraries can convert Ruby objects into serialized strings.
341
1213
 
342
1214
  === Streaming performance
343
1215
 
344
- Evaluates streaming event-based parsing performance for libraries that support it, which processes data sequentially and is memory-efficient for large files.
1216
+ Evaluates streaming event-based parsing performance for libraries that support
1217
+ it, which processes data sequentially and is memory-efficient for large files.
345
1218
 
346
1219
  === Memory usage analysis
347
1220
 
348
- Profiles memory allocation and retention during serialization operations using the `memory_profiler` gem.
1221
+ Profiles memory allocation and retention during serialization operations using
1222
+ the `memory_profiler` gem.
1223
+
349
1224
 
350
1225
  == Output and reports
351
1226
 
@@ -381,17 +1256,16 @@ results/
381
1256
  * **Performance rankings**: Fastest to slowest for each category
382
1257
  * **Memory profiling**: Detailed memory allocation analysis
383
1258
  * **Feature matrix**: Capability comparison across libraries
384
- * **Recommendations**: Use-case specific library suggestions
385
1259
  * **Environment details**: Ruby version, platform, and library versions
386
1260
 
387
1261
  === Sample output
388
1262
 
389
- [source]
390
- ----
1263
+ [example]
1264
+ ====
391
1265
  Serialbench - Comprehensive Serialization Performance Tests
392
1266
  ===========================================================
393
1267
  Environment: Ruby 3.3.2 on arm64-darwin23
394
- Timestamp: 2024-01-15T10:30:00Z
1268
+ Timestamp: 2025-06-07T10:30:00Z
395
1269
 
396
1270
  Available serializers: rexml, json, oj, toml-rb
397
1271
  Test formats: xml, json, toml
@@ -409,7 +1283,7 @@ Parsing Performance:
409
1283
  JSON/json: 12.67ms
410
1284
  XML/rexml: 28.45ms
411
1285
  TOML/toml-rb: 35.21ms
412
- ----
1286
+ ====
413
1287
 
414
1288
  == Methodology
415
1289
 
@@ -443,57 +1317,11 @@ The benchmark suite uses carefully crafted synthetic data that represents common
443
1317
  * Memory measurements account for garbage collection
444
1318
  * Results include both absolute and relative performance metrics
445
1319
 
446
- == Library comparison matrix
447
-
448
- [cols="1,1,1,1,1,1,1"]
449
- |===
450
- |Format |Library |Parsing |Generation |Streaming |Memory |Features
451
-
452
- |XML |REXML |⭐⭐ |⭐⭐ |⭐⭐⭐ |⭐⭐ |Built-in
453
- |XML |Ox |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |High-performance
454
- |XML |Nokogiri |⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐ |Feature-rich
455
- |XML |LibXML |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |High-performance
456
- |XML |Oga |⭐⭐ |⭐⭐ |⭐⭐⭐ |⭐⭐ |Pure Ruby
457
- |JSON |JSON |⭐⭐⭐ |⭐⭐⭐ |❌ |⭐⭐⭐ |Built-in
458
- |JSON |Oj |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |High-performance
459
- |JSON |YAJL |⭐⭐⭐⭐ |⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐ |Streaming
460
- |TOML |TOML-RB |⭐⭐⭐ |⭐⭐⭐ |❌ |⭐⭐⭐ |Standard
461
- |TOML |Tomlib |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐ |❌ |⭐⭐⭐⭐⭐ |High-performance
462
- |===
463
-
464
- _Performance ratings: ⭐⭐⭐⭐⭐ Excellent, ⭐⭐⭐⭐ Good, ⭐⭐⭐ Average, ⭐⭐ Below average, ⭐ Poor, ❌ Not supported_
465
-
466
- == Recommendations
467
-
468
- === For high-performance JSON applications
469
-
470
- **Oj** is recommended for applications where JSON parsing/generation speed is critical. It consistently outperforms the built-in JSON library.
471
-
472
- === For configuration files
473
-
474
- **TOML** provides human-readable configuration with good parsing performance. **JSON** is faster but less readable for configuration.
475
-
476
- === For data interchange
477
-
478
- **JSON** offers the best balance of performance, compatibility, and tooling support across different systems.
479
-
480
- === For document processing
481
-
482
- **XML** with **REXML** provides built-in support, though performance is lower than JSON alternatives.
483
-
484
- === For memory-constrained environments
485
-
486
- **Oj** demonstrates superior memory efficiency. For large file processing, streaming approaches are recommended where available.
487
-
488
- === For minimal dependencies
489
-
490
- **JSON** and **REXML** are included with Ruby and require no additional gems, making them suitable for environments with strict dependency constraints.
491
-
492
1320
  == Development
493
1321
 
494
1322
  === Running tests
495
1323
 
496
- [source,shell]
1324
+ [source]
497
1325
  ----
498
1326
  $ bundle exec rake
499
1327
  $ bundle exec rspec
@@ -501,21 +1329,21 @@ $ bundle exec rspec
501
1329
 
502
1330
  === Contributing
503
1331
 
504
- 1. Fork the repository
505
- 2. Create your feature branch (`git checkout -b feature/my-new-feature`)
506
- 3. Commit your changes (`git commit -am 'Add some feature'`)
507
- 4. Push to the branch (`git push origin feature/my-new-feature`)
508
- 5. Create a new Pull Request
1332
+ . Fork the repository
1333
+ . Create your feature branch (`git checkout -b feature/my-new-feature`)
1334
+ . Commit your changes (`git commit -am 'Add some feature'`)
1335
+ . Push to the branch (`git push origin feature/my-new-feature`)
1336
+ . Create a new Pull Request
509
1337
 
510
1338
  === Adding new serializers
511
1339
 
512
1340
  To add support for additional serialization libraries:
513
1341
 
514
- 1. Create a new serializer class in `lib/serialbench/serializers/{format}/`
515
- 2. Inherit from the appropriate base class (`BaseXmlSerializer`, `BaseJsonSerializer`, etc.)
516
- 3. Implement the required methods: `parse`, `generate`, `name`, `version`
517
- 4. Add the serializer to the registry in `lib/serialbench/serializers.rb`
518
- 5. Update documentation and tests
1342
+ . Create a new serializer class in `lib/serialbench/serializers/{format}/`
1343
+ . Inherit from the appropriate base class (`BaseXmlSerializer`, `BaseJsonSerializer`, etc.)
1344
+ . Implement the required methods: `parse`, `generate`, `name`, `version`
1345
+ . Add the serializer to the registry in `lib/serialbench/serializers.rb`
1346
+ . Update documentation and tests
519
1347
 
520
1348
  ==== Example: Adding a new JSON serializer
521
1349