serialbench 0.1.0 → 0.1.1
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/benchmark.yml +173 -30
- data/.github/workflows/ci.yml +3 -3
- data/.github/workflows/docker.yml +246 -0
- data/.github/workflows/release.yml +25 -0
- data/Gemfile +5 -30
- data/README.adoc +962 -134
- data/config/ci.yml +22 -0
- data/config/full.yml +30 -0
- data/docker/Dockerfile.benchmark +31 -0
- data/docker/README.md +214 -0
- data/docker/run-benchmarks.sh +356 -0
- data/lib/serialbench/benchmark_runner.rb +82 -0
- data/lib/serialbench/cli.rb +201 -9
- data/lib/serialbench/result_merger.rb +5 -5
- data/lib/serialbench/serializers/json/rapidjson_serializer.rb +50 -0
- data/lib/serialbench/serializers/json/yajl_serializer.rb +6 -2
- data/lib/serialbench/serializers/xml/nokogiri_serializer.rb +21 -3
- data/lib/serialbench/serializers/xml/rexml_serializer.rb +32 -2
- data/lib/serialbench/serializers/yaml/base_yaml_serializer.rb +55 -0
- data/lib/serialbench/serializers/yaml/psych_serializer.rb +54 -0
- data/lib/serialbench/serializers/yaml/syck_serializer.rb +65 -0
- data/lib/serialbench/serializers.rb +11 -0
- data/lib/serialbench/version.rb +1 -1
- data/serialbench.gemspec +25 -17
- metadata +113 -46
data/README.adoc
CHANGED
@@ -1,31 +1,282 @@
|
|
1
|
-
= Serialbench:
|
1
|
+
= Serialbench: Ruby serialization library performance benchmarker
|
2
2
|
|
3
3
|
image:https://img.shields.io/gem/v/serialbench.svg["Gem Version", link="https://rubygems.org/gems/serialbench"]
|
4
|
-
image:https://github.com/
|
5
|
-
image:https://
|
6
|
-
image:https://img.shields.io/github/
|
4
|
+
image:https://github.com/metanorma/serialbench/actions/workflows/ci.yml/badge.svg["Build Status", link="https://github.com/metanorma/serialbench/actions/workflows/ci.yml"]
|
5
|
+
image:https://github.com/metanorma/serialbench/actions/workflows/benchmark.yml/badge.svg["Benchmark Status", link="https://github.com/metanorma/serialbench/actions/workflows/benchmark.yml"]
|
6
|
+
image:https://img.shields.io/github/issues-pr-raw/metanorma/serialbench.svg["Pull Requests", link="https://github.com/metanorma/serialbench/pulls"]
|
7
7
|
|
8
|
-
==
|
8
|
+
== Overview
|
9
9
|
|
10
|
-
Serialbench is a comprehensive benchmarking suite that evaluates the performance of popular Ruby serialization libraries across multiple formats and
|
10
|
+
Serialbench is a comprehensive benchmarking suite that evaluates the performance of popular Ruby serialization libraries across multiple formats. It provides detailed performance comparisons and analysis to help developers make informed decisions when choosing serialization libraries for their Ruby applications.
|
11
11
|
|
12
|
-
|
12
|
+
**Supported Formats**: XML, JSON, YAML, TOML, and more
|
13
13
|
|
14
|
-
|
14
|
+
**Key Metrics**: Parsing speed, generation speed, memory usage, streaming capabilities, and feature completeness
|
15
15
|
|
16
|
-
|
17
|
-
* **Ox** - High-performance XML parser optimized for speed and low memory usage
|
18
|
-
* **Nokogiri** - Feature-rich XML/HTML parser with XPath support and comprehensive DOM manipulation
|
19
|
-
* **LibXML** - Ruby bindings for the libxml2 C library with excellent performance characteristics
|
20
|
-
* **Oga** - Pure Ruby XML parser with XPath support and streaming capabilities
|
21
|
-
* **REXML** - Ruby's built-in XML parser with streaming support (reference implementation)
|
16
|
+
**Docker Support**: Multi-Ruby version benchmarking with automated result aggregation and GitHub Pages generation
|
22
17
|
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
18
|
+
== Supported serialization libraries
|
19
|
+
|
20
|
+
[cols="1,3,1,4", options="header"]
|
21
|
+
|===
|
22
|
+
| Format | Name | Version | Description
|
23
|
+
|
24
|
+
| XML
|
25
|
+
| https://github.com/ohler55/ox[Ox]
|
26
|
+
| v2.14.23
|
27
|
+
| C extension XML parser
|
28
|
+
|
29
|
+
| XML
|
30
|
+
| https://github.com/xml4r/libxml-ruby[LibXML]
|
31
|
+
| v4.1.2
|
32
|
+
| Ruby bindings for libxml2
|
33
|
+
|
34
|
+
| XML
|
35
|
+
| https://github.com/sparklemotion/nokogiri[Nokogiri]
|
36
|
+
| v1.18.8
|
37
|
+
| XML/HTML parser with XPath and CSS selectors
|
38
|
+
|
39
|
+
| XML
|
40
|
+
| https://github.com/YorickPeterse/oga[Oga]
|
41
|
+
| v3.4
|
42
|
+
| Pure Ruby XML parser with XPath support
|
43
|
+
|
44
|
+
| XML
|
45
|
+
| https://github.com/ruby/rexml[REXML]
|
46
|
+
| v3.4.1
|
47
|
+
| Ruby's standard library XML parser
|
48
|
+
|
49
|
+
| JSON
|
50
|
+
| https://github.com/ohler55/oj[Oj]
|
51
|
+
| v3.16.11
|
52
|
+
| JSON parser with multiple parsing modes
|
53
|
+
|
54
|
+
| JSON
|
55
|
+
| https://github.com/brianmario/yajl-ruby[YAJL]
|
56
|
+
| v1.4.3
|
57
|
+
| JSON library with streaming capabilities
|
58
|
+
|
59
|
+
| JSON
|
60
|
+
| https://github.com/flori/json[JSON]
|
61
|
+
| v2.12.2
|
62
|
+
| Ruby's standard library JSON parser
|
63
|
+
|
64
|
+
| YAML
|
65
|
+
| https://github.com/ruby/psych[Psych]
|
66
|
+
| v5.1.2
|
67
|
+
| Ruby's standard library YAML parser
|
68
|
+
|
69
|
+
| YAML
|
70
|
+
| https://github.com/ruby/syck[Syck]
|
71
|
+
| v1.5.1.1
|
72
|
+
| Legacy YAML parser
|
73
|
+
|
74
|
+
| TOML
|
75
|
+
| https://github.com/fbernier/tomlib[Tomlib]
|
76
|
+
| v0.7.3
|
77
|
+
| TOML parser implemented in C
|
78
|
+
|
79
|
+
| TOML
|
80
|
+
| https://github.com/emancu/toml-rb[TOML-RB]
|
81
|
+
| v2.2.0
|
82
|
+
| Pure Ruby TOML parser
|
83
|
+
|===
|
84
|
+
|
85
|
+
== Benchmark report JSON format
|
86
|
+
|
87
|
+
=== Single Ruby version results
|
88
|
+
|
89
|
+
The benchmark results are saved as JSON files with the following structure:
|
90
|
+
|
91
|
+
[source,json]
|
92
|
+
----
|
93
|
+
{
|
94
|
+
"environment": {
|
95
|
+
"ruby_version": "3.3.8",
|
96
|
+
"ruby_platform": "aarch64-linux",
|
97
|
+
"serializer_versions": {
|
98
|
+
"rexml": "3.4.1",
|
99
|
+
"ox": "2.14.23",
|
100
|
+
"nokogiri": "1.18.8",
|
101
|
+
"oga": "3.4",
|
102
|
+
"libxml": "4.1.2",
|
103
|
+
"json": "2.12.2",
|
104
|
+
"oj": "3.16.11",
|
105
|
+
"yajl": "1.4.3",
|
106
|
+
"psych": "5.1.2",
|
107
|
+
"syck": "1.5.1.1",
|
108
|
+
"toml-rb": "2.2.0",
|
109
|
+
"tomlib": "0.7.3"
|
110
|
+
},
|
111
|
+
"timestamp": "2025-06-07T09:00:25+00:00"
|
112
|
+
},
|
113
|
+
"parsing": {
|
114
|
+
"small": {
|
115
|
+
"xml": {
|
116
|
+
"rexml": {
|
117
|
+
"time_per_iterations": 0.002514416000053643,
|
118
|
+
"time_per_iteration": 0.00012572080000268216,
|
119
|
+
"iterations_per_second": 7954.133285650949,
|
120
|
+
"iterations_count": 20
|
121
|
+
},
|
122
|
+
"ox": {
|
123
|
+
"time_per_iterations": 0.00005258399994545471,
|
124
|
+
"time_per_iteration": 0.0000026291999972727353,
|
125
|
+
"iterations_per_second": 380343.83121759404,
|
126
|
+
"iterations_count": 20
|
127
|
+
}
|
128
|
+
},
|
129
|
+
"json": {
|
130
|
+
"json": {
|
131
|
+
"time_per_iterations": 0.000029707999942729657,
|
132
|
+
"time_per_iteration": 0.000001485399997136483,
|
133
|
+
"iterations_per_second": 673219.3361571126,
|
134
|
+
"iterations_count": 20
|
135
|
+
},
|
136
|
+
"oj": {
|
137
|
+
"time_per_iterations": 0.00003158300000905001,
|
138
|
+
"time_per_iteration": 0.0000015791500004525006,
|
139
|
+
"iterations_per_second": 633252.0658034089,
|
140
|
+
"iterations_count": 20
|
141
|
+
}
|
142
|
+
}
|
143
|
+
},
|
144
|
+
"medium": {
|
145
|
+
"xml": { /* ... */ },
|
146
|
+
"json": { /* ... */ }
|
147
|
+
},
|
148
|
+
"large": {
|
149
|
+
"xml": { /* ... */ },
|
150
|
+
"json": { /* ... */ }
|
151
|
+
}
|
152
|
+
},
|
153
|
+
"ruby_version": "3.3.8",
|
154
|
+
"ruby_platform": "aarch64-linux",
|
155
|
+
"timestamp": "2025-06-07T09:01:02+00:00"
|
156
|
+
}
|
157
|
+
----
|
158
|
+
|
159
|
+
=== Multi-Ruby version merged results
|
160
|
+
|
161
|
+
When results from multiple Ruby versions are merged, the structure becomes:
|
162
|
+
|
163
|
+
[source,json]
|
164
|
+
----
|
165
|
+
{
|
166
|
+
"environments": {
|
167
|
+
"3_3_8_aarch64_linux": {
|
168
|
+
"ruby_version": "3.3.8",
|
169
|
+
"ruby_platform": "aarch64-linux",
|
170
|
+
"source_file": "docker-results/ruby-3.3/data/results.json",
|
171
|
+
"timestamp": "2025-06-07T09:01:02+00:00",
|
172
|
+
"environment": {
|
173
|
+
"ruby_version": "3.3.8",
|
174
|
+
"ruby_platform": "aarch64-linux",
|
175
|
+
"serializer_versions": { /* ... */ },
|
176
|
+
"timestamp": "2025-06-07T09:00:25+00:00"
|
177
|
+
}
|
178
|
+
},
|
179
|
+
"3_4_4_aarch64_linux": {
|
180
|
+
"ruby_version": "3.4.4",
|
181
|
+
"ruby_platform": "aarch64-linux",
|
182
|
+
/* ... */
|
183
|
+
}
|
184
|
+
},
|
185
|
+
"combined_results": {
|
186
|
+
"parsing": {
|
187
|
+
"small": {
|
188
|
+
"xml": {
|
189
|
+
"rexml": {
|
190
|
+
"3_3_8_aarch64_linux": {
|
191
|
+
"time_per_iterations": 0.002514416000053643,
|
192
|
+
"time_per_iteration": 0.00012572080000268216,
|
193
|
+
"iterations_per_second": 7954.133285650949,
|
194
|
+
"iterations_count": 20
|
195
|
+
},
|
196
|
+
"3_4_4_aarch64_linux": {
|
197
|
+
"time_per_iterations": 0.0025308329999234047,
|
198
|
+
"time_per_iteration": 0.00012654164999617023,
|
199
|
+
"iterations_per_second": 7902.536437846866,
|
200
|
+
"iterations_count": 20
|
201
|
+
}
|
202
|
+
}
|
203
|
+
}
|
204
|
+
}
|
205
|
+
}
|
206
|
+
},
|
207
|
+
"metadata": {
|
208
|
+
"merged_at": "2025-06-07T17:01:48+08:00",
|
209
|
+
"ruby_versions": ["3.3.8", "3.4.4"],
|
210
|
+
"platforms": ["aarch64-linux"]
|
211
|
+
}
|
212
|
+
}
|
213
|
+
----
|
214
|
+
|
215
|
+
=== JSON schema specification
|
216
|
+
|
217
|
+
The benchmark results follow this schema:
|
218
|
+
|
219
|
+
[source,json]
|
220
|
+
----
|
221
|
+
{
|
222
|
+
"$schema": "http://json-schema.org/draft-07/schema#",
|
223
|
+
"title": "Serialbench Results",
|
224
|
+
"type": "object",
|
225
|
+
"properties": {
|
226
|
+
"environment": {
|
227
|
+
"type": "object",
|
228
|
+
"properties": {
|
229
|
+
"ruby_version": { "type": "string" },
|
230
|
+
"ruby_platform": { "type": "string" },
|
231
|
+
"serializer_versions": {
|
232
|
+
"type": "object",
|
233
|
+
"additionalProperties": { "type": "string" }
|
234
|
+
},
|
235
|
+
"timestamp": { "type": "string", "format": "date-time" }
|
236
|
+
},
|
237
|
+
"required": ["ruby_version", "ruby_platform", "serializer_versions", "timestamp"]
|
238
|
+
},
|
239
|
+
"parsing": {
|
240
|
+
"type": "object",
|
241
|
+
"properties": {
|
242
|
+
"small": { "$ref": "#/definitions/sizeResults" },
|
243
|
+
"medium": { "$ref": "#/definitions/sizeResults" },
|
244
|
+
"large": { "$ref": "#/definitions/sizeResults" }
|
245
|
+
}
|
246
|
+
},
|
247
|
+
"ruby_version": { "type": "string" },
|
248
|
+
"ruby_platform": { "type": "string" },
|
249
|
+
"timestamp": { "type": "string", "format": "date-time" }
|
250
|
+
},
|
251
|
+
"definitions": {
|
252
|
+
"sizeResults": {
|
253
|
+
"type": "object",
|
254
|
+
"properties": {
|
255
|
+
"xml": { "$ref": "#/definitions/formatResults" },
|
256
|
+
"json": { "$ref": "#/definitions/formatResults" },
|
257
|
+
"yaml": { "$ref": "#/definitions/formatResults" },
|
258
|
+
"toml": { "$ref": "#/definitions/formatResults" }
|
259
|
+
}
|
260
|
+
},
|
261
|
+
"formatResults": {
|
262
|
+
"type": "object",
|
263
|
+
"additionalProperties": {
|
264
|
+
"$ref": "#/definitions/serializerResults"
|
265
|
+
}
|
266
|
+
},
|
267
|
+
"serializerResults": {
|
268
|
+
"type": "object",
|
269
|
+
"properties": {
|
270
|
+
"time_per_iterations": { "type": "number" },
|
271
|
+
"time_per_iteration": { "type": "number" },
|
272
|
+
"iterations_per_second": { "type": "number" },
|
273
|
+
"iterations_count": { "type": "integer" }
|
274
|
+
},
|
275
|
+
"required": ["time_per_iterations", "time_per_iteration", "iterations_per_second", "iterations_count"]
|
276
|
+
}
|
277
|
+
}
|
278
|
+
}
|
279
|
+
----
|
29
280
|
|
30
281
|
== Installation
|
31
282
|
|
@@ -38,14 +289,14 @@ gem 'serialbench'
|
|
38
289
|
|
39
290
|
And then execute:
|
40
291
|
|
41
|
-
[source
|
292
|
+
[source]
|
42
293
|
----
|
43
294
|
$ bundle install
|
44
295
|
----
|
45
296
|
|
46
297
|
Or install it yourself as:
|
47
298
|
|
48
|
-
[source
|
299
|
+
[source]
|
49
300
|
----
|
50
301
|
$ gem install serialbench
|
51
302
|
----
|
@@ -54,7 +305,8 @@ $ gem install serialbench
|
|
54
305
|
|
55
306
|
To run benchmarks for all supported XML libraries, install the following gems:
|
56
307
|
|
57
|
-
|
308
|
+
|
309
|
+
[source]
|
58
310
|
----
|
59
311
|
# Core XML libraries
|
60
312
|
$ gem install ox nokogiri libxml-ruby oga
|
@@ -66,20 +318,25 @@ $ gem install oj toml-rb
|
|
66
318
|
$ gem install memory_profiler
|
67
319
|
----
|
68
320
|
|
69
|
-
|
321
|
+
|
322
|
+
NOTE: REXML, JSON, and Psych (YAML) are included with Ruby and require no additional installation.
|
70
323
|
|
71
324
|
=== Library-specific installation notes
|
72
325
|
|
73
326
|
==== Ox
|
327
|
+
|
74
328
|
High-performance C extension requiring compilation:
|
75
|
-
|
329
|
+
|
330
|
+
[source]
|
76
331
|
----
|
77
332
|
$ gem install ox
|
78
333
|
----
|
79
334
|
|
80
335
|
==== Nokogiri
|
336
|
+
|
81
337
|
May require system dependencies on some platforms:
|
82
|
-
|
338
|
+
|
339
|
+
[source]
|
83
340
|
----
|
84
341
|
# macOS with Homebrew
|
85
342
|
$ brew install libxml2 libxslt
|
@@ -91,8 +348,10 @@ $ gem install nokogiri
|
|
91
348
|
----
|
92
349
|
|
93
350
|
==== LibXML
|
351
|
+
|
94
352
|
Ruby bindings for libxml2:
|
95
|
-
|
353
|
+
|
354
|
+
[source]
|
96
355
|
----
|
97
356
|
# macOS with Homebrew
|
98
357
|
$ brew install libxml2
|
@@ -104,77 +363,531 @@ $ gem install libxml-ruby
|
|
104
363
|
----
|
105
364
|
|
106
365
|
==== Oga
|
366
|
+
|
107
367
|
Pure Ruby implementation with no system dependencies:
|
108
|
-
|
368
|
+
|
369
|
+
[source]
|
109
370
|
----
|
110
371
|
$ gem install oga
|
111
372
|
----
|
112
373
|
|
113
|
-
==
|
374
|
+
== Docker containers and cross-platform benchmarking
|
114
375
|
|
115
|
-
|
376
|
+
Serialbench provides pre-built Docker containers and supports cross-platform benchmarking on Windows, macOS, and Linux. Our containers are automatically built and published to GitHub Container Registry for easy access.
|
377
|
+
|
378
|
+
=== Published Docker containers
|
379
|
+
|
380
|
+
We publish multi-architecture Docker containers for all supported Ruby versions:
|
381
|
+
|
382
|
+
**Registry**: `ghcr.io/metanorma/serialbench`
|
383
|
+
|
384
|
+
**Available tags**:
|
385
|
+
* `ghcr.io/metanorma/serialbench:main-ruby-3.1` - Ruby 3.1 (linux/amd64, linux/arm64)
|
386
|
+
* `ghcr.io/metanorma/serialbench:main-ruby-3.2` - Ruby 3.2 (linux/amd64, linux/arm64)
|
387
|
+
* `ghcr.io/metanorma/serialbench:main-ruby-3.3` - Ruby 3.3 (linux/amd64, linux/arm64)
|
388
|
+
* `ghcr.io/metanorma/serialbench:main-ruby-3.4` - Ruby 3.4 (linux/amd64, linux/arm64)
|
389
|
+
|
390
|
+
**Container features**:
|
391
|
+
* All serialization libraries pre-installed (XML, JSON, YAML, TOML)
|
392
|
+
* Multi-architecture support (Intel x64 and ARM64)
|
393
|
+
* Optimized for CI/CD and local development
|
394
|
+
* Automatic updates on code changes
|
395
|
+
|
396
|
+
=== Running benchmarks on your computer
|
116
397
|
|
117
|
-
|
398
|
+
==== Windows (PowerShell/Command Prompt)
|
118
399
|
|
119
|
-
[source,
|
400
|
+
[source,powershell]
|
120
401
|
----
|
121
|
-
|
402
|
+
# Pull and run latest Ruby 3.3 container
|
403
|
+
docker pull ghcr.io/metanorma/serialbench:main-ruby-3.3
|
404
|
+
|
405
|
+
# Create results directory
|
406
|
+
mkdir results
|
407
|
+
|
408
|
+
# Run benchmarks with volume mounting
|
409
|
+
docker run --rm -v ${PWD}/results:/app/results ghcr.io/metanorma/serialbench:main-ruby-3.3
|
410
|
+
|
411
|
+
# View results
|
412
|
+
dir results
|
122
413
|
----
|
123
414
|
|
124
|
-
|
415
|
+
==== macOS (Terminal)
|
125
416
|
|
126
|
-
[source,
|
417
|
+
[source,bash]
|
127
418
|
----
|
128
|
-
|
419
|
+
# Pull and run latest Ruby 3.3 container
|
420
|
+
docker pull ghcr.io/metanorma/serialbench:main-ruby-3.3
|
421
|
+
|
422
|
+
# Create results directory
|
423
|
+
mkdir -p results
|
424
|
+
|
425
|
+
# Run benchmarks with volume mounting
|
426
|
+
docker run --rm -v $(pwd)/results:/app/results ghcr.io/metanorma/serialbench:main-ruby-3.3
|
427
|
+
|
428
|
+
# View results
|
429
|
+
ls -la results/
|
129
430
|
----
|
130
431
|
|
131
|
-
|
432
|
+
==== Ubuntu/Linux (Terminal)
|
132
433
|
|
133
|
-
[source,
|
434
|
+
[source,bash]
|
134
435
|
----
|
135
|
-
|
136
|
-
|
436
|
+
# Pull and run latest Ruby 3.3 container
|
437
|
+
docker pull ghcr.io/metanorma/serialbench:main-ruby-3.3
|
438
|
+
|
439
|
+
# Create results directory
|
440
|
+
mkdir -p results
|
441
|
+
|
442
|
+
# Run benchmarks with volume mounting
|
443
|
+
docker run --rm -v $(pwd)/results:/app/results ghcr.io/metanorma/serialbench:main-ruby-3.3
|
444
|
+
|
445
|
+
# View results
|
446
|
+
ls -la results/
|
137
447
|
----
|
138
448
|
|
139
|
-
|
449
|
+
=== Multi-Ruby version comparison
|
450
|
+
|
451
|
+
Run benchmarks across all supported Ruby versions:
|
452
|
+
|
453
|
+
==== Windows (PowerShell)
|
140
454
|
|
141
|
-
[source,
|
455
|
+
[source,powershell]
|
142
456
|
----
|
143
|
-
|
457
|
+
# Create results directories
|
458
|
+
$versions = @("3.1", "3.2", "3.3", "3.4")
|
459
|
+
foreach ($version in $versions) {
|
460
|
+
mkdir "results-ruby-$version" -Force
|
461
|
+
docker pull "ghcr.io/metanorma/serialbench:main-ruby-$version"
|
462
|
+
docker run --rm -v "${PWD}/results-ruby-${version}:/app/results" "ghcr.io/metanorma/serialbench:main-ruby-$version"
|
463
|
+
}
|
464
|
+
|
465
|
+
# View all results
|
466
|
+
dir results-ruby-*
|
144
467
|
----
|
145
468
|
|
146
|
-
|
469
|
+
==== macOS/Linux (Bash)
|
147
470
|
|
148
|
-
[source,
|
471
|
+
[source,bash]
|
149
472
|
----
|
150
|
-
|
473
|
+
# Run benchmarks for all Ruby versions
|
474
|
+
for version in 3.1 3.2 3.3 3.4; do
|
475
|
+
echo "Running benchmarks for Ruby $version..."
|
476
|
+
mkdir -p "results-ruby-$version"
|
477
|
+
docker pull "ghcr.io/metanorma/serialbench:main-ruby-$version"
|
478
|
+
docker run --rm \
|
479
|
+
-v "$(pwd)/results-ruby-$version:/app/results" \
|
480
|
+
"ghcr.io/metanorma/serialbench:main-ruby-$version"
|
481
|
+
done
|
482
|
+
|
483
|
+
# View all results
|
484
|
+
ls -la results-ruby-*/
|
151
485
|
----
|
152
486
|
|
153
|
-
|
487
|
+
=== Custom benchmark configuration
|
488
|
+
|
489
|
+
Run benchmarks with custom parameters:
|
154
490
|
|
155
|
-
[source,
|
491
|
+
[source,bash]
|
156
492
|
----
|
157
|
-
|
493
|
+
# Run specific formats only
|
494
|
+
docker run --rm \
|
495
|
+
-v $(pwd)/results:/app/results \
|
496
|
+
ghcr.io/metanorma/serialbench:main-ruby-3.3 \
|
497
|
+
bundle exec serialbench benchmark --formats xml json --iterations 10
|
498
|
+
|
499
|
+
# Run with memory profiling
|
500
|
+
docker run --rm \
|
501
|
+
-v $(pwd)/results:/app/results \
|
502
|
+
ghcr.io/metanorma/serialbench:main-ruby-3.3 \
|
503
|
+
bundle exec serialbench benchmark --memory-profiling
|
504
|
+
|
505
|
+
# List available serializers
|
506
|
+
docker run --rm ghcr.io/metanorma/serialbench:main-ruby-3.3 \
|
507
|
+
bundle exec serialbench list
|
158
508
|
----
|
159
509
|
|
160
|
-
|
510
|
+
=== Quick start with Docker
|
161
511
|
|
162
|
-
|
512
|
+
==== Prerequisites
|
513
|
+
|
514
|
+
* Docker installed and running
|
515
|
+
* Command line access (PowerShell, Terminal, or Bash)
|
516
|
+
|
517
|
+
==== Running multi-Ruby benchmarks
|
518
|
+
|
519
|
+
[source]
|
163
520
|
----
|
164
|
-
|
521
|
+
# From the project root directory (if you have the source)
|
522
|
+
$ ./docker/run-benchmarks.sh
|
523
|
+
|
524
|
+
# Or using published containers directly
|
525
|
+
$ docker pull ghcr.io/metanorma/serialbench:main-ruby-3.3
|
526
|
+
$ docker run --rm -v $(pwd)/results:/app/results ghcr.io/metanorma/serialbench:main-ruby-3.3
|
527
|
+
----
|
528
|
+
|
529
|
+
This will:
|
530
|
+
|
531
|
+
. **Pull pre-built containers** from GitHub Container Registry
|
532
|
+
. **Run comprehensive benchmarks** in isolated environments
|
533
|
+
. **Generate detailed reports** with performance comparisons
|
534
|
+
. **Output results** to your local `results/` directory
|
535
|
+
|
536
|
+
==== Results structure
|
537
|
+
|
538
|
+
Results are organized in `docker-results/`:
|
539
|
+
|
540
|
+
[source]
|
541
|
+
----
|
542
|
+
docker-results/
|
543
|
+
├── ruby-3.0/ # Ruby 3.0 individual results
|
544
|
+
│ ├── benchmark.log # Execution log
|
545
|
+
│ ├── data/
|
546
|
+
│ │ ├── results.json # Raw benchmark data
|
547
|
+
│ │ └── results.yaml # YAML format results
|
548
|
+
│ ├── reports/
|
549
|
+
│ │ └── benchmark_report.html
|
550
|
+
│ └── assets/
|
551
|
+
├── ruby-3.1/ # Ruby 3.1 individual results
|
552
|
+
├── ruby-3.2/ # Ruby 3.2 individual results
|
553
|
+
├── ruby-3.3/ # Ruby 3.3 individual results
|
554
|
+
├── ruby-3.4/ # Ruby 3.4 individual results
|
555
|
+
├── merged/ # Aggregated cross-version results
|
556
|
+
│ └── merged_results.json # Combined performance data
|
557
|
+
└── docs/ # GitHub Pages ready output
|
558
|
+
├── index.html # Interactive comparison report
|
559
|
+
├── styles.css # Report styling
|
560
|
+
└── merged_results.json # Data for interactive charts
|
561
|
+
----
|
562
|
+
|
563
|
+
=== Manual Docker usage
|
564
|
+
|
565
|
+
==== Build image for specific Ruby version
|
566
|
+
|
567
|
+
[source]
|
568
|
+
----
|
569
|
+
$ docker build \
|
570
|
+
--build-arg RUBY_VERSION=3.3 \
|
571
|
+
-t serialbench:ruby-3.3 \
|
572
|
+
-f docker/Dockerfile.benchmark \
|
573
|
+
.
|
574
|
+
----
|
575
|
+
|
576
|
+
==== Run benchmarks in container
|
577
|
+
|
578
|
+
[source]
|
579
|
+
----
|
580
|
+
# Create results directory
|
581
|
+
$ mkdir -p results
|
582
|
+
|
583
|
+
# Run benchmarks with volume mounting
|
584
|
+
$ docker run \
|
585
|
+
--rm \
|
586
|
+
-v $(pwd)/results:/app/results \
|
587
|
+
serialbench:ruby-3.3
|
588
|
+
----
|
589
|
+
|
590
|
+
==== Custom configuration
|
591
|
+
|
592
|
+
[source]
|
593
|
+
----
|
594
|
+
# Use custom config file
|
595
|
+
$ docker run \
|
596
|
+
--rm \
|
597
|
+
-v $(pwd)/results:/app/results \
|
598
|
+
-v $(pwd)/config:/app/config \
|
599
|
+
serialbench:ruby-3.3 \
|
600
|
+
bundle exec serialbench benchmark --config config/ci.yml
|
601
|
+
----
|
602
|
+
|
603
|
+
=== Supported Ruby versions
|
604
|
+
|
605
|
+
The Docker setup supports the following Ruby versions:
|
606
|
+
|
607
|
+
* **Ruby 3.0** - Stable release with good performance baseline
|
608
|
+
* **Ruby 3.1** - Improved performance and new features
|
609
|
+
* **Ruby 3.2** - Enhanced YJIT and memory optimizations
|
610
|
+
* **Ruby 3.3** - Latest stable with performance improvements
|
611
|
+
* **Ruby 3.4** - Current development version
|
612
|
+
|
613
|
+
Each version includes all supported serialization libraries:
|
614
|
+
|
615
|
+
* **XML**: REXML (built-in), Ox, Nokogiri, Oga, LibXML
|
616
|
+
* **JSON**: JSON (built-in), Oj, YAJL
|
617
|
+
* **YAML**: Psych (built-in), Syck
|
618
|
+
* **TOML**: TOML-RB, Tomlib
|
619
|
+
|
620
|
+
=== Environment variables
|
621
|
+
|
622
|
+
The Docker images support these environment variables:
|
623
|
+
|
624
|
+
* `BUNDLE_PATH` - Bundle installation path
|
625
|
+
* `BUNDLE_BIN` - Bundle binary path
|
626
|
+
* `PATH` - System PATH including bundle binaries
|
627
|
+
* `RUBY_VERSION` - Ruby version for build-time configuration
|
628
|
+
|
629
|
+
=== Result aggregation and GitHub Pages
|
630
|
+
|
631
|
+
==== Merging multi-version results
|
632
|
+
|
633
|
+
The Docker workflow automatically merges results from all Ruby versions:
|
634
|
+
|
635
|
+
[source]
|
636
|
+
----
|
637
|
+
# Manual result merging
|
638
|
+
$ serialbench merge_results \
|
639
|
+
docker-results/ruby-3.0 \
|
640
|
+
docker-results/ruby-3.1 \
|
641
|
+
docker-results/ruby-3.2 \
|
642
|
+
docker-results/ruby-3.3 \
|
643
|
+
docker-results/ruby-3.4 \
|
644
|
+
docker-results/merged
|
645
|
+
----
|
646
|
+
|
647
|
+
==== GitHub Pages generation
|
648
|
+
|
649
|
+
Generate interactive HTML reports ready for GitHub Pages deployment:
|
650
|
+
|
651
|
+
[source]
|
652
|
+
----
|
653
|
+
# Generate GitHub Pages from multi-version results
|
654
|
+
$ serialbench github_pages \
|
655
|
+
docker-results/ruby-3.0 \
|
656
|
+
docker-results/ruby-3.1 \
|
657
|
+
docker-results/ruby-3.2 \
|
658
|
+
docker-results/ruby-3.3 \
|
659
|
+
docker-results/ruby-3.4 \
|
660
|
+
docker-results/docs
|
165
661
|
----
|
166
662
|
|
167
|
-
|
663
|
+
The generated GitHub Pages include:
|
168
664
|
|
169
|
-
|
665
|
+
* **Interactive Performance Charts**: Compare serializers across Ruby versions
|
666
|
+
* **Multi-Version Analysis**: See how performance changes between Ruby releases
|
667
|
+
* **Environment Details**: Ruby versions, platforms, and library versions
|
668
|
+
* **Responsive Design**: Works on desktop and mobile devices
|
669
|
+
* **Direct Deployment**: Ready for GitHub Pages, Netlify, or any static hosting
|
670
|
+
|
671
|
+
==== Deploying to GitHub Pages
|
672
|
+
|
673
|
+
. **Commit the generated files**:
|
674
|
+
+
|
675
|
+
[source]
|
676
|
+
----
|
677
|
+
$ git add docker-results/docs/
|
678
|
+
$ git commit -m "Add multi-Ruby benchmark results"
|
679
|
+
$ git push origin main
|
680
|
+
----
|
681
|
+
|
682
|
+
. **Enable GitHub Pages** in repository settings:
|
683
|
+
.. Go to Settings → Pages
|
684
|
+
.. Set source to "Deploy from a branch"
|
685
|
+
.. Select branch containing the `docs/` folder
|
686
|
+
.. Set folder to `/docker-results/docs`
|
687
|
+
|
688
|
+
. **Access your results** at: `https://yourusername.github.io/yourrepo/`
|
689
|
+
|
690
|
+
=== Troubleshooting Docker issues
|
691
|
+
|
692
|
+
==== Build failures
|
693
|
+
|
694
|
+
Check build logs for specific Ruby versions:
|
695
|
+
|
696
|
+
[source]
|
697
|
+
----
|
698
|
+
$ cat docker-results/build-ruby-3.3.log
|
699
|
+
----
|
700
|
+
|
701
|
+
Common build issues:
|
702
|
+
|
703
|
+
* **Missing system dependencies**: Ensure libxml2-dev and libxslt1-dev are available
|
704
|
+
* **Network timeouts**: Retry the build or use a different network
|
705
|
+
* **Disk space**: Ensure sufficient disk space for multiple Ruby images
|
706
|
+
|
707
|
+
==== Runtime failures
|
708
|
+
|
709
|
+
Check benchmark execution logs:
|
710
|
+
|
711
|
+
[source]
|
712
|
+
----
|
713
|
+
$ cat docker-results/ruby-3.3/benchmark.log
|
714
|
+
----
|
715
|
+
|
716
|
+
Common runtime issues:
|
717
|
+
|
718
|
+
* **Memory constraints**: Increase Docker memory allocation
|
719
|
+
* **Timeout issues**: Some benchmarks may take longer on slower systems
|
720
|
+
* **Permission errors**: Ensure proper volume mounting permissions
|
721
|
+
|
722
|
+
==== Docker system issues
|
723
|
+
|
724
|
+
Verify Docker is running properly:
|
725
|
+
|
726
|
+
[source]
|
727
|
+
----
|
728
|
+
$ docker info
|
729
|
+
$ docker system df # Check disk usage
|
730
|
+
$ docker system prune # Clean up unused resources
|
731
|
+
----
|
732
|
+
|
733
|
+
Clean up Serialbench Docker resources:
|
734
|
+
|
735
|
+
[source]
|
736
|
+
----
|
737
|
+
# Remove all Serialbench images
|
738
|
+
$ docker rmi $(docker images serialbench -q)
|
739
|
+
|
740
|
+
# Remove all containers
|
741
|
+
$ docker container prune
|
742
|
+
----
|
743
|
+
|
744
|
+
=== Customization options
|
745
|
+
|
746
|
+
==== Adding Ruby versions
|
747
|
+
|
748
|
+
Edit the `RUBY_VERSIONS` array in `docker/run-benchmarks.sh`:
|
749
|
+
|
750
|
+
[source,bash]
|
751
|
+
----
|
752
|
+
RUBY_VERSIONS=("3.0" "3.1" "3.2" "3.3" "3.4" "head")
|
753
|
+
----
|
754
|
+
|
755
|
+
==== Custom benchmark configuration
|
756
|
+
|
757
|
+
Create custom config files in the `config/` directory:
|
758
|
+
|
759
|
+
[source,yaml]
|
760
|
+
----
|
761
|
+
# config/custom.yml
|
762
|
+
formats:
|
763
|
+
- xml
|
764
|
+
- json
|
765
|
+
iterations: 50
|
766
|
+
warmup: 5
|
767
|
+
data_sizes:
|
768
|
+
- small
|
769
|
+
- medium
|
770
|
+
----
|
771
|
+
|
772
|
+
Reference the custom config in the run script:
|
773
|
+
|
774
|
+
[source,bash]
|
775
|
+
----
|
776
|
+
# In docker/run-benchmarks.sh
|
777
|
+
CONFIG_FILE="config/custom.yml"
|
778
|
+
----
|
779
|
+
|
780
|
+
==== Output directory customization
|
781
|
+
|
782
|
+
Change the output directory in the run script:
|
783
|
+
|
784
|
+
[source,bash]
|
785
|
+
----
|
786
|
+
# In docker/run-benchmarks.sh
|
787
|
+
OUTPUT_DIR="my-benchmark-results"
|
788
|
+
----
|
789
|
+
|
790
|
+
=== Integration with CI/CD
|
791
|
+
|
792
|
+
==== GitHub Actions integration
|
793
|
+
|
794
|
+
The Docker setup integrates seamlessly with GitHub Actions:
|
795
|
+
|
796
|
+
[source,yaml]
|
797
|
+
----
|
798
|
+
# .github/workflows/benchmark.yml
|
799
|
+
name: Multi-Ruby Benchmarks
|
800
|
+
|
801
|
+
on:
|
802
|
+
schedule:
|
803
|
+
- cron: '0 2 * * 0' # Weekly on Sunday at 2 AM
|
804
|
+
workflow_dispatch:
|
805
|
+
|
806
|
+
permissions:
|
807
|
+
contents: read
|
808
|
+
pages: write
|
809
|
+
id-token: write
|
810
|
+
|
811
|
+
concurrency:
|
812
|
+
group: "pages"
|
813
|
+
cancel-in-progress: false
|
814
|
+
|
815
|
+
jobs:
|
816
|
+
benchmark:
|
817
|
+
runs-on: ubuntu-latest
|
818
|
+
steps:
|
819
|
+
- name: Checkout repository
|
820
|
+
uses: actions/checkout@v4
|
821
|
+
|
822
|
+
- name: Setup Docker Buildx
|
823
|
+
uses: docker/setup-buildx-action@v3
|
824
|
+
|
825
|
+
- name: Run Docker Benchmarks
|
826
|
+
run: ./docker/run-benchmarks.sh
|
827
|
+
|
828
|
+
- name: Upload Results
|
829
|
+
uses: actions/upload-artifact@v4
|
830
|
+
with:
|
831
|
+
name: benchmark-results
|
832
|
+
path: docker-results/
|
833
|
+
|
834
|
+
- name: Setup Pages
|
835
|
+
uses: actions/configure-pages@v4
|
836
|
+
|
837
|
+
- name: Upload Pages artifact
|
838
|
+
uses: actions/upload-pages-artifact@v3
|
839
|
+
with:
|
840
|
+
path: ./docker-results/docs
|
841
|
+
|
842
|
+
deploy:
|
843
|
+
environment:
|
844
|
+
name: github-pages
|
845
|
+
url: ${{ steps.deployment.outputs.page_url }}
|
846
|
+
runs-on: ubuntu-latest
|
847
|
+
needs: benchmark
|
848
|
+
steps:
|
849
|
+
- name: Deploy to GitHub Pages
|
850
|
+
id: deployment
|
851
|
+
uses: actions/deploy-pages@v4
|
852
|
+
----
|
853
|
+
|
854
|
+
==== Performance considerations
|
855
|
+
|
856
|
+
* **Parallel builds**: Docker builds can run in parallel for faster execution
|
857
|
+
* **Build caching**: Subsequent runs use cached layers for faster builds
|
858
|
+
* **Memory profiling**: Enabled by default but can be disabled for faster runs
|
859
|
+
* **Result compression**: Large result files can be compressed for storage
|
860
|
+
|
861
|
+
=== Security considerations
|
862
|
+
|
863
|
+
* **Containers run with minimal privileges**: No root access required
|
864
|
+
* **No network access during benchmarks**: Isolated execution environment
|
865
|
+
* **Volume mounting**: Results written only to specified mounted volumes
|
866
|
+
* **Image scanning**: Regular security updates for base Ruby images
|
867
|
+
|
868
|
+
== Usage
|
869
|
+
|
870
|
+
=== Command line interface
|
871
|
+
|
872
|
+
==== Basic usage
|
873
|
+
|
874
|
+
Run benchmarks for all available formats:
|
875
|
+
|
876
|
+
[source]
|
877
|
+
----
|
878
|
+
$ serialbench benchmark
|
879
|
+
----
|
880
|
+
|
881
|
+
List all available serializers:
|
882
|
+
|
883
|
+
[source]
|
170
884
|
----
|
171
885
|
$ serialbench list
|
172
|
-
$ serialbench list --format xml
|
173
886
|
----
|
174
887
|
|
175
888
|
Show help information:
|
176
889
|
|
177
|
-
[source
|
890
|
+
[source]
|
178
891
|
----
|
179
892
|
$ serialbench help
|
180
893
|
$ serialbench help benchmark
|
@@ -182,47 +895,142 @@ $ serialbench help benchmark
|
|
182
895
|
|
183
896
|
Show version:
|
184
897
|
|
185
|
-
[source
|
898
|
+
[source]
|
186
899
|
----
|
187
900
|
$ serialbench version
|
188
901
|
----
|
189
902
|
|
190
|
-
|
903
|
+
==== Format-specific benchmarks
|
904
|
+
|
905
|
+
===== XML benchmarks
|
191
906
|
|
192
|
-
Run
|
907
|
+
Run all XML library benchmarks:
|
193
908
|
|
194
|
-
[source
|
909
|
+
[source]
|
910
|
+
----
|
911
|
+
$ serialbench benchmark --formats xml
|
912
|
+
----
|
913
|
+
|
914
|
+
Test specific XML libraries:
|
915
|
+
|
916
|
+
[source]
|
195
917
|
----
|
196
918
|
$ serialbench benchmark --formats xml --parsers ox,nokogiri
|
197
|
-
$ serialbench benchmark --formats xml --parsers rexml,oga
|
919
|
+
$ serialbench benchmark --formats xml --parsers rexml,oga,libxml
|
920
|
+
----
|
921
|
+
|
922
|
+
XML-only parsing performance:
|
923
|
+
|
924
|
+
[source]
|
925
|
+
----
|
926
|
+
$ serialbench benchmark --formats xml --parsing-only
|
927
|
+
----
|
928
|
+
|
929
|
+
XML generation benchmarks:
|
930
|
+
|
931
|
+
[source]
|
932
|
+
----
|
933
|
+
$ serialbench benchmark --formats xml --generation-only
|
934
|
+
----
|
935
|
+
|
936
|
+
XML streaming/SAX parsing:
|
937
|
+
|
938
|
+
[source]
|
939
|
+
----
|
940
|
+
$ serialbench benchmark --formats xml --streaming-only
|
198
941
|
----
|
199
942
|
|
200
|
-
|
943
|
+
===== JSON benchmarks
|
944
|
+
|
945
|
+
Run all JSON library benchmarks:
|
201
946
|
|
202
|
-
[source
|
947
|
+
[source]
|
203
948
|
----
|
204
|
-
$ serialbench benchmark --formats
|
949
|
+
$ serialbench benchmark --formats json
|
205
950
|
----
|
206
951
|
|
207
|
-
|
952
|
+
Test specific JSON libraries:
|
208
953
|
|
209
|
-
[source
|
954
|
+
[source]
|
210
955
|
----
|
211
|
-
$ serialbench benchmark --formats
|
956
|
+
$ serialbench benchmark --formats json --parsers oj,json
|
957
|
+
$ serialbench benchmark --formats json --parsers yajl,oj
|
212
958
|
----
|
213
959
|
|
214
|
-
|
960
|
+
===== TOML benchmarks
|
961
|
+
|
962
|
+
Run all TOML library benchmarks:
|
963
|
+
|
964
|
+
[source]
|
965
|
+
----
|
966
|
+
$ serialbench benchmark --formats toml
|
967
|
+
----
|
968
|
+
|
969
|
+
Test specific TOML libraries:
|
970
|
+
|
971
|
+
[source]
|
972
|
+
----
|
973
|
+
$ serialbench benchmark --formats toml --parsers tomlib,toml-rb
|
974
|
+
----
|
975
|
+
|
976
|
+
==== Cross-format comparisons
|
977
|
+
|
978
|
+
Compare XML vs JSON performance:
|
979
|
+
|
980
|
+
[source]
|
981
|
+
----
|
982
|
+
$ serialbench benchmark --formats xml json
|
983
|
+
----
|
984
|
+
|
985
|
+
Compare all supported formats:
|
986
|
+
|
987
|
+
[source]
|
988
|
+
----
|
989
|
+
$ serialbench benchmark --formats xml json toml
|
990
|
+
----
|
991
|
+
|
992
|
+
==== Advanced options
|
993
|
+
|
994
|
+
Memory profiling across formats:
|
995
|
+
|
996
|
+
[source]
|
997
|
+
----
|
998
|
+
$ serialbench benchmark --memory-profiling
|
999
|
+
----
|
1000
|
+
|
1001
|
+
Generate detailed reports:
|
1002
|
+
|
1003
|
+
[source]
|
1004
|
+
----
|
1005
|
+
$ serialbench benchmark --detailed-reports
|
1006
|
+
----
|
1007
|
+
|
1008
|
+
Output results in JSON format:
|
1009
|
+
|
1010
|
+
[source]
|
1011
|
+
----
|
1012
|
+
$ serialbench benchmark --output-format json
|
1013
|
+
----
|
1014
|
+
|
1015
|
+
Custom data sizes and iterations:
|
1016
|
+
|
1017
|
+
[source]
|
1018
|
+
----
|
1019
|
+
$ serialbench benchmark --data-sizes small,medium --iterations 100
|
1020
|
+
----
|
1021
|
+
|
1022
|
+
=== Multi-Ruby version comparison
|
215
1023
|
|
216
1024
|
Merge benchmark results from multiple Ruby versions:
|
217
1025
|
|
218
|
-
[source
|
1026
|
+
[source]
|
219
1027
|
----
|
220
1028
|
$ serialbench merge_results ruby-3.0/results ruby-3.1/results ruby-3.2/results merged_output/
|
221
1029
|
----
|
222
1030
|
|
223
1031
|
Generate GitHub Pages HTML from multiple benchmark runs:
|
224
1032
|
|
225
|
-
[source
|
1033
|
+
[source]
|
226
1034
|
----
|
227
1035
|
$ serialbench github_pages ruby-3.0/results ruby-3.1/results ruby-3.2/results docs/
|
228
1036
|
----
|
@@ -234,6 +1042,70 @@ This creates an interactive HTML report with:
|
|
234
1042
|
* **Environment details**: Ruby versions, platforms, and serializer versions
|
235
1043
|
* **GitHub Pages ready**: Deploy directly to GitHub Pages for public sharing
|
236
1044
|
|
1045
|
+
=== Cross-platform performance analysis
|
1046
|
+
|
1047
|
+
Analyze performance data from multiple benchmark runs across different platforms and Ruby versions:
|
1048
|
+
|
1049
|
+
[source]
|
1050
|
+
----
|
1051
|
+
$ serialbench analyze_performance artifacts/benchmark-results-*/ performance_analysis.json
|
1052
|
+
----
|
1053
|
+
|
1054
|
+
This command:
|
1055
|
+
|
1056
|
+
* **Processes multiple result directories** from different platforms and Ruby versions
|
1057
|
+
* **Extracts platform and Ruby version** information from directory names
|
1058
|
+
* **Generates comprehensive JSON** with detailed performance metrics
|
1059
|
+
* **Handles both parsing and generation** benchmark results
|
1060
|
+
* **Provides summary statistics** about processed data
|
1061
|
+
|
1062
|
+
Generate platform comparison reports:
|
1063
|
+
|
1064
|
+
[source]
|
1065
|
+
----
|
1066
|
+
$ serialbench platform_comparison performance_analysis.json platform_comparison.json
|
1067
|
+
----
|
1068
|
+
|
1069
|
+
This creates a JSON report with:
|
1070
|
+
|
1071
|
+
* **Cross-platform statistics**: Average, min, max performance by platform
|
1072
|
+
* **Format-specific analysis**: Performance breakdown by serialization format
|
1073
|
+
* **Operation comparison**: Separate analysis for parsing vs generation
|
1074
|
+
* **Sample counts**: Number of data points for statistical confidence
|
1075
|
+
* **Standard deviation**: Statistical variance for performance consistency
|
1076
|
+
|
1077
|
+
==== Example workflow for cross-platform analysis
|
1078
|
+
|
1079
|
+
[source]
|
1080
|
+
----
|
1081
|
+
# 1. Run benchmarks on different platforms (or use CI artifacts)
|
1082
|
+
$ serialbench benchmark --formats xml json yaml toml
|
1083
|
+
|
1084
|
+
# 2. Collect results from multiple platforms/Ruby versions
|
1085
|
+
$ mkdir analysis
|
1086
|
+
$ cp -r platform1-ruby3.2/results analysis/benchmark-results-ubuntu-ruby-3.2
|
1087
|
+
$ cp -r platform2-ruby3.3/results analysis/benchmark-results-macos-ruby-3.3
|
1088
|
+
$ cp -r platform3-ruby3.4/results analysis/benchmark-results-windows-ruby-3.4
|
1089
|
+
|
1090
|
+
# 3. Generate performance analysis
|
1091
|
+
$ serialbench analyze_performance analysis/benchmark-results-*/ performance_analysis.json
|
1092
|
+
|
1093
|
+
# 4. Create platform comparison report
|
1094
|
+
$ serialbench platform_comparison performance_analysis.json platform_comparison.json
|
1095
|
+
|
1096
|
+
# 5. View results
|
1097
|
+
$ cat performance_analysis.json | jq '.summary'
|
1098
|
+
$ cat performance_analysis.json | jq '.platforms'
|
1099
|
+
$ cat platform_comparison.json | jq '.platforms'
|
1100
|
+
----
|
1101
|
+
|
1102
|
+
The analysis commands are particularly useful for:
|
1103
|
+
|
1104
|
+
* **CI/CD integration**: Automated cross-platform performance tracking
|
1105
|
+
* **Performance regression detection**: Compare results across builds
|
1106
|
+
* **Platform optimization**: Identify platform-specific performance characteristics
|
1107
|
+
* **Ruby version migration**: Analyze performance impact of Ruby upgrades
|
1108
|
+
|
237
1109
|
=== Programmatic usage
|
238
1110
|
|
239
1111
|
==== Basic benchmark execution
|
@@ -341,11 +1213,14 @@ Tests how quickly libraries can convert Ruby objects into serialized strings.
|
|
341
1213
|
|
342
1214
|
=== Streaming performance
|
343
1215
|
|
344
|
-
Evaluates streaming event-based parsing performance for libraries that support
|
1216
|
+
Evaluates streaming event-based parsing performance for libraries that support
|
1217
|
+
it, which processes data sequentially and is memory-efficient for large files.
|
345
1218
|
|
346
1219
|
=== Memory usage analysis
|
347
1220
|
|
348
|
-
Profiles memory allocation and retention during serialization operations using
|
1221
|
+
Profiles memory allocation and retention during serialization operations using
|
1222
|
+
the `memory_profiler` gem.
|
1223
|
+
|
349
1224
|
|
350
1225
|
== Output and reports
|
351
1226
|
|
@@ -381,17 +1256,16 @@ results/
|
|
381
1256
|
* **Performance rankings**: Fastest to slowest for each category
|
382
1257
|
* **Memory profiling**: Detailed memory allocation analysis
|
383
1258
|
* **Feature matrix**: Capability comparison across libraries
|
384
|
-
* **Recommendations**: Use-case specific library suggestions
|
385
1259
|
* **Environment details**: Ruby version, platform, and library versions
|
386
1260
|
|
387
1261
|
=== Sample output
|
388
1262
|
|
389
|
-
[
|
390
|
-
|
1263
|
+
[example]
|
1264
|
+
====
|
391
1265
|
Serialbench - Comprehensive Serialization Performance Tests
|
392
1266
|
===========================================================
|
393
1267
|
Environment: Ruby 3.3.2 on arm64-darwin23
|
394
|
-
Timestamp:
|
1268
|
+
Timestamp: 2025-06-07T10:30:00Z
|
395
1269
|
|
396
1270
|
Available serializers: rexml, json, oj, toml-rb
|
397
1271
|
Test formats: xml, json, toml
|
@@ -409,7 +1283,7 @@ Parsing Performance:
|
|
409
1283
|
JSON/json: 12.67ms
|
410
1284
|
XML/rexml: 28.45ms
|
411
1285
|
TOML/toml-rb: 35.21ms
|
412
|
-
|
1286
|
+
====
|
413
1287
|
|
414
1288
|
== Methodology
|
415
1289
|
|
@@ -443,57 +1317,11 @@ The benchmark suite uses carefully crafted synthetic data that represents common
|
|
443
1317
|
* Memory measurements account for garbage collection
|
444
1318
|
* Results include both absolute and relative performance metrics
|
445
1319
|
|
446
|
-
== Library comparison matrix
|
447
|
-
|
448
|
-
[cols="1,1,1,1,1,1,1"]
|
449
|
-
|===
|
450
|
-
|Format |Library |Parsing |Generation |Streaming |Memory |Features
|
451
|
-
|
452
|
-
|XML |REXML |⭐⭐ |⭐⭐ |⭐⭐⭐ |⭐⭐ |Built-in
|
453
|
-
|XML |Ox |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |High-performance
|
454
|
-
|XML |Nokogiri |⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐ |Feature-rich
|
455
|
-
|XML |LibXML |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |High-performance
|
456
|
-
|XML |Oga |⭐⭐ |⭐⭐ |⭐⭐⭐ |⭐⭐ |Pure Ruby
|
457
|
-
|JSON |JSON |⭐⭐⭐ |⭐⭐⭐ |❌ |⭐⭐⭐ |Built-in
|
458
|
-
|JSON |Oj |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐⭐ |High-performance
|
459
|
-
|JSON |YAJL |⭐⭐⭐⭐ |⭐⭐⭐ |⭐⭐⭐⭐ |⭐⭐⭐⭐ |Streaming
|
460
|
-
|TOML |TOML-RB |⭐⭐⭐ |⭐⭐⭐ |❌ |⭐⭐⭐ |Standard
|
461
|
-
|TOML |Tomlib |⭐⭐⭐⭐⭐ |⭐⭐⭐⭐ |❌ |⭐⭐⭐⭐⭐ |High-performance
|
462
|
-
|===
|
463
|
-
|
464
|
-
_Performance ratings: ⭐⭐⭐⭐⭐ Excellent, ⭐⭐⭐⭐ Good, ⭐⭐⭐ Average, ⭐⭐ Below average, ⭐ Poor, ❌ Not supported_
|
465
|
-
|
466
|
-
== Recommendations
|
467
|
-
|
468
|
-
=== For high-performance JSON applications
|
469
|
-
|
470
|
-
**Oj** is recommended for applications where JSON parsing/generation speed is critical. It consistently outperforms the built-in JSON library.
|
471
|
-
|
472
|
-
=== For configuration files
|
473
|
-
|
474
|
-
**TOML** provides human-readable configuration with good parsing performance. **JSON** is faster but less readable for configuration.
|
475
|
-
|
476
|
-
=== For data interchange
|
477
|
-
|
478
|
-
**JSON** offers the best balance of performance, compatibility, and tooling support across different systems.
|
479
|
-
|
480
|
-
=== For document processing
|
481
|
-
|
482
|
-
**XML** with **REXML** provides built-in support, though performance is lower than JSON alternatives.
|
483
|
-
|
484
|
-
=== For memory-constrained environments
|
485
|
-
|
486
|
-
**Oj** demonstrates superior memory efficiency. For large file processing, streaming approaches are recommended where available.
|
487
|
-
|
488
|
-
=== For minimal dependencies
|
489
|
-
|
490
|
-
**JSON** and **REXML** are included with Ruby and require no additional gems, making them suitable for environments with strict dependency constraints.
|
491
|
-
|
492
1320
|
== Development
|
493
1321
|
|
494
1322
|
=== Running tests
|
495
1323
|
|
496
|
-
[source
|
1324
|
+
[source]
|
497
1325
|
----
|
498
1326
|
$ bundle exec rake
|
499
1327
|
$ bundle exec rspec
|
@@ -501,21 +1329,21 @@ $ bundle exec rspec
|
|
501
1329
|
|
502
1330
|
=== Contributing
|
503
1331
|
|
504
|
-
|
505
|
-
|
506
|
-
|
507
|
-
|
508
|
-
|
1332
|
+
. Fork the repository
|
1333
|
+
. Create your feature branch (`git checkout -b feature/my-new-feature`)
|
1334
|
+
. Commit your changes (`git commit -am 'Add some feature'`)
|
1335
|
+
. Push to the branch (`git push origin feature/my-new-feature`)
|
1336
|
+
. Create a new Pull Request
|
509
1337
|
|
510
1338
|
=== Adding new serializers
|
511
1339
|
|
512
1340
|
To add support for additional serialization libraries:
|
513
1341
|
|
514
|
-
|
515
|
-
|
516
|
-
|
517
|
-
|
518
|
-
|
1342
|
+
. Create a new serializer class in `lib/serialbench/serializers/{format}/`
|
1343
|
+
. Inherit from the appropriate base class (`BaseXmlSerializer`, `BaseJsonSerializer`, etc.)
|
1344
|
+
. Implement the required methods: `parse`, `generate`, `name`, `version`
|
1345
|
+
. Add the serializer to the registry in `lib/serialbench/serializers.rb`
|
1346
|
+
. Update documentation and tests
|
519
1347
|
|
520
1348
|
==== Example: Adding a new JSON serializer
|
521
1349
|
|