moxml 0.5.178 → 0.6.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/.github/workflows/nightly-fuzz.yml +0 -14
- data/.github/workflows/performance-release.yml +0 -13
- data/.github/workflows/performance.yml +0 -17
- data/.github/workflows/rake.yml +0 -38
- data/.github/workflows/round-trip.yml +0 -45
- data/.github/workflows/windows.yml +0 -12
- data/Gemfile +4 -14
- data/README.adoc +5 -388
- data/Rakefile +2 -179
- data/docs/_references/processing-model.adoc +3 -2
- data/lib/moxml/adapter/customized_oga.rb +6 -4
- data/lib/moxml/adapter/customized_rexml.rb +5 -3
- data/lib/moxml/adapter/headed_ox.rb +0 -2
- data/lib/moxml/adapter/leptris.rb +0 -2
- data/lib/moxml/adapter/libxml.rb +0 -2
- data/lib/moxml/adapter/nokogiri.rb +0 -2
- data/lib/moxml/adapter/ox.rb +0 -2
- data/lib/moxml/adapter/rexml.rb +1 -3
- data/lib/moxml/adapter.rb +1 -6
- data/lib/moxml/config.rb +0 -17
- data/lib/moxml/entity_registry.rb +1 -5
- data/lib/moxml/native_attachment.rb +1 -3
- data/lib/moxml/version.rb +1 -1
- data/lib/moxml.rb +1 -2
- data/spec/moxml/adapter/platform_spec.rb +2 -47
- data/spec/moxml/native_attachment_spec.rb +4 -50
- data/spec/moxml/signature/edge_cases_spec.rb +0 -1
- data/spec/spec_helper.rb +13 -31
- metadata +1 -21
- data/.github/workflows/opal.yml +0 -61
- data/.gitmodules +0 -6
- data/lib/compat/opal/moxml_boot.rb +0 -68
- data/lib/compat/opal/rexml/namespace.rb +0 -59
- data/lib/compat/opal/rexml/parsers/baseparser.rb +0 -1016
- data/lib/compat/opal/rexml/source.rb +0 -214
- data/lib/compat/opal/rexml/text.rb +0 -426
- data/lib/compat/opal/rexml/xmltokens.rb +0 -45
- data/lib/compat/opal/rexml_compat.rb +0 -77
- data/lib/moxml/entity_registry_opal_data.rb +0 -2138
- data/lib/moxml/native_attachment/opal.rb +0 -38
- data/spec/moxml/native_attachment/opal_spec.rb +0 -66
- data/spec/moxml/opal_oga_adapter_spec.rb +0 -20
- data/spec/moxml/opal_oga_features_spec.rb +0 -212
- data/spec/moxml/opal_oga_smoke_spec.rb +0 -35
- data/spec/moxml/opal_rexml_adapter_spec.rb +0 -14
- data/spec/moxml/opal_smoke_spec.rb +0 -61
- data/spec/moxml/reference_opal_spec.rb +0 -29
- data/spec/moxml/signature_autoload_spec.rb +0 -52
- data/spec/support/opal.rb +0 -16
data/README.adoc
CHANGED
|
@@ -56,8 +56,7 @@ widely used implementation which wraps around the performant
|
|
|
56
56
|
https://github.com/GNOME/libxml2[libxml2] C library.
|
|
57
57
|
|
|
58
58
|
Oga:: https://github.com/yorickpeterse/oga[Oga], a pure Ruby XML parser.
|
|
59
|
-
|
|
60
|
-
https://github.com/opal/opal[Opal].
|
|
59
|
+
Useful when you need a dependency with no native code.
|
|
61
60
|
|
|
62
61
|
Ox:: https://github.com/ohler55/ox[Ox], a fast XML parser.
|
|
63
62
|
|
|
@@ -100,9 +99,8 @@ or :nokogiri. Removal after the deprecation cycle.
|
|
|
100
99
|
|
|
101
100
|
|Oga
|
|
102
101
|
|3.4 / 3.5
|
|
103
|
-
|
|
|
104
|
-
|
|
105
|
-
moxml's customized generator and decoder overrides.
|
|
102
|
+
|Oga 4.x is not yet validated against moxml's customized generator
|
|
103
|
+
and decoder overrides.
|
|
106
104
|
|
|
107
105
|
|LibXML (libxml-ruby)
|
|
108
106
|
|>= 5.0
|
|
@@ -131,8 +129,7 @@ matrix tests exactly those floors.
|
|
|
131
129
|
When no adapter is explicitly specified, Moxml selects one automatically based on
|
|
132
130
|
the runtime environment:
|
|
133
131
|
|
|
134
|
-
. If
|
|
135
|
-
. Otherwise, if the installed leptris gem supports programmatic document
|
|
132
|
+
. If the installed leptris gem supports programmatic document
|
|
136
133
|
construction (`Leptris::XML::Document.create`, leptris >= 1.9.386.0), Leptris is used
|
|
137
134
|
. Otherwise, Moxml detects already-loaded XML libraries in this order:
|
|
138
135
|
.. Nokogiri
|
|
@@ -144,7 +141,7 @@ the runtime environment:
|
|
|
144
141
|
----
|
|
145
142
|
# Automatic detection — picks the best available adapter
|
|
146
143
|
ctx = Moxml.new
|
|
147
|
-
ctx.config.adapter_name # => :leptris (leptris >= 1.9.386.0), :nokogiri otherwise
|
|
144
|
+
ctx.config.adapter_name # => :leptris (leptris >= 1.9.386.0), :nokogiri otherwise
|
|
148
145
|
|
|
149
146
|
# Explicit override — always takes precedence
|
|
150
147
|
ctx = Moxml.new(:ox)
|
|
@@ -1238,384 +1235,4 @@ performance optimization, and testing strategies, see
|
|
|
1238
1235
|
link:docs/_pages/best-practices.adoc[Best Practices Guide].
|
|
1239
1236
|
|
|
1240
1237
|
|
|
1241
|
-
== Running under Opal (JavaScript)
|
|
1242
|
-
|
|
1243
|
-
Opal compiles Ruby to JavaScript and cannot load C extensions. To run
|
|
1244
|
-
Moxml on Opal you need three things: an adapter with no native code, a
|
|
1245
|
-
XML library fork that exposes pure-Ruby fallbacks for its lexer and
|
|
1246
|
-
parser driver, and an Opal build that puts those pure-Ruby
|
|
1247
|
-
implementations on the load path.
|
|
1248
|
-
|
|
1249
|
-
=== Default adapter under Opal
|
|
1250
|
-
|
|
1251
|
-
Under `RUBY_ENGINE == "opal"`, Moxml's default adapter is `:oga` (see
|
|
1252
|
-
<<Default adapter selection>>). Oga is a pure-Ruby XML parser; under
|
|
1253
|
-
standard CRuby its lexer and parsers are accelerated by the `liboga`
|
|
1254
|
-
and `libll` C extensions. Those extensions cannot be compiled into
|
|
1255
|
-
JavaScript, so on Opal they are replaced by the pure-Ruby fallbacks
|
|
1256
|
-
shipped in `ext/pureruby/` of the Opal-compatible forks below.
|
|
1257
|
-
|
|
1258
|
-
The REXML adapter is also available on Opal (`Moxml.new(:rexml)`) but
|
|
1259
|
-
relies on a number of compatibility shims because REXML uses Ruby
|
|
1260
|
-
regular-expression features (the `/n` flag, `\u{...}` codepoint
|
|
1261
|
-
escapes, inline flag groups) and String APIs that do not all transpile
|
|
1262
|
-
to JavaScript today. `:oga` is the recommended path for new Opal
|
|
1263
|
-
projects.
|
|
1264
|
-
|
|
1265
|
-
=== Required XML library forks
|
|
1266
|
-
|
|
1267
|
-
Upstream https://github.com/yorickpeterse/oga[oga] and
|
|
1268
|
-
https://github.com/yorickpeterse/ruby-ll[ruby-ll] do not currently
|
|
1269
|
-
ship Opal-compatible fallbacks. Moxml vendors the following forks:
|
|
1270
|
-
|
|
1271
|
-
* https://github.com/lutaml/opal-oga[lutaml/opal-oga] — oga with an
|
|
1272
|
-
`ext/pureruby/oga/native/lexer.rb` fallback that fires when
|
|
1273
|
-
`RUBY_PLATFORM == 'opal'`. Under CRuby/JRuby the fork behaves
|
|
1274
|
-
identically to upstream oga.
|
|
1275
|
-
* https://github.com/lutaml/opal-ruby-ll[lutaml/opal-ruby-ll] —
|
|
1276
|
-
ruby-ll with `ext/pureruby/ll/native/{driver,driver_config}.rb`
|
|
1277
|
-
fallbacks selected by the same conditional in `lib/ll/setup.rb`.
|
|
1278
|
-
|
|
1279
|
-
Both forks are vendored as submodules under `vendor/opal-oga` and
|
|
1280
|
-
`vendor/opal-ruby-ll` in the Moxml repository. Moxml's own `Gemfile`
|
|
1281
|
-
references them via `path:` source so that any consumer of Moxml under
|
|
1282
|
-
Opal resolves to the same fork commits Moxml was tested against.
|
|
1283
|
-
|
|
1284
|
-
=== Wiring the forks in a downstream Gemfile
|
|
1285
|
-
|
|
1286
|
-
Downstream libraries that run on Opal (for example
|
|
1287
|
-
https://github.com/plurimath/plurimath-js[plurimath-js]) must also use
|
|
1288
|
-
the Opal-compatible forks rather than upstream `oga` and `ruby-ll`.
|
|
1289
|
-
Wire them as `path:` or `git:` sources so the fork's
|
|
1290
|
-
`RUBY_PLATFORM == 'opal'` conditional can fire:
|
|
1291
|
-
|
|
1292
|
-
[source,ruby]
|
|
1293
|
-
----
|
|
1294
|
-
# In your downstream gem's Gemfile (or gemspec)
|
|
1295
|
-
gem "moxml", "~> 0.1.24" # provides the :oga adapter
|
|
1296
|
-
gem "oga", git: "https://github.com/lutaml/opal-oga.git",
|
|
1297
|
-
ref: "<commit-sha-moxml-tests-against>"
|
|
1298
|
-
gem "ruby-ll", git: "https://github.com/lutaml/opal-ruby-ll.git",
|
|
1299
|
-
ref: "<commit-sha-moxml-tests-against>"
|
|
1300
|
-
----
|
|
1301
|
-
|
|
1302
|
-
Pin the `ref:` to the submodule commits Moxml ships in
|
|
1303
|
-
`vendor/opal-oga` and `vendor/opal-ruby-ll` (recorded in
|
|
1304
|
-
`.gitmodules`) so your fork and Moxml's fork stay in lockstep. If you
|
|
1305
|
-
prefer to vendor the forks yourself, replace the `git:` source with
|
|
1306
|
-
`path: "vendor/opal-oga"` and `path: "vendor/opal-ruby-ll"` after
|
|
1307
|
-
copying them in.
|
|
1308
|
-
|
|
1309
|
-
=== Opal build configuration
|
|
1310
|
-
|
|
1311
|
-
Under Opal, the forks' `lib/` directory is on the load path (via the
|
|
1312
|
-
gem), but the pure-Ruby implementations live under `ext/pureruby/`.
|
|
1313
|
-
That directory must be added to Opal's load path so the
|
|
1314
|
-
`if RUBY_PLATFORM == 'opal'` branch in `lib/oga.rb` and
|
|
1315
|
-
`lib/ll/setup.rb` resolves to the pure-Ruby files instead of trying to
|
|
1316
|
-
load the C extension.
|
|
1317
|
-
|
|
1318
|
-
For consumers using https://github.com/opal/opal[Opal] directly:
|
|
1319
|
-
|
|
1320
|
-
[source,ruby]
|
|
1321
|
-
----
|
|
1322
|
-
require "opal"
|
|
1323
|
-
|
|
1324
|
-
# Opal path: put both lib/ and ext/pureruby/ on the load path so the
|
|
1325
|
-
# fork's conditional requires resolve to the pure-Ruby fallback.
|
|
1326
|
-
%w[opal-oga opal-ruby-ll].each do |fork|
|
|
1327
|
-
Opal.append_path File.expand_path("vendor/#{fork}/lib", __dir__)
|
|
1328
|
-
Opal.append_path File.expand_path("vendor/#{fork}/ext/pureruby", __dir__)
|
|
1329
|
-
end
|
|
1330
|
-
|
|
1331
|
-
# Moxml's Opal boot file (lib/compat/opal/moxml_boot.rb) explicitly
|
|
1332
|
-
# requires every module that would normally be autoloaded under CRuby.
|
|
1333
|
-
require "moxml_boot"
|
|
1334
|
-
----
|
|
1335
|
-
|
|
1336
|
-
For consumers using https://github.com/opal/opal-compiler[Opal via
|
|
1337
|
-
Sprockets/Webpack], the equivalent is to add both `lib/` and
|
|
1338
|
-
`ext/pureruby/` from each fork to the Opal `paths` array.
|
|
1339
|
-
|
|
1340
|
-
=== Verifying the Opal build
|
|
1341
|
-
|
|
1342
|
-
Moxml's own Opal test suite exercises the wiring described above. To
|
|
1343
|
-
reproduce:
|
|
1344
|
-
|
|
1345
|
-
[source,shell]
|
|
1346
|
-
----
|
|
1347
|
-
# One-time setup: install ragel + ruby-ll so the forks' gitignored
|
|
1348
|
-
# generated lexer/parser sources can be produced. ruby-ll must be
|
|
1349
|
-
# installed outside the bundle: vendor:prepare runs it outside Bundler.
|
|
1350
|
-
gem install ruby-ll --no-document
|
|
1351
|
-
# (Install ragel via your package manager, e.g. apt-get install ragel
|
|
1352
|
-
# or brew install ragel.)
|
|
1353
|
-
|
|
1354
|
-
# Generate the gitignored outputs in the vendored forks.
|
|
1355
|
-
bundle exec rake vendor:prepare
|
|
1356
|
-
|
|
1357
|
-
# Run the Opal (JavaScript) test suite.
|
|
1358
|
-
bundle exec rake spec:opal
|
|
1359
|
-
----
|
|
1360
|
-
|
|
1361
|
-
Reference implementation: https://github.com/plurimath/plurimath-js[plurimath-js]
|
|
1362
|
-
is the canonical downstream consumer; its `Rakefile` and `env/` wrapper
|
|
1363
|
-
scripts show the full pattern end-to-end.
|
|
1364
|
-
|
|
1365
|
-
|
|
1366
|
-
== Specific adapter limitations
|
|
1367
|
-
|
|
1368
|
-
=== Ox adapter
|
|
1369
|
-
|
|
1370
|
-
The Ox adapter provides maximum parsing speed but has XPath limitations.
|
|
1371
|
-
|
|
1372
|
-
**XPath limitations:**
|
|
1373
|
-
|
|
1374
|
-
* No attribute value predicates: `//book[@id='123']` ❌
|
|
1375
|
-
* No logical operators, position predicates, text predicates ❌
|
|
1376
|
-
* No namespace queries, parent axis, sibling axes ❌
|
|
1377
|
-
* No XPath functions ❌
|
|
1378
|
-
|
|
1379
|
-
**Workaround:** Use Ruby enumerable methods:
|
|
1380
|
-
|
|
1381
|
-
[source,ruby]
|
|
1382
|
-
----
|
|
1383
|
-
# Instead of: doc.xpath("//book[@id='123']")
|
|
1384
|
-
doc.xpath("//book").find { |book| book["id"] == "123" }
|
|
1385
|
-
----
|
|
1386
|
-
|
|
1387
|
-
**Upstream Ox gem limitations:**
|
|
1388
|
-
|
|
1389
|
-
These limitations exist in the Ox gem itself and cannot be worked around in Moxml
|
|
1390
|
-
without changes to the Ox C extension:
|
|
1391
|
-
|
|
1392
|
-
* *Namespace introspection* — Ox stores `xmlns` attributes but does not expose
|
|
1393
|
-
namespace accessors on `Ox::Element`. Methods like `node.namespace`,
|
|
1394
|
-
`node.namespaces`, and namespace inheritance are unavailable.
|
|
1395
|
-
* *Parent node reparenting* — Ox has no method to change a node's parent after
|
|
1396
|
-
creation, preventing `node.parent=` functionality. Nodes are immutable with
|
|
1397
|
-
respect to their parent relationship.
|
|
1398
|
-
|
|
1399
|
-
For complete Ox adapter documentation including all limitations and workarounds,
|
|
1400
|
-
see link:docs/_pages/adapters/ox.adoc[Ox Adapter Guide].
|
|
1401
|
-
|
|
1402
|
-
=== HeadedOx adapter
|
|
1403
|
-
|
|
1404
|
-
The HeadedOx adapter combines Ox's fast C-based XML parsing with Moxml's
|
|
1405
|
-
comprehensive pure Ruby XPath 1.0 engine.
|
|
1406
|
-
|
|
1407
|
-
**Status:** Production-ready v1.2 (99.20% pass rate, 1,992/2,008 tests)
|
|
1408
|
-
|
|
1409
|
-
**Key features:**
|
|
1410
|
-
|
|
1411
|
-
* Fast XML parsing (Ox C extension)
|
|
1412
|
-
* All 27 XPath 1.0 functions
|
|
1413
|
-
* 6 XPath axes (child, descendant, parent, attribute, self, descendant-or-self)
|
|
1414
|
-
* Expression caching for performance
|
|
1415
|
-
* Pure Ruby XPath engine (debuggable)
|
|
1416
|
-
|
|
1417
|
-
**When to use:**
|
|
1418
|
-
|
|
1419
|
-
* Need Ox's fast parsing with comprehensive XPath
|
|
1420
|
-
* Want XPath functions (count, sum, contains, etc.)
|
|
1421
|
-
* Prefer pure Ruby XPath for debugging
|
|
1422
|
-
* Basic namespace queries are sufficient
|
|
1423
|
-
|
|
1424
|
-
**Inherited Ox limitations:**
|
|
1425
|
-
|
|
1426
|
-
HeadedOx inherits the upstream Ox gem limitations described above (namespace
|
|
1427
|
-
introspection and parent node reparenting). Additionally, some sibling axes
|
|
1428
|
-
are not fully supported due to Ox's tree structure.
|
|
1429
|
-
|
|
1430
|
-
[source,ruby]
|
|
1431
|
-
----
|
|
1432
|
-
# Use HeadedOx adapter
|
|
1433
|
-
context = Moxml.new(:headed_ox)
|
|
1434
|
-
doc = context.parse(xml_string)
|
|
1435
|
-
|
|
1436
|
-
# Full XPath 1.0 support
|
|
1437
|
-
books = doc.xpath('//book[@price < 20]')
|
|
1438
|
-
count = doc.xpath('count(//book)')
|
|
1439
|
-
titles = doc.xpath('//book/title[contains(., "Ruby")]')
|
|
1440
|
-
----
|
|
1441
|
-
|
|
1442
|
-
For complete HeadedOx documentation including architecture, XPath capabilities, known limitations, and usage examples, see link:docs/_pages/adapters/headed-ox.adoc[HeadedOx Adapter Guide] and link:docs/HEADED_OX_LIMITATIONS.md[Limitations Documentation].
|
|
1443
|
-
|
|
1444
|
-
|
|
1445
|
-
==== LibXML adapter
|
|
1446
|
-
|
|
1447
|
-
*Performance:*
|
|
1448
|
-
|
|
1449
|
-
* Serialization speed: ~120 ips (slower than target)
|
|
1450
|
-
* Parsing speed: Good
|
|
1451
|
-
* For high-throughput serialization, consider Ox or Nokogiri
|
|
1452
|
-
|
|
1453
|
-
=== Other adapters
|
|
1454
|
-
|
|
1455
|
-
*Nokogiri, Oga, REXML:*
|
|
1456
|
-
|
|
1457
|
-
All three adapters have near-complete feature support with only minor edge case
|
|
1458
|
-
limitations. Use these adapters when you need full XPath and namespace support.
|
|
1459
|
-
|
|
1460
|
-
|
|
1461
|
-
|
|
1462
|
-
== Round-trip XML Testing
|
|
1463
|
-
|
|
1464
|
-
Moxml includes comprehensive round-trip testing to verify that XML documents remain
|
|
1465
|
-
semantically equivalent when parsed and serialized across different adapters.
|
|
1466
|
-
|
|
1467
|
-
=== Purpose
|
|
1468
|
-
|
|
1469
|
-
Round-trip testing ensures:
|
|
1470
|
-
|
|
1471
|
-
* **Cross-adapter compatibility** - XML parsed with one adapter (e.g., Nokogiri) can be
|
|
1472
|
-
serialized and re-parsed with another adapter (e.g., Oga) while preserving content
|
|
1473
|
-
* **Structural fidelity** - Element names, attributes, and document structure are maintained
|
|
1474
|
-
* **Content preservation** - Text content and entity references survive multiple parse/serialize cycles
|
|
1475
|
-
* **Double round-trip verification** - Source → Target → Source sequences produce semantically
|
|
1476
|
-
equivalent output
|
|
1477
|
-
|
|
1478
|
-
=== Test Fixtures
|
|
1479
|
-
|
|
1480
|
-
Round-trip tests use real-world XML documents organized into collections:
|
|
1481
|
-
|
|
1482
|
-
**rfcxml** - IETF RFC documents in XML format. These provide complex, standards-compliant
|
|
1483
|
-
XML with mixed content, namespaces, and attributes. The collection includes:
|
|
1484
|
-
|
|
1485
|
-
* Large documents (500KB-2.4MB) for stress testing
|
|
1486
|
-
* Rich metadata and cross-references
|
|
1487
|
-
* Various XML schema patterns
|
|
1488
|
-
|
|
1489
|
-
**metanorma** - Metanorma document processing XML. These test:
|
|
1490
|
-
|
|
1491
|
-
* Document structure preservation
|
|
1492
|
-
* Nested elements and complex hierarchies
|
|
1493
|
-
* Standard XML vocabularies
|
|
1494
|
-
|
|
1495
|
-
**niso-jats** - NISO Journal Article Tag Suite XML. These provide:
|
|
1496
|
-
|
|
1497
|
-
* Scholarly publishing XML schemas
|
|
1498
|
-
* Rich bibliographic metadata
|
|
1499
|
-
* Mixed content models
|
|
1500
|
-
|
|
1501
|
-
=== Running Round-trip Tests
|
|
1502
|
-
|
|
1503
|
-
[source,bash]
|
|
1504
|
-
----
|
|
1505
|
-
# Run all round-trip tests
|
|
1506
|
-
bundle exec rake spec:consistency
|
|
1507
|
-
|
|
1508
|
-
# Exclude REXML for larger fixtures (faster, REXML is pure Ruby)
|
|
1509
|
-
MOXML_ROUNDTRIP_REXML_MAX_SIZE=0 bundle exec rake spec:consistency
|
|
1510
|
-
|
|
1511
|
-
# Adjust the per-example timeout (default: 120 seconds)
|
|
1512
|
-
MOXML_ROUNDTRIP_TIMEOUT=300 bundle exec rake spec:consistency
|
|
1513
|
-
----
|
|
1514
|
-
|
|
1515
|
-
REXML is a pure Ruby XML parser and becomes very slow on large documents (500KB+).
|
|
1516
|
-
By default, REXML adapter pairs are skipped for fixtures exceeding 500KB. All other
|
|
1517
|
-
adapters (Nokogiri, Oga, Ox) are tested against every fixture.
|
|
1518
|
-
|
|
1519
|
-
=== Test Mechanics
|
|
1520
|
-
|
|
1521
|
-
For each fixture, tests run across all adapter pairs (4 adapters = 12 combinations):
|
|
1522
|
-
|
|
1523
|
-
1. Parse with source adapter
|
|
1524
|
-
2. Serialize to XML string
|
|
1525
|
-
3. Parse serialized output with target adapter
|
|
1526
|
-
4. Compare semantic equivalence (element names, attributes, text content)
|
|
1527
|
-
|
|
1528
|
-
A "double round-trip" test additionally verifies: Source → Target → Source → Target
|
|
1529
|
-
produces consistent results.
|
|
1530
|
-
|
|
1531
|
-
NOTE: REXML is excluded from adapter pairs for fixtures larger than 500KB (configurable
|
|
1532
|
-
via `MOXML_ROUNDTRIP_REXML_MAX_SIZE`). This is because REXML is pure Ruby and cannot
|
|
1533
|
-
parse large XML documents in a practical timeframe. A per-example timeout
|
|
1534
|
-
(`MOXML_ROUNDTRIP_TIMEOUT`, default 120s) prevents tests from hanging indefinitely.
|
|
1535
|
-
|
|
1536
|
-
=== Ox Adapter Element Ordering Caveat
|
|
1537
|
-
|
|
1538
|
-
The Ox adapter produces elements in a different order than other adapters for certain
|
|
1539
|
-
fixtures with complex nested structures (e.g., `element_citation.xml`,
|
|
1540
|
-
`collection1nested.xml`, `pnas_sample.xml`). This causes the `elements_with_attributes`
|
|
1541
|
-
comparison to fail with "Array length mismatch" even though the semantic equivalence
|
|
1542
|
-
check (double round-trip) passes.
|
|
1543
|
-
|
|
1544
|
-
Round-trip tests automatically skip the `elements_with_attributes` comparison for these
|
|
1545
|
-
known Ox ordering issues. The `ruby-versions` CI job tests only Nokogiri and Oga adapters;
|
|
1546
|
-
the `nokogiri-ox` and `nokogiri-rexml` CI jobs test Ox and REXML respectively but are
|
|
1547
|
-
marked as **experimental** since these adapters lack full XML feature support:
|
|
1548
|
-
|
|
1549
|
-
* **Ox**: Lacks proper namespace support, XPath with predicates, and uses a custom
|
|
1550
|
-
`locate()` method instead of standard XPath
|
|
1551
|
-
* **REXML**: Pure Ruby, exponential time complexity with document size, impractical for
|
|
1552
|
-
documents over ~500KB
|
|
1553
|
-
|
|
1554
|
-
For production use, prefer Nokogiri or Oga which provide complete XML conformance.
|
|
1555
|
-
|
|
1556
|
-
To run tests with a specific adapter set locally:
|
|
1557
|
-
|
|
1558
|
-
[source,bash]
|
|
1559
|
-
----
|
|
1560
|
-
# Nokogiri + Oga only (fast, full test suite)
|
|
1561
|
-
MOXML_ROUNDTRIP_ADAPTERS=nokogiri,oga bundle exec rspec spec/consistency/ --tag round_trip
|
|
1562
|
-
|
|
1563
|
-
# Nokogiri × Ox only (experimental)
|
|
1564
|
-
MOXML_ROUNDTRIP_ADAPTERS=nokogiri,ox MOXML_ROUNDTRIP_TIMEOUT=300 bundle exec rspec spec/consistency/ --tag round_trip
|
|
1565
|
-
|
|
1566
|
-
# Nokogiri × REXML only (experimental, small fixtures due to exponential complexity)
|
|
1567
|
-
MOXML_ROUNDTRIP_ADAPTERS=nokogiri,rexml MOXML_ROUNDTRIP_TIMEOUT=300 MOXML_ROUNDTRIP_REXML_MAX_SIZE=50000 bundle exec rspec spec/consistency/ --tag round_trip
|
|
1568
|
-
----
|
|
1569
|
-
|
|
1570
|
-
=== Why Semantic Equivalence?
|
|
1571
|
-
|
|
1572
|
-
While a pure round-trip test with raw XML comparison would be ideal, different XML adapters
|
|
1573
|
-
have fundamentally different philosophies for handling:
|
|
1574
|
-
|
|
1575
|
-
* **Element ordering** - Some preserve document order, others sort alphabetically
|
|
1576
|
-
* **Whitespace handling** - Some normalize spaces, others preserve exactly
|
|
1577
|
-
* **Attribute representation** - Different data structures for the same attributes
|
|
1578
|
-
* **Text extraction** - Varying approaches to concatenating text content
|
|
1579
|
-
|
|
1580
|
-
Instead of raw comparison, Moxml implements semantic equivalence testing that focuses on
|
|
1581
|
-
meaningful XML structure and content:
|
|
1582
|
-
|
|
1583
|
-
[source,ruby]
|
|
1584
|
-
----
|
|
1585
|
-
# Element name must match
|
|
1586
|
-
expect(target_element.name).to eq(source_element.name)
|
|
1587
|
-
|
|
1588
|
-
# Attributes must be semantically equivalent
|
|
1589
|
-
expect(target_attributes).to eq(source_attributes)
|
|
1590
|
-
|
|
1591
|
-
# Text content must be preserved (whitespace-normalized)
|
|
1592
|
-
expect(normalized_text(target)).to eq(normalized_text(source))
|
|
1593
|
-
|
|
1594
|
-
# Document structure (element count) must match
|
|
1595
|
-
expect(doc.xpath("//*").size).to eq(original.xpath("//*").size)
|
|
1596
|
-
----
|
|
1597
|
-
|
|
1598
|
-
This approach tolerates adapter-specific serialization differences while ensuring
|
|
1599
|
-
the actual XML content remains intact.
|
|
1600
|
-
|
|
1601
|
-
|
|
1602
|
-
== Development and testing
|
|
1603
|
-
|
|
1604
|
-
For complete information on development setup, testing strategies, benchmarking,
|
|
1605
|
-
and coverage reporting, see the
|
|
1606
|
-
link:docs/_guides/development-testing.adoc[Development and Testing Guide].
|
|
1607
|
-
|
|
1608
|
-
== Contributing
|
|
1609
|
-
|
|
1610
|
-
. Fork the repository
|
|
1611
|
-
. Create your feature branch (`git checkout -b feature/my-new-feature`)
|
|
1612
|
-
. Commit your changes (`git commit -am 'Add some feature'`)
|
|
1613
|
-
. Push to the branch (`git push origin feature/my-new-feature`)
|
|
1614
|
-
. Create a new Pull Request
|
|
1615
|
-
|
|
1616
|
-
== License
|
|
1617
|
-
|
|
1618
|
-
Copyright Ribose.
|
|
1619
|
-
|
|
1620
|
-
This project is licensed under the Ribose 3-Clause BSD License. See the
|
|
1621
1238
|
link:LICENSE.md[] file for details.
|
data/Rakefile
CHANGED
|
@@ -2,161 +2,22 @@
|
|
|
2
2
|
|
|
3
3
|
require "bundler/gem_tasks"
|
|
4
4
|
|
|
5
|
-
#
|
|
6
|
-
#
|
|
7
|
-
# outputs exist before their extconf.rb runs). Guard the rspec/opal
|
|
8
|
-
# requires so the file loads with only `rake` + `bundler` available.
|
|
5
|
+
# Guard the rspec requires so the file loads with only `rake` +
|
|
6
|
+
# `bundler` available.
|
|
9
7
|
begin
|
|
10
8
|
require "rspec/core/rake_task"
|
|
11
9
|
RSpec::Core::RakeTask.new(:spec)
|
|
12
10
|
rescue LoadError
|
|
13
11
|
end
|
|
14
12
|
|
|
15
|
-
begin
|
|
16
|
-
require "opal/rspec/rake_task"
|
|
17
|
-
|
|
18
|
-
# REXML is a bundled gem since Ruby 3.4; its source must be on Opal's
|
|
19
|
-
# global load path so the compiler can follow `require "rexml/document"`.
|
|
20
|
-
# Opal-compatible overrides go first so they shadow the gem originals.
|
|
21
|
-
if defined?(Opal)
|
|
22
|
-
Opal.append_path File.expand_path("lib/compat/opal", __dir__)
|
|
23
|
-
rexml_lib = $LOAD_PATH.find do |p|
|
|
24
|
-
File.exist?(File.join(p, "rexml", "document.rb"))
|
|
25
|
-
end
|
|
26
|
-
Opal.append_path rexml_lib if rexml_lib
|
|
27
|
-
|
|
28
|
-
# The Opal-compatible oga and ruby-ll forks (vendored as submodules)
|
|
29
|
-
# expose pure-Ruby implementations under ext/pureruby/. Their top-level
|
|
30
|
-
# lib/oga.rb and lib/ll/setup.rb conditionally require them when
|
|
31
|
-
# RUBY_PLATFORM == 'opal'. Both lib/ and ext/pureruby/ must be on
|
|
32
|
-
# Opal's load path so the conditional resolves correctly.
|
|
33
|
-
%w[opal-oga opal-ruby-ll].each do |fork_name|
|
|
34
|
-
fork_path = File.expand_path("vendor/#{fork_name}", __dir__)
|
|
35
|
-
Opal.append_path File.join(fork_path, "lib")
|
|
36
|
-
Opal.append_path File.join(fork_path, "ext/pureruby")
|
|
37
|
-
end
|
|
38
|
-
end
|
|
39
|
-
rescue LoadError
|
|
40
|
-
# Opal not available or incompatible with current Ruby version
|
|
41
|
-
end
|
|
42
|
-
|
|
43
13
|
begin
|
|
44
14
|
require "rubocop/rake_task"
|
|
45
15
|
RuboCop::RakeTask.new
|
|
46
16
|
rescue LoadError
|
|
47
17
|
end
|
|
48
18
|
|
|
49
|
-
# Regenerate the ragel / ruby-ll outputs that the Opal-compatible forks
|
|
50
|
-
# (vendored as submodules under vendor/) gitignore. The forks ship the
|
|
51
|
-
# grammar sources (.rl / .rll) but not the generated .rb / .c, since
|
|
52
|
-
# those are large and version-controllable upstream. Both ragel and
|
|
53
|
-
# ruby-ll must be on PATH; the upstream ruby-ll gem is sufficient for
|
|
54
|
-
# generation (the fork is only needed at runtime).
|
|
55
|
-
namespace :vendor do
|
|
56
|
-
desc "Generate ragel / ruby-ll outputs in vendored opal-oga and opal-ruby-ll"
|
|
57
|
-
task :prepare do
|
|
58
|
-
require "fileutils"
|
|
59
|
-
|
|
60
|
-
oga = File.expand_path("vendor/opal-oga", __dir__)
|
|
61
|
-
ruby_ll = File.expand_path("vendor/opal-ruby-ll", __dir__)
|
|
62
|
-
|
|
63
|
-
generators = [
|
|
64
|
-
# oga: ruby-ll grammar → Ruby parser
|
|
65
|
-
["ruby-ll #{oga}/lib/oga/xml/parser.rll -o #{oga}/lib/oga/xml/parser.rb",
|
|
66
|
-
"#{oga}/lib/oga/xml/parser.rb",
|
|
67
|
-
"#{oga}/lib/oga/xml/parser.rll"],
|
|
68
|
-
["ruby-ll #{oga}/lib/oga/xpath/parser.rll -o #{oga}/lib/oga/xpath/parser.rb",
|
|
69
|
-
"#{oga}/lib/oga/xpath/parser.rb",
|
|
70
|
-
"#{oga}/lib/oga/xpath/parser.rll"],
|
|
71
|
-
["ruby-ll #{oga}/lib/oga/css/parser.rll -o #{oga}/lib/oga/css/parser.rb",
|
|
72
|
-
"#{oga}/lib/oga/css/parser.rb",
|
|
73
|
-
"#{oga}/lib/oga/css/parser.rll"],
|
|
74
|
-
# oga: ragel Ruby lexer
|
|
75
|
-
["ragel -R -F1 #{oga}/lib/oga/xpath/lexer.rl -o #{oga}/lib/oga/xpath/lexer.rb",
|
|
76
|
-
"#{oga}/lib/oga/xpath/lexer.rb",
|
|
77
|
-
"#{oga}/lib/oga/xpath/lexer.rl"],
|
|
78
|
-
["ragel -R -F1 #{oga}/lib/oga/css/lexer.rl -o #{oga}/lib/oga/css/lexer.rb",
|
|
79
|
-
"#{oga}/lib/oga/css/lexer.rb",
|
|
80
|
-
"#{oga}/lib/oga/css/lexer.rl"],
|
|
81
|
-
# oga: ragel C lexer for liboga
|
|
82
|
-
["ragel -C -I #{oga}/ext/ragel -G2 #{oga}/ext/c/lexer.rl -o #{oga}/ext/c/lexer.c",
|
|
83
|
-
"#{oga}/ext/c/lexer.c",
|
|
84
|
-
"#{oga}/ext/c/lexer.rl"],
|
|
85
|
-
# ruby-ll: ruby-ll grammar → Ruby parser
|
|
86
|
-
["ruby-ll #{ruby_ll}/lib/ll/parser.rll -o #{ruby_ll}/lib/ll/parser.rb --no-requires",
|
|
87
|
-
"#{ruby_ll}/lib/ll/parser.rb",
|
|
88
|
-
"#{ruby_ll}/lib/ll/parser.rll"],
|
|
89
|
-
]
|
|
90
|
-
|
|
91
|
-
# Under bundle exec, ruby-ll resolves to the vendored opal-ruby-ll
|
|
92
|
-
# binstub, which needs the not-yet-compiled libll.
|
|
93
|
-
Bundler.with_unbundled_env do
|
|
94
|
-
generators.each do |cmd, output, source|
|
|
95
|
-
if File.exist?(output) && File.mtime(output) >= File.mtime(source)
|
|
96
|
-
next
|
|
97
|
-
end
|
|
98
|
-
|
|
99
|
-
FileUtils.mkdir_p(File.dirname(output))
|
|
100
|
-
sh cmd
|
|
101
|
-
end
|
|
102
|
-
end
|
|
103
|
-
end
|
|
104
|
-
|
|
105
|
-
# Bundler does not reliably compile native extensions for path-source
|
|
106
|
-
# gems. Build liboga/libll explicitly so `require "oga"` resolves.
|
|
107
|
-
desc "Compile liboga / libll native extensions for vendored forks"
|
|
108
|
-
task :compile do
|
|
109
|
-
require "fileutils"
|
|
110
|
-
require "rbconfig"
|
|
111
|
-
|
|
112
|
-
dlext = RbConfig::CONFIG["DLEXT"]
|
|
113
|
-
|
|
114
|
-
{
|
|
115
|
-
"vendor/opal-ruby-ll" => "libll",
|
|
116
|
-
"vendor/opal-oga" => "liboga",
|
|
117
|
-
}.each do |fork_path, ext_name|
|
|
118
|
-
abs_fork = File.expand_path(fork_path, __dir__)
|
|
119
|
-
lib_bundle = File.join(abs_fork, "lib", "#{ext_name}.#{dlext}")
|
|
120
|
-
ext_dir = File.join(abs_fork, "ext", "c")
|
|
121
|
-
extconf = File.join(ext_dir, "extconf.rb")
|
|
122
|
-
lib_dir = File.join(abs_fork, "lib")
|
|
123
|
-
next if File.exist?(lib_bundle)
|
|
124
|
-
|
|
125
|
-
Dir.chdir(ext_dir) do
|
|
126
|
-
sh "ruby #{extconf}"
|
|
127
|
-
sh "make"
|
|
128
|
-
FileUtils.cp("#{ext_name}.#{dlext}", lib_dir)
|
|
129
|
-
end
|
|
130
|
-
end
|
|
131
|
-
end
|
|
132
|
-
end
|
|
133
|
-
|
|
134
19
|
if defined?(RSpec)
|
|
135
20
|
namespace :spec do
|
|
136
|
-
if defined?(Opal::RSpec::RakeTask)
|
|
137
|
-
desc "Run Opal (JavaScript) tests"
|
|
138
|
-
Opal::RSpec::RakeTask.new(:opal) do |server, runner|
|
|
139
|
-
server.append_path "lib"
|
|
140
|
-
server.append_path "spec"
|
|
141
|
-
|
|
142
|
-
runner.default_path = "spec"
|
|
143
|
-
# `oga` and `ll/setup` must be required before moxml_boot so that
|
|
144
|
-
# the forks' Opal-aware conditional requires fire (lib/oga.rb calls
|
|
145
|
-
# `require 'oga/native/lexer'` when RUBY_PLATFORM == 'opal'; that
|
|
146
|
-
# resolves against vendor/opal-oga/ext/pureruby/, which the global
|
|
147
|
-
# Opal.append_path calls above add to the load path).
|
|
148
|
-
runner.requires = %w[rexml_compat rexml/document rexml/xpath
|
|
149
|
-
oga ll/setup
|
|
150
|
-
moxml_boot spec_helper support/opal]
|
|
151
|
-
runner.files = Dir.glob("spec/moxml/*opal*_spec.rb") +
|
|
152
|
-
Dir.glob("spec/moxml/native_attachment/opal_spec.rb") +
|
|
153
|
-
Dir.glob("spec/moxml/adapter/shared_examples/*.rb")
|
|
154
|
-
end
|
|
155
|
-
|
|
156
|
-
desc "Alias for spec:opal that also runs vendor:prepare"
|
|
157
|
-
task opal: "vendor:prepare"
|
|
158
|
-
end
|
|
159
|
-
|
|
160
21
|
desc "Validate XML fixtures are well-formed (requires xmllint)"
|
|
161
22
|
task :validate_fixtures do
|
|
162
23
|
fixtures = Dir.glob("spec/fixtures/**/*.xml")
|
|
@@ -305,44 +166,6 @@ namespace :benchmark do
|
|
|
305
166
|
end
|
|
306
167
|
end
|
|
307
168
|
|
|
308
|
-
namespace :opal do
|
|
309
|
-
desc "Regenerate entity data for Opal from w3c_entities.json"
|
|
310
|
-
task :generate_entity_data do
|
|
311
|
-
require "json"
|
|
312
|
-
|
|
313
|
-
source = File.join(__dir__, "data", "w3c_entities.json")
|
|
314
|
-
target = File.join(__dir__, "lib", "moxml", "entity_registry_opal_data.rb")
|
|
315
|
-
|
|
316
|
-
data = JSON.parse(File.read(source))
|
|
317
|
-
chars = data["characters"]
|
|
318
|
-
|
|
319
|
-
lines = []
|
|
320
|
-
lines << "# frozen_string_literal: true"
|
|
321
|
-
lines << "#"
|
|
322
|
-
lines << "# Auto-generated entity data for Opal runtime."
|
|
323
|
-
lines << "# Source: data/w3c_entities.json (#{chars.size} entities)"
|
|
324
|
-
lines << "# Regenerate with: rake opal:generate_entity_data"
|
|
325
|
-
lines << ""
|
|
326
|
-
lines << "module Moxml"
|
|
327
|
-
lines << " class EntityRegistry"
|
|
328
|
-
lines << " OPAL_ENTITY_DATA = {"
|
|
329
|
-
chars.each do |name, char|
|
|
330
|
-
codepoint = if char.start_with?("\\u")
|
|
331
|
-
char.unicode_normalize(:nfc)[2..].to_i(16)
|
|
332
|
-
else
|
|
333
|
-
char.ord
|
|
334
|
-
end
|
|
335
|
-
lines << " #{name.inspect} => #{codepoint},"
|
|
336
|
-
end
|
|
337
|
-
lines << " }.freeze"
|
|
338
|
-
lines << " end"
|
|
339
|
-
lines << "end"
|
|
340
|
-
|
|
341
|
-
File.write(target, "#{lines.join("\n")}\n")
|
|
342
|
-
puts "Generated #{target} (#{chars.size} entities, #{lines.size} lines)"
|
|
343
|
-
end
|
|
344
|
-
end
|
|
345
|
-
|
|
346
169
|
namespace :docs do
|
|
347
170
|
desc "Regenerate the published model-coverage page from the matrix"
|
|
348
171
|
task :coverage do
|
|
@@ -25,8 +25,9 @@ Pins:: none
|
|
|
25
25
|
reference** (a pure-Ruby reader, writer, and normalizer). Adapters
|
|
26
26
|
are differentially tested against the reference
|
|
27
27
|
(`spec/consistency/reference_differential_spec.rb`), not only
|
|
28
|
-
against each other; the reference is pure Ruby
|
|
29
|
-
the
|
|
28
|
+
against each other; the reference is pure Ruby. (The Opal
|
|
29
|
+
transpile lane was removed in #489 phase 4 — the JS surface is
|
|
30
|
+
the moxml-ts wasm binding.)
|
|
30
31
|
3. Adapter implementations and capability gates — representations.
|
|
31
32
|
|
|
32
33
|
A pin cites the model section it enforces
|
|
@@ -2,10 +2,12 @@
|
|
|
2
2
|
|
|
3
3
|
module Moxml
|
|
4
4
|
module Adapter
|
|
5
|
-
# Lifecycle record (#495): every override serves the :oga
|
|
6
|
-
#
|
|
7
|
-
# moxml#489 phase 4
|
|
8
|
-
#
|
|
5
|
+
# Lifecycle record (#495): every override serves the :oga
|
|
6
|
+
# adapter. The Opal surface that motivated them is deleted
|
|
7
|
+
# (moxml#489 phase 4, 2026-10-31); the overrides remain because
|
|
8
|
+
# the CRuby :oga adapter calls them on core paths, and the
|
|
9
|
+
# adapter's fate -- the owner's removal decision, consumer
|
|
10
|
+
# evidence in moxml#490 -- governs this file.
|
|
9
11
|
module CustomizedOga
|
|
10
12
|
autoload :EntityDecoderOverride, "moxml/adapter/customized_oga/entity_decoder"
|
|
11
13
|
autoload :RawValueOverride, "moxml/adapter/customized_oga/raw_value_override"
|
|
@@ -3,9 +3,11 @@
|
|
|
3
3
|
module Moxml
|
|
4
4
|
module Adapter
|
|
5
5
|
# Lifecycle record (#495): engine-gap shims for the :rexml
|
|
6
|
-
# adapter
|
|
7
|
-
# moxml#489 phase 4)
|
|
8
|
-
#
|
|
6
|
+
# adapter. The Opal surface (this adapter's historical fallback
|
|
7
|
+
# role) is deleted (moxml#489 phase 4, 2026-10-31); the shims
|
|
8
|
+
# remain because the CRuby :rexml adapter calls them. The
|
|
9
|
+
# adapter's fate -- the owner's removal decision, consumer
|
|
10
|
+
# evidence in moxml#490 -- governs this file.
|
|
9
11
|
module CustomizedRexml
|
|
10
12
|
autoload :EntityReference,
|
|
11
13
|
"moxml/adapter/customized_rexml/entity_reference"
|