leptris 1.6.0 → 1.6.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (55) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +70 -0
  3. data/CLAUDE.md +76 -87
  4. data/README.adoc +80 -6
  5. data/Rakefile +1 -1
  6. data/benchmark/README.md +8 -164
  7. data/lib/leptris/version.rb +1 -1
  8. data/lib/leptris/xml/cdata.rb +1 -0
  9. data/lib/leptris/xml/comment.rb +1 -0
  10. data/lib/leptris/xml/document.rb +29 -14
  11. data/lib/leptris/xml/element.rb +21 -3
  12. data/lib/leptris/xml/ffi.rb +3 -13
  13. data/lib/leptris/xml/node.rb +32 -10
  14. data/lib/leptris/xml/node_set.rb +6 -5
  15. data/lib/leptris/xml/processing_instruction.rb +2 -0
  16. data/lib/leptris/xml/searchable.rb +1 -1
  17. data/lib/leptris/xml/serialization.rb +20 -2
  18. data/lib/leptris/xml/text.rb +1 -0
  19. data/lib/leptris/xml.rb +3 -2
  20. metadata +1 -36
  21. data/TODO.impl/01-architecture.md +0 -217
  22. data/TODO.impl/02-ffi-declarations.md +0 -236
  23. data/TODO.impl/03-document-node-element-nodeset.md +0 -382
  24. data/TODO.impl/04-sax-parser.md +0 -203
  25. data/TODO.impl/05-serialize-c14n-memory-specs-css.md +0 -276
  26. data/docs/ARCHITECTURE.adoc +0 -559
  27. data/docs/BUILD.md +0 -395
  28. data/docs/ERROR_MESSAGES.md +0 -458
  29. data/docs/FFI_ARCHITECTURE.md +0 -439
  30. data/docs/FUTURE_VISION.md +0 -303
  31. data/docs/GITHUB_ACTIONS.md +0 -293
  32. data/docs/OPTIMIZATIONS_IMPLEMENTED.adoc +0 -459
  33. data/docs/PERFORMANCE.adoc +0 -668
  34. data/docs/PERFORMANCE.md +0 -448
  35. data/docs/RELEASE_NOTES_v1.0.0.md +0 -515
  36. data/docs/XPATH_SPEC_COMPLIANCE.md +0 -298
  37. data/docs/completion/leptris.bash +0 -86
  38. data/docs/completion/leptris.zsh +0 -74
  39. data/docs/man/leptris-format.1 +0 -227
  40. data/docs/man/leptris-parse.1 +0 -178
  41. data/docs/man/leptris-xpath.1 +0 -312
  42. data/docs/man/leptris.1 +0 -160
  43. data/docs/v0.9.0_PERFORMANCE_IMPROVEMENTS.md +0 -217
  44. data/docs/v0.9.0_RELEASE_SUMMARY.md +0 -281
  45. data/docs/v1.0.0_CONTINUATION_PLAN.md +0 -172
  46. data/docs/v1.0.0_CONTINUATION_PROMPT.md +0 -382
  47. data/docs/v1.0.0_SESSION_6_CONTINUATION.md +0 -434
  48. data/docs/v1.0.0_SESSION_6_PROMPT.md +0 -231
  49. data/docs/v1.0.0_STATUS_TRACKER.md +0 -224
  50. data/docs/v1.1.0_CONTINUATION_PLAN.md +0 -299
  51. data/docs/v1.1.0_FINAL_CONTINUATION_PLAN.md +0 -201
  52. data/docs/v1.1.0_SESSION_3_PROMPT.md +0 -223
  53. data/docs/v1.1.0_STATUS_TRACKER.md +0 -355
  54. data/docs/xml-performance.adoc +0 -115
  55. data/docs/xpath-performance.adoc +0 -379
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 28f24417196438a9a1aef17c3e1f648b7d85048659268a332341c36663a4a16a
4
- data.tar.gz: 542fa2497b09a734f2cd65ca3eb398c90d24841bc4ed70577e7ed177e323f521
3
+ metadata.gz: 6cb4c2a8d3cb9da6f2c79fd03dc85b0f55f1a590b15a18ffedabd63177332f8e
4
+ data.tar.gz: df6b3611943af4f69c30d2a1650c86276fd89b7261f4d8da9dd43d6421b17de3
5
5
  SHA512:
6
- metadata.gz: d46d2a265f3c07326fda5e152f338373772b9f804f5c5c7d8174dcb78dff3f10e5a875972b5635ab94e2309001552e85f0a45c26e14687c713d8647cdd32815c
7
- data.tar.gz: a7ae147afbd49cf1afe4eebfd6f9e2ba4dea2e682a2da7cbd3e7b892aaf5bc3150eeffca6bdecd840a166499d17eefd6debf4f4df3e0fe59d34cdfe0613da84f
6
+ metadata.gz: 82fde4f6f7d6645c1eafe9d130f532437fb28703ab27aa6753d9afc2ad53f2eabb6e852bf8338b8e8c1e551225fd2548045430f93b2063cd31adf7c2e6132e2e
7
+ data.tar.gz: 726699586511ef79fdc5c51e4a102e49329c480bd66d39089c2dd0201c0d96117127f6a5cbb9a90051a115e0c130877c957e6342b124fb1a18211104a8bad591
data/CHANGELOG.md CHANGED
@@ -5,6 +5,41 @@ All notable changes to Leptris will be documented in this file.
5
5
  The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
6
  and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
7
 
8
+ ## [1.6.2] - 2026-08-24
9
+
10
+ Lockstep with libleptris 1.6.2 (fix releases, no public API changes).
11
+
12
+ ### Changed
13
+
14
+ - CI/build pin libleptris v1.6.2, carrying the serializer fix for
15
+ mixed-content indentation (upstream #534 — never indent inside
16
+ mixed-content elements).
17
+
18
+ ## [1.6.1] - 2026-08-24
19
+
20
+ ### Added
21
+
22
+ - **Readonly mode**: `Leptris::XML.parse(xml, readonly: true)` /
23
+ `Document#readonly!` (one-way). Mutations raise
24
+ `Leptris::XML::ReadOnlyError`; read paths memoize aggressively
25
+ (names, content, children NodeSets, attribute hashes) since they can
26
+ never go stale; the C document is frozen (advisory upstream).
27
+ Detached factories (`create_element` etc.) remain usable.
28
+ Purpose: steady-state read performance for the dominant
29
+ parse-query-serialize workload — see Changed.
30
+
31
+ ### Changed
32
+
33
+ - Micro-optimizations targeting the small-document gap versus C
34
+ extension bindings:
35
+ - node type is memoized from `Node.wrap`'s dispatch call — every
36
+ predicate and `#type` is now FFI-free
37
+ - the default serialize options struct is built once and reused
38
+ - `Document.parse` skips the per-parse status MemoryPointer (the C
39
+ out-param is nullable; failure detail comes from the thread-local
40
+ last error)
41
+ - `Element#name` memoizes (invalidated by `name=`)
42
+
8
43
  ## [1.6.0] - 2026-08-24
9
44
 
10
45
  Lockstep with libleptris 1.6.0 — the moxml-adapter blockers fixed:
@@ -26,6 +61,41 @@ release: pull (StAX) API, bounded iterparse, compiled XPath
26
61
  expressions, per-parse options, truthful serialization encoding
27
62
  declarations. Binding-side adoption of the new APIs follows.
28
63
 
64
+ ## [1.6.2] - 2026-08-24
65
+
66
+ Lockstep with libleptris 1.6.2 (fix releases, no public API changes).
67
+
68
+ ### Changed
69
+
70
+ - CI/build pin libleptris v1.6.2, carrying the serializer fix for
71
+ mixed-content indentation (upstream #534 — never indent inside
72
+ mixed-content elements).
73
+
74
+ ## [1.6.1] - 2026-08-24
75
+
76
+ ### Added
77
+
78
+ - **Readonly mode**: `Leptris::XML.parse(xml, readonly: true)` /
79
+ `Document#readonly!` (one-way). Mutations raise
80
+ `Leptris::XML::ReadOnlyError`; read paths memoize aggressively
81
+ (names, content, children NodeSets, attribute hashes) since they can
82
+ never go stale; the C document is frozen (advisory upstream).
83
+ Detached factories (`create_element` etc.) remain usable.
84
+ Purpose: steady-state read performance for the dominant
85
+ parse-query-serialize workload — see Changed.
86
+
87
+ ### Changed
88
+
89
+ - Micro-optimizations targeting the small-document gap versus C
90
+ extension bindings:
91
+ - node type is memoized from `Node.wrap`'s dispatch call — every
92
+ predicate and `#type` is now FFI-free
93
+ - the default serialize options struct is built once and reused
94
+ - `Document.parse` skips the per-parse status MemoryPointer (the C
95
+ out-param is nullable; failure detail comes from the thread-local
96
+ last error)
97
+ - `Element#name` memoizes (invalidated by `name=`)
98
+
29
99
  ## [1.6.0] - 2026-08-24
30
100
 
31
101
  Lockstep with libleptris 1.6.0 (v1.4/v1.5 were fix releases with no
data/CLAUDE.md CHANGED
@@ -4,101 +4,90 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
4
4
 
5
5
  ## Project
6
6
 
7
- `leptris-ruby` is a Ruby gem that wraps the native C library `libleptris` (built via CMake) via the `ffi` gem. It exposes an XML parser with complete XPath 1.0 support and a CLI. Target users want Nokogiri-like ergonomics with native speed.
8
-
9
- - Native dep: `libleptris` shared library, built by `ext/leptris/extconf.rb` (CMake) and copied to `lib/libleptris.{dylib,so,dll}`.
10
- - Ruby entry point: `lib/leptris.rb` (uses `require_relative` see "Conventions" below).
11
- - CLI: `bin/leptris` (Thor-based), defined in `lib/leptris/cli.rb`.
7
+ `leptris-ruby` is the official Ruby binding for `libleptris`
8
+ (https://github.com/leptris/leptris), a pure-C99 XML 1.0 parser with a
9
+ W3C-conformant XPath 1.0 engine (438/438 on the W3C suite), full SAX,
10
+ pull parsing, and C14N. The binding is **FFI-based** (no C extension,
11
+ no compilation at install) and ships **precompiled platform gems**
12
+ with the shared library vendored inside: x86_64/aarch64-linux (glibc
13
+ and musl), x86_64/arm64-darwin, x64-mingw32/ucrt, aarch64-mingw-ucrt,
14
+ plus the pure-Ruby fallback gem.
15
+
16
+ Version lockstep: the gem's major.minor tracks the C library
17
+ (`libleptris 1.6.x` <-> `leptris 1.6.x`). A C release means one PR
18
+ carrying version.rb + the Rakefile `LIBLEPTRIS_VERSION` pin + the
19
+ CHANGELOG entry, then the release workflow.
12
20
 
13
21
  ## Commands
14
22
 
15
23
  ```bash
16
- # Build the C library and install it into lib/ for FFI to load
17
- bundle exec rake compile # runs ext/leptris/extconf.rb (CMake)
18
-
19
- # Run the full test suite (builds first)
20
- bundle exec rake test # = spec, depends on :compile
21
-
22
- # Run RSpec directly (skips build dependency)
23
- bundle exec rspec # full suite
24
- bundle exec rspec spec/leptris/document_spec.rb # one file
25
- bundle exec rspec spec/leptris/document_spec.rb:42 # one example by line
26
-
27
- # Lint
28
- bundle exec rubocop
29
-
30
- # Clean build artifacts (lib/libleptris.*, ext/leptris/build, Makefile, tmp, pkg)
31
- bundle exec rake clean
24
+ bundle exec rake compile # build libleptris (pinned tarball) into lib/
25
+ bundle exec rspec # full suite (uses the vendored library)
26
+ bundle exec rspec spec/xml/xpath_spec.rb:42 # one example
27
+ bundle exec rake gem:native:arm64-darwin # platform gem
32
28
  ```
33
29
 
34
- CI (`.github/workflows/test.yml`) runs `bundle exec rspec` + `bundle exec rubocop` on Ubuntu + macOS across Ruby 3.0–3.3.
30
+ `LEPTRIS_LIB_PATH=/path/to/libleptris.dylib bundle exec rspec` runs
31
+ against an arbitrary build (e.g. a libleptris checkout under test).
32
+
33
+ ## Architecture
35
34
 
36
- ## Architecture: current state (v1.1.0)
35
+ The C DOM is the single source of truth; Ruby objects are thin FFI
36
+ handles. One Ruby method = one FFI call wherever possible.
37
37
 
38
38
  ```
39
- User Ruby code
40
-
41
- Leptris.parse / Leptris.parse_file (lib/leptris.rb)
42
- FFI call (leptris_parse) one-shot tree copy
43
- C document → FFI::Bridge.document_from_ptr (lib/leptris/ffi/bridge.rb)
44
- ↓ recursive hydration
45
- Ruby Document → Element → Node → NodeSet (lib/leptris/{document,element,node,node_set}.rb)
39
+ lib/leptris.rb — Leptris::VERSION
40
+ lib/leptris/xml.rb — autoload registry, error classes,
41
+ Leptris::XML.parse / parse_file
42
+ lib/leptris/xml/ffi.rb — every public C declaration + seam
43
+ helpers (check_status, read_owned_string,
44
+ status_message)
45
+ lib/leptris/xml/document.rb — the only C-memory owner (finalizer);
46
+ factories, PI accessors, exslt, last_error
47
+ lib/leptris/xml/node.rb — Node.wrap: the ONLY wrapper constructor
48
+ (identity cache + type dispatch)
49
+ lib/leptris/xml/element.rb — attributes via the v1.1.0 iteration
50
+ face; namespace/mutation surface
51
+ lib/leptris/xml/node_set.rb — lazy XPath results; batch fetch via
52
+ get_nodes_ex
53
+ lib/leptris/xml/searchable.rb — xpath/css/at_*; namespace-bound path
54
+ lib/leptris/xml/xpath.rb — compiled expressions (parse once,
55
+ eval many)
56
+ lib/leptris/xml/sax/ — callback SAX
57
+ lib/leptris/xml/pull.rb — StAX-style pull parsing
58
+ lib/leptris/xml/iterparse.rb — bounded-memory element iteration
59
+ lib/leptris/xml/serialization.rb — serialize/c14n (Document + Element
60
+ one-liners over this module)
61
+ lib/leptris/xml/c_string_array.rb — NULL-terminated char** adapter
62
+ lib/leptris/xml/css_to_xpath.rb — minimal CSS translation
46
63
  ```
47
64
 
48
- Key directories:
49
-
50
- - `lib/leptris.rb` top-level module, `parse`, `parse_file`, `xpath_evaluate`, error classes (`ParseError`, `XPathError`, `EvaluationError`).
51
- - `lib/leptris/ffi/` FFI plumbing: `library.rb` (bindings), `types.rb` (constants), `memory.rb` (AutoPointer wrappers), `errors.rb` (thread-local error check), `bridge.rb` (C ptr → Ruby object).
52
- - `lib/leptris/{document,element,node,node_set}.rb` pure-Ruby tree model (full hydration on parse).
53
- - `lib/leptris/xpath/` pure-Ruby XPath engine (lexer, parser, compiler, VM). XPath DOES NOT go through C currently; `lib/leptris.rb#xpath_evaluate` calls `FFI.leptris_xpath_eval` only as a wrapper, but the result materialization in `FFI::Bridge` recursively re-walks via the Ruby tree.
54
- - `lib/leptris/adapter*` third-party format adapters.
55
- - `spec/leptris/` — 250+ RSpec examples covering parser, XPath, namespaces, errors, ox-compatibility.
56
- - `ext/leptris/` CMake-based build of `libleptris` (sources come from the separate `leptris/leptris` repo at build time).
57
-
58
- ## Architecture: planned rewrite (see `TODO.impl/`)
59
-
60
- The five files under `TODO.impl/` describe a planned rewrite that has NOT been implemented yet. Read them before touching the Ruby/COM layer. Summary:
61
-
62
- 1. **`01-architecture.md`** — Rewrite `leptris-ruby` as a **thin FFI wrapper** around libleptris v0.4.2 with a **Nokogiri-compatible API**. Current code does a one-shot C→Ruby tree copy and runs XPath in Ruby; planned code keeps the C DOM as the single source of truth (Ruby objects = handles wrapping opaque pointers), so every Ruby method = one FFI call and XPath/SAX go through the C engine.
63
-
64
- 2. **`02-ffi-declarations.md`** Complete FFI attachment: every public function in libleptris v0.4.2 (document lifecycle, node access, element queries/mutation, creation, text/comment/CDATA/PI access, XPath + variable set, SAX, serialization, `leptris_free_string`). Opaque typedefs: `document`, `element`, `node_ref`, `xpath_result`, `sax_parser`, `attribute`. Structs: `SAXHandler`, `SerializeOptions`. Constants for status codes, node types, XPath result types.
65
-
66
- 3. **`03-document-node-element-nodeset.md`** Target file layout:
67
- ```
68
- lib/leptris.rb
69
- lib/leptris/xml.rb
70
- lib/leptris/xml/{ffi,document,node,element,text,comment,cdata,
71
- processing_instruction,attr,node_set,searchable,
72
- parse_options}.rb
73
- ```
74
- `Node.wrap(ptr, doc)` dispatches on the C node type. Document is the only memory-owning object; Node/Element/Text/Comment/CDATA/PI/Attr are non-owning handles valid until `Document#free`.
75
-
76
- 4. **`04-sax-parser.md`** `Leptris::XML::SAX::{Parser, Document}` wrapping `leptris_sax_parse` / `leptris_sax_parser_feed`. FFI::Function callbacks for each event; `start_element` walks the NULL-terminated `const char**` attribute array. Streaming via incremental `feed`.
77
-
78
- 5. **`05-serialize-c14n-memory-specs-css.md`** — `SerializeOptions` struct for `leptris_serialize_document`; `Document#canonicalize` (modes `C14N_1_0`, `C14N_1_1`, `C14N_EXCLUSIVE`); `UseAfterFreeError` guard; minimal CSS-to-XPath converter (`.class`, `#id`, `[attr]`, `[attr=val]`, `:first-child`, `:last-child`); spec layout under `spec/xml/{parse,document,node,element,node_set,xpath,sax,serialize,c14n,memory}_spec.rb`.
79
-
80
- ### Memory ownership rules (planned)
81
-
82
- | Ruby class | Owns C memory? | Free function |
83
- |---|---|---|
84
- | `Document` | YES | `leptris_document_free` |
85
- | `Node`/`Element`/text/CDATA/PI/Attr | NO (borrowed) | freed transitively by Document |
86
- | `NodeSet` (XPath result) | YES | `leptris_xpath_result_free` |
87
- | SAX handler closures | callback lifetime | `leptris_sax_parser_free` |
88
-
89
- GC safety net: `ObjectSpace.define_finalizer` capturing the **raw pointer value**, not the Ruby wrapper. Finalizers must not double-free after explicit `#free`.
90
-
91
- ## Reference material
92
-
93
- - Nokogiri source (`~/src/external/nokogiri/`) — `lib/nokogiri/xml/{node,node_set,document,searchable}.rb` are the API shape targets.
94
- - libleptris public headers (`src/include/leptris/{types,leptris}.h`, `src/include/leptris/{dom,xpath,sax}/*.h`) — the single source of truth for FFI declarations. Target tag: `v0.4.2`.
95
- - `docs/FFI_ARCHITECTURE.md` — describes the v0.5.0 FFI design (AutoPointer, two-pointer strategy for XPath). The planned rewrite supersedes some of this (no recursive hydration, no two-pointer — the Document pointer alone suffices because Node objects stay as C handles).
96
- - `docs/BUILD.md` — CMake build reference for libleptris itself.
97
-
98
- ## Conventions (project-specific)
99
-
100
- - **Autoload, not require_relative.** TODO 3 is explicit: `lib/leptris.rb` → `autoload :XML, 'leptris/xml'`; `lib/leptris/xml.rb` → `autoload :Document, 'leptris/xml/document'`. Autoload entries live in the **immediate parent namespace's file** (create that file if missing). The current `lib/leptris.rb` uses `require_relative` — when implementing TODO 3, do not retrofit require_relative into the new layout.
101
- - **No `instance_variable_set`/`_get` cross-object.** TODO 3 is explicit. The current `lib/leptris.rb` and `lib/leptris/ffi/bridge.rb` use `instance_variable_get(:@_c_ptr)` heavily — that pattern is debt to migrate, not a model to copy. In the rewrite, expose `c_ptr`/`document` as public `attr_reader`s and access via those.
102
- - **No `respond_to?` type checks.** Use `is_a?`. The current `lib/leptris.rb#xpath_evaluate` checks `context_node != doc` to disambiguate — fine. Don't add `respond_to?(:c_ptr)` style checks.
103
- - **No doubles in specs.** The existing `spec/leptris/` specs use real model instances (XML strings → `Leptris.parse` → real `Document`/`Element`/`NodeSet`). Keep it that way.
104
- - **Forward compatibility:** Keep `Leptris.parse` / `Leptris.parse_file` as the existing top-level API during the rewrite. The new `Leptris::XML.parse` may coexist.
65
+ ### Ownership and seams
66
+
67
+ - `Document` owns C memory (explicit `#free` or GC finalizer; wrapper
68
+ cache is a strong Hash cleared on free). Every other class is a
69
+ borrowed handle valid while its Document lives.
70
+ - All wrapper construction goes through `Node.wrap` wrapper
71
+ identity (`doc.root.equal?(doc.root)`) is guaranteed by the
72
+ per-document cache.
73
+ - Status checks go through `FFI.check_status`; C `char*` returns go
74
+ through `FFI.read_owned_string`; array wire format (both
75
+ directions) goes through `CStringArray`. Never hand-roll these.
76
+ - The release workflow's publish loop is idempotent (already-published
77
+ gems are skipped).
78
+
79
+ ## Conventions
80
+
81
+ - Autoload only no `require_relative` inside `lib/`.
82
+ - No `instance_variable_set`/`_get` across objects; no `respond_to?`
83
+ type checks; specs use real documents, never doubles.
84
+ - New C surface: attach in `ffi.rb`, sugar where it earns its keep,
85
+ specs against a locally built library, CHANGELOG, lockstep release.
86
+ - All changes via PRs; no AI attribution; `git add` explicit paths.
87
+
88
+ ## Reference
89
+
90
+ - libleptris public headers (`src/include/leptris/*.h`) are the
91
+ contract; when symbols change, bump lockstep and audit
92
+ attached-vs-exported (`nm -gU` on a fresh build).
93
+ - Upstream issues worth tracking live at leptris/leptris.
data/README.adoc CHANGED
@@ -136,6 +136,25 @@ doc.root.traverse do |node|
136
136
  end
137
137
  ----
138
138
 
139
+ == Readonly mode
140
+
141
+ For the dominant parse-query-serialize workload, parse with
142
+ `readonly: true` (or call `Document#readonly!`, one-way):
143
+
144
+ [source,ruby]
145
+ ----
146
+ doc = Leptris::XML.parse(xml, readonly: true)
147
+ doc.root.children.first["id"] # reads are memoized
148
+ doc.root.name # plain ivar after first call
149
+ doc.root << doc.create_element("x") # raises ReadOnlyError
150
+ ----
151
+
152
+ Reads (`name`, `content`, `children`, `attributes`) memoize
153
+ aggressively — they cannot go stale because mutation is forbidden.
154
+ Every mutator raises `Leptris::XML::ReadOnlyError`. Detached factories
155
+ (`create_element` and friends) still work: building a *new* tree
156
+ against a readonly document is legal; mutating the frozen one is not.
157
+
139
158
  == Searching: XPath and CSS
140
159
 
141
160
  `Document` and `Element` (via `Leptris::XML::Searchable`) support:
@@ -376,11 +395,66 @@ Notable differences:
376
395
 
377
396
  == Performance
378
397
 
379
- On the benchmark suite in `benchmark/leptris_vs_nokogiri.rb` (Ruby 3.3,
380
- libleptris v0.13+, macOS arm64), Leptris matches or beats Nokogiri on
381
- parse, XPath, serialize, and full-tree traverse for the small and
382
- medium documents that dominate real-world XML workloads. Run the
383
- benchmark locally for numbers on your hardware:
398
+ Measured with metanorma/serialbench (Ruby 3.4.8, leptris 1.6.0,
399
+ libleptris 1.6.0, macOS arm64) Leptris vs the full field:
400
+
401
+ [cols="2,1,1,1,1",options="header"]
402
+ |===
403
+ |Operation |Leptris |Ox |Nokogiri |vs Ox
404
+
405
+ |parse medium (300 KB)
406
+ |0.97 ms
407
+ |3.00 ms
408
+ |8.57 ms
409
+ |3.1x faster
410
+
411
+ |parse large (4.4 MB)
412
+ |13.51 ms
413
+ |77.06 ms
414
+ |104.22 ms
415
+ |5.7x faster
416
+
417
+ |generate medium
418
+ |1.75 ms
419
+ |14.19 ms
420
+ |23.93 ms
421
+ |8.1x faster
422
+
423
+ |generate large
424
+ |74.57 ms
425
+ |80.91 ms
426
+ |117.16 ms
427
+ |1.1x faster
428
+
429
+ |streaming medium
430
+ |0.73 ms
431
+ |103.82 ms
432
+ |43.74 ms
433
+ |142x faster
434
+
435
+ |streaming large
436
+ |8.52 ms
437
+ |204.14 ms
438
+ |239.14 ms
439
+ |24x faster
440
+
441
+ |Ruby allocations (medium)
442
+ |0.01 MB
443
+ |18.43 MB
444
+ |39.05 MB
445
+ |~1800x less
446
+ |===
447
+
448
+ Leptris takes first place on every XML operation above small-document
449
+ size, including against Ox (the C-extension speed champion). On
450
+ small-document micro-operations (sub-5 µs), C extensions hold an
451
+ inherent edge per call; Leptris's readonly mode closes the
452
+ steady-state gap by memoizing reads behind an immutability guarantee.
453
+
454
+ For XPath-heavy workloads, `Leptris::XML::XPath.compile` evaluates a
455
+ parsed expression repeatedly without re-parsing.
456
+
457
+ Run the local benchmark for numbers on your hardware:
384
458
 
385
459
  [source,shell]
386
460
  ----
@@ -392,7 +466,7 @@ bundle exec ruby benchmark/leptris_vs_nokogiri.rb
392
466
  [source,shell]
393
467
  ----
394
468
  bundle install # install Ruby deps
395
- bundle exec rspec # full test suite (176 specs)
469
+ bundle exec rspec # full test suite (229 specs)
396
470
  bundle exec rspec spec/xml/xpath_spec.rb:42 # one example by line
397
471
  bundle exec rubocop # lint
398
472
  ----
data/Rakefile CHANGED
@@ -8,7 +8,7 @@ RSpec::Core::RakeTask.new(:spec)
8
8
  # Pin for `rake compile` and the platform-gem builds. Keep in lockstep
9
9
  # with .github/workflows/build.yml (which calls `rake compile`) and the
10
10
  # CHANGELOG when libleptris releases.
11
- LIBLEPTRIS_VERSION = "1.6.0"
11
+ LIBLEPTRIS_VERSION = "1.6.2"
12
12
 
13
13
  CMAKE_FLAGS = %w[
14
14
  -DCMAKE_BUILD_TYPE=Release
data/benchmark/README.md CHANGED
@@ -1,168 +1,12 @@
1
- # leptris-ruby vs Nokogiri — Ruby-level benchmarks
1
+ # leptris vs the field
2
2
 
3
- Compares `Leptris::XML` (FFI libleptris v0.12.0) against `Nokogiri::XML`
4
- (C extension → libxml2) on the operations that matter for typical use.
3
+ The authoritative cross-library numbers live in the README's
4
+ Performance section (measured with metanorma/serialbench).
5
5
 
6
- Run with:
6
+ This local benchmark compares against Nokogiri only:
7
7
 
8
- ```
9
- bundle exec ruby -Ilib benchmark/leptris_vs_nokogiri.rb
10
- ```
8
+ LEPTRIS_LIB_PATH=/path/to/libleptris.dylib bundle exec ruby benchmark/leptris_vs_nokogiri.rb
11
9
 
12
- ## Latest run (M1, libleptris v0.12.0, Nokogiri 1.19.4)
13
-
14
- All libleptris upstream issues closed (#166–#262, 21 issues total).
15
- v0.12.0 ships the [#262](https://github.com/leptris/leptris/issues/262) proposals:
16
- batch XPath result accessor (`leptris_xpath_result_get_nodes`) and per-node
17
- `binding_wrapper` field. The Ruby binding uses the batch accessor in
18
- `NodeSet#each`; the `binding_wrapper` is left for the libleptris team's
19
- other bindings (Python ctypes, etc.) — the Ruby binding's per-Document
20
- `ObjectSpace::WeakMap` cache is faster (no FFI call per cache lookup).
21
-
22
- Median of 3 runs (run-to-run variance is significant on cold starts and
23
- shared-runner workloads; the trend is stable):
24
-
25
- | Operation | Leptris | Nokogiri | Leptris / Nokogiri | Winner |
26
- |---|---:|---:|---:|---|
27
- | Parse small (431 B) | ~6 µs | ~14 µs | **~2×** | Leptris (variable) |
28
- | Parse medium (12 KB) | ~32 µs | ~200 µs | **~6×** | Leptris |
29
- | XPath `count(//book)` | ~1.5 µs | ~9 µs | **~6×** | Leptris |
30
- | XPath `//book` (100-node nodeset) | ~4 µs | ~13 µs | **~5×** | Leptris |
31
- | XPath `//book[@id='50']` (1 match) | ~6 µs | ~80 µs | **~10×** | Leptris |
32
- | XPath `//book[price > 50]` | ~80 µs | ~100 µs | **~1.3×** | Leptris |
33
- | XPath `//author \| //title` (union) | ~14 µs | ~25 µs | **~3×** | Leptris |
34
- | Tree traversal | ~900 µs | ~500 µs | 0.55× | Nokogiri (1.8× faster) |
35
- | Serialize | ~28 µs | ~85 µs | **~3×** | Leptris |
36
-
37
- **Leptris beats Nokogiri on 8 of 9 operations.** Only tree traversal still
38
- loses, by ~1.8×.
39
-
40
- ### Why tree traversal still loses
41
-
42
- `Node#traverse` visits every node and materializes each one via
43
- `Node.wrap`. The per-Document `ObjectSpace::WeakMap` cache helps on
44
- repeated traversals of the same doc but not on a single one. Each
45
- visited node pays:
46
- - 1 FFI call to `leptris_node_first_child` / `_next_sibling`
47
- - 1 FFI call to `leptris_node_get_type` (for wrap dispatch)
48
- - 1 Ruby object allocation (cache miss)
49
-
50
- Nokogiri's libxml2 C extension handles traversal in C and only crosses
51
- into Ruby when the user's block is called. No per-node FFI.
52
-
53
- The libleptris `binding_wrapper` field shipped in v0.12.0 doesn't help
54
- here in the Ruby binding — Ruby FFI still needs an FFI call to read
55
- `binding_wrapper`. The Ruby-side `WeakMap` cache avoids that FFI call
56
- on cache hits. So the binding's existing cache is already optimal for
57
- Ruby; the `binding_wrapper` field is more useful for bindings that
58
- don't have a native GC hook (Python ctypes, Go cgo, Rust bindgen).
59
-
60
- ## What changed from earlier runs
61
-
62
- ### v0.11.0 → v0.11.2 (binding-side)
63
-
64
- - **Lazy NodeSet** — `NodeSet.from_result` keeps the
65
- `LeptrisXPathResult*` alive (via `FFI::AutoPointer`) and materializes
66
- `self[i]` on demand. Eager materialization was the #1 cost.
67
- - **Per-Document wrapper cache** — `ObjectSpace::WeakMap` keyed on c_ptr
68
- address. Eliminates re-allocation on repeated access to the same node.
69
-
70
- ### v0.11.4 (libleptris-side)
71
-
72
- - **`leptris_xpath_result_get_nodes`** — batch accessor. The Ruby binding
73
- now uses this in `NodeSet#each` to fetch all node pointers in one FFI
74
- call instead of N calls.
75
-
76
- ### v0.12.0 (libleptris-side)
77
-
78
- - **`binding_wrapper` field on `LeptrisNode`** — present in the C struct
79
- but not used by the Ruby binding (see "Why tree traversal still loses"
80
- above). Useful for non-Ruby bindings.
81
- - **`#261` fix** — `benchmark-ips` on 38 KB docs no longer segfaults.
82
- All upstream issues closed.
83
-
84
- ### Before/after the binding + libleptris v0.12.0 optimizations
85
-
86
- | Operation | v0.11.0 (eager) | v0.12.0 (lazy + batch) | Speedup |
87
- |---|---:|---:|---:|
88
- | XPath `//book` (100 nodes) | 87.77 µs (0.15×) | ~4 µs (5×) | **22×** |
89
- | XPath union (200 nodes) | 188.40 µs (0.14×) | ~14 µs (3×) | **13×** |
90
- | XPath complex | 126.88 µs (0.67×) | ~80 µs (1.3×) | 1.6× |
91
- | Tree traversal | 1203 µs (0.50×) | ~900 µs (0.55×) | 1.3× |
92
-
93
- ## What this means for the v0.1.0 release
94
-
95
- - Leptris is the right choice for almost every Nokogiri workload on
96
- small-to-medium docs (≤20 KB): 2–10× faster than Nokogiri.
97
- - Tree-traversal-heavy workloads (single-pass DOM scraping where you
98
- touch every node once) are 1.8× slower than Nokogiri. Acceptable
99
- for v0.1.0; the binding could ship a "fast traverse" path later that
100
- skips wrapping for read-only blocks.
101
- - All known libleptris bugs are fixed. No upstream blockers.
102
-
103
- ## Analysis
104
-
105
- ### Where Leptris wins
106
-
107
- **Parse (1.69×–6.90×)** — libleptris's single-pass direct parser (the only
108
- parser since v0.11.0, after flat + legacy were deleted) is dramatically
109
- faster than libxml2's parser. The gap widens with document size.
110
-
111
- **XPath returning scalars (4.56×)** — `count()`, `boolean()`, `string()`,
112
- `number()` queries skip NodeSet materialization entirely. libleptris's XPath
113
- bytecode VM evaluates these in a single C call, no Ruby objects allocated
114
- per match.
115
-
116
- **XPath predicate match (13.01×)** — when the predicate narrows to a small
117
- result set (single match in this test), Leptris is much faster than
118
- Nokogiri/libxml2.
119
-
120
- **Serialize (2.84×)** — single C call into `leptris_document_serialize`,
121
- no Ruby traversal.
122
-
123
- ### Where Nokogiri wins
124
-
125
- **Nodeset-returning XPath (0.13×–0.16×)** — when a query returns a 100-node
126
- NodeSet, Leptris materializes every node eagerly (100× `Node.wrap` calls,
127
- each dispatching on `leptris_node_get_type`). Nokogiri caches wrappers
128
- lazily.
129
-
130
- **Tree traversal (0.49×)** — same root cause. `Node#traverse` creates a
131
- new wrapper per visited node via `Node.wrap`; Nokogiri reuses cached
132
- wrappers.
133
-
134
- ### Optimization opportunities (Ruby-side, no libleptris work needed)
135
-
136
- 1. **Lazy NodeSet materialization.** Currently `NodeSet.from_result`
137
- iterates the C result and calls `Node.wrap` for each entry on
138
- construction. Switch to lazy: keep the `LeptrisXPathResult*` alive,
139
- materialize `self[i]` on demand. Frees the eager 100× wrap.
140
- 2. **Node wrapper cache.** Weak-ref map keyed on the c_ptr address.
141
- `Node.wrap(ptr)` checks the cache first; only creates a new wrapper
142
- if none exists. Matches Nokogiri's behavior.
143
- 3. **Specialized traverse path.** For pure-traversal use cases (no
144
- per-node mutation), skip the wrapper and call FFI directly. Lower
145
- overhead but less idiomatic.
146
-
147
- ### Blockers
148
-
149
- **Parse-loop segfault on >20 KB docs with explicit `Document#free`
150
- (libleptris #256).** The v0.11.1 fix addressed one stale-thread-local
151
- path but not the parse+free cycle path. Long-running services and batch
152
- processors parsing medium/large XML cannot rely on the standard Ruby
153
- "let GC handle document lifetime" pattern OR the explicit `Document#free`
154
- pattern. Tracked upstream; workaround: cap doc size or avoid tight loops.
155
-
156
- **DOCTYPE PUBLIC/SYSTEM not exposed (libleptris #253).** Unrelated to
157
- benchmarks but blocks 4 Ruby specs. Low impact on perf-sensitive workloads.
158
-
159
- ## What this means for the v0.1.0 release
160
-
161
- - For **parse-heavy / XPath-aggregate / serialize** workloads on small-to-
162
- medium docs (≤20 KB): Leptris is clearly the right choice. 1.7-7× faster
163
- than Nokogiri.
164
- - For **heavy nodeset manipulation** (scraping, large DOM traversal):
165
- Nokogiri is faster today. The Ruby-side optimizations above would close
166
- most of the gap.
167
- - For **long-running services on medium/large docs**: blocked by #256
168
- until libleptris ships a complete fix.
10
+ For the full-field race (Ox, Nokogiri, Oga, REXML, Leptris), use
11
+ https://github.com/metanorma/serialbench — leptris's adapter ships
12
+ there under `lib/serialbench/serializers/xml/leptris_serializer.rb`.
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module Leptris
4
- VERSION = "1.6.0"
4
+ VERSION = "1.6.2"
5
5
  end
@@ -8,6 +8,7 @@ class Leptris::XML::CDATA < Leptris::XML::Text
8
8
  end
9
9
 
10
10
  def content=(new_content)
11
+ ensure_writable!
11
12
  Leptris::XML::FFI.check_status(
12
13
  Leptris::XML::FFI.leptris_cdata_node_set_content(@c_ptr, new_content.to_s))
13
14
  new_content
@@ -8,6 +8,7 @@ class Leptris::XML::Comment < Leptris::XML::Node
8
8
  end
9
9
 
10
10
  def content=(new_content)
11
+ ensure_writable!
11
12
  Leptris::XML::FFI.check_status(
12
13
  Leptris::XML::FFI.leptris_comment_node_set_content(@c_ptr, new_content.to_s))
13
14
  new_content