leptris 1.6.0 → 1.6.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +70 -0
- data/CLAUDE.md +76 -87
- data/README.adoc +80 -6
- data/Rakefile +1 -1
- data/benchmark/README.md +8 -164
- data/lib/leptris/version.rb +1 -1
- data/lib/leptris/xml/cdata.rb +1 -0
- data/lib/leptris/xml/comment.rb +1 -0
- data/lib/leptris/xml/document.rb +29 -14
- data/lib/leptris/xml/element.rb +21 -3
- data/lib/leptris/xml/ffi.rb +3 -13
- data/lib/leptris/xml/node.rb +32 -10
- data/lib/leptris/xml/node_set.rb +6 -5
- data/lib/leptris/xml/processing_instruction.rb +2 -0
- data/lib/leptris/xml/searchable.rb +1 -1
- data/lib/leptris/xml/serialization.rb +20 -2
- data/lib/leptris/xml/text.rb +1 -0
- data/lib/leptris/xml.rb +3 -2
- metadata +1 -36
- data/TODO.impl/01-architecture.md +0 -217
- data/TODO.impl/02-ffi-declarations.md +0 -236
- data/TODO.impl/03-document-node-element-nodeset.md +0 -382
- data/TODO.impl/04-sax-parser.md +0 -203
- data/TODO.impl/05-serialize-c14n-memory-specs-css.md +0 -276
- data/docs/ARCHITECTURE.adoc +0 -559
- data/docs/BUILD.md +0 -395
- data/docs/ERROR_MESSAGES.md +0 -458
- data/docs/FFI_ARCHITECTURE.md +0 -439
- data/docs/FUTURE_VISION.md +0 -303
- data/docs/GITHUB_ACTIONS.md +0 -293
- data/docs/OPTIMIZATIONS_IMPLEMENTED.adoc +0 -459
- data/docs/PERFORMANCE.adoc +0 -668
- data/docs/PERFORMANCE.md +0 -448
- data/docs/RELEASE_NOTES_v1.0.0.md +0 -515
- data/docs/XPATH_SPEC_COMPLIANCE.md +0 -298
- data/docs/completion/leptris.bash +0 -86
- data/docs/completion/leptris.zsh +0 -74
- data/docs/man/leptris-format.1 +0 -227
- data/docs/man/leptris-parse.1 +0 -178
- data/docs/man/leptris-xpath.1 +0 -312
- data/docs/man/leptris.1 +0 -160
- data/docs/v0.9.0_PERFORMANCE_IMPROVEMENTS.md +0 -217
- data/docs/v0.9.0_RELEASE_SUMMARY.md +0 -281
- data/docs/v1.0.0_CONTINUATION_PLAN.md +0 -172
- data/docs/v1.0.0_CONTINUATION_PROMPT.md +0 -382
- data/docs/v1.0.0_SESSION_6_CONTINUATION.md +0 -434
- data/docs/v1.0.0_SESSION_6_PROMPT.md +0 -231
- data/docs/v1.0.0_STATUS_TRACKER.md +0 -224
- data/docs/v1.1.0_CONTINUATION_PLAN.md +0 -299
- data/docs/v1.1.0_FINAL_CONTINUATION_PLAN.md +0 -201
- data/docs/v1.1.0_SESSION_3_PROMPT.md +0 -223
- data/docs/v1.1.0_STATUS_TRACKER.md +0 -355
- data/docs/xml-performance.adoc +0 -115
- data/docs/xpath-performance.adoc +0 -379
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 6cb4c2a8d3cb9da6f2c79fd03dc85b0f55f1a590b15a18ffedabd63177332f8e
|
|
4
|
+
data.tar.gz: df6b3611943af4f69c30d2a1650c86276fd89b7261f4d8da9dd43d6421b17de3
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: 82fde4f6f7d6645c1eafe9d130f532437fb28703ab27aa6753d9afc2ad53f2eabb6e852bf8338b8e8c1e551225fd2548045430f93b2063cd31adf7c2e6132e2e
|
|
7
|
+
data.tar.gz: 726699586511ef79fdc5c51e4a102e49329c480bd66d39089c2dd0201c0d96117127f6a5cbb9a90051a115e0c130877c957e6342b124fb1a18211104a8bad591
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,41 @@ All notable changes to Leptris will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [1.6.2] - 2026-08-24
|
|
9
|
+
|
|
10
|
+
Lockstep with libleptris 1.6.2 (fix releases, no public API changes).
|
|
11
|
+
|
|
12
|
+
### Changed
|
|
13
|
+
|
|
14
|
+
- CI/build pin libleptris v1.6.2, carrying the serializer fix for
|
|
15
|
+
mixed-content indentation (upstream #534 — never indent inside
|
|
16
|
+
mixed-content elements).
|
|
17
|
+
|
|
18
|
+
## [1.6.1] - 2026-08-24
|
|
19
|
+
|
|
20
|
+
### Added
|
|
21
|
+
|
|
22
|
+
- **Readonly mode**: `Leptris::XML.parse(xml, readonly: true)` /
|
|
23
|
+
`Document#readonly!` (one-way). Mutations raise
|
|
24
|
+
`Leptris::XML::ReadOnlyError`; read paths memoize aggressively
|
|
25
|
+
(names, content, children NodeSets, attribute hashes) since they can
|
|
26
|
+
never go stale; the C document is frozen (advisory upstream).
|
|
27
|
+
Detached factories (`create_element` etc.) remain usable.
|
|
28
|
+
Purpose: steady-state read performance for the dominant
|
|
29
|
+
parse-query-serialize workload — see Changed.
|
|
30
|
+
|
|
31
|
+
### Changed
|
|
32
|
+
|
|
33
|
+
- Micro-optimizations targeting the small-document gap versus C
|
|
34
|
+
extension bindings:
|
|
35
|
+
- node type is memoized from `Node.wrap`'s dispatch call — every
|
|
36
|
+
predicate and `#type` is now FFI-free
|
|
37
|
+
- the default serialize options struct is built once and reused
|
|
38
|
+
- `Document.parse` skips the per-parse status MemoryPointer (the C
|
|
39
|
+
out-param is nullable; failure detail comes from the thread-local
|
|
40
|
+
last error)
|
|
41
|
+
- `Element#name` memoizes (invalidated by `name=`)
|
|
42
|
+
|
|
8
43
|
## [1.6.0] - 2026-08-24
|
|
9
44
|
|
|
10
45
|
Lockstep with libleptris 1.6.0 — the moxml-adapter blockers fixed:
|
|
@@ -26,6 +61,41 @@ release: pull (StAX) API, bounded iterparse, compiled XPath
|
|
|
26
61
|
expressions, per-parse options, truthful serialization encoding
|
|
27
62
|
declarations. Binding-side adoption of the new APIs follows.
|
|
28
63
|
|
|
64
|
+
## [1.6.2] - 2026-08-24
|
|
65
|
+
|
|
66
|
+
Lockstep with libleptris 1.6.2 (fix releases, no public API changes).
|
|
67
|
+
|
|
68
|
+
### Changed
|
|
69
|
+
|
|
70
|
+
- CI/build pin libleptris v1.6.2, carrying the serializer fix for
|
|
71
|
+
mixed-content indentation (upstream #534 — never indent inside
|
|
72
|
+
mixed-content elements).
|
|
73
|
+
|
|
74
|
+
## [1.6.1] - 2026-08-24
|
|
75
|
+
|
|
76
|
+
### Added
|
|
77
|
+
|
|
78
|
+
- **Readonly mode**: `Leptris::XML.parse(xml, readonly: true)` /
|
|
79
|
+
`Document#readonly!` (one-way). Mutations raise
|
|
80
|
+
`Leptris::XML::ReadOnlyError`; read paths memoize aggressively
|
|
81
|
+
(names, content, children NodeSets, attribute hashes) since they can
|
|
82
|
+
never go stale; the C document is frozen (advisory upstream).
|
|
83
|
+
Detached factories (`create_element` etc.) remain usable.
|
|
84
|
+
Purpose: steady-state read performance for the dominant
|
|
85
|
+
parse-query-serialize workload — see Changed.
|
|
86
|
+
|
|
87
|
+
### Changed
|
|
88
|
+
|
|
89
|
+
- Micro-optimizations targeting the small-document gap versus C
|
|
90
|
+
extension bindings:
|
|
91
|
+
- node type is memoized from `Node.wrap`'s dispatch call — every
|
|
92
|
+
predicate and `#type` is now FFI-free
|
|
93
|
+
- the default serialize options struct is built once and reused
|
|
94
|
+
- `Document.parse` skips the per-parse status MemoryPointer (the C
|
|
95
|
+
out-param is nullable; failure detail comes from the thread-local
|
|
96
|
+
last error)
|
|
97
|
+
- `Element#name` memoizes (invalidated by `name=`)
|
|
98
|
+
|
|
29
99
|
## [1.6.0] - 2026-08-24
|
|
30
100
|
|
|
31
101
|
Lockstep with libleptris 1.6.0 (v1.4/v1.5 were fix releases with no
|
data/CLAUDE.md
CHANGED
|
@@ -4,101 +4,90 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
|
|
|
4
4
|
|
|
5
5
|
## Project
|
|
6
6
|
|
|
7
|
-
`leptris-ruby` is
|
|
8
|
-
|
|
9
|
-
-
|
|
10
|
-
|
|
11
|
-
|
|
7
|
+
`leptris-ruby` is the official Ruby binding for `libleptris`
|
|
8
|
+
(https://github.com/leptris/leptris), a pure-C99 XML 1.0 parser with a
|
|
9
|
+
W3C-conformant XPath 1.0 engine (438/438 on the W3C suite), full SAX,
|
|
10
|
+
pull parsing, and C14N. The binding is **FFI-based** (no C extension,
|
|
11
|
+
no compilation at install) and ships **precompiled platform gems**
|
|
12
|
+
with the shared library vendored inside: x86_64/aarch64-linux (glibc
|
|
13
|
+
and musl), x86_64/arm64-darwin, x64-mingw32/ucrt, aarch64-mingw-ucrt,
|
|
14
|
+
plus the pure-Ruby fallback gem.
|
|
15
|
+
|
|
16
|
+
Version lockstep: the gem's major.minor tracks the C library
|
|
17
|
+
(`libleptris 1.6.x` <-> `leptris 1.6.x`). A C release means one PR
|
|
18
|
+
carrying version.rb + the Rakefile `LIBLEPTRIS_VERSION` pin + the
|
|
19
|
+
CHANGELOG entry, then the release workflow.
|
|
12
20
|
|
|
13
21
|
## Commands
|
|
14
22
|
|
|
15
23
|
```bash
|
|
16
|
-
|
|
17
|
-
bundle exec
|
|
18
|
-
|
|
19
|
-
|
|
20
|
-
bundle exec rake test # = spec, depends on :compile
|
|
21
|
-
|
|
22
|
-
# Run RSpec directly (skips build dependency)
|
|
23
|
-
bundle exec rspec # full suite
|
|
24
|
-
bundle exec rspec spec/leptris/document_spec.rb # one file
|
|
25
|
-
bundle exec rspec spec/leptris/document_spec.rb:42 # one example by line
|
|
26
|
-
|
|
27
|
-
# Lint
|
|
28
|
-
bundle exec rubocop
|
|
29
|
-
|
|
30
|
-
# Clean build artifacts (lib/libleptris.*, ext/leptris/build, Makefile, tmp, pkg)
|
|
31
|
-
bundle exec rake clean
|
|
24
|
+
bundle exec rake compile # build libleptris (pinned tarball) into lib/
|
|
25
|
+
bundle exec rspec # full suite (uses the vendored library)
|
|
26
|
+
bundle exec rspec spec/xml/xpath_spec.rb:42 # one example
|
|
27
|
+
bundle exec rake gem:native:arm64-darwin # platform gem
|
|
32
28
|
```
|
|
33
29
|
|
|
34
|
-
|
|
30
|
+
`LEPTRIS_LIB_PATH=/path/to/libleptris.dylib bundle exec rspec` runs
|
|
31
|
+
against an arbitrary build (e.g. a libleptris checkout under test).
|
|
32
|
+
|
|
33
|
+
## Architecture
|
|
35
34
|
|
|
36
|
-
|
|
35
|
+
The C DOM is the single source of truth; Ruby objects are thin FFI
|
|
36
|
+
handles. One Ruby method = one FFI call wherever possible.
|
|
37
37
|
|
|
38
38
|
```
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
Leptris.parse /
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
39
|
+
lib/leptris.rb — Leptris::VERSION
|
|
40
|
+
lib/leptris/xml.rb — autoload registry, error classes,
|
|
41
|
+
Leptris::XML.parse / parse_file
|
|
42
|
+
lib/leptris/xml/ffi.rb — every public C declaration + seam
|
|
43
|
+
helpers (check_status, read_owned_string,
|
|
44
|
+
status_message)
|
|
45
|
+
lib/leptris/xml/document.rb — the only C-memory owner (finalizer);
|
|
46
|
+
factories, PI accessors, exslt, last_error
|
|
47
|
+
lib/leptris/xml/node.rb — Node.wrap: the ONLY wrapper constructor
|
|
48
|
+
(identity cache + type dispatch)
|
|
49
|
+
lib/leptris/xml/element.rb — attributes via the v1.1.0 iteration
|
|
50
|
+
face; namespace/mutation surface
|
|
51
|
+
lib/leptris/xml/node_set.rb — lazy XPath results; batch fetch via
|
|
52
|
+
get_nodes_ex
|
|
53
|
+
lib/leptris/xml/searchable.rb — xpath/css/at_*; namespace-bound path
|
|
54
|
+
lib/leptris/xml/xpath.rb — compiled expressions (parse once,
|
|
55
|
+
eval many)
|
|
56
|
+
lib/leptris/xml/sax/ — callback SAX
|
|
57
|
+
lib/leptris/xml/pull.rb — StAX-style pull parsing
|
|
58
|
+
lib/leptris/xml/iterparse.rb — bounded-memory element iteration
|
|
59
|
+
lib/leptris/xml/serialization.rb — serialize/c14n (Document + Element
|
|
60
|
+
one-liners over this module)
|
|
61
|
+
lib/leptris/xml/c_string_array.rb — NULL-terminated char** adapter
|
|
62
|
+
lib/leptris/xml/css_to_xpath.rb — minimal CSS translation
|
|
46
63
|
```
|
|
47
64
|
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
- `
|
|
51
|
-
|
|
52
|
-
|
|
53
|
-
-
|
|
54
|
-
|
|
55
|
-
-
|
|
56
|
-
-
|
|
57
|
-
|
|
58
|
-
|
|
59
|
-
|
|
60
|
-
|
|
61
|
-
|
|
62
|
-
|
|
63
|
-
|
|
64
|
-
|
|
65
|
-
|
|
66
|
-
|
|
67
|
-
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
|
|
71
|
-
|
|
72
|
-
|
|
73
|
-
|
|
74
|
-
|
|
75
|
-
|
|
76
|
-
|
|
77
|
-
|
|
78
|
-
5. **`05-serialize-c14n-memory-specs-css.md`** — `SerializeOptions` struct for `leptris_serialize_document`; `Document#canonicalize` (modes `C14N_1_0`, `C14N_1_1`, `C14N_EXCLUSIVE`); `UseAfterFreeError` guard; minimal CSS-to-XPath converter (`.class`, `#id`, `[attr]`, `[attr=val]`, `:first-child`, `:last-child`); spec layout under `spec/xml/{parse,document,node,element,node_set,xpath,sax,serialize,c14n,memory}_spec.rb`.
|
|
79
|
-
|
|
80
|
-
### Memory ownership rules (planned)
|
|
81
|
-
|
|
82
|
-
| Ruby class | Owns C memory? | Free function |
|
|
83
|
-
|---|---|---|
|
|
84
|
-
| `Document` | YES | `leptris_document_free` |
|
|
85
|
-
| `Node`/`Element`/text/CDATA/PI/Attr | NO (borrowed) | freed transitively by Document |
|
|
86
|
-
| `NodeSet` (XPath result) | YES | `leptris_xpath_result_free` |
|
|
87
|
-
| SAX handler closures | callback lifetime | `leptris_sax_parser_free` |
|
|
88
|
-
|
|
89
|
-
GC safety net: `ObjectSpace.define_finalizer` capturing the **raw pointer value**, not the Ruby wrapper. Finalizers must not double-free after explicit `#free`.
|
|
90
|
-
|
|
91
|
-
## Reference material
|
|
92
|
-
|
|
93
|
-
- Nokogiri source (`~/src/external/nokogiri/`) — `lib/nokogiri/xml/{node,node_set,document,searchable}.rb` are the API shape targets.
|
|
94
|
-
- libleptris public headers (`src/include/leptris/{types,leptris}.h`, `src/include/leptris/{dom,xpath,sax}/*.h`) — the single source of truth for FFI declarations. Target tag: `v0.4.2`.
|
|
95
|
-
- `docs/FFI_ARCHITECTURE.md` — describes the v0.5.0 FFI design (AutoPointer, two-pointer strategy for XPath). The planned rewrite supersedes some of this (no recursive hydration, no two-pointer — the Document pointer alone suffices because Node objects stay as C handles).
|
|
96
|
-
- `docs/BUILD.md` — CMake build reference for libleptris itself.
|
|
97
|
-
|
|
98
|
-
## Conventions (project-specific)
|
|
99
|
-
|
|
100
|
-
- **Autoload, not require_relative.** TODO 3 is explicit: `lib/leptris.rb` → `autoload :XML, 'leptris/xml'`; `lib/leptris/xml.rb` → `autoload :Document, 'leptris/xml/document'`. Autoload entries live in the **immediate parent namespace's file** (create that file if missing). The current `lib/leptris.rb` uses `require_relative` — when implementing TODO 3, do not retrofit require_relative into the new layout.
|
|
101
|
-
- **No `instance_variable_set`/`_get` cross-object.** TODO 3 is explicit. The current `lib/leptris.rb` and `lib/leptris/ffi/bridge.rb` use `instance_variable_get(:@_c_ptr)` heavily — that pattern is debt to migrate, not a model to copy. In the rewrite, expose `c_ptr`/`document` as public `attr_reader`s and access via those.
|
|
102
|
-
- **No `respond_to?` type checks.** Use `is_a?`. The current `lib/leptris.rb#xpath_evaluate` checks `context_node != doc` to disambiguate — fine. Don't add `respond_to?(:c_ptr)` style checks.
|
|
103
|
-
- **No doubles in specs.** The existing `spec/leptris/` specs use real model instances (XML strings → `Leptris.parse` → real `Document`/`Element`/`NodeSet`). Keep it that way.
|
|
104
|
-
- **Forward compatibility:** Keep `Leptris.parse` / `Leptris.parse_file` as the existing top-level API during the rewrite. The new `Leptris::XML.parse` may coexist.
|
|
65
|
+
### Ownership and seams
|
|
66
|
+
|
|
67
|
+
- `Document` owns C memory (explicit `#free` or GC finalizer; wrapper
|
|
68
|
+
cache is a strong Hash cleared on free). Every other class is a
|
|
69
|
+
borrowed handle valid while its Document lives.
|
|
70
|
+
- All wrapper construction goes through `Node.wrap` — wrapper
|
|
71
|
+
identity (`doc.root.equal?(doc.root)`) is guaranteed by the
|
|
72
|
+
per-document cache.
|
|
73
|
+
- Status checks go through `FFI.check_status`; C `char*` returns go
|
|
74
|
+
through `FFI.read_owned_string`; array wire format (both
|
|
75
|
+
directions) goes through `CStringArray`. Never hand-roll these.
|
|
76
|
+
- The release workflow's publish loop is idempotent (already-published
|
|
77
|
+
gems are skipped).
|
|
78
|
+
|
|
79
|
+
## Conventions
|
|
80
|
+
|
|
81
|
+
- Autoload only — no `require_relative` inside `lib/`.
|
|
82
|
+
- No `instance_variable_set`/`_get` across objects; no `respond_to?`
|
|
83
|
+
type checks; specs use real documents, never doubles.
|
|
84
|
+
- New C surface: attach in `ffi.rb`, sugar where it earns its keep,
|
|
85
|
+
specs against a locally built library, CHANGELOG, lockstep release.
|
|
86
|
+
- All changes via PRs; no AI attribution; `git add` explicit paths.
|
|
87
|
+
|
|
88
|
+
## Reference
|
|
89
|
+
|
|
90
|
+
- libleptris public headers (`src/include/leptris/*.h`) are the
|
|
91
|
+
contract; when symbols change, bump lockstep and audit
|
|
92
|
+
attached-vs-exported (`nm -gU` on a fresh build).
|
|
93
|
+
- Upstream issues worth tracking live at leptris/leptris.
|
data/README.adoc
CHANGED
|
@@ -136,6 +136,25 @@ doc.root.traverse do |node|
|
|
|
136
136
|
end
|
|
137
137
|
----
|
|
138
138
|
|
|
139
|
+
== Readonly mode
|
|
140
|
+
|
|
141
|
+
For the dominant parse-query-serialize workload, parse with
|
|
142
|
+
`readonly: true` (or call `Document#readonly!`, one-way):
|
|
143
|
+
|
|
144
|
+
[source,ruby]
|
|
145
|
+
----
|
|
146
|
+
doc = Leptris::XML.parse(xml, readonly: true)
|
|
147
|
+
doc.root.children.first["id"] # reads are memoized
|
|
148
|
+
doc.root.name # plain ivar after first call
|
|
149
|
+
doc.root << doc.create_element("x") # raises ReadOnlyError
|
|
150
|
+
----
|
|
151
|
+
|
|
152
|
+
Reads (`name`, `content`, `children`, `attributes`) memoize
|
|
153
|
+
aggressively — they cannot go stale because mutation is forbidden.
|
|
154
|
+
Every mutator raises `Leptris::XML::ReadOnlyError`. Detached factories
|
|
155
|
+
(`create_element` and friends) still work: building a *new* tree
|
|
156
|
+
against a readonly document is legal; mutating the frozen one is not.
|
|
157
|
+
|
|
139
158
|
== Searching: XPath and CSS
|
|
140
159
|
|
|
141
160
|
`Document` and `Element` (via `Leptris::XML::Searchable`) support:
|
|
@@ -376,11 +395,66 @@ Notable differences:
|
|
|
376
395
|
|
|
377
396
|
== Performance
|
|
378
397
|
|
|
379
|
-
|
|
380
|
-
libleptris
|
|
381
|
-
|
|
382
|
-
|
|
383
|
-
|
|
398
|
+
Measured with metanorma/serialbench (Ruby 3.4.8, leptris 1.6.0,
|
|
399
|
+
libleptris 1.6.0, macOS arm64) — Leptris vs the full field:
|
|
400
|
+
|
|
401
|
+
[cols="2,1,1,1,1",options="header"]
|
|
402
|
+
|===
|
|
403
|
+
|Operation |Leptris |Ox |Nokogiri |vs Ox
|
|
404
|
+
|
|
405
|
+
|parse medium (300 KB)
|
|
406
|
+
|0.97 ms
|
|
407
|
+
|3.00 ms
|
|
408
|
+
|8.57 ms
|
|
409
|
+
|3.1x faster
|
|
410
|
+
|
|
411
|
+
|parse large (4.4 MB)
|
|
412
|
+
|13.51 ms
|
|
413
|
+
|77.06 ms
|
|
414
|
+
|104.22 ms
|
|
415
|
+
|5.7x faster
|
|
416
|
+
|
|
417
|
+
|generate medium
|
|
418
|
+
|1.75 ms
|
|
419
|
+
|14.19 ms
|
|
420
|
+
|23.93 ms
|
|
421
|
+
|8.1x faster
|
|
422
|
+
|
|
423
|
+
|generate large
|
|
424
|
+
|74.57 ms
|
|
425
|
+
|80.91 ms
|
|
426
|
+
|117.16 ms
|
|
427
|
+
|1.1x faster
|
|
428
|
+
|
|
429
|
+
|streaming medium
|
|
430
|
+
|0.73 ms
|
|
431
|
+
|103.82 ms
|
|
432
|
+
|43.74 ms
|
|
433
|
+
|142x faster
|
|
434
|
+
|
|
435
|
+
|streaming large
|
|
436
|
+
|8.52 ms
|
|
437
|
+
|204.14 ms
|
|
438
|
+
|239.14 ms
|
|
439
|
+
|24x faster
|
|
440
|
+
|
|
441
|
+
|Ruby allocations (medium)
|
|
442
|
+
|0.01 MB
|
|
443
|
+
|18.43 MB
|
|
444
|
+
|39.05 MB
|
|
445
|
+
|~1800x less
|
|
446
|
+
|===
|
|
447
|
+
|
|
448
|
+
Leptris takes first place on every XML operation above small-document
|
|
449
|
+
size, including against Ox (the C-extension speed champion). On
|
|
450
|
+
small-document micro-operations (sub-5 µs), C extensions hold an
|
|
451
|
+
inherent edge per call; Leptris's readonly mode closes the
|
|
452
|
+
steady-state gap by memoizing reads behind an immutability guarantee.
|
|
453
|
+
|
|
454
|
+
For XPath-heavy workloads, `Leptris::XML::XPath.compile` evaluates a
|
|
455
|
+
parsed expression repeatedly without re-parsing.
|
|
456
|
+
|
|
457
|
+
Run the local benchmark for numbers on your hardware:
|
|
384
458
|
|
|
385
459
|
[source,shell]
|
|
386
460
|
----
|
|
@@ -392,7 +466,7 @@ bundle exec ruby benchmark/leptris_vs_nokogiri.rb
|
|
|
392
466
|
[source,shell]
|
|
393
467
|
----
|
|
394
468
|
bundle install # install Ruby deps
|
|
395
|
-
bundle exec rspec # full test suite (
|
|
469
|
+
bundle exec rspec # full test suite (229 specs)
|
|
396
470
|
bundle exec rspec spec/xml/xpath_spec.rb:42 # one example by line
|
|
397
471
|
bundle exec rubocop # lint
|
|
398
472
|
----
|
data/Rakefile
CHANGED
|
@@ -8,7 +8,7 @@ RSpec::Core::RakeTask.new(:spec)
|
|
|
8
8
|
# Pin for `rake compile` and the platform-gem builds. Keep in lockstep
|
|
9
9
|
# with .github/workflows/build.yml (which calls `rake compile`) and the
|
|
10
10
|
# CHANGELOG when libleptris releases.
|
|
11
|
-
LIBLEPTRIS_VERSION = "1.6.
|
|
11
|
+
LIBLEPTRIS_VERSION = "1.6.2"
|
|
12
12
|
|
|
13
13
|
CMAKE_FLAGS = %w[
|
|
14
14
|
-DCMAKE_BUILD_TYPE=Release
|
data/benchmark/README.md
CHANGED
|
@@ -1,168 +1,12 @@
|
|
|
1
|
-
# leptris
|
|
1
|
+
# leptris vs the field
|
|
2
2
|
|
|
3
|
-
|
|
4
|
-
(
|
|
3
|
+
The authoritative cross-library numbers live in the README's
|
|
4
|
+
Performance section (measured with metanorma/serialbench).
|
|
5
5
|
|
|
6
|
-
|
|
6
|
+
This local benchmark compares against Nokogiri only:
|
|
7
7
|
|
|
8
|
-
|
|
9
|
-
bundle exec ruby -Ilib benchmark/leptris_vs_nokogiri.rb
|
|
10
|
-
```
|
|
8
|
+
LEPTRIS_LIB_PATH=/path/to/libleptris.dylib bundle exec ruby benchmark/leptris_vs_nokogiri.rb
|
|
11
9
|
|
|
12
|
-
|
|
13
|
-
|
|
14
|
-
|
|
15
|
-
v0.12.0 ships the [#262](https://github.com/leptris/leptris/issues/262) proposals:
|
|
16
|
-
batch XPath result accessor (`leptris_xpath_result_get_nodes`) and per-node
|
|
17
|
-
`binding_wrapper` field. The Ruby binding uses the batch accessor in
|
|
18
|
-
`NodeSet#each`; the `binding_wrapper` is left for the libleptris team's
|
|
19
|
-
other bindings (Python ctypes, etc.) — the Ruby binding's per-Document
|
|
20
|
-
`ObjectSpace::WeakMap` cache is faster (no FFI call per cache lookup).
|
|
21
|
-
|
|
22
|
-
Median of 3 runs (run-to-run variance is significant on cold starts and
|
|
23
|
-
shared-runner workloads; the trend is stable):
|
|
24
|
-
|
|
25
|
-
| Operation | Leptris | Nokogiri | Leptris / Nokogiri | Winner |
|
|
26
|
-
|---|---:|---:|---:|---|
|
|
27
|
-
| Parse small (431 B) | ~6 µs | ~14 µs | **~2×** | Leptris (variable) |
|
|
28
|
-
| Parse medium (12 KB) | ~32 µs | ~200 µs | **~6×** | Leptris |
|
|
29
|
-
| XPath `count(//book)` | ~1.5 µs | ~9 µs | **~6×** | Leptris |
|
|
30
|
-
| XPath `//book` (100-node nodeset) | ~4 µs | ~13 µs | **~5×** | Leptris |
|
|
31
|
-
| XPath `//book[@id='50']` (1 match) | ~6 µs | ~80 µs | **~10×** | Leptris |
|
|
32
|
-
| XPath `//book[price > 50]` | ~80 µs | ~100 µs | **~1.3×** | Leptris |
|
|
33
|
-
| XPath `//author \| //title` (union) | ~14 µs | ~25 µs | **~3×** | Leptris |
|
|
34
|
-
| Tree traversal | ~900 µs | ~500 µs | 0.55× | Nokogiri (1.8× faster) |
|
|
35
|
-
| Serialize | ~28 µs | ~85 µs | **~3×** | Leptris |
|
|
36
|
-
|
|
37
|
-
**Leptris beats Nokogiri on 8 of 9 operations.** Only tree traversal still
|
|
38
|
-
loses, by ~1.8×.
|
|
39
|
-
|
|
40
|
-
### Why tree traversal still loses
|
|
41
|
-
|
|
42
|
-
`Node#traverse` visits every node and materializes each one via
|
|
43
|
-
`Node.wrap`. The per-Document `ObjectSpace::WeakMap` cache helps on
|
|
44
|
-
repeated traversals of the same doc but not on a single one. Each
|
|
45
|
-
visited node pays:
|
|
46
|
-
- 1 FFI call to `leptris_node_first_child` / `_next_sibling`
|
|
47
|
-
- 1 FFI call to `leptris_node_get_type` (for wrap dispatch)
|
|
48
|
-
- 1 Ruby object allocation (cache miss)
|
|
49
|
-
|
|
50
|
-
Nokogiri's libxml2 C extension handles traversal in C and only crosses
|
|
51
|
-
into Ruby when the user's block is called. No per-node FFI.
|
|
52
|
-
|
|
53
|
-
The libleptris `binding_wrapper` field shipped in v0.12.0 doesn't help
|
|
54
|
-
here in the Ruby binding — Ruby FFI still needs an FFI call to read
|
|
55
|
-
`binding_wrapper`. The Ruby-side `WeakMap` cache avoids that FFI call
|
|
56
|
-
on cache hits. So the binding's existing cache is already optimal for
|
|
57
|
-
Ruby; the `binding_wrapper` field is more useful for bindings that
|
|
58
|
-
don't have a native GC hook (Python ctypes, Go cgo, Rust bindgen).
|
|
59
|
-
|
|
60
|
-
## What changed from earlier runs
|
|
61
|
-
|
|
62
|
-
### v0.11.0 → v0.11.2 (binding-side)
|
|
63
|
-
|
|
64
|
-
- **Lazy NodeSet** — `NodeSet.from_result` keeps the
|
|
65
|
-
`LeptrisXPathResult*` alive (via `FFI::AutoPointer`) and materializes
|
|
66
|
-
`self[i]` on demand. Eager materialization was the #1 cost.
|
|
67
|
-
- **Per-Document wrapper cache** — `ObjectSpace::WeakMap` keyed on c_ptr
|
|
68
|
-
address. Eliminates re-allocation on repeated access to the same node.
|
|
69
|
-
|
|
70
|
-
### v0.11.4 (libleptris-side)
|
|
71
|
-
|
|
72
|
-
- **`leptris_xpath_result_get_nodes`** — batch accessor. The Ruby binding
|
|
73
|
-
now uses this in `NodeSet#each` to fetch all node pointers in one FFI
|
|
74
|
-
call instead of N calls.
|
|
75
|
-
|
|
76
|
-
### v0.12.0 (libleptris-side)
|
|
77
|
-
|
|
78
|
-
- **`binding_wrapper` field on `LeptrisNode`** — present in the C struct
|
|
79
|
-
but not used by the Ruby binding (see "Why tree traversal still loses"
|
|
80
|
-
above). Useful for non-Ruby bindings.
|
|
81
|
-
- **`#261` fix** — `benchmark-ips` on 38 KB docs no longer segfaults.
|
|
82
|
-
All upstream issues closed.
|
|
83
|
-
|
|
84
|
-
### Before/after the binding + libleptris v0.12.0 optimizations
|
|
85
|
-
|
|
86
|
-
| Operation | v0.11.0 (eager) | v0.12.0 (lazy + batch) | Speedup |
|
|
87
|
-
|---|---:|---:|---:|
|
|
88
|
-
| XPath `//book` (100 nodes) | 87.77 µs (0.15×) | ~4 µs (5×) | **22×** |
|
|
89
|
-
| XPath union (200 nodes) | 188.40 µs (0.14×) | ~14 µs (3×) | **13×** |
|
|
90
|
-
| XPath complex | 126.88 µs (0.67×) | ~80 µs (1.3×) | 1.6× |
|
|
91
|
-
| Tree traversal | 1203 µs (0.50×) | ~900 µs (0.55×) | 1.3× |
|
|
92
|
-
|
|
93
|
-
## What this means for the v0.1.0 release
|
|
94
|
-
|
|
95
|
-
- Leptris is the right choice for almost every Nokogiri workload on
|
|
96
|
-
small-to-medium docs (≤20 KB): 2–10× faster than Nokogiri.
|
|
97
|
-
- Tree-traversal-heavy workloads (single-pass DOM scraping where you
|
|
98
|
-
touch every node once) are 1.8× slower than Nokogiri. Acceptable
|
|
99
|
-
for v0.1.0; the binding could ship a "fast traverse" path later that
|
|
100
|
-
skips wrapping for read-only blocks.
|
|
101
|
-
- All known libleptris bugs are fixed. No upstream blockers.
|
|
102
|
-
|
|
103
|
-
## Analysis
|
|
104
|
-
|
|
105
|
-
### Where Leptris wins
|
|
106
|
-
|
|
107
|
-
**Parse (1.69×–6.90×)** — libleptris's single-pass direct parser (the only
|
|
108
|
-
parser since v0.11.0, after flat + legacy were deleted) is dramatically
|
|
109
|
-
faster than libxml2's parser. The gap widens with document size.
|
|
110
|
-
|
|
111
|
-
**XPath returning scalars (4.56×)** — `count()`, `boolean()`, `string()`,
|
|
112
|
-
`number()` queries skip NodeSet materialization entirely. libleptris's XPath
|
|
113
|
-
bytecode VM evaluates these in a single C call, no Ruby objects allocated
|
|
114
|
-
per match.
|
|
115
|
-
|
|
116
|
-
**XPath predicate match (13.01×)** — when the predicate narrows to a small
|
|
117
|
-
result set (single match in this test), Leptris is much faster than
|
|
118
|
-
Nokogiri/libxml2.
|
|
119
|
-
|
|
120
|
-
**Serialize (2.84×)** — single C call into `leptris_document_serialize`,
|
|
121
|
-
no Ruby traversal.
|
|
122
|
-
|
|
123
|
-
### Where Nokogiri wins
|
|
124
|
-
|
|
125
|
-
**Nodeset-returning XPath (0.13×–0.16×)** — when a query returns a 100-node
|
|
126
|
-
NodeSet, Leptris materializes every node eagerly (100× `Node.wrap` calls,
|
|
127
|
-
each dispatching on `leptris_node_get_type`). Nokogiri caches wrappers
|
|
128
|
-
lazily.
|
|
129
|
-
|
|
130
|
-
**Tree traversal (0.49×)** — same root cause. `Node#traverse` creates a
|
|
131
|
-
new wrapper per visited node via `Node.wrap`; Nokogiri reuses cached
|
|
132
|
-
wrappers.
|
|
133
|
-
|
|
134
|
-
### Optimization opportunities (Ruby-side, no libleptris work needed)
|
|
135
|
-
|
|
136
|
-
1. **Lazy NodeSet materialization.** Currently `NodeSet.from_result`
|
|
137
|
-
iterates the C result and calls `Node.wrap` for each entry on
|
|
138
|
-
construction. Switch to lazy: keep the `LeptrisXPathResult*` alive,
|
|
139
|
-
materialize `self[i]` on demand. Frees the eager 100× wrap.
|
|
140
|
-
2. **Node wrapper cache.** Weak-ref map keyed on the c_ptr address.
|
|
141
|
-
`Node.wrap(ptr)` checks the cache first; only creates a new wrapper
|
|
142
|
-
if none exists. Matches Nokogiri's behavior.
|
|
143
|
-
3. **Specialized traverse path.** For pure-traversal use cases (no
|
|
144
|
-
per-node mutation), skip the wrapper and call FFI directly. Lower
|
|
145
|
-
overhead but less idiomatic.
|
|
146
|
-
|
|
147
|
-
### Blockers
|
|
148
|
-
|
|
149
|
-
**Parse-loop segfault on >20 KB docs with explicit `Document#free`
|
|
150
|
-
(libleptris #256).** The v0.11.1 fix addressed one stale-thread-local
|
|
151
|
-
path but not the parse+free cycle path. Long-running services and batch
|
|
152
|
-
processors parsing medium/large XML cannot rely on the standard Ruby
|
|
153
|
-
"let GC handle document lifetime" pattern OR the explicit `Document#free`
|
|
154
|
-
pattern. Tracked upstream; workaround: cap doc size or avoid tight loops.
|
|
155
|
-
|
|
156
|
-
**DOCTYPE PUBLIC/SYSTEM not exposed (libleptris #253).** Unrelated to
|
|
157
|
-
benchmarks but blocks 4 Ruby specs. Low impact on perf-sensitive workloads.
|
|
158
|
-
|
|
159
|
-
## What this means for the v0.1.0 release
|
|
160
|
-
|
|
161
|
-
- For **parse-heavy / XPath-aggregate / serialize** workloads on small-to-
|
|
162
|
-
medium docs (≤20 KB): Leptris is clearly the right choice. 1.7-7× faster
|
|
163
|
-
than Nokogiri.
|
|
164
|
-
- For **heavy nodeset manipulation** (scraping, large DOM traversal):
|
|
165
|
-
Nokogiri is faster today. The Ruby-side optimizations above would close
|
|
166
|
-
most of the gap.
|
|
167
|
-
- For **long-running services on medium/large docs**: blocked by #256
|
|
168
|
-
until libleptris ships a complete fix.
|
|
10
|
+
For the full-field race (Ox, Nokogiri, Oga, REXML, Leptris), use
|
|
11
|
+
https://github.com/metanorma/serialbench — leptris's adapter ships
|
|
12
|
+
there under `lib/serialbench/serializers/xml/leptris_serializer.rb`.
|
data/lib/leptris/version.rb
CHANGED
data/lib/leptris/xml/cdata.rb
CHANGED
data/lib/leptris/xml/comment.rb
CHANGED