leptris 1.0.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.rspec +3 -0
- data/.rubocop.yml +8 -0
- data/CHANGELOG.md +529 -0
- data/CLAUDE.md +104 -0
- data/LICENSE.md +33 -0
- data/README.adoc +405 -0
- data/Rakefile +7 -0
- data/TODO.impl/01-architecture.md +217 -0
- data/TODO.impl/02-ffi-declarations.md +236 -0
- data/TODO.impl/03-document-node-element-nodeset.md +382 -0
- data/TODO.impl/04-sax-parser.md +203 -0
- data/TODO.impl/05-serialize-c14n-memory-specs-css.md +276 -0
- data/benchmark/README.md +168 -0
- data/benchmark/leptris_vs_nokogiri.rb +105 -0
- data/docs/ARCHITECTURE.adoc +559 -0
- data/docs/BUILD.md +395 -0
- data/docs/ERROR_MESSAGES.md +458 -0
- data/docs/FFI_ARCHITECTURE.md +439 -0
- data/docs/FUTURE_VISION.md +303 -0
- data/docs/GITHUB_ACTIONS.md +293 -0
- data/docs/OPTIMIZATIONS_IMPLEMENTED.adoc +459 -0
- data/docs/PERFORMANCE.adoc +668 -0
- data/docs/PERFORMANCE.md +448 -0
- data/docs/RELEASE_NOTES_v1.0.0.md +515 -0
- data/docs/XPATH_SPEC_COMPLIANCE.md +298 -0
- data/docs/completion/leptris.bash +86 -0
- data/docs/completion/leptris.zsh +74 -0
- data/docs/man/leptris-format.1 +227 -0
- data/docs/man/leptris-parse.1 +178 -0
- data/docs/man/leptris-xpath.1 +312 -0
- data/docs/man/leptris.1 +160 -0
- data/docs/v0.9.0_PERFORMANCE_IMPROVEMENTS.md +217 -0
- data/docs/v0.9.0_RELEASE_SUMMARY.md +281 -0
- data/docs/v1.0.0_CONTINUATION_PLAN.md +172 -0
- data/docs/v1.0.0_CONTINUATION_PROMPT.md +382 -0
- data/docs/v1.0.0_SESSION_6_CONTINUATION.md +434 -0
- data/docs/v1.0.0_SESSION_6_PROMPT.md +231 -0
- data/docs/v1.0.0_STATUS_TRACKER.md +224 -0
- data/docs/v1.1.0_CONTINUATION_PLAN.md +299 -0
- data/docs/v1.1.0_FINAL_CONTINUATION_PLAN.md +201 -0
- data/docs/v1.1.0_SESSION_3_PROMPT.md +223 -0
- data/docs/v1.1.0_STATUS_TRACKER.md +355 -0
- data/docs/xml-performance.adoc +115 -0
- data/docs/xpath-performance.adoc +379 -0
- data/leptris.gemspec +42 -0
- data/lib/leptris/version.rb +5 -0
- data/lib/leptris/xml/attr.rb +43 -0
- data/lib/leptris/xml/c14n.rb +23 -0
- data/lib/leptris/xml/cdata.rb +16 -0
- data/lib/leptris/xml/comment.rb +16 -0
- data/lib/leptris/xml/css_to_xpath.rb +177 -0
- data/lib/leptris/xml/doc_type.rb +54 -0
- data/lib/leptris/xml/document.rb +202 -0
- data/lib/leptris/xml/document_fragment.rb +42 -0
- data/lib/leptris/xml/element.rb +278 -0
- data/lib/leptris/xml/ffi.rb +420 -0
- data/lib/leptris/xml/namespace.rb +43 -0
- data/lib/leptris/xml/node.rb +221 -0
- data/lib/leptris/xml/node_set.rb +143 -0
- data/lib/leptris/xml/parse_options.rb +19 -0
- data/lib/leptris/xml/processing_instruction.rb +26 -0
- data/lib/leptris/xml/sax/document.rb +45 -0
- data/lib/leptris/xml/sax/parser.rb +148 -0
- data/lib/leptris/xml/sax.rb +12 -0
- data/lib/leptris/xml/searchable.rb +93 -0
- data/lib/leptris/xml/text.rb +16 -0
- data/lib/leptris/xml.rb +39 -0
- data/lib/leptris.rb +7 -0
- metadata +157 -0
data/LICENSE.md
ADDED
|
@@ -0,0 +1,33 @@
|
|
|
1
|
+
Licenses & Copyright
|
|
2
|
+
====================
|
|
3
|
+
|
|
4
|
+
This license file adheres to the formatting guidelines of
|
|
5
|
+
[readable-licenses](https://github.com/nevir/readable-licenses).
|
|
6
|
+
|
|
7
|
+
|
|
8
|
+
Ribose's BSD 2-Clause License
|
|
9
|
+
-----------------------------
|
|
10
|
+
|
|
11
|
+
Copyright (c) 2025, [Ribose Inc](https://www.ribose.com).
|
|
12
|
+
All rights reserved.
|
|
13
|
+
|
|
14
|
+
Redistribution and use in source and binary forms, with or without modification,
|
|
15
|
+
are permitted provided that the following conditions are met:
|
|
16
|
+
|
|
17
|
+
1. Redistributions of source code must retain the above copyright notice,
|
|
18
|
+
this list of conditions and the following disclaimer.
|
|
19
|
+
|
|
20
|
+
2. Redistributions in binary form must reproduce the above copyright notice,
|
|
21
|
+
this list of conditions and the following disclaimer in the documentation
|
|
22
|
+
and/or other materials provided with the distribution.
|
|
23
|
+
|
|
24
|
+
THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" AND
|
|
25
|
+
ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED
|
|
26
|
+
WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE
|
|
27
|
+
DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE LIABLE
|
|
28
|
+
FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL
|
|
29
|
+
DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR
|
|
30
|
+
SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER
|
|
31
|
+
CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY,
|
|
32
|
+
OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF
|
|
33
|
+
THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE.
|
data/README.adoc
ADDED
|
@@ -0,0 +1,405 @@
|
|
|
1
|
+
= leptris-ruby
|
|
2
|
+
:toc:
|
|
3
|
+
:toclevels: 3
|
|
4
|
+
|
|
5
|
+
image:https://img.shields.io/gem/v/leptris.svg[RubyGems Version,link=https://rubygems.org/gems/leptris]
|
|
6
|
+
image:https://github.com/leptris/leptris-ruby/actions/workflows/build.yml/badge.svg[CI,link=https://github.com/leptris/leptris-ruby/actions/workflows/build.yml]
|
|
7
|
+
|
|
8
|
+
A Nokogiri-compatible Ruby binding for
|
|
9
|
+
https://github.com/leptris/leptris[libleptris], a pure-C99 XML 1.0 parser
|
|
10
|
+
with full https://www.w3.org/TR/1999/REC-xpath-19991116/[XPath 1.0],
|
|
11
|
+
XML Namespaces 1.0, SAX, and C14N (1.0 / 1.1 / Exclusive).
|
|
12
|
+
|
|
13
|
+
The C DOM is the single source of truth — Ruby objects are thin FFI
|
|
14
|
+
handles over the C pointers, so every Ruby method maps to one FFI call.
|
|
15
|
+
No tree hydration, no parallel Ruby-side model.
|
|
16
|
+
|
|
17
|
+
== Installation
|
|
18
|
+
|
|
19
|
+
Add to your Gemfile:
|
|
20
|
+
|
|
21
|
+
[source,ruby]
|
|
22
|
+
----
|
|
23
|
+
gem "leptris"
|
|
24
|
+
----
|
|
25
|
+
|
|
26
|
+
Then `bundle install`.
|
|
27
|
+
|
|
28
|
+
=== Runtime requirement: libleptris
|
|
29
|
+
|
|
30
|
+
`leptris` shells out to the native `libleptris` shared library via FFI.
|
|
31
|
+
You need `libleptris.{dylib,so,dll}` installed on the host. Options:
|
|
32
|
+
|
|
33
|
+
1. **Homebrew (macOS, easiest):** `brew install lutaml/tap/libleptris`
|
|
34
|
+
(if packaged) or build from source (see below).
|
|
35
|
+
2. **Build from source** (Linux/macOS/Windows):
|
|
36
|
+
+
|
|
37
|
+
[source,shell]
|
|
38
|
+
----
|
|
39
|
+
git clone https://github.com/leptris/leptris.git
|
|
40
|
+
cd leptris
|
|
41
|
+
cmake -B build -S . \
|
|
42
|
+
-DCMAKE_BUILD_TYPE=Release \
|
|
43
|
+
-DLEPTRIS_BUILD_SHARED=ON \
|
|
44
|
+
-DLEPTRIS_BUILD_STATIC=OFF \
|
|
45
|
+
-DCMAKE_WINDOWS_EXPORT_ALL_SYMBOLS=ON
|
|
46
|
+
cmake --build build -j
|
|
47
|
+
sudo cmake --install build # optional, system-wide
|
|
48
|
+
----
|
|
49
|
+
|
|
50
|
+
3. **Point Leptris at a specific path** by setting `LEPTRIS_LIB_PATH`:
|
|
51
|
+
+
|
|
52
|
+
[source,shell]
|
|
53
|
+
----
|
|
54
|
+
export LEPTRIS_LIB_PATH=/usr/local/lib/libleptris.dylib
|
|
55
|
+
----
|
|
56
|
+
|
|
57
|
+
If Leptris can't find the library at startup, every `parse` call raises
|
|
58
|
+
`LoadError`.
|
|
59
|
+
|
|
60
|
+
== Parsing
|
|
61
|
+
|
|
62
|
+
The top-level entry point is `Leptris::XML`. Parse a string or an IO:
|
|
63
|
+
|
|
64
|
+
[source,ruby]
|
|
65
|
+
----
|
|
66
|
+
require "leptris"
|
|
67
|
+
|
|
68
|
+
doc = Leptris::XML.parse(<<~XML)
|
|
69
|
+
<library xmlns="http://example.org/ns">
|
|
70
|
+
<book id="b1" lang="en">
|
|
71
|
+
<title>Refactoring</title>
|
|
72
|
+
<author>Martin Fowler</author>
|
|
73
|
+
</book>
|
|
74
|
+
<book id="b2" lang="fr">
|
|
75
|
+
<title>Programmer en Ruby</title>
|
|
76
|
+
</book>
|
|
77
|
+
</library>
|
|
78
|
+
XML
|
|
79
|
+
|
|
80
|
+
doc.root.name # => "library"
|
|
81
|
+
doc.root.children.size # => 5 (2 element children + 3 whitespace text nodes)
|
|
82
|
+
----
|
|
83
|
+
|
|
84
|
+
Or a file:
|
|
85
|
+
|
|
86
|
+
[source,ruby]
|
|
87
|
+
----
|
|
88
|
+
doc = Leptris::XML.parse_file("books.xml")
|
|
89
|
+
----
|
|
90
|
+
|
|
91
|
+
This is the direct Nokogiri equivalent of `Nokogiri::XML(...)`. The
|
|
92
|
+
returned object is a `Leptris::XML::Document`.
|
|
93
|
+
|
|
94
|
+
== Reading nodes
|
|
95
|
+
|
|
96
|
+
[horizontal]
|
|
97
|
+
`Document#root` :: root `Element`, or `nil` for an empty document.
|
|
98
|
+
`Node#name` :: element name (e.g. `"book"`).
|
|
99
|
+
`Node#content` (alias `#text`, `#inner_text`) :: all descendant text concatenated.
|
|
100
|
+
`Node#[]` (alias `#attr`, `#get_attribute`) :: attribute value by name.
|
|
101
|
+
`Node#attributes` :: hash of `{name => Attr}`.
|
|
102
|
+
`Node#key?` (alias `#has_attribute?`) :: attribute presence.
|
|
103
|
+
`Node#children` :: `NodeSet` of all children (elements, text, comments, …).
|
|
104
|
+
`Node#element_children` :: `NodeSet` of element children only.
|
|
105
|
+
`Node#first_element_child`, `#last_element_child` :: first/last element child (skip text nodes).
|
|
106
|
+
`Node#next_element`, `#previous_element` :: next/prev sibling element.
|
|
107
|
+
`Node#parent`, `#next_sibling`, `#previous_sibling` :: tree navigation.
|
|
108
|
+
`Node#line` :: 1-based source line number.
|
|
109
|
+
`Node#type` (alias `#node_type`) :: integer type code. Element predicates: `#element?`, `#text?`, `#comment?`, `#cdata?`, `#processing_instruction?`.
|
|
110
|
+
|
|
111
|
+
Example — walk all book titles:
|
|
112
|
+
|
|
113
|
+
[source,ruby]
|
|
114
|
+
----
|
|
115
|
+
doc.root.children.select(&:element?).each do |book|
|
|
116
|
+
title = book.children.find { |c| c.element? && c.name == "title" }
|
|
117
|
+
puts "#{book[:id]}: #{title&.content}"
|
|
118
|
+
end
|
|
119
|
+
# b1: Refactoring
|
|
120
|
+
# b2: Programmer en Ruby
|
|
121
|
+
----
|
|
122
|
+
|
|
123
|
+
=== Tree iteration
|
|
124
|
+
|
|
125
|
+
`Node#traverse` walks the subtree in document order via a single C-side
|
|
126
|
+
callback (one FFI call for the whole traversal, not one per node):
|
|
127
|
+
|
|
128
|
+
[source,ruby]
|
|
129
|
+
----
|
|
130
|
+
doc.root.traverse do |node|
|
|
131
|
+
case node
|
|
132
|
+
when Leptris::XML::Element then puts "E #{node.name}"
|
|
133
|
+
when Leptris::XML::Text then puts "T #{node.content.inspect}"
|
|
134
|
+
when Leptris::XML::Comment then puts "C #{node.content.inspect}"
|
|
135
|
+
end
|
|
136
|
+
end
|
|
137
|
+
----
|
|
138
|
+
|
|
139
|
+
== Searching: XPath and CSS
|
|
140
|
+
|
|
141
|
+
`Document` and `Element` (via `Leptris::XML::Searchable`) support:
|
|
142
|
+
|
|
143
|
+
[horizontal]
|
|
144
|
+
`#xpath(*exprs)` :: evaluate XPath; returns `NodeSet`, `true`/`false`, `Float`, or `String` depending on the expression.
|
|
145
|
+
`#at_xpath(*exprs)` :: first match (or scalar), like `xpath(*exprs).first`.
|
|
146
|
+
`#css(*selectors)` :: minimal CSS-to-XPath translation, then `xpath`.
|
|
147
|
+
`#at_css(*selectors)` :: first match of `css`.
|
|
148
|
+
`#search(*exprs)` :: dispatches on syntax — `/`-prefixed or `,`-separated → `xpath`, otherwise `css`.
|
|
149
|
+
`#at(*exprs)` :: first match of `search`.
|
|
150
|
+
|
|
151
|
+
[source,ruby]
|
|
152
|
+
----
|
|
153
|
+
doc.xpath("//book") # => NodeSet of both <book>
|
|
154
|
+
doc.xpath("count(//book)") # => 2.0
|
|
155
|
+
doc.xpath("//book[@lang='fr']/title") # => NodeSet[<title>Programmer en Ruby</title>]
|
|
156
|
+
doc.at_xpath("//book[@id='b1']") # => <book id="b1" ...>
|
|
157
|
+
doc.at_xpath("string(//book[1]/@id)") # => "b1"
|
|
158
|
+
|
|
159
|
+
doc.css("book[lang='en'] title") # => NodeSet[<title>Refactoring</title>]
|
|
160
|
+
doc.at_css("book#b1 title") # => <title>Refactoring</title> (id selector)
|
|
161
|
+
doc.css("book:first-child") # first <book>
|
|
162
|
+
----
|
|
163
|
+
|
|
164
|
+
XPath result type follows XPath 1.0 semantics:
|
|
165
|
+
`count(...)` → `Float`, `boolean(...)` → `true`/`false`,
|
|
166
|
+
`string(...)` → `String`, otherwise a `Leptris::XML::NodeSet`.
|
|
167
|
+
|
|
168
|
+
=== Supported CSS selectors
|
|
169
|
+
|
|
170
|
+
Minimal subset (translated to XPath via `Leptris::XML::CssToXPath`):
|
|
171
|
+
|
|
172
|
+
* Type/universal: `book`, `*`
|
|
173
|
+
* Class/ID: `.highlight`, `#b1`
|
|
174
|
+
* Attribute presence: `[lang]`
|
|
175
|
+
* Attribute value: `[lang='en']`, `[lang~='en']`, `[lang^='en']`, `[lang$='en']`, `[lang*='en']`
|
|
176
|
+
* Combinators: descendant (space), child (`>`), comma (multi-selector)
|
|
177
|
+
* Pseudo-classes: `:first-child`, `:last-child`, `:only-child`, `:empty`, `:root`, `:not(...)`
|
|
178
|
+
|
|
179
|
+
For anything more sophisticated, drop down to `xpath`.
|
|
180
|
+
|
|
181
|
+
== Building and mutating
|
|
182
|
+
|
|
183
|
+
Documents expose factory methods; elements expose mutation methods:
|
|
184
|
+
|
|
185
|
+
[source,ruby]
|
|
186
|
+
----
|
|
187
|
+
doc = Leptris::XML.parse("<root/>")
|
|
188
|
+
book = doc.create_element("book")
|
|
189
|
+
book[:id] = "b3"
|
|
190
|
+
book.add_child(doc.create_element("title")).content = "New book"
|
|
191
|
+
doc.root.add_child(book)
|
|
192
|
+
|
|
193
|
+
puts doc.to_xml
|
|
194
|
+
# <?xml version="1.0"?>
|
|
195
|
+
# <root><book id="b3"><title>New book</title></book></root>
|
|
196
|
+
----
|
|
197
|
+
|
|
198
|
+
[horizontal]
|
|
199
|
+
`Document#create_element(name)` :: detached element owned by the document.
|
|
200
|
+
`Document#create_text_node(str)`, `#create_comment(str)`, `#create_cdata(str)` :: text-class factories.
|
|
201
|
+
`Document#create_processing_instruction(target, data)` :: PI factory.
|
|
202
|
+
`Document#fragment(markup)` :: parse a markup fragment (multiple top-level children allowed).
|
|
203
|
+
`Element#name=`, `#content=` :: rename / replace inner text.
|
|
204
|
+
`Element#[]=` (alias `#set_attribute`) :: add/update an attribute.
|
|
205
|
+
`Element#remove_attribute` (alias `#delete`) :: drop an attribute.
|
|
206
|
+
`Element#add_child(node_or_markup)` (alias `#<<`) :: append a Node, or parse+append a markup String.
|
|
207
|
+
`Element#prepend_child(node)` :: insert as the first child.
|
|
208
|
+
`Element#add_next_sibling(node)`, `#add_previous_sibling(node)` :: sibling insertion.
|
|
209
|
+
`Element#remove_child(node)` :: detach (does not free).
|
|
210
|
+
`Element#children=` :: replace all children.
|
|
211
|
+
`Element#replace(node)` / `#swap(node)` :: replace in parent.
|
|
212
|
+
`Element#wrap(node_or_markup)` :: wrap this element in a new one.
|
|
213
|
+
`Node#unlink` :: detach from the tree.
|
|
214
|
+
|
|
215
|
+
=== Building from scratch (no parse)
|
|
216
|
+
|
|
217
|
+
[source,ruby]
|
|
218
|
+
----
|
|
219
|
+
# Create an empty Document by parsing a sentinel and replacing the root,
|
|
220
|
+
# or build incrementally on a one-element seed.
|
|
221
|
+
doc = Leptris::XML.parse("<root/>")
|
|
222
|
+
doc.root.name = "catalog"
|
|
223
|
+
# ... then create_element / add_child as above.
|
|
224
|
+
----
|
|
225
|
+
|
|
226
|
+
== Namespaces
|
|
227
|
+
|
|
228
|
+
[horizontal]
|
|
229
|
+
`Element#namespace` :: the element's in-scope namespace as a `Namespace` (or `nil`).
|
|
230
|
+
`Element#namespaces` :: all in-scope namespaces (inherited from ancestors) as a `{prefix_or_xmlns => href}` hash.
|
|
231
|
+
`Element#namespace_definitions` :: only namespaces declared directly on this element.
|
|
232
|
+
`Element#add_namespace_definition(prefix, href)` (alias `#add_namespace`) :: declare `xmlns:prefix="href"` on this element.
|
|
233
|
+
`Element#default_namespace=(href)` :: declare/replace `xmlns="href"`.
|
|
234
|
+
`Element#remove_namespace_definition(prefix)` :: drop a declaration.
|
|
235
|
+
|
|
236
|
+
[source,ruby]
|
|
237
|
+
----
|
|
238
|
+
root = doc.root
|
|
239
|
+
root.add_namespace_definition("t", "https://example.org/types")
|
|
240
|
+
puts root.namespaces
|
|
241
|
+
# {"xmlns"=>"http://example.org/ns", "xmlns:t"=>"https://example.org/types"}
|
|
242
|
+
|
|
243
|
+
# XPath with prefixes is dispatched straight to libleptris, which resolves
|
|
244
|
+
# prefixes using the in-scope namespace declarations.
|
|
245
|
+
doc.xpath("//t:title")
|
|
246
|
+
----
|
|
247
|
+
|
|
248
|
+
== Serialization and canonicalization
|
|
249
|
+
|
|
250
|
+
[horizontal]
|
|
251
|
+
`Document#to_xml(indent: 0, no_decl: false, encoding: nil)` (aliases `#to_s`, `#serialize`) :: serialize the whole document.
|
|
252
|
+
`Element#to_xml(...)` :: serialize a subtree.
|
|
253
|
+
`Document#save(path, **opts)` :: serialize to a file.
|
|
254
|
+
`Document#canonicalize(version, inclusive_ns, with_comments:, exclusive:, mode:)` (alias `#c14n`) :: canonical XML.
|
|
255
|
+
`Element#canonicalize(...)` :: subtree canonicalization.
|
|
256
|
+
|
|
257
|
+
[source,ruby]
|
|
258
|
+
----
|
|
259
|
+
doc.to_xml # one-line, no indent
|
|
260
|
+
doc.to_xml(indent: 2) # pretty-printed
|
|
261
|
+
doc.canonicalize # C14N 1.0
|
|
262
|
+
doc.canonicalize(Leptris::XML::FFI::C14N_1_1) # C14N 1.1
|
|
263
|
+
doc.canonicalize(exclusive: true) # Exclusive C14N
|
|
264
|
+
doc.canonicalize(with_comments: true) # keep comments
|
|
265
|
+
doc.canonicalize(exclusive: true, inclusive_namespaces: ["ds"]) # InclusiveNamespaces
|
|
266
|
+
----
|
|
267
|
+
|
|
268
|
+
== SAX parsing
|
|
269
|
+
|
|
270
|
+
For very large documents, use the streaming SAX parser. Subclass
|
|
271
|
+
`Leptris::XML::SAX::Document` and override the events you care about:
|
|
272
|
+
|
|
273
|
+
[source,ruby]
|
|
274
|
+
----
|
|
275
|
+
class Counter < Leptris::XML::SAX::Document
|
|
276
|
+
attr_reader :elements, :depth
|
|
277
|
+
def initialize
|
|
278
|
+
@elements = 0
|
|
279
|
+
@depth = 0
|
|
280
|
+
end
|
|
281
|
+
|
|
282
|
+
def start_element(name, attrs = [])
|
|
283
|
+
@elements += 1
|
|
284
|
+
@depth += 1
|
|
285
|
+
puts " " * (@depth - 1) + "<#{name}>"
|
|
286
|
+
end
|
|
287
|
+
|
|
288
|
+
def end_element(name)
|
|
289
|
+
@depth -= 1
|
|
290
|
+
end
|
|
291
|
+
|
|
292
|
+
def characters(str)
|
|
293
|
+
puts " " * @depth + "text: #{str.inspect}" unless str.strip.empty?
|
|
294
|
+
end
|
|
295
|
+
end
|
|
296
|
+
|
|
297
|
+
parser = Leptris::XML::SAX::Parser.new(Counter.new)
|
|
298
|
+
parser.parse(File.open("huge.xml")) # streams in 4 KB chunks
|
|
299
|
+
----
|
|
300
|
+
|
|
301
|
+
`SAX::Parser#parse` accepts a `String`, an `IO`, or any object responding
|
|
302
|
+
to `#read`. The handler callbacks are:
|
|
303
|
+
|
|
304
|
+
[horizontal]
|
|
305
|
+
`start_document`, `end_document` :: document boundaries.
|
|
306
|
+
`xmldecl(version, encoding, standalone)` :: XML declaration.
|
|
307
|
+
`start_element(name, attrs)`, `end_element(name)` :: element events; `attrs` is an array of `[name, value]` pairs in source order.
|
|
308
|
+
`characters(str)`, `comment(str)`, `cdata_block(str)` :: text-class events.
|
|
309
|
+
`processing_instruction(name, content)` :: PI event.
|
|
310
|
+
`start_prefix_mapping(prefix, uri)`, `end_prefix_mapping(prefix)` :: namespace events.
|
|
311
|
+
`warning(str)`, `error(msg, line, col)` :: recoverable parser messages.
|
|
312
|
+
|
|
313
|
+
== Memory model
|
|
314
|
+
|
|
315
|
+
`Document` is the only object that owns C memory. Everything else
|
|
316
|
+
(`Element`, `Text`, `Attr`, `NodeSet`, …) is a *borrowed handle* that
|
|
317
|
+
is valid only while its Document is alive.
|
|
318
|
+
|
|
319
|
+
* Free a document explicitly with `Document#free`. After `#free`, any
|
|
320
|
+
further method call on the document or its nodes raises
|
|
321
|
+
`Leptris::XML::UseAfterFreeError`.
|
|
322
|
+
* If you don't call `#free`, GC will — a finalizer captures the raw
|
|
323
|
+
pointer address (not the Ruby wrapper) and calls
|
|
324
|
+
`leptris_document_free` exactly once.
|
|
325
|
+
* `NodeSet`s holding XPath results own their own
|
|
326
|
+
`LeptrisXPathResult` and free it on GC.
|
|
327
|
+
* Don't hold a `Node` reference past the lifetime of its `Document`.
|
|
328
|
+
The C memory is gone; using the wrapper is undefined behaviour.
|
|
329
|
+
|
|
330
|
+
== Errors
|
|
331
|
+
|
|
332
|
+
All Leptris errors descend from `Leptris::XML::Error`:
|
|
333
|
+
|
|
334
|
+
[horizontal]
|
|
335
|
+
`ParseError` :: raised by `parse` / `parse_file` / SAX on malformed input.
|
|
336
|
+
`XPathError` :: raised by `xpath` on malformed or unsupported expressions.
|
|
337
|
+
`UseAfterFreeError` :: raised when calling methods on a freed `Document`.
|
|
338
|
+
`Error` :: generic (mutation precondition failures, etc.).
|
|
339
|
+
|
|
340
|
+
[source,ruby]
|
|
341
|
+
----
|
|
342
|
+
begin
|
|
343
|
+
Leptris::XML.parse("<unclosed>")
|
|
344
|
+
rescue Leptris::XML::ParseError => e
|
|
345
|
+
warn "parse failed: #{e.message}"
|
|
346
|
+
end
|
|
347
|
+
----
|
|
348
|
+
|
|
349
|
+
== Migrating from Nokogiri
|
|
350
|
+
|
|
351
|
+
For most read-only XPath use cases the swap is mechanical:
|
|
352
|
+
|
|
353
|
+
[source,ruby]
|
|
354
|
+
----
|
|
355
|
+
# Nokogiri
|
|
356
|
+
require "nokogiri"
|
|
357
|
+
doc = Nokogiri::XML(File.read("doc.xml"))
|
|
358
|
+
doc.xpath("//item[@id='1']").each { |n| puts n.text }
|
|
359
|
+
|
|
360
|
+
# Leptris
|
|
361
|
+
require "leptris"
|
|
362
|
+
doc = Leptris::XML.parse(File.read("doc.xml"))
|
|
363
|
+
doc.xpath("//item[@id='1']").each { |n| puts n.content }
|
|
364
|
+
----
|
|
365
|
+
|
|
366
|
+
Notable differences:
|
|
367
|
+
|
|
368
|
+
* `Node#text` exists but the canonical name is `#content` (Nokogiri uses both).
|
|
369
|
+
* `Node#children` includes whitespace text nodes (same as Nokogiri); use
|
|
370
|
+
`#element_children` or `#first_element_child` to skip them.
|
|
371
|
+
* CSS support is intentionally minimal — for advanced selectors, drop to
|
|
372
|
+
`xpath`.
|
|
373
|
+
* No `Nokogiri::HTML` or `Nokogiri::CSS` parser. Leptris is XML-only.
|
|
374
|
+
* No XSLT, no RelaxNG / DTD validation, no schema caching.
|
|
375
|
+
* No built-in JRuby / TruffleRuby support — only CRuby via `ffi`.
|
|
376
|
+
|
|
377
|
+
== Performance
|
|
378
|
+
|
|
379
|
+
On the benchmark suite in `benchmark/leptris_vs_nokogiri.rb` (Ruby 3.3,
|
|
380
|
+
libleptris v0.13+, macOS arm64), Leptris matches or beats Nokogiri on
|
|
381
|
+
parse, XPath, serialize, and full-tree traverse for the small and
|
|
382
|
+
medium documents that dominate real-world XML workloads. Run the
|
|
383
|
+
benchmark locally for numbers on your hardware:
|
|
384
|
+
|
|
385
|
+
[source,shell]
|
|
386
|
+
----
|
|
387
|
+
bundle exec ruby benchmark/leptris_vs_nokogiri.rb
|
|
388
|
+
----
|
|
389
|
+
|
|
390
|
+
== Development
|
|
391
|
+
|
|
392
|
+
[source,shell]
|
|
393
|
+
----
|
|
394
|
+
bundle install # install Ruby deps
|
|
395
|
+
bundle exec rspec # full test suite (176 specs)
|
|
396
|
+
bundle exec rspec spec/xml/xpath_spec.rb:42 # one example by line
|
|
397
|
+
bundle exec rubocop # lint
|
|
398
|
+
----
|
|
399
|
+
|
|
400
|
+
CI pins libleptris to a released tag (currently v0.26.7) and builds it
|
|
401
|
+
from source on each runner; see `.github/workflows/build.yml`.
|
|
402
|
+
|
|
403
|
+
== License
|
|
404
|
+
|
|
405
|
+
MIT — see link:LICENSE[LICENSE].
|
data/Rakefile
ADDED
|
@@ -0,0 +1,217 @@
|
|
|
1
|
+
# TODO 1 — Architecture: C-backed Nokogiri-compatible Ruby binding
|
|
2
|
+
|
|
3
|
+
## Goal
|
|
4
|
+
|
|
5
|
+
Rewrite `leptris-ruby` to be a **thin FFI wrapper** around libleptris v0.4.2,
|
|
6
|
+
exposing a **Nokogiri-compatible API**. The C DOM is the single source of
|
|
7
|
+
truth — no Ruby-side tree copy.
|
|
8
|
+
|
|
9
|
+
## Current state (problem)
|
|
10
|
+
|
|
11
|
+
The existing `leptris-ruby` has:
|
|
12
|
+
- A **pure-Ruby** XML tree model (Document < Element, Node, NodeSet)
|
|
13
|
+
- A FFI bridge that does a **one-shot copy** from C to Ruby on parse
|
|
14
|
+
- A **pure-Ruby XPath engine** (lexer, parser, compiler, VM in Ruby)
|
|
15
|
+
|
|
16
|
+
This defeats libleptris's performance: the C library's optimized bytecode
|
|
17
|
+
VM + element index are never used. XPath goes through the slow Ruby engine.
|
|
18
|
+
|
|
19
|
+
## Target architecture
|
|
20
|
+
|
|
21
|
+
```
|
|
22
|
+
User Ruby code
|
|
23
|
+
↓
|
|
24
|
+
Leptris::XML::Document / Node / NodeSet (thin Ruby wrappers)
|
|
25
|
+
↓ FFI
|
|
26
|
+
libleptris v0.4.2 (C99: DOM, XPath bytecode VM, element index)
|
|
27
|
+
```
|
|
28
|
+
|
|
29
|
+
Key principles:
|
|
30
|
+
- **Every Ruby method is a single FFI call** to the C library.
|
|
31
|
+
- **No Ruby-side tree copy.** The C DOM is the truth; Ruby objects are
|
|
32
|
+
handles (wrapping opaque pointers).
|
|
33
|
+
- **XPath goes through C.** `doc.xpath('//book')` calls
|
|
34
|
+
`leptris_xpath_eval` directly. No Ruby XPath engine.
|
|
35
|
+
- **SAX goes through C.** `Leptris::XML::SAX::Parser` wraps
|
|
36
|
+
`leptris_sax_parse` with Ruby callback dispatch.
|
|
37
|
+
|
|
38
|
+
## Module structure (Nokogiri-compatible)
|
|
39
|
+
|
|
40
|
+
```ruby
|
|
41
|
+
module Leptris
|
|
42
|
+
module XML
|
|
43
|
+
# Top-level parse entry points
|
|
44
|
+
def self.parse(string_or_io) → Document
|
|
45
|
+
def self.parse_options → ParseOptions
|
|
46
|
+
|
|
47
|
+
class Document < Node
|
|
48
|
+
def root → Element (or nil)
|
|
49
|
+
def create_element(name) → Element
|
|
50
|
+
def create_text_node(text) → Text
|
|
51
|
+
def to_xml(options) → String
|
|
52
|
+
def xpath(expr) → NodeSet | Float | String | Boolean
|
|
53
|
+
def at_xpath(expr) → Node (or nil)
|
|
54
|
+
def search(expr) → NodeSet
|
|
55
|
+
def canonicalize(...) → String
|
|
56
|
+
def free → void (explicit)
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
class Node
|
|
60
|
+
include Searchable
|
|
61
|
+
|
|
62
|
+
def name → String
|
|
63
|
+
def content / text → String
|
|
64
|
+
def [](attr_name) → String (or nil)
|
|
65
|
+
def []=(attr_name, value)
|
|
66
|
+
def attributes → Hash {String => Attr}
|
|
67
|
+
def children → NodeSet
|
|
68
|
+
def child → Node (or nil)
|
|
69
|
+
def first_element_child → Element (or nil)
|
|
70
|
+
def last_element_child → Element (or nil)
|
|
71
|
+
def next_sibling → Node (or nil)
|
|
72
|
+
def previous_sibling → Node (or nil)
|
|
73
|
+
def parent → Node (or nil)
|
|
74
|
+
def document → Document
|
|
75
|
+
def type → Integer (element/text/comment/cdata/pi)
|
|
76
|
+
def element? → Boolean
|
|
77
|
+
def text? → Boolean
|
|
78
|
+
def comment? → Boolean
|
|
79
|
+
def cdata? → Boolean
|
|
80
|
+
def processing_instruction? → Boolean
|
|
81
|
+
def add_child(node) → Node
|
|
82
|
+
def add_next_sibling(node) → Node
|
|
83
|
+
def add_previous_sibling(node) → Node
|
|
84
|
+
def remove → Node
|
|
85
|
+
def replace(node) → Node
|
|
86
|
+
def to_xml(options) → String
|
|
87
|
+
def inner_html → String
|
|
88
|
+
def traverse(&block)
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
class Element < Node
|
|
92
|
+
def add_class(name)
|
|
93
|
+
def remove_class(name)
|
|
94
|
+
def classes → Array<String>
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
class Text < Node; end
|
|
98
|
+
class Comment < Node; end
|
|
99
|
+
class CDATA < Node; end
|
|
100
|
+
class ProcessingInstruction < Node; end
|
|
101
|
+
class Attr
|
|
102
|
+
def name → String
|
|
103
|
+
def value → String
|
|
104
|
+
def value=(val)
|
|
105
|
+
def parent → Element
|
|
106
|
+
def remove
|
|
107
|
+
end
|
|
108
|
+
|
|
109
|
+
class NodeSet
|
|
110
|
+
include Enumerable
|
|
111
|
+
include Searchable
|
|
112
|
+
|
|
113
|
+
def length / size → Integer
|
|
114
|
+
def first(n) → Node | NodeSet
|
|
115
|
+
def last → Node
|
|
116
|
+
def [](index) → Node
|
|
117
|
+
def each(&block)
|
|
118
|
+
def empty? → Boolean
|
|
119
|
+
def xpath(expr) → NodeSet
|
|
120
|
+
def search(expr) → NodeSet
|
|
121
|
+
def to_xml → String
|
|
122
|
+
def inner_text → String
|
|
123
|
+
end
|
|
124
|
+
|
|
125
|
+
module Searchable
|
|
126
|
+
def xpath(*paths) → NodeSet | Float | String | Boolean
|
|
127
|
+
def at_xpath(*paths) → Node (or nil)
|
|
128
|
+
def css(*selectors) → NodeSet (converts CSS to XPath)
|
|
129
|
+
def at_css(*selectors) → Node (or nil)
|
|
130
|
+
def search(*args) → NodeSet (auto-detect CSS/XPath)
|
|
131
|
+
def at(*args) → Node (or nil)
|
|
132
|
+
end
|
|
133
|
+
|
|
134
|
+
class ParseOptions
|
|
135
|
+
DEFAULT_XML = ...
|
|
136
|
+
RECOVER = ...
|
|
137
|
+
NOERROR = ...
|
|
138
|
+
NOWARNING = ...
|
|
139
|
+
NOCDATA = ...
|
|
140
|
+
STRICT = ...
|
|
141
|
+
end
|
|
142
|
+
|
|
143
|
+
module SAX
|
|
144
|
+
class Parser
|
|
145
|
+
def initialize(handler = DocHandler.new)
|
|
146
|
+
def parse(io_or_string)
|
|
147
|
+
end
|
|
148
|
+
|
|
149
|
+
class Document
|
|
150
|
+
def start_element(name, attrs = [])
|
|
151
|
+
def end_element(name)
|
|
152
|
+
def characters(string)
|
|
153
|
+
def start_document
|
|
154
|
+
def end_document
|
|
155
|
+
def comment(string)
|
|
156
|
+
def cdata(string)
|
|
157
|
+
def processing_instruction(name, content)
|
|
158
|
+
def error(message, line, column)
|
|
159
|
+
end
|
|
160
|
+
end
|
|
161
|
+
end
|
|
162
|
+
end
|
|
163
|
+
```
|
|
164
|
+
|
|
165
|
+
## Memory model
|
|
166
|
+
|
|
167
|
+
- **Document** owns the C document pool. `Leptris::XML::Document.new`
|
|
168
|
+
calls `leptris_parse_string` → returns a `LeptrisDocument` pointer.
|
|
169
|
+
`Document#free` calls `leptris_document_free`. Auto-free via
|
|
170
|
+
`ObjectSpace.define_finalizer` as a safety net (but callers should
|
|
171
|
+
call `#free` explicitly for predictable lifecycle).
|
|
172
|
+
- **Node / Element / Text etc.** are **non-owning handles** wrapping
|
|
173
|
+
a C pointer. The pointer is valid as long as the parent Document
|
|
174
|
+
is alive. Freeing a Node just drops the Ruby wrapper; the C node
|
|
175
|
+
lives until `Document#free`.
|
|
176
|
+
- **NodeSet** wraps a `LeptrisXPathResult` pointer from
|
|
177
|
+
`leptris_xpath_eval`. Freeing a NodeSet calls
|
|
178
|
+
`leptris_xpath_result_free`.
|
|
179
|
+
- **Attr** wraps a C attribute pointer (owned by the parent element's
|
|
180
|
+
pool). Non-owning.
|
|
181
|
+
|
|
182
|
+
## CSS support
|
|
183
|
+
|
|
184
|
+
Nokogiri supports CSS selectors via `css()` and `at_css()`. Leptris
|
|
185
|
+
doesn't have a CSS engine in C, so CSS-to-XPath conversion must be
|
|
186
|
+
done in Ruby. Options:
|
|
187
|
+
1. Use the `css_parser` gem (depends on `racc`).
|
|
188
|
+
2. Write a minimal CSS-to-XPath converter in pure Ruby.
|
|
189
|
+
|
|
190
|
+
For v0.4.2 compatibility, option 2 (minimal converter) is recommended.
|
|
191
|
+
Nokogiri's CSS selector support is comprehensive but the common subset
|
|
192
|
+
is small: `tag`, `.class`, `#id`, `> child`, `descendant`,
|
|
193
|
+
`[attr]`, `[attr=value]`, `:first-child`, `:last-child`, `:not(...)`.
|
|
194
|
+
|
|
195
|
+
## Dependencies
|
|
196
|
+
|
|
197
|
+
```ruby
|
|
198
|
+
# leptris.gemspec
|
|
199
|
+
spec.add_dependency 'ffi', '~> 1.16'
|
|
200
|
+
```
|
|
201
|
+
|
|
202
|
+
No other runtime dependencies. No C extension compilation needed —
|
|
203
|
+
just FFI to the pre-built libleptris shared library.
|
|
204
|
+
|
|
205
|
+
## Reference material
|
|
206
|
+
|
|
207
|
+
- Nokogiri source: `~/src/external/nokogiri/`
|
|
208
|
+
- `lib/nokogiri/xml/node.rb` — 77 public methods
|
|
209
|
+
- `lib/nokogiri/xml/node_set.rb` — 31 public methods
|
|
210
|
+
- `lib/nokogiri/xml/document.rb` — 22 public methods
|
|
211
|
+
- `lib/nokogiri/xml/searchable.rb` — xpath/css/search module
|
|
212
|
+
- libleptris public headers: `src/include/leptris/`
|
|
213
|
+
- `types.h` — opaque handle typedefs
|
|
214
|
+
- `dom/document.h`, `dom/element.h`, `dom/serialize.h`
|
|
215
|
+
- `xpath/xpath.h` — XPath eval API
|
|
216
|
+
- `sax/sax.h` — SAX parser API
|
|
217
|
+
- libleptris v0.4.2: tag `v0.4.2` on `github.com:leptris/leptris`
|