makiri 0.1.0-x86_64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +7 -0
- data/.github/workflows/ci.yml +73 -0
- data/.github/workflows/conformance.yml +94 -0
- data/.github/workflows/release.yml +223 -0
- data/.github/workflows/security.yml +95 -0
- data/.gitmodules +3 -0
- data/CHANGELOG.md +102 -0
- data/LICENSE +176 -0
- data/NOTICE +12 -0
- data/README.md +134 -0
- data/Rakefile +150 -0
- data/lib/makiri/3.2/makiri.so +0 -0
- data/lib/makiri/3.3/makiri.so +0 -0
- data/lib/makiri/3.4/makiri.so +0 -0
- data/lib/makiri/4.0/makiri.so +0 -0
- data/lib/makiri/attribute.rb +13 -0
- data/lib/makiri/cdata.rb +6 -0
- data/lib/makiri/comment.rb +6 -0
- data/lib/makiri/css.rb +11 -0
- data/lib/makiri/document.rb +82 -0
- data/lib/makiri/document_fragment.rb +21 -0
- data/lib/makiri/document_type.rb +14 -0
- data/lib/makiri/element.rb +17 -0
- data/lib/makiri/node.rb +221 -0
- data/lib/makiri/node_set.rb +105 -0
- data/lib/makiri/processing_instruction.rb +8 -0
- data/lib/makiri/text.rb +16 -0
- data/lib/makiri/version.rb +5 -0
- data/lib/makiri/xpath.rb +14 -0
- data/lib/makiri/xpath_context.rb +41 -0
- data/lib/makiri.rb +47 -0
- data/script/build_native_gem.rb +50 -0
- data/script/check_c_safety.rb +238 -0
- data/script/check_c_safety_allowlist.yml +12 -0
- data/sig/makiri.rbs +4 -0
- metadata +127 -0
data/LICENSE
ADDED
|
@@ -0,0 +1,176 @@
|
|
|
1
|
+
Apache License
|
|
2
|
+
Version 2.0, January 2004
|
|
3
|
+
http://www.apache.org/licenses/
|
|
4
|
+
|
|
5
|
+
TERMS AND CONDITIONS FOR USE, REPRODUCTION, AND DISTRIBUTION
|
|
6
|
+
|
|
7
|
+
1. Definitions.
|
|
8
|
+
|
|
9
|
+
"License" shall mean the terms and conditions for use, reproduction,
|
|
10
|
+
and distribution as defined by Sections 1 through 9 of this document.
|
|
11
|
+
|
|
12
|
+
"Licensor" shall mean the copyright owner or entity authorized by
|
|
13
|
+
the copyright owner that is granting the License.
|
|
14
|
+
|
|
15
|
+
"Legal Entity" shall mean the union of the acting entity and all
|
|
16
|
+
other entities that control, are controlled by, or are under common
|
|
17
|
+
control with that entity. For the purposes of this definition,
|
|
18
|
+
"control" means (i) the power, direct or indirect, to cause the
|
|
19
|
+
direction or management of such entity, whether by contract or
|
|
20
|
+
otherwise, or (ii) ownership of fifty percent (50%) or more of the
|
|
21
|
+
outstanding shares, or (iii) beneficial ownership of such entity.
|
|
22
|
+
|
|
23
|
+
"You" (or "Your") shall mean an individual or Legal Entity
|
|
24
|
+
exercising permissions granted by this License.
|
|
25
|
+
|
|
26
|
+
"Source" form shall mean the preferred form for making modifications,
|
|
27
|
+
including but not limited to software source code, documentation
|
|
28
|
+
source, and configuration files.
|
|
29
|
+
|
|
30
|
+
"Object" form shall mean any form resulting from mechanical
|
|
31
|
+
transformation or translation of a Source form, including but
|
|
32
|
+
not limited to compiled object code, generated documentation,
|
|
33
|
+
and conversions to other media types.
|
|
34
|
+
|
|
35
|
+
"Work" shall mean the work of authorship, whether in Source or
|
|
36
|
+
Object form, made available under the License, as indicated by a
|
|
37
|
+
copyright notice that is included in or attached to the work
|
|
38
|
+
(an example is provided in the Appendix below).
|
|
39
|
+
|
|
40
|
+
"Derivative Works" shall mean any work, whether in Source or Object
|
|
41
|
+
form, that is based on (or derived from) the Work and for which the
|
|
42
|
+
editorial revisions, annotations, elaborations, or other modifications
|
|
43
|
+
represent, as a whole, an original work of authorship. For the purposes
|
|
44
|
+
of this License, Derivative Works shall not include works that remain
|
|
45
|
+
separable from, or merely link (or bind by name) to the interfaces of,
|
|
46
|
+
the Work and Derivative Works thereof.
|
|
47
|
+
|
|
48
|
+
"Contribution" shall mean any work of authorship, including
|
|
49
|
+
the original version of the Work and any modifications or additions
|
|
50
|
+
to that Work or Derivative Works thereof, that is intentionally
|
|
51
|
+
submitted to Licensor for inclusion in the Work by the copyright owner
|
|
52
|
+
or by an individual or Legal Entity authorized to submit on behalf of
|
|
53
|
+
the copyright owner. For the purposes of this definition, "submitted"
|
|
54
|
+
means any form of electronic, verbal, or written communication sent
|
|
55
|
+
to the Licensor or its representatives, including but not limited to
|
|
56
|
+
communication on electronic mailing lists, source code control systems,
|
|
57
|
+
and issue tracking systems that are managed by, or on behalf of, the
|
|
58
|
+
Licensor for the purpose of discussing and improving the Work, but
|
|
59
|
+
excluding communication that is conspicuously marked or otherwise
|
|
60
|
+
designated in writing by the copyright owner as "Not a Contribution."
|
|
61
|
+
|
|
62
|
+
"Contributor" shall mean Licensor and any individual or Legal Entity
|
|
63
|
+
on behalf of whom a Contribution has been received by Licensor and
|
|
64
|
+
subsequently incorporated within the Work.
|
|
65
|
+
|
|
66
|
+
2. Grant of Copyright License. Subject to the terms and conditions of
|
|
67
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
68
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
69
|
+
copyright license to reproduce, prepare Derivative Works of,
|
|
70
|
+
publicly display, publicly perform, sublicense, and distribute the
|
|
71
|
+
Work and such Derivative Works in Source or Object form.
|
|
72
|
+
|
|
73
|
+
3. Grant of Patent License. Subject to the terms and conditions of
|
|
74
|
+
this License, each Contributor hereby grants to You a perpetual,
|
|
75
|
+
worldwide, non-exclusive, no-charge, royalty-free, irrevocable
|
|
76
|
+
(except as stated in this section) patent license to make, have made,
|
|
77
|
+
use, offer to sell, sell, import, and otherwise transfer the Work,
|
|
78
|
+
where such license applies only to those patent claims licensable
|
|
79
|
+
by such Contributor that are necessarily infringed by their
|
|
80
|
+
Contribution(s) alone or by combination of their Contribution(s)
|
|
81
|
+
with the Work to which such Contribution(s) was submitted. If You
|
|
82
|
+
institute patent litigation against any entity (including a
|
|
83
|
+
cross-claim or counterclaim in a lawsuit) alleging that the Work
|
|
84
|
+
or a Contribution incorporated within the Work constitutes direct
|
|
85
|
+
or contributory patent infringement, then any patent licenses
|
|
86
|
+
granted to You under this License for that Work shall terminate
|
|
87
|
+
as of the date such litigation is filed.
|
|
88
|
+
|
|
89
|
+
4. Redistribution. You may reproduce and distribute copies of the
|
|
90
|
+
Work or Derivative Works thereof in any medium, with or without
|
|
91
|
+
modifications, and in Source or Object form, provided that You
|
|
92
|
+
meet the following conditions:
|
|
93
|
+
|
|
94
|
+
(a) You must give any other recipients of the Work or
|
|
95
|
+
Derivative Works a copy of this License; and
|
|
96
|
+
|
|
97
|
+
(b) You must cause any modified files to carry prominent notices
|
|
98
|
+
stating that You changed the files; and
|
|
99
|
+
|
|
100
|
+
(c) You must retain, in the Source form of any Derivative Works
|
|
101
|
+
that You distribute, all copyright, patent, trademark, and
|
|
102
|
+
attribution notices from the Source form of the Work,
|
|
103
|
+
excluding those notices that do not pertain to any part of
|
|
104
|
+
the Derivative Works; and
|
|
105
|
+
|
|
106
|
+
(d) If the Work includes a "NOTICE" text file as part of its
|
|
107
|
+
distribution, then any Derivative Works that You distribute must
|
|
108
|
+
include a readable copy of the attribution notices contained
|
|
109
|
+
within such NOTICE file, excluding those notices that do not
|
|
110
|
+
pertain to any part of the Derivative Works, in at least one
|
|
111
|
+
of the following places: within a NOTICE text file distributed
|
|
112
|
+
as part of the Derivative Works; within the Source form or
|
|
113
|
+
documentation, if provided along with the Derivative Works; or,
|
|
114
|
+
within a display generated by the Derivative Works, if and
|
|
115
|
+
wherever such third-party notices normally appear. The contents
|
|
116
|
+
of the NOTICE file are for informational purposes only and
|
|
117
|
+
do not modify the License. You may add Your own attribution
|
|
118
|
+
notices within Derivative Works that You distribute, alongside
|
|
119
|
+
or as an addendum to the NOTICE text from the Work, provided
|
|
120
|
+
that such additional attribution notices cannot be construed
|
|
121
|
+
as modifying the License.
|
|
122
|
+
|
|
123
|
+
You may add Your own copyright statement to Your modifications and
|
|
124
|
+
may provide additional or different license terms and conditions
|
|
125
|
+
for use, reproduction, or distribution of Your modifications, or
|
|
126
|
+
for any such Derivative Works as a whole, provided Your use,
|
|
127
|
+
reproduction, and distribution of the Work otherwise complies with
|
|
128
|
+
the conditions stated in this License.
|
|
129
|
+
|
|
130
|
+
5. Submission of Contributions. Unless You explicitly state otherwise,
|
|
131
|
+
any Contribution intentionally submitted for inclusion in the Work
|
|
132
|
+
by You to the Licensor shall be under the terms and conditions of
|
|
133
|
+
this License, without any additional terms or conditions.
|
|
134
|
+
Notwithstanding the above, nothing herein shall supersede or modify
|
|
135
|
+
the terms of any separate license agreement you may have executed
|
|
136
|
+
with Licensor regarding such Contributions.
|
|
137
|
+
|
|
138
|
+
6. Trademarks. This License does not grant permission to use the trade
|
|
139
|
+
names, trademarks, service marks, or product names of the Licensor,
|
|
140
|
+
except as required for describing the origin of the Work and
|
|
141
|
+
reproducing the content of the NOTICE file.
|
|
142
|
+
|
|
143
|
+
7. Disclaimer of Warranty. Unless required by applicable law or
|
|
144
|
+
agreed to in writing, Licensor provides the Work (and each
|
|
145
|
+
Contributor provides its Contributions) on an "AS IS" BASIS,
|
|
146
|
+
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or
|
|
147
|
+
implied, including, without limitation, any warranties or conditions
|
|
148
|
+
of TITLE, NON-INFRINGEMENT, MERCHANTABILITY, or FITNESS FOR A
|
|
149
|
+
PARTICULAR PURPOSE. You are solely responsible for determining the
|
|
150
|
+
appropriateness of using or redistributing the Work and assume any
|
|
151
|
+
risks associated with Your exercise of permissions under this License.
|
|
152
|
+
|
|
153
|
+
8. Limitation of Liability. In no event and under no legal theory,
|
|
154
|
+
whether in tort (including negligence), contract, or otherwise,
|
|
155
|
+
unless required by applicable law (such as deliberate and grossly
|
|
156
|
+
negligent acts) or agreed to in writing, shall any Contributor be
|
|
157
|
+
liable to You for damages, including any direct, indirect, special,
|
|
158
|
+
incidental, or consequential damages of any character arising as a
|
|
159
|
+
result of this License or out of the use or inability to use the
|
|
160
|
+
Work (including but not limited to damages for loss of goodwill,
|
|
161
|
+
work stoppage, computer failure or malfunction, or any and all
|
|
162
|
+
other commercial damages or losses), even if such Contributor
|
|
163
|
+
has been advised of the possibility of such damages.
|
|
164
|
+
|
|
165
|
+
9. Accepting Warranty or Support. While redistributing the Work or
|
|
166
|
+
Derivative Works thereof, You may choose to offer, and charge a
|
|
167
|
+
fee for, acceptance of support, warranty, indemnity, or other
|
|
168
|
+
liability obligations and/or rights consistent with this License.
|
|
169
|
+
However, in accepting such obligations, You may act only on Your
|
|
170
|
+
own behalf and on Your sole responsibility, not on behalf of any
|
|
171
|
+
other Contributor, and only if You agree to indemnify, defend,
|
|
172
|
+
and hold each Contributor harmless for any liability incurred by,
|
|
173
|
+
or claims asserted against, such Contributor by reason of your
|
|
174
|
+
accepting any such warranty or support.
|
|
175
|
+
|
|
176
|
+
END OF TERMS AND CONDITIONS
|
data/NOTICE
ADDED
|
@@ -0,0 +1,12 @@
|
|
|
1
|
+
Makiri
|
|
2
|
+
Copyright 2026 Makiri contributors
|
|
3
|
+
|
|
4
|
+
This product includes software developed by:
|
|
5
|
+
|
|
6
|
+
* Lexbor (vendored as a git submodule under vendor/lexbor)
|
|
7
|
+
Copyright (C) Alexander Borisov
|
|
8
|
+
Licensed under the Apache License, Version 2.0
|
|
9
|
+
https://lexbor.com / https://github.com/lexbor/lexbor
|
|
10
|
+
|
|
11
|
+
The XPath 1.0 engine under ext/makiri/xpath/ is an original implementation
|
|
12
|
+
written for this project. No code is derived from libxml2 or libxslt.
|
data/README.md
ADDED
|
@@ -0,0 +1,134 @@
|
|
|
1
|
+
# Makiri
|
|
2
|
+
|
|
3
|
+
Standards-oriented HTML5 parsing, CSS selector querying, and XPath 1.0
|
|
4
|
+
querying for Ruby, powered by Lexbor and a native XPath engine.
|
|
5
|
+
|
|
6
|
+
> [!WARNING]
|
|
7
|
+
> Status: early release. APIs and behavior may change before v1.0.
|
|
8
|
+
|
|
9
|
+
## What / Why
|
|
10
|
+
|
|
11
|
+
Makiri uses Lexbor for HTML5 parsing and CSS selector support, and implements
|
|
12
|
+
XPath 1.0 evaluation in its own native engine, with no libxml2 dependency.
|
|
13
|
+
|
|
14
|
+
* HTML5 parsing via [Lexbor](https://lexbor.com)
|
|
15
|
+
* Makiri uses Lexbor as the parsing backend and provides a Ruby-facing DOM/query layer.
|
|
16
|
+
* Lexbor-specific behavior is isolated in a thin compatibility layer
|
|
17
|
+
(`ext/makiri/lexbor_compat/`).
|
|
18
|
+
* CSS selector support via Lexbor
|
|
19
|
+
* Supports Lexbor-backed standard CSS selector querying, including `:is`/`:where`/`:has`
|
|
20
|
+
* Native XPath 1.0 engine
|
|
21
|
+
* XPath is parsed and evaluated by Makiri's own engine, written from scratch.
|
|
22
|
+
* Makiri does not depend on libxml2 for parsing, DOM representation, or XPath evaluation.
|
|
23
|
+
* Bounded, fail-closed execution
|
|
24
|
+
* XPath evaluation is bounded by per-evaluation limits on work, memory, and recursion.
|
|
25
|
+
* Ownership and borrowing are kept explicit across layers, with owned/borrowed
|
|
26
|
+
string types and verified text at engine boundaries.
|
|
27
|
+
* Programmatic invalid input, limit violations, allocation failures, and unsupported constructs
|
|
28
|
+
fail closed instead of producing partial or silently truncated results.
|
|
29
|
+
|
|
30
|
+
## Usage
|
|
31
|
+
|
|
32
|
+
```ruby
|
|
33
|
+
require "makiri"
|
|
34
|
+
|
|
35
|
+
doc = Makiri::HTML(<<~HTML)
|
|
36
|
+
<html><body>
|
|
37
|
+
<div id="main" class="container">
|
|
38
|
+
<p class="lead">Hello</p>
|
|
39
|
+
<a href="/a">one</a>
|
|
40
|
+
<a href="/b">two</a>
|
|
41
|
+
</div>
|
|
42
|
+
</body></html>
|
|
43
|
+
HTML
|
|
44
|
+
|
|
45
|
+
# CSS selectors (Lexbor's selector engine)
|
|
46
|
+
doc.css("a").map { |a| a["href"] } # => ["/a", "/b"]
|
|
47
|
+
doc.at_css("p.lead").text # => "Hello"
|
|
48
|
+
|
|
49
|
+
# XPath 1.0 (native engine — no libxml2)
|
|
50
|
+
doc.xpath("//a").length # => 2
|
|
51
|
+
doc.xpath("count(//a)") # => 2.0
|
|
52
|
+
doc.at_xpath('//*[@id="main"]/p').text # => "Hello"
|
|
53
|
+
|
|
54
|
+
# Attributes and navigation
|
|
55
|
+
link = doc.at_css("a")
|
|
56
|
+
link["href"] # => "/a"
|
|
57
|
+
link.parent.name # => "div"
|
|
58
|
+
|
|
59
|
+
# Source location (reconstructed from the tokenizer, no Lexbor patches)
|
|
60
|
+
doc.at_css("p").line # => 3
|
|
61
|
+
|
|
62
|
+
# Serialization
|
|
63
|
+
doc.at_css("#main").to_html # => "<div id=\"main\" ...>...</div>"
|
|
64
|
+
doc.at_css("#main").inner_html # => "\n <p class=\"lead\">Hello</p>\n..."
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
### XPathContext (namespaces and variables)
|
|
68
|
+
|
|
69
|
+
```ruby
|
|
70
|
+
ctx = Makiri::XPathContext.new(doc)
|
|
71
|
+
ctx.register_variable("cls", "lead")
|
|
72
|
+
ctx.evaluate('//p[@class=$cls]').first.text # => "Hello"
|
|
73
|
+
```
|
|
74
|
+
|
|
75
|
+
## Non-goals (v1.0)
|
|
76
|
+
|
|
77
|
+
* XML parsing (HTML only).
|
|
78
|
+
* XSLT, DTD / Schema / RelaxNG validation, XPointer, XInclude.
|
|
79
|
+
* Streaming / SAX parsing.
|
|
80
|
+
* Drop-in replacement for every Nokogiri method. Makiri covers the common
|
|
81
|
+
HTML-scraping and manipulation surface. Deliberately not provided:
|
|
82
|
+
- XML/XHTML serialization variants (`to_xml`, `to_xhtml`, `write_xml_to`)
|
|
83
|
+
- XML/DTD construction (`create_internal_subset`, `external_subset`)
|
|
84
|
+
- namespace introspection beyond `namespace-uri()` (`namespace_definitions`, `add_namespace`, `collect_namespaces`)
|
|
85
|
+
- Nokogiri internals (`decorate`, `slop!`, `validate`).
|
|
86
|
+
|
|
87
|
+
## Differences from Nokogiri
|
|
88
|
+
|
|
89
|
+
Makiri targets a Nokogiri-compatible API, but a few query behaviours differ.
|
|
90
|
+
Detailed, test-backed notes live in `spec/conformance/README.md`.
|
|
91
|
+
|
|
92
|
+
### XPath
|
|
93
|
+
|
|
94
|
+
* The `namespace::` axis is not implemented
|
|
95
|
+
* It raises `Makiri::Error` rather than returning a silently-empty result.
|
|
96
|
+
* Nokogiri (libxml2) supports it (for `<svg>` in HTML it yields the `xml` and `svg` namespace nodes).
|
|
97
|
+
For an element's namespace use `namespace-uri()` / `local-name()`, which are implemented.
|
|
98
|
+
* Unprefixed name tests are namespace-strict by default (HTML5/WHATWG-faithful, like browsers' `document.evaluate` and `Nokogiri::HTML5`)
|
|
99
|
+
* `//div` matches, but foreign elements need a registered prefix (`//svg:path`).
|
|
100
|
+
Pass `namespace_matching: :lax` to `Node#xpath` / `XPathContext.new` for the
|
|
101
|
+
namespace-agnostic match where `//path` finds an SVG element (the
|
|
102
|
+
`Nokogiri::HTML`/libxml2-HTML4 behaviour).
|
|
103
|
+
* `namespace-uri()` of an HTML element returns the XHTML URI (DOM-correct, as browsers report)
|
|
104
|
+
* `Nokogiri::HTML5` returns `""`.
|
|
105
|
+
|
|
106
|
+
### CSS
|
|
107
|
+
|
|
108
|
+
* jQuery/Nokogiri CSS extensions are not supported (`:contains`, `:gt`, `:lt`, `:eq`, `:first`, …)
|
|
109
|
+
* Makiri uses Lexbor's standards-only selector engine.
|
|
110
|
+
Use XPath (`xpath("//p[contains(., 'x')]")`) or Enumerable (`css('li')[1]`).
|
|
111
|
+
Standard Level-4 selectors (`:is` / `:where` / `:has`) are supported; some of which Nokogiri rejects.
|
|
112
|
+
* Type selectors are ASCII case-insensitive (CSS-correct for HTML; `LI` matches `<li>`)
|
|
113
|
+
* `Nokogiri::HTML5` is case-sensitive there.
|
|
114
|
+
* Class/ID selectors are matched case-insensitively regardless of quirks mode (a Lexbor behaviour)
|
|
115
|
+
* In a no-quirks document browsers and `Nokogiri::HTML5` match them case-sensitively.
|
|
116
|
+
|
|
117
|
+
## Requirements
|
|
118
|
+
|
|
119
|
+
* CRuby 3.2 or newer.
|
|
120
|
+
* CMake (to build vendored Lexbor at install time).
|
|
121
|
+
* C99 toolchain.
|
|
122
|
+
|
|
123
|
+
## Build (development)
|
|
124
|
+
|
|
125
|
+
```sh
|
|
126
|
+
git submodule update --init --recursive
|
|
127
|
+
bundle install
|
|
128
|
+
bundle exec rake compile
|
|
129
|
+
bundle exec rake spec
|
|
130
|
+
```
|
|
131
|
+
|
|
132
|
+
## License
|
|
133
|
+
|
|
134
|
+
Apache License 2.0. See [LICENSE](LICENSE) and [NOTICE](NOTICE).
|
data/Rakefile
ADDED
|
@@ -0,0 +1,150 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "bundler/gem_tasks"
|
|
4
|
+
require "rspec/core/rake_task"
|
|
5
|
+
require "rake/extensiontask"
|
|
6
|
+
require "shellwords"
|
|
7
|
+
|
|
8
|
+
GEMSPEC = Gem::Specification.load("makiri.gemspec")
|
|
9
|
+
|
|
10
|
+
Rake::ExtensionTask.new("makiri", GEMSPEC) do |ext|
|
|
11
|
+
ext.lib_dir = "lib/makiri"
|
|
12
|
+
ext.ext_dir = "ext/makiri"
|
|
13
|
+
ext.source_pattern = "**/*.{c,h}"
|
|
14
|
+
end
|
|
15
|
+
|
|
16
|
+
RSpec::Core::RakeTask.new(:spec)
|
|
17
|
+
|
|
18
|
+
task default: %i[compile spec]
|
|
19
|
+
|
|
20
|
+
namespace :security do
|
|
21
|
+
desc "Run mechanical C safety lint over ext/makiri"
|
|
22
|
+
task :clint do
|
|
23
|
+
sh FileUtils::RUBY, "script/check_c_safety.rb", *Shellwords.split(ENV.fetch("C_LINT_ARGS", ""))
|
|
24
|
+
end
|
|
25
|
+
end
|
|
26
|
+
|
|
27
|
+
# `rake clean` (from rake-compiler) removes the ext build dir under tmp/,
|
|
28
|
+
# including the generated Makefile. The next `rake compile` re-runs extconf,
|
|
29
|
+
# so newly-added .c files are picked up — without this, a stale Makefile omits
|
|
30
|
+
# new sources and macOS's -undefined dynamic_lookup turns the missing symbols
|
|
31
|
+
# into runtime NULL calls. The vendored Lexbor build is deliberately NOT wiped
|
|
32
|
+
# here (it is slow to rebuild and rarely changes); use `rake clean:lexbor` for
|
|
33
|
+
# a from-scratch Lexbor build.
|
|
34
|
+
#
|
|
35
|
+
# rake clean compile # regenerate ext Makefile + recompile (fast)
|
|
36
|
+
# rake clean:lexbor # force a full Lexbor rebuild next compile
|
|
37
|
+
|
|
38
|
+
namespace :clean do
|
|
39
|
+
desc "Remove the vendored Lexbor build/install output (forces a full rebuild)"
|
|
40
|
+
task :lexbor do
|
|
41
|
+
require "fileutils"
|
|
42
|
+
FileUtils.rm_rf("vendor/lexbor/build")
|
|
43
|
+
FileUtils.rm_rf("vendor/lexbor/dist")
|
|
44
|
+
end
|
|
45
|
+
end
|
|
46
|
+
|
|
47
|
+
# Locate the AddressSanitizer runtime shared library for the active compiler, so
|
|
48
|
+
# it can be preloaded ahead of Ruby (sanitized extensions dlopen'd late
|
|
49
|
+
# otherwise abort with "ASan runtime does not come first").
|
|
50
|
+
def asan_runtime_path
|
|
51
|
+
cc = RbConfig::CONFIG["CC"] || "cc"
|
|
52
|
+
names =
|
|
53
|
+
if RbConfig::CONFIG["target_os"] =~ /darwin/
|
|
54
|
+
%w[libclang_rt.asan_osx_dynamic.dylib]
|
|
55
|
+
else
|
|
56
|
+
arch = RUBY_PLATFORM[/x86_64|aarch64|arm64/] || "x86_64"
|
|
57
|
+
["libasan.so", "libclang_rt.asan-#{arch}.so", "libclang_rt.asan.so"]
|
|
58
|
+
end
|
|
59
|
+
names.each do |name|
|
|
60
|
+
path = `#{cc} -print-file-name=#{name} 2>/dev/null`.strip
|
|
61
|
+
return path if path != name && !path.empty? && File.exist?(path)
|
|
62
|
+
end
|
|
63
|
+
nil
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
desc "Build the extension with sanitizers (MAKIRI_SANITIZE, default " \
|
|
67
|
+
"address,undefined) and run the spec suite under them"
|
|
68
|
+
task :sanitize do
|
|
69
|
+
sanitize = ENV["MAKIRI_SANITIZE"] || "address,undefined"
|
|
70
|
+
sh({ "MAKIRI_SANITIZE" => sanitize }, "#{FileUtils::RUBY} -S rake clean compile")
|
|
71
|
+
|
|
72
|
+
env = {
|
|
73
|
+
# LeakSanitizer would flag Ruby's intentional caches; the interpreter is not
|
|
74
|
+
# instrumented, so silence the noise and keep real heap/UB findings fatal.
|
|
75
|
+
"ASAN_OPTIONS" => "detect_leaks=0:detect_container_overflow=0:" \
|
|
76
|
+
"detect_odr_violation=0:abort_on_error=1:halt_on_error=1",
|
|
77
|
+
"UBSAN_OPTIONS" => "print_stacktrace=1:halt_on_error=1",
|
|
78
|
+
}
|
|
79
|
+
if sanitize.include?("address")
|
|
80
|
+
runtime = asan_runtime_path or
|
|
81
|
+
abort "sanitize: could not locate the ASan runtime for #{RbConfig::CONFIG['CC']}"
|
|
82
|
+
preload = RbConfig::CONFIG["target_os"] =~ /darwin/ ? "DYLD_INSERT_LIBRARIES" : "LD_PRELOAD"
|
|
83
|
+
env[preload] = runtime
|
|
84
|
+
puts "sanitize: preloading #{runtime} via #{preload}"
|
|
85
|
+
end
|
|
86
|
+
|
|
87
|
+
sh(env, "#{FileUtils::RUBY} -S rspec")
|
|
88
|
+
end
|
|
89
|
+
|
|
90
|
+
desc "Run the robustness fuzzer (override options via FUZZ_ARGS)"
|
|
91
|
+
task fuzz: :compile do
|
|
92
|
+
sh "#{FileUtils::RUBY} -Ilib spec/fuzz/run.rb #{ENV['FUZZ_ARGS']}"
|
|
93
|
+
end
|
|
94
|
+
|
|
95
|
+
desc "Run the performance benchmark (Makiri vs Nokogiri reference)"
|
|
96
|
+
task bench: :compile do
|
|
97
|
+
# Run outside the bundle so the bench-only gems (nokogiri, benchmark-ips)
|
|
98
|
+
# resolve from system RubyGems without polluting the runtime dependency set.
|
|
99
|
+
Bundler.with_unbundled_env do
|
|
100
|
+
sh "#{FileUtils::RUBY} -Ilib bench/bench.rb"
|
|
101
|
+
end
|
|
102
|
+
end
|
|
103
|
+
|
|
104
|
+
namespace :conformance do
|
|
105
|
+
desc "WHATWG HTML5 parsing conformance: run html5lib-tests through Makiri"
|
|
106
|
+
task html5: :compile do
|
|
107
|
+
sh "#{FileUtils::RUBY} -Ilib spec/conformance/html5lib_runner.rb #{ENV['H5_ARGS']}"
|
|
108
|
+
end
|
|
109
|
+
|
|
110
|
+
desc "XPath 1.0 differential conformance vs Nokogiri (libxml2 reference)"
|
|
111
|
+
task xpath: :compile do
|
|
112
|
+
# Like `bench`, run outside the bundle so the bench-only Nokogiri resolves
|
|
113
|
+
# from system RubyGems without entering the runtime dependency set.
|
|
114
|
+
Bundler.with_unbundled_env do
|
|
115
|
+
sh "#{FileUtils::RUBY} -Ilib spec/conformance/xpath_diff.rb #{ENV['XPATH_ARGS']}"
|
|
116
|
+
end
|
|
117
|
+
end
|
|
118
|
+
|
|
119
|
+
desc "CSS Selectors differential conformance vs Nokogiri::HTML5"
|
|
120
|
+
task css: :compile do
|
|
121
|
+
Bundler.with_unbundled_env do
|
|
122
|
+
sh "#{FileUtils::RUBY} -Ilib spec/conformance/css_diff.rb #{ENV['CSS_ARGS']}"
|
|
123
|
+
end
|
|
124
|
+
end
|
|
125
|
+
end
|
|
126
|
+
|
|
127
|
+
desc "Run all conformance suites (html5lib-tests + XPath & CSS differentials)"
|
|
128
|
+
task conformance: %w[conformance:html5 conformance:xpath conformance:css]
|
|
129
|
+
|
|
130
|
+
namespace :fuzz do
|
|
131
|
+
desc "Run the fuzzer under AddressSanitizer (rebuilds the ext; --isolated)"
|
|
132
|
+
task :sanitize do
|
|
133
|
+
sanitize = ENV["MAKIRI_SANITIZE"] || "address,undefined"
|
|
134
|
+
sh({ "MAKIRI_SANITIZE" => sanitize }, "#{FileUtils::RUBY} -S rake clean compile")
|
|
135
|
+
|
|
136
|
+
env = {
|
|
137
|
+
"ASAN_OPTIONS" => "detect_leaks=0:detect_container_overflow=0:" \
|
|
138
|
+
"detect_odr_violation=0:abort_on_error=1:halt_on_error=1",
|
|
139
|
+
"UBSAN_OPTIONS" => "print_stacktrace=1:halt_on_error=1",
|
|
140
|
+
}
|
|
141
|
+
if sanitize.include?("address")
|
|
142
|
+
runtime = asan_runtime_path or
|
|
143
|
+
abort "fuzz:sanitize: could not locate the ASan runtime"
|
|
144
|
+
preload = RbConfig::CONFIG["target_os"] =~ /darwin/ ? "DYLD_INSERT_LIBRARIES" : "LD_PRELOAD"
|
|
145
|
+
env[preload] = runtime
|
|
146
|
+
end
|
|
147
|
+
args = ENV["FUZZ_ARGS"] || "--isolated --time 120"
|
|
148
|
+
sh(env, "#{FileUtils::RUBY} -Ilib spec/fuzz/run.rb #{args}")
|
|
149
|
+
end
|
|
150
|
+
end
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
Binary file
|
|
@@ -0,0 +1,13 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Makiri
|
|
4
|
+
# An attribute node. Most of the API lives in C (#name, #value).
|
|
5
|
+
#
|
|
6
|
+
# An attribute knows its owning element through the document's lazily-built
|
|
7
|
+
# attr->owner index, exposed as both #parent and #element.
|
|
8
|
+
class Attribute < Node
|
|
9
|
+
# The element this attribute belongs to, or nil if detached.
|
|
10
|
+
# @return [Makiri::Element, nil]
|
|
11
|
+
alias element parent
|
|
12
|
+
end
|
|
13
|
+
end
|
data/lib/makiri/cdata.rb
ADDED
data/lib/makiri/css.rb
ADDED
|
@@ -0,0 +1,11 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Makiri
|
|
4
|
+
# CSS selector support. Queries are served by Lexbor's selector engine via
|
|
5
|
+
# {Makiri::Node#css} / {Makiri::Node#at_css}; this module only defines the
|
|
6
|
+
# error type. See ext/makiri/glue/ruby_css.c.
|
|
7
|
+
module CSS
|
|
8
|
+
# Raised when a CSS selector string fails to parse.
|
|
9
|
+
class SyntaxError < ::Makiri::Error; end
|
|
10
|
+
end
|
|
11
|
+
end
|
|
@@ -0,0 +1,82 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Makiri
|
|
4
|
+
# Root container for a parsed HTML document.
|
|
5
|
+
class Document < Node
|
|
6
|
+
# Parse +source+ as HTML5 and return a Document.
|
|
7
|
+
#
|
|
8
|
+
# +source+ may be a String or any object responding to +#read+ (e.g. an
|
|
9
|
+
# IO). The native parser (#_parse) expects UTF-8 bytes. Source locations
|
|
10
|
+
# for {Node#line} are always tracked (the cost is negligible).
|
|
11
|
+
#
|
|
12
|
+
# @param source [String, #read]
|
|
13
|
+
# @return [Makiri::Document]
|
|
14
|
+
def self.parse(source)
|
|
15
|
+
source = source.read if source.respond_to?(:read)
|
|
16
|
+
_parse(String(source))
|
|
17
|
+
end
|
|
18
|
+
|
|
19
|
+
# The document's <body> element, or nil.
|
|
20
|
+
# @return [Makiri::Element, nil]
|
|
21
|
+
def body
|
|
22
|
+
at_css("body")
|
|
23
|
+
end
|
|
24
|
+
|
|
25
|
+
# The document's <head> element, or nil.
|
|
26
|
+
# @return [Makiri::Element, nil]
|
|
27
|
+
def head
|
|
28
|
+
at_css("head")
|
|
29
|
+
end
|
|
30
|
+
|
|
31
|
+
# Set the document title, creating <title> (in <head>) if absent.
|
|
32
|
+
# @param text [String]
|
|
33
|
+
# @return [String]
|
|
34
|
+
def title=(text)
|
|
35
|
+
t = at_css("title")
|
|
36
|
+
unless t
|
|
37
|
+
t = Element.new("title", self)
|
|
38
|
+
(head || root).add_child(t)
|
|
39
|
+
end
|
|
40
|
+
t.content = text
|
|
41
|
+
text
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# Makiri parses and stores everything as UTF-8 (callers decode bytes before
|
|
45
|
+
# parsing), so the in-memory encoding is always UTF-8.
|
|
46
|
+
# @return [String]
|
|
47
|
+
def encoding
|
|
48
|
+
"UTF-8"
|
|
49
|
+
end
|
|
50
|
+
|
|
51
|
+
# The charset declared in the document's markup, or nil. Reads
|
|
52
|
+
# <meta charset> first, then <meta http-equiv="Content-Type">.
|
|
53
|
+
# @return [String, nil]
|
|
54
|
+
def meta_encoding
|
|
55
|
+
if (m = at_css("meta[charset]"))
|
|
56
|
+
return m["charset"]
|
|
57
|
+
end
|
|
58
|
+
|
|
59
|
+
css("meta").each do |meta|
|
|
60
|
+
http_equiv = meta["http-equiv"]
|
|
61
|
+
next unless http_equiv&.downcase == "content-type"
|
|
62
|
+
|
|
63
|
+
content = meta["content"].to_s
|
|
64
|
+
return Regexp.last_match(1) if content =~ /charset\s*=\s*"?([^\s;"]+)/i
|
|
65
|
+
end
|
|
66
|
+
nil
|
|
67
|
+
end
|
|
68
|
+
|
|
69
|
+
# Set (or insert) a <meta charset> declaration.
|
|
70
|
+
# @param value [String]
|
|
71
|
+
# @return [String]
|
|
72
|
+
def meta_encoding=(value)
|
|
73
|
+
meta = at_css("meta[charset]")
|
|
74
|
+
unless meta
|
|
75
|
+
meta = Element.new("meta", self)
|
|
76
|
+
(head || root).add_child(meta)
|
|
77
|
+
end
|
|
78
|
+
meta["charset"] = value
|
|
79
|
+
value
|
|
80
|
+
end
|
|
81
|
+
end
|
|
82
|
+
end
|
|
@@ -0,0 +1,21 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Makiri
|
|
4
|
+
# A detached group of sibling nodes. Build one with {.parse} (its own backing
|
|
5
|
+
# document) or {Makiri::Document#fragment} (bound to an existing document, so
|
|
6
|
+
# its nodes can be spliced in with {Node#add_child} and friends). Inserting a
|
|
7
|
+
# fragment contributes its children, not the fragment node itself.
|
|
8
|
+
#
|
|
9
|
+
# Both +.parse+ and +Document#fragment+ accept a Nokogiri-compatible
|
|
10
|
+
# <tt>context:</tt> keyword naming the element the HTML is parsed inside of
|
|
11
|
+
# (the fragment-parsing algorithm is context-sensitive). It may be:
|
|
12
|
+
# * a tag-name String (HTML namespace), e.g. <tt>context: "tr"</tt>; the
|
|
13
|
+
# bare strings <tt>"svg"</tt> / <tt>"math"</tt> name the foreign roots;
|
|
14
|
+
# * a {Makiri::Node} element — its tag and namespace are used (the way to
|
|
15
|
+
# reach a foreign non-root context such as an SVG <desc>).
|
|
16
|
+
# The default context is <tt><body></tt>. See also {Makiri::Node#parse}.
|
|
17
|
+
#
|
|
18
|
+
# +.parse+ is defined in C (ext/makiri/glue/ruby_doc.c).
|
|
19
|
+
class DocumentFragment < Node
|
|
20
|
+
end
|
|
21
|
+
end
|
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Makiri
|
|
4
|
+
# A `<!DOCTYPE ...>` node. Reachable via {Document#internal_subset} or as a
|
|
5
|
+
# child of the document. Exposes the doctype name (via {Node#name}) and its
|
|
6
|
+
# public/system identifiers; XPath cannot reach it (XPath 1.0 has no doctype
|
|
7
|
+
# node type), matching Nokogiri/libxml2.
|
|
8
|
+
class DocumentType < Node
|
|
9
|
+
# @return [String, nil] the public identifier, or nil if absent. An empty
|
|
10
|
+
# `PUBLIC ""` literal returns "".
|
|
11
|
+
# `public_id`, `external_id` (Nokogiri-compatible alias), and `system_id`
|
|
12
|
+
# are defined in the C extension.
|
|
13
|
+
end
|
|
14
|
+
end
|
|
@@ -0,0 +1,17 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
module Makiri
|
|
4
|
+
# An HTML element node. Attribute access lives in C.
|
|
5
|
+
class Element < Node
|
|
6
|
+
# Create a detached element named +name+ owned by +document+ (Nokogiri-style
|
|
7
|
+
# constructor; delegates to {Document#create_element}). Attach it with
|
|
8
|
+
# {Node#add_child} and friends.
|
|
9
|
+
#
|
|
10
|
+
# @param name [String]
|
|
11
|
+
# @param document [Makiri::Document]
|
|
12
|
+
# @return [Makiri::Element]
|
|
13
|
+
def self.new(name, document)
|
|
14
|
+
document.create_element(name)
|
|
15
|
+
end
|
|
16
|
+
end
|
|
17
|
+
end
|