makiri 0.15.0 → 0.17.0.rc1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 383a2feea9ae25aca8e66985862eb3dfbbb64682771fc4578d9f0f6c11100d43
-  data.tar.gz: 774f2089e9b581b757cf0e142e937f892a87ccd71bb7c60f74666d9007bcac5c
+  metadata.gz: f0b28460a4edb95ca0dd366fe5142748c39e80b1c3d4a4f227a949531d08f106
+  data.tar.gz: f099573777282c714387339251261fa7aedc619a5c57be65fba4bd002fcc7b0a
 SHA512:
-  metadata.gz: 575cdf673cf1c0f2f76bc641c978a5c946e99a19ebc9bd92d761aa23ac9e7d9a72d980120c22b7bc344a95d0236bcfff6887fc09e28836f64739ca87a18c7e8d
-  data.tar.gz: 8b5f74af6c317ce0928c2b700d4061828621c4115685b80c3b49c47bf65f521f46431e4cf22ce064eadf0c30dc2f68611f84b3e4e4ead2b162ef5369c5544b7a
+  metadata.gz: 839d2291c32bd040d244e054567a214a5c5ee60be1b23f6b7c44edf3631001fb64c7fa3e7f3956818fa138ae5f011ab22824a96ef00395608d894d7aae5efeaa
+  data.tar.gz: c0a7ed1945f42162ad8445d7cc2fafac32f6ff3c218e0734dd1354dc80844b634a86ddf44531c24a7365c3964f4e1b26172ba635257e3c220132d2a3ee647d99

data/CHANGELOG.md

@@ -1,5 +1,121 @@
 # Changelog
 
+## [Unreleased]
+
+## [0.17.0.rc1] - 2026-10-22
+
+### Added
+
+* `Makiri::Lexbor::CSS.parse_stylesheet` returns the declarations in an
+  at-rule's block (`@font-face`, `@page`, ...) as its `:declarations`; they
+  were dropped.
+
+### Changed
+
+* HTML is parsed by Makiri's own Rust port of Lexbor's tokenizer and tree
+  builder, into a DOM of Makiri's own, and the CSS selector and stylesheet
+  parsers are ported too. A parsed document takes about 40% less memory.
+* Installing from source needs Rust 1.84 or newer (and libclang), and no
+  longer CMake: the gem carries none of Lexbor's sources, only its LICENSE
+  and NOTICE.
+* `Makiri::XML`'s `#css`, `#at_css` and `#matches?` run on the same matcher
+  as HTML's and answer as a browser's `querySelectorAll` does in an XML
+  document, 10-20x faster on compound selectors (`.item`, `ul li.item`).
+  This changes some answers:
+  * a bare type selector (`p`) matches that name in any namespace;
+  * the root element's namespace declarations are no longer picked up: a
+    prefix must be given in the ns Hash, whose `"xmlns"` entry is the
+    default namespace (on HTML too);
+  * a selector run on a node may match through ancestors outside it
+    (`node.css("body p")`); only the node's descendants are returned;
+  * `[a=v i]`, `:nth-child(... of S)`, and the form-state and link
+    pseudo-classes on XHTML elements are supported;
+  * `||` and `:current()` raise `Makiri::CSS::SyntaxError`, and an over-long
+    chain `Makiri::Error` (it was `Makiri::XPath::LimitExceeded`).
+* HTML `css(selector, ns)` resolves namespace prefixes: `svg|path` is the SVG
+  `path`, `|p` an element in no namespace, `[x|href]` the attribute in `x`'s
+  namespace. A prefix the Hash does not bind raises
+  `Makiri::CSS::SyntaxError`; prefixes used to be ignored.
+* HTML parsing follows the HTML Standard where Lexbor did not. This affects
+  malformed or unusual markup only: `<frameset>`, `<form>` and tables inside
+  `<template>`, `<![CDA` or `<!` at the end of the input, the adoption
+  agency, `<textarea>`, `<input type=HIDDEN>` and `<image>` in a table,
+  `<sup>` in SVG or MathML, and a repeated attribute name (the first is
+  kept). Fragment parsing (`#inner_html=`, `#outer_html=`, `Node#parse`,
+  `context:` given an element) takes the context element's quirks mode,
+  namespace and ancestor `<form>` into account, as browsers do.
+* CSS selectors are parsed as Selectors 4 says: `:has()` inside `:has()` and
+  an invalid argument to `:has()` make the selector invalid, an empty
+  `:is()` / `:where()` is valid and matches nothing, and `#1a` is not an ID
+  selector.
+* `parse_stylesheet` reads names and values by one rule for every property: a
+  declaration name is lower-cased unless it is a custom property, an at-rule
+  name is lower-cased, and a value is the text as written with whitespace
+  runs collapsed (`color: RED` stays `RED`; it came back as `red`).
+  `:specificity` is counted as Selectors 4 does (`:is()` / `:not()` /
+  `:has()` take their most specific argument).
+
+### Removed
+
+* The 10,000-`<option>` limit on a `<select>`: a select with any number of
+  options parses in linear time, as browsers parse it.
+
+### Fixed
+
+* `parse_stylesheet` keeps a rule such as `a:hover { ... }` inside `@media`
+  or `@supports`; it was read as a declaration, which dropped that rule and
+  every one after it in the block. A comment between a declaration's name
+  and its `:` no longer drops the declarations that follow.
+* A namespaced type selector in the middle of a compound (`.x|a`) raises
+  `Makiri::CSS::SyntaxError`, as browsers do.
+* `:any-link` / `:link` match only an HTML `a` or `area`, or an SVG `a`, with
+  an `href` (`<link href>` and `<map>` matched). `:read-write` /
+  `:read-only` follow the HTML Standard (a checkbox is read-only, an element
+  inside `contenteditable` is read-write), and `:empty` ignores processing
+  instructions.
+* HTML's `:nth-child` family counts element siblings only:
+  `html:first-child` matched nothing after `<!doctype html>`.
+* An attribute keeps the namespace it was made in: `set_attribute_ns` in the
+  element's own namespace replaces the attribute it set before instead of
+  adding a second, and an attribute removed from its element keeps its
+  `namespace_uri`.
+* An attribute name is serialized in the case it was set (`e["viewBox"]` on
+  an element in no namespace was written lower-cased).
+* A void element's `#inner_html` is empty even when it has children, and a
+  prefixed HTML, SVG or MathML element is written by its local name.
+* `Node#line` of a void element fostered out of a table is its line, not nil.
+* Many namespaces created with `create_element_ns` no longer make lookups
+  quadratic.
+* `HTML::Document#import_node` given a document raises "cannot import a
+  document; use #dup", as `#clone_node` does.
+
+## [0.16.0] - 2026-10-10
+
+### Added
+
+* `Makiri::NATIVE_VERSION` and `Makiri::NATIVE_RUBY_API_VERSION`: the gem
+  version and Ruby `MAJOR.MINOR` the native extension was built as.
+  `require "makiri"` raises `LoadError` for an extension built for another
+  Ruby (refused by the extension itself, before it defines anything) or as
+  another makiri version (e.g. a stale `lib/makiri/makiri.bundle` in a
+  checkout).
+* `Node#child_count`, `#child_at(i)`, `#element_child_count` and
+  `#element_child_at(i)` (HTML and XML): count and index a child list without
+  building `#children`. A loop over the indexes in order costs O(1) a step,
+  also while other lists are read in between. Any child-list edit in the
+  document resets that, so a loop that edits as it indexes walks from the
+  nearer end after each edit. `child_at` returns nil for a negative index or
+  one past the end.
+* `Node#first_child` (an alias of `#child`) and `#last_child`.
+* `Node#root_node`, DOM `getRootNode()`: the Document, a DocumentFragment, or
+  the top of a detached subtree. An Attr is its own root.
+
+### Changed
+
+* `#local_name`, `#prefix`, `#namespace_uri` and `#tag_name` return frozen,
+  interned Strings: every read of one name returns the same object. `#name`
+  still returns a new, mutable String.
+
 ## [0.15.0] - 2026-10-09
 
 ### Added

data/NOKOGIRI_DIFFERENCES.md

@@ -149,7 +149,7 @@ what browsers do - rather than libxml2. Detailed, test-backed notes live in
 * `<?php ... ?>` in HTML input is a **ProcessingInstruction** node; `#to_html`
   writes it back as `<?php ... ?>`, and `<?` at end of input is ignored.
   * The HTML Standard added processing-instruction tokens and tree-construction
-    rules for them; Makiri follows them through Lexbor. `Nokogiri::HTML5` (gumbo)
+    rules for them, and Makiri's parser follows them. `Nokogiri::HTML5` (gumbo)
     still produces the older bogus comment (`<!--?php ... ?-->`), and
     `Nokogiri::HTML` (libxml2) its own comment.
 
@@ -158,17 +158,14 @@ what browsers do - rather than libxml2. Detailed, test-backed notes live in
   Nokogiri's. `inner_html=`, `outer_html=` and `Node#parse` take no keyword.
   `Nokogiri::HTML` (libxml2) instead stops at depth 256 and returns the
   truncated document.
-* More than 10,000 `<option>`s in one `<select>` raise `Makiri::Error`;
-  Nokogiri has no such limit.
 
 ## HTML mutation
 
 * There is no `Node#name=` / `#node_name=` (on HTML or XML nodes). Nokogiri
   renames a node in place and keeps its identity; the DOM has no rename, and
-  Lexbor keeps elements such as `<template>`, `<option>` and `<style>` in
-  structs of their own, so a node cannot change its tag safely. Create an
-  element of the new name, move the attributes and children, and `replace`
-  the old one (see CHANGELOG.md).
+  an element's tag decides what it is (a `<template>` owns its contents), so a
+  node cannot change its tag. Create an element of the new name, move the
+  attributes and children, and `replace` the old one (see CHANGELOG.md).
 * Element and attribute names follow the WHATWG DOM's rules
   * `create_element`, `[]=` and `set_attribute_ns` raise `ArgumentError` for a
     name the DOM refuses - one holding whitespace, `/`, `>` (or `=` for an
@@ -224,26 +221,19 @@ what browsers do - rather than libxml2. Detailed, test-backed notes live in
   * One exception to the DOM: an HTML attribute named `xml:lang` (in no
     namespace) becomes the XML namespace's `xml:lang`, as the XML reader reads
     that name, so XHTML-style HTML still writes as XML.
-* A known gap, in Lexbor's tag table: an HTML document that already holds a
-  parsed element named with a colon (`<x:y>`, one local name) and then
-  receives, by `import_node` from another document, a prefixed element
-  written the same way (`x:y` from XML: prefix `x`, local name `y`) re-points
-  the table's entry for that spelling - the parsed element then no longer
-  matches CSS `x\:y`. Copies within one document do not touch the table.
 * An XML element with a prefix, imported into HTML, keeps it (`h:div` in
   XHTML has the local name `div`). Two readers then disagree, as they do in
-  browsers: CSS's `div` matches it (Lexbor matches the local name), XPath's
-  `//div` does not (an HTML element's name test reads its qualified name). And
-  Lexbor's HTML serializer writes the prefix (`<h:div>`), where the HTML
-  standard writes the local name, so the HTML does not re-parse to the same
-  element.
+  browsers: CSS's `div` matches it (the matcher compares the local name),
+  XPath's `//div` does not (an HTML element's name test reads its qualified
+  name). `#to_html` writes the local name (`<div>`), as the HTML standard
+  does.
 
 ## CSS
 
 * Most jQuery/Nokogiri CSS extensions are not supported (`:gt`, `:lt`, `:eq`, `:first`, ...)
-  * Makiri uses Lexbor's selector engine, which is standards-based apart from one
-    text-containment extension. Use XPath (`xpath("//p[contains(., 'x')]")`) or
-    Enumerable (`css('li')[1]`) for the rest.
+  * Makiri's selector engine is standards-based. Use XPath
+    (`xpath("//p[contains(., 'x')]")`) or Enumerable (`css('li')[1]`) for the
+    rest.
     Standard Level-4 selectors (`:is` / `:where` / `:has`) are supported; some of which Nokogiri rejects.
   * There is no text-containment pseudo-class. Nokogiri's `:contains()` is
     not supported, and neither is Lexbor's `:lexbor-contains()`: it raises
@@ -255,33 +245,45 @@ what browsers do - rather than libxml2. Detailed, test-backed notes live in
   selector) is supported and correct on both HTML and XML - the "type" is the
   element's own expanded name.
   * Nokogiri (XML and HTML5) mistranslates these to first-/only-child
-    (`//*[position()=1]` / `//*[last()=1]`), so it under-matches; Makiri matches
-    Lexbor's HTML matcher.
-* HTML CSS takes a `{prefix => uri}` Hash (`css(selector, ns)`) but does not
-  resolve prefixes against it: Lexbor's matcher matches a prefixed type selector
-  loosely, so `svg|path`, `|path` and `path` all find the SVG element whatever
-  is bound. `Nokogiri::HTML5` honours the binding (a wrong URI finds nothing).
-  * The Hash is accepted and unused rather than refused, so the `css(selector,
-    ns)` a caller writes for both representations works. Use `#xpath`, where a
-    prefix IS resolved against the bindings, when the namespace matters.
-  * `Makiri::XML` resolves CSS prefixes properly - it lowers the selector to the
-    XPath engine, which registers the bindings.
-  * The same holds for attribute selectors: HTML `[|href]` (no namespace) and
-    plain `[href]` also find an SVG `xlink:href`, which `Nokogiri::HTML5`
-    does not.
-    `Makiri::XML` reads `[|a]` as the no-namespace attribute.
+    (`//*[position()=1]` / `//*[last()=1]`), so it under-matches; Makiri does
+    not.
+* CSS resolves a namespace prefix against the `{prefix => uri}` Hash
+  (`css(selector, ns)`), on HTML and XML alike: `svg|path` is the SVG `path`
+  once `svg` is bound, `|path` an element in no namespace, and a prefix the
+  Hash does not bind raises `Makiri::CSS::SyntaxError`. `[x|href]` finds the
+  XLink `href` once `x` is bound to XLink. The Hash's `"xmlns"` entry is the
+  default namespace, as CSS `@namespace` without a prefix sets one: a type or
+  universal selector without a prefix is then in it.
+  * Without a default, a bare type selector matches its name in any namespace
+    (`path` finds the SVG element, `entry` an Atom entry), as a browser's
+    `querySelectorAll` does. `Nokogiri::XML` binds a bare name to the root's
+    default namespace when no Hash is given, and to no namespace once one is;
+    pass `"xmlns" => uri` (or `"xmlns" => ""`) for that reading.
+  * `Makiri::XML` does not collect the document's own declarations: a prefix
+    is declared by the Hash or not at all. `Nokogiri::XML` collects the
+    root's.
+  * A bare `*` under a default namespace is in it, as CSS has it;
+    `Nokogiri::XML` matches any element.
+  * Up to 0.16 HTML accepted the Hash and ignored every prefix, and
+    `Makiri::XML` collected the root's declarations as Nokogiri does.
 * A selector under a node matches the way `Element#querySelectorAll` does in a
-  browser, not scoped to that node, on HTML: `at_css("#c").css("div p")` finds a
-  `p` inside `#c` when `#c` is itself a `div`, since the selector is matched
-  against the whole document and only the results are kept to descendants.
-  `Nokogiri::HTML5` and `Nokogiri::XML` scope the selector to the node (`#c`
-  cannot be the `div`), and so does `Makiri::XML`, which lowers the selector to
-  an XPath from the node. Lexbor has no `:scope`; for a scoped match on HTML,
-  use XPath from the node (`xpath(".//div//p")`).
-* The attribute case modifiers (`[a="x" i]`, `[a="x" s]`): HTML supports both,
-  through Lexbor's matcher. `Makiri::XML` accepts `s` (case-sensitive, which XML
-  values are anyway) and refuses `i` with `Makiri::CSS::SyntaxError`. Nokogiri
-  refuses both on either representation.
+  browser, not scoped to that node, on HTML and XML alike:
+  `at_css("#c").css("div p")` finds a `p` inside `#c` when `#c` is itself a
+  `div`, since the selector is matched against the whole tree and only the
+  results are kept to descendants. `Nokogiri::HTML5` and `Nokogiri::XML` scope
+  the selector to the node (`#c` cannot be the `div`), and so did
+  `Makiri::XML` up to 0.16. There is no `:scope`; for a scoped match, use XPath
+  from the node (`xpath(".//div//p")`).
+* The attribute case modifiers (`[a="x" i]`, `[a="x" s]`) are supported on
+  both representations. Nokogiri refuses both on either.
+* The form-state and link pseudo-classes (`:checked`, `:disabled`,
+  `:enabled`, `:required`, `:optional`, `:placeholder-shown`, `:link`, ...)
+  answer on `Makiri::XML` too, for XHTML elements, as browsers do in an XML
+  document; on any other element they match nothing. `Nokogiri::XML` refuses
+  them.
+* A selector `Makiri` cannot run (the column combinator `||`, `:current()`)
+  raises `Makiri::CSS::SyntaxError`, as a browser's `querySelectorAll` throws a
+  SyntaxError, on both representations.
 * `#matches?` answers for a DETACHED node (`document.create_element("p")
   .matches?("p")` is true, on both representations). Nokogiri raises
   `NoMethodError` there - it implements `#matches?` as a search from
@@ -294,7 +296,9 @@ what browsers do - rather than libxml2. Detailed, test-backed notes live in
   therefore matches no type selector.
   * `Nokogiri::HTML5` is case-sensitive on HTML elements too, so `LI` does not
     match `<li>` there. `Makiri::XML`'s `#css` is case-sensitive, as XML names
-    are.
+    are - an XHTML element's included, as in a browser's XML document - and so
+    are its attribute values, the HTML Standard's case-insensitive list of
+    attributes applying only to an HTML document.
 
 ## Serialization
 

data/NOTICE

@@ -8,6 +8,13 @@ This product includes software developed by:
   Licensed under the Apache License, Version 2.0
   https://lexbor.com / https://github.com/lexbor/lexbor
 
+  The HTML tokenizer and tree builder under ext/makiri/rust/src/html/ and the
+  CSS syntax, selector and stylesheet parser under
+  ext/makiri/rust/src/selectors/ are Rust ports of Lexbor's, as are the HTML
+  CSS matcher under ext/makiri/rust/src/css_match/ and the tag and
+  attribute tables generated from Lexbor's. The extension links no Lexbor
+  code; Lexbor's LICENSE and NOTICE are kept under vendor/lexbor/ in the gem.
+
 The XPath 1.0 engine under ext/makiri/rust/src/xpath/ is an original
 implementation written for this project. No code is derived from libxml2 or
 libxslt.

data/README.md

@@ -28,8 +28,7 @@ XPath 1.0 evaluation in its own native engine, with no libxml2 dependency.
     differential, and property-based testing vs Nokogiri (see below).
 * Bounded, fail-closed execution
   * XPath evaluation is bounded by per-evaluation limits on work, memory, and recursion.
-  * HTML parsing bounds the tree depth (`max_tree_depth:`, default 400) and the
-    `<option>`s per `<select>` (10,000).
+  * HTML parsing bounds the tree depth (`max_tree_depth:`, default 400).
   * Ownership and borrowing are kept explicit across layers, with owned/borrowed
     string types and verified text at engine boundaries.
   * Programmatic invalid input, limit violations, allocation failures, and unsupported constructs
@@ -109,9 +108,10 @@ el = doc.at_xpath("//a:entry", ns)
 el.local_name                                  # => "entry"
 el.namespace_uri                               # => "http://www.w3.org/2005/Atom"
 
-# CSS selectors work too (lowered to the native XPath engine): a bare type
-# selector binds to the document's default namespace, so this just works.
+# CSS selectors work too, as in a browser: a bare type selector matches its
+# name in any namespace, so this just works.
 doc.css("entry").length                        # => 2
+doc.css("a|entry", ns).length                  # => 2 (a prefix the Hash binds)
 doc.css("feed > entry").map { |e| e.at_css("title").text }  # => ["Hello", "World"]
 
 # Serialize back to XML
@@ -256,10 +256,13 @@ See also [`spec/conformance/README.md`](spec/conformance/README.md).
 ## Requirements
 
 * CRuby 3.2 or newer.
-* A Rust toolchain (stable) with `cargo` - the extension is a Rust crate.
+* Rust 1.84 or newer with `cargo` - the extension is a Rust crate. (Binary
+  gems are published for the common platforms, which need no toolchain.)
 * libclang (clang's development library): rb-sys runs bindgen at build time and
   reads Ruby's headers through it.
-* CMake and a C toolchain, to build the vendored Lexbor static library.
+
+Development also needs CMake and a C toolchain, to build the vendored Lexbor
+that the Rust port is tested against (`rake lexbor`).
 
 ## Build (development)
 

data/ext/makiri/rust/Cargo.toml

@@ -2,6 +2,11 @@
 name = "makiri_rs"
 version = "0.0.1"
 edition = "2021"
+# The oldest Rust a source install builds with - what the release's
+# `source-gem-install` job installs and builds on, so the declaration is
+# tested. It is today's floor (1.84 brought `core::ptr::without_provenance`),
+# not a promise: raise it whenever a newer feature is worth having.
+rust-version = "1.84"
 publish = false
 description = "The Makiri gem's extension: HTML5 parsing, XPath 1.0 and an XML reader"
 
@@ -23,7 +28,7 @@ path = "src/lib.rs"
 # about an unexpected cfg name (and CI's `clippy -D warnings` turns that into an
 # error). The proof modules are compiled only under it.
 [lints.rust]
-unexpected_cfgs = { level = "warn", check-cfg = ['cfg(kani)', 'cfg(makiri_asan)'] }
+unexpected_cfgs = { level = "warn", check-cfg = ['cfg(kani)', 'cfg(fuzzing)'] }
 
 # The panic gate. A panic now reaches Ruby as an exception (see the profile
 # below), but it is still an INTERNAL error nobody asked for - `rescue => e`
@@ -44,9 +49,11 @@ expect_used = "warn"
 panic = "warn"
 
 [features]
-# Four features. Three name a real optional dependency rather than a step of
-# the C port - a Ruby runtime, a vendored C library, and a test hook - and the
-# fourth, `css-reference`, a test reference (below).
+# Seven features. Three name a real optional dependency rather than a step of
+# the C port - a Ruby runtime, a vendored C library, and a test hook - three,
+# `css-reference`, `lexbor-reference` and `xml-css-reference`, test
+# references, and the last, `html-dom`, the HTML DOM and everything over it
+# (below).
 #
 # The port's thirty-seven per-file flags are gone. They existed so one C file at
 # a time could be replaced behind an identical ABI; there is no C left to
@@ -55,15 +62,17 @@ default = ["ruby"]
 
 # The Ruby boundary: magnus, rb-sys, and everything under `bridge/`, `glue/`
 # and `init.rs`. Off, the crate is the engine alone - which is what Kani and
-# cargo-fuzz build, since neither can link a live Ruby.
-ruby = ["lexbor", "dep:magnus", "dep:rb-sys"]
+# cargo-fuzz build, since neither can link a live Ruby. It needs no Lexbor:
+# its HTML DOM is the arena's (`html-dom`).
+ruby = ["dep:magnus", "dep:rb-sys", "html-dom"]
 
 # The vendored Lexbor: build.rs runs bindgen over its headers, so this also
 # decides whether a build needs `vendor/lexbor/dist` to exist at all. `rake
 # kani` turns it OFF deliberately - the proofs are over the XML reader and the
 # XPath front end, and not needing a Lexbor build is what lets them run without
-# `rake compile` first.
-lexbor = []
+# `rake compile` first. Lexbor is only ever the reference the ports are
+# tested against; the HTML DOM beside it is still the arena's.
+lexbor = ["html-dom"]
 
 # Allocation-failure injection, the Rust half of `rake oom`. Off,
 # `falloc::should_fail` is a const false and the branch disappears.
@@ -74,6 +83,24 @@ alloc-inject = []
 # part of a build that ships.
 css-reference = ["lexbor"]
 
+# The CSS-to-XPath lowering (`crate::css`) outside `cfg(test)`, as the
+# reference the `css` fuzz harness holds the matcher's XML instance to - the
+# engine `Makiri::XML`'s CSS ran on before it moved to `css_match`. No
+# dependency, no Lexbor. Never part of a build that ships.
+xml-css-reference = []
+
+# Lexbor as the parser port's reference (`lexbor::tree_diff`): its tree,
+# copied into the arena, beside the ported tree builder's, for the
+# `html_diff` fuzz harness. Never part of a build that ships.
+lexbor-reference = ["lexbor"]
+
+# The parser port's index-arena DOM (`crate::html::dom`) behind its facade
+# (`crate::html_dom`), parsed by the ported tree builder (`crate::html::tree`),
+# and what sits over it: the HTML CSS matcher, XPath's HTML instance, the
+# bridge's HTML half. Off only in the bare engine (Kani). `ruby` and `lexbor`
+# both turn it on.
+html-dom = []
+
 [dependencies]
 # Only the `ruby` feature needs these; the engine builds without them, which is
 # what lets Kani and cargo-fuzz compile it. `rb-sys` is magnus's own binding

data/ext/makiri/rust/build.rs

@@ -41,6 +41,10 @@ fn main() {
     println!("cargo:rerun-if-changed=build.rs");
     println!("cargo:rerun-if-env-changed=MAKIRI_LEXBOR_INCLUDE");
 
+    if std::env::var_os("CARGO_FEATURE_RUBY").is_some() {
+        gem_version();
+    }
+
     // Only a build that reads Lexbor's layout pays for this. Kani builds with
     // `--no-default-features` and has no Lexbor headers to point at, which is
     // the whole reason `rake kani` needs no `rake compile` first.
@@ -240,6 +244,39 @@ fn main() {
         .allowlist_function("lxb_css_parser_status_noi")
         .allowlist_function("lxb_css_parser_memory_set_noi")
         .allowlist_function("lxb_css_parser_selectors_set_noi")
+        // The tokenizer on its own, for the parser conformance harness only
+        // (`lexbor::conformance_tests`): the html5lib tokenizer tests drive it
+        // from a given initial state with no tree builder behind it. Nothing
+        // that ships calls these, so they add declarations and no link.
+        .allowlist_function("lxb_html_tokenizer_create")
+        .allowlist_function("lxb_html_tokenizer_init")
+        .allowlist_function("lxb_html_tokenizer_destroy")
+        .allowlist_function("lxb_html_tokenizer_tags_make")
+        .allowlist_function("lxb_html_tokenizer_tags_destroy")
+        .allowlist_function("lxb_html_tokenizer_attrs_make")
+        .allowlist_function("lxb_html_tokenizer_attrs_destroy")
+        .allowlist_function("lxb_html_tokenizer_begin")
+        .allowlist_function("lxb_html_tokenizer_chunk")
+        .allowlist_function("lxb_html_tokenizer_end")
+        .allowlist_function("lxb_html_tokenizer_state_data_before")
+        .allowlist_function("lxb_html_tokenizer_state_plaintext_before")
+        .allowlist_function("lxb_html_tokenizer_state_rcdata_before")
+        .allowlist_function("lxb_html_tokenizer_state_rawtext_before")
+        .allowlist_function("lxb_html_tokenizer_state_script_data_before")
+        .allowlist_function("lxb_html_tokenizer_state_cdata_section_before")
+        .allowlist_function("lxb_html_token_attr_name")
+        .allowlist_function("lxb_tag_name_by_id_noi")
+        .allowlist_type("lxb_dom_attr_id_enum_t")
+        // Lexbor's static name tables by id, which the port's generated tables
+        // (`html::res`) are checked against, entry by entry.
+        .allowlist_function("lxb_tag_name_upper_by_id_noi")
+        .allowlist_function("lxb_dom_attr_data_by_id")
+        .allowlist_function("lxb_ns_data_by_id")
+        .allowlist_function("lxb_ns_prefix_data_by_id")
+        .allowlist_type("lxb_ns_prefix_data_t")
+        // The tokenizer's recorded parse errors, which the port's are compared
+        // with (`lexbor::conformance_tests`).
+        .allowlist_type("lxb_html_tokenizer_error_t")
         // The fragment parse, CHUNKED - begin/process/end rather than the
         // one-shot `lxb_html_parse_fragment*`, because only between `begin` and
         // `process` can the tree-depth guard be installed on the tokenizer
@@ -312,8 +349,6 @@ fn main() {
         .allowlist_function("lxb_selectors_find")
         .allowlist_function("lxb_selectors_match_node")
         .allowlist_function("lxb_selectors_opt_set_noi")
-        // Lexbor's allocator hook (`lexbor::memory`).
-        .allowlist_function("lexbor_memory_setup")
         .allowlist_type("lxb_selectors_opt_t")
         // Top-level consts, not modules: the names then match the headers
         // exactly and do not depend on bindgen's numbering of anonymous types.
@@ -358,9 +393,9 @@ fn main() {
     // of them left to keep in agreement.
 }
 
-/// Where the vendored Lexbor headers are. extconf builds them into
-/// `vendor/lexbor/dist` before it invokes cargo, so the default is that path
-/// relative to this crate; `MAKIRI_LEXBOR_INCLUDE` overrides it for anything
+/// Where the vendored Lexbor headers are, for the reference builds that
+/// read them (the `lexbor` feature). `rake lexbor` builds them into
+/// `vendor/lexbor/dist`, so the default is that path relative to this crate; `MAKIRI_LEXBOR_INCLUDE` overrides it for anything
 /// that builds the crate from somewhere else.
 fn lexbor_include() -> std::path::PathBuf {
     if let Some(p) = std::env::var_os("MAKIRI_LEXBOR_INCLUDE") {
@@ -370,7 +405,7 @@ fn lexbor_include() -> std::path::PathBuf {
     let p = manifest.join("../../../vendor/lexbor/dist/include");
     if !p.join("lexbor/dom/dom.h").exists() {
         panic!(
-            "vendored Lexbor headers not found at {} - run `bundle exec rake compile` \
+            "vendored Lexbor headers not found at {} - run `bundle exec rake lexbor` \
              once, or set MAKIRI_LEXBOR_INCLUDE",
             p.display()
         );
@@ -472,3 +507,36 @@ fn definition(text: &str, name: &str) -> Option<(String, String)> {
     }
     None
 }
+
+/// Embed the gem's version (`lib/makiri/version.rb`, the one place it is
+/// written) as `MAKIRI_GEM_VERSION`, which `init` publishes as
+/// `Makiri::NATIVE_VERSION` for `lib/makiri.rb` to check the loaded binary
+/// against. Read from the source rather than passed in by extconf, so every
+/// cargo build of the extension carries it and none can carry a stale one.
+fn gem_version() {
+    let manifest = std::path::PathBuf::from(std::env::var("CARGO_MANIFEST_DIR").unwrap());
+    let path = manifest.join("../../../lib/makiri/version.rb");
+    println!("cargo:rerun-if-changed={}", path.display());
+    let source = std::fs::read_to_string(&path)
+        .unwrap_or_else(|e| panic!("cannot read {}: {e}", path.display()));
+    /* `VERSION = "x"` or `VERSION = 'x'`, a trailing `.freeze` or comment
+     * allowed: the literal is what is between the first quote and its match. */
+    let version = source
+        .lines()
+        .find_map(|line| {
+            let rest = line.trim().strip_prefix("VERSION")?.trim_start();
+            let rest = rest.strip_prefix('=')?.trim_start();
+            let quote = rest.chars().next().filter(|c| *c == '"' || *c == '\'')?;
+            let body = &rest[1..];
+            body.find(quote).map(|end| &body[..end])
+        })
+        .filter(|v| !v.is_empty())
+        .unwrap_or_else(|| {
+            panic!(
+                "build.rs reads the gem version from a `VERSION = \"x.y.z\"` line in {}, \
+                 and found none",
+                path.display()
+            )
+        });
+    println!("cargo:rustc-env=MAKIRI_GEM_VERSION={version}");
+}