leptris 1.9.6 → 1.9.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 68bba6b5b68f0266571ec17a5809ea42ce7b1f37f0ba3577f9a06afd57db578f
-  data.tar.gz: d15442dd463ba2b4d09c6415e51c478c385c851eb6630ffc8c4d00e6a9ad7a57
+  metadata.gz: 8f306f9bc45d304b82c3db1ac2936dcc0d708b21d2caf595e4a9484da5c6b48a
+  data.tar.gz: ce9b04f6f1e91e6dffd1ff68ce70967b8b2bcff8351132db2f3312e4b534e803
 SHA512:
-  metadata.gz: 52a5f4a5307b9b219911a9ebb7050672be1ad27096266e1c0732eb98ab40779055dcf25e4dd31c0ef0813966539e43894c6d4cdce20bd79e778a7df043c5631a
-  data.tar.gz: cd2e0732f88060f6ff02e3496bedca3d986fde26c9400c91efe458c05e96379caeddd50a5950eb65b965be23487aac243ada697855ade71d0ed6017bcfe49433
+  metadata.gz: 40618f16214157768b018c774816f738a8cbce00e757286822a69f22f71413605c165448a22644b1ff5cc66a387ce63d6e2c496167b8049f2e80ab7cdf124582
+  data.tar.gz: f3c5d00e54f051510d76db44582806252ccf84091c9c9eb11915fc43675282721f4224fada0331f45616250be411362f981d545fb5de47bfcaadfaf09a4954bb

data/CHANGELOG.md

@@ -5,6 +5,56 @@ All notable changes to Leptris will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.9.8] - 2026-08-26
+
+### Added
+
+- **`rake audit:symbols`**: the lockstep drift detector as an
+  executable step — `nm -gU` on the vendored library vs the ffi.rb
+  attach list, failing with both directions of drift (ADR 0001's
+  enforcement arm; previously prose plus ad-hoc pipelines).
+
+### Changed
+
+- ffi.rb attaches the five header-declared exports past cleanups
+  had dropped (element_children, serialize_document,
+  xpath_ns_set_add, xpath_result_get, xpath_result_get_nodes) —
+  mirror-only, supersession noted inline. The audit now runs green:
+  **224/224 symbols in lockstep**. Pull-attribute batch fetch asked
+  upstream (leptris/leptris#562).
+
+## [1.9.7] - 2026-08-26
+
+### Added
+
+- **DocumentFragment is searchable**: `fragment.xpath/at_xpath/
+  css/at_css/search` — Nokogiri fragment parity. The engine already
+  accepted the fragment handle as an XPath context node; the Ruby
+  class now includes Searchable.
+- **`Document#last_error_position`** → `[line, column]` (1-based,
+  thread-global and sticky per the C contract) — the position
+  companion to the recover mode's failure detail. Closes an
+  export-audit gap: `leptris_last_error_position` was exported but
+  unattached.
+
+### Changed
+
+- **css is receiver-relative** (Nokogiri semantics): `element.css`
+  and `fragment.css` scope to the receiver (`.//`), `Document#css`
+  stays document-absolute (`//`). Previously every receiver
+  translated to document-absolute XPath — wrong results for any
+  non-root element or fragment context. Document-context behavior
+  is unchanged.
+
+### Meta
+
+- **CONTEXT.md + docs/adr/**: the domain glossary (ownership model,
+  borrowed handles, wrapper identity, readonly contract, the seam
+  and its two faces) and five ADRs recording the settled decisions
+  (lockstep mirror, UTF-8 at the seam, hand-rolled memoization
+  pattern, the lifetime guard and its measured price, autoload
+  ordering) — future reviews stop re-deriving them.
+
 ## [1.9.6] - 2026-08-26
 
 ### Changed

data/CLAUDE.md

@@ -87,7 +87,11 @@ lib/leptris/xml/css_to_xpath.rb   — minimal CSS translation
 
 ## Reference
 
+- Domain vocabulary: CONTEXT.md. Settled decisions: docs/adr/ —
+  check both before proposing refactors; they exist so reviews
+  stop re-deriving settled questions.
 - libleptris public headers (`src/include/leptris/*.h`) are the
-  contract; when symbols change, bump lockstep and audit
-  attached-vs-exported (`nm -gU` on a fresh build).
+  contract; when symbols change, bump lockstep and run
+  `rake audit:symbols` (attached == exported on the vendored
+  library; fails listing both directions of drift).
 - Upstream issues worth tracking live at leptris/leptris.

data/CONTEXT.md

@@ -0,0 +1,39 @@
+# Domain glossary
+
+The vocabulary for talking about leptris-ruby. Architecture reviews
+use these names; see docs/adr/ for settled decisions.
+
+**The DOM** — the C tree built by libleptris. Single source of
+truth: every Ruby object is a view over it, and one Ruby method is
+one FFI call wherever possible.
+
+**The seam** — `Leptris::XML::FFI`: every attach, every struct
+layout, every buffer protocol, and the encoding policy (UTF-8 on
+both faces: returns and callback params). ABI knowledge appears
+nowhere else (document.rb's finalizer is ownership machinery;
+CStringArray is the char** adapter).
+
+**Ownership / the borrowed handle** — Document owns C memory
+(explicit `#free` or GC finalizer); every other object (Node,
+Element, Text, Comment, CDATA, PI, Attr, DocumentFragment, DocType)
+is a handle borrowed for its document's lifetime. Mutations raise
+ReadOnlyError on readonly documents; use of a handle after `#free`
+raises UseAfterFreeError.
+
+**Wrapper identity** — `Node.wrap` is the only wrapper constructor;
+the per-document strong cache makes the same C node always yield the
+same Ruby object (`doc.root.equal?(doc.root)`).
+
+**The readonly contract** — `readonly: true` makes reads memoizable
+because they cannot go stale; every read method follows the same
+three-line pattern (return memo → compute → store memo). Mutations
+are refused; detached factories still work.
+
+**Lockstep** — the gem's major.minor tracks libleptris; a C release
+is one PR (version.rb + Rakefile pin + CHANGELOG) then the release
+workflow; ffi.rb mirrors the public header so the attached-vs-
+exported audit (`nm -gU`) catches drift.
+
+**The recover mode** — `recover: true` parses return an empty
+document with the failure recorded on the thread-global last error
+(+ position), instead of raising ParseError.

data/README.adoc

@@ -169,12 +169,13 @@ against a readonly document is legal; mutating the frozen one is not.
 
 == Searching: XPath and CSS
 
-`Document` and `Element` (via `Leptris::XML::Searchable`) support:
+`Document`, `Element`, and `DocumentFragment` (via
+`Leptris::XML::Searchable`) support:
 
 [horizontal]
 `#xpath(*exprs)` :: evaluate XPath; returns `NodeSet`, `true`/`false`, `Float`, or `String` depending on the expression.
 `#at_xpath(*exprs)` :: first match (or scalar), like `xpath(*exprs).first`.
-`#css(*selectors)` :: minimal CSS-to-XPath translation, then `xpath`.
+`#css(*selectors)` :: minimal CSS-to-XPath translation, then `xpath`. Receiver-relative (Nokogiri semantics): scoped to the element or fragment, document-wide from a Document.
 `#at_css(*selectors)` :: first match of `css`.
 `#search(*exprs)` :: dispatches on syntax — `/`-prefixed or `,`-separated → `xpath`, otherwise `css`.
 `#at(*exprs)` :: first match of `search`.
@@ -190,6 +191,11 @@ doc.at_xpath("string(//book[1]/@id)")     # => "b1"
 doc.css("book[lang='en'] title")          # => NodeSet[<title>Refactoring</title>]
 doc.at_css("book#b1 title")               # => <title>Refactoring</title>  (id selector)
 doc.css("book:first-child")               # first <book>
+
+# css is receiver-relative: scoped to the receiver, not the document
+doc.root.at_css("book").css("title")      # titles under THAT book only
+frag = doc.fragment("<a x='1'><n/></a>")
+frag.css("a > n")                         # searches the fragment
 ----
 
 XPath result type follows XPath 1.0 semantics:

data/Rakefile

@@ -49,6 +49,41 @@ task spec: :compile unless ENV.key?("LEPTRIS_LIB_PATH")
 
 task default: :spec
 
+# Lockstep drift detector (ADR 0001): ffi.rb mirrors the public
+# header, so attached == exported must hold on the vendored library.
+# Run after `rake compile` and before any lockstep release.
+namespace :audit do
+  desc "Fail when ffi.rb attachments and library exports drift"
+  task :symbols do
+    lib = Dir.glob("lib/libleptris.{dylib,so,dll}").first
+    unless lib && system("which nm > /dev/null 2>&1")
+      abort "audit:symbols: vendored library or nm not found — run rake compile"
+    end
+    exported = `nm -gU #{lib}`
+      .lines.map { |l| l.split[2] }.compact
+      .map { |n| n.sub(/\A_/, "") }
+      .select { |n| n.start_with?("leptris_") }
+      .map { |n| n.sub(/\Aleptris_/, "") }.sort
+    attached = File.read("lib/leptris/xml/ffi.rb")
+      .scan(/attach_function :leptris_([a-z_0-9]+)/).flatten.sort
+    unattached = exported - attached
+    unexported = attached - exported
+    unless unattached.empty?
+      puts "exported but NOT attached (upstream surface drift):"
+      unattached.each { |s| puts "  leptris_#{s}" }
+    end
+    unless unexported.empty?
+      puts "attached but NOT exported (stale attachment):"
+      unexported.each { |s| puts "  leptris_#{s}" }
+    end
+    if unattached.empty? && unexported.empty?
+      puts "audit:symbols: #{attached.length}/#{exported.length} symbols in lockstep"
+    else
+      abort "audit:symbols: drift detected"
+    end
+  end
+end
+
 require "rubygems/package_task"
 
 desc "Build the pure-Ruby gem"

data/docs/adr/0001-lockstep-mirror.md

@@ -0,0 +1,17 @@
+# ADR 0001: ffi.rb mirrors the public header
+
+## Context
+ffi.rb attaches ~30 functions the binding never calls (typed
+attribute accessors, copy-inserts, allocator hooks, xinclude
+getters). Reviews keep proposing to strip them.
+
+## Decision
+Keep attaching every public-header symbol. The mirror makes the
+attached-vs-exported audit (`nm -gU` on a fresh build) a drift
+detector: when libleptris changes its surface, the audit fails
+loudly instead of the binding silently missing ABI.
+
+## Consequences
+Attachment is declarative and cheap; unused functions cost nothing.
+When a call site starts using a string-returning mirror function,
+add it to the UTF8_RETURNS wrap-list (ADR 0002).

data/docs/adr/0002-utf8-at-the-seam.md

@@ -0,0 +1,19 @@
+# ADR 0002: UTF-8 at the seam, both faces
+
+## Context
+The headers contract every C string as UTF-8, but FFI's implicit
+string conversion returns ASCII-8BIT — on returns (2026-08-25,
+release 1.9.3) and on FFI::Function callback params (2026-08-25,
+1.9.4; only `characters` was corrected before).
+
+## Decision
+The encoding policy lives entirely in ffi.rb: the UTF8_RETURNS
+wrap-list for called string-returning attaches, plus
+read_owned_string, CStringArray.to_ruby, and the SAX adapter's
+nil-safe utf8 for callback params. Mirror-only attachments stay
+raw; they join the list when a call site appears.
+
+## Consequences
+Memoized readonly results can safely cache strings (they freeze as
+UTF-8, the correct encoding, not BINARY). Non-ASCII content can no
+longer leak BINARY into consumer code.

data/docs/adr/0003-readonly-memoization-pattern.md

@@ -0,0 +1,18 @@
+# ADR 0003: readonly memoization stays hand-rolled
+
+## Context
+Every readonly-mode read repeats the same three lines:
+`return @x if readonly_cached?(:@x); v = compute; @x = v if
+@document&.readonly?`. ~25 sites. Reviews keep proposing a
+`readonly_memo(ivar) { }` concentrator.
+
+## Decision
+Keep the pattern hand-rolled. Concentrating it requires
+instance_variable_get/set — banned project-wide (encapsulation:
+no reaching into another object's ivars, and metaprogrammed ivar
+access hides the memo identity from readers).
+
+## Consequences
+The repetition IS the invariant — greppable, one file at a time.
+New readonly reads copy the three lines verbatim; a site that
+deviates is visible in review.

data/docs/adr/0004-lifetime-guard.md

@@ -0,0 +1,21 @@
+# ADR 0004: borrowed handles validate their lender
+
+## Context
+Before 1.9.5, only Document's own methods checked freed state:
+`doc.free; node.name` read freed memory silently (verified
+returning ""), stale mutations risked segfaults.
+
+## Decision
+`Node#ensure_alive!` guards every c_ptr-derefercing entry point —
+after memo checks (memoized readonly results stay check-free) and
+folded into `ensure_writable!` for mutations. The check is the
+cheapest sufficient one: `Document#c_ptr.nil?` (#free nils it; the
+GC finalizer cannot fire while any handle exists). `Document#freed?`
+is the accurate public predicate.
+
+## Consequences
+Measured cost (harness, 1.9.4→1.9.6): visible only on the
+single-cheapest call shape (Element#[] hot loops, ~13-28%); every
+other loop at parity. Accepted as the price of the contract;
+readonly hot loops have the memoized attributes/keys alternatives.
+Do not re-litigate without a C-side lifetime mechanism.

data/docs/adr/0005-autoload-manifest-ordering.md

@@ -0,0 +1,19 @@
+# ADR 0005: the eager FFI require runs after autoload registration
+
+## Context
+Issue #53: eager-loading ffi.rb from inside `module Leptris`
+BEFORE `autoload :XML` was registered let ffi.rb's module opening
+create Leptris::XML first; the registration was shadowed, xml.rb
+never loaded, and the whole API was unreachable
+(constants == [:FFI]).
+
+## Decision
+leptris.rb: register `autoload :XML` first, then the eager
+`require "leptris/xml/ffi"` at top level — ffi.rb's module opening
+triggers the autoload, landing FFI inside the real manifest module;
+a downstream `require "leptris/xml"` is then a no-op. A subprocess
+spec guards the cold path (the in-process suite cannot see it).
+
+## Consequences
+Any new eager require in leptris.rb must come after the autoload
+block. Eager resolution at require time (issue #49) is unchanged.

data/lib/leptris/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Leptris
-  VERSION = "1.9.6"
+  VERSION = "1.9.8"
 end

data/lib/leptris/xml/css_to_xpath.rb

@@ -35,30 +35,34 @@ module Leptris
 
       module_function
 
-      # Translation is a pure function of the rule string, and real
-      # workloads repeat a small selector vocabulary in loops —
-      # memoize. Failed translations raise before caching.
+      # Translation is a pure function of the rule string AND the
+      # context prefix, and real workloads repeat a small selector
+      # vocabulary in loops — memoize. Failed translations raise
+      # before caching. +prefix+ scopes the result: "//" for
+      # document receivers, ".//" for elements and fragments
+      # (Nokogiri receiver-relative semantics).
       CACHE = {}
       private_constant :CACHE
 
-      def convert(rule)
-        key = rule.to_s
-        CACHE.fetch(key) { CACHE[key] = convert_rule(key) }
+      def convert(rule, prefix: "//")
+        key = "#{prefix}\t#{rule}"
+        CACHE.fetch(key) { CACHE[key] = convert_rule(rule, prefix) }
       end
 
-      def convert_rule(rule)
-        rule.split(COMMA_SPLIT).map { |r| convert_one(r.strip) }.join(" | ")
+      def convert_rule(rule, prefix)
+        rule.split(COMMA_SPLIT)
+          .map { |r| convert_one(r.strip, prefix) }.join(" | ")
       end
 
-      def convert_one(rule)
-        return "//*" if rule == "*"
+      def convert_one(rule, prefix)
+        return "#{prefix}*" if rule == "*"
 
         # Tokenize chain first (handles > and whitespace)
         if rule =~ /\s/ || rule.include?(">")
-          return convert_chain(rule)
+          return convert_chain(rule, prefix)
         end
 
-        convert_simple(rule, prefix: "//")
+        convert_simple(rule, prefix: prefix)
       end
 
       # Parse a single simple selector into (tag, predicates) where
@@ -140,9 +144,9 @@ module Leptris
 
       # Tokenize chain into [sel, op, sel, op, sel, ...] where op is :child or
       # :descendant. Then build XPath.
-      def convert_chain(rule)
+      def convert_chain(rule, prefix)
         tokens = tokenize_chain(rule)
-        build_chain_xpath(tokens)
+        build_chain_xpath(tokens, prefix)
       end
       private_class_method :convert_chain
 
@@ -167,11 +171,11 @@ module Leptris
       end
       private_class_method :tokenize_chain
 
-      def build_chain_xpath(tokens)
+      def build_chain_xpath(tokens, prefix)
         first = tokens.shift
         raise ArgumentError, "empty CSS chain" unless first.is_a?(String)
 
-        xpath = convert_simple(first, prefix: "//")
+        xpath = convert_simple(first, prefix: prefix)
         until tokens.empty?
           op = tokens.shift
           sel = tokens.shift

data/lib/leptris/xml/document.rb

@@ -287,6 +287,22 @@ class Leptris::XML::Document
     @freed.state == :freed || @c_ptr.nil?
   end
 
+  # The thread-global last-failure [line, column] (1-based), or nil
+  # when no error is recorded — the position companion to
+  # Document#last_error; populated by recover parses.
+  def last_error_position
+    line = ::FFI::MemoryPointer.new(:int)
+    column = ::FFI::MemoryPointer.new(:int)
+    begin
+      Leptris::XML::FFI.leptris_last_error_position(line, column)
+      line.read_int.zero? && column.read_int.zero? ? nil :
+        [line.read_int, column.read_int]
+    ensure
+      line.free
+      column.free
+    end
+  end
+
   # The most recent error recorded against this document, or nil.
   def last_error
     msg = Leptris::XML::FFI.leptris_document_last_error(@c_ptr)

data/lib/leptris/xml/document_fragment.rb

@@ -5,6 +5,8 @@
 # the fragment itself isn't part of any document tree but borrows its
 # document's lifetime.
 class Leptris::XML::DocumentFragment
+  include Leptris::XML::Searchable
+
   attr_reader :document, :c_ptr
 
   def initialize(document, c_ptr)

data/lib/leptris/xml/ffi.rb

@@ -173,6 +173,10 @@ module Leptris
       # count-only query returning the TOTAL across every child kind.
       attach_function :leptris_node_children,
         [:leptris_node_ref, :pointer, :size_t], :size_t
+      # Element-only batch; mirror-only — fetch_children rides the
+      # all-kind node_children above.
+      attach_function :leptris_element_children,
+        [:leptris_element, :pointer, :size_t], :size_t
       attach_function :leptris_node_as_element,
         [:leptris_node_ref], :leptris_element
       attach_function :leptris_element_as_node,
@@ -402,6 +406,12 @@ module Leptris
       # which (element / synthetic attribute / text / other).
       attach_function :leptris_xpath_result_node_kind,
         [:leptris_xpath_result, :size_t], :int
+      # Pre-_ex variants, superseded by xpath_result_get_nodes_ex /
+      # xpath_result_get_node; mirror-only (ADR 0001).
+      attach_function :leptris_xpath_result_get,
+        [:leptris_xpath_result, :size_t], :leptris_element
+      attach_function :leptris_xpath_result_get_nodes,
+        [:leptris_xpath_result, :pointer, :size_t], :size_t
       attach_function :leptris_xpath_result_get_node,
         [:leptris_xpath_result, :size_t], :leptris_node_ref
       attach_function :leptris_xpath_result_node_name,
@@ -447,6 +457,10 @@ module Leptris
         [:leptris_xpath_ns_set], :void
       # One-call constructor: flat array of 2*pair_count alternating
       # prefix/URI strings.
+      # Per-pair add; superseded by ns_set_new_from_pairs (one call);
+      # mirror-only (ADR 0001).
+      attach_function :leptris_xpath_ns_set_add,
+        [:leptris_xpath_ns_set, :string, :string], :leptris_status
       attach_function :leptris_xpath_ns_set_new_from_pairs,
         [:pointer, :size_t], :leptris_xpath_ns_set
       attach_function :leptris_xpath_eval_ns,
@@ -507,6 +521,10 @@ module Leptris
 
       attach_function :leptris_document_serialize,
         [:leptris_document, :pointer], :pointer
+      # Deprecated automatic-options alias (header: prefer
+      # leptris_document_serialize); mirror-only.
+      attach_function :leptris_serialize_document,
+        [:leptris_document], :pointer
       # libleptris >= 1.9.0: caller-buffer serialization with options
       # (leptris#541). buf=NULL is a size query; the size-query +
       # fill pair reuses one serialization through a per-document
@@ -553,6 +571,10 @@ module Leptris
         [:leptris_document, :pointer, :pointer], :leptris_status
       attach_function :leptris_status_string, [:leptris_status], :string
       attach_function :leptris_error_message, [:leptris_status], :string
+      # Thread-global last-failure position (error.h): companion to
+      # leptris_last_error; populated by recover parses (#547).
+      attach_function :leptris_last_error_position,
+        [:pointer, :pointer], :void
       # Thread-local since v1.3.0; reliable under the
       # one-document-per-thread contract.
       attach_function :leptris_last_error, [], :string

data/lib/leptris/xml/node.rb

@@ -205,15 +205,9 @@ class Leptris::XML::Node
 
   # Walks the subtree in post-order DFS (matches Nokogiri's semantics).
   #
-  # Specialized hot path: skips the intermediate NodeSet allocation that
-  # Element#children would create, walking via raw FFI calls and wrapping
-  # nodes directly. Saves one Array + one NodeSet allocation per parent
-  # node. For a tree of N nodes that's ~N fewer allocations on a full
-  # traversal.
-  #
-  # Still pays ~2 FFI calls per visited node (first_child + next_sibling).
-  # Beating Nokogiri on this benchmark needs C-side traverse with a
-  # callback (libleptris #273); the per-node FFI cost is the floor.
+  # One FFI call dispatches the whole walk; the C engine invokes the
+  # callback once per visited node (the only per-node cost is the
+  # C-to-Ruby callback dispatch, not FFI round-trips).
   def traverse
     return enum_for(:traverse) unless block_given?
     ensure_alive!

data/lib/leptris/xml/searchable.rb

@@ -44,7 +44,11 @@ module Leptris::XML::Searchable
     handler, ns, _ = parse_search_args(args)
     raise ArgumentError, "namespace bindings not supported in css" if ns && !ns.empty?
     raise ArgumentError, "custom CSS handlers not supported" if handler
-    expr = args.map { |r| Leptris::XML::CssToXPath.convert(r) }.join(" | ")
+    # Nokogiri semantics: css is receiver-relative — absolute "//"
+    # from a Document, descendant ".//" from elements and fragments.
+    prefix = is_a?(Leptris::XML::Document) ? "//" : ".//"
+    expr = args.map { |r| Leptris::XML::CssToXPath.convert(r, prefix: prefix) }
+      .join(" | ")
     xpath(expr)
   end
 

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: leptris
 version: !ruby/object:Gem::Version
-  version: 1.9.6
+  version: 1.9.8
 platform: ruby
 authors:
 - Ribose Inc.
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-08-25 00:00:00.000000000 Z
+date: 2026-08-26 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: ffi
@@ -65,9 +65,15 @@ extra_rdoc_files: []
 files:
 - CHANGELOG.md
 - CLAUDE.md
+- CONTEXT.md
 - LICENSE.md
 - README.adoc
 - Rakefile
+- docs/adr/0001-lockstep-mirror.md
+- docs/adr/0002-utf8-at-the-seam.md
+- docs/adr/0003-readonly-memoization-pattern.md
+- docs/adr/0004-lifetime-guard.md
+- docs/adr/0005-autoload-manifest-ordering.md
 - leptris.gemspec
 - lib/leptris.rb
 - lib/leptris/version.rb