leptris 1.9.54 → 1.9.56

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: '028bd827a38f7d8caceadedb1bc882ff6a0bb3ae15151914abe5047c8668ed60'
-  data.tar.gz: 45a28f45bfe6546c24cbf8c83800d0883d82d1f486a0169c55ca78a992e05432
+  metadata.gz: 6e63d8af535ae0bfc72372d3ed8add688494d9a05b40175b038c7d83c4cdff15
+  data.tar.gz: 110517ea3d517fb76f2f1b6adb623f6c50fbe731fd41ec821efb21e54de458d9
 SHA512:
-  metadata.gz: 5c47e9fbc105d04e4f4504e56d0ab2584492ddb3f613e95e776eb527982858648ebbe3d05c95c7864e9676b1385def4d7fe5b6314567baf03b0a2c52dc05af41
-  data.tar.gz: 65b68638bad6fcea08332875c21222b81467a074dbc5426a1e2f3596fb0cc84214895c5ed336b677288860d34566758b4bc83018365d7ca400f10deb027e0787
+  metadata.gz: 14054ad4ad00851c244ffc8c9493df25c6a300bfd96f4eb9b827001e75aa5e35e06e3dcf310efc08c4fe61bc3f4030121fa5d3127cc9ce2213a367546dd2ca46
+  data.tar.gz: a301bc6f767c7b8c67ea4c06c15b99cbe2fbec50d848eefb4c743d03aa0e1fa64d2eb49083a046c7859c6506f4ad5c7cde5dcc65779161ceff35821b29e45708

data/CHANGELOG.md

@@ -5,6 +5,93 @@ All notable changes to Leptris will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.9.56] - 2026-09-03
+
+### Added
+
+- **Lockstep with libleptris 1.9.66** (1.9.51 → 1.9.66 — sixteen
+  releases, the XPath 3.1 value-level + XQuery wave). Four new C
+  symbols attached (audit 259/259):
+  - **`Leptris::XML::XQuery`** (1.9.64-1.9.66): compile-once /
+    evaluate-many XQuery 1.0 core — prolog (`declare variable` /
+    `namespace` / `function local:*`), nested `for` with `at`
+    positions, `let`, `where`, stable multi-key `order by`, direct
+    and computed constructors with attribute value templates, and
+    plain XPath expression bodies. Results reuse the XPath result
+    handle; constructor results arrive as readable strings, FLWOR
+    sequences through the sequence channel. Twelve specs.
+  - **`Leptris::XML.buffer_has_nonstandard_entity?`** (1.9.62,
+    closes leptris-ruby#124 / upstream #745): the one-pass C
+    entity pre-scan — measured 7.5 µs vs 24 µs for the Ruby regex
+    it replaces on a 13 KB document (+18% vs +59% over parse).
+  - **XPath 3.1 value-level surface** (1.9.51-1.9.63), all
+    resolving standalone from `#xpath` and specced: map/array
+    constructors and accessors (`?key`, `map:get`, `array:get`,
+    sizes), `parse-json`, `json-to-xml` / `xml-to-json`,
+    `serialize(…, map { 'method': 'json' })`, postfix `?lookup`,
+    inline function items and dynamic calls, `function-lookup`,
+    `fold-left`, `for-each`, named function references.
+  - 1.9.61 (upstream #739/#744): `xs:boolean('0')` casts to
+    false, `xs:` string lexicals validated.
+- **README**: XQuery section, value-level examples in the
+  standalone-subset block, the migration list drops "No XQuery".
+
+### Fixed
+
+- **XQuery v1 grammar gaps found on first contact, filed as
+  leptris/leptris#790** (pending specs against it): constructor as
+  the FLWOR return clause fails to parse; `where`/`at` clauses
+  inside a function-argument FLWOR fail to parse (bare for-return
+  works); `'nope' cast as xs:integer` NaNs instead of erroring, so
+  `try/catch` never reaches its catch arm.
+
+## [1.9.55] - 2026-09-02
+
+### Changed
+
+- **Lockstep with libleptris 1.9.50**: XPath 2.0 type operators —
+  `instance of`, `castable as`, `cast as`, `treat as` with
+  SequenceType v1 (`node()`, `item()`, occurrence `*`/`+`) —
+  resolving standalone from `#xpath`, specced against the Saxon
+  ground truth including the falsifiable negative
+  (`'x' castable as xs:integer` = false, `1.9 cast as xs:integer`
+  = 1). No new C surface (audit 255/255).
+- **README**: the standalone-subset example block grows the type
+  operators; the #683 gap list drops sequence types (still out:
+  value comparators, quantifiers, `intersect`/`except`, string
+  templates, XQuery).
+
+### Improved
+
+- **Serialize + inner_html, one dispatch instead of two**: the
+  `_serialize_into` contract is snprintf-like (the return is
+  always the bytes needed incl. NUL; the copy happens only when
+  the buffer fits), so `serialize_into_string` now fills the
+  grow-only byte scratch opportunistically and probes the size
+  only when a child outgrows it — previously every call paid a
+  size-probe dispatch before the fill. The bound serializer
+  methods are hoisted to constants (`Element#to_xml` and
+  `Document#to_xml` allocated a Method object per call), and
+  `inner_html` serializes element children through a dedicated
+  default-options fast path (no kwargs, no options rebuild).
+  Measured on a 1.0 MB / 42k-element document: document serialize
+  1.1x → **3.2x faster** than Nokogiri 1.19.4; `inner_html` 0.8x →
+  **4.4x faster** (4.2 ms → 1.1 ms per 200 elements). Parse (11x),
+  attr reads (9x), text aggregation (11x), and element_children
+  walks (parity) all hold.
+
+### Fixed
+
+- **#120 closed with a guard**: text ampersands escape on
+  serialize — byte-parity specs (parse-and-serialize round-trip
+  plus a `content=` write path) now run on every CI leg including
+  both Linuxes, so a platform-build serializer divergence fails CI
+  instead of shipping. The 1.9.50 Linux build was superseded per
+  the verification on the issue.
+- **#124 filed upstream** (leptris/leptris#745): the C-side
+  non-standard-entity pre-scan entry — the moxml +61% parse → ~+2%
+  case. The binding attaches and wraps it the day it lands.
+
 ## [1.9.54] - 2026-09-02
 
 ### Changed

data/README.adoc

@@ -9,9 +9,9 @@ A Nokogiri-compatible Ruby binding for
 https://github.com/leptris/leptris[libleptris], a pure-C99 XML 1.0 parser
 with full https://www.w3.org/TR/1999/REC-xpath-19991116/[XPath 1.0],
 XML Namespaces 1.0, SAX, and C14N (1.0 / 1.1 / Exclusive), plus
-XSLT 1.0–3.0 transforms (the XPath 3.1 core rides inside them) and a
-growing XPath 2/3.1 expression subset available standalone from
-`#xpath` itself.
+XSLT 1.0–3.0 transforms, an XQuery 1.0 core face, and a growing
+XPath 2/3.1 expression subset available standalone from `#xpath`
+itself.
 
 The C DOM is the single source of truth — Ruby objects are thin FFI
 handles over the C pointers, so every Ruby method maps to one FFI call.
@@ -239,6 +239,14 @@ doc.xpath("count(1 to 4)")                           # => 4.0
 doc.xpath("//book => count()")                       # 3.1 arrow
 doc.xpath("//title ! string(.)")                     # 3.1 simple map
 doc.xpath("'v' || count(//book)")                    # 3.1 concat
+doc.xpath("'42' castable as xs:integer")             # 2.0 type ops
+doc.xpath("1.9 cast as xs:integer")                  # => 1.0 (truncates)
+doc.xpath("//title instance of node()+")             # 2.0 sequence types
+doc.xpath("map { 'b': 'beta' }?b")                   # 3.1 maps
+doc.xpath("[10, 20, 30]?2")                          # 3.1 arrays
+doc.xpath(%q{serialize(parse-json('{"b":"beta"}'), map { 'method': 'json' })})
+doc.xpath("function($x) { $x + 1 }(41)")             # 3.1 function items
+doc.xpath("fold-left(1 to 4, 0, function($a, $b) { $a + $b })")
 ----
 
 Two caveats: `for … return` sequences come back as opaque result
@@ -247,8 +255,8 @@ nodes — consume them through an aggregate (`count(...)`,
 items; and XQuery-only syntax (`order by`, `try/catch`, map
 constructors) raises `XPathError` rather than evaluating silently
 wrong. The remaining grammar gaps — value comparators (`eq`),
-quantifiers, sequence types (`instance of`), `intersect`/`except`,
-string templates — and XQuery itself are
+quantifiers, `intersect`/`except`, string templates — and XQuery
+itself are
 https://github.com/leptris/leptris/issues/683[tracked upstream].
 
 === Supported CSS selectors
@@ -379,6 +387,39 @@ dispositions drop unmatched subtrees in the built-in initial
 descent (https://github.com/leptris/leptris/issues/705[#705]) —
 the other four dispositions are spec-correct. XQuery has no
 entry point yet.
+== XQuery
+
+XQuery 1.0 core through `Leptris::XML::XQuery` (compile once,
+evaluate many):
+
+[source,ruby]
+----
+query = Leptris::XML::XQuery.parse(<<~XQ)
+  declare variable $min := 3;
+  for $i in //item
+  where number($i/@qty) > $min
+  order by $i/@qty descending
+  return <big>{$i/name/text()}</big>
+XQ
+query.eval(doc)   # plain expressions keep their XPath result type;
+                  # FLWOR results arrive as the sequence channel —
+                  # read them through an aggregate until the engine
+                  # materializes readable sequence items
+----
+
+Supported: the prolog (`declare variable` / `declare namespace` /
+`declare function local:*`), nested `for` with `at` positions,
+`let`, `where`, stable multi-key `order by`, direct and computed
+constructors with attribute value templates, and plain XPath
+expression bodies. Known grammar gaps are
+https://github.com/leptris/leptris/issues/790[tracked upstream].
+
+`Leptris::XML.buffer_has_nonstandard_entity?(string)` is a
+one-pass C pre-scan (for adapter layers): true when the buffer
+contains a named entity outside the five predefined ones (or
+numeric) — ~3x faster than the equivalent Ruby regex and no
+false positives on bare `&`.
+
 `Document#save(path, **opts)` :: serialize to a file.
 `Document#canonicalize(version, inclusive_ns, with_comments:, exclusive:, mode:)` (alias `#c14n`) :: canonical XML.
 `Element#canonicalize(...)` :: subtree canonicalization.
@@ -545,8 +586,8 @@ Notable differences:
   past Document disposal will see the error; silence-replace-UAF patterns
   from Nokogiri do not apply.
 * No `Nokogiri::HTML` or `Nokogiri::CSS` parser. Leptris is XML-only.
-* No XQuery, no RelaxNG / DTD validation, no schema caching
-  (XSLT 1.0–3.0 is supported — see above).
+* No RelaxNG / DTD validation, no schema caching (XSLT 1.0–3.0
+  and an XQuery 1.0 core are supported — see above).
 * No built-in JRuby / TruffleRuby support — only CRuby via `ffi`.
 
 == Performance

data/Rakefile

@@ -8,7 +8,7 @@ RSpec::Core::RakeTask.new(:spec)
 # Pin for `rake compile` and the platform-gem builds. Keep in lockstep
 # with .github/workflows/build.yml (which calls `rake compile`) and the
 # CHANGELOG when libleptris releases.
-LIBLEPTRIS_VERSION = "1.9.49"
+LIBLEPTRIS_VERSION = "1.9.66"
 
 CMAKE_FLAGS = %w[
   -DCMAKE_BUILD_TYPE=Release

data/lib/leptris/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Leptris
-  VERSION = "1.9.54"
+  VERSION = "1.9.56"
 end

data/lib/leptris/xml/document.rb

@@ -285,7 +285,7 @@ class Leptris::XML::Document
         @c_ptr, indent: indent, no_decl: no_decl, encoding: encoding)
     end
     Leptris::XML::Serialization.to_xml(
-      Leptris::XML::FFI.method(:leptris_document_serialize_into), @c_ptr,
+      Leptris::XML::Serialization::DOCUMENT_SERIALIZE_INTO, @c_ptr,
       indent: indent, no_decl: no_decl, encoding: encoding)
   end
   alias_method :to_s, :to_xml

data/lib/leptris/xml/element.rb

@@ -349,7 +349,8 @@ class Leptris::XML::Element < Leptris::XML::Node
   def inner_html
     children.map do |child|
       case child
-      when Leptris::XML::Element then child.to_xml
+      when Leptris::XML::Element
+        Leptris::XML::Serialization.element_xml_default(child.c_ptr)
       when Leptris::XML::CDATA # before Text: CDATA subclasses it
         "<![CDATA[#{child.content}]]>"
       when Leptris::XML::Text
@@ -377,7 +378,7 @@ class Leptris::XML::Element < Leptris::XML::Node
     return Leptris::XML::Serialization.to_xml_element_unit(
       self, indent_text, indent: indent) if indent_text.is_a?(String)
     Leptris::XML::Serialization.to_xml(
-      Leptris::XML::FFI.method(:leptris_element_serialize_into), @c_ptr,
+      Leptris::XML::Serialization::ELEMENT_SERIALIZE_INTO, @c_ptr,
       indent: indent, no_decl: no_decl, encoding: encoding)
   end
 

data/lib/leptris/xml/ffi.rb

@@ -43,6 +43,7 @@ module Leptris
       typedef :pointer, :leptris_xpath_ns_set
       typedef :pointer, :leptris_xpath_compiled
       typedef :pointer, :leptris_xslt
+      typedef :pointer, :leptris_xquery
       typedef :pointer, :leptris_sax_recorder
       typedef :pointer, :leptris_pull_parser
       typedef :pointer, :leptris_iterparse
@@ -671,6 +672,25 @@ module Leptris
       attach_function :leptris_document_save_file,
         [:leptris_document, :string, :pointer], :leptris_status
       # XSLT 1.0 engine (libleptris 1.9.1): compile once, apply many.
+      # XQuery 1.0 core (libleptris 1.9.64-1.9.66): orchestration
+      # over the XPath engine — prolog (declare variable/namespace/
+      # function local:*), FLWOR, computed and direct constructors.
+      # Results reuse the XPath result handle (free with
+      # leptris_xpath_result_free); context_node NULL = document.
+      attach_function :leptris_xquery_parse,
+        [:string, :size_t], :leptris_xquery
+      attach_function :leptris_xquery_eval,
+        [:leptris_xquery, :leptris_document, :leptris_element],
+        :leptris_xpath_result
+      attach_function :leptris_xquery_free, [:leptris_xquery], :void
+
+      # One-pass entity pre-scan (libleptris 1.9.62, leptris#745):
+      # 1 when the buffer holds a named entity that is not one of
+      # the five predefined XML entities (nor numeric) — the moxml
+      # pre-scan at C speed instead of a Ruby regex.
+      attach_function :leptris_str_has_nonstandard_entity,
+        [:string, :size_t], :int
+
       attach_function :leptris_xslt_parse,
         [:string, :size_t], :leptris_xslt
       attach_function :leptris_xslt_parse_file,
@@ -894,12 +914,29 @@ module Leptris
       # leptris_element_serialize_into. The buffer holds the
       # serialization + NUL and XML output is NUL-free, so the
       # bounded read is exact.
+      # snprintf contract (leptris.h #535): the return is ALWAYS the
+      # bytes needed incl. the NUL; the copy happens only when the
+      # buffer fits. So the size probe is paid only when the child
+      # outgrows the grow-only scratch — one dispatch per call on
+      # the fast path instead of two (inner_html serializes each
+      # child; the probe was most of the per-child FFI cost).
+      SERIALIZE_INITIAL = 256
+      private_constant :SERIALIZE_INITIAL
+
       def self.serialize_into_string(ffi_function, c_ptr, options)
-        need = ffi_function.call(c_ptr, nil, 0, nil, options)
-        return "" if need.zero?
-        buffer = scratch_bytes(need)
-        ffi_function.call(c_ptr, buffer, need, nil, options)
-        buffer.read_string.force_encoding(Encoding::UTF_8)
+        scratch = (Thread.current[:leptris_scratch] ||= {})
+        buffer = scratch[:bytes]
+        if buffer.nil?
+          buffer = scratch[:bytes] =
+            ::FFI::MemoryPointer.new(:char, SERIALIZE_INITIAL)
+        end
+        needed = ffi_function.call(c_ptr, buffer, buffer.size, nil, options)
+        if needed > buffer.size
+          buffer = scratch_bytes(needed)
+          needed = ffi_function.call(c_ptr, buffer, buffer.size, nil, options)
+        end
+        return "" if needed <= 1
+        buffer.read_string(needed - 1).force_encoding(Encoding::UTF_8)
       end
 
       # Byte scratch for serialize-into cycles: the size+fill pair

data/lib/leptris/xml/serialization.rb

@@ -18,6 +18,16 @@ module Leptris::XML::Serialization
   end
   private_constant :DEFAULT_OPTIONS
 
+  # Bound serializer entries, hoisted: Element#to_xml and the
+  # inner_html per-child loop passed FFI.method(:...) — one Method
+  # allocation per serialized child.
+  ELEMENT_SERIALIZE_INTO =
+    Leptris::XML::FFI.method(:leptris_element_serialize_into)
+  DOCUMENT_SERIALIZE_INTO =
+    Leptris::XML::FFI.method(:leptris_document_serialize_into)
+  # Public within the library: Element#to_xml and Document#to_xml
+  # pass the hoisted bound methods straight through.
+
   # +ffi_function+ is a bound _serialize_into function. The buffer
   # cycle lives at the FFI seam (FFI.serialize_into_string); this
   # module owns only options selection and construction.
@@ -33,6 +43,14 @@ module Leptris::XML::Serialization
     Leptris::XML::FFI.serialize_into_string(ffi_function, c_ptr, opts.pointer)
   end
 
+  # Default-options element serialization for inner_html's per-child
+  # loop: no kwargs, no options rebuild, no method allocation —
+  # one FFI fast-path dispatch per child.
+  def self.element_xml_default(c_ptr)
+    Leptris::XML::FFI.serialize_into_string(
+      ELEMENT_SERIALIZE_INTO, c_ptr, DEFAULT_OPTIONS.pointer)
+  end
+
   # Display-form document serialization (libleptris 1.9.9, #129):
   # the ext struct's indent_text hands ALL whitespace to the
   # formatter — text and mixed content indent too. Display-oriented:

data/lib/leptris/xml/xquery.rb

@@ -0,0 +1,69 @@
+# frozen_string_literal: true
+
+require "ffi"
+
+# Compiled XQuery 1.0 query (libleptris 1.9.64+): parse once,
+# evaluate many times against any document. The engine layer
+# orchestrates the XPath engine — the prolog (declare variable /
+# namespace / function local:*) binds into the evaluation context,
+# FLWOR clauses run as a tuple stream, and results reuse the XPath
+# result handle.
+#
+#     query = Leptris::XML::XQuery.parse(<<~XQ)
+#       for $i in //item
+#       where $i/@qty > 3
+#       order by $i/@qty descending
+#       return <big>{$i/name/text()}</big>
+#     XQ
+#     query.eval(doc)   # => Leptris::XML::NodeSet
+#
+class Leptris::XML::XQuery
+  class Handle < ::FFI::AutoPointer
+    def self.release(ptr)
+      Leptris::XML::FFI.leptris_xquery_free(ptr)
+    end
+  end
+
+  def self.parse(query)
+    query = query.to_s
+    raw = Leptris::XML::FFI.leptris_xquery_parse(query, query.bytesize)
+    if raw.null?
+      raise Leptris::XML::XPathError,
+        "query parse failed: #{Leptris::XML::FFI.leptris_last_error}"
+    end
+    new(query, Handle.new(raw))
+  end
+
+  attr_reader :query
+
+  def initialize(query, handle)
+    @query = query
+    @handle = handle
+  end
+
+  # Evaluates against +doc_or_element+. Plain-expression bodies keep
+  # their XPath result type (NodeSet / true/false / Float / String);
+  # FLWOR results arrive as the sequence channel — read them through
+  # an aggregate until the engine materializes readable sequence
+  # items (same caveat as the XPath for-return subset).
+  def eval(doc_or_element)
+    case doc_or_element
+    when Leptris::XML::Document
+      document = doc_or_element
+      context = nil
+    when Leptris::XML::Element
+      document = doc_or_element.document
+      context = doc_or_element.c_ptr
+    else
+      raise ArgumentError,
+        "expected a Leptris::XML::Document or Element, got #{doc_or_element.class}"
+    end
+    result_ptr = Leptris::XML::FFI.leptris_xquery_eval(
+      @handle, document.c_ptr, context)
+    if result_ptr.null?
+      raise Leptris::XML::XPathError,
+        Leptris::XML::FFI.leptris_last_error.to_s
+    end
+    Leptris::XML::Searchable.wrap_xpath_result(document, result_ptr)
+  end
+end

data/lib/leptris/xml.rb

@@ -23,6 +23,7 @@ module Leptris
     autoload :SAX, "leptris/xml/sax"
     autoload :XPath, "leptris/xml/xpath"
     autoload :XSLT, "leptris/xml/xslt"
+    autoload :XQuery, "leptris/xml/xquery"
     autoload :Pull, "leptris/xml/pull"
     autoload :Iterparse, "leptris/xml/iterparse"
 
@@ -33,6 +34,16 @@ module Leptris
                      recover: recover)
     end
 
+    # Does +string+ contain a named entity reference that is not
+    # one of the five predefined XML entities (nor numeric)? One C
+    # pass (libleptris 1.9.62) — downstream adapters use it to skip
+    # whole-buffer rewrites when a document is already clean.
+    def self.buffer_has_nonstandard_entity?(string)
+      string = string.to_s
+      Leptris::XML::FFI.leptris_str_has_nonstandard_entity(
+        string, string.bytesize) != 0
+    end
+
     def self.parse_file(path)
       Document.parse_file(path)
     end

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: leptris
 version: !ruby/object:Gem::Version
-  version: 1.9.54
+  version: 1.9.56
 platform: ruby
 authors:
 - Ribose Inc.
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2026-09-02 00:00:00.000000000 Z
+date: 2026-09-03 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: ffi
@@ -105,6 +105,7 @@ files:
 - lib/leptris/xml/serialization.rb
 - lib/leptris/xml/text.rb
 - lib/leptris/xml/xpath.rb
+- lib/leptris/xml/xquery.rb
 - lib/leptris/xml/xslt.rb
 homepage: https://github.com/leptris/leptris-ruby
 licenses: