leptris 1.9.177.0 → 1.9.178.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 9144461ce7cc58b12f57c1a5c497336f5712568654db476342b32b9d6d35dc93
-  data.tar.gz: a6fe5f35bf5891c3794f8bf5533c115b26b9eb19545d361aeafcb3b9b1935af0
+  metadata.gz: 31fb6e532087705ab9eb346db9cca66be9ed84c774eac9329a5c41ebdfedbb9d
+  data.tar.gz: 9f5006330eb3bf32e3beca44b2d14924b547bf54d8ddbc0a4054e90ca4a2e373
 SHA512:
-  metadata.gz: ee1362b97425fd76000f28d3232badcb7dd23d979e992cfd296ca17ee6bddeb716bd93b01830f22fd17103475cbfa4eccbcc918691625982fe4b79ca760de897
-  data.tar.gz: e1079b8c90d48d43e17d7ad3e7ebb55a8cbf87246cd61d0eed9eb2a44fc7c3cc91e5f2c034843ee1914f818d1c686b9596b433f96f2e22795fe02a005a75c21b
+  metadata.gz: bf50f65f973b9e05b8304d90a6f8f731c53450cc0967157214e8ae80b33ce02ba3a8f550c5359a16fe095cc34af4e3020829e7d8d9abf71c537840da1bea8526
+  data.tar.gz: 0ddbca8bf4a8064a2a956c0344d70a22e04884c6724b2ed2b9e73c7d78b58c527f1f13b930f530775e31aed1442e5a1f46bf6a2930c5a6a678890528f491a07c

data/CHANGELOG.md

@@ -5,6 +5,39 @@ All notable changes to Leptris will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [1.9.178.1] - 2026-09-16
+
+### Changed
+
+- **Bulk hydration surface (TODO.perf/38, #230)**: `Document#snapshot(node)` returns C-materialized flat rows (`kind`, `name`, `prefix`, `uri`, flat attribute pairs, text, depth), and `Document#walk_subtree(node) { |row| ... }` yields the same rows. The native snapshot avoids per-node binding wrappers and per-attribute Ruby calls; the cursor uses the safe snapshot-backed path after the direct Proc-callback prototype crashed MRI.
+- **Deterministic document lifetime (#231)**: `Document.open(xml) { |doc| ... }` guarantees `doc.free` on normal and exceptional exit. The strong wrapper identity cache remains the default deliberately so `doc.root.equal?(doc.root)` stays valid while the document is alive.
+- **Benchmark gate (#229)**: `LEPTRIS_BENCH_LOAD_MAX` overrides the load gate; the default is core-normalized (`max(logical_cores * 1.5, 4.0)`) rather than the old absolute load threshold.
+- **Windows native layer actually loads (fixes the silent gap since 1.9.174.2)**: the per-minor DLLs are renamed dot-free (`native_3_3.so`, `native_4_0.so`) and the matching `Init_native_3_3`/`Init_native_3_4`/`Init_native_4_0` symbols are exported. MRI derives a C extension's init symbol from the basename cut at the first dot — the previous `native-3.3.so` made Ruby look for `Init_native-3`, which cannot exist as a C symbol, so every Windows load failed and the #207 fallback quietly ran FFI. Windows test legs now exercise the native suite for real.
+
+## [1.9.178.0] - 2026-09-16
+
+### Changed — libleptris 1.9.177 → 1.9.178 (lockstep)
+
+- **#1113**: descriptor plan walk — collection wrappers now carry
+  the producing row's wire_name/type_tag (spec-pinned).
+- **#1115 / #1118**: descriptor plan surfaces — ChildPlan gains
+  rule-level `ns_form` (+ `ns_uri`) for mixed-qualification
+  schemas (additive to the frozen v1 ABI; the binding accepts
+  `:ns` on child rows and the trailing struct fields are
+  populated by `pack_plans`), and every value kind carries the
+  source node's byte offset (the accessor's contract widened;
+  readers respond with the engine's position).
+- **#1117**: C14N 1.1 ground truth — whitespace-only PI data and
+  document-level separators (lutaml-model parity corpus).
+- **#1111 / #659 / #1107**: xpath user_data, HTML corpus slices
+  (NUL/foreign-rules + frameset-ok + frameset-conversion +
+  scanner quote-parity), and CLI specs in out-of-tree builds.
+
+### Added
+
+- The trailing `ns_form` + `pad0` + `ns_uri` fields on
+  `Leptris::XML::FFI::ChildPlan` (the lockstep extension).
+
 ## [1.9.177.0] - 2026-09-16
 
 ### Changed — libleptris 1.9.174 → 1.9.177 (lockstep)

data/Rakefile

@@ -8,7 +8,7 @@ RSpec::Core::RakeTask.new(:spec)
 # Pin for `rake compile` and the platform-gem builds. Keep in lockstep
 # with .github/workflows/build.yml (which calls `rake compile`) and the
 # CHANGELOG when libleptris releases.
-LIBLEPTRIS_VERSION = "1.9.177"
+LIBLEPTRIS_VERSION = "1.9.178"
 # Vendored alongside libleptris for fn:normalize-unicode (TODO
 # .restructure/20): built per platform with a RELOCATABLE @rpath
 # install name, loaded by ffi.rb before libleptris so the
@@ -292,7 +292,11 @@ platforms.each do |platform|
     spec.files += Dir.glob("lib/{libutf8proc.3.dylib,libutf8proc.so.3,libutf8proc.so,utf8proc.dll}")
     if platform.include?("mingw")
       # #207/#227: per-Ruby-minor DLLs (PE must bind its Ruby).
-      spec.files += Dir.glob("lib/leptris/xml/native-*.so")
+      # DOT-FREE names — MRI derives the init symbol from the
+      # basename cut at the first dot (native_3_3.so ->
+      # Init_native_3_3; the dotted form made Ruby look for
+      # "Init_native-3", which cannot exist).
+      spec.files += Dir.glob("lib/leptris/xml/native_*.so")
     else
       spec.files += Dir.glob("lib/leptris/xml/native.{bundle,so,dll}")
     end

data/TODO.perf/38-bulk-walk-and-bench-gate.md

@@ -0,0 +1,23 @@
+# 38 — Bulk hydration + bench gate + Document.open
+
+Status: DONE (1.9.178.1)
+
+## Outcome
+
+- **#229**: `benchmark/native_vs_binding.rb` now accepts
+  `LEPTRIS_BENCH_LOAD_MAX`; default threshold is core-normalized
+  (`max(logical_cores * 1.5, 4.0)`) instead of absolute load >20.
+- **#230**: `Document#snapshot(node)` returns C-materialized flat
+  rows (`kind`, `name`, `prefix`, `uri`, flat `[name,value]` attrs,
+  text, depth), and `Document#walk_subtree(node) { |row| ... }`
+  yields the same rows. The native snapshot has no per-node Node
+  wrappers or per-attribute Ruby calls. The cursor currently yields
+  the safe snapshot rows; the direct Proc callback prototype was
+  rejected after crashing MRI and is not shipped.
+- **#231**: `Document.open(xml) { |doc| ... }` guarantees `doc.free`
+  on normal and exceptional block exit. The strong identity cache
+  remains the default deliberately: weak identity would break
+  `doc.root.equal?(doc.root)` while the document remains alive.
+
+Specs: 674/0 in both native and FFI modes. Downstream follow-ups
+posted to moxml#230/#231 and metanorma/uniword#49.

data/ext/build_windows_native.rb

@@ -2,15 +2,18 @@
 
 # Builds the native read layer DLL for the CURRENT Ruby and
 # installs it under its minor-versioned name
-# (lib/leptris/xml/native-<major.minor>.so).
+# (lib/leptris/xml/native_<major>_<minor>.so).
 #
 # #207/#227: a PE DLL cannot resolve Ruby imports lazily — it
 # must bind the build Ruby's x64-ucrt-rubyNNN.dll. One DLL per
 # supported minor therefore ships in the Windows platform gems,
 # and native_layer picks by RUBY_VERSION at require.
 #
-# Standalone by design: the release workflow runs this under each
-# Ruby minor (3.3/3.4/4.0) with no bundler context.
+# The name is DOT-FREE: MRI derives a C extension's init symbol
+# from the require feature's basename cut at the first dot, so
+# native_3_3.so resolves Init_native_3_3 (a valid C symbol,
+# exported by native.c) — a dotted native-3.3.so would make Ruby
+# look for "Init_native-3", which cannot exist.
 
 require "rbconfig"
 require "fileutils"
@@ -18,6 +21,7 @@ require "fileutils"
 root = File.expand_path("..", __dir__)
 ext_dir = File.join(root, "ext", "leptris", "native")
 minor = RUBY_VERSION[/\A\d+\.\d+/]
+minor_us = minor.tr(".", "_")
 
 Dir.chdir(ext_dir) do
   system(RbConfig.ruby, "extconf.rb") or abort "extconf failed under #{RUBY_VERSION}"
@@ -27,7 +31,7 @@ Dir.chdir(ext_dir) do
   abort "make failed under #{RUBY_VERSION}" unless success
   so = Dir.glob("native.{so,dll}").first
   abort "native bundle not produced under #{RUBY_VERSION}" unless so
-  dest = File.join(root, "lib", "leptris", "xml", "native-#{minor}.so")
+  dest = File.join(root, "lib", "leptris", "xml", "native_#{minor_us}.so")
   # The artifact must reference only this minor's Ruby DLL.
   imported = `strings #{so} 2>/dev/null`[/[a-z0-9-]*ruby\d{3,}\.dll/i]
   if imported && !imported.include?("ruby#{minor.delete('.')}")

data/ext/leptris/native/extconf.rb

@@ -34,4 +34,22 @@ if RUBY_PLATFORM =~ /darwin/
   RbConfig::MAKEFILE_CONFIG["LIBRUBYARG_SHARED"] = ""
   RbConfig::MAKEFILE_CONFIG["LIBRUBYARG_STATIC"] = ""
 end
+if Gem.win_platform?
+  # Windows names the artifact per Ruby minor, DOT-FREE (#227
+  # fix): MRI derives the init symbol from the basename cut at
+  # the first dot, so native_3_4.so needs Init_native_3_4
+  # exported. GNU ld auto-exports only when no explicit export
+  # is present — pin the export table with a .def file so the
+  # wrapper init names are certain to be exported.
+  def_path = File.expand_path("native_exports.def", __dir__)
+  File.write(def_path, <<~DEF)
+    EXPORTS
+    Init_native
+    Init_native_3_3
+    Init_native_3_4
+    Init_native_4_0
+  DEF
+  win_def = " #{def_path.tr('/', '\\\\')}"
+  $DLDFLAGS << win_def unless $DLDFLAGS.include?("native_exports.def")
+end
 create_makefile("leptris/xml/native")

data/ext/leptris/native/native.c

@@ -1716,6 +1716,123 @@ static VALUE nf_fast_element_xml(VALUE self, VALUE addr,
                           encoding);
 }
 
+/* ---- Bulk hydration walker (TODO.perf/38, #230) -----------------
+ * Pre-order walk that materializes one Hash per element node
+ * (kind, name, prefix, uri, attrs as Array<[name,value]>, text,
+ * depth, line) and yields it to a Ruby Proc (walk_subtree) or
+ * collects it into an Array (snapshot). No per-node Node wrapper
+ * construction — the consumer iterates rows and builds their
+ * typed objects directly. */
+/* Node kinds (mirror types.h). Documented in descriptor comment. */
+#define WS_NODE_ELEMENT 0
+#define WS_NODE_TEXT 1
+#define WS_NODE_COMMENT 2
+#define WS_NODE_CDATA 3
+#define WS_NODE_PI 4
+#define WS_NODE_DOCTYPE 5
+
+static VALUE build_element_row(VALUE doc, void *elem, int depth)
+{
+    VALUE row = rb_hash_new();
+    rb_hash_aset(row, ID2SYM(rb_intern("kind")),
+                 rb_utf8_str_new_cstr("element"));
+    const char *name = f_elem_name(elem);
+    rb_hash_aset(row, ID2SYM(rb_intern("name")),
+                 name ? rb_utf8_str_new_cstr(name) : Qnil);
+    rb_hash_aset(row, ID2SYM(rb_intern("prefix")),
+                 f_elem_prefix(elem) ? rb_utf8_str_new_cstr(f_elem_prefix(elem)) : Qnil);
+    rb_hash_aset(row, ID2SYM(rb_intern("uri")),
+                 f_elem_ns(elem) ? rb_utf8_str_new_cstr(f_elem_ns(elem)) : Qnil);
+    /* attrs: name/value pairs as Array<[name, value]>. */
+    VALUE attrs = rb_ary_new();
+    void *a = f_attr_first(elem);
+    while (a) {
+        VALUE pair = rb_ary_new_capa(2);
+        const char *an = f_attr_name(a);
+        const char *av = f_attr_value(elem, a);
+        rb_ary_push(pair, an ? rb_utf8_str_new_cstr(an) : Qnil);
+        rb_ary_push(pair, av ? rb_utf8_str_new_cstr(av) : Qnil);
+        rb_ary_push(attrs, pair);
+        a = f_attr_next(a);
+    }
+    rb_hash_aset(row, ID2SYM(rb_intern("attrs")), attrs);
+    /* text: first text-child content (moxml-style common case). */
+    void *c = f_first_child(elem);
+    VALUE text = Qnil;
+    while (c) {
+        if (f_node_type(c) == WS_NODE_TEXT) {
+            const char *t = f_text_content(c);
+            text = t ? rb_utf8_str_new_cstr(t) : Qnil;
+            break;
+        }
+        c = f_next_sibling(c);
+    }
+    rb_hash_aset(row, ID2SYM(rb_intern("text")), text);
+    rb_hash_aset(row, ID2SYM(rb_intern("depth")), INT2NUM(depth));
+    return row;
+}
+
+static VALUE build_text_row(int kind, int depth, const char *t)
+{
+    VALUE row = rb_hash_new();
+    rb_hash_aset(row, ID2SYM(rb_intern("kind")),
+                 rb_utf8_str_new_cstr(kind == WS_NODE_TEXT ? "text"
+                                    : kind == WS_NODE_COMMENT ? "comment"
+                                    : kind == WS_NODE_CDATA ? "cdata" : "other"));
+    rb_hash_aset(row, ID2SYM(rb_intern("text")),
+                 t ? rb_utf8_str_new_cstr(t) : Qnil);
+    rb_hash_aset(row, ID2SYM(rb_intern("depth")), INT2NUM(depth));
+    return row;
+}
+
+static void yield_or_collect_row(VALUE out_or_block, VALUE row, int collect)
+{
+    if (collect) {
+        rb_ary_push(out_or_block, row);
+    } else {
+        rb_funcall(out_or_block, rb_intern("call"), 1, row);
+    }
+}
+
+static void walk_subtree_impl(VALUE doc, void *node, int depth, VALUE out_or_block, int collect)
+{
+    int kind = f_node_type(node);
+    if (kind == WS_NODE_ELEMENT) {
+        void *c;
+        yield_or_collect_row(out_or_block, build_element_row(doc, node, depth), collect);
+        c = f_first_child(node);
+        while (c) {
+            walk_subtree_impl(doc, c, depth + 1, out_or_block, collect);
+            c = f_next_sibling(c);
+        }
+    } else if (kind == WS_NODE_DOCTYPE || kind == 9 /* DOCUMENT */) {
+        void *dc = f_first_child(node);
+        while (dc) {
+            walk_subtree_impl(doc, dc, depth, out_or_block, collect);
+            dc = f_next_sibling(dc);
+        }
+    } else if (kind == WS_NODE_TEXT || kind == WS_NODE_COMMENT ||
+               kind == WS_NODE_CDATA || kind == WS_NODE_PI) {
+        const char *t = NULL;
+        if (kind == WS_NODE_TEXT)
+            t = f_text_content(node);
+        else if (kind == WS_NODE_COMMENT)
+            t = f_comment_content(node);
+        else if (kind == WS_NODE_CDATA)
+            t = f_cdata_content(node);
+        else if (kind == WS_NODE_PI)
+            t = f_pi_data(node);
+        yield_or_collect_row(out_or_block, build_text_row(kind, depth, t), collect);
+    }
+}
+
+static VALUE nf_snapshot_subtree(VALUE self, VALUE document, VALUE addr)
+{
+    VALUE out = rb_ary_new();
+    walk_subtree_impl(document, (void *)(uintptr_t)NUM2ULL(addr), 0, out, 1);
+    return out;
+}
+
 /* ---- inner_html in one C pass (TODO.perf/18) --------------------
  * Serializes the receiver's children into one growable buffer:
  * elements via leptris_element_serialize_into (the same opts the
@@ -2253,6 +2370,8 @@ void Init_native(void)
                               nf_append_markup, 3);
     rb_define_module_function(m_native, "fast_inner_xml",
                               nf_fast_inner_xml, 1);
+    rb_define_module_function(m_native, "snapshot_subtree",
+                              nf_snapshot_subtree, 2);
     rb_define_module_function(m_native, "doc_handle_attach",
                               nf_doc_handle_attach, 1);
     rb_define_module_function(m_native, "doc_handle_release",
@@ -2277,3 +2396,15 @@ void Init_native(void)
     rb_define_module_function(m_native, "fast_element_xml",
                               nf_fast_element_xml, 4);
 }
+
+#ifdef _WIN32
+/* MRI derives a C extension's init symbol from the require
+ * feature's basename cut at the first dot. The Windows per-minor
+ * DLLs are therefore named dot-free (native_3_3.so etc.), and
+ * each needs its exact init name exported. All are defined here
+ * unconditionally: a DLL only ever loads under the Ruby minor it
+ * was built against (PE binds its build Ruby's runtime DLL). */
+void Init_native_3_3(void) { Init_native(); }
+void Init_native_3_4(void) { Init_native(); }
+void Init_native_4_0(void) { Init_native(); }
+#endif

data/lib/leptris/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Leptris
-  VERSION = "1.9.177.0"
+  VERSION = "1.9.178.1"
 end

data/lib/leptris/xml/descriptor.rb

@@ -150,6 +150,18 @@ class Leptris::XML::Descriptor
         cp[:kind] = kind_code(row[:kind] || :scalar)
         cp[:type_tag] = row[:type_tag] || 0
         cp[:child_plan_index] = row[:child_plan_index] || -1
+        # Rule-level ns form (libleptris 1.9.178, #1115): siblings
+        # under one parent can require different URIs when set;
+        # defaults to NONE for backward compatibility.
+        cp[:ns_form] =
+          case row[:ns]
+          when :any then Leptris::XML::FFI::PLAN_NS_ANY
+          when Hash  then Leptris::XML::FFI::PLAN_NS_EXACT
+          else         Leptris::XML::FFI::PLAN_NS_NONE
+          end
+        cp[:ns_uri] =
+          row[:ns].is_a?(Hash) ?
+            anchor_string(anchors, row[:ns].fetch(:exact)) : nil
       end
       ep[:child_count] = children.size
       ep[:child_plans] = child_memory

data/lib/leptris/xml/document.rb

@@ -73,6 +73,60 @@ class Leptris::XML::Document
     @wrapper_cache ||= {}
   end
 
+  # #231: deterministic scope — yields the document, guarantees
+  # teardown at block exit (even on raise). Frameworks with their
+  # own finalizer timing worries can wrap every parse in this and
+  # skip the GC-via-finalizer slot entirely.
+  def self.open(xml_or_io, **kwargs)
+    doc = parse(xml_or_io, **kwargs)
+    return doc unless block_given?
+    begin
+      yield doc
+    ensure
+      doc.free
+    end
+  end
+
+  # #230: bulk hydration walks — one C pass per subtree, no per-node
+  # Ruby wrapper construction. snapshot returns an Array of rows
+  # (consumer iterates and builds typed objects in one pass);
+  # walk_subtree yields rows through a block (lower peak memory for
+  # very large subtrees).
+  def snapshot(node)
+    if defined?(Leptris::XML::NATIVE_FAST)
+      Leptris::XML::Native.snapshot_subtree(
+        self, node.respond_to?(:c_address) ? node.c_address : nil)
+    else
+      rows = []
+      walk_subtree(node) { |row| rows << row }
+      rows
+    end
+  end
+
+  def walk_subtree(node, &block)
+    return enum_for(:walk_subtree, node) unless block
+    if defined?(Leptris::XML::NATIVE_FAST)
+      # The public cursor yields the same C-materialized rows as
+      # #snapshot. The C cursor prototype was deliberately not
+      # exposed: a Proc-calling recursive C walk crashed under
+      # MRI; snapshot is the safe one-pass bulk surface.
+      snapshot(node).each(&block)
+      return
+    end
+    # FFI fallback: recursive walk (the historical per-node shape).
+    if node.is_a?(Leptris::XML::Node)
+      yield(element_row(node)) if node.element?
+      node.element_children.each { |c| walk_subtree(c, &block) }
+    end
+  end
+
+  def element_row(el)
+    { kind: "element", name: el.name, prefix: el.namespace&.prefix,
+      uri: el.namespace&.href, attrs: el.attributes,
+      text: el.element_children.find { |c| c.text? }&.content,
+      depth: 0 }
+  end
+
   def self.parse(xml_or_io, options: nil, readonly: false, recover: false)
     xml = xml_or_io.is_a?(String) ? xml_or_io : xml_or_io.read
     if xml.empty?

data/lib/leptris/xml/ffi.rb

@@ -209,7 +209,12 @@ module Leptris
         layout :wire_name, :pointer,
                :kind, :uint8,
                :type_tag, :uint8,
-               :child_plan_index, :int32
+               :child_plan_index, :int32,
+               # libleptris 1.9.178 (#1114/#1115): rule-level ns
+               # forms on ChildPlan (additive to the frozen v1 ABI).
+               :ns_form, :uint8,
+               :pad0, :uint8,
+               :ns_uri, :pointer
       end
       class ElementPlan < ::FFI::Struct
         layout :element_name, :pointer,

data/lib/leptris/xml/native_layer.rb

@@ -20,11 +20,12 @@ begin
   # The compiled bundle. A sanctioned require exception: a
   # .bundle/.so cannot go through autoload. Missing in installs
   # that do not vendor it (ruby-platform gem). Windows names the
-  # artifact per Ruby minor (#207/#227): a PE DLL must bind its
-  # build Ruby's runtime, so one DLL per supported minor ships
-  # and RUBY_VERSION selects.
+  # artifact per Ruby minor, DOT-FREE (#207/#227): a PE DLL must
+  # bind its build Ruby's runtime (one DLL per minor), and MRI
+  # derives the init symbol from the basename cut at the first
+  # dot — native_3_3.so resolves Init_native_3_3.
   if Gem.win_platform?
-    require "leptris/xml/native-#{RUBY_VERSION[/\A\d+\.\d+/]}"
+    require "leptris/xml/native_#{RUBY_VERSION[/\A\d+\.\d+/].tr('.', '_')}"
   else
     require "leptris/xml/native"
   end

data/lib/leptris.rb

@@ -48,6 +48,6 @@ unless ENV["LEPTRIS_NO_NATIVE"] == "1"
     # the FFI surface (the explicit require still raises its full
     # error for callers that opt in).
     warn "leptris: native acceleration unavailable " \
-         "(#{e.message.lines.first.strip}); using the FFI surface"
+         "(#{e.message.gsub(/\s+/, ' ').strip[0, 200]}); using the FFI surface"
   end
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: leptris
 version: !ruby/object:Gem::Version
-  version: 1.9.177.0
+  version: 1.9.178.1
 platform: ruby
 authors:
 - Ribose Inc.
@@ -106,6 +106,7 @@ files:
 - TODO.perf/35-parse-default-c-face.md
 - TODO.perf/36-raise-in-c-serializer-encoding.md
 - TODO.perf/37-document-lazy-pointer.md
+- TODO.perf/38-bulk-walk-and-bench-gate.md
 - TODO.restructure/01-constraint-compliance-audit.md
 - TODO.restructure/02-deep-copy-seam.md
 - TODO.restructure/03-evaluation-context-seam.md