leptris 1.9.21-aarch64-linux → 1.9.24-aarch64-linux
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +68 -0
- data/Rakefile +1 -1
- data/lib/leptris/version.rb +1 -1
- data/lib/leptris/xml/ffi.rb +54 -0
- data/lib/leptris/xml/iterparse.rb +89 -14
- data/lib/leptris/xml/sax/parser.rb +60 -24
- data/lib/leptris/xml/sax/recorder.rb +179 -0
- data/lib/leptris/xml/sax.rb +1 -0
- data/lib/libleptris.so +0 -0
- metadata +2 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1be15b3c37fddd0b1f5c30c79b08adae540825a34605ffa2d44f8c9800dc1cd5
|
|
4
|
+
data.tar.gz: e008a194237148e03af56e37032cd3e3e30c1762680d3013b5cb3053af4a7c21
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: '04893f33c1d19df655b5ae1e5aeb15cd753988e1f7f7872f5984fa6a011863c59c860919186d7f688dbc200e3d2cc76c5266917ff0293f7ad125d8d1913daa6e'
|
|
7
|
+
data.tar.gz: 4698e55f88198a6058302c6df78062c57771e809b49b99fbdee63e21fe7b348f0bf707650e9690b46e6c975b474d1daa51ba5782dd3074655c62be67f24df621
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,74 @@ All notable changes to Leptris will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [1.9.24] - 2026-08-27
|
|
9
|
+
|
|
10
|
+
### Changed
|
|
11
|
+
|
|
12
|
+
- **Interest-proportional SAX (both transports)**: the handler's
|
|
13
|
+
declared interest — which event methods it overrides — now decides
|
|
14
|
+
what the transport dispatches or drains.
|
|
15
|
+
- `SAX::Parser` attaches a callback only when the handler defines
|
|
16
|
+
the method beyond `SAX::Document`'s no-ops; the C engine already
|
|
17
|
+
skips NULL callbacks, so unwanted events cost nothing on either
|
|
18
|
+
side. Measured per parse of a 1.9 MB / 250k-event document:
|
|
19
|
+
text-only handler 151 ms -> 22 ms (**6.9x**); a handler that
|
|
20
|
+
overrides nothing runs at the C floor, 151 ms -> 7 ms
|
|
21
|
+
(**22x**); elements-only 151 ms -> 121 ms (-20%, the remainder
|
|
22
|
+
is attribute walking). Handlers overriding most kinds stay at
|
|
23
|
+
parity. Duck-typed handlers attach exactly what they define.
|
|
24
|
+
- `SAX::Recorder#each_event(*kinds)` (and `Recorder.parse(...,
|
|
25
|
+
kinds:)`, `#feed_stream(..., kinds:)`) skips records of other
|
|
26
|
+
kinds BEFORE slicing any strings from the arena — an unwanted
|
|
27
|
+
event costs one Array read. Filtered `:characters` drain 201 ms
|
|
28
|
+
-> 119 ms (**-41%**); unknown kinds raise ArgumentError. The
|
|
29
|
+
kind lookup table is now an Array indexed by the event code.
|
|
30
|
+
- Transport choice, by measurement: the pruned callback transport
|
|
31
|
+
beats the filtered recorder for selective consumers (C-side
|
|
32
|
+
emission is skipped entirely), so callbacks remain the default;
|
|
33
|
+
the recorder keeps its niche — bulk raw-event drains per chunk.
|
|
34
|
+
|
|
35
|
+
## [1.9.23] - 2026-08-27
|
|
36
|
+
|
|
37
|
+
### Added
|
|
38
|
+
|
|
39
|
+
- **libleptris 1.9.4 lockstep** (leptris/leptris#585, #586): Rakefile
|
|
40
|
+
pin bumped; symbol audit 242/242 attached-vs-exported with the ten
|
|
41
|
+
new functions.
|
|
42
|
+
- **`SAX::Recorder`**: public chunked-event API over the C recorder —
|
|
43
|
+
`Recorder.parse(xml_or_io)` / `Recorder#feed` / `#each_event`
|
|
44
|
+
yield `kind, name, text, attrs, line, column` with UTF-8 strings,
|
|
45
|
+
drained as one bulk read per chunk (records via `read_bytes` +
|
|
46
|
+
`unpack` against the packed string arena — no per-event FFI
|
|
47
|
+
struct). NOT wired into `SAX::Parser#parse_memory`: measured
|
|
48
|
+
parity-to-−30% vs the callback path (per-parse recorder setup
|
|
49
|
+
scales with input size; findings and measurements filed upstream
|
|
50
|
+
as leptris/leptris#594). It remains the right shape for hosts
|
|
51
|
+
that drain bulk events per chunk.
|
|
52
|
+
- **Iterparse v2** (#586): `Iterparse.parse` / `.parse_file` take
|
|
53
|
+
`mode:` — `:top_level` (v1: the root's completed children) or
|
|
54
|
+
`:full_document` (every element, post-order, child before parent).
|
|
55
|
+
`#namespace_uri(prefix)` / `#namespace_count` resolve prefixes on
|
|
56
|
+
the last yielded element's in-scope snapshot (call inside the run
|
|
57
|
+
block); `#error` reports truncated/malformed input.
|
|
58
|
+
|
|
59
|
+
### Changed
|
|
60
|
+
|
|
61
|
+
- **Iterparse lifecycle**: `#run` no longer frees in `ensure` — the
|
|
62
|
+
block form of `.parse`/`.parse_file` frees on return (Pull's
|
|
63
|
+
pattern), the no-block form returns a live iterator for explicit
|
|
64
|
+
`#free`. This fixes the `enum_for` path, which previously freed
|
|
65
|
+
the handle before the enumerator drove it, and lets `#error`
|
|
66
|
+
stay readable after the run (`#free` snapshots the terminal
|
|
67
|
+
message). Full-document mode measures +36% CPU over top-level —
|
|
68
|
+
proportional to yielding every element instead of root children.
|
|
69
|
+
|
|
70
|
+
### Meta
|
|
71
|
+
|
|
72
|
+
- Head-to-head vs published 1.9.21 (engine 1.9.0 vs 1.9.4): SAX
|
|
73
|
+
callback parsing and iterparse at parity across two passes; no
|
|
74
|
+
regression from the engine bump.
|
|
75
|
+
|
|
8
76
|
## [1.9.21] - 2026-08-27
|
|
9
77
|
|
|
10
78
|
### Changed
|
data/Rakefile
CHANGED
|
@@ -8,7 +8,7 @@ RSpec::Core::RakeTask.new(:spec)
|
|
|
8
8
|
# Pin for `rake compile` and the platform-gem builds. Keep in lockstep
|
|
9
9
|
# with .github/workflows/build.yml (which calls `rake compile`) and the
|
|
10
10
|
# CHANGELOG when libleptris releases.
|
|
11
|
-
LIBLEPTRIS_VERSION = "1.9.
|
|
11
|
+
LIBLEPTRIS_VERSION = "1.9.4"
|
|
12
12
|
|
|
13
13
|
CMAKE_FLAGS = %w[
|
|
14
14
|
-DCMAKE_BUILD_TYPE=Release
|
data/lib/leptris/version.rb
CHANGED
data/lib/leptris/xml/ffi.rb
CHANGED
|
@@ -43,6 +43,7 @@ module Leptris
|
|
|
43
43
|
typedef :pointer, :leptris_xpath_ns_set
|
|
44
44
|
typedef :pointer, :leptris_xpath_compiled
|
|
45
45
|
typedef :pointer, :leptris_xslt
|
|
46
|
+
typedef :pointer, :leptris_sax_recorder
|
|
46
47
|
typedef :pointer, :leptris_pull_parser
|
|
47
48
|
typedef :pointer, :leptris_iterparse
|
|
48
49
|
typedef :pointer, :leptris_xpath_var_set
|
|
@@ -81,6 +82,33 @@ module Leptris
|
|
|
81
82
|
:recover, :int
|
|
82
83
|
end
|
|
83
84
|
|
|
85
|
+
# LeptrisSaxEventRecord (types.h, issue #585): fixed layout the
|
|
86
|
+
# header explicitly freezes for FFI mirrors. Strings are NOT
|
|
87
|
+
# inline — name/text/attrs slice the recorder's arena.
|
|
88
|
+
class SaxEventRecord < ::FFI::Struct
|
|
89
|
+
layout :kind, :uint8,
|
|
90
|
+
:reserved, [:uint8, 7],
|
|
91
|
+
:name_off, :uint32, :name_len, :uint32,
|
|
92
|
+
:text_off, :uint32, :text_len, :uint32,
|
|
93
|
+
:attrs_off, :uint32, :attr_count, :uint32,
|
|
94
|
+
:line, :uint32, :column, :uint32
|
|
95
|
+
end
|
|
96
|
+
|
|
97
|
+
SAX_EVENT_START_DOCUMENT = 0
|
|
98
|
+
SAX_EVENT_END_DOCUMENT = 1
|
|
99
|
+
SAX_EVENT_START_ELEMENT = 2
|
|
100
|
+
SAX_EVENT_END_ELEMENT = 3
|
|
101
|
+
SAX_EVENT_CHARACTERS = 4
|
|
102
|
+
SAX_EVENT_COMMENT = 5
|
|
103
|
+
SAX_EVENT_CDATA = 6
|
|
104
|
+
SAX_EVENT_PI = 7
|
|
105
|
+
SAX_EVENT_START_PREFIX = 8
|
|
106
|
+
SAX_EVENT_END_PREFIX = 9
|
|
107
|
+
SAX_EVENT_ERROR = 10
|
|
108
|
+
|
|
109
|
+
ITERPARSE_TOP_LEVEL = 0
|
|
110
|
+
ITERPARSE_FULL_DOCUMENT = 1
|
|
111
|
+
|
|
84
112
|
# struct LeptrisPullEvent (leptris/sax.h): FFI::Struct derives
|
|
85
113
|
# the field offsets from the layout, so ABI changes are a
|
|
86
114
|
# one-line edit here instead of silent offset drift in callers.
|
|
@@ -499,6 +527,19 @@ module Leptris
|
|
|
499
527
|
[:pointer, :pointer], :leptris_sax_parser
|
|
500
528
|
attach_function :leptris_sax_parser_feed,
|
|
501
529
|
[:leptris_sax_parser, :string, :size_t, :int], :int
|
|
530
|
+
# Chunked event recorder (libleptris 1.9.4, #585): buffers
|
|
531
|
+
# events C-side (fixed records + packed arena); the host
|
|
532
|
+
# drains in bulk — callbacks become O(chunks), not O(events).
|
|
533
|
+
attach_function :leptris_sax_recorder_new,
|
|
534
|
+
[], :leptris_sax_recorder
|
|
535
|
+
attach_function :leptris_sax_recorder_feed,
|
|
536
|
+
[:leptris_sax_recorder, :string, :size_t, :int], :int
|
|
537
|
+
attach_function :leptris_sax_recorder_records,
|
|
538
|
+
[:leptris_sax_recorder, :pointer], :pointer
|
|
539
|
+
attach_function :leptris_sax_recorder_arena,
|
|
540
|
+
[:leptris_sax_recorder, :pointer], :pointer
|
|
541
|
+
attach_function :leptris_sax_recorder_free,
|
|
542
|
+
[:leptris_sax_recorder], :void
|
|
502
543
|
attach_function :leptris_sax_parser_free,
|
|
503
544
|
[:leptris_sax_parser], :void
|
|
504
545
|
attach_function :leptris_sax_parser_set_streaming,
|
|
@@ -528,6 +569,19 @@ module Leptris
|
|
|
528
569
|
[:string], :leptris_iterparse
|
|
529
570
|
attach_function :leptris_iterparse_next,
|
|
530
571
|
[:leptris_iterparse], :leptris_element
|
|
572
|
+
# iterparse v2 (libleptris 1.9.4, #586): explicit yield mode
|
|
573
|
+
# (top-level children or full-document post-order), namespace
|
|
574
|
+
# resolution on the last yielded element, error channel.
|
|
575
|
+
attach_function :leptris_iterparse_new_ex,
|
|
576
|
+
[:string, :size_t, :int], :leptris_iterparse
|
|
577
|
+
attach_function :leptris_iterparse_new_file_ex,
|
|
578
|
+
[:string, :int], :leptris_iterparse
|
|
579
|
+
attach_function :leptris_iterparse_ns_count,
|
|
580
|
+
[:leptris_iterparse], :size_t
|
|
581
|
+
attach_function :leptris_iterparse_ns_uri,
|
|
582
|
+
[:leptris_iterparse, :string], :string
|
|
583
|
+
attach_function :leptris_iterparse_error,
|
|
584
|
+
[:leptris_iterparse], :string
|
|
531
585
|
attach_function :leptris_iterparse_free,
|
|
532
586
|
[:leptris_iterparse], :void
|
|
533
587
|
|
|
@@ -12,34 +12,78 @@ require "ffi"
|
|
|
12
12
|
# process(element) # subtree valid until the next iteration
|
|
13
13
|
# end
|
|
14
14
|
#
|
|
15
|
-
#
|
|
16
|
-
#
|
|
17
|
-
#
|
|
18
|
-
#
|
|
19
|
-
#
|
|
15
|
+
# v2 (libleptris 1.9.4, #586): an explicit yield mode —
|
|
16
|
+
# :top_level (v1: the root's completed children) or
|
|
17
|
+
# :full_document (every element in document-order completion —
|
|
18
|
+
# post-order, child before parent, subtree attached) — plus
|
|
19
|
+
# namespace resolution on the last yielded element and an error
|
|
20
|
+
# channel for truncated input:
|
|
21
|
+
#
|
|
22
|
+
# it = Leptris::XML::Iterparse.parse(xml, mode: :full_document)
|
|
23
|
+
# it.run do |el|
|
|
24
|
+
# it.namespace_uri("p") # in scope on el, the last yielded
|
|
25
|
+
# end
|
|
26
|
+
# it.free
|
|
27
|
+
#
|
|
28
|
+
# Leptris::XML::Iterparse.parse(xml) { |el| ... }.error
|
|
29
|
+
# # => nil, or the parse failure (readable after the run: the
|
|
30
|
+
# # block form frees the iterator on return, snapshotting it)
|
|
31
|
+
#
|
|
32
|
+
# The yielded elements have no parent Document; #document returns
|
|
33
|
+
# nil and they must not outlive the iteration.
|
|
20
34
|
class Leptris::XML::Iterparse
|
|
35
|
+
MODES = {
|
|
36
|
+
top_level: Leptris::XML::FFI::ITERPARSE_TOP_LEVEL,
|
|
37
|
+
full_document: Leptris::XML::FFI::ITERPARSE_FULL_DOCUMENT,
|
|
38
|
+
}.freeze
|
|
39
|
+
|
|
21
40
|
# NOTE: an IO argument is read fully into memory before the C
|
|
22
41
|
# iterator starts — the bounded-memory path is parse_file (C-side
|
|
23
42
|
# file streaming). The C API takes one (xml, len) buffer.
|
|
24
|
-
def self.parse(xml_or_io, &block)
|
|
43
|
+
def self.parse(xml_or_io, mode: :top_level, &block)
|
|
25
44
|
xml = xml_or_io.respond_to?(:read) ? xml_or_io.read : xml_or_io.to_s
|
|
26
|
-
iterator = new(Leptris::XML::FFI.
|
|
27
|
-
|
|
45
|
+
iterator = new(Leptris::XML::FFI.leptris_iterparse_new_ex(
|
|
46
|
+
xml, xml.bytesize, mode_code(mode)))
|
|
47
|
+
return iterator unless block
|
|
48
|
+
begin
|
|
49
|
+
iterator.run(&block)
|
|
50
|
+
ensure
|
|
51
|
+
iterator.free
|
|
52
|
+
end
|
|
53
|
+
iterator
|
|
28
54
|
end
|
|
29
55
|
|
|
30
|
-
def self.parse_file(path, &block)
|
|
31
|
-
iterator = new(Leptris::XML::FFI.
|
|
32
|
-
|
|
56
|
+
def self.parse_file(path, mode: :top_level, &block)
|
|
57
|
+
iterator = new(Leptris::XML::FFI.leptris_iterparse_new_file_ex(
|
|
58
|
+
path, mode_code(mode)))
|
|
59
|
+
return iterator unless block
|
|
60
|
+
begin
|
|
61
|
+
iterator.run(&block)
|
|
62
|
+
ensure
|
|
63
|
+
iterator.free
|
|
64
|
+
end
|
|
65
|
+
iterator
|
|
66
|
+
end
|
|
67
|
+
|
|
68
|
+
def self.mode_code(mode)
|
|
69
|
+
MODES.fetch(mode) do
|
|
70
|
+
raise ArgumentError,
|
|
71
|
+
"mode must be :top_level or :full_document, got #{mode.inspect}"
|
|
72
|
+
end
|
|
33
73
|
end
|
|
74
|
+
private_class_method :mode_code
|
|
34
75
|
|
|
35
76
|
def initialize(handle)
|
|
36
77
|
raise Leptris::XML::ParseError, "leptris_iterparse_new failed" if handle.null?
|
|
37
78
|
@handle = handle
|
|
38
79
|
end
|
|
39
80
|
|
|
81
|
+
# Yields completed elements until the document is exhausted or the
|
|
82
|
+
# parse fails (see #error). Does not free the iterator — the block
|
|
83
|
+
# form of .parse/.parse_file does, or call #free explicitly.
|
|
40
84
|
def run
|
|
41
85
|
return enum_for(:run) unless block_given?
|
|
42
|
-
|
|
86
|
+
while @handle
|
|
43
87
|
ptr = Leptris::XML::FFI.leptris_iterparse_next(@handle)
|
|
44
88
|
break if ptr.null?
|
|
45
89
|
# document: nil — the subtree belongs to the iterator's pool,
|
|
@@ -48,13 +92,44 @@ class Leptris::XML::Iterparse
|
|
|
48
92
|
yield Leptris::XML::Node.wrap(ptr, nil)
|
|
49
93
|
end
|
|
50
94
|
self
|
|
51
|
-
ensure
|
|
52
|
-
free
|
|
53
95
|
end
|
|
54
96
|
|
|
97
|
+
# The URI bound to +prefix+ ("" for the default namespace) in the
|
|
98
|
+
# last yielded element's in-scope namespace snapshot, or nil when
|
|
99
|
+
# unbound (v2, #586). Valid only while the iterator is live —
|
|
100
|
+
# call it inside the #run block; nil after #free.
|
|
101
|
+
def namespace_uri(prefix)
|
|
102
|
+
return nil if @handle.nil?
|
|
103
|
+
Leptris::XML::FFI.leptris_iterparse_ns_uri(@handle, prefix.to_s)
|
|
104
|
+
end
|
|
105
|
+
|
|
106
|
+
# Number of namespace bindings in the last yielded element's
|
|
107
|
+
# scope. Valid only while the iterator is live; 0 after #free.
|
|
108
|
+
def namespace_count
|
|
109
|
+
return 0 if @handle.nil?
|
|
110
|
+
Leptris::XML::FFI.leptris_iterparse_ns_count(@handle)
|
|
111
|
+
end
|
|
112
|
+
|
|
113
|
+
# The parse error for malformed/truncated input, or nil when the
|
|
114
|
+
# document is well-formed and exhausted. Yielded elements stop at
|
|
115
|
+
# the failure point; #free snapshots the message so this stays
|
|
116
|
+
# readable after the block form of .parse returns.
|
|
117
|
+
def error
|
|
118
|
+
@handle ? live_error : @last_error
|
|
119
|
+
end
|
|
120
|
+
|
|
121
|
+
# Releases the C iterator. Safe to call twice.
|
|
55
122
|
def free
|
|
56
123
|
return if @handle.nil?
|
|
124
|
+
@last_error = live_error
|
|
57
125
|
Leptris::XML::FFI.leptris_iterparse_free(@handle)
|
|
58
126
|
@handle = nil
|
|
59
127
|
end
|
|
128
|
+
|
|
129
|
+
private
|
|
130
|
+
|
|
131
|
+
def live_error
|
|
132
|
+
msg = Leptris::XML::FFI.leptris_iterparse_error(@handle)
|
|
133
|
+
msg.nil? || msg.empty? ? nil : msg
|
|
134
|
+
end
|
|
60
135
|
end
|
|
@@ -83,59 +83,95 @@ class Leptris::XML::SAX::Parser
|
|
|
83
83
|
private
|
|
84
84
|
|
|
85
85
|
# Build a LeptrisSAXHandler struct populated with FFI::Function callbacks
|
|
86
|
-
# that dispatch to the Ruby handler.
|
|
86
|
+
# that dispatch to the Ruby handler. Only the callbacks the handler
|
|
87
|
+
# actually overrides are attached: the C engine skips NULL callbacks,
|
|
88
|
+
# so cost scales with the handler's declared interest, not with the
|
|
89
|
+
# document's event mix (measured 4.9x for an elements-only handler on
|
|
90
|
+
# a 1.9 MB stream).
|
|
87
91
|
def build_handler_struct
|
|
88
92
|
s = Leptris::XML::FFI::SAXHandler.new
|
|
89
93
|
handler = @document # capture in closures
|
|
90
94
|
|
|
91
|
-
|
|
92
|
-
|
|
95
|
+
if overridden?(:start_document)
|
|
96
|
+
s[:start_document] = callback(:void, [:pointer]) do |_|
|
|
97
|
+
handler.start_document
|
|
98
|
+
end
|
|
93
99
|
end
|
|
94
100
|
|
|
95
|
-
|
|
96
|
-
|
|
101
|
+
if overridden?(:end_document)
|
|
102
|
+
s[:end_document] = callback(:void, [:pointer]) do |_|
|
|
103
|
+
handler.end_document
|
|
104
|
+
end
|
|
97
105
|
end
|
|
98
106
|
|
|
99
|
-
|
|
100
|
-
|
|
101
|
-
|
|
107
|
+
if overridden?(:start_element)
|
|
108
|
+
s[:start_element] = callback(:void, [:pointer, :string, :pointer]) do |_, name, attrs_ptr|
|
|
109
|
+
attrs = walk_attr_array(attrs_ptr)
|
|
110
|
+
handler.start_element(utf8(name), attrs)
|
|
111
|
+
end
|
|
102
112
|
end
|
|
103
113
|
|
|
104
|
-
|
|
105
|
-
|
|
114
|
+
if overridden?(:end_element)
|
|
115
|
+
s[:end_element] = callback(:void, [:pointer, :string]) do |_, name|
|
|
116
|
+
handler.end_element(utf8(name))
|
|
117
|
+
end
|
|
106
118
|
end
|
|
107
119
|
|
|
108
|
-
|
|
109
|
-
|
|
120
|
+
if overridden?(:characters)
|
|
121
|
+
s[:characters] = callback(:void, [:pointer, :pointer, :size_t]) do |_, text_ptr, len|
|
|
122
|
+
handler.characters(text_ptr.read_bytes(len).force_encoding(Encoding::UTF_8))
|
|
123
|
+
end
|
|
110
124
|
end
|
|
111
125
|
|
|
112
|
-
|
|
113
|
-
|
|
126
|
+
if overridden?(:comment)
|
|
127
|
+
s[:comment] = callback(:void, [:pointer, :string]) do |_, comment|
|
|
128
|
+
handler.comment(utf8(comment))
|
|
129
|
+
end
|
|
114
130
|
end
|
|
115
131
|
|
|
116
|
-
|
|
117
|
-
|
|
132
|
+
if overridden?(:cdata_block)
|
|
133
|
+
s[:cdata] = callback(:void, [:pointer, :string]) do |_, cdata|
|
|
134
|
+
handler.cdata_block(utf8(cdata))
|
|
135
|
+
end
|
|
118
136
|
end
|
|
119
137
|
|
|
120
|
-
|
|
121
|
-
|
|
138
|
+
if overridden?(:processing_instruction)
|
|
139
|
+
s[:processing_instruction] = callback(:void, [:pointer, :string, :string]) do |_, target, data|
|
|
140
|
+
handler.processing_instruction(utf8(target), utf8(data))
|
|
141
|
+
end
|
|
122
142
|
end
|
|
123
143
|
|
|
124
|
-
|
|
125
|
-
|
|
144
|
+
if overridden?(:start_prefix_mapping)
|
|
145
|
+
s[:start_prefix_mapping] = callback(:void, [:pointer, :string, :string]) do |_, prefix, uri|
|
|
146
|
+
handler.start_prefix_mapping(utf8(prefix), utf8(uri))
|
|
147
|
+
end
|
|
126
148
|
end
|
|
127
149
|
|
|
128
|
-
|
|
129
|
-
|
|
150
|
+
if overridden?(:end_prefix_mapping)
|
|
151
|
+
s[:end_prefix_mapping] = callback(:void, [:pointer, :string]) do |_, prefix|
|
|
152
|
+
handler.end_prefix_mapping(utf8(prefix))
|
|
153
|
+
end
|
|
130
154
|
end
|
|
131
155
|
|
|
132
|
-
|
|
133
|
-
|
|
156
|
+
if overridden?(:error)
|
|
157
|
+
s[:error] = callback(:void, [:pointer, :string, :int, :int]) do |_, msg, line, col|
|
|
158
|
+
handler.error(utf8(msg), line, col)
|
|
159
|
+
end
|
|
134
160
|
end
|
|
135
161
|
|
|
136
162
|
s
|
|
137
163
|
end
|
|
138
164
|
|
|
165
|
+
# True when the handler defines the event method beyond the no-op
|
|
166
|
+
# every SAX::Document carries — subclasses, included modules, and
|
|
167
|
+
# duck-typed handlers all qualify; a handler that overrides nothing
|
|
168
|
+
# attaches nothing and the parse runs at the C floor.
|
|
169
|
+
def overridden?(name)
|
|
170
|
+
@document.method(name).owner != Leptris::XML::SAX::Document
|
|
171
|
+
rescue NameError
|
|
172
|
+
false
|
|
173
|
+
end
|
|
174
|
+
|
|
139
175
|
def callback(return_type, params, blocking: true, &block)
|
|
140
176
|
::FFI::Function.new(return_type, params, blocking: blocking, &block)
|
|
141
177
|
end
|
|
@@ -0,0 +1,179 @@
|
|
|
1
|
+
# frozen_string_literal: true
|
|
2
|
+
|
|
3
|
+
require "ffi"
|
|
4
|
+
|
|
5
|
+
# Chunked event recorder (libleptris 1.9.4, issue #585): a SAX
|
|
6
|
+
# transport that buffers events C-side — fixed LeptrisSaxEventRecord
|
|
7
|
+
# entries plus a packed string arena — and drains them in one bulk
|
|
8
|
+
# read per fed chunk. The handler receives the identical calls the
|
|
9
|
+
# callback API makes; only the transport changes, and the callback
|
|
10
|
+
# count becomes O(chunks), not O(events): through FFI, per-event
|
|
11
|
+
# C-to-Ruby dispatch cost more than the parse itself.
|
|
12
|
+
#
|
|
13
|
+
# Leptris::XML::SAX::Recorder.parse(xml) do |kind, name, text, attrs|
|
|
14
|
+
# # kind: :start_element, :characters, ... (see KINDS)
|
|
15
|
+
# end
|
|
16
|
+
#
|
|
17
|
+
class Leptris::XML::SAX::Recorder
|
|
18
|
+
KINDS = {
|
|
19
|
+
Leptris::XML::FFI::SAX_EVENT_START_DOCUMENT => :start_document,
|
|
20
|
+
Leptris::XML::FFI::SAX_EVENT_END_DOCUMENT => :end_document,
|
|
21
|
+
Leptris::XML::FFI::SAX_EVENT_START_ELEMENT => :start_element,
|
|
22
|
+
Leptris::XML::FFI::SAX_EVENT_END_ELEMENT => :end_element,
|
|
23
|
+
Leptris::XML::FFI::SAX_EVENT_CHARACTERS => :characters,
|
|
24
|
+
Leptris::XML::FFI::SAX_EVENT_COMMENT => :comment,
|
|
25
|
+
Leptris::XML::FFI::SAX_EVENT_CDATA => :cdata,
|
|
26
|
+
Leptris::XML::FFI::SAX_EVENT_PI => :pi,
|
|
27
|
+
Leptris::XML::FFI::SAX_EVENT_START_PREFIX => :start_prefix,
|
|
28
|
+
Leptris::XML::FFI::SAX_EVENT_END_PREFIX => :end_prefix,
|
|
29
|
+
Leptris::XML::FFI::SAX_EVENT_ERROR => :error,
|
|
30
|
+
}.freeze
|
|
31
|
+
|
|
32
|
+
# Kind symbol per code byte — Array indexing in the drain loop,
|
|
33
|
+
# not a Hash lookup per event.
|
|
34
|
+
KIND_BY_CODE = Array.new(KINDS.size) { |code| KINDS[code] }.freeze
|
|
35
|
+
|
|
36
|
+
def self.open
|
|
37
|
+
raw = Leptris::XML::FFI.leptris_sax_recorder_new
|
|
38
|
+
if raw.null?
|
|
39
|
+
raise Leptris::XML::Error, "leptris_sax_recorder_new failed"
|
|
40
|
+
end
|
|
41
|
+
new(raw)
|
|
42
|
+
end
|
|
43
|
+
|
|
44
|
+
# One-shot parse over a complete document, yielding Record structs.
|
|
45
|
+
# +kinds+ filters the drain (see #each_event) — pass e.g.
|
|
46
|
+
# `kinds: [:start_element]` to slice strings only for the events
|
|
47
|
+
# you consume.
|
|
48
|
+
def self.parse(xml_or_io, kinds: nil)
|
|
49
|
+
recorder = open
|
|
50
|
+
begin
|
|
51
|
+
if xml_or_io.respond_to?(:read)
|
|
52
|
+
recorder.feed_stream(xml_or_io, kinds: kinds) { |*args| yield(*args) }
|
|
53
|
+
else
|
|
54
|
+
recorder.feed(xml_or_io.to_s, final: true)
|
|
55
|
+
recorder.each_event(*Array(kinds)) { |*args| yield(*args) }
|
|
56
|
+
end
|
|
57
|
+
ensure
|
|
58
|
+
recorder.free
|
|
59
|
+
end
|
|
60
|
+
end
|
|
61
|
+
|
|
62
|
+
def initialize(handle)
|
|
63
|
+
@handle = handle
|
|
64
|
+
end
|
|
65
|
+
|
|
66
|
+
def free
|
|
67
|
+
return if @handle.nil?
|
|
68
|
+
Leptris::XML::FFI.leptris_sax_recorder_free(@handle)
|
|
69
|
+
@handle = nil
|
|
70
|
+
end
|
|
71
|
+
|
|
72
|
+
# Feed one chunk; records/arena reset at feed entry, so drain
|
|
73
|
+
# after every feed. Returns self.
|
|
74
|
+
def feed(chunk, final: false)
|
|
75
|
+
chunk = chunk.dup.force_encoding(Encoding::UTF_8)
|
|
76
|
+
rc = Leptris::XML::FFI.leptris_sax_recorder_feed(
|
|
77
|
+
@handle, chunk, chunk.bytesize, final ? 1 : 0)
|
|
78
|
+
rc
|
|
79
|
+
end
|
|
80
|
+
|
|
81
|
+
# Feed an IO in chunks, draining events after each chunk.
|
|
82
|
+
def feed_stream(io, chunk_size = 65_536, kinds: nil, &block)
|
|
83
|
+
while (chunk = io.read(chunk_size))
|
|
84
|
+
feed(chunk)
|
|
85
|
+
each_event(*Array(kinds), &block)
|
|
86
|
+
end
|
|
87
|
+
feed("", final: true)
|
|
88
|
+
each_event(*Array(kinds), &block)
|
|
89
|
+
end
|
|
90
|
+
|
|
91
|
+
# Drains the current chunk's records: bulk-reads the record array
|
|
92
|
+
# and the packed arena (two FFI calls), then slices every string
|
|
93
|
+
# from the arena in Ruby — no per-string FFI. With +kinds+, records
|
|
94
|
+
# of other kinds are skipped BEFORE any string is sliced: an
|
|
95
|
+
# unwanted event costs one Array read, and cost scales with what
|
|
96
|
+
# the consumer asked for, not with the document's event mix.
|
|
97
|
+
def each_event(*kinds)
|
|
98
|
+
wanted = kinds.empty? ? nil : wanted_set(kinds)
|
|
99
|
+
count_ptr = ::FFI::MemoryPointer.new(:size_t)
|
|
100
|
+
len_ptr = ::FFI::MemoryPointer.new(:size_t)
|
|
101
|
+
begin
|
|
102
|
+
records_ptr = Leptris::XML::FFI.leptris_sax_recorder_records(
|
|
103
|
+
@handle, count_ptr)
|
|
104
|
+
return self if records_ptr.null? || count_ptr.read_uint64.zero?
|
|
105
|
+
|
|
106
|
+
arena_ptr = Leptris::XML::FFI.leptris_sax_recorder_arena(
|
|
107
|
+
@handle, len_ptr)
|
|
108
|
+
arena = arena_ptr.read_bytes(len_ptr.read_uint64)
|
|
109
|
+
count = count_ptr.read_uint64
|
|
110
|
+
|
|
111
|
+
# The whole record block in ONE read: read_bytes + String#unpack
|
|
112
|
+
# (C-fast) — two FFI calls per chunk total (records + arena),
|
|
113
|
+
# the true bulk discipline. Template per record: kind byte,
|
|
114
|
+
# 7 pad, then the eight uint32 fields (little-endian hosts —
|
|
115
|
+
# the layout the header freezes for FFI mirrors).
|
|
116
|
+
stride = Leptris::XML::FFI::SaxEventRecord.size
|
|
117
|
+
fields = records_ptr.read_bytes(count * stride)
|
|
118
|
+
.unpack("C x7 V8" * count)
|
|
119
|
+
|
|
120
|
+
i = 0
|
|
121
|
+
while i < count
|
|
122
|
+
kind = KIND_BY_CODE[fields[i * 9]]
|
|
123
|
+
if kind && (wanted.nil? || wanted[kind])
|
|
124
|
+
base = i * 9 + 1
|
|
125
|
+
yield kind,
|
|
126
|
+
slice(arena, fields[base], fields[base + 1]),
|
|
127
|
+
slice(arena, fields[base + 2], fields[base + 3]),
|
|
128
|
+
attrs_from(arena, fields[base + 4], fields[base + 5]),
|
|
129
|
+
fields[base + 6], fields[base + 7]
|
|
130
|
+
end
|
|
131
|
+
i += 1
|
|
132
|
+
end
|
|
133
|
+
ensure
|
|
134
|
+
count_ptr.free
|
|
135
|
+
len_ptr.free
|
|
136
|
+
end
|
|
137
|
+
self
|
|
138
|
+
end
|
|
139
|
+
|
|
140
|
+
private
|
|
141
|
+
|
|
142
|
+
def wanted_set(kinds)
|
|
143
|
+
unknown = kinds.each_with_object([]) do |kind, acc|
|
|
144
|
+
acc << kind unless KINDS.value?(kind)
|
|
145
|
+
end
|
|
146
|
+
unless unknown.empty?
|
|
147
|
+
raise ArgumentError,
|
|
148
|
+
"unknown event kinds #{unknown.inspect} " \
|
|
149
|
+
"(known: #{KINDS.values.inspect})"
|
|
150
|
+
end
|
|
151
|
+
kinds.each_with_object({}) { |kind, set| set[kind] = true }
|
|
152
|
+
end
|
|
153
|
+
|
|
154
|
+
# Zero-length slices are empty strings, not nil — the default
|
|
155
|
+
# namespace prefix is legitimately "" (START_PREFIX: name may be
|
|
156
|
+
# ""), matching the callback transport's delivery.
|
|
157
|
+
def slice(arena, off, len)
|
|
158
|
+
arena.byteslice(off, len).force_encoding(Encoding::UTF_8)
|
|
159
|
+
end
|
|
160
|
+
|
|
161
|
+
# attrs_off addresses name\0value\0... pairs, attr_count pairs;
|
|
162
|
+
# nil when the element carries none.
|
|
163
|
+
def attrs_from(arena, off, count)
|
|
164
|
+
return nil if count.zero?
|
|
165
|
+
hash = {}
|
|
166
|
+
pos = off
|
|
167
|
+
count.times do
|
|
168
|
+
name_end = arena.index("\0", pos)
|
|
169
|
+
name = arena.byteslice(pos, name_end - pos)
|
|
170
|
+
pos = name_end + 1
|
|
171
|
+
value_end = arena.index("\0", pos)
|
|
172
|
+
value = arena.byteslice(pos, value_end - pos)
|
|
173
|
+
pos = value_end + 1
|
|
174
|
+
hash[name.force_encoding(Encoding::UTF_8)] =
|
|
175
|
+
value.force_encoding(Encoding::UTF_8)
|
|
176
|
+
end
|
|
177
|
+
hash
|
|
178
|
+
end
|
|
179
|
+
end
|
data/lib/leptris/xml/sax.rb
CHANGED
data/lib/libleptris.so
CHANGED
|
Binary file
|
metadata
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: leptris
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 1.9.
|
|
4
|
+
version: 1.9.24
|
|
5
5
|
platform: aarch64-linux
|
|
6
6
|
authors:
|
|
7
7
|
- Ribose Inc.
|
|
@@ -99,6 +99,7 @@ files:
|
|
|
99
99
|
- lib/leptris/xml/sax.rb
|
|
100
100
|
- lib/leptris/xml/sax/document.rb
|
|
101
101
|
- lib/leptris/xml/sax/parser.rb
|
|
102
|
+
- lib/leptris/xml/sax/recorder.rb
|
|
102
103
|
- lib/leptris/xml/searchable.rb
|
|
103
104
|
- lib/leptris/xml/serialization.rb
|
|
104
105
|
- lib/leptris/xml/text.rb
|