purehtml 0.1.0 → 0.2.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a768efe71c391b97100948faf0ac5de6131971e9db477b8c1711aeb4c847a506
4
- data.tar.gz: f18783bfa9e3d67dbd7280dabde3d395a0224e52b71da21cdde76ac94787c9b5
3
+ metadata.gz: '0874a4a517f3ccdc7513231cde4084e1ad7f81d8985ecdb44485d6b307ad81ca'
4
+ data.tar.gz: 13c64400abf0eb074dd09465d7b78288c678cccc3062be327653d1d57af7055f
5
5
  SHA512:
6
- metadata.gz: 5191e380c661c485c89cd2d066bfb783d637618fa3b72216af3bd9fb8ea4c466d84abb41928db8d1df76ed93530800274d3a0feff9dded233f1e6690370a9a8d
7
- data.tar.gz: 366d5960a80c655c64641a644d1fffa65c2d35809d8fc474676612d65acebb93c7570683a15010573ab2daf47b5379b7fb5118cb1083fba59020789a5e069cd0
6
+ metadata.gz: fc7ed845c4c834faea208018a30234eeaa38525418652d39c3e3d4885b7d4f3ac69ec86421b09d8d2b379804ca91afe57a843f4860415bdf103d753712b1b20b
7
+ data.tar.gz: af30cd26d55ea3bd3b037b84c4f82cf23d20620397f9eb040745d9f739dbccb557a7f94e5646bbe7f6fd8c666aac235bbdb833aa53e20b614a991b9ba830b3e6
data/CHANGELOG.md ADDED
@@ -0,0 +1,42 @@
1
+ # Changelog
2
+
3
+ All notable changes to the PureHTML Ruby gem will be documented in this file.
4
+
5
+ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
6
+ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
7
+
8
+ ## [Unreleased]
9
+
10
+ ## [0.2.0] - 2026-11-20
11
+
12
+ ### Added
13
+ - Support for `nokolexbor` backend (`PureHTML::NokolexborBackend`) for high-performance HTML5 parsing.
14
+ - Flexible backend configuration supporting symbols (`:nokogiri`, `:nokolexbor`), classes, or instances via `PureHTML.backend=` and `PureHTML.extract(..., backend: ...)`.
15
+ - Automatic detection of pre-parsed `Nokolexbor::Node` / `Nokolexbor::Document` and `Nokogiri::XML::Node` objects passed to `PureHTML.extract`.
16
+ - Dual-backend test suite running all PureHTML specification tests against both Nokogiri and Nokolexbor.
17
+ - Multi-backend benchmarking in `bench/bench.rb`.
18
+
19
+ ### Changed
20
+ - Added `nokolexbor` development dependency and relaxed `rake` development dependency constraint to `>= 12.0`.
21
+
22
+ ## [0.1.1] - 2026-11-19
23
+
24
+ ### Changed
25
+ - Constrained gem dependencies (`nokogiri`, `rake`, `minitest`) to silence open-ended dependency warnings.
26
+ - Cleaned up duplicate `documentation_uri` in gemspec metadata.
27
+ - Added repository metadata URIs (`changelog_uri`, `source_code_uri`, `bug_tracker_uri`).
28
+
29
+ ## [0.1.0] - 2026-11-18
30
+
31
+ ### Added
32
+ - Initial Ruby implementation of PureHTML specification.
33
+ - Built on top of Nokogiri for fast HTML/XML parsing.
34
+ - Support for declarative YAML configuration schemas (`PureHTML::ConfigFactory.from_yaml`).
35
+ - Support for primitives, nested objects, arrays, unions, constants, and default values.
36
+ - Built-in transformers (`trim`, `number`, `boolean`, `attr`, `html`, `json`, `jsonpath`, `split`, `join`, `replace`, `capitalize`, `upper`, `lower`, etc.).
37
+ - Comprehensive test suite matching the PureHTML multi-language test specifications.
38
+
39
+ [Unreleased]: https://github.com/purescraps/purehtml/compare/ruby-v0.2.0...HEAD
40
+ [0.2.0]: https://github.com/purescraps/purehtml/compare/ruby-v0.1.1...ruby-v0.2.0
41
+ [0.1.1]: https://github.com/purescraps/purehtml/compare/ruby-v0.1.0...ruby-v0.1.1
42
+ [0.1.0]: https://github.com/purescraps/purehtml/releases/tag/ruby-v0.1.0
data/README.md CHANGED
@@ -7,7 +7,7 @@ PureHTML is a declarative HTML parsing and extraction library for Ruby. Extract
7
7
  ## Features
8
8
 
9
9
  - **Declarative Schemas**: Simple YAML definitions for HTML data extraction.
10
- - **Pluggable Backends**: Built on top of [Nokogiri](https://nokogiri.org).
10
+ - **Pluggable Backends**: Built on top of [Nokogiri](https://nokogiri.org) with optional high-performance [Nokolexbor](https://github.com/serpapi/nokolexbor) support.
11
11
  - **Extensible Transformers**: Value transformation chain (e.g. `trim`, `number`, `attr`, `html`, `json`, `jsonpath`, `split`, `join`, `replace`, etc.).
12
12
  - **Complex Structures**: Supports primitives, nested objects, arrays, unions, constants, and default values.
13
13
 
@@ -51,6 +51,36 @@ puts result.inspect
51
51
  # => ["a", "b", "c"]
52
52
  ```
53
53
 
54
+ ## Backends
55
+
56
+ PureHTML supports pluggable HTML parsing backends:
57
+
58
+ - **Nokogiri** (default): Robust and widely compatible HTML/XML parser.
59
+ - **Nokolexbor**: Ultra-fast HTML5 parser backed by Lexbor (significantly faster parsing and CSS selector execution).
60
+
61
+ ### Using Nokolexbor
62
+
63
+ Add `nokolexbor` to your Gemfile:
64
+
65
+ ```ruby
66
+ gem 'nokolexbor'
67
+ ```
68
+
69
+ You can select the backend globally or pass it per extraction:
70
+
71
+ ```ruby
72
+ # Set globally
73
+ PureHTML.backend = :nokolexbor
74
+ # or PureHTML.backend = PureHTML::NokolexborBackend.new
75
+
76
+ # Per extraction
77
+ PureHTML.extract(config, html_string, backend: :nokolexbor)
78
+
79
+ # Or pass a pre-parsed Nokolexbor document directly
80
+ doc = Nokolexbor::HTML(html_string)
81
+ PureHTML.extract(config, doc)
82
+ ```
83
+
54
84
  ## Configuration Types
55
85
 
56
86
  ### Primitive Value
@@ -2,6 +2,12 @@
2
2
 
3
3
  require "nokogiri"
4
4
 
5
+ begin
6
+ require "nokolexbor"
7
+ rescue LoadError
8
+ # nokolexbor is an optional backend
9
+ end
10
+
5
11
  module PureHTML
6
12
  class Backend
7
13
  def load(html)
@@ -43,12 +49,19 @@ module PureHTML
43
49
 
44
50
  class NokogiriBackend < Backend
45
51
  def load(html)
46
- NokogiriDocument.new(Nokogiri::HTML(html))
52
+ if html.is_a?(NokogiriDocument)
53
+ html
54
+ elsif html.is_a?(::Nokogiri::XML::Node)
55
+ NokogiriDocument.new(html)
56
+ else
57
+ NokogiriDocument.new(Nokogiri::HTML(html))
58
+ end
47
59
  end
48
60
  end
49
61
 
50
62
  class NokogiriDocument < Document
51
63
  attr_reader :nokogiri_doc
64
+ alias_method :doc, :nokogiri_doc
52
65
 
53
66
  def initialize(nokogiri_doc)
54
67
  @nokogiri_doc = nokogiri_doc
@@ -117,4 +130,109 @@ module PureHTML
117
130
  end
118
131
  end
119
132
  end
133
+
134
+ class NokolexborBackend < Backend
135
+ def initialize
136
+ require_nokolexbor!
137
+ end
138
+
139
+ def load(html)
140
+ require_nokolexbor!
141
+ if html.is_a?(NokolexborDocument)
142
+ html
143
+ elsif defined?(::Nokolexbor::Node) && html.is_a?(::Nokolexbor::Node)
144
+ NokolexborDocument.new(html)
145
+ else
146
+ NokolexborDocument.new(::Nokolexbor::HTML(html))
147
+ end
148
+ end
149
+
150
+ private
151
+
152
+ def require_nokolexbor!
153
+ return if defined?(::Nokolexbor)
154
+
155
+ begin
156
+ require "nokolexbor"
157
+ rescue LoadError => e
158
+ raise LoadError, "NokolexborBackend requires the 'nokolexbor' gem: #{e.message}"
159
+ end
160
+ end
161
+ end
162
+
163
+ class NokolexborDocument < Document
164
+ attr_reader :nokolexbor_doc
165
+ alias_method :doc, :nokolexbor_doc
166
+
167
+ def initialize(nokolexbor_doc)
168
+ @nokolexbor_doc = nokolexbor_doc
169
+ end
170
+
171
+ def root
172
+ NokolexborNode.new(@nokolexbor_doc, @nokolexbor_doc)
173
+ end
174
+
175
+ def find(selector)
176
+ @nokolexbor_doc.css(selector).map { |node| NokolexborNode.new(@nokolexbor_doc, node) }
177
+ end
178
+ end
179
+
180
+ class NokolexborNode < Node
181
+ attr_reader :doc, :node
182
+
183
+ def initialize(doc, node)
184
+ @doc = doc
185
+ @node = node
186
+ end
187
+
188
+ def is_selector?(selector)
189
+ return false if @node.nil? || @node.document?
190
+
191
+ if @node.respond_to?(:matches?)
192
+ begin
193
+ return @node.matches?(selector)
194
+ rescue StandardError
195
+ # Fall through to doc.css check if matches? encounters an unsupported selector syntax
196
+ end
197
+ end
198
+
199
+ @doc.css(selector).include?(@node)
200
+ end
201
+
202
+ def find(selector)
203
+ nodes = if @node.document?
204
+ @doc.css(selector)
205
+ else
206
+ @node.css(selector)
207
+ end
208
+
209
+ nodes.map { |n| NokolexborNode.new(@doc, n) }
210
+ end
211
+
212
+ def text
213
+ return "" if @node.nil?
214
+
215
+ if @node.document? && @node.respond_to?(:root) && @node.root
216
+ return @node.root.text
217
+ end
218
+
219
+ @node.text
220
+ end
221
+
222
+ def html
223
+ return nil if @node.nil?
224
+
225
+ @node.inner_html
226
+ end
227
+
228
+ def attr(name = nil)
229
+ return nil if @node.nil? || @node.document?
230
+
231
+ if name
232
+ @node[name]
233
+ else
234
+ @node.respond_to?(:attributes) ? @node.attributes.transform_values(&:value) : {}
235
+ end
236
+ end
237
+ end
120
238
  end
@@ -1,5 +1,5 @@
1
1
  # frozen_string_literal: true
2
2
 
3
3
  module PureHTML
4
- VERSION = "0.1.0"
4
+ VERSION = "0.2.0"
5
5
  end
data/lib/purehtml.rb CHANGED
@@ -19,14 +19,25 @@ require_relative "purehtml/config_factory"
19
19
 
20
20
  module PureHTML
21
21
  class << self
22
- attr_writer :backend
23
-
24
22
  def backend
25
23
  @backend ||= NokogiriBackend.new
26
24
  end
27
25
 
26
+ def backend=(backend)
27
+ @backend = backend.nil? ? nil : resolve_backend(backend)
28
+ end
29
+
28
30
  def extract(config, html, url = "", backend: nil)
29
- active_backend = backend || self.backend
31
+ active_backend = if backend
32
+ resolve_backend(backend)
33
+ elsif defined?(::Nokolexbor::Node) && html.is_a?(::Nokolexbor::Node)
34
+ NokolexborBackend.new
35
+ elsif defined?(::Nokogiri::XML::Node) && html.is_a?(::Nokogiri::XML::Node)
36
+ NokogiriBackend.new
37
+ else
38
+ self.backend
39
+ end
40
+
30
41
  doc = html.is_a?(Document) ? html : active_backend.load(html)
31
42
  params = ExtractParams.new(
32
43
  doc: doc,
@@ -38,5 +49,22 @@ module PureHTML
38
49
  )
39
50
  config.extract(params)
40
51
  end
52
+
53
+ def resolve_backend(backend)
54
+ case backend
55
+ when :nokogiri, "nokogiri"
56
+ NokogiriBackend.new
57
+ when :nokolexbor, "nokolexbor"
58
+ NokolexborBackend.new
59
+ when Class
60
+ backend.new
61
+ else
62
+ if backend.respond_to?(:load)
63
+ backend
64
+ else
65
+ raise ArgumentError, "Unsupported backend: #{backend.inspect}"
66
+ end
67
+ end
68
+ end
41
69
  end
42
70
  end
data/purehtml.gemspec CHANGED
@@ -16,14 +16,17 @@ Gem::Specification.new do |spec|
16
16
 
17
17
  spec.metadata["homepage_uri"] = spec.homepage
18
18
  spec.metadata["source_code_uri"] = "https://github.com/purescraps/purehtml"
19
+ spec.metadata["changelog_uri"] = "https://github.com/purescraps/purehtml/blob/master/ruby/CHANGELOG.md"
20
+ spec.metadata["bug_tracker_uri"] = "https://github.com/purescraps/purehtml/issues"
19
21
 
20
22
  spec.files = Dir.chdir(File.expand_path(__dir__)) do
21
- Dir.glob("{lib}/**/*") + %w[README.md LICENSE purehtml.gemspec Rakefile]
23
+ Dir.glob("{lib}/**/*") + %w[README.md LICENSE CHANGELOG.md purehtml.gemspec Rakefile]
22
24
  end
23
25
  spec.require_paths = ["lib"]
24
26
 
25
- spec.add_dependency "nokogiri", ">= 1.10.0"
27
+ spec.add_dependency "nokogiri", "~> 1.10", ">= 1.10.0"
26
28
 
27
29
  spec.add_development_dependency "rake", ">= 12.0"
28
30
  spec.add_development_dependency "minitest", ">= 5.0"
31
+ spec.add_development_dependency "nokolexbor", ">= 0.5.0"
29
32
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: purehtml
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.1.0
4
+ version: 0.2.0
5
5
  platform: ruby
6
6
  authors:
7
7
  - Fatih Kaya
@@ -13,6 +13,9 @@ dependencies:
13
13
  name: nokogiri
14
14
  requirement: !ruby/object:Gem::Requirement
15
15
  requirements:
16
+ - - "~>"
17
+ - !ruby/object:Gem::Version
18
+ version: '1.10'
16
19
  - - ">="
17
20
  - !ruby/object:Gem::Version
18
21
  version: 1.10.0
@@ -20,6 +23,9 @@ dependencies:
20
23
  prerelease: false
21
24
  version_requirements: !ruby/object:Gem::Requirement
22
25
  requirements:
26
+ - - "~>"
27
+ - !ruby/object:Gem::Version
28
+ version: '1.10'
23
29
  - - ">="
24
30
  - !ruby/object:Gem::Version
25
31
  version: 1.10.0
@@ -51,6 +57,20 @@ dependencies:
51
57
  - - ">="
52
58
  - !ruby/object:Gem::Version
53
59
  version: '5.0'
60
+ - !ruby/object:Gem::Dependency
61
+ name: nokolexbor
62
+ requirement: !ruby/object:Gem::Requirement
63
+ requirements:
64
+ - - ">="
65
+ - !ruby/object:Gem::Version
66
+ version: 0.5.0
67
+ type: :development
68
+ prerelease: false
69
+ version_requirements: !ruby/object:Gem::Requirement
70
+ requirements:
71
+ - - ">="
72
+ - !ruby/object:Gem::Version
73
+ version: 0.5.0
54
74
  description: PureHTML is a parsing specification and library for extracting JSON from
55
75
  HTML documents using simple YAML schemas.
56
76
  email:
@@ -59,6 +79,7 @@ executables: []
59
79
  extensions: []
60
80
  extra_rdoc_files: []
61
81
  files:
82
+ - CHANGELOG.md
62
83
  - LICENSE
63
84
  - README.md
64
85
  - Rakefile
@@ -106,6 +127,8 @@ licenses:
106
127
  metadata:
107
128
  homepage_uri: https://purescraps.github.io/purehtml/
108
129
  source_code_uri: https://github.com/purescraps/purehtml
130
+ changelog_uri: https://github.com/purescraps/purehtml/blob/master/ruby/CHANGELOG.md
131
+ bug_tracker_uri: https://github.com/purescraps/purehtml/issues
109
132
  rdoc_options: []
110
133
  require_paths:
111
134
  - lib
@@ -120,7 +143,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
120
143
  - !ruby/object:Gem::Version
121
144
  version: '0'
122
145
  requirements: []
123
- rubygems_version: 4.0.20
146
+ rubygems_version: 3.6.9
124
147
  specification_version: 4
125
148
  summary: Declarative HTML parsing and data extraction library for Ruby
126
149
  test_files: []