parsec-html 0.1.0__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- parsec_html-0.1.0/Cargo.toml +14 -0
- parsec_html-0.1.0/PKG-INFO +186 -0
- parsec_html-0.1.0/README.md +168 -0
- parsec_html-0.1.0/bindings/python/Cargo.lock +610 -0
- parsec_html-0.1.0/bindings/python/Cargo.toml +39 -0
- parsec_html-0.1.0/bindings/python/README.md +168 -0
- parsec_html-0.1.0/bindings/python/bench.py +98 -0
- parsec_html-0.1.0/bindings/python/ci-requirements.in +4 -0
- parsec_html-0.1.0/bindings/python/ci-requirements.txt +133 -0
- parsec_html-0.1.0/bindings/python/parsec_html.pyi +213 -0
- parsec_html-0.1.0/bindings/python/src/lib.rs +1876 -0
- parsec_html-0.1.0/bindings/python/src/units.rs +133 -0
- parsec_html-0.1.0/bindings/python/tests/test_encoding.py +203 -0
- parsec_html-0.1.0/bindings/python/tests/test_parsec_html.py +476 -0
- parsec_html-0.1.0/crates/parsec/Cargo.toml +25 -0
- parsec_html-0.1.0/crates/parsec/LICENSE +201 -0
- parsec_html-0.1.0/crates/parsec/NOTICE +64 -0
- parsec_html-0.1.0/crates/parsec/README.md +511 -0
- parsec_html-0.1.0/crates/parsec/examples/hostile.rs +38 -0
- parsec_html-0.1.0/crates/parsec/examples/parallel.rs +32 -0
- parsec_html-0.1.0/crates/parsec/examples/parse.rs +19 -0
- parsec_html-0.1.0/crates/parsec/examples/select.rs +78 -0
- parsec_html-0.1.0/crates/parsec/examples/serialize.rs +21 -0
- parsec_html-0.1.0/crates/parsec/examples/tokenize.rs +41 -0
- parsec_html-0.1.0/crates/parsec/scripts/fetch-tree-tests.sh +15 -0
- parsec_html-0.1.0/crates/parsec/scripts/fetch-wpt-selectors.sh +33 -0
- parsec_html-0.1.0/crates/parsec/scripts/gen-entities.py +36 -0
- parsec_html-0.1.0/crates/parsec/scripts/gen-unicode.py +257 -0
- parsec_html-0.1.0/crates/parsec/scripts/wpt-selectors-cases.js +454 -0
- parsec_html-0.1.0/crates/parsec/src/css.rs +3925 -0
- parsec_html-0.1.0/crates/parsec/src/dom.rs +1137 -0
- parsec_html-0.1.0/crates/parsec/src/edit.rs +1086 -0
- parsec_html-0.1.0/crates/parsec/src/entities.rs +2239 -0
- parsec_html-0.1.0/crates/parsec/src/lib.rs +222 -0
- parsec_html-0.1.0/crates/parsec/src/pattern.rs +1518 -0
- parsec_html-0.1.0/crates/parsec/src/serialize.rs +1451 -0
- parsec_html-0.1.0/crates/parsec/src/shadow.rs +801 -0
- parsec_html-0.1.0/crates/parsec/src/tags.rs +232 -0
- parsec_html-0.1.0/crates/parsec/src/tokenizer.rs +2752 -0
- parsec_html-0.1.0/crates/parsec/src/tree.rs +4457 -0
- parsec_html-0.1.0/crates/parsec/src/unicode.rs +3177 -0
- parsec_html-0.1.0/crates/parsec/tests/chunks.rs +178 -0
- parsec_html-0.1.0/crates/parsec/tests/css.rs +1162 -0
- parsec_html-0.1.0/crates/parsec/tests/errors.rs +160 -0
- parsec_html-0.1.0/crates/parsec/tests/inputs/mod.rs +152 -0
- parsec_html-0.1.0/crates/parsec/tests/serializer.rs +252 -0
- parsec_html-0.1.0/crates/parsec/tests/shadow-dom/COMMIT +1 -0
- parsec_html-0.1.0/crates/parsec/tests/shadow-dom/LICENSE.md +11 -0
- parsec_html-0.1.0/crates/parsec/tests/shadow-dom/declarative.dat +691 -0
- parsec_html-0.1.0/crates/parsec/tests/shadow-dom/gethtml-ordering.html +69 -0
- parsec_html-0.1.0/crates/parsec/tests/shadow-dom.rs +671 -0
- parsec_html-0.1.0/crates/parsec/tests/spans.rs +260 -0
- parsec_html-0.1.0/crates/parsec/tests/summary/mod.rs +50 -0
- parsec_html-0.1.0/crates/parsec/tests/threads.rs +23 -0
- parsec_html-0.1.0/crates/parsec/tests/tokenizer.rs +335 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/COMMIT +1 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/LICENSE.md +11 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/adoption01.dat +354 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/adoption02.dat +72 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/blocks.dat +695 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/comments01.dat +214 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/doctype01.dat +474 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/domjs-unsafe.dat +0 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/entities01.dat +943 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/entities02.dat +309 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/foreign-fragment.dat +645 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/html5test-com.dat +365 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/inbody01.dat +54 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/isindex.dat +49 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/main-element.dat +46 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/math.dat +104 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/menuitem-element.dat +241 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/namespace-sensitivity.dat +22 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/noscript01.dat +237 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/pending-spec-changes-plain-text-unsafe.dat +0 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/pending-spec-changes.dat +46 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/plain-text-unsafe.dat +0 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/processing-instructions.dat +1132 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/quirks01.dat +53 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/ruby.dat +302 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/scriptdata01.dat +372 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/scripted_adoption01.dat +16 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/scripted_ark.dat +27 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/scripted_foster01.dat +21 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/scripted_webkit01.dat +30 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/search-element.dat +132 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/svg.dat +104 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tables01.dat +323 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/template.dat +1879 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests1.dat +1949 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests10.dat +853 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests11.dat +523 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests12.dat +62 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests14.dat +75 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests15.dat +216 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests16.dat +2602 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests17.dat +177 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests18.dat +572 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests19.dat +1456 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests2.dat +831 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests20.dat +842 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests21.dat +306 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests22.dat +190 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests23.dat +168 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests24.dat +79 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests25.dat +288 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests26.dat +453 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests3.dat +305 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests4.dat +80 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests5.dat +210 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests6.dat +663 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests7.dat +453 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests8.dat +165 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests9.dat +482 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests_innerHTML_1.dat +844 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/tricky01.dat +336 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/void-in-phrasing.dat +151 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/webkit01.dat +783 -0
- parsec_html-0.1.0/crates/parsec/tests/tree-construction/webkit02.dat +775 -0
- parsec_html-0.1.0/crates/parsec/tests/tree.rs +280 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/COMMIT +1 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/LICENSE.md +11 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/cases.json +11353 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/class-prefix-selector-has-matching.tentative.html +92 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/class-prefix-selector-matching-quirks.tentative.html +36 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/class-prefix-selector-matching.tentative.html +114 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/class-prefix-selector-queryselector.tentative.html +42 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/dir-selector-querySelector.html +71 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/first-child.html +52 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/first-of-type.html +90 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/focus-visible-017-2.html +93 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/focus-visible-017.html +44 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/focus-visible-023.html +33 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/focus-visible-script-focus-001.html +54 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/forgiving-selector-list-comma-in-block.html +45 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/has-argument-with-explicit-scope.html +69 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/has-basic.html +81 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/has-relative-argument.html +184 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/is-where-basic.html +41 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/is-where-not.html +45 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/last-child.html +52 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/last-of-type.html +89 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/missing-right-token.html +31 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/not-complex.html +46 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/only-child.html +61 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/only-of-type.html +64 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/invalid-pseudos.html +25 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-attribute.html +30 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-child.html +11 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-class-prefix.tentative.html +50 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-class.html +15 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-descendant.html +13 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-focus-visible.html +13 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has-class-prefix.tentative.html +34 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has-disallow-nesting-has-inside-has.html +19 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has-forgiving-selector.html +15 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has-slotted.tentative.html +32 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has.html +40 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-heading.html +38 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-id.html +13 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-is-where.html +59 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-is.html +18 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-not.html +37 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-part.html +44 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-sibling.html +12 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-slotted.html +28 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-state.html +35 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-universal.html +14 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-where.html +18 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/query/query-is.html +57 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/query/query-where.html +57 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/selectors-4/lang-singleton-subtag-matching.html +78 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/webkit-pseudo-element.html +57 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/dom/nodes/Element-closest.html +73 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/dom/nodes/ParentNode-querySelector-All-content.html +377 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/known-failures.txt +4 -0
- parsec_html-0.1.0/crates/parsec/tests/wpt-selectors.rs +557 -0
- parsec_html-0.1.0/crates/parsec-encoding/Cargo.toml +26 -0
- parsec_html-0.1.0/crates/parsec-encoding/LICENSE +201 -0
- parsec_html-0.1.0/crates/parsec-encoding/NOTICE +64 -0
- parsec_html-0.1.0/crates/parsec-encoding/README.md +511 -0
- parsec_html-0.1.0/crates/parsec-encoding/src/lib.rs +405 -0
- parsec_html-0.1.0/crates/parsec-encoding/src/offsets.rs +239 -0
- parsec_html-0.1.0/crates/parsec-encoding/src/prescan.rs +351 -0
- parsec_html-0.1.0/crates/parsec-encoding/src/stream.rs +380 -0
- parsec_html-0.1.0/crates/parsec-xpath/Cargo.toml +19 -0
- parsec_html-0.1.0/crates/parsec-xpath/LICENSE +201 -0
- parsec_html-0.1.0/crates/parsec-xpath/NOTICE +64 -0
- parsec_html-0.1.0/crates/parsec-xpath/README.md +511 -0
- parsec_html-0.1.0/crates/parsec-xpath/src/eval.rs +1461 -0
- parsec_html-0.1.0/crates/parsec-xpath/src/index.rs +251 -0
- parsec_html-0.1.0/crates/parsec-xpath/src/lexer.rs +274 -0
- parsec_html-0.1.0/crates/parsec-xpath/src/lib.rs +274 -0
- parsec_html-0.1.0/crates/parsec-xpath/src/parser.rs +719 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/COMMIT +1 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/Copyright +24 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/base +68 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/compare +184 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/equality +104 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/floats +252 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/functions +172 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/strings +192 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/base +17 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/compare +46 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/equality +26 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/floats +63 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/functions +43 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/strings +50 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2.rs +134 -0
- parsec_html-0.1.0/crates/parsec-xpath/tests/xpath.rs +537 -0
- parsec_html-0.1.0/pyproject.toml +28 -0
|
@@ -0,0 +1,14 @@
|
|
|
1
|
+
[workspace]
|
|
2
|
+
resolver = "3"
|
|
3
|
+
members = ["crates/parsec", "crates/parsec-html", "crates/parsec-capi", "crates/parsec-css", "crates/parsec-encoding", "crates/parsec-url", "crates/parsec-xpath"]
|
|
4
|
+
|
|
5
|
+
[workspace.package]
|
|
6
|
+
version = "0.1.0"
|
|
7
|
+
edition = "2024"
|
|
8
|
+
license = "Apache-2.0"
|
|
9
|
+
repository = "https://github.com/parsec-html/parsec"
|
|
10
|
+
rust-version = "1.88"
|
|
11
|
+
|
|
12
|
+
[profile.release]
|
|
13
|
+
lto = "thin"
|
|
14
|
+
codegen-units = 1
|
|
@@ -0,0 +1,186 @@
|
|
|
1
|
+
Metadata-Version: 2.4
|
|
2
|
+
Name: parsec-html
|
|
3
|
+
Version: 0.1.0
|
|
4
|
+
Classifier: Development Status :: 3 - Alpha
|
|
5
|
+
Classifier: Programming Language :: Python :: 3
|
|
6
|
+
Classifier: Programming Language :: Python :: Free Threading :: 3 - Stable
|
|
7
|
+
Classifier: Programming Language :: Rust
|
|
8
|
+
Classifier: Topic :: Text Processing :: Markup :: HTML
|
|
9
|
+
Summary: A fast, spec-following HTML parser with CSS selectors and an editable tree, with a selectolax-style API.
|
|
10
|
+
Keywords: html,parser,html5,css-selectors,selectolax
|
|
11
|
+
License-Expression: Apache-2.0
|
|
12
|
+
Requires-Python: >=3.9
|
|
13
|
+
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
14
|
+
Project-URL: Documentation, https://github.com/parsec-html/parsec/wiki/Python
|
|
15
|
+
Project-URL: Homepage, https://github.com/parsec-html/parsec
|
|
16
|
+
Project-URL: Issues, https://github.com/parsec-html/parsec/issues
|
|
17
|
+
|
|
18
|
+
# parsec-html for Python
|
|
19
|
+
|
|
20
|
+
Parsec is an HTML parser in Rust that reads pages the way a browser does,
|
|
21
|
+
finds what you want with CSS selectors, and lets you change the tree and
|
|
22
|
+
write it back out. This is its Python package, and if you've used
|
|
23
|
+
selectolax you already know the API:
|
|
24
|
+
|
|
25
|
+
```python
|
|
26
|
+
from parsec_html import HTMLParser
|
|
27
|
+
|
|
28
|
+
page = HTMLParser("<ul><li class=hit>one<li>two</ul>")
|
|
29
|
+
assert page.css_first("li.hit").text() == "one"
|
|
30
|
+
for li in page.css("li"):
|
|
31
|
+
print(li.tag, li.attributes, li.html)
|
|
32
|
+
```
|
|
33
|
+
|
|
34
|
+
What you get over selectolax:
|
|
35
|
+
|
|
36
|
+
- **Threads.** One parsed page can be read from as many threads as you
|
|
37
|
+
like, and edited from any of them. On free-threaded Python (3.14t
|
|
38
|
+
and later) the module runs with the GIL off, and on any Python, parsing and
|
|
39
|
+
`css()` let go of it while they work.
|
|
40
|
+
- **Selectors selectolax 1.0.2 won't run**: `:lang()`, `:dir()`,
|
|
41
|
+
`:valid`, `:in-range`, `:scope` and more, which it rejects as
|
|
42
|
+
unparseable, and `:target` and `:local-link` once you pass the page's
|
|
43
|
+
`url`.
|
|
44
|
+
- **Limits** for pages you don't trust: `budget=` (seconds) and
|
|
45
|
+
`max_depth=`.
|
|
46
|
+
- **Nodes that outlive edits.** A removed node can still be read, and put
|
|
47
|
+
back.
|
|
48
|
+
- **XPath 1.0**, which selectolax doesn't have. `xpath()` gives back what
|
|
49
|
+
lxml's does, so code written for lxml or parsel reads the same:
|
|
50
|
+
|
|
51
|
+
```python
|
|
52
|
+
page.xpath("//li/text()") # ['one', 'two']
|
|
53
|
+
page.xpath("count(//li)") # 2.0
|
|
54
|
+
page.xpath("//a[@rel=$r]/@href", r="next")
|
|
55
|
+
```
|
|
56
|
+
|
|
57
|
+
Elements and comments come back as nodes, text and attribute values as
|
|
58
|
+
`str`, and a number, string or boolean as `float`, `str` or `bool`.
|
|
59
|
+
- **Where everything was written.** Parse with `positions=True` and every
|
|
60
|
+
node knows its place in your input: `node.source_range` and
|
|
61
|
+
`node.end_tag_range` are `(start, end)` indexes into what you passed
|
|
62
|
+
(code points into a `str`, bytes into `bytes`), so `src[start:end]` is
|
|
63
|
+
the tag as you wrote it; `node.position` is
|
|
64
|
+
its `(line, column)`, and `node.attribute_range("href")` finds one
|
|
65
|
+
attribute.
|
|
66
|
+
- **Parse errors.** With `errors=True`, `page.errors` lists each one with
|
|
67
|
+
the HTML Standard's name for it and where it is:
|
|
68
|
+
`2:1: unexpected-end-tag`.
|
|
69
|
+
- **Bytes in any encoding.** Hand `HTMLParser` the `bytes` you
|
|
70
|
+
downloaded and it decodes them as a browser would: a byte order mark
|
|
71
|
+
first, then the HTTP `charset`, which you pass as `transport_encoding`,
|
|
72
|
+
then a `<meta>`, then UTF-8 if the bytes are valid UTF-8, then
|
|
73
|
+
`default_encoding` (windows-1252 unless you say otherwise;
|
|
74
|
+
`encoding_for_locale("ja")` gives the one a Japanese browser uses). `page.encoding` says which it was, and
|
|
75
|
+
`page.encoding_confidence` whether it's `"certain"` or `"tentative"`.
|
|
76
|
+
Positions stay byte offsets into your bytes, whatever the encoding.
|
|
77
|
+
|
|
78
|
+
```python
|
|
79
|
+
with urllib.request.urlopen(url) as r:
|
|
80
|
+
page = HTMLParser(r.read(), transport_encoding=r.headers.get_content_charset())
|
|
81
|
+
page.encoding # 'Shift_JIS'
|
|
82
|
+
```
|
|
83
|
+
|
|
84
|
+
A `charset` that names no encoding is passed over, as browsers pass
|
|
85
|
+
it over.
|
|
86
|
+
|
|
87
|
+
`StreamParser` does the same as the bytes arrive, and you get the same
|
|
88
|
+
page however they're cut:
|
|
89
|
+
|
|
90
|
+
```python
|
|
91
|
+
with urllib.request.urlopen(url) as r:
|
|
92
|
+
s = StreamParser(transport_encoding=r.headers.get_content_charset())
|
|
93
|
+
while chunk := r.read(65536):
|
|
94
|
+
s.feed(chunk)
|
|
95
|
+
page = s.close() # an HTMLParser
|
|
96
|
+
```
|
|
97
|
+
|
|
98
|
+
The names are those of lxml's feed parser and the standard library's
|
|
99
|
+
`html.parser`; the keywords are html5lib's.
|
|
100
|
+
|
|
101
|
+
**A small change.** `bytes` used to be read as UTF-8 whatever they
|
|
102
|
+
were. UTF-8 still comes out as UTF-8, declared or not. Bytes that
|
|
103
|
+
aren't valid UTF-8 and don't say what they are used to get U+FFFD in
|
|
104
|
+
place of each bad byte; now they decode as windows-1252, as in a
|
|
105
|
+
browser. Pass `transport_encoding` if you know better.
|
|
106
|
+
|
|
107
|
+
## Installing
|
|
108
|
+
|
|
109
|
+
Parsec isn't released yet. Wheels go to PyPI as `parsec-html` with 0.1.0.
|
|
110
|
+
Until then, build it from git (you need Rust 1.88 or newer):
|
|
111
|
+
|
|
112
|
+
```sh
|
|
113
|
+
pip install "git+https://github.com/parsec-html/parsec#subdirectory=bindings/python"
|
|
114
|
+
```
|
|
115
|
+
|
|
116
|
+
## Moving from selectolax
|
|
117
|
+
|
|
118
|
+
Change the import, and most code runs as it was:
|
|
119
|
+
|
|
120
|
+
```python
|
|
121
|
+
# from selectolax.lexbor import LexborHTMLParser as HTMLParser
|
|
122
|
+
from parsec_html import HTMLParser
|
|
123
|
+
```
|
|
124
|
+
|
|
125
|
+
`HTMLParser(html, is_fragment=False, fragment_tag="div",
|
|
126
|
+
fragment_namespace="html", scripting=False)` takes what
|
|
127
|
+
`LexborHTMLParser` does (plus `budget`, `max_depth`, `url`, `positions`,
|
|
128
|
+
`errors`, `transport_encoding` and `default_encoding`), and the parser and its nodes have selectolax's
|
|
129
|
+
names: `root`, `head`, `body`, `html`, `css`, `css_first` (with `default`
|
|
130
|
+
and `strict`), `css_matches`, `any_css_matches`, `tags`, `text(deep,
|
|
131
|
+
separator, strip, skip_empty)`, `merge_text_nodes`, `strip_tags`,
|
|
132
|
+
`unwrap_tags`, `create_node`; on a node also `tag`, `id`, `attributes`,
|
|
133
|
+
`attrs`, `inner_html` (which you can set), `text_content`,
|
|
134
|
+
`comment_content`, `parent`, `next`, `prev`, `first_child`, `last_child`,
|
|
135
|
+
`iter`, `traverse`, `decompose`, `remove`, `unwrap`, `replace_with`,
|
|
136
|
+
`insert_before`, `insert_after`, `insert_child` (all four taking a string
|
|
137
|
+
as text, or a node to copy, or to move with `move=True`), and the
|
|
138
|
+
`is_*_node` checks.
|
|
139
|
+
|
|
140
|
+
Where it behaves differently:
|
|
141
|
+
|
|
142
|
+
| | selectolax 1.0.2 | parsec_html |
|
|
143
|
+
|---|---|---|
|
|
144
|
+
| `<a href>`'s `attributes` | `{'href': None}` | `{'href': ''}`: Parsec doesn't yet record whether a value was written |
|
|
145
|
+
| A selector that doesn't parse | `SelectolaxError` | `SelectorError`, a `ValueError` |
|
|
146
|
+
| A node after `decompose()` | freed | reads as empty (no parent, children, attributes or text, `html` is `""`, no selector matches) but keeps its `tag`; `xpath()` and putting it back raise `ValueError` |
|
|
147
|
+
| `remove()` | the same as `decompose()` | takes the node out but keeps it readable, to put back with `move=True` |
|
|
148
|
+
| `attrs.items()`, `keys()`, `values()` | iterators | lists |
|
|
149
|
+
| `bytes` in | UTF-8, or sniffed with `encoding=True` | sniffed as a browser sniffs them: byte order mark, `transport_encoding`, `<meta>`, then `default_encoding` |
|
|
150
|
+
| Class names | `LexborHTMLParser`, `LexborNode` | `HTMLParser`, `Node` |
|
|
151
|
+
|
|
152
|
+
Not there yet: `clone()`, `raw_html`, `raw_value`, `html_pretty`,
|
|
153
|
+
`select()` and its filters, `scripts_contain`, `script_srcs_contain`,
|
|
154
|
+
`template_fragments`, `mem_id`, `tag_id` and the `:lexbor-contains()`
|
|
155
|
+
pseudo-class.
|
|
156
|
+
|
|
157
|
+
## Speed
|
|
158
|
+
|
|
159
|
+
`bench.py` times both on the 13 real pages `bench/` uses (24 MB in all):
|
|
160
|
+
|
|
161
|
+
| | parsec_html | selectolax | |
|
|
162
|
+
|---|---:|---:|---:|
|
|
163
|
+
| Parse all 13 pages | 66 ms | 106 ms | 1.6× |
|
|
164
|
+
| The 32 selectors over all 13 | 283 ms | 781 ms | 2.8× |
|
|
165
|
+
| Every link's `href` and text | 43 ms | 50 ms | 1.2× |
|
|
166
|
+
|
|
167
|
+
Parsec parses each of the 13 pages faster, by 1.5 to 2.8 times, and runs
|
|
168
|
+
each of the 32 selectors faster, by 1.04 to 29 times
|
|
169
|
+
(`p:nth-child(2n+1)`). The narrowest margins are the selectors that
|
|
170
|
+
match most of a page, such as `*` and `[class]`, where the time goes on
|
|
171
|
+
making a Python object per match. Medians of 20 runs on an Apple M2 Max,
|
|
172
|
+
CPython 3.14, at a load of 3; a second run gave the same to within 6%.
|
|
173
|
+
|
|
174
|
+
## Working on it
|
|
175
|
+
|
|
176
|
+
```sh
|
|
177
|
+
cd bindings/python
|
|
178
|
+
pip install maturin pytest
|
|
179
|
+
maturin develop --release
|
|
180
|
+
pytest tests
|
|
181
|
+
python bench.py # after bench/fetch-pages.sh and pip install selectolax
|
|
182
|
+
```
|
|
183
|
+
|
|
184
|
+
The [Python page](https://github.com/parsec-html/parsec/wiki/Python) on
|
|
185
|
+
the wiki has the whole API.
|
|
186
|
+
|
|
@@ -0,0 +1,168 @@
|
|
|
1
|
+
# parsec-html for Python
|
|
2
|
+
|
|
3
|
+
Parsec is an HTML parser in Rust that reads pages the way a browser does,
|
|
4
|
+
finds what you want with CSS selectors, and lets you change the tree and
|
|
5
|
+
write it back out. This is its Python package, and if you've used
|
|
6
|
+
selectolax you already know the API:
|
|
7
|
+
|
|
8
|
+
```python
|
|
9
|
+
from parsec_html import HTMLParser
|
|
10
|
+
|
|
11
|
+
page = HTMLParser("<ul><li class=hit>one<li>two</ul>")
|
|
12
|
+
assert page.css_first("li.hit").text() == "one"
|
|
13
|
+
for li in page.css("li"):
|
|
14
|
+
print(li.tag, li.attributes, li.html)
|
|
15
|
+
```
|
|
16
|
+
|
|
17
|
+
What you get over selectolax:
|
|
18
|
+
|
|
19
|
+
- **Threads.** One parsed page can be read from as many threads as you
|
|
20
|
+
like, and edited from any of them. On free-threaded Python (3.14t
|
|
21
|
+
and later) the module runs with the GIL off, and on any Python, parsing and
|
|
22
|
+
`css()` let go of it while they work.
|
|
23
|
+
- **Selectors selectolax 1.0.2 won't run**: `:lang()`, `:dir()`,
|
|
24
|
+
`:valid`, `:in-range`, `:scope` and more, which it rejects as
|
|
25
|
+
unparseable, and `:target` and `:local-link` once you pass the page's
|
|
26
|
+
`url`.
|
|
27
|
+
- **Limits** for pages you don't trust: `budget=` (seconds) and
|
|
28
|
+
`max_depth=`.
|
|
29
|
+
- **Nodes that outlive edits.** A removed node can still be read, and put
|
|
30
|
+
back.
|
|
31
|
+
- **XPath 1.0**, which selectolax doesn't have. `xpath()` gives back what
|
|
32
|
+
lxml's does, so code written for lxml or parsel reads the same:
|
|
33
|
+
|
|
34
|
+
```python
|
|
35
|
+
page.xpath("//li/text()") # ['one', 'two']
|
|
36
|
+
page.xpath("count(//li)") # 2.0
|
|
37
|
+
page.xpath("//a[@rel=$r]/@href", r="next")
|
|
38
|
+
```
|
|
39
|
+
|
|
40
|
+
Elements and comments come back as nodes, text and attribute values as
|
|
41
|
+
`str`, and a number, string or boolean as `float`, `str` or `bool`.
|
|
42
|
+
- **Where everything was written.** Parse with `positions=True` and every
|
|
43
|
+
node knows its place in your input: `node.source_range` and
|
|
44
|
+
`node.end_tag_range` are `(start, end)` indexes into what you passed
|
|
45
|
+
(code points into a `str`, bytes into `bytes`), so `src[start:end]` is
|
|
46
|
+
the tag as you wrote it; `node.position` is
|
|
47
|
+
its `(line, column)`, and `node.attribute_range("href")` finds one
|
|
48
|
+
attribute.
|
|
49
|
+
- **Parse errors.** With `errors=True`, `page.errors` lists each one with
|
|
50
|
+
the HTML Standard's name for it and where it is:
|
|
51
|
+
`2:1: unexpected-end-tag`.
|
|
52
|
+
- **Bytes in any encoding.** Hand `HTMLParser` the `bytes` you
|
|
53
|
+
downloaded and it decodes them as a browser would: a byte order mark
|
|
54
|
+
first, then the HTTP `charset`, which you pass as `transport_encoding`,
|
|
55
|
+
then a `<meta>`, then UTF-8 if the bytes are valid UTF-8, then
|
|
56
|
+
`default_encoding` (windows-1252 unless you say otherwise;
|
|
57
|
+
`encoding_for_locale("ja")` gives the one a Japanese browser uses). `page.encoding` says which it was, and
|
|
58
|
+
`page.encoding_confidence` whether it's `"certain"` or `"tentative"`.
|
|
59
|
+
Positions stay byte offsets into your bytes, whatever the encoding.
|
|
60
|
+
|
|
61
|
+
```python
|
|
62
|
+
with urllib.request.urlopen(url) as r:
|
|
63
|
+
page = HTMLParser(r.read(), transport_encoding=r.headers.get_content_charset())
|
|
64
|
+
page.encoding # 'Shift_JIS'
|
|
65
|
+
```
|
|
66
|
+
|
|
67
|
+
A `charset` that names no encoding is passed over, as browsers pass
|
|
68
|
+
it over.
|
|
69
|
+
|
|
70
|
+
`StreamParser` does the same as the bytes arrive, and you get the same
|
|
71
|
+
page however they're cut:
|
|
72
|
+
|
|
73
|
+
```python
|
|
74
|
+
with urllib.request.urlopen(url) as r:
|
|
75
|
+
s = StreamParser(transport_encoding=r.headers.get_content_charset())
|
|
76
|
+
while chunk := r.read(65536):
|
|
77
|
+
s.feed(chunk)
|
|
78
|
+
page = s.close() # an HTMLParser
|
|
79
|
+
```
|
|
80
|
+
|
|
81
|
+
The names are those of lxml's feed parser and the standard library's
|
|
82
|
+
`html.parser`; the keywords are html5lib's.
|
|
83
|
+
|
|
84
|
+
**A small change.** `bytes` used to be read as UTF-8 whatever they
|
|
85
|
+
were. UTF-8 still comes out as UTF-8, declared or not. Bytes that
|
|
86
|
+
aren't valid UTF-8 and don't say what they are used to get U+FFFD in
|
|
87
|
+
place of each bad byte; now they decode as windows-1252, as in a
|
|
88
|
+
browser. Pass `transport_encoding` if you know better.
|
|
89
|
+
|
|
90
|
+
## Installing
|
|
91
|
+
|
|
92
|
+
Parsec isn't released yet. Wheels go to PyPI as `parsec-html` with 0.1.0.
|
|
93
|
+
Until then, build it from git (you need Rust 1.88 or newer):
|
|
94
|
+
|
|
95
|
+
```sh
|
|
96
|
+
pip install "git+https://github.com/parsec-html/parsec#subdirectory=bindings/python"
|
|
97
|
+
```
|
|
98
|
+
|
|
99
|
+
## Moving from selectolax
|
|
100
|
+
|
|
101
|
+
Change the import, and most code runs as it was:
|
|
102
|
+
|
|
103
|
+
```python
|
|
104
|
+
# from selectolax.lexbor import LexborHTMLParser as HTMLParser
|
|
105
|
+
from parsec_html import HTMLParser
|
|
106
|
+
```
|
|
107
|
+
|
|
108
|
+
`HTMLParser(html, is_fragment=False, fragment_tag="div",
|
|
109
|
+
fragment_namespace="html", scripting=False)` takes what
|
|
110
|
+
`LexborHTMLParser` does (plus `budget`, `max_depth`, `url`, `positions`,
|
|
111
|
+
`errors`, `transport_encoding` and `default_encoding`), and the parser and its nodes have selectolax's
|
|
112
|
+
names: `root`, `head`, `body`, `html`, `css`, `css_first` (with `default`
|
|
113
|
+
and `strict`), `css_matches`, `any_css_matches`, `tags`, `text(deep,
|
|
114
|
+
separator, strip, skip_empty)`, `merge_text_nodes`, `strip_tags`,
|
|
115
|
+
`unwrap_tags`, `create_node`; on a node also `tag`, `id`, `attributes`,
|
|
116
|
+
`attrs`, `inner_html` (which you can set), `text_content`,
|
|
117
|
+
`comment_content`, `parent`, `next`, `prev`, `first_child`, `last_child`,
|
|
118
|
+
`iter`, `traverse`, `decompose`, `remove`, `unwrap`, `replace_with`,
|
|
119
|
+
`insert_before`, `insert_after`, `insert_child` (all four taking a string
|
|
120
|
+
as text, or a node to copy, or to move with `move=True`), and the
|
|
121
|
+
`is_*_node` checks.
|
|
122
|
+
|
|
123
|
+
Where it behaves differently:
|
|
124
|
+
|
|
125
|
+
| | selectolax 1.0.2 | parsec_html |
|
|
126
|
+
|---|---|---|
|
|
127
|
+
| `<a href>`'s `attributes` | `{'href': None}` | `{'href': ''}`: Parsec doesn't yet record whether a value was written |
|
|
128
|
+
| A selector that doesn't parse | `SelectolaxError` | `SelectorError`, a `ValueError` |
|
|
129
|
+
| A node after `decompose()` | freed | reads as empty (no parent, children, attributes or text, `html` is `""`, no selector matches) but keeps its `tag`; `xpath()` and putting it back raise `ValueError` |
|
|
130
|
+
| `remove()` | the same as `decompose()` | takes the node out but keeps it readable, to put back with `move=True` |
|
|
131
|
+
| `attrs.items()`, `keys()`, `values()` | iterators | lists |
|
|
132
|
+
| `bytes` in | UTF-8, or sniffed with `encoding=True` | sniffed as a browser sniffs them: byte order mark, `transport_encoding`, `<meta>`, then `default_encoding` |
|
|
133
|
+
| Class names | `LexborHTMLParser`, `LexborNode` | `HTMLParser`, `Node` |
|
|
134
|
+
|
|
135
|
+
Not there yet: `clone()`, `raw_html`, `raw_value`, `html_pretty`,
|
|
136
|
+
`select()` and its filters, `scripts_contain`, `script_srcs_contain`,
|
|
137
|
+
`template_fragments`, `mem_id`, `tag_id` and the `:lexbor-contains()`
|
|
138
|
+
pseudo-class.
|
|
139
|
+
|
|
140
|
+
## Speed
|
|
141
|
+
|
|
142
|
+
`bench.py` times both on the 13 real pages `bench/` uses (24 MB in all):
|
|
143
|
+
|
|
144
|
+
| | parsec_html | selectolax | |
|
|
145
|
+
|---|---:|---:|---:|
|
|
146
|
+
| Parse all 13 pages | 66 ms | 106 ms | 1.6× |
|
|
147
|
+
| The 32 selectors over all 13 | 283 ms | 781 ms | 2.8× |
|
|
148
|
+
| Every link's `href` and text | 43 ms | 50 ms | 1.2× |
|
|
149
|
+
|
|
150
|
+
Parsec parses each of the 13 pages faster, by 1.5 to 2.8 times, and runs
|
|
151
|
+
each of the 32 selectors faster, by 1.04 to 29 times
|
|
152
|
+
(`p:nth-child(2n+1)`). The narrowest margins are the selectors that
|
|
153
|
+
match most of a page, such as `*` and `[class]`, where the time goes on
|
|
154
|
+
making a Python object per match. Medians of 20 runs on an Apple M2 Max,
|
|
155
|
+
CPython 3.14, at a load of 3; a second run gave the same to within 6%.
|
|
156
|
+
|
|
157
|
+
## Working on it
|
|
158
|
+
|
|
159
|
+
```sh
|
|
160
|
+
cd bindings/python
|
|
161
|
+
pip install maturin pytest
|
|
162
|
+
maturin develop --release
|
|
163
|
+
pytest tests
|
|
164
|
+
python bench.py # after bench/fetch-pages.sh and pip install selectolax
|
|
165
|
+
```
|
|
166
|
+
|
|
167
|
+
The [Python page](https://github.com/parsec-html/parsec/wiki/Python) on
|
|
168
|
+
the wiki has the whole API.
|