parsec-html 0.1.0__tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (211) hide show
  1. parsec_html-0.1.0/Cargo.toml +14 -0
  2. parsec_html-0.1.0/PKG-INFO +186 -0
  3. parsec_html-0.1.0/README.md +168 -0
  4. parsec_html-0.1.0/bindings/python/Cargo.lock +610 -0
  5. parsec_html-0.1.0/bindings/python/Cargo.toml +39 -0
  6. parsec_html-0.1.0/bindings/python/README.md +168 -0
  7. parsec_html-0.1.0/bindings/python/bench.py +98 -0
  8. parsec_html-0.1.0/bindings/python/ci-requirements.in +4 -0
  9. parsec_html-0.1.0/bindings/python/ci-requirements.txt +133 -0
  10. parsec_html-0.1.0/bindings/python/parsec_html.pyi +213 -0
  11. parsec_html-0.1.0/bindings/python/src/lib.rs +1876 -0
  12. parsec_html-0.1.0/bindings/python/src/units.rs +133 -0
  13. parsec_html-0.1.0/bindings/python/tests/test_encoding.py +203 -0
  14. parsec_html-0.1.0/bindings/python/tests/test_parsec_html.py +476 -0
  15. parsec_html-0.1.0/crates/parsec/Cargo.toml +25 -0
  16. parsec_html-0.1.0/crates/parsec/LICENSE +201 -0
  17. parsec_html-0.1.0/crates/parsec/NOTICE +64 -0
  18. parsec_html-0.1.0/crates/parsec/README.md +511 -0
  19. parsec_html-0.1.0/crates/parsec/examples/hostile.rs +38 -0
  20. parsec_html-0.1.0/crates/parsec/examples/parallel.rs +32 -0
  21. parsec_html-0.1.0/crates/parsec/examples/parse.rs +19 -0
  22. parsec_html-0.1.0/crates/parsec/examples/select.rs +78 -0
  23. parsec_html-0.1.0/crates/parsec/examples/serialize.rs +21 -0
  24. parsec_html-0.1.0/crates/parsec/examples/tokenize.rs +41 -0
  25. parsec_html-0.1.0/crates/parsec/scripts/fetch-tree-tests.sh +15 -0
  26. parsec_html-0.1.0/crates/parsec/scripts/fetch-wpt-selectors.sh +33 -0
  27. parsec_html-0.1.0/crates/parsec/scripts/gen-entities.py +36 -0
  28. parsec_html-0.1.0/crates/parsec/scripts/gen-unicode.py +257 -0
  29. parsec_html-0.1.0/crates/parsec/scripts/wpt-selectors-cases.js +454 -0
  30. parsec_html-0.1.0/crates/parsec/src/css.rs +3925 -0
  31. parsec_html-0.1.0/crates/parsec/src/dom.rs +1137 -0
  32. parsec_html-0.1.0/crates/parsec/src/edit.rs +1086 -0
  33. parsec_html-0.1.0/crates/parsec/src/entities.rs +2239 -0
  34. parsec_html-0.1.0/crates/parsec/src/lib.rs +222 -0
  35. parsec_html-0.1.0/crates/parsec/src/pattern.rs +1518 -0
  36. parsec_html-0.1.0/crates/parsec/src/serialize.rs +1451 -0
  37. parsec_html-0.1.0/crates/parsec/src/shadow.rs +801 -0
  38. parsec_html-0.1.0/crates/parsec/src/tags.rs +232 -0
  39. parsec_html-0.1.0/crates/parsec/src/tokenizer.rs +2752 -0
  40. parsec_html-0.1.0/crates/parsec/src/tree.rs +4457 -0
  41. parsec_html-0.1.0/crates/parsec/src/unicode.rs +3177 -0
  42. parsec_html-0.1.0/crates/parsec/tests/chunks.rs +178 -0
  43. parsec_html-0.1.0/crates/parsec/tests/css.rs +1162 -0
  44. parsec_html-0.1.0/crates/parsec/tests/errors.rs +160 -0
  45. parsec_html-0.1.0/crates/parsec/tests/inputs/mod.rs +152 -0
  46. parsec_html-0.1.0/crates/parsec/tests/serializer.rs +252 -0
  47. parsec_html-0.1.0/crates/parsec/tests/shadow-dom/COMMIT +1 -0
  48. parsec_html-0.1.0/crates/parsec/tests/shadow-dom/LICENSE.md +11 -0
  49. parsec_html-0.1.0/crates/parsec/tests/shadow-dom/declarative.dat +691 -0
  50. parsec_html-0.1.0/crates/parsec/tests/shadow-dom/gethtml-ordering.html +69 -0
  51. parsec_html-0.1.0/crates/parsec/tests/shadow-dom.rs +671 -0
  52. parsec_html-0.1.0/crates/parsec/tests/spans.rs +260 -0
  53. parsec_html-0.1.0/crates/parsec/tests/summary/mod.rs +50 -0
  54. parsec_html-0.1.0/crates/parsec/tests/threads.rs +23 -0
  55. parsec_html-0.1.0/crates/parsec/tests/tokenizer.rs +335 -0
  56. parsec_html-0.1.0/crates/parsec/tests/tree-construction/COMMIT +1 -0
  57. parsec_html-0.1.0/crates/parsec/tests/tree-construction/LICENSE.md +11 -0
  58. parsec_html-0.1.0/crates/parsec/tests/tree-construction/adoption01.dat +354 -0
  59. parsec_html-0.1.0/crates/parsec/tests/tree-construction/adoption02.dat +72 -0
  60. parsec_html-0.1.0/crates/parsec/tests/tree-construction/blocks.dat +695 -0
  61. parsec_html-0.1.0/crates/parsec/tests/tree-construction/comments01.dat +214 -0
  62. parsec_html-0.1.0/crates/parsec/tests/tree-construction/doctype01.dat +474 -0
  63. parsec_html-0.1.0/crates/parsec/tests/tree-construction/domjs-unsafe.dat +0 -0
  64. parsec_html-0.1.0/crates/parsec/tests/tree-construction/entities01.dat +943 -0
  65. parsec_html-0.1.0/crates/parsec/tests/tree-construction/entities02.dat +309 -0
  66. parsec_html-0.1.0/crates/parsec/tests/tree-construction/foreign-fragment.dat +645 -0
  67. parsec_html-0.1.0/crates/parsec/tests/tree-construction/html5test-com.dat +365 -0
  68. parsec_html-0.1.0/crates/parsec/tests/tree-construction/inbody01.dat +54 -0
  69. parsec_html-0.1.0/crates/parsec/tests/tree-construction/isindex.dat +49 -0
  70. parsec_html-0.1.0/crates/parsec/tests/tree-construction/main-element.dat +46 -0
  71. parsec_html-0.1.0/crates/parsec/tests/tree-construction/math.dat +104 -0
  72. parsec_html-0.1.0/crates/parsec/tests/tree-construction/menuitem-element.dat +241 -0
  73. parsec_html-0.1.0/crates/parsec/tests/tree-construction/namespace-sensitivity.dat +22 -0
  74. parsec_html-0.1.0/crates/parsec/tests/tree-construction/noscript01.dat +237 -0
  75. parsec_html-0.1.0/crates/parsec/tests/tree-construction/pending-spec-changes-plain-text-unsafe.dat +0 -0
  76. parsec_html-0.1.0/crates/parsec/tests/tree-construction/pending-spec-changes.dat +46 -0
  77. parsec_html-0.1.0/crates/parsec/tests/tree-construction/plain-text-unsafe.dat +0 -0
  78. parsec_html-0.1.0/crates/parsec/tests/tree-construction/processing-instructions.dat +1132 -0
  79. parsec_html-0.1.0/crates/parsec/tests/tree-construction/quirks01.dat +53 -0
  80. parsec_html-0.1.0/crates/parsec/tests/tree-construction/ruby.dat +302 -0
  81. parsec_html-0.1.0/crates/parsec/tests/tree-construction/scriptdata01.dat +372 -0
  82. parsec_html-0.1.0/crates/parsec/tests/tree-construction/scripted_adoption01.dat +16 -0
  83. parsec_html-0.1.0/crates/parsec/tests/tree-construction/scripted_ark.dat +27 -0
  84. parsec_html-0.1.0/crates/parsec/tests/tree-construction/scripted_foster01.dat +21 -0
  85. parsec_html-0.1.0/crates/parsec/tests/tree-construction/scripted_webkit01.dat +30 -0
  86. parsec_html-0.1.0/crates/parsec/tests/tree-construction/search-element.dat +132 -0
  87. parsec_html-0.1.0/crates/parsec/tests/tree-construction/svg.dat +104 -0
  88. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tables01.dat +323 -0
  89. parsec_html-0.1.0/crates/parsec/tests/tree-construction/template.dat +1879 -0
  90. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests1.dat +1949 -0
  91. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests10.dat +853 -0
  92. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests11.dat +523 -0
  93. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests12.dat +62 -0
  94. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests14.dat +75 -0
  95. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests15.dat +216 -0
  96. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests16.dat +2602 -0
  97. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests17.dat +177 -0
  98. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests18.dat +572 -0
  99. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests19.dat +1456 -0
  100. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests2.dat +831 -0
  101. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests20.dat +842 -0
  102. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests21.dat +306 -0
  103. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests22.dat +190 -0
  104. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests23.dat +168 -0
  105. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests24.dat +79 -0
  106. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests25.dat +288 -0
  107. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests26.dat +453 -0
  108. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests3.dat +305 -0
  109. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests4.dat +80 -0
  110. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests5.dat +210 -0
  111. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests6.dat +663 -0
  112. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests7.dat +453 -0
  113. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests8.dat +165 -0
  114. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests9.dat +482 -0
  115. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tests_innerHTML_1.dat +844 -0
  116. parsec_html-0.1.0/crates/parsec/tests/tree-construction/tricky01.dat +336 -0
  117. parsec_html-0.1.0/crates/parsec/tests/tree-construction/void-in-phrasing.dat +151 -0
  118. parsec_html-0.1.0/crates/parsec/tests/tree-construction/webkit01.dat +783 -0
  119. parsec_html-0.1.0/crates/parsec/tests/tree-construction/webkit02.dat +775 -0
  120. parsec_html-0.1.0/crates/parsec/tests/tree.rs +280 -0
  121. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/COMMIT +1 -0
  122. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/LICENSE.md +11 -0
  123. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/cases.json +11353 -0
  124. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/class-prefix-selector-has-matching.tentative.html +92 -0
  125. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/class-prefix-selector-matching-quirks.tentative.html +36 -0
  126. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/class-prefix-selector-matching.tentative.html +114 -0
  127. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/class-prefix-selector-queryselector.tentative.html +42 -0
  128. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/dir-selector-querySelector.html +71 -0
  129. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/first-child.html +52 -0
  130. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/first-of-type.html +90 -0
  131. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/focus-visible-017-2.html +93 -0
  132. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/focus-visible-017.html +44 -0
  133. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/focus-visible-023.html +33 -0
  134. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/focus-visible-script-focus-001.html +54 -0
  135. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/forgiving-selector-list-comma-in-block.html +45 -0
  136. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/has-argument-with-explicit-scope.html +69 -0
  137. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/has-basic.html +81 -0
  138. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/has-relative-argument.html +184 -0
  139. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/is-where-basic.html +41 -0
  140. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/is-where-not.html +45 -0
  141. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/last-child.html +52 -0
  142. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/last-of-type.html +89 -0
  143. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/missing-right-token.html +31 -0
  144. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/not-complex.html +46 -0
  145. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/only-child.html +61 -0
  146. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/only-of-type.html +64 -0
  147. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/invalid-pseudos.html +25 -0
  148. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-attribute.html +30 -0
  149. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-child.html +11 -0
  150. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-class-prefix.tentative.html +50 -0
  151. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-class.html +15 -0
  152. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-descendant.html +13 -0
  153. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-focus-visible.html +13 -0
  154. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has-class-prefix.tentative.html +34 -0
  155. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has-disallow-nesting-has-inside-has.html +19 -0
  156. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has-forgiving-selector.html +15 -0
  157. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has-slotted.tentative.html +32 -0
  158. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-has.html +40 -0
  159. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-heading.html +38 -0
  160. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-id.html +13 -0
  161. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-is-where.html +59 -0
  162. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-is.html +18 -0
  163. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-not.html +37 -0
  164. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-part.html +44 -0
  165. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-sibling.html +12 -0
  166. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-slotted.html +28 -0
  167. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-state.html +35 -0
  168. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-universal.html +14 -0
  169. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/parsing/parse-where.html +18 -0
  170. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/query/query-is.html +57 -0
  171. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/query/query-where.html +57 -0
  172. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/selectors-4/lang-singleton-subtag-matching.html +78 -0
  173. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/css/selectors/webkit-pseudo-element.html +57 -0
  174. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/dom/nodes/Element-closest.html +73 -0
  175. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/dom/nodes/ParentNode-querySelector-All-content.html +377 -0
  176. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors/known-failures.txt +4 -0
  177. parsec_html-0.1.0/crates/parsec/tests/wpt-selectors.rs +557 -0
  178. parsec_html-0.1.0/crates/parsec-encoding/Cargo.toml +26 -0
  179. parsec_html-0.1.0/crates/parsec-encoding/LICENSE +201 -0
  180. parsec_html-0.1.0/crates/parsec-encoding/NOTICE +64 -0
  181. parsec_html-0.1.0/crates/parsec-encoding/README.md +511 -0
  182. parsec_html-0.1.0/crates/parsec-encoding/src/lib.rs +405 -0
  183. parsec_html-0.1.0/crates/parsec-encoding/src/offsets.rs +239 -0
  184. parsec_html-0.1.0/crates/parsec-encoding/src/prescan.rs +351 -0
  185. parsec_html-0.1.0/crates/parsec-encoding/src/stream.rs +380 -0
  186. parsec_html-0.1.0/crates/parsec-xpath/Cargo.toml +19 -0
  187. parsec_html-0.1.0/crates/parsec-xpath/LICENSE +201 -0
  188. parsec_html-0.1.0/crates/parsec-xpath/NOTICE +64 -0
  189. parsec_html-0.1.0/crates/parsec-xpath/README.md +511 -0
  190. parsec_html-0.1.0/crates/parsec-xpath/src/eval.rs +1461 -0
  191. parsec_html-0.1.0/crates/parsec-xpath/src/index.rs +251 -0
  192. parsec_html-0.1.0/crates/parsec-xpath/src/lexer.rs +274 -0
  193. parsec_html-0.1.0/crates/parsec-xpath/src/lib.rs +274 -0
  194. parsec_html-0.1.0/crates/parsec-xpath/src/parser.rs +719 -0
  195. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/COMMIT +1 -0
  196. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/Copyright +24 -0
  197. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/base +68 -0
  198. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/compare +184 -0
  199. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/equality +104 -0
  200. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/floats +252 -0
  201. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/functions +172 -0
  202. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/result/XPath/expr/strings +192 -0
  203. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/base +17 -0
  204. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/compare +46 -0
  205. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/equality +26 -0
  206. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/floats +63 -0
  207. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/functions +43 -0
  208. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2/test/XPath/expr/strings +50 -0
  209. parsec_html-0.1.0/crates/parsec-xpath/tests/libxml2.rs +134 -0
  210. parsec_html-0.1.0/crates/parsec-xpath/tests/xpath.rs +537 -0
  211. parsec_html-0.1.0/pyproject.toml +28 -0
@@ -0,0 +1,14 @@
1
+ [workspace]
2
+ resolver = "3"
3
+ members = ["crates/parsec", "crates/parsec-html", "crates/parsec-capi", "crates/parsec-css", "crates/parsec-encoding", "crates/parsec-url", "crates/parsec-xpath"]
4
+
5
+ [workspace.package]
6
+ version = "0.1.0"
7
+ edition = "2024"
8
+ license = "Apache-2.0"
9
+ repository = "https://github.com/parsec-html/parsec"
10
+ rust-version = "1.88"
11
+
12
+ [profile.release]
13
+ lto = "thin"
14
+ codegen-units = 1
@@ -0,0 +1,186 @@
1
+ Metadata-Version: 2.4
2
+ Name: parsec-html
3
+ Version: 0.1.0
4
+ Classifier: Development Status :: 3 - Alpha
5
+ Classifier: Programming Language :: Python :: 3
6
+ Classifier: Programming Language :: Python :: Free Threading :: 3 - Stable
7
+ Classifier: Programming Language :: Rust
8
+ Classifier: Topic :: Text Processing :: Markup :: HTML
9
+ Summary: A fast, spec-following HTML parser with CSS selectors and an editable tree, with a selectolax-style API.
10
+ Keywords: html,parser,html5,css-selectors,selectolax
11
+ License-Expression: Apache-2.0
12
+ Requires-Python: >=3.9
13
+ Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
14
+ Project-URL: Documentation, https://github.com/parsec-html/parsec/wiki/Python
15
+ Project-URL: Homepage, https://github.com/parsec-html/parsec
16
+ Project-URL: Issues, https://github.com/parsec-html/parsec/issues
17
+
18
+ # parsec-html for Python
19
+
20
+ Parsec is an HTML parser in Rust that reads pages the way a browser does,
21
+ finds what you want with CSS selectors, and lets you change the tree and
22
+ write it back out. This is its Python package, and if you've used
23
+ selectolax you already know the API:
24
+
25
+ ```python
26
+ from parsec_html import HTMLParser
27
+
28
+ page = HTMLParser("<ul><li class=hit>one<li>two</ul>")
29
+ assert page.css_first("li.hit").text() == "one"
30
+ for li in page.css("li"):
31
+ print(li.tag, li.attributes, li.html)
32
+ ```
33
+
34
+ What you get over selectolax:
35
+
36
+ - **Threads.** One parsed page can be read from as many threads as you
37
+ like, and edited from any of them. On free-threaded Python (3.14t
38
+ and later) the module runs with the GIL off, and on any Python, parsing and
39
+ `css()` let go of it while they work.
40
+ - **Selectors selectolax 1.0.2 won't run**: `:lang()`, `:dir()`,
41
+ `:valid`, `:in-range`, `:scope` and more, which it rejects as
42
+ unparseable, and `:target` and `:local-link` once you pass the page's
43
+ `url`.
44
+ - **Limits** for pages you don't trust: `budget=` (seconds) and
45
+ `max_depth=`.
46
+ - **Nodes that outlive edits.** A removed node can still be read, and put
47
+ back.
48
+ - **XPath 1.0**, which selectolax doesn't have. `xpath()` gives back what
49
+ lxml's does, so code written for lxml or parsel reads the same:
50
+
51
+ ```python
52
+ page.xpath("//li/text()") # ['one', 'two']
53
+ page.xpath("count(//li)") # 2.0
54
+ page.xpath("//a[@rel=$r]/@href", r="next")
55
+ ```
56
+
57
+ Elements and comments come back as nodes, text and attribute values as
58
+ `str`, and a number, string or boolean as `float`, `str` or `bool`.
59
+ - **Where everything was written.** Parse with `positions=True` and every
60
+ node knows its place in your input: `node.source_range` and
61
+ `node.end_tag_range` are `(start, end)` indexes into what you passed
62
+ (code points into a `str`, bytes into `bytes`), so `src[start:end]` is
63
+ the tag as you wrote it; `node.position` is
64
+ its `(line, column)`, and `node.attribute_range("href")` finds one
65
+ attribute.
66
+ - **Parse errors.** With `errors=True`, `page.errors` lists each one with
67
+ the HTML Standard's name for it and where it is:
68
+ `2:1: unexpected-end-tag`.
69
+ - **Bytes in any encoding.** Hand `HTMLParser` the `bytes` you
70
+ downloaded and it decodes them as a browser would: a byte order mark
71
+ first, then the HTTP `charset`, which you pass as `transport_encoding`,
72
+ then a `<meta>`, then UTF-8 if the bytes are valid UTF-8, then
73
+ `default_encoding` (windows-1252 unless you say otherwise;
74
+ `encoding_for_locale("ja")` gives the one a Japanese browser uses). `page.encoding` says which it was, and
75
+ `page.encoding_confidence` whether it's `"certain"` or `"tentative"`.
76
+ Positions stay byte offsets into your bytes, whatever the encoding.
77
+
78
+ ```python
79
+ with urllib.request.urlopen(url) as r:
80
+ page = HTMLParser(r.read(), transport_encoding=r.headers.get_content_charset())
81
+ page.encoding # 'Shift_JIS'
82
+ ```
83
+
84
+ A `charset` that names no encoding is passed over, as browsers pass
85
+ it over.
86
+
87
+ `StreamParser` does the same as the bytes arrive, and you get the same
88
+ page however they're cut:
89
+
90
+ ```python
91
+ with urllib.request.urlopen(url) as r:
92
+ s = StreamParser(transport_encoding=r.headers.get_content_charset())
93
+ while chunk := r.read(65536):
94
+ s.feed(chunk)
95
+ page = s.close() # an HTMLParser
96
+ ```
97
+
98
+ The names are those of lxml's feed parser and the standard library's
99
+ `html.parser`; the keywords are html5lib's.
100
+
101
+ **A small change.** `bytes` used to be read as UTF-8 whatever they
102
+ were. UTF-8 still comes out as UTF-8, declared or not. Bytes that
103
+ aren't valid UTF-8 and don't say what they are used to get U+FFFD in
104
+ place of each bad byte; now they decode as windows-1252, as in a
105
+ browser. Pass `transport_encoding` if you know better.
106
+
107
+ ## Installing
108
+
109
+ Parsec isn't released yet. Wheels go to PyPI as `parsec-html` with 0.1.0.
110
+ Until then, build it from git (you need Rust 1.88 or newer):
111
+
112
+ ```sh
113
+ pip install "git+https://github.com/parsec-html/parsec#subdirectory=bindings/python"
114
+ ```
115
+
116
+ ## Moving from selectolax
117
+
118
+ Change the import, and most code runs as it was:
119
+
120
+ ```python
121
+ # from selectolax.lexbor import LexborHTMLParser as HTMLParser
122
+ from parsec_html import HTMLParser
123
+ ```
124
+
125
+ `HTMLParser(html, is_fragment=False, fragment_tag="div",
126
+ fragment_namespace="html", scripting=False)` takes what
127
+ `LexborHTMLParser` does (plus `budget`, `max_depth`, `url`, `positions`,
128
+ `errors`, `transport_encoding` and `default_encoding`), and the parser and its nodes have selectolax's
129
+ names: `root`, `head`, `body`, `html`, `css`, `css_first` (with `default`
130
+ and `strict`), `css_matches`, `any_css_matches`, `tags`, `text(deep,
131
+ separator, strip, skip_empty)`, `merge_text_nodes`, `strip_tags`,
132
+ `unwrap_tags`, `create_node`; on a node also `tag`, `id`, `attributes`,
133
+ `attrs`, `inner_html` (which you can set), `text_content`,
134
+ `comment_content`, `parent`, `next`, `prev`, `first_child`, `last_child`,
135
+ `iter`, `traverse`, `decompose`, `remove`, `unwrap`, `replace_with`,
136
+ `insert_before`, `insert_after`, `insert_child` (all four taking a string
137
+ as text, or a node to copy, or to move with `move=True`), and the
138
+ `is_*_node` checks.
139
+
140
+ Where it behaves differently:
141
+
142
+ | | selectolax 1.0.2 | parsec_html |
143
+ |---|---|---|
144
+ | `<a href>`'s `attributes` | `{'href': None}` | `{'href': ''}`: Parsec doesn't yet record whether a value was written |
145
+ | A selector that doesn't parse | `SelectolaxError` | `SelectorError`, a `ValueError` |
146
+ | A node after `decompose()` | freed | reads as empty (no parent, children, attributes or text, `html` is `""`, no selector matches) but keeps its `tag`; `xpath()` and putting it back raise `ValueError` |
147
+ | `remove()` | the same as `decompose()` | takes the node out but keeps it readable, to put back with `move=True` |
148
+ | `attrs.items()`, `keys()`, `values()` | iterators | lists |
149
+ | `bytes` in | UTF-8, or sniffed with `encoding=True` | sniffed as a browser sniffs them: byte order mark, `transport_encoding`, `<meta>`, then `default_encoding` |
150
+ | Class names | `LexborHTMLParser`, `LexborNode` | `HTMLParser`, `Node` |
151
+
152
+ Not there yet: `clone()`, `raw_html`, `raw_value`, `html_pretty`,
153
+ `select()` and its filters, `scripts_contain`, `script_srcs_contain`,
154
+ `template_fragments`, `mem_id`, `tag_id` and the `:lexbor-contains()`
155
+ pseudo-class.
156
+
157
+ ## Speed
158
+
159
+ `bench.py` times both on the 13 real pages `bench/` uses (24 MB in all):
160
+
161
+ | | parsec_html | selectolax | |
162
+ |---|---:|---:|---:|
163
+ | Parse all 13 pages | 66 ms | 106 ms | 1.6× |
164
+ | The 32 selectors over all 13 | 283 ms | 781 ms | 2.8× |
165
+ | Every link's `href` and text | 43 ms | 50 ms | 1.2× |
166
+
167
+ Parsec parses each of the 13 pages faster, by 1.5 to 2.8 times, and runs
168
+ each of the 32 selectors faster, by 1.04 to 29 times
169
+ (`p:nth-child(2n+1)`). The narrowest margins are the selectors that
170
+ match most of a page, such as `*` and `[class]`, where the time goes on
171
+ making a Python object per match. Medians of 20 runs on an Apple M2 Max,
172
+ CPython 3.14, at a load of 3; a second run gave the same to within 6%.
173
+
174
+ ## Working on it
175
+
176
+ ```sh
177
+ cd bindings/python
178
+ pip install maturin pytest
179
+ maturin develop --release
180
+ pytest tests
181
+ python bench.py # after bench/fetch-pages.sh and pip install selectolax
182
+ ```
183
+
184
+ The [Python page](https://github.com/parsec-html/parsec/wiki/Python) on
185
+ the wiki has the whole API.
186
+
@@ -0,0 +1,168 @@
1
+ # parsec-html for Python
2
+
3
+ Parsec is an HTML parser in Rust that reads pages the way a browser does,
4
+ finds what you want with CSS selectors, and lets you change the tree and
5
+ write it back out. This is its Python package, and if you've used
6
+ selectolax you already know the API:
7
+
8
+ ```python
9
+ from parsec_html import HTMLParser
10
+
11
+ page = HTMLParser("<ul><li class=hit>one<li>two</ul>")
12
+ assert page.css_first("li.hit").text() == "one"
13
+ for li in page.css("li"):
14
+ print(li.tag, li.attributes, li.html)
15
+ ```
16
+
17
+ What you get over selectolax:
18
+
19
+ - **Threads.** One parsed page can be read from as many threads as you
20
+ like, and edited from any of them. On free-threaded Python (3.14t
21
+ and later) the module runs with the GIL off, and on any Python, parsing and
22
+ `css()` let go of it while they work.
23
+ - **Selectors selectolax 1.0.2 won't run**: `:lang()`, `:dir()`,
24
+ `:valid`, `:in-range`, `:scope` and more, which it rejects as
25
+ unparseable, and `:target` and `:local-link` once you pass the page's
26
+ `url`.
27
+ - **Limits** for pages you don't trust: `budget=` (seconds) and
28
+ `max_depth=`.
29
+ - **Nodes that outlive edits.** A removed node can still be read, and put
30
+ back.
31
+ - **XPath 1.0**, which selectolax doesn't have. `xpath()` gives back what
32
+ lxml's does, so code written for lxml or parsel reads the same:
33
+
34
+ ```python
35
+ page.xpath("//li/text()") # ['one', 'two']
36
+ page.xpath("count(//li)") # 2.0
37
+ page.xpath("//a[@rel=$r]/@href", r="next")
38
+ ```
39
+
40
+ Elements and comments come back as nodes, text and attribute values as
41
+ `str`, and a number, string or boolean as `float`, `str` or `bool`.
42
+ - **Where everything was written.** Parse with `positions=True` and every
43
+ node knows its place in your input: `node.source_range` and
44
+ `node.end_tag_range` are `(start, end)` indexes into what you passed
45
+ (code points into a `str`, bytes into `bytes`), so `src[start:end]` is
46
+ the tag as you wrote it; `node.position` is
47
+ its `(line, column)`, and `node.attribute_range("href")` finds one
48
+ attribute.
49
+ - **Parse errors.** With `errors=True`, `page.errors` lists each one with
50
+ the HTML Standard's name for it and where it is:
51
+ `2:1: unexpected-end-tag`.
52
+ - **Bytes in any encoding.** Hand `HTMLParser` the `bytes` you
53
+ downloaded and it decodes them as a browser would: a byte order mark
54
+ first, then the HTTP `charset`, which you pass as `transport_encoding`,
55
+ then a `<meta>`, then UTF-8 if the bytes are valid UTF-8, then
56
+ `default_encoding` (windows-1252 unless you say otherwise;
57
+ `encoding_for_locale("ja")` gives the one a Japanese browser uses). `page.encoding` says which it was, and
58
+ `page.encoding_confidence` whether it's `"certain"` or `"tentative"`.
59
+ Positions stay byte offsets into your bytes, whatever the encoding.
60
+
61
+ ```python
62
+ with urllib.request.urlopen(url) as r:
63
+ page = HTMLParser(r.read(), transport_encoding=r.headers.get_content_charset())
64
+ page.encoding # 'Shift_JIS'
65
+ ```
66
+
67
+ A `charset` that names no encoding is passed over, as browsers pass
68
+ it over.
69
+
70
+ `StreamParser` does the same as the bytes arrive, and you get the same
71
+ page however they're cut:
72
+
73
+ ```python
74
+ with urllib.request.urlopen(url) as r:
75
+ s = StreamParser(transport_encoding=r.headers.get_content_charset())
76
+ while chunk := r.read(65536):
77
+ s.feed(chunk)
78
+ page = s.close() # an HTMLParser
79
+ ```
80
+
81
+ The names are those of lxml's feed parser and the standard library's
82
+ `html.parser`; the keywords are html5lib's.
83
+
84
+ **A small change.** `bytes` used to be read as UTF-8 whatever they
85
+ were. UTF-8 still comes out as UTF-8, declared or not. Bytes that
86
+ aren't valid UTF-8 and don't say what they are used to get U+FFFD in
87
+ place of each bad byte; now they decode as windows-1252, as in a
88
+ browser. Pass `transport_encoding` if you know better.
89
+
90
+ ## Installing
91
+
92
+ Parsec isn't released yet. Wheels go to PyPI as `parsec-html` with 0.1.0.
93
+ Until then, build it from git (you need Rust 1.88 or newer):
94
+
95
+ ```sh
96
+ pip install "git+https://github.com/parsec-html/parsec#subdirectory=bindings/python"
97
+ ```
98
+
99
+ ## Moving from selectolax
100
+
101
+ Change the import, and most code runs as it was:
102
+
103
+ ```python
104
+ # from selectolax.lexbor import LexborHTMLParser as HTMLParser
105
+ from parsec_html import HTMLParser
106
+ ```
107
+
108
+ `HTMLParser(html, is_fragment=False, fragment_tag="div",
109
+ fragment_namespace="html", scripting=False)` takes what
110
+ `LexborHTMLParser` does (plus `budget`, `max_depth`, `url`, `positions`,
111
+ `errors`, `transport_encoding` and `default_encoding`), and the parser and its nodes have selectolax's
112
+ names: `root`, `head`, `body`, `html`, `css`, `css_first` (with `default`
113
+ and `strict`), `css_matches`, `any_css_matches`, `tags`, `text(deep,
114
+ separator, strip, skip_empty)`, `merge_text_nodes`, `strip_tags`,
115
+ `unwrap_tags`, `create_node`; on a node also `tag`, `id`, `attributes`,
116
+ `attrs`, `inner_html` (which you can set), `text_content`,
117
+ `comment_content`, `parent`, `next`, `prev`, `first_child`, `last_child`,
118
+ `iter`, `traverse`, `decompose`, `remove`, `unwrap`, `replace_with`,
119
+ `insert_before`, `insert_after`, `insert_child` (all four taking a string
120
+ as text, or a node to copy, or to move with `move=True`), and the
121
+ `is_*_node` checks.
122
+
123
+ Where it behaves differently:
124
+
125
+ | | selectolax 1.0.2 | parsec_html |
126
+ |---|---|---|
127
+ | `<a href>`'s `attributes` | `{'href': None}` | `{'href': ''}`: Parsec doesn't yet record whether a value was written |
128
+ | A selector that doesn't parse | `SelectolaxError` | `SelectorError`, a `ValueError` |
129
+ | A node after `decompose()` | freed | reads as empty (no parent, children, attributes or text, `html` is `""`, no selector matches) but keeps its `tag`; `xpath()` and putting it back raise `ValueError` |
130
+ | `remove()` | the same as `decompose()` | takes the node out but keeps it readable, to put back with `move=True` |
131
+ | `attrs.items()`, `keys()`, `values()` | iterators | lists |
132
+ | `bytes` in | UTF-8, or sniffed with `encoding=True` | sniffed as a browser sniffs them: byte order mark, `transport_encoding`, `<meta>`, then `default_encoding` |
133
+ | Class names | `LexborHTMLParser`, `LexborNode` | `HTMLParser`, `Node` |
134
+
135
+ Not there yet: `clone()`, `raw_html`, `raw_value`, `html_pretty`,
136
+ `select()` and its filters, `scripts_contain`, `script_srcs_contain`,
137
+ `template_fragments`, `mem_id`, `tag_id` and the `:lexbor-contains()`
138
+ pseudo-class.
139
+
140
+ ## Speed
141
+
142
+ `bench.py` times both on the 13 real pages `bench/` uses (24 MB in all):
143
+
144
+ | | parsec_html | selectolax | |
145
+ |---|---:|---:|---:|
146
+ | Parse all 13 pages | 66 ms | 106 ms | 1.6× |
147
+ | The 32 selectors over all 13 | 283 ms | 781 ms | 2.8× |
148
+ | Every link's `href` and text | 43 ms | 50 ms | 1.2× |
149
+
150
+ Parsec parses each of the 13 pages faster, by 1.5 to 2.8 times, and runs
151
+ each of the 32 selectors faster, by 1.04 to 29 times
152
+ (`p:nth-child(2n+1)`). The narrowest margins are the selectors that
153
+ match most of a page, such as `*` and `[class]`, where the time goes on
154
+ making a Python object per match. Medians of 20 runs on an Apple M2 Max,
155
+ CPython 3.14, at a load of 3; a second run gave the same to within 6%.
156
+
157
+ ## Working on it
158
+
159
+ ```sh
160
+ cd bindings/python
161
+ pip install maturin pytest
162
+ maturin develop --release
163
+ pytest tests
164
+ python bench.py # after bench/fetch-pages.sh and pip install selectolax
165
+ ```
166
+
167
+ The [Python page](https://github.com/parsec-html/parsec/wiki/Python) on
168
+ the wiki has the whole API.