justhtml 3.6.0__tar.gz → 3.7.0__tar.gz

{justhtml-3.6.0 → justhtml-3.7.0}/CHANGELOG.md

@@ -5,7 +5,26 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
-## [Unreleased]
+## [3.7.0] - 2026-07-15
+
+### Fixed
+
+- Keep templates open in the head when they contain a processing instruction, matching the latest web-platform-tests tree-construction fixture.
+
+### Changed
+
+- Refresh the web-platform-tests tree-construction fixtures and comparison scores.
+- Make the correctness benchmark skip an optional `html5-parser` installation when its libxml2 ABI conflicts with `lxml`.
+
+## [3.6.1] - 2026-07-13
+
+### Fixed
+
+- Keep processing instructions inside a scripting-disabled head `noscript`, and process them in body mode inside templates as required by the latest WHATWG parsing rules.
+
+### Changed
+
+- Move raw parser input/output regressions from Python unit tests into the shared tree-construction fixture format.
 
 ## [3.6.0] - 2026-07-13
 

{justhtml-3.6.0 → justhtml-3.7.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: justhtml
-Version: 3.6.0
+Version: 3.7.0
 Summary: A pure Python HTML5 parser that just works.
 Project-URL: Homepage, https://github.com/emilstenstrom/justhtml
 Project-URL: Issues, https://github.com/emilstenstrom/justhtml/issues
@@ -40,7 +40,7 @@ Requires-Dist: html5-parser; extra == 'benchmark'
 Requires-Dist: html5lib; extra == 'benchmark'
 Requires-Dist: lxml; extra == 'benchmark'
 Requires-Dist: markupever; extra == 'benchmark'
-Requires-Dist: selectolax>=0.4.8; extra == 'benchmark'
+Requires-Dist: selectolax>=0.4.11; extra == 'benchmark'
 Requires-Dist: turbohtml>=1.3.1; extra == 'benchmark'
 Requires-Dist: zstandard; extra == 'benchmark'
 Provides-Extra: dev

{justhtml-3.6.0 → justhtml-3.7.0}/benchmarks/correctness.py

@@ -148,7 +148,7 @@ def check_parser_available(parser_name):
             import html5_parser  # noqa: F401
 
             return True
-        except ImportError:
+        except (ImportError, RuntimeError):
             return False
     if parser_name == "markupever":
         try:

{justhtml-3.6.0 → justhtml-3.7.0}/docs/comparison.md

@@ -11,22 +11,22 @@ Use a different tool when one narrow requirement matters more than the whole pip
 | Tool | HTML5 parsing [1] | Speed | Query | Build | Sanitize | Notes |
 |------|------------------------------------------|-------|----------|-------|------------------|-------|
 | **JustHTML**<br>Pure Python | ✅ 100% | ⚡ Fast | ✅ CSS selectors | ✅ `element()` | ✅ Built-in | Correct, secure, easy to install, and fast enough. |
-| **`selectolax`**<br>Python wrapper of C-based Lexbor | 🟡 95.4% | 🚀 Very Fast | ✅ CSS selectors | ✅ `create_node()` | ❌ Needs sanitization | Very fast. |
-| **Chromium**<br>browser engine | 🟡 95.1% [2] | 🚀 Very Fast | — | — | — | Current browser-harness result. |
-| **`turbohtml`**<br>Python wrapper of a C core | 🟡 94.3% | 🚀 Very Fast | ✅ CSS selectors, XPath | ✅ `E.*` builder | ✅ Built-in | Broad, compiled alternative with parsing, querying, and sanitization. |
-| **WebKit**<br>browser engine | 🟡 94.0% [2] | 🚀 Very Fast | — | — | — | Current browser-harness result. |
-| **Firefox**<br>browser engine | 🟡 93.2% [2] | 🚀 Very Fast | — | — | — | Current browser-harness result. |
+| **`selectolax`**<br>Python wrapper of C-based Lexbor | ✅ 100% | 🚀 Very Fast | ✅ CSS selectors | ✅ `create_node()` | ❌ Needs sanitization | Very fast. |
+| **Chromium**<br>browser engine | 🟡 95.0% [2] | 🚀 Very Fast | — | — | — | Current browser-harness result. |
+| **`turbohtml`**<br>Python wrapper of a C core | 🟡 94.2% | 🚀 Very Fast | ✅ CSS selectors, XPath | ✅ `E.*` builder | ✅ Built-in | Broad, compiled alternative with parsing, querying, and sanitization. |
+| **WebKit**<br>browser engine | 🟡 93.9% [2] | 🚀 Very Fast | — | — | — | Current browser-harness result. |
+| **Firefox**<br>browser engine | 🟡 93.1% [2] | 🚀 Very Fast | — | — | — | Current browser-harness result. |
 | **`html5lib`**<br>Pure Python | 🟡 82.3% | 🐢 Slow | 🟡 XPath (lxml) | 🟡 Tree API | 🔴 [Deprecated](https://github.com/html5lib/html5lib-python/issues/443) | Unmaintained reference implementation; incomplete coverage of the tree-construction fixtures. |
-| **`markupever`**<br>Python wrapper of Rust-based html5ever | 🟡 79.4% | 🚀 Very Fast | ✅ CSS selectors | ✅ `TreeDom .create_*()` | ❌ Needs sanitization | Fast, but 105 fixture cases abort its current parser process. |
+| **`markupever`**<br>Python wrapper of Rust-based html5ever | 🟡 79.4% | 🚀 Very Fast | ✅ CSS selectors | ✅ `TreeDom .create_*()` | ❌ Needs sanitization | Fast, but many fixtures cases abort its current parser process. |
 | **`html5_parser`**<br>Python wrapper of C-based Gumbo | 🔴 47.8% | 🚀 Very Fast | 🟡 XPath (lxml) | 🟡 `etree` (lxml) | ❌ Needs sanitization | Fast, but its public tree API loses information needed by many fixtures. |
 | **`BeautifulSoup`**<br>Pure Python | 🔴 0.3% (default) | 🐢 Slow | 🟡 Custom API | ✅ `new_tag()` API | ❌ Needs sanitization | Wraps `html.parser` (default). Can use lxml or html5lib. |
 | **`html.parser`**<br>Python stdlib | 🔴 0.3% | ⚡ Fast | ❌ None | ❌ None | ❌ Needs sanitization | Standard library. Chokes on malformed HTML. |
-| **`lxml`**<br>Python wrapper of C-based libxml2 | 🔴 0.3% | 🚀 Very Fast | 🟡 XPath | ✅ `etree` / E-factory | ❌ Needs sanitization | Fast but not HTML5 compliant. Context-fragment cases are skipped; supported cases still perform poorly. Don't use the old lxml.html.clean module! |
+| **`lxml`**<br>Python wrapper of C-based libxml2 | 🔴 0.9% | 🚀 Very Fast | 🟡 XPath | ✅ `etree` / E-factory | ❌ Needs sanitization | Fast but not HTML5 compliant. Context-fragment cases are skipped; supported cases still perform poorly. Don't use the old lxml.html.clean module! |
 
-[1]: Correctness scores are counted against the [web-platform-tests tree-construction tests](https://github.com/web-platform-tests/wpt/tree/master/html/syntax/parsing/resources). They were run locally at [2026-07-11](https://github.com/web-platform-tests/wpt/commit/ba458f52c8a848bbd9c525ac98fb1f7b30e41dc2).
+[1]: Library correctness scores are counted against the [web-platform-tests tree-construction tests](https://github.com/web-platform-tests/wpt/tree/master/html/syntax/parsing/resources). They were run locally at [2026-07-15](https://github.com/web-platform-tests/wpt/commit/4830edb033cb486fd0cd6f85b5e937cfc718704d) and cover 1,880 cases after excluding scripting-directed fixtures. JustHTML and Selectolax 0.4.11 passed all 1,880 cases; MarkupEver's score counts its 103 process-aborting cases as errors.
 
 
-[2]: Current local rerun with [`justhtml-html5lib-tests-bench`](https://github.com/EmilStenstrom/justhtml-html5lib-tests-bench) against WPT commit [`ba458f52`](https://github.com/web-platform-tests/wpt/commit/ba458f52c8a848bbd9c525ac98fb1f7b30e41dc2). The browser harness skips 12 `#script-on` cases but includes `#script-off` cases, so these scores are not directly comparable to the 1,879-case Python-parser scores above.
+[2]: Current local rerun with [`justhtml-html5lib-tests-bench`](https://github.com/EmilStenstrom/justhtml-html5lib-tests-bench) against WPT commit [`4830edb`](https://github.com/web-platform-tests/wpt/commit/4830edb033cb486fd0cd6f85b5e937cfc718704d). Chromium 143.0.7499.4, WebKit 26.0, and Firefox 144.0.2 were compared against 1,908 cases; the harness skips 12 `#script-on` cases but includes `#script-off` cases, so these scores are not directly comparable to the 1,880-case Python-parser scores above.
 
 ## Why JustHTML
 

{justhtml-3.6.0 → justhtml-3.7.0}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "justhtml"
 authors = [{ name = "Emil Stenström", email = "emil@emilstenstrom.se" }]
-version = "3.6.0"
+version = "3.7.0"
 description = "A pure Python HTML5 parser that just works."
 readme = "README.md"
 license = { file = "LICENSE" }
@@ -18,7 +18,7 @@ benchmark = [
     "html5lib",
     "lxml",
     "beautifulsoup4",
-    "selectolax>=0.4.8",
+    "selectolax>=0.4.11",
     "html5-parser",
     "markupever",
     "turbohtml>=1.3.1",

{justhtml-3.6.0 → justhtml-3.7.0}/src/justhtml/parser/engine.py

@@ -1632,7 +1632,7 @@ class ParseEngine:
                     append_text(html[lt:end] if gt == -1 else html[lt : gt + 1], lt)
                     pos = end if gt == -1 else gt + 1
                     continue
-                if self._raw_mode and len(self._stack) > 1 and self._stack[-1].name in _RAWTEXT_ELEMENT_TAGS:
+                if self._raw_mode and len(self._stack) > 1 and self._stack[-1].name in self._rawtext_element_tags:
                     comment_end = end if gt == -1 else gt
                     append_text(html[lt:comment_end] if gt == -1 else html[lt : gt + 1], lt)
                     pos = end if gt == -1 else gt + 1

justhtml-3.7.0/tests/justhtml-tests/character_preprocessing.dat

@@ -0,0 +1,252 @@
+#data
+<math><mi>\x00</mi></math>
+#errors
+(1,6): expected-doctype-but-got-start-tag
+(1,11): unexpected-null-character
+
+#document
+| <html>
+|   <head>
+|   <body>
+|     <math math>
+|       <math mi>
+
+#data
+<math><mi>\x00\x0c</mi></math>
+#errors
+(1,6): expected-doctype-but-got-start-tag
+(1,11): unexpected-null-character
+
+#document
+| <html>
+|   <head>
+|   <body>
+|     <math math>
+|       <math mi>
+|         ""
+
+#data
+<math><mi>   </mi></math>
+#errors
+(1,1): expected-doctype-but-got-start-tag
+
+#document
+| <html>
+|   <head>
+|   <body>
+|     <math math>
+|       <math mi>
+|         "   "
+
+#data
+<math><mi>\x0ctext</mi></math>
+#errors
+(1,1): expected-doctype-but-got-start-tag
+
+#document
+| <html>
+|   <head>
+|   <body>
+|     <math math>
+|       <math mi>
+|         "text"
+
+#data
+<select>\x0ctext</select>
+#errors
+(1,1): expected-doctype-but-got-start-tag
+
+#document
+| <html>
+|   <head>
+|   <body>
+|     <select>
+|       "text"
+
+#data
+<table>\x00</table>
+#errors
+(1,7): expected-doctype-but-got-start-tag
+(1,8): unexpected-null-character
+
+#document
+| <html>
+|   <head>
+|   <body>
+|     <table>
+
+#data
+<head></head>\x00
+#errors
+(1,6): expected-doctype-but-got-start-tag
+(1,14): unexpected-null-character
+
+#document
+| <html>
+|   <head>
+|   <body>
+
+#data
+<table>\x0c</table>
+#errors
+(1,1): expected-doctype-but-got-start-tag
+
+#document
+| <html>
+|   <head>
+|   <body>
+|     <table>
+|       ""
+
+#data
+<!doctype html>\x00
+#errors
+(1,16): unexpected-null-character
+
+#document
+| <!DOCTYPE html>
+| <html>
+|   <head>
+|   <body>
+
+
+#data
+<!--a\x00b-->
+#errors
+(1,1): expected-doctype-but-got-chars
+(1,6): unexpected-null-character
+#document
+| <!-- a�b -->
+| <html>
+|   <head>
+|   <body>
+
+#data
+<frameset>\x0d\x0a
+#errors
+#document
+| <html>
+|   <head>
+|   <frameset>
+|     "
+"
+#data
+<!--a-
+#errors
+(1,1): expected-doctype-but-got-chars
+(1,6): eof-in-comment
+#document
+| <!-- a -->
+| <html>
+|   <head>
+|   <body>
+
+#data
+<!x\x0d\x0ay>
+#errors
+(1,1): expected-doctype-but-got-chars
+#document
+| <!-- x
+y -->
+| <html>
+|   <head>
+|   <body>
+
+#data
+<\x00div>
+#errors
+(1,1): expected-doctype-but-got-chars
+(1,2): unexpected-null-character
+#document
+| <html>
+|   <head>
+|   <body>
+|     "<�div>"
+
+#data
+</>
+#errors
+(1,1): expected-doctype-but-got-chars
+#document
+| <html>
+|   <head>
+|   <body>
+
+#data
+\x00\x0cA
+#errors
+(1,1): unexpected-null-character
+(1,1): expected-doctype-but-got-chars
+#document
+| <html>
+|   <head>
+|   <body>
+|     "A"
+
+#data
+<!DOCTYPE\x00html>
+#errors
+(1,10): unexpected-null-character
+(1,15): unknown-doctype
+#document
+| <!DOCTYPE �html>
+| <html>
+|   <head>
+|   <body>
+
+#data
+&#x;
+#errors
+(1,1): expected-doctype-but-got-chars
+#document
+| <html>
+|   <head>
+|   <body>
+|     "&#x;"
+
+#data
+<!--first--><!DOCTYPE\x0d\x0ahtml>
+#errors
+(1,1): expected-doctype-but-got-chars
+#document
+| <!-- first -->
+| <!DOCTYPE html>
+| <html>
+|   <head>
+|   <body>
+
+#data
+<o\x0ce="\x0d">
+#errors
+(1,9): expected-doctype-but-got-start-tag
+#document
+| <html>
+|   <head>
+|   <body>
+|     <o>
+|       e="
+"
+
+#data
+<d 2="\x0d">
+#errors
+(1,9): expected-doctype-but-got-start-tag
+#document
+| <html>
+|   <head>
+|   <body>
+|     <d>
+|       2="
+"
+
+#data
+<a 8=&#13\x0d>
+#errors
+(1,11): expected-doctype-but-got-start-tag
+#document
+| <html>
+|   <head>
+|   <body>
+|     <a>
+|       8="
+"