justhtml 3.0.0__tar.gz → 3.2.0__tar.gz

{justhtml-3.0.0 → justhtml-3.2.0}/.github/workflows/ci.yml

@@ -36,20 +36,21 @@ jobs:
       - name: Checkout html5lib-tests
         run: |
           cd ..
-          git clone https://github.com/html5lib/html5lib-tests.git
+          git clone --no-checkout https://github.com/html5lib/html5lib-tests.git
+          git -C html5lib-tests fetch origin 9fb614afaa42ce8787840f057b32084308e76549
+          git -C html5lib-tests checkout 9fb614afaa42ce8787840f057b32084308e76549
           git -C html5lib-tests rev-parse HEAD
 
-      - name: Setup test symlinks
+      - name: Setup test fixtures
         run: |
-          cd tests
-          ln -s ../../html5lib-tests/tokenizer html5lib-tests-tokenizer
-          ln -s ../../html5lib-tests/tree-construction html5lib-tests-tree
-          ln -s ../../html5lib-tests/serializer html5lib-tests-serializer
-          ln -s ../../html5lib-tests/encoding html5lib-tests-encoding
+          cp -a ../html5lib-tests/tokenizer tests/html5lib-tests-tokenizer
+          cp -a ../html5lib-tests/tree-construction tests/html5lib-tests-tree
+          cp -a ../html5lib-tests/serializer tests/html5lib-tests-serializer
+          cp -a ../html5lib-tests/encoding tests/html5lib-tests-encoding
 
       - name: Run pre-commit
         id: precommit
-        uses: pre-commit/action@v3.0.1
+        run: pre-commit run --show-diff-on-failure --color=always --all-files
         env:
           SKIP: mypy
 

{justhtml-3.0.0 → justhtml-3.2.0}/CHANGELOG.md

@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [3.2.0] - 2026-07-04
+
+### Changed
+- Use the web platform html5 treebuilder tests from WPT for tree-construction compliance fixtures.
+
+### Fixed
+- Preserve HTML processing-instruction nodes in unsanitized parsing while keeping default-safe sanitization comment-like and conservative, and neutralize `>` in serialized processing-instruction data so a hand-built DOM or a policy that preserves comment-like nodes cannot serialize a processing instruction that a browser parses as a bogus comment terminating early and exposing following markup.
+- Keep raw-text, RCDATA, doctype, and foreign-content CDATA scans aligned after Unicode characters whose Python lowercase mapping changes length or becomes ASCII. Previously, content such as `İ` before a later `<script>` close tag could make the parser miss the real end tag and treat following markup as raw text.
+
+### Performance
+- Short-circuit open-elements-stack scope checks (`<p>`, `<li>`, `<dd>`/`<dt>`, table-cell scoping, and similar) when the target element isn't open anywhere on the stack, avoiding quadratic parse time for deeply nested common block elements such as `<div>`, `<table>`, `<section>`, and heading tags.
+
+### Security
+- (Severity: Low) Reject numeric character references with more digits than any valid Unicode code point could have before converting them to an integer. Previously, a decimal reference such as `&#` followed by thousands of digits could crash parsing with an uncaught `ValueError` before sanitization, since Python limits decimal string-to-integer conversion length.
+- (Severity: Moderate) Prioritize a byte-order mark over a caller-declared transport encoding when decoding byte input, matching the HTML encoding-sniffing algorithm. Previously, passing an explicit `encoding=` alongside untrusted bytes that began with a BOM for a different encoding could make JustHTML decode and sanitize content differently than a browser does, letting markup such as `<script>` pass through undetected while a browser executes it under the BOM-indicated encoding.
+
+## [3.1.0] - 2026-07-02
+
+### Changed
+- `CollapseWhitespace(...)` now trims leading and trailing whitespace at block-container edges by default. Pass `trim_blocks=False` to keep the previous html5lib-like behavior, or customize `block_tags=(...)` for project-specific block containers.
+- Document that DOM insertion methods adopt existing nodes, and show how to clone nodes before appending when using parsed DOMs as templates.
+
+### Fixed
+- Apply explicit `UrlPolicy.allow_rules` to custom URL attributes in constructor sanitization and `DropUrlAttrs(...)`. Previously, rules such as `("el-custom", "data-url")` were ignored unless the attribute name was one of JustHTML's built-in URL sinks.
+- Restore strict-mode reporting for unexpected `</p>` end tags after implicit paragraph closure.
+
+### Performance
+- Speed up pretty HTML serialization for direct text-only elements, repeated tag/attribute names, and common empty-attribute elements; reduce repeated selector name normalization and hot parser namespace checks; and fix the real-world profiler script when run directly.
+- Speed up parser URL sanitization for common lowercase `http://` and `https://` attributes.
+
 ## [3.0.0] - 2026-06-21
 
 JustHTML 3.0.0 ships a new parser core. The old tokenizer and tree-builder are now one integrated system, which makes the parser much faster while keeping browser-style HTML5 recovery and default-safe behavior intact. In practice, this release is mainly about getting materially better performance without asking most users to change how they use the library.

{justhtml-3.0.0 → justhtml-3.2.0}/CONTRIBUTING.md

@@ -24,9 +24,9 @@ Thanks for considering contributing to JustHTML! This document explains how to s
 
 ## Running Tests
 
-The test suite uses the html5lib test cases plus additional tests for selector functionality.
+The test suite uses the web platform html5 treebuilder tests plus additional tests for selector functionality.
 
-If you want to run the full html5lib test suite locally, clone `html5lib-tests` next to this repository and create the symlinks described in [tests/README.md](tests/README.md) (tokenizer, tree-construction, and serializer).
+If you want to run the full external fixture suite locally, clone the web platform html5 treebuilder tests and `html5lib-tests` serializer/encoding fixtures next to this repository, then create the symlinks described in [tests/README.md](tests/README.md).
 
 ```bash
 # Run all tests
@@ -58,7 +58,7 @@ Pre-commit runs automatically on every commit and checks:
 - **Ruff check** - linting with auto-fix
 - **Ruff format** - code formatting
 - **Tests & Coverage** - full test suite with 100% coverage requirement
-- **Parser Differential** - exact agreement with the reference parser path across scored html5lib tree cases
+- **Parser Differential** - exact agreement with the reference parser path across scored web platform html5 treebuilder cases
 
 Run manually:
 ```bash

{justhtml-3.0.0 → justhtml-3.2.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: justhtml
-Version: 3.0.0
+Version: 3.2.0
 Summary: A pure Python HTML5 parser that just works.
 Project-URL: Homepage, https://github.com/emilstenstrom/justhtml
 Project-URL: Issues, https://github.com/emilstenstrom/justhtml/issues
@@ -82,7 +82,7 @@ Most Python HTML libraries optimize for one part of the problem.
 
 JustHTML is for applications that want a boring, inspectable, pure-Python default:
 
-- **Correct parsing:** browser-style HTML5 recovery, tested against the official html5lib fixtures.
+- **Correct parsing:** browser-style HTML5 recovery, tested against the web platform html5 treebuilder tests.
 - **Safe by default:** `JustHTML(html)` sanitizes before you query or serialize.
 - **One DOM:** parse once, then sanitize, query, transform, serialize, extract text, or produce Markdown.
 - **Easy deployment:** zero runtime dependencies, no compiler, works on PyPy and Pyodide.
@@ -164,10 +164,10 @@ justhtml index.html --selector "article" --format markdown
 
 ## Correctness
 
-JustHTML is tested against the official html5lib tree-construction, serializer, and encoding fixtures, plus project-specific sanitizer, selector, transform, CLI, and regression tests.
+JustHTML is tested against the web platform html5 treebuilder tests, plus html5lib serializer/encoding fixtures and project-specific sanitizer, selector, transform, CLI, and regression tests.
 
 The current test suite enforces 100% combined line and branch coverage, including the parser engine.
-The parser engine additionally requires exact agreement with the reference path across the html5lib tree suite.
+The parser engine additionally requires exact agreement with the reference path across the web platform html5 treebuilder suite.
 See [Correctness Testing](docs/correctness.md) for details.
 
 ## Documentation

{justhtml-3.0.0 → justhtml-3.2.0}/README.md

@@ -28,7 +28,7 @@ Most Python HTML libraries optimize for one part of the problem.
 
 JustHTML is for applications that want a boring, inspectable, pure-Python default:
 
-- **Correct parsing:** browser-style HTML5 recovery, tested against the official html5lib fixtures.
+- **Correct parsing:** browser-style HTML5 recovery, tested against the web platform html5 treebuilder tests.
 - **Safe by default:** `JustHTML(html)` sanitizes before you query or serialize.
 - **One DOM:** parse once, then sanitize, query, transform, serialize, extract text, or produce Markdown.
 - **Easy deployment:** zero runtime dependencies, no compiler, works on PyPy and Pyodide.
@@ -110,10 +110,10 @@ justhtml index.html --selector "article" --format markdown
 
 ## Correctness
 
-JustHTML is tested against the official html5lib tree-construction, serializer, and encoding fixtures, plus project-specific sanitizer, selector, transform, CLI, and regression tests.
+JustHTML is tested against the web platform html5 treebuilder tests, plus html5lib serializer/encoding fixtures and project-specific sanitizer, selector, transform, CLI, and regression tests.
 
 The current test suite enforces 100% combined line and branch coverage, including the parser engine.
-The parser engine additionally requires exact agreement with the reference path across the html5lib tree suite.
+The parser engine additionally requires exact agreement with the reference path across the web platform html5 treebuilder suite.
 See [Correctness Testing](docs/correctness.md) for details.
 
 ## Documentation

{justhtml-3.0.0 → justhtml-3.2.0}/benchmarks/profile.py

@@ -2,9 +2,10 @@
 """Profile JustHTML on real-world HTML."""
 
 import argparse
-import cProfile
+import importlib
 import pathlib
 import pstats
+import sys
 import tarfile
 
 import zstandard as zstd
@@ -49,15 +50,30 @@ def load_html_files(batch_path: pathlib.Path, dict_bytes: bytes, limit: int = 10
 
 def parse_args() -> argparse.Namespace:
     parser = argparse.ArgumentParser(description=__doc__)
+    parser.add_argument(
+        "--mode",
+        choices=["parse", "compact-html", "pretty-html", "text", "fragment"],
+        default="parse",
+        help="Pipeline to profile (default: parse)",
+    )
     parser.add_argument(
         "--to-html",
         action="store_true",
-        help="Profile parse + serialization via to_html() (pretty=True by default)",
+        help="Deprecated alias for --mode pretty-html",
     )
     return parser.parse_args()
 
 
+def remove_script_dir_from_import_path() -> None:
+    script_dir = str(pathlib.Path(__file__).resolve().parent)
+    if sys.path and sys.path[0] == script_dir:
+        sys.path.pop(0)
+
+
 def main() -> None:
+    remove_script_dir_from_import_path()
+    cprofile = importlib.import_module("cProfile")
+
     args = parse_args()
 
     dict_bytes = load_dict(pathlib.Path("/home/emilstenstrom/Projects/web100k/html.dict"))
@@ -69,12 +85,26 @@ def main() -> None:
 
     print(f"Loaded {len(html_files)} files")
 
-    profiler = cProfile.Profile()
+    mode = "pretty-html" if args.to_html else args.mode
+
+    profiler = cprofile.Profile()
     profiler.enable()
 
     for _filename, html in html_files:
+        if mode == "fragment":
+            parser = JustHTML(html, fragment=True)
+            _ = parser.root
+            continue
+
         parser = JustHTML(html)
-        _ = parser.to_html() if args.to_html else parser.root
+        if mode == "compact-html":
+            _ = parser.to_html(pretty=False)
+        elif mode == "pretty-html":
+            _ = parser.to_html()
+        elif mode == "text":
+            _ = parser.to_text()
+        else:
+            _ = parser.root
 
     profiler.disable()
 

{justhtml-3.0.0 → justhtml-3.2.0}/docs/api.md

@@ -408,10 +408,16 @@ When you build documents with `JustHTML(..., sanitize=True)` (the default), this
 
 Append a child node to this node.
 
+If `node` already has a parent, it is moved from that parent to this node. Use
+`node.clone_node(deep=True)` before appending when you want to keep the source
+DOM unchanged.
+
 #### `insert_before(node, reference_node)`
 
 Insert `node` before `reference_node` (or append if `reference_node` is `None`).
 
+If `node` already has a parent, it is moved from that parent to this node.
+
 #### `remove_child(node)`
 
 Remove a direct child node.
@@ -420,6 +426,9 @@ Remove a direct child node.
 
 Replace a direct child node with a new node.
 
+If `new_node` already has a parent, it is moved from that parent before it
+replaces `old_node`.
+
 #### `clone_node(deep=False, override_attrs=None)`
 
 Clone this node. If `deep=True`, children are cloned recursively.

{justhtml-3.0.0 → justhtml-3.2.0}/docs/building.md

@@ -435,6 +435,33 @@ lead = doc.query_one("p.lead")
 lead.to_html(pretty=False)  # => <p class="lead">Hello</p>
 ```
 
+### Clone from template DOMs before appending
+
+DOM insertion methods adopt existing nodes. If you append a node that already
+belongs to another tree, it is removed from its old parent and moved into the new
+tree.
+
+When you use one parsed DOM as a template for another document, clone the source
+node before appending it:
+
+```python
+from justhtml import JustHTML
+from justhtml.dom.builder import element
+
+template = JustHTML('<section><p class="lead">Hello</p></section>', fragment=True, sanitize=False)
+output = JustHTML(element("main"), fragment=True, sanitize=False)
+
+lead = template.query_one("p.lead")
+assert lead is not None
+main = output.query_one("main")
+assert main is not None
+
+main.append_child(lead.clone_node(deep=True))
+
+template.to_html(pretty=False)  # => <section><p class="lead">Hello</p></section>
+output.to_html(pretty=False)  # => <main><p class="lead">Hello</p></main>
+```
+
 ## When Not to Use the Builder
 
 Don’t use the builder just because it exists.

{justhtml-3.0.0 → justhtml-3.2.0}/docs/correctness.md

@@ -2,18 +2,18 @@
 
 # Correctness Testing
 
-JustHTML is the only pure-Python HTML5 parser that passes 100% of the official html5lib test suite. This page explains how we verify and maintain that compliance.
+JustHTML is tested against the web platform html5 treebuilder tests. This page explains how we verify and maintain that compliance.
 
-## The html5lib Test Suite
+## The Web Platform Tests
 
-The [html5lib-tests](https://github.com/html5lib/html5lib-tests) repository is the gold standard for HTML5 parsing compliance. It's used by browser vendors to verify their implementations against the [WHATWG HTML5 specification](https://html.spec.whatwg.org/).
+The web platform html5 treebuilder tests live in [web-platform-tests/wpt](https://github.com/web-platform-tests/wpt/tree/master/html/syntax/parsing/resources). Serializer and encoding fixtures remain in [html5lib-tests](https://github.com/html5lib/html5lib-tests).
 
-Our checked-in test inputs contain:
+The external fixture inputs contain:
 
-- **56 tree-construction test files** - Testing how the parser builds the DOM tree
+- **61 treebuilder test files** - Testing how the parser builds the DOM tree
 - **5 serializer fixture files** - Testing how token streams are serialized back to HTML
 - **Encoding sniffing tests** - Testing BOM/meta charset/transport overrides and legacy fallbacks
-- **1,791 enabled tree-construction cases** - Covering edge cases, error recovery, and spec compliance
+- **1,914 enabled treebuilder cases** - Covering edge cases, error recovery, and spec compliance
 
 ### What the Tests Cover
 
@@ -54,12 +54,11 @@ This tests the adoption agency algorithm - when `</b>` is encountered inside `<p
 ## Compliance Comparison
 
 We run the same test suite against other Python parsers to compare compliance.
-The cross-parser snapshot below used the 1,743 cases available when it was
-recorded; the current JustHTML gate covers 1,791 enabled cases.
+The cross-parser snapshot below used the 1,743 cases available when it was recorded.
 
 | Parser | Tests Passed | Compliance | Notes |
 |--------|-------------|------------|-------|
-| **JustHTML** | 1743/1743 | **100%** | Full spec compliance in this comparison snapshot; current gate: 1791/1791 |
+| **JustHTML** | 1743/1743 | **100%** | Full spec compliance in this comparison snapshot |
 | selectolax | 1743/1743 | 100% | C-based (Lexbor), fast and spec-compliant with dev `html5test` output API |
 | markupever | 1545/1743 | 89% | Rust-based (html5ever), mostly correct |
 | html5lib | 1496/1743 | 86% | Reference implementation, but incomplete |
@@ -68,9 +67,9 @@ recorded; the current JustHTML gate covers 1,791 enabled cases.
 | html.parser | 6/1743 | <1% | Python stdlib, basic error recovery only |
 | lxml | 5/1743 | <1% | XML-based, not HTML5 compliant |
 
-*Run `python benchmarks/correctness.py` to reproduce these results. The selectolax score requires its dev `html5test` output and fragment-context APIs. These scores were refreshed against html5lib-tests `e446320`.*
+*Run `python benchmarks/correctness.py` to reproduce these results. The selectolax score requires its dev `html5test` output and fragment-context APIs. These historical scores were refreshed against html5lib-tests `e446320`.*
 
-These numbers come from a strict tree comparison against the expected output in the `html5lib-tests` tree-construction fixtures (excluding `#script-on` / `#script-off` cases). Unsupported parser capabilities count as failures for this compliance table. The numbers will not match the `html5lib` project’s own reported totals, because `html5lib` runs the suite in multiple configurations and also has its own skip/xfail lists.
+These numbers come from a strict tree comparison against the expected output in the treebuilder fixtures available at the time (excluding `#script-on` / `#script-off` cases). Unsupported parser capabilities count as failures for this compliance table. The numbers will not match the `html5lib` project’s own reported totals, because `html5lib` runs the suite in multiple configurations and also has its own skip/xfail lists.
 
 ## Our Testing Strategy
 
@@ -123,7 +122,7 @@ PYTHONPATH=src python benchmarks/html5lib_engine_diff.py \
   --fail-on-current-exceptions
 ```
 
-This requires exact agreement with the reference parser path across every scored html5lib tree-construction case.
+This requires exact agreement with the reference parser path across every scored web platform html5 treebuilder case.
 
 ### 3. Fuzz Testing (millions of cases)
 
@@ -169,14 +168,18 @@ We maintain additional tests in `tests/justhtml-tests/` for:
 ### Quick Start
 
 ```bash
-# Clone the test suite (one-time setup)
+# Clone the test suites (one-time setup)
+cd ..
+git clone --filter=blob:none --sparse https://github.com/web-platform-tests/wpt.git
+cd wpt
+git sparse-checkout set html/syntax/parsing/resources
 cd ..
 git clone https://github.com/html5lib/html5lib-tests.git
 cd justhtml
 
 # Create symlinks
 cd tests
-ln -s ../../html5lib-tests/tree-construction html5lib-tests-tree
+ln -s ../../wpt/html/syntax/parsing/resources html5lib-tests-tree
 ln -s ../../html5lib-tests/serializer html5lib-tests-serializer
 ln -s ../../html5lib-tests/encoding html5lib-tests-encoding
 cd ..

{justhtml-3.0.0 → justhtml-3.2.0}/docs/playground/app.js

@@ -65,6 +65,7 @@ async function installJusthtmlFromLocalRepo(pyodideInstance) {
 		"parser/encoding.py",
 		"core/entities.py",
 		"core/errors.py",
+		"core/text.py",
 		"core/types.py",
 		"core/doctype.py",
 		"serializer/markdown.py",

{justhtml-3.0.0 → justhtml-3.2.0}/docs/transforms.md

@@ -268,11 +268,16 @@ Advanced building blocks (useful for policy-driven pipelines):
 
 See [`Linkify(...)`](linkify.md) for full documentation and examples.
 
-### `CollapseWhitespace(skip_tags=(...), enabled=True, callback=None, report=None)`
+### `CollapseWhitespace(skip_tags=(...), trim_blocks=True, block_tags=(...), enabled=True, callback=None, report=None)`
 
 Collapses runs of HTML whitespace characters in text nodes to a single space.
+By default, it also trims leading and trailing whitespace at the edges of block
+containers such as `<p>`, `<div>`, and `<li>`.
 
-This is similar to `html5lib.filters.whitespace.Filter`.
+Pass `trim_blocks=False` for html5lib-like whitespace filtering without edge
+trimming. Use `block_tags=(...)` to define which containers trim their edge
+whitespace; `skip_tags` still wins and preserves whitespace inside matching
+tags.
 
 By default it skips `<pre>`, `<textarea>`, `<code>`, `<title>`, `<script>`, and `<style>`.
 
@@ -280,7 +285,7 @@ By default it skips `<pre>`, `<textarea>`, `<code>`, `<title>`, `<script>`, and
 from justhtml import CollapseWhitespace, JustHTML
 
 doc = JustHTML(
-    "<p>Hello \n\t world</p><pre>a  b</pre>",
+    "<p>\n  Hello \n\t world\n</p><pre>a  b</pre>",
     fragment=True,
     transforms=[CollapseWhitespace()],
 )

{justhtml-3.0.0 → justhtml-3.2.0}/docs/url-cleaning.md

@@ -85,7 +85,8 @@ Output:
 
 ## How URL cleaning works (in order)
 
-For a URL-like attribute (like `img[src]` or `a[href]`), JustHTML applies these steps:
+For a URL-like attribute (like `img[src]` or `a[href]`) or a custom attribute with an explicit
+`UrlPolicy(allow_rules=...)` entry, JustHTML applies these steps:
 
 1. The tag must be allowed by `SanitizationPolicy.allowed_tags`.
 2. The attribute name must be allowed by `SanitizationPolicy.allowed_attributes`.
@@ -103,7 +104,9 @@ URL behavior is controlled by `UrlPolicy`:
 - `default_handling`: the default post-validation action for URL-like attributes that match an allow rule and do not set `UrlRule.handling`.
 - `default_allow_relative`: whether **relative** URLs (like `/path`, `./path`, `../path`, `?q`) are allowed by default.
 
-For URL-like attributes that match an explicit `(tag, attr)` rule in `allow_rules`, validated URLs use `UrlPolicy.default_handling` unless that specific `UrlRule` overrides it with `handling=...`.
+For URL-like attributes, or custom attributes that match an explicit `(tag, attr)` or `("*", attr)` rule in
+`allow_rules`, validated URLs use `UrlPolicy.default_handling` unless that specific `UrlRule` overrides it with
+`handling=...`.
 
 Note: URL validation is always enforced by `UrlRule`.
 

{justhtml-3.0.0 → justhtml-3.2.0}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "justhtml"
 authors = [{ name = "Emil Stenström", email = "emil@emilstenstrom.se" }]
-version = "3.0.0"
+version = "3.2.0"
 description = "A pure Python HTML5 parser that just works."
 readme = "README.md"
 license = { file = "LICENSE" }

{justhtml-3.0.0 → justhtml-3.2.0}/run_tests.py

@@ -114,7 +114,7 @@ def parse_args():
         "--check-errors",
         action="store_true",
         help=(
-            "Enable additional error validation. For html5lib tree-construction .dat tests, "
+            "Enable additional error validation. For web platform html5 treebuilder .dat tests, "
             "validates the number of parse errors; for tests/justhtml-tests, validates the exact "
             "ordered list of error codes."
             "parse errors (code+line+col) when provided by the fixture."
@@ -295,7 +295,7 @@ def main():
     run_encoding = suite in {"all", "encoding"}
     run_unit = suite in {"all", "unit"}
 
-    # Check that html5lib-tests symlinks exist (only for the selected suites)
+    # Check that external fixture symlinks exist (only for the selected suites)
     tree_tests = test_dir / "html5lib-tests-tree"
     serializer_tests = test_dir / "html5lib-tests-serializer"
     encoding_tests = test_dir / "html5lib-tests-encoding"
@@ -307,13 +307,19 @@ def main():
     if run_encoding and not encoding_tests.exists():
         missing.append(str(encoding_tests))
     if len(missing) > 0:
-        print("ERROR: html5lib-tests not found. Please create symlinks:", file=sys.stderr)
+        print("ERROR: external test fixtures not found. Please create symlinks:", file=sys.stderr)
         for path in missing:
             print(f"  {path}", file=sys.stderr)
-        print("\nTo set up, clone html5lib-tests and create symlinks:", file=sys.stderr)
-        print("  git clone https://github.com/html5lib/html5lib-tests.git ../html5lib-tests", file=sys.stderr)
+        print("\nTo set up, clone the fixture repositories and create symlinks:", file=sys.stderr)
         if run_tree:
-            print("  ln -s ../../html5lib-tests/tree-construction tests/html5lib-tests-tree", file=sys.stderr)
+            print(
+                "  git clone --filter=blob:none --sparse https://github.com/web-platform-tests/wpt.git ../wpt",
+                file=sys.stderr,
+            )
+            print("  (cd ../wpt && git sparse-checkout set html/syntax/parsing/resources)", file=sys.stderr)
+            print("  ln -s ../../wpt/html/syntax/parsing/resources tests/html5lib-tests-tree", file=sys.stderr)
+        if run_serializer or run_encoding:
+            print("  git clone https://github.com/html5lib/html5lib-tests.git ../html5lib-tests", file=sys.stderr)
         if run_serializer:
             print("  ln -s ../../html5lib-tests/serializer tests/html5lib-tests-serializer", file=sys.stderr)
         if run_encoding:
@@ -343,7 +349,7 @@ def main():
         combined_results.update(runner.file_results)
 
     if run_justhtml_tree:
-        # Run JustHTML-specific tree-construction tests (custom .dat fixtures).
+        # Run JustHTML-specific treebuilder tests (custom .dat fixtures).
         # These live outside the upstream html5lib-tests checkout.
         justhtml_tree_tests = test_dir / "justhtml-tests"
         justhtml_runner = TestRunner(justhtml_tree_tests, config)

{justhtml-3.0.0 → justhtml-3.2.0}/src/justhtml/__init__.py

@@ -1,5 +1,16 @@
 from .core.types import ParseError
-from .dom import Comment, Document, DocumentFragment, Element, Node, NodeType, QueryMatch, Template, Text
+from .dom import (
+    Comment,
+    Document,
+    DocumentFragment,
+    Element,
+    Node,
+    NodeType,
+    ProcessingInstruction,
+    QueryMatch,
+    Template,
+    Text,
+)
 from .parser import JustHTML, StrictModeError
 from .parser.stream import stream
 from .sanitizer import (
@@ -70,6 +81,7 @@ __all__ = [
     "Node",
     "NodeType",
     "ParseError",
+    "ProcessingInstruction",
     "PruneEmpty",
     "QueryMatch",
     "SanitizationPolicy",

{justhtml-3.0.0 → justhtml-3.2.0}/src/justhtml/core/constants.py

@@ -200,6 +200,57 @@ WHITESPACE_PRESERVING_ELEMENTS: Final[frozenset[str]] = frozenset(
     }
 )
 
+WHITESPACE_TRIMMING_BLOCK_ELEMENTS: Final[frozenset[str]] = frozenset(
+    {
+        "address",
+        "article",
+        "aside",
+        "blockquote",
+        "body",
+        "caption",
+        "center",
+        "dd",
+        "details",
+        "dialog",
+        "dir",
+        "div",
+        "dl",
+        "dt",
+        "fieldset",
+        "figcaption",
+        "figure",
+        "footer",
+        "form",
+        "h1",
+        "h2",
+        "h3",
+        "h4",
+        "h5",
+        "h6",
+        "header",
+        "hgroup",
+        "html",
+        "li",
+        "listing",
+        "main",
+        "menu",
+        "nav",
+        "ol",
+        "p",
+        "search",
+        "section",
+        "summary",
+        "table",
+        "tbody",
+        "td",
+        "tfoot",
+        "th",
+        "thead",
+        "tr",
+        "ul",
+    }
+)
+
 FORMATTING_ELEMENTS = {
     "a",
     "b",

{justhtml-3.0.0 → justhtml-3.2.0}/src/justhtml/core/entities.py

@@ -199,7 +199,16 @@ def decode_numeric_entity(
         The decoded character, or None if invalid
     """
     base = 16 if is_hex else 10
-    codepoint = int(text, base)
+
+    # A numeric character reference this long is already guaranteed to exceed
+    # the valid Unicode range, so treat it as out-of-range without calling
+    # int() on an attacker-controlled digit string long enough to exceed
+    # Python's integer-string conversion limit (see sys.set_int_max_str_digits).
+    digits = text.lstrip("0") or "0"
+    if len(digits) > 16:
+        return "\ufffd"  # REPLACEMENT CHARACTER
+
+    codepoint = int(digits, base)
 
     # Invalid ranges per HTML5 spec
     if codepoint > 0x10FFFF:

{justhtml-3.0.0 → justhtml-3.2.0}/src/justhtml/core/rawtext.py

@@ -1,13 +1,14 @@
 from __future__ import annotations
 
 from .constants import HTML_SPACE_OR_TAG_END_CHARACTERS
+from .text import ascii_lower
 
 
 def neutralize_rawtext_end_tag_sequences(text: str, tag_name: str) -> tuple[str, bool]:
     if not text:
         return text, False
 
-    lower_text = text.lower()
+    lower_text = ascii_lower(text)
     needle = f"</{tag_name}"
     needle_len = len(needle)
     out: list[str] = []

justhtml-3.2.0/src/justhtml/core/text.py

@@ -0,0 +1,30 @@
+from __future__ import annotations
+
+import string
+
+_ASCII_LOWER_TABLE = str.maketrans(string.ascii_uppercase, string.ascii_lowercase)
+
+
+def ascii_lower(text: str) -> str:
+    """Fold A-Z for ASCII marker searches that reuse original offsets.
+
+    Some parser and serializer paths search a folded copy of full text for
+    ASCII HTML markers such as ``</script``, ``<!doctype``, and ``<![cdata[``.
+    Match offsets are then reused against the original string, so the folded
+    copy must preserve length, and it must not invent ASCII marker characters
+    from non-ASCII text content. HTML markup matching is ASCII
+    case-insensitive; it is not Unicode case-insensitive.
+
+    ``str.lower()`` is still the fast path when it is provably equivalent for
+    these searches. In Python's Unicode data, U+0130 LATIN CAPITAL LETTER I
+    WITH DOT ABOVE is the only character whose lowercase mapping changes
+    length, and U+212A KELVIN SIGN is the only non-ASCII character whose
+    lowercase mapping becomes an ASCII letter (``k``). Inputs containing either
+    shape take the strict A-Z translation path. That path deliberately keeps
+    U+212A as U+212A; it is not a tag-name fold.
+    """
+    if text.isascii():
+        return text.lower()
+    if "\u0130" in text or "\u212a" in text:
+        return text.translate(_ASCII_LOWER_TABLE)
+    return text.lower()