justhtml 2.3.0__tar.gz → 2.4.1__tar.gz

{justhtml-2.3.0 → justhtml-2.4.1}/CHANGELOG.md

@@ -7,6 +7,18 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+## [2.4.1] - 2026-06-21
+
+### Fixed
+- Prevent the leading-line-feed exception for `<pre>`, `<listing>`, and `<textarea>` from leaking past intervening non-character tokens and removing a later newline.
+- Apply paragraph closing and leading-line-feed handling to `<pre>` and `<listing>` inside customizable `<select>` content.
+- Preserve U+000C FORM FEED in HTML text like browsers do, while retaining its intentional conversion to a space in XML-coercion mode.
+
+## [2.4.0] - 2026-06-20
+
+### Fixed
+- Handle `<hr>` using the HTML in-body void-element rules in delegated insertion modes, preserving following content in table cells and captions, closing open paragraphs, and acknowledging self-closing syntax.
+
 ## [2.3.0] - 2026-06-12
 
 ### Fixed

{justhtml-2.3.0 → justhtml-2.4.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: justhtml
-Version: 2.3.0
+Version: 2.4.1
 Summary: A pure Python HTML5 parser that just works.
 Project-URL: Homepage, https://github.com/emilstenstrom/justhtml
 Project-URL: Issues, https://github.com/emilstenstrom/justhtml/issues
@@ -88,6 +88,8 @@ JustHTML is for applications that want a boring, inspectable, pure-Python defaul
 - **Easy deployment:** zero runtime dependencies, no compiler, works on PyPy and Pyodide.
 - **Honest tradeoff:** if you are parsing terabytes of trusted HTML, use a C/Rust parser. If you need reliable handling of untrusted or malformed HTML inside a Python app, use JustHTML.
 
+Real-world signal: [Mozilla Support migrated from Bleach to JustHTML](https://github.com/mozilla/kitsune/pull/7236) in Kitsune, the Django application behind support.mozilla.org.
+
 ## Quick Start
 
 ```python

{justhtml-2.3.0 → justhtml-2.4.1}/README.md

@@ -34,6 +34,8 @@ JustHTML is for applications that want a boring, inspectable, pure-Python defaul
 - **Easy deployment:** zero runtime dependencies, no compiler, works on PyPy and Pyodide.
 - **Honest tradeoff:** if you are parsing terabytes of trusted HTML, use a C/Rust parser. If you need reliable handling of untrusted or malformed HTML inside a Python app, use JustHTML.
 
+Real-world signal: [Mozilla Support migrated from Bleach to JustHTML](https://github.com/mozilla/kitsune/pull/7236) in Kitsune, the Django application behind support.mozilla.org.
+
 ## Quick Start
 
 ```python

{justhtml-2.3.0 → justhtml-2.4.1}/docs/bleach-migration.md

@@ -10,6 +10,107 @@ In 2023, Bleach’s maintainer announced that **Bleach is deprecated** (but will
 
 This guide covers common migration patterns.
 
+## Real-world migration pattern: add a local wrapper first
+
+A large application migration often works best in two steps:
+
+1. Add a small application-local `clean(...)` wrapper that accepts the Bleach-shaped options your codebase already uses.
+2. Migrate call sites from `bleach.clean(...)` / `bleach.linkify(...)` to that wrapper.
+
+That keeps most call-site changes mechanical while leaving the security policy in one place.
+
+Example wrapper:
+
+```python
+import re
+from collections.abc import Collection, Mapping
+
+from justhtml import JustHTML, Linkify, SanitizationPolicy, SetAttrs, UrlPolicy, UrlRule
+
+URL_LIKE_ATTRS = {
+    "href",
+    "src",
+    "srcset",
+    "poster",
+    "action",
+    "formaction",
+    "data",
+    "cite",
+    "background",
+    "ping",
+}
+
+
+def build_url_policy(
+    allowed_tags: Collection[str],
+    allowed_attributes: Mapping[str, Collection[str]],
+) -> UrlPolicy:
+    rules = {}
+
+    global_attrs = allowed_attributes.get("*", ())
+    for tag in allowed_tags:
+        for attr in global_attrs:
+            if attr in URL_LIKE_ATTRS:
+                rules[(tag, attr)] = UrlRule(
+                    allowed_schemes={"http", "https", "mailto", "tel"},
+                    allow_relative=True,
+                )
+
+    for tag, attrs in allowed_attributes.items():
+        if tag == "*":
+            continue
+        for attr in attrs:
+            if attr in URL_LIKE_ATTRS:
+                rules[(tag, attr)] = UrlRule(
+                    allowed_schemes={"http", "https", "mailto", "tel"},
+                    allow_relative=True,
+                )
+    return UrlPolicy(allow_rules=rules)
+
+
+def clean(
+    html: str,
+    *,
+    tags: Collection[str] = (),
+    attributes: Mapping[str, Collection[str]] | None = None,
+    css_properties: Collection[str] = (),
+    strip: bool = True,
+    strip_comments: bool = True,
+) -> str:
+    attrs = dict(attributes or {})
+    disallowed_tag_handling = "unwrap" if strip else "escape"
+
+    # Bleach defaults to strip_comments=True. If you use escape mode
+    # (`strip=False`) and still want comments removed rather than displayed,
+    # remove comments before parsing.
+    if strip_comments and disallowed_tag_handling == "escape":
+        html = re.sub(r"<!--.*?-->", "", html, flags=re.DOTALL)
+
+    policy = SanitizationPolicy(
+        allowed_tags=tags,
+        allowed_attributes=attrs,
+        allowed_css_properties=css_properties,
+        url_policy=build_url_policy(tags, attrs),
+        drop_comments=strip_comments,
+        disallowed_tag_handling=disallowed_tag_handling,
+    )
+    return JustHTML(html, fragment=True, policy=policy).to_html(pretty=False)
+
+
+def linkify(text: str, *, nofollow: bool = False) -> str:
+    transforms = [Linkify()]
+    if nofollow:
+        transforms.append(SetAttrs("a", rel="nofollow"))
+    return JustHTML(
+        text,
+        fragment=True,
+        sanitize=False,
+        transforms=transforms,
+    ).to_html(pretty=False)
+```
+
+Treat this as a compatibility shim, not a universal policy. Tighten `build_url_policy(...)` for your application, especially for attributes that load remote resources such as `img[src]`.
+
 ## Mental model differences
 
 - Bleach takes a string and returns a cleaned string.
@@ -97,6 +198,7 @@ print(doc.to_html(pretty=False))
 Bleach’s `protocols=[...]` concept maps to JustHTML’s URL policy rules.
 
 - Configure allowed schemes per attribute via `UrlRule(allowed_schemes=[...])`.
+- URL-like attributes need explicit `(tag, attr)` allow rules even when the attribute itself is allowed.
 - Under `sanitize=True`, URLs can be rewritten or stripped according to policy (see [URL Cleaning](url-cleaning.md)).
 
 ## What to do with “strip=True/False”
@@ -118,9 +220,20 @@ If you need to display untrusted HTML with no HTML in the output, prefer `to_tex
 
 If you need additional structural cleanup beyond policy decisions, prefer doing it explicitly with transforms.
 
+## Migration test expectations
+
+Expect some output differences even when the security behavior is equivalent:
+
+- Inline styles may be normalized, for example `vertical-align:top;` can serialize as `vertical-align: top`.
+- Malformed HTML can serialize differently because JustHTML parses with its own HTML5 tree builder.
+- `strip=False` maps to escaped disallowed tags. If your old Bleach call also relied on `strip_comments=True`, add an explicit test for comments.
+- Linkification should be tested separately from sanitization when your old code used both `bleach.linkify(...)` and `bleach.clean(...)`.
+
 ## Suggested migration approach
 
-1. Start by using JustHTML’s default sanitizer (`JustHTML(...).to_html()`).
-2. Add a `SanitizationPolicy` that matches your allowlist and URL requirements.
-3. Add transforms to replace any Bleach filter/linkify behavior.
-4. Add tests for your exact input corpus and output expectations (especially around URLs and allowed attributes).
+1. Inventory existing `bleach.clean(...)`, `bleach.Cleaner(...)`, and `bleach.linkify(...)` call shapes.
+2. For a large app, add a local compatibility wrapper so most call sites can move mechanically.
+3. Start by using JustHTML’s default sanitizer (`JustHTML(...).to_html()`) for new code.
+4. Add a `SanitizationPolicy` that matches your allowlist and URL requirements.
+5. Add transforms to replace any Bleach filter/linkify behavior.
+6. Add tests for your exact input corpus and output expectations (especially around URLs, allowed attributes, comments, and malformed HTML).

{justhtml-2.3.0 → justhtml-2.4.1}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "justhtml"
 authors = [{ name = "Emil Stenström", email = "emil@emilstenstrom.se" }]
-version = "2.3.0"
+version = "2.4.1"
 description = "A pure Python HTML5 parser that just works."
 readme = "README.md"
 license = { file = "LICENSE" }

{justhtml-2.3.0 → justhtml-2.4.1}/src/justhtml/treebuilder/core.py

@@ -411,6 +411,11 @@ class TreeBuilder(TreeBuilderModesMixin):
     def process_token(self, token: Any) -> Any:
         # Optimization: Use type() identity check instead of isinstance
         token_type = type(token)
+        # The leading-LF exception for <pre>, <listing>, and <textarea>
+        # applies only to the immediately following token. Character tokens
+        # consume it in _append_text; any other token cancels it.
+        if self.ignore_lf and token_type is not CharacterTokens:
+            self.ignore_lf = False
         if token_type is DoctypeToken:
             # Check for foreign content first - DOCTYPE in SVG/MathML is a parse error
             if self.open_elements:
@@ -668,9 +673,6 @@ class TreeBuilder(TreeBuilderModesMixin):
                 if not text:
                     return
 
-        if "\f" in text:
-            text = text.replace("\f", " ")
-
         # Guard against empty stack
         if not self.open_elements:  # pragma: no cover
             return

{justhtml-2.3.0 → justhtml-2.4.1}/src/justhtml/treebuilder/modes.py

@@ -954,6 +954,12 @@ class TreeBuilderModesMixin:
         self._insert_element(token, push=False)
         return
 
+    def _handle_body_start_hr(self, token: Tag) -> None:
+        self._close_p_element()
+        self._insert_element(token, push=False)
+        self.frameset_ok = False
+        return
+
     def _handle_body_start_input(self, token: Tag) -> None:
         if (
             self.fragment_context is not None
@@ -1671,6 +1677,10 @@ class TreeBuilderModesMixin:
                     self.ignore_lf = True
                     self.frameset_ok = False
                     return None
+                if name in {"pre", "listing"}:
+                    self._parse_error("unexpected-start-tag-in-select", tag_name=name)
+                    self._handle_body_start_pre_listing(token)
+                    return None
                 # Any other start tag is part of customizable select content.
                 self._parse_error("unexpected-start-tag-in-select", tag_name=name)
                 self._reconstruct_active_formatting_elements()
@@ -2054,6 +2064,7 @@ class TreeBuilderModesMixin:
         "head": _handle_body_start_head,
         "header": _handle_body_start_block_with_p,
         "hgroup": _handle_body_start_block_with_p,
+        "hr": _handle_body_start_hr,
         "html": _handle_body_start_html,
         "i": _handle_body_start_formatting,
         "image": _handle_body_start_image,

{justhtml-2.3.0 → justhtml-2.4.1}/tests/justhtml-tests/branch_coverage.dat

@@ -24,7 +24,7 @@
 |   <body>
 |     <math math>
 |       <math mi>
-|         " "
+|         ""
 
 
 #data

{justhtml-2.3.0 → justhtml-2.4.1}/tests/justhtml-tests/treebuilder_coverage.dat

@@ -21,7 +21,7 @@
 |   <body>
 |     <math math>
 |       <math mi>
-|         " text"
+|         "text"
 
 
 #data
@@ -75,7 +75,7 @@
 |   <head>
 |   <body>
 |     <select>
-|       " text"
+|       "text"
 
 
 #data
@@ -175,7 +175,7 @@
 |   <head>
 |   <body>
 |     <table>
-|       " "
+|       ""
 
 
 #data

{justhtml-2.3.0 → justhtml-2.4.1}/tests/test_treebuilder.py

@@ -220,6 +220,93 @@ class TestTreeBuilder(unittest.TestCase):
                 doc = JustHTML(html, fragment=True, sanitize=False)
                 self.assertEqual(doc.to_html(pretty=False), expected)
 
+    def test_hr_in_table_context_uses_in_body_void_element_rules(self) -> None:
+        cases = {
+            "<table><tr><td>before<hr>after</td></tr></table>": (
+                "<table><tbody><tr><td>before<hr>after</td></tr></tbody></table>"
+            ),
+            "<table><tr><th>a<hr>b</th></tr></table>": ("<table><tbody><tr><th>a<hr>b</th></tr></tbody></table>"),
+            "<table><caption>a<hr>b</caption></table>": "<table><caption>a<hr>b</caption></table>",
+            "<table><tr><td><p>a<hr>b</td></tr></table>": (
+                "<table><tbody><tr><td><p>a</p><hr>b</td></tr></tbody></table>"
+            ),
+        }
+
+        for html, expected in cases.items():
+            with self.subTest(html=html):
+                doc = JustHTML(html, fragment=True, sanitize=False)
+                self.assertEqual(doc.to_html(pretty=False), expected)
+
+    def test_self_closing_hr_in_table_context_is_acknowledged(self) -> None:
+        doc = JustHTML(
+            "<table><tr><td><hr/></td></tr></table>",
+            fragment=True,
+            sanitize=False,
+            collect_errors=True,
+        )
+
+        self.assertNotIn(
+            "non-void-html-element-start-tag-with-trailing-solidus",
+            [error.code for error in doc.errors],
+        )
+
+    def test_leading_lf_skip_does_not_leak_past_non_character_tokens(self) -> None:
+        cases = {
+            "<div><pre></pre>\nX</div>": "<div><pre></pre>\nX</div>",
+            "<div><pre><code></code></pre>\nX</div>": "<div><pre><code></code></pre>\nX</div>",
+            "<div><pre><!--x--></pre>\nX</div>": "<div><pre><!--x--></pre>\nX</div>",
+            "<div><listing></listing>\nX</div>": "<div><listing></listing>\nX</div>",
+            "<div><textarea></textarea>\nX</div>": "<div><textarea></textarea>\nX</div>",
+            "<div><pre>\nX</pre></div>": "<div><pre>X</pre></div>",
+            "<div><listing>\nX</listing></div>": "<div><listing>X</listing></div>",
+            "<div><textarea>\nX</textarea></div>": "<div><textarea>X</textarea></div>",
+        }
+
+        for html, expected in cases.items():
+            with self.subTest(html=html):
+                doc = JustHTML(html, fragment=True, sanitize=False)
+                self.assertEqual(doc.to_html(pretty=False), expected)
+
+    def test_pre_and_listing_in_select_use_their_in_body_start_rules(self) -> None:
+        cases = {
+            "<select><pre>\nX</pre></select>": "<select><pre>X</pre></select>",
+            "<select><listing>\nX</listing></select>": "<select><listing>X</listing></select>",
+            "<select><p>P<pre>\nX</pre></select>": "<select><p>P</p><pre>X</pre></select>",
+            "<select><p>P<listing>\nX</listing></select>": "<select><p>P</p><listing>X</listing></select>",
+            "<select><pre><code></code>\nX</pre></select>": ("<select><pre><code></code>\nX</pre></select>"),
+        }
+
+        for html, expected in cases.items():
+            with self.subTest(html=html):
+                doc = JustHTML(html, fragment=True, sanitize=False)
+                self.assertEqual(doc.to_html(pretty=False), expected)
+
+    def test_form_feed_is_preserved_in_html_text_like_chromium(self) -> None:
+        cases = {
+            "<div>A\fB</div>": "<div>A\fB</div>",
+            "<pre>A\fB</pre>": "<pre>A\fB</pre>",
+            "<textarea>A\fB</textarea>": "<textarea>A\fB</textarea>",
+            "<style>A\fB</style>": "<style>A\fB</style>",
+            "<script>A\fB</script>": "<script>A\fB</script>",
+            "<table>A\fB<tr><td>C\fD</td></tr></table>": ("A\fB<table><tbody><tr><td>C\fD</td></tr></tbody></table>"),
+            "<svg><text>A\fB</text></svg>": "<svg><text>A\fB</text></svg>",
+        }
+
+        for html, expected in cases.items():
+            with self.subTest(html=html):
+                doc = JustHTML(html, fragment=True, sanitize=False)
+                self.assertEqual(doc.to_html(pretty=False), expected)
+
+    def test_form_feed_is_still_coerced_for_xml_output(self) -> None:
+        doc = JustHTML(
+            "<div>A\fB</div>",
+            fragment=True,
+            sanitize=False,
+            _tokenizer_opts=TokenizerOpts(xml_coercion=True),
+        )
+
+        self.assertEqual(doc.to_html(pretty=False), "<div>A B</div>")
+
     def test_select_end_tags_create_p_and_close_custom_children_like_chromium(self) -> None:
         cases = {
             "<select>x</p>": "<select>x<p></p></select>",