PyPI - justhtml - Versions diffs - 3.5.0__tar.gz → 3.6.0__tar.gz - Mend

justhtml 3.5.0tar.gz → 3.6.0tar.gz

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (157) hide show

{justhtml-3.5.0 → justhtml-3.6.0}/CHANGELOG.md RENAMED Viewed

@@ -7,6 +7,36 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ## [Unreleased]
+## [3.6.0] - 2026-07-13
+### Fixed
+Note: The below are all minor parsing differences found when diffing against Chromium output, that is not covered in WPT's html5 parsing tests.
+- Add a formatting element that breaks out of foreign content (such as `<i>` inside `<svg>`) to the active formatting list, so it is reconstructed later. Previously, the breakout element was inserted but not tracked, so `<b><svg><i></b>z` dropped the reconstructed `<i>` around the trailing text and `<svg><code><h2></code>` lost the reconstructed `<code>`.
+- Start the body for a `<noscript>` start tag in the "after head" insertion mode instead of re-entering the head. The after-head metadata list does not include noscript, so markup such as `</head><noscript>` now places the element in the body.
+- Hand an end tag to the HTML rules when the foreign-content end-tag walk reaches a matching HTML element, rather than popping the foreign elements above it. Previously, `<form><math></form>` popped the `<math>` along with the form instead of splicing only the form out of the stack.
+- Clear back to the template contents before opening table structure in a template with no table element, so an open caption is closed. Previously, `<template><caption><tr>` nested the row inside the caption instead of making it a sibling.
+- Switch the "in template" mode into the body for a non-head raw-text or RCDATA start tag (`iframe`, `noembed`, `xmp`, `textarea`), which are dispatched before the template-mode handler. Previously, a following table-structure tag such as the `<td>` in `<template><xmp></xmp><td>` created a spurious cell instead of being dropped.
+- Ignore a `</td>` or `</th>` end tag in a template cell when the matching cell type is not in table scope. Previously, `<template><td></th>x` closed the `<td>` on the mismatched `</th>`, so the following text became a sibling instead of staying in the cell.
+- Insert a non-breakout HTML-named start tag inside foreign content as a foreign element without running HTML in-body recovery. Foreign elements sharing a name with an HTML scope boundary or table foster target (an SVG or MathML `td`, `tr`, `caption`, ...) no longer stop scope walks or foster their children out. Previously, markup such as `<p><svg><article>`, `<p><math><td><listing>`, and `<table><math><tr><d>` broke out of or escaped the foreign subtree.
+- Ignore `</body>` and `</html>` whenever a select is open, not only when the select holds just options. Previously, an element retained inside the select (as in `<select><o></body><!--x-->`) let the end tag switch to the after-body mode, moving following content out of the select.
+- Close a raw-text or RCDATA element when its end tag name is terminated by `/` or a space even without a closing `>`. Previously, `<style></style/x` kept the unterminated end tag as text and left the element open.
+- Do not reconstruct the active formatting elements for `rb`, `rp`, `rt`, or `rtc` start tags. Previously, `<div><tt></div><rt>` wrapped the `<rt>` in a stale `<tt>` instead of inserting it directly.
+- Ignore `</form>` when the form element is not in scope, such as behind an open `<table>`. Previously, `<form><table></form><table>` removed the form from the stack, so the following table became a sibling of the form instead of nesting inside it.
+- Respect special-element blocking for generic end tags inside a template. Previously, `<template><dialog><object></dialog>e` closed the dialog through the open `<object>` instead of ignoring the end tag, so following text escaped the object.
+- Place a comment after an explicit `</head>` between the head and body even when the head contains a template. Previously, `<head><template></template></head><!--c-->` kept the comment inside the head.
+- Process in-body `base`, `link`, and `meta` start tags with the in-head rules without reconstructing the active formatting elements. Previously, markup such as `<p><u><section><meta>` could wrongly wrap the metadata element in a stale formatting element instead of inserting it directly.
+- Check the script-data-double-escape-start terminator against the real following character. Previously, a `<script` immediately followed by `</script>` inside a script (for example `<script><!--<script</script>`) entered the double-escaped state and swallowed the closing tag, leaving the script element unclosed.
+- Pop foreign-content ancestors before applying the heading start-tag rule so a heading that breaks out of MathML or SVG closes an open heading. Previously, markup such as `<h1><math><h3>` nested the second heading inside the first instead of making it a sibling.
+- Ignore `</html>` and `</body>` end tags in the "in select" insertion mode instead of switching to the after-body/after-html modes. Previously, markup such as `<select></html><!--c-->` moved trailing comments out of the still-open select to the document level.
+- Restrict character foster parenting to HTML table elements. Previously, a foreign SVG or MathML element that merely shared a name with a table foster target (such as an SVG `<tr>`) had following text foster-parented out of the foreign subtree instead of kept as its child, as in `<table><svg><tr>n`.
+- Force quirks mode when a doctype has trailing junk after its name without a PUBLIC or SYSTEM keyword. Previously, `<!DOCTYPE html foo>` was treated as no-quirks, so a following `<table>` incorrectly closed an open `<p>`.
+- Ignore end tags other than `</template>` in the "in template" insertion mode. Previously, the `</br>` special case still ran, so `<template></br>` created a stray `<br>` inside the template contents.
+- Let table-scoped end tags follow the foreign-content end-tag algorithm when the current node is a MathML or SVG element. Previously, an end tag such as `</th>` inside foreign content was routed to HTML table recovery, so `<svg><th><o></th>0` left the foreign cell open and misplaced the following text.
+- Ignore `</body>` and `</html>` when no body element is in scope. Previously, an open scope marker such as `<marquee>`, `<object>`, or `<applet>` did not prevent the switch to the after-body mode, so following content such as a comment in `<marquee></body><!--x-->` was placed outside the still-open element.
+- Close an escaped script comment on `<!-->`, whose `<!--` dashes already count toward the `-->`. Previously, `<script><!--><script></script>` treated the following `<script>` as a double-escape start and swallowed the closing tag, leaving the script element unclosed.
+- Push active-formatting markers only for HTML scope-marker elements. A foreign element sharing a name such as an SVG or MathML `<td>` no longer pushes a marker that would linger after the element is popped and block later formatting reconstruction, as in `<p><a><math><td></td><listing>U`.
 ## [3.5.0] - 2026-07-13
 ### Changed

{justhtml-3.5.0 → justhtml-3.6.0}/PKG-INFO RENAMED Viewed

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: justhtml
-Version: 3.5.0
+Version: 3.6.0
 Summary: A pure Python HTML5 parser that just works.
 Project-URL: Homepage, https://github.com/emilstenstrom/justhtml
 Project-URL: Issues, https://github.com/emilstenstrom/justhtml/issues

justhtml-3.6.0/project_ideas.md ADDED Viewed

@@ -0,0 +1,47 @@
+# Project Ideas
+## Treebuilder Refactors
+### Open element stack helper
+Replace direct `open_elements` list mutation with an `OpenElementStack` helper. The helper would own push, pop, remove, replace, and truncate operations, and it would keep cached counters for common stack queries such as open `<p>` elements, template presence, and scope-relevant names.
+Expected value: high maintainability, safer future performance caches, and fewer hidden invariants around `_note_open_element_pushed()` and `_note_open_element_removed()`.
+Performance requirement: no meaningful regression on the general `web100k` benchmark.
+### Active formatting list helper
+Move active formatting marker and entry operations into an `ActiveFormattingList` helper. It would own marker handling, duplicate detection, lookup by name, lookup by node, removal, and clear-to-marker behavior.
+Expected value: lower adoption-agency complexity and safer future optimizations around active formatting reconstruction.
+### Declarative insertion-mode dispatch
+Generate or centralize start/end tag dispatch tables for major insertion modes, especially `IN_BODY`, table modes, and select mode. Keep fallback behavior explicit so tag priority remains easy to audit.
+Expected value: possible hot-path speedup and clearer mode behavior. Needs before/after benchmark validation because dispatch overhead may vary.
+### Split insertion modes into smaller modules
+Break `modes.py` into mode-group modules such as head, body, table, select, template, and foreign-content handling while keeping the public `TreeBuilder` API unchanged.
+Expected value: large maintainability improvement and easier auditing. Performance should be neutral if done mechanically.
+### Stack index helpers
+Centralize repeated reverse stack scans into helpers such as `_find_open_element_index()`, `_find_last_open_element_index()`, and `_truncate_open_elements_from()`. Add caching only where benchmarks prove it helps.
+Expected value: clearer code first, optional targeted performance improvements later.
+### Separate parsing decisions from DOM mutation
+Create a tighter mutation API for element insertion, text insertion, foster parenting, child movement, and stack popping. Keep mode handlers focused on parser decisions.
+Expected value: maintainability and auditability. Runtime effect must be measured because extra calls could slow hot paths.
+### Narrow internal node types
+Replace some `Any` usage around stack nodes and active formatting entries with protocols or narrower internal types.
+Expected value: safer refactors and better mypy coverage. Runtime effect should be neutral.

{justhtml-3.5.0 → justhtml-3.6.0}/pyproject.toml RENAMED Viewed

@@ -1,7 +1,7 @@
 [project]
 name = "justhtml"
 authors = [{ name = "Emil Stenström", email = "emil@emilstenstrom.se" }]
-version = "3.5.0"
+version = "3.6.0"
 description = "A pure Python HTML5 parser that just works."
 readme = "README.md"
 license = { file = "LICENSE" }

{justhtml-3.5.0 → justhtml-3.6.0}/src/justhtml/parser/engine.py RENAMED Viewed

@@ -138,6 +138,15 @@ _P_CLOSING_START_TAGS = {
 _HEAD_NOSCRIPT_ALLOWED_START_TAGS = {"basefont", "bgsound", "link", "meta", "noframes", "style"}
 _HEAD_NOSCRIPT_VOID_START_TAGS = {"basefont", "bgsound"}
 _HEAD_ONLY_VOID_START_TAGS = {"basefont", "bgsound"}
+# In "in body", base/basefont/bgsound/link/meta are processed with the "in head"
+# rules (§13.2.6.4.7), which insert-and-pop without reconstructing the active
+# formatting elements. Reconstructing here would wrongly wrap them (and their
+# following siblings) in a stale formatting element.
+_INLINE_HEAD_VOID_START_TAGS = {"base", "basefont", "bgsound", "link", "meta"}
+# The in-body rb/rp/rt/rtc rules (§13.2.6.4.7) only generate implied end tags
+# and insert; unlike "any other start tag" they do not reconstruct the active
+# formatting elements.
+_RUBY_START_TAGS = {"rb", "rp", "rt", "rtc"}
 _HTML_VOID_COMPAT_TAGS = {"basefont", "bgsound", "frame", "keygen"}
 _DEFINITION_SCOPE_BOUNDARIES = frozenset(DEFINITION_SCOPE_TERMINATORS)
 _LIST_ITEM_SCOPE_BOUNDARIES = frozenset(LIST_ITEM_SCOPE_TERMINATORS)
@@ -160,6 +169,20 @@ _TEMPLATE_MODE_COLGROUP = "colgroup"
 _TEMPLATE_TABLE_CONTEXT_START_TAGS = {"caption", "col", "colgroup", "tbody", "td", "tfoot", "th", "thead", "tr"}
 _TEMPLATE_TABLE_BODY_IGNORED_START_TAGS = {"caption", "col", "colgroup", "table"}
 _TEMPLATE_ROW_STRUCTURE_START_TAGS = {"caption", "col", "colgroup", "tbody", "tfoot", "thead", "tr", "table"}
+# Start tags that move the "in template" mode into the body without going through
+# _handle_template_mode_start: void head elements handled inline plus the raw
+# text / RCDATA elements that are not head content (iframe, noembed, xmp,
+# textarea), which are dispatched before the template-mode handler runs.
+_TEMPLATE_INITIAL_BODY_SWITCH_TAGS = {
+    "base",
+    "basefont",
+    "bgsound",
+    "noframes",
+    "iframe",
+    "noembed",
+    "xmp",
+    "textarea",
+}
 _FRAMESET_BODY_OK_TAGS = {"div", "figure", "p", "param", "source", "track"} | FORMATTING_ELEMENTS
 _FRAMESET_BLOCKING_START_TAGS = {
     "applet",
@@ -931,8 +954,6 @@ class ParseEngine:
             return False
         if name in {"br", "p"}:
             return False
-        if name in _TABLE_SCOPED_END_TAGS:
-            return False
         crossed_integration_point = False
         for idx in range(len(stack) - 1, 0, -1):
@@ -942,6 +963,12 @@ class ParseEngine:
                     return True
                 if self._fragment_context_node is not None and node is self._fragment_context_node:
                     return True
+                if node.namespace in {None, "html", _PARSER_ONLY_NAMESPACE}:
+                    # The foreign-content walk reached a matching HTML element
+                    # (§13.2.6.5 steps 6-7): hand the token to the HTML end-tag
+                    # rules, which may splice a form or run adoption, rather than
+                    # popping the foreign elements sitting above it.
+                    return False
                 if self._track_tag_spans:
                     self._set_end_span(node, name, tag_start, tag_end)
                 del stack[idx:]
@@ -1279,6 +1306,7 @@ class ParseEngine:
         parent: Node
         if self._current_parent() is self._head or (
             not self._fragment
+            and not self._after_head
             and self._head is not None
             and not self._body_mode_seen
             and not self._body_has_content()
@@ -1931,7 +1959,14 @@ class ParseEngine:
                     self._insert_at(parent, position, node)
                     return
             foster = None
-            if parent.name in self._table_foster_targets:
+            # Foster parenting only applies in HTML table insertion modes. A
+            # foreign element that merely shares a name with a table foster
+            # target (e.g. an SVG or MathML "tr") keeps character data as its
+            # own child instead.
+            parent_is_table_foster = parent.name in self._table_foster_targets and getattr(
+                parent, "namespace", None
+            ) in {None, "html", _PARSER_ONLY_NAMESPACE}
+            if parent_is_table_foster:
                 is_table_space = text_is_space if text_is_space is not None else text.strip(_SPACE) == ""
                 if not is_table_space:
                     if pending_table_whitespace:
@@ -1961,7 +1996,7 @@ class ParseEngine:
                         node = self._new_text(text, source_pos) if self._track_node_locations else Text(text)
                         children.append(node)
                         node.parent = parent
-                if parent.name in self._table_foster_targets and is_table_space:
+                if parent_is_table_foster and is_table_space:
                     self._foster_next_table_whitespace = pending_table_whitespace + len(text)
             else:
                 foster_parent, position = foster
@@ -2002,9 +2037,15 @@ class ParseEngine:
                 else:
                     public_id = None
                     system_id = None
-                doctype = Doctype(name, public_id, system_id, force_quirks=name is None)
+                # Junk after the name with no PUBLIC/SYSTEM keyword lands in the
+                # "after DOCTYPE name" state, which sets the force-quirks flag
+                # (§13.2.5.55). The regex tolerates that trailing text, so detect
+                # it here to reproduce the quirks-mode transition.
+                trailing = raw[match.end() :].strip("\t\n\f\r ")
+                bogus = kind is None and bool(trailing)
+                doctype = Doctype(name, public_id, system_id, force_quirks=name is None or bogus)
                 doctype_error, self._quirks_mode = doctype_error_and_quirks(doctype, self._iframe_srcdoc)
-                if doctype_error and self._collect_errors:
+                if (doctype_error or bogus) and self._collect_errors:
                     self._emit_error(
                         "unknown-doctype",
                         doctype_end if gt != -1 else max(0, end - 1),
@@ -2297,8 +2338,21 @@ class ParseEngine:
             self._body_mode_seen = True
         if not self._fragment and name in {"html", "body"}:
-            parent_name = getattr(self._current_parent(), "name", None)
-            if self._find_open_index("table") is not None and parent_name not in {"body", "html"}:
+            if self._find_open_html_index("select") is not None:
+                # In the "in select" insertion mode, </html> and </body> are
+                # unhandled end tags and must be ignored rather than switching to
+                # the after-body/after-html modes. Any elements kept inside the
+                # open select remain part of that insertion mode.
+                return pos
+            if (
+                self._find_open_html_index("body") is not None
+                and self._find_open_index_before_boundary("body", _DEFAULT_SCOPE_BOUNDARIES) is None
+            ):
+                # With body on the open stack, </body> and </html> are ignored
+                # unless body is in scope (§13.2.6.4.7); an open scope marker such
+                # as marquee, object, or applet keeps it out of scope. When body
+                # is not on the stack yet (in head/after head), fall through so
+                # the head-to-body transition still runs.
                 return pos
             self._in_colgroup = False
             if self._head is not None and self._stack[-1] is self._head:
@@ -2369,11 +2423,16 @@ class ParseEngine:
             self._form_element = None
             if form_node is None:
                 return pos
+            # §13.2.6.4.7: ignore </form> when the form element is not in scope,
+            # e.g. behind an open <table>. Without this the form is removed from
+            # the stack and following content escapes it.
+            form_idx = self._find_open_index_before_boundary("form", _DEFAULT_SCOPE_BOUNDARIES)
+            if form_idx is None or self._stack[form_idx] is not form_node:
+                return pos
+            # form_node is confirmed in scope on the stack and implied end tags
+            # never pop a form, so the removal below always succeeds.
             self._generate_implied_end_tags()
-            try:
-                self._stack.remove(form_node)
-            except ValueError:
-                pass
+            self._stack.remove(form_node)
             return pos
         select_idx = self._find_open_html_index("select")
         if select_idx is not None and name not in {"optgroup", "option", "select", "selectedcontent", "template"}:
@@ -2494,7 +2553,7 @@ class ParseEngine:
         elif name == "summary":
             idx = self._find_open_index_before_boundary(name, _DEFAULT_SCOPE_BOUNDARIES)
         elif self._template_modes:
-            idx = self._find_open_index_in_current_scope(name)
+            idx = self._find_open_index_before_boundary(name, self._template_end_tag_boundaries(name, action))
         elif name not in self._special_elements and (action is None or not action.p_closing):
             idx = self._find_open_index_before_boundary(name, _GENERAL_END_TAG_BOUNDARIES)
         else:
@@ -2671,12 +2730,9 @@ class ParseEngine:
             self._close_until("select")
             return pos
-        if self._current_template_mode() == _TEMPLATE_MODE_INITIAL and name in {
-            "base",
-            "basefont",
-            "bgsound",
-            "noframes",
-        }:  # pragma: no cover - compiled sanitizer drops parser-only template contents
+        if (
+            self._current_template_mode() == _TEMPLATE_MODE_INITIAL and name in _TEMPLATE_INITIAL_BODY_SWITCH_TAGS
+        ):  # pragma: no cover - compiled sanitizer drops parser-only template contents
             self._set_current_template_mode(_TEMPLATE_MODE_BODY)  # pragma: no cover
         if in_parser_only_template and self._current_template_mode() == _TEMPLATE_MODE_COLGROUP:
@@ -2981,6 +3037,9 @@ class ParseEngine:
                 action is not None
                 and allowed
                 and action.head_content
+                # "after head" re-enters the head only for the metadata start
+                # tags it lists; noscript is not among them and starts the body.
+                and name != "noscript"
                 and self._head is not None
                 and not self._in_head_noscript
                 and not self._body_mode_seen
@@ -3088,12 +3147,7 @@ class ParseEngine:
         ):
             self._close_html_until("select")
-        if self._current_template_mode() == _TEMPLATE_MODE_INITIAL and name in {
-            "base",
-            "basefont",
-            "bgsound",
-            "noframes",
-        }:
+        if self._current_template_mode() == _TEMPLATE_MODE_INITIAL and name in _TEMPLATE_INITIAL_BODY_SWITCH_TAGS:
             self._set_current_template_mode(_TEMPLATE_MODE_BODY)
         if html_text_parsing and self._current_template_mode() == _TEMPLATE_MODE_COLGROUP:
@@ -3202,7 +3256,14 @@ class ParseEngine:
                 )
                 return pos
-        if html_text_parsing and action is not None and action.active_formatting:
+        if (
+            action is not None
+            and action.active_formatting
+            and (html_text_parsing or self._is_foreign_breakout_start(name, attrs))
+        ):
+            # A formatting element that breaks out of foreign content (e.g. <i>
+            # inside <svg>) becomes an HTML formatting element and must join the
+            # active formatting list so it is reconstructed later.
             return self._parse_formatting_start(name, attrs, pos, tag_start=tag_start, tag_end=tag_end)
         if html_text_parsing and name == "menuitem" and self._active_formatting_dirty:
@@ -3270,6 +3331,11 @@ class ParseEngine:
             and not self._body_has_content()
         ):
             parent = self._head
+        elif not html_text_parsing and not self._is_foreign_breakout_start(name, attrs):
+            # A non-breakout start tag inside foreign content is inserted as a
+            # foreign element in the current namespace; the HTML in-body repair
+            # (p-closing, heading/list handling, ...) must not run.
+            parent = self._current_parent()
         else:
             self._repair_stack_for_start(name)
             parent = self._current_parent()
@@ -3312,8 +3378,9 @@ class ParseEngine:
         reconstruct_before_insert = (
             name not in _TABLE_STRUCTURE_START_TAGS
-            and name not in _HEAD_ONLY_VOID_START_TAGS
+            and name not in _INLINE_HEAD_VOID_START_TAGS
             and name not in {"param", "source", "track"}
+            and name not in _RUBY_START_TAGS
             and name != "template"
             and (action is None or not action.p_closing)
         )
@@ -3484,9 +3551,11 @@ class ParseEngine:
                 "html",
             }:  # pragma: no branch - opposite edge requires invalid parser state
                 self._enter_template_mode()  # pragma: no cover - unreachable after parser-state guards
-            if name in self._table_cell_tags:
-                self._push_active_formatting_marker()
-            elif name in _ACTIVE_FORMATTING_MARKER_TAGS:
+            # Active-formatting markers are pushed only for the HTML scope-marker
+            # elements. A foreign element that merely shares a name (e.g. an SVG
+            # or MathML "td") must not push one, or the stray marker would block
+            # later reconstruction once the foreign element is popped.
+            if is_html_namespace and (name in self._table_cell_tags or name in _ACTIVE_FORMATTING_MARKER_TAGS):
                 self._push_active_formatting_marker()
         return node
@@ -3540,9 +3609,10 @@ class ParseEngine:
             self._stack.append(node)
             if type(node) is Template and node.namespace in {None, "html"}:
                 self._enter_template_mode()
-            if name in self._table_cell_tags:
-                self._push_active_formatting_marker()
-            elif name in _ACTIVE_FORMATTING_MARKER_TAGS:
+            # Only HTML scope-marker elements push an active-formatting marker; a
+            # foreign element sharing a name (e.g. an SVG or MathML "td") must not,
+            # or the stray marker blocks reconstruction once it is popped.
+            if is_html_namespace and (name in self._table_cell_tags or name in _ACTIVE_FORMATTING_MARKER_TAGS):
                 self._push_active_formatting_marker()
         return node
@@ -3558,6 +3628,25 @@ class ParseEngine:
             attrs = self._prepare_foreign_attrs(namespace, attrs)
         return name, attrs, namespace
+    def _pop_foreign_for_breakout(self) -> None:
+        # Pop the foreign-content elements a breakout start tag returns through,
+        # stopping at the fragment context node and at integration points where
+        # HTML content resumes.
+        while (
+            len(self._stack) > 1
+            and self._stack[-1].namespace not in {None, "html", _PARSER_ONLY_NAMESPACE}
+            and self._stack[-1] is not self._fragment_context_node
+            and not self._is_html_integration_point(self._stack[-1])
+            and not self._is_mathml_text_integration_point(self._stack[-1])
+        ):
+            self._stack.pop()
+    def _is_foreign_breakout_start(self, name: str, attrs: dict[str, str | None]) -> bool:
+        # Start tags that break out of foreign content back to HTML (§13.2.6.5).
+        return name in FOREIGN_BREAKOUT_ELEMENTS or (
+            name == "font" and any(attr.lower() in {"color", "face", "size"} for attr in attrs)
+        )
     def _namespace_for_raw_start(self, name: str, attrs: dict[str, str | None]) -> str:
         current = self._stack[-1] if self._stack else None
         current_ns = getattr(current, "namespace", None)
@@ -3568,19 +3657,9 @@ class ParseEngine:
                 return "svg" if name == "svg" else "math" if name == "math" else "html"
             if current_ns == "math" and current.name == "annotation-xml" and name == "svg":
                 return "svg"
-            breaks_out = name in FOREIGN_BREAKOUT_ELEMENTS or (
-                name == "font" and any(attr.lower() in {"color", "face", "size"} for attr in attrs)
-            )
-            if not breaks_out:
+            if not self._is_foreign_breakout_start(name, attrs):
                 return current_ns or "html"
-            while (
-                len(self._stack) > 1
-                and self._stack[-1].namespace not in {None, "html", _PARSER_ONLY_NAMESPACE}
-                and self._stack[-1] is not self._fragment_context_node
-                and not self._is_html_integration_point(self._stack[-1])
-                and not self._is_mathml_text_integration_point(self._stack[-1])
-            ):
-                self._stack.pop()
+            self._pop_foreign_for_breakout()
         if name == "svg":
             return "svg"
@@ -3732,6 +3811,15 @@ class ParseEngine:
                 return None
         return None
+    def _template_end_tag_boundaries(self, name: str, action: TagAction | None) -> frozenset[str]:
+        # Generic in-body end tags inside a template still obey scope: block and
+        # special names close within the default scope, while other names stop at
+        # any special element (§13.2.6.4.7 "any other end tag"). The template
+        # element is a member of both sets, so the walk cannot escape it.
+        if name in self._special_elements or (action is not None and action.p_closing):
+            return _DEFAULT_SCOPE_BOUNDARIES
+        return _GENERAL_END_TAG_BOUNDARIES
     def _find_open_heading_index(self) -> int | None:
         stack = self._stack
         for idx in range(len(stack) - 1, 0, -1):  # pragma: no branch
@@ -3994,7 +4082,7 @@ class ParseEngine:
                 del self._stack[table_idx:]
                 self._insert_template_mode_element(name, attrs, self_closing)
                 return pos
-            if table_idx is not None and name in {
+            if name in {
                 "caption",
                 "col",
                 "colgroup",
@@ -4005,7 +4093,12 @@ class ParseEngine:
                 "thead",
                 "tr",
             }:
-                del self._stack[table_idx + 1 :]
+                # Clear back to the table context before opening table structure.
+                # In a template with no table element, that context is the
+                # template contents, so an open caption is popped first.
+                clear_idx = table_idx if table_idx is not None else template_idx
+                if clear_idx is not None:  # pragma: no branch - a template is always open in this mode
+                    del self._stack[clear_idx + 1 :]
             if name in {"tbody", "tfoot", "thead"}:
                 self._set_current_template_mode(_TEMPLATE_MODE_TABLE_BODY)
                 self._insert_template_mode_element(name, attrs, self_closing)
@@ -4134,9 +4227,20 @@ class ParseEngine:
             return False  # pragma: no cover - unreachable after parser-state guards
         if name in {"html", "head", "body"}:
             return True
+        if mode == _TEMPLATE_MODE_INITIAL:
+            # "In template" handles only </template>; every other end tag with
+            # nothing to close is a parse error that is ignored (§13.2.6.4.18),
+            # including the </br> special case. An element left open in the
+            # template (e.g. a scripting-disabled <noscript>) is still closeable
+            # through the normal in-body handling.
+            if name == "template":
+                return False
+            return self._find_open_index_before_boundary(name, _TEMPLATE_SCOPE_BOUNDARIES) is None
         if mode == _TEMPLATE_MODE_CELL:
             if name in self._table_cell_tags:
-                return self._close_template_cell()
+                # Only the matching cell type closes; </th> with a <td> open (or
+                # vice versa) leaves the cell open.
+                return self._close_template_cell(name)
             if name in {"table", "tbody", "tfoot", "thead", "tr"}:
                 if self._find_open_index_before_boundary(name, _TEMPLATE_SCOPE_BOUNDARIES) is None:
                     return True
@@ -4200,11 +4304,16 @@ class ParseEngine:
                 return False
         return False  # pragma: no cover - template scope always terminates the scan
-    def _close_template_cell(self) -> bool:
+    def _close_template_cell(self, name: str | None = None) -> bool:
         stack = self._stack
         for idx in range(len(stack) - 1, 0, -1):
             node_name = stack[idx].name
             if node_name in self._table_cell_tags:
+                # A </td>/</th> end tag only closes the matching cell type; a
+                # mismatched end tag (e.g. </th> for an open <td>) is left for
+                # the generic handling, which ignores it.
+                if name is not None and node_name != name:
+                    return False
                 self._mark_active_formatting_dirty()
                 self._clear_active_formatting_to_marker()
                 del stack[idx:]
@@ -4229,7 +4338,8 @@ class ParseEngine:
     def _close_until_before_boundary(self, name: str, boundaries: frozenset[str]) -> bool:
         stack = self._stack
         for idx in range(len(stack) - 1, 0, -1):
-            node_name = getattr(stack[idx], "name", None)
+            node = stack[idx]
+            node_name = getattr(node, "name", None)
             if node_name == name:
                 if (
                     self._fragment_context_node is not None and stack[idx] is self._fragment_context_node
@@ -4238,7 +4348,11 @@ class ParseEngine:
                 self._mark_active_formatting_dirty()
                 del stack[idx:]
                 return True
-            if node_name in boundaries:
+            # Scope boundaries are HTML elements. A foreign element that merely
+            # shares a name with a boundary (e.g. an SVG or MathML "td") must not
+            # stop the scan; callers guard this walk with an in-scope check that
+            # already accounts for foreign integration points.
+            if getattr(node, "namespace", None) in {None, "html", _PARSER_ONLY_NAMESPACE} and node_name in boundaries:
                 return False
         return False
@@ -4301,9 +4415,14 @@ class ParseEngine:
                 self._stack.pop()
             return
-        if name in HEADING_ELEMENTS and getattr(self._stack[-1], "name", None) in HEADING_ELEMENTS:
-            self._mark_active_formatting_dirty()
-            self._stack.pop()
+        if name in HEADING_ELEMENTS:
+            # Headings are foreign-content breakout elements; pop the foreign
+            # ancestors first so the "current node is a heading" check runs
+            # against the HTML element the breakout returns to (§13.2.6.5).
+            self._pop_foreign_for_breakout()
+            if getattr(self._stack[-1], "name", None) in HEADING_ELEMENTS:
+                self._mark_active_formatting_dirty()
+                self._stack.pop()
     def _repair_table_for_start(self, name: str) -> None:
         table_idx = self._find_open_index("table")
@@ -4427,7 +4546,12 @@ class ParseEngine:
         self._refresh_active_formatting_dirty()
     def _foster_parent_for(self, parent: Node, *, for_tag: str | None = None) -> tuple[Node, int] | None:
-        if getattr(parent, "name", None) not in self._table_foster_targets:
+        # Foster parenting only applies within an HTML table. A foreign element
+        # sharing a name with a foster target (e.g. an SVG or MathML "tr") keeps
+        # its children rather than fostering them out.
+        if getattr(parent, "name", None) not in self._table_foster_targets or getattr(
+            parent, "namespace", None
+        ) not in {None, "html", _PARSER_ONLY_NAMESPACE}:
             return None
         if (
             for_tag is not None and for_tag in self._table_allowed_children
@@ -4481,7 +4605,7 @@ class ParseEngine:
     def _find_script_start_marker(self, pos: int, end: int) -> int:
         return _scanner.find_script_start_marker(
-            self._html_input, self._lower_input, pos, end
+            self._html_input, self._lower_input, pos, end, end
         )  # pragma: no cover - unreachable after parser-state guards
     def _find_tag_end(self, pos: int, end: int) -> int:

{justhtml-3.5.0 → justhtml-3.6.0}/src/justhtml/parser/scanner.py RENAMED Viewed

@@ -38,7 +38,11 @@ def find_rawtext_end_tag(html: str, lower: str, name: str, pos: int, end: int) -
             continue
         tag_end = find_tag_end(html, after_name, end)
         if tag_end == -1:
-            if after_name < end and not html[after_name:end].strip(SPACE + "/"):
+            if after_name < end:
+                # A space or "/" terminated the end-tag name but no ">" follows.
+                # At EOF the end tag is still emitted (its attributes dropped), so
+                # the raw text ends here. Only a bare "</name" at EOF, with no
+                # terminator, stays raw text.
                 return close, end
             search = after_name
             continue
@@ -59,10 +63,14 @@ def find_script_end_tag(html: str, lower: str, pos: int, end: int) -> tuple[int
             if comment_start == -1 or close < comment_start:
                 return close, next_pos
             escaped = True
-            search = comment_start + 4
+            # After "<!--" the tokenizer is in the script-data-escaped-dash-dash
+            # state, so the two trailing dashes already count toward a closing
+            # "-->". Resume the comment-end search at those dashes so "<!-->"
+            # leaves the escaped state instead of being treated as still open.
+            search = comment_start + 2
             continue
-        script_start = find_script_start_marker(html, lower, search, close) if not double_escaped else -1
+        script_start = find_script_start_marker(html, lower, search, close, end) if not double_escaped else -1
         comment_end = lower.find("-->", search, close)
         if (
             comment_end != -1
@@ -85,20 +93,21 @@ def find_script_end_tag(html: str, lower: str, pos: int, end: int) -> tuple[int
                     double_escaped = False
                     search = close + 8
                     continue
-            double_end = lower.find("</script", search, close)
-            if double_end != -1:
-                after_name = double_end + 8
-                if after_name >= end or html[after_name] in TAG_END_NAME_STOP:
-                    double_escaped = False
-                    search = after_name
-                    continue
+            # No earlier terminated "</script" can precede `close`: find_rawtext_end_tag
+            # already returns the first terminated end tag, so the region between
+            # `search` and `close` holds only non-terminated occurrences.
             double_escaped = False
             search = next_pos
             continue
         return close, next_pos
-def find_script_start_marker(html: str, lower: str, pos: int, end: int) -> int:
+def find_script_start_marker(html: str, lower: str, pos: int, end: int, text_end: int) -> int:
+    # `end` bounds where a "<script" marker may start; `text_end` bounds the real
+    # input so the script-data-double-escape-start terminator is checked against
+    # the actual following character. When these differ, a marker whose name ends
+    # exactly at `end` still has a following character (e.g. the "<" of a trailing
+    # "</script>") that must be inspected rather than treated as end-of-input.
     search = pos
     needle = "<script"
     needle_len = len(needle)
@@ -107,6 +116,6 @@ def find_script_start_marker(html: str, lower: str, pos: int, end: int) -> int:
         if start == -1:
             return -1
         after_name = start + needle_len
-        if after_name >= end or html[after_name] in TAG_END_NAME_STOP:
+        if after_name >= text_end or html[after_name] in TAG_END_NAME_STOP:
             return start
         search = after_name

{justhtml-3.5.0 → justhtml-3.6.0}/tests/justhtml-tests/document_shell_recovery.dat RENAMED Viewed

@@ -91,3 +91,16 @@
 |   <body>
 |     "x"
 |     <meta>
+#data
+<!DOCTYPE html foo><p><table>
+#errors
+(1,19): unknown-doctype
+#document
+| <!DOCTYPE html>
+| <html>
+|   <head>
+|   <body>
+|     <p>
+|       <table>

justhtml 3.5.0__tar.gz → 3.6.0__tar.gz

justhtml 3.5.0tar.gz → 3.6.0tar.gz