justhtml 2.0.0__tar.gz → 2.2.0__tar.gz

{justhtml-2.0.0 → justhtml-2.2.0}/CHANGELOG.md

@@ -5,6 +5,35 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## [Unreleased]
+
+## [2.2.0] - 2026-06-07
+
+### Fixed
+- Handle `<select><selectedcontent></selectedcontent></select>` without crashing when no `<option>` is present, replace selectedcontent fallback content during parser finalization, and avoid repeated selectedcontent subtree scans.
+- Preserve source order and tag text when escape-mode sanitization handles disallowed rawtext/RCDATA elements with attributed or self-closing end tags.
+- Make `stream()` use namespace-aware tokenizer context for SVG/MathML CDATA, rawtext decisions, self-closing foreign tags, and foreign end-tag stack updates.
+- Use the correct initial tokenizer states for HTML fragment contexts such as `<title>`, `<textarea>`, `<script>`, `<style>`, and scripting-disabled `<noscript>`.
+- Use HTML rawtext/RCDATA tokenizer states for text-like elements inside SVG/MathML HTML integration points and MathML text integration points.
+- Generate implied end tags before removing `<form>` on `</form>` so following controls do not remain inside still-open descendants.
+- Keep `<form>` elements inside `<template>` from claiming the global form pointer, including table-template form insertion.
+- Close open `<p>` elements correctly around `<option>`, `<optgroup>`, `<hr>`, `<p>`, and `<div>` starts in `<select>` parsing.
+- Close `<template>` correctly when `</template>` is seen while parsing inside `<select>`.
+- Keep `</p>` and `</br>` foreign-content breakouts inside MathML text integration points such as `<mi>` and `<mtext>`.
+- Align customizable `<select>` parsing with Chromium for phantom `</p>` handling and generic custom child elements.
+
+### Security
+- (Severity: Low) Strip invisible Unicode during URL sink validation even when general invisible-Unicode stripping is disabled. Previously, custom policies using `strip_invisible_unicode=False` could preserve scheme-obfuscated values such as `javascript\u200b:` in otherwise URL-validated attributes.
+
+## [2.1.0] - 2026-06-06
+
+### Performance
+- Avoid quadratic work for deeply nested HTML during default sanitization by carrying foreign-content context through the transform traversal instead of rescanning each node's ancestors, and by short-circuiting repeated `<p>` scope checks when no `<p>` is open.
+- Speed up default sanitization by constructing selector matchers only for transforms that actually need selector matching.
+- Speed up sanitizer attribute and text cleanup by skipping URL-sink resolution for non-URL attributes and bypassing invisible-Unicode regex scans for ASCII values.
+- Avoid unnecessary default work by skipping selectedcontent finalization when no `<select>` was parsed and by bypassing invisible-Unicode transform helper calls for ASCII node values.
+- Skip the sanitizer rawtext hardening pass when the active policy cannot preserve `<script>` or `<style>` elements.
+
 ## [2.0.0] - 2026-05-24
 
 ### Changed

justhtml-2.2.0/PKG-INFO

@@ -0,0 +1,187 @@
+Metadata-Version: 2.4
+Name: justhtml
+Version: 2.2.0
+Summary: A pure Python HTML5 parser that just works.
+Project-URL: Homepage, https://github.com/emilstenstrom/justhtml
+Project-URL: Issues, https://github.com/emilstenstrom/justhtml/issues
+Author-email: Emil Stenström <emil@emilstenstrom.se>
+License: MIT License
+        
+        Copyright (c) 2025 Emil Stenström (JustHTML)
+        Copyright (c) 2014-2017, The html5ever Project Developers (html5ever inspiration)
+        Copyright (c) 2006-2013 James Graham, Sam Sneddon, and
+        other contributors (html5lib-tests)
+        
+        Permission is hereby granted, free of charge, to any person obtaining a copy
+        of this software and associated documentation files (the "Software"), to deal
+        in the Software without restriction, including without limitation the rights
+        to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+        copies of the Software, and to permit persons to whom the Software is
+        furnished to do so, subject to the following conditions:
+        
+        The above copyright notice and this permission notice shall be included in all
+        copies or substantial portions of the Software.
+        
+        THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+        IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+        FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+        AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+        LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+        OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+        SOFTWARE.
+License-File: LICENSE
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Requires-Python: >=3.10
+Provides-Extra: benchmark
+Requires-Dist: beautifulsoup4; extra == 'benchmark'
+Requires-Dist: html5-parser; extra == 'benchmark'
+Requires-Dist: html5lib; extra == 'benchmark'
+Requires-Dist: lxml; extra == 'benchmark'
+Requires-Dist: markupever; extra == 'benchmark'
+Requires-Dist: psutil; extra == 'benchmark'
+Requires-Dist: selectolax>=0.4.8; extra == 'benchmark'
+Requires-Dist: zstandard; extra == 'benchmark'
+Provides-Extra: dev
+Requires-Dist: build; extra == 'dev'
+Requires-Dist: coverage; extra == 'dev'
+Requires-Dist: mypy>=1.0; (platform_python_implementation != 'PyPy') and extra == 'dev'
+Requires-Dist: pre-commit; extra == 'dev'
+Requires-Dist: ruff==0.14.7; extra == 'dev'
+Requires-Dist: twine; extra == 'dev'
+Description-Content-Type: text/markdown
+
+# JustHTML
+
+HTML from the real web is messy. It is often malformed, user supplied, scraped from unknown pages, or headed for a browser where small parsing differences can become security bugs.
+
+JustHTML gives Python projects one small dependency for the common HTML jobs:
+
+- parse HTML like a browser, including broken markup
+- sanitize untrusted HTML by default
+- query with CSS selectors
+- transform, serialize, extract text, or convert to Markdown
+- run anywhere Python runs, with no C extension and no system package to install
+
+```bash
+pip install justhtml
+```
+
+Requires Python 3.10 or later.
+
+[Documentation](https://emilstenstrom.github.io/justhtml/) | [Comparison](docs/comparison.md) | [Playground](https://emilstenstrom.github.io/justhtml/playground/) | [Security policy](SECURITY.md)
+
+![JustHTML turns messy unsafe HTML into a sanitized, queryable DOM, then serializes it to text, Markdown, or HTML.](assets/justhtml-readme-explainer.png)
+
+## Why Use It?
+
+Most Python HTML libraries optimize for one part of the problem.
+
+`html.parser` is built in, but not HTML5-correct. BeautifulSoup is convenient, but depends heavily on the parser underneath. `lxml` and C/Rust-backed parsers are fast, but usually leave sanitization as a separate concern. `html5lib` and Bleach shaped the Python ecosystem, but both are no longer the obvious foundation for new projects.
+
+JustHTML is for applications that want a boring, inspectable, pure-Python default:
+
+- **Correct parsing:** browser-style HTML5 recovery, tested against the official html5lib fixtures.
+- **Safe by default:** `JustHTML(html)` sanitizes before you query or serialize.
+- **One DOM:** parse once, then sanitize, query, transform, serialize, extract text, or produce Markdown.
+- **Easy deployment:** zero runtime dependencies, no compiler, works on PyPy and Pyodide.
+- **Honest tradeoff:** if you are parsing terabytes of trusted HTML, use a C/Rust parser. If you need reliable handling of untrusted or malformed HTML inside a Python app, use JustHTML.
+
+## Quick Start
+
+```python
+from justhtml import JustHTML
+
+doc = JustHTML(
+    "<p>Hello<script>alert(1)</script> "
+    "<a href='javascript:alert(1)'>bad</a> "
+    "<a href='https://example.com'>ok</a></p>",
+    fragment=True,
+)
+
+print(doc.to_html(pretty=False))
+# => <p>Hello <a>bad</a> <a href="https://example.com">ok</a></p>
+```
+
+Sanitization is enabled by default. Disable it only for trusted input:
+
+```python
+doc = JustHTML("<main><p class='intro'>Hello</p></main>", sanitize=False)
+intro = doc.query_one("p.intro")
+
+print(intro.to_text())
+# => Hello
+```
+
+## What You Can Do
+
+```python
+from justhtml import JustHTML, Linkify, SetAttrs, Unwrap
+
+doc = JustHTML(
+    "<p>Hello <span>world</span> example.com</p>",
+    fragment=True,
+    sanitize=False,
+    transforms=[
+        Unwrap("span"),
+        Linkify(),
+        SetAttrs("a", rel="nofollow"),
+    ],
+)
+
+print(doc.to_html(pretty=False))
+# => <p>Hello world <a href="http://example.com" rel="nofollow">example.com</a></p>
+```
+
+JustHTML includes:
+
+- [CSS selectors](docs/selectors.md): `query()` and `query_one()`
+- [Sanitization](docs/sanitization.md): allowlisted HTML cleaning, URL policies, inline CSS controls
+- [Transforms](docs/transforms.md): unwrap, drop, edit attributes, linkify, compose cleanup pipelines
+- [Text output](docs/text.md): `to_text()` and Markdown generation
+- [Builder API](docs/building.md): construct nodes directly from Python
+- [Streaming](docs/streaming.md): process large inputs incrementally
+- [Bleach migration guide](docs/bleach-migration.md): move existing sanitizer code to JustHTML policies
+
+## Command Line
+
+```bash
+# Pretty-print an HTML file
+justhtml index.html
+
+# Parse from stdin
+curl -s https://example.com | justhtml -
+
+# Extract text from selected nodes
+justhtml index.html --selector "main p" --format text
+
+# Convert selected HTML to Markdown
+justhtml index.html --selector "article" --format markdown
+```
+
+## Correctness
+
+JustHTML is tested against the official html5lib tokenizer, tree-construction, serializer, and encoding fixtures, plus project-specific sanitizer, selector, transform, CLI, and regression tests.
+
+The current test summary is 10,257 passing tests with 100% line and branch coverage. See [Correctness Testing](docs/correctness.md) for details.
+
+## Documentation
+
+- [Quickstart](docs/quickstart.md)
+- [Comparison](docs/comparison.md)
+- [API Reference](docs/api.md)
+- [Sanitization & Security](docs/sanitization.md)
+- [Migrating from Bleach](docs/bleach-migration.md)
+- [Command Line](docs/cli.md)
+- [Full documentation site](https://emilstenstrom.github.io/justhtml/)
+
+## Security
+
+JustHTML sanitizes by default, but output safety still depends on where you put it. HTML body output is not automatically safe inside JavaScript, CSS, URL attributes, or other contexts.
+
+For the supported-version policy and vulnerability reporting, see [SECURITY.md](SECURITY.md).
+
+## License
+
+MIT. Free to use for commercial and non-commercial projects.

justhtml-2.2.0/README.md

@@ -0,0 +1,133 @@
+# JustHTML
+
+HTML from the real web is messy. It is often malformed, user supplied, scraped from unknown pages, or headed for a browser where small parsing differences can become security bugs.
+
+JustHTML gives Python projects one small dependency for the common HTML jobs:
+
+- parse HTML like a browser, including broken markup
+- sanitize untrusted HTML by default
+- query with CSS selectors
+- transform, serialize, extract text, or convert to Markdown
+- run anywhere Python runs, with no C extension and no system package to install
+
+```bash
+pip install justhtml
+```
+
+Requires Python 3.10 or later.
+
+[Documentation](https://emilstenstrom.github.io/justhtml/) | [Comparison](docs/comparison.md) | [Playground](https://emilstenstrom.github.io/justhtml/playground/) | [Security policy](SECURITY.md)
+
+![JustHTML turns messy unsafe HTML into a sanitized, queryable DOM, then serializes it to text, Markdown, or HTML.](assets/justhtml-readme-explainer.png)
+
+## Why Use It?
+
+Most Python HTML libraries optimize for one part of the problem.
+
+`html.parser` is built in, but not HTML5-correct. BeautifulSoup is convenient, but depends heavily on the parser underneath. `lxml` and C/Rust-backed parsers are fast, but usually leave sanitization as a separate concern. `html5lib` and Bleach shaped the Python ecosystem, but both are no longer the obvious foundation for new projects.
+
+JustHTML is for applications that want a boring, inspectable, pure-Python default:
+
+- **Correct parsing:** browser-style HTML5 recovery, tested against the official html5lib fixtures.
+- **Safe by default:** `JustHTML(html)` sanitizes before you query or serialize.
+- **One DOM:** parse once, then sanitize, query, transform, serialize, extract text, or produce Markdown.
+- **Easy deployment:** zero runtime dependencies, no compiler, works on PyPy and Pyodide.
+- **Honest tradeoff:** if you are parsing terabytes of trusted HTML, use a C/Rust parser. If you need reliable handling of untrusted or malformed HTML inside a Python app, use JustHTML.
+
+## Quick Start
+
+```python
+from justhtml import JustHTML
+
+doc = JustHTML(
+    "<p>Hello<script>alert(1)</script> "
+    "<a href='javascript:alert(1)'>bad</a> "
+    "<a href='https://example.com'>ok</a></p>",
+    fragment=True,
+)
+
+print(doc.to_html(pretty=False))
+# => <p>Hello <a>bad</a> <a href="https://example.com">ok</a></p>
+```
+
+Sanitization is enabled by default. Disable it only for trusted input:
+
+```python
+doc = JustHTML("<main><p class='intro'>Hello</p></main>", sanitize=False)
+intro = doc.query_one("p.intro")
+
+print(intro.to_text())
+# => Hello
+```
+
+## What You Can Do
+
+```python
+from justhtml import JustHTML, Linkify, SetAttrs, Unwrap
+
+doc = JustHTML(
+    "<p>Hello <span>world</span> example.com</p>",
+    fragment=True,
+    sanitize=False,
+    transforms=[
+        Unwrap("span"),
+        Linkify(),
+        SetAttrs("a", rel="nofollow"),
+    ],
+)
+
+print(doc.to_html(pretty=False))
+# => <p>Hello world <a href="http://example.com" rel="nofollow">example.com</a></p>
+```
+
+JustHTML includes:
+
+- [CSS selectors](docs/selectors.md): `query()` and `query_one()`
+- [Sanitization](docs/sanitization.md): allowlisted HTML cleaning, URL policies, inline CSS controls
+- [Transforms](docs/transforms.md): unwrap, drop, edit attributes, linkify, compose cleanup pipelines
+- [Text output](docs/text.md): `to_text()` and Markdown generation
+- [Builder API](docs/building.md): construct nodes directly from Python
+- [Streaming](docs/streaming.md): process large inputs incrementally
+- [Bleach migration guide](docs/bleach-migration.md): move existing sanitizer code to JustHTML policies
+
+## Command Line
+
+```bash
+# Pretty-print an HTML file
+justhtml index.html
+
+# Parse from stdin
+curl -s https://example.com | justhtml -
+
+# Extract text from selected nodes
+justhtml index.html --selector "main p" --format text
+
+# Convert selected HTML to Markdown
+justhtml index.html --selector "article" --format markdown
+```
+
+## Correctness
+
+JustHTML is tested against the official html5lib tokenizer, tree-construction, serializer, and encoding fixtures, plus project-specific sanitizer, selector, transform, CLI, and regression tests.
+
+The current test summary is 10,257 passing tests with 100% line and branch coverage. See [Correctness Testing](docs/correctness.md) for details.
+
+## Documentation
+
+- [Quickstart](docs/quickstart.md)
+- [Comparison](docs/comparison.md)
+- [API Reference](docs/api.md)
+- [Sanitization & Security](docs/sanitization.md)
+- [Migrating from Bleach](docs/bleach-migration.md)
+- [Command Line](docs/cli.md)
+- [Full documentation site](https://emilstenstrom.github.io/justhtml/)
+
+## Security
+
+JustHTML sanitizes by default, but output safety still depends on where you put it. HTML body output is not automatically safe inside JavaScript, CSS, URL attributes, or other contexts.
+
+For the supported-version policy and vulnerability reporting, see [SECURITY.md](SECURITY.md).
+
+## License
+
+MIT. Free to use for commercial and non-commercial projects.

{justhtml-2.0.0 → justhtml-2.2.0}/SECURITY.md

@@ -15,7 +15,8 @@
 
 | Version | Supported                                |
 | ------- | ---------------------------------------- |
-| 1.x     | :white_check_mark: (until 2.0 is released) |
+| 2.x     | :white_check_mark: (until 3.0 is released) |
+| 1.x     | :x:                                      |
 | < 1.0   | :x:                                      |
 
 ## Security Domains

{justhtml-2.0.0 → justhtml-2.2.0}/benchmarks/correctness.py

@@ -18,8 +18,8 @@ from enum import Enum
 from pathlib import Path
 
 from justhtml import JustHTML
-from justhtml.context import FragmentContext
-from justhtml.serialize import to_test_format
+from justhtml.parser.context import FragmentContext
+from justhtml.serializer import to_test_format
 
 # Available parsers
 PARSERS = ["justhtml", "html5lib", "html5_parser", "lxml", "bs4", "html.parser", "selectolax", "markupever"]

justhtml-2.2.0/docs/comparison.md

@@ -0,0 +1,82 @@
+[← Back to docs](index.md)
+
+# Comparison
+
+Use JustHTML when you want browser-grade HTML parsing, safe-by-default sanitization, CSS selectors, transforms, text extraction, and serialization in one pure-Python package.
+
+Use a different tool when one narrow requirement matters more than the whole pipeline: maximum throughput, a BeautifulSoup-specific API, XPath-heavy XML work, or integration with an existing lxml tree.
+
+## At a Glance
+
+| Tool | HTML5 parsing [1][2] | Speed | Query | Build | Sanitize | Notes |
+|------|------------------------------------------|-------|----------|-------|------------------|-------|
+| **JustHTML**<br>Pure Python | ✅&nbsp;100% | ⚡ Fast | ✅ CSS selectors | ✅ `element()` | ✅ Built-in | Correct, secure, easy to install, and fast enough. |
+| **`selectolax`**<br>Python wrapper of C-based Lexbor | ✅&nbsp;100% | 🚀 Very Fast | ✅ CSS selectors | ✅ `create_node()` | ❌ Needs sanitization | Very fast and spec-compliant. |
+| **Chromium**<br>browser engine | ✅&nbsp;99.5% | 🚀&nbsp;Very&nbsp;Fast | — | — | — | — |
+| **WebKit**<br>browser engine | ✅ 98.4% | 🚀 Very Fast | — | — | — | — |
+| **Firefox**<br>browser engine | ✅ 97.6% | 🚀 Very Fast | — | — | — | — |
+| **`markupever`**<br>Python wrapper of Rust-based html5ever | 🟡 89% | 🚀 Very Fast | ✅ CSS selectors | ✅ `TreeDom .create_*()` | ❌ Needs sanitization | Fast and mostly correct, but missing benchmarked capabilities count against compliance. |
+| **`html5lib`**<br>Pure Python | 🟡 86% | 🐢 Slow | 🟡 XPath (lxml) | 🟡 Tree API | 🔴 [Deprecated](https://github.com/html5lib/html5lib-python/issues/443) | Unmaintained reference implementation; incomplete coverage of the tree-construction fixtures. |
+| **`html5_parser`**<br>Python wrapper of C-based Gumbo | 🔴 49% | 🚀 Very Fast | 🟡 XPath (lxml) | 🟡 `etree` (lxml) | ❌ Needs sanitization | Fast, but its public tree API loses information needed by many fixtures. |
+| **`BeautifulSoup`**<br>Pure Python | 🔴 <1% (default) | 🐢 Slow | 🟡 Custom API | ✅ `new_tag()` API | ❌ Needs sanitization | Wraps `html.parser` (default). Can use lxml or html5lib. |
+| **`html.parser`**<br>Python stdlib | 🔴 <1% | ⚡ Fast | ❌ None | ❌ None | ❌ Needs sanitization | Standard library. Chokes on malformed HTML. |
+| **`lxml`**<br>Python wrapper of C-based libxml2 | 🔴 <1% | 🚀 Very Fast | 🟡 XPath | ✅ `etree` / E-factory | ❌ Needs sanitization | Fast but not HTML5 compliant. Context-fragment cases are skipped; supported cases still perform poorly. Don't use the old lxml.html.clean module! |
+
+[1]: Parser compliance scores are from a strict run of the [html5lib-tests](https://github.com/html5lib/html5lib-tests) tree-construction fixtures (1,743 non-script tests). The score is `pass / (pass + fail + error)`; unsupported public API capabilities count as failures rather than being faked. The benchmark may compose multiple public APIs from the same parser, but does not use testcase-specific shims or synthetic adapters when an API surface is missing. See [Correctness Testing](correctness.md) for details.
+
+[2]: Browser numbers are from a local rerun of [`justhtml-html5lib-tests-bench`](https://github.com/EmilStenstrom/justhtml-html5lib-tests-bench) against this repo's `tests/html5lib-tests-tree/*.dat` corpus: Chromium 1762/1770, WebKit 1742/1770, Firefox 1728/1770, with 12 skipped scripting-enabled cases per engine.
+
+## Why JustHTML
+
+Most Python HTML projects start simple and then accumulate extra tools:
+
+- a parser for broken HTML
+- a sanitizer for user input
+- a selector engine
+- a serializer
+- linkification or cleanup filters
+- text or Markdown extraction
+
+JustHTML keeps those operations on one DOM. That makes the behavior easier to reason about, especially when the input is untrusted.
+
+```python
+from justhtml import JustHTML
+
+doc = JustHTML("<p>Hello<script>alert(1)</script><a href='javascript:x'>link</a></p>", fragment=True)
+
+print(doc.to_html(pretty=False))
+# <p>Hello<a>link</a></p>
+```
+
+Sanitization happens before you query or serialize unless you explicitly disable it with `sanitize=False`.
+
+## When to Choose Another Tool
+
+Choose **selectolax** when raw speed is the main requirement and the HTML is trusted or sanitized elsewhere.
+
+Choose **markupever** or **html5_parser** when you specifically want their underlying parser engines or tree APIs and can accept their compatibility tradeoffs.
+
+Choose **BeautifulSoup** when you want its forgiving, familiar scraping API and parser correctness is not the main risk.
+
+Choose **lxml** when your project is already built around XPath, etree, or XML-style processing.
+
+Choose **nh3** when you only need fast sanitization and are happy with a Rust-backed dependency.
+
+Choose **html.parser** when you need a tiny stdlib-only script for trusted input and HTML5 correctness does not matter.
+
+Choose **Bleach** only for existing codebases that already depend on it. For new projects, prefer an actively maintained sanitizer path. See [Migrating from Bleach](bleach-migration.md).
+
+## Tradeoffs
+
+JustHTML is pure Python. That makes it easy to install, inspect, debug, and run in environments like Pyodide, but it will not beat C or Rust parsers on raw throughput.
+
+JustHTML sanitizes HTML output by default. That is the right default for user-generated content, CMS snippets, comments, scraped fragments, and transform pipelines that eventually return to a browser. If all of your input is trusted, pass `sanitize=False`.
+
+JustHTML's sanitizer emits HTML-only output. SVG and MathML can still be parsed when sanitization is disabled, but sanitized output drops foreign-namespace content to keep the security model smaller and more reviewable.
+
+## Related Pages
+
+- [Correctness Testing](correctness.md)
+- [Sanitization & Security](sanitization.md)
+- [Migrating from Bleach](bleach-migration.md)
+- [Performance Benchmark](../benchmarks/performance.py)

{justhtml-2.0.0 → justhtml-2.2.0}/docs/index.md

@@ -21,6 +21,7 @@ A pure Python HTML5 parser that just works.
 ## Contents
 
 - **[Quickstart](quickstart.md)** - Get up and running in 2 minutes
+- **[Comparison](comparison.md)** - How JustHTML compares with other Python HTML tools
 - **[Learn by examples](migration-examples.md)** - Real-world StackOverflow tasks rewritten with JustHTML
 - **[API Reference](api.md)** - Complete public API documentation
 - **[Command Line](cli.md)** - Use `justhtml` to extract HTML, text, or Markdown

{justhtml-2.0.0 → justhtml-2.2.0}/pyproject.toml

@@ -1,7 +1,7 @@
 [project]
 name = "justhtml"
 authors = [{ name = "Emil Stenström", email = "emil@emilstenstrom.se" }]
-version = "2.0.0"
+version = "2.2.0"
 description = "A pure Python HTML5 parser that just works."
 readme = "README.md"
 license = { file = "LICENSE" }

{justhtml-2.0.0 → justhtml-2.2.0}/scripts/release.py

@@ -2,18 +2,18 @@
 """Interactive release helper for JustHTML.
 
 What it does (in order):
-1) Bumps version in pyproject.toml ([project].version)
-2) Commits the change
-3) Creates an annotated git tag
-4) Pushes commit + tag
-5) Creates a GitHub release (marked as latest) via `gh`
-
-This script is intentionally minimal and uses only `git` and the GitHub CLI (`gh`).
+1) Verifies the repo-wide pre-commit checks CI runs for releases
+2) Bumps version in pyproject.toml ([project].version)
+3) Commits the change
+4) Creates an annotated git tag
+5) Pushes commit + tag
+6) Creates a GitHub release (marked as latest) via `gh`
 """
 
 from __future__ import annotations
 
 import argparse
+import os
 import re
 import shlex
 import subprocess
@@ -55,6 +55,29 @@ def _run(cmd: list[str], *, check: bool = True) -> CmdResult:
     return out
 
 
+def _run_with_env(cmd: list[str], *, env: dict[str, str], check: bool = True) -> CmdResult:
+    merged_env = os.environ.copy()
+    merged_env.update(env)
+    p = subprocess.run(  # noqa: S603
+        cmd,
+        check=False,
+        text=True,
+        stdout=subprocess.PIPE,
+        stderr=subprocess.STDOUT,
+        env=merged_env,
+    )
+    out = CmdResult(stdout=p.stdout, returncode=p.returncode)
+    if check and p.returncode != 0:
+        raise RuntimeError(
+            "Command failed (exit {code}): {cmd}\n{out}".format(
+                code=p.returncode,
+                cmd=_quote_cmd(cmd),
+                out=(p.stdout or "").rstrip(),
+            )
+        )
+    return out
+
+
 def _run_quiet_ok(cmd: list[str]) -> bool:
     p = subprocess.run(  # noqa: S603
         cmd,
@@ -248,6 +271,13 @@ def _default_repo_from_remote(remote: str) -> str:
     return f"{m.group('owner')}/{m.group('repo')}"
 
 
+def _run_release_checks() -> None:
+    print("Running release checks: SKIP=mypy pre-commit run --all-files")
+    out = _run_with_env(["pre-commit", "run", "--all-files"], env={"SKIP": "mypy"}).stdout
+    if out.strip():
+        print(out.rstrip())
+
+
 def main(argv: list[str] | None = None) -> int:
     parser = argparse.ArgumentParser(description="Bump version, tag, and create a GitHub release.")
     parser.add_argument("--version", help="New version, e.g. 0.21.0 (will be tagged as v0.21.0 unless --tag is set)")
@@ -294,12 +324,20 @@ def main(argv: list[str] | None = None) -> int:
         action="store_true",
         help="Do not prompt for confirmation before push/release.",
     )
+    parser.add_argument(
+        "--skip-checks",
+        action="store_true",
+        help="Skip repo-wide pre-commit validation. Use only when you have already validated the exact release commit.",
+    )
 
     args = parser.parse_args(argv)
 
     try:
         _require_clean_git()
 
+        if not args.skip_checks:
+            _run_release_checks()
+
         py_text = PYPROJECT_PATH.read_text(encoding="utf-8")
         current_version = _read_current_version(py_text)
 

{justhtml-2.0.0 → justhtml-2.2.0}/src/justhtml/parser/__init__.py

@@ -163,14 +163,17 @@ class JustHTML:
         if needs_escape_incomplete_tags:
             opts.emit_bogus_markup_as_text = True
 
-        # For RAWTEXT fragment contexts, set initial tokenizer state and rawtext tag
-        if fragment_context and not fragment_context.namespace:
-            rawtext_elements = {"textarea", "title", "style"}
+        # For text-like HTML fragment contexts, set the initial tokenizer state
+        # to match the context element.
+        if fragment_context and fragment_context.namespace in {None, "html"}:
             tag_name = fragment_context.tag_name.lower()
-            if tag_name in rawtext_elements:
+            if tag_name in {"textarea", "title"}:
+                opts.initial_state = Tokenizer.RCDATA
+            elif tag_name in {"iframe", "noembed", "noframes", "script", "style", "xmp"} or (
+                tag_name == "noscript" and opts.scripting_enabled
+            ):
                 opts.initial_state = Tokenizer.RAWTEXT
-                opts.initial_rawtext_tag = tag_name
-            elif tag_name in ("plaintext", "script"):
+            elif tag_name == "plaintext":
                 opts.initial_state = Tokenizer.PLAINTEXT
 
         self.tokenizer = Tokenizer(