pyhtml2text 0.3.1__tar.gz → 0.4.1__tar.gz
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/.gitignore +2 -1
- pyhtml2text-0.4.1/CHANGELOG.md +87 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/Cargo.lock +45 -126
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/Cargo.toml +9 -8
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/LICENSE +0 -1
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/PKG-INFO +27 -16
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/README.md +15 -13
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/pyproject.toml +13 -4
- pyhtml2text-0.4.1/src/dom.rs +290 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/src/lib.rs +19 -29
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/src/markdown.rs +9 -8
- pyhtml2text-0.4.1/src/table.rs +1215 -0
- pyhtml2text-0.4.1/src/text.rs +1378 -0
- pyhtml2text-0.4.1/src/url.rs +116 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/tests/test_basic.py +30 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/README.md +19 -17
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/Cargo.toml +5 -10
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/encoding.rs +51 -1
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/serialize/mod.rs +43 -9
- pyhtml2text-0.4.1/vendor/html5ever/src/tokenizer/mod.rs +3292 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tree_builder/mod.rs +13 -21
- pyhtml2text-0.3.1/CHANGELOG.md +0 -42
- pyhtml2text-0.3.1/vendor/html5ever/src/tokenizer/mod.rs +0 -2324
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/python/pyhtml2text/__init__.py +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/python/pyhtml2text/__init__.pyi +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/python/pyhtml2text/py.typed +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/tests/markdown_check.py +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/tests/test_markdown.py +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/LICENSE-APACHE +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/LICENSE-MIT +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/README.md +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/driver.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/lib.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/macros.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tokenizer/char_ref/mod.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tokenizer/interface.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tokenizer/states.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tree_builder/data.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tree_builder/open_elems.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tree_builder/rules.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tree_builder/tag_sets.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/tree_builder/types.rs +0 -0
- {pyhtml2text-0.3.1 → pyhtml2text-0.4.1}/vendor/html5ever/src/util/str.rs +0 -0
|
@@ -0,0 +1,87 @@
|
|
|
1
|
+
# Changelog
|
|
2
|
+
|
|
3
|
+
## 0.4.1 (2026-10-30)
|
|
4
|
+
|
|
5
|
+
- Deeply nested lists, definitions, paragraphs and emphasis are rendered in
|
|
6
|
+
linear time again. In 0.4.0 each level cost time for every level around
|
|
7
|
+
it: 400,000 nested lists took 49 s, and now take 0.5 s. Output is
|
|
8
|
+
unchanged.
|
|
9
|
+
- Wheels for free-threaded Python 3.15 (3.15t), alongside 3.14t. The abi3
|
|
10
|
+
wheels already cover Python 3.15. CI now tests on 3.10, 3.14t, 3.15 and
|
|
11
|
+
3.15t.
|
|
12
|
+
- Character widths from Unicode 18 (unicode-width 0.2.3). The halfwidth
|
|
13
|
+
katakana sound marks ゙ and ゚ count one column, as terminals show them,
|
|
14
|
+
rather than none, and characters new in Unicode 18 (emoji, kana,
|
|
15
|
+
ideographic characters and combining marks) get their own widths. Other
|
|
16
|
+
text wraps as before.
|
|
17
|
+
- html5ever 0.40.1, still with
|
|
18
|
+
[servo/html5ever#800](https://github.com/servo/html5ever/pull/800).
|
|
19
|
+
Output is unchanged. Building from source needs Rust 1.85 or later.
|
|
20
|
+
|
|
21
|
+
## 0.4.0 (2026-10-22)
|
|
22
|
+
|
|
23
|
+
- `html2text()` is rendered by this package's own code. The library it used
|
|
24
|
+
before came from a fork that can no longer be fetched, so the source
|
|
25
|
+
distribution and fresh builds of 0.3.1 fail. The new renderer is a
|
|
26
|
+
clean-room implementation: it was written from 0.3.1's output, the HTML
|
|
27
|
+
standard and RFC 3986, without reference to the old library's source.
|
|
28
|
+
- Output is the same as 0.3.1's, byte for byte, with every option set: on
|
|
29
|
+
the benchmark corpus, on 38 of 39 other real pages (the last differs at
|
|
30
|
+
width 40 only), and on 1,500 random documents without tables. With tables,
|
|
31
|
+
99.2% of 2,000 random documents match under each option set. The known
|
|
32
|
+
differences are all in unusual markup:
|
|
33
|
+
- A cell with nothing in it, beside a cell spanning rows: 0.3.1 draws the
|
|
34
|
+
span against the wrong cell, so borders and line breaks land a column
|
|
35
|
+
off. They now follow the spanning cell.
|
|
36
|
+
- A table with nothing to show whose cells span into further empty rows,
|
|
37
|
+
an empty `<pre>` or heading directly inside `<pre>`, and a table with
|
|
38
|
+
nothing to show in a paragraph: 0.3.1 writes an extra blank line.
|
|
39
|
+
- Headings, quotes and list items squeezed to fewer than three columns
|
|
40
|
+
break `**` and other markers in different places.
|
|
41
|
+
- Some emoji sequences (joined with U+200D) and Arabic lam-alef pairs, cut
|
|
42
|
+
at the end of a line in a narrow table cell.
|
|
43
|
+
- `html2text()` is 1.2–2× faster on real pages (7.0 ms against 10.8 ms for
|
|
44
|
+
a 780 KB Wikipedia article) and 2× faster on short snippets.
|
|
45
|
+
- `html2markdown()` output is unchanged.
|
|
46
|
+
- PyO3 0.30. Building from source needs maturin 1.9.4 or later.
|
|
47
|
+
|
|
48
|
+
## 0.3.1 (2026-10-19)
|
|
49
|
+
|
|
50
|
+
- The vendored html5ever now carries
|
|
51
|
+
[servo/html5ever#800](https://github.com/servo/html5ever/pull/800) by Josef
|
|
52
|
+
Guenther in place of our narrower patch. Every "in scope" check and the
|
|
53
|
+
other searches of the stack of open elements are now O(1), so unmatched
|
|
54
|
+
end tags and end tags inside SVG or MathML no longer make deeply nested
|
|
55
|
+
HTML quadratic: 32,000 nested `<span>` followed by as many unmatched
|
|
56
|
+
`</em>` went from 3.4 s to 0.02 s. Output is unchanged.
|
|
57
|
+
- About 0.4 µs more per call, and 0–2% on real pages.
|
|
58
|
+
|
|
59
|
+
## 0.3.0 (2026-10-15)
|
|
60
|
+
|
|
61
|
+
- New `html2markdown(html, *, base_url=None)`: HTML to CommonMark with
|
|
62
|
+
GitHub's tables and strikethrough. Links and images keep their URLs (made
|
|
63
|
+
absolute with `base_url`), tables become pipe tables, code blocks are
|
|
64
|
+
fenced with their language, and characters are escaped only where Markdown
|
|
65
|
+
would misread them. 3–13× faster than markdownify, html2text and
|
|
66
|
+
html-to-markdown on real pages.
|
|
67
|
+
- `html2text()` output is unchanged.
|
|
68
|
+
|
|
69
|
+
## 0.2.0 (2026-10-15)
|
|
70
|
+
|
|
71
|
+
- 1.5–1.7× faster than 0.1.0 on real web pages, and 2× faster on short
|
|
72
|
+
snippets. Output is unchanged.
|
|
73
|
+
- Wheels for Linux x86_64/aarch64, macOS and Windows x64 are built with
|
|
74
|
+
profile-guided optimisation.
|
|
75
|
+
- Uses mimalloc as the allocator (except on glibc armv7, ppc64le and s390x).
|
|
76
|
+
- Style rules are shared rather than rebuilt and copied for every call.
|
|
77
|
+
- Deeply nested HTML no longer takes quadratic time to parse: 40,000 nested
|
|
78
|
+
`<ul>` lists went from 6s to 0.14s (patched html5ever, see `vendor/`).
|
|
79
|
+
- `html` can be UTF-8 `bytes` as well as `str`.
|
|
80
|
+
- An internal error now raises `ValueError` rather than `PanicException`
|
|
81
|
+
(which `except Exception` doesn't catch).
|
|
82
|
+
- Supports free-threaded Python without re-enabling the GIL.
|
|
83
|
+
- Requires Python 3.10 or later (3.9 is end-of-life).
|
|
84
|
+
|
|
85
|
+
## 0.1.0 (2026-10-15)
|
|
86
|
+
|
|
87
|
+
- First release: `html2text()` with `base_url` support.
|
|
@@ -2,12 +2,6 @@
|
|
|
2
2
|
# It is not intended for manual editing.
|
|
3
3
|
version = 4
|
|
4
4
|
|
|
5
|
-
[[package]]
|
|
6
|
-
name = "autocfg"
|
|
7
|
-
version = "1.5.1"
|
|
8
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
9
|
-
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
|
|
10
|
-
|
|
11
5
|
[[package]]
|
|
12
6
|
name = "bitflags"
|
|
13
7
|
version = "2.13.2"
|
|
@@ -16,9 +10,9 @@ checksum = "3ded4057c258ba199e2d26386d3af3780957ecaee6c4ef4041c6b4b8b97c0b06"
|
|
|
16
10
|
|
|
17
11
|
[[package]]
|
|
18
12
|
name = "cc"
|
|
19
|
-
version = "1.
|
|
13
|
+
version = "1.10.0"
|
|
20
14
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
21
|
-
checksum = "
|
|
15
|
+
checksum = "86e055fa6e3e40ad6187ba25b6855eeb635d5271f8e0f0e1e9a5826b8d75bea0"
|
|
22
16
|
dependencies = [
|
|
23
17
|
"find-msvc-tools",
|
|
24
18
|
"shlex",
|
|
@@ -38,9 +32,9 @@ checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223"
|
|
|
38
32
|
|
|
39
33
|
[[package]]
|
|
40
34
|
name = "find-msvc-tools"
|
|
41
|
-
version = "0.1.
|
|
35
|
+
version = "0.1.15"
|
|
42
36
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
43
|
-
checksum = "
|
|
37
|
+
checksum = "1c66915cc5391097de7a2b0b972e8505aa33c0df401fd3a6811dbdb32881ba48"
|
|
44
38
|
|
|
45
39
|
[[package]]
|
|
46
40
|
name = "heck"
|
|
@@ -48,39 +42,20 @@ version = "0.5.0"
|
|
|
48
42
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
49
43
|
checksum = "2304e00983f87ffb38b55b444b5e3b60a884b5d30c0fca7d82fe33449bbe55ea"
|
|
50
44
|
|
|
51
|
-
[[package]]
|
|
52
|
-
name = "html2text"
|
|
53
|
-
version = "0.17.1"
|
|
54
|
-
source = "git+https://github.com/jaredlynskey/rust-html2text?rev=b42930f8e6688ea121daf392e27a7d9812e6192d#b42930f8e6688ea121daf392e27a7d9812e6192d"
|
|
55
|
-
dependencies = [
|
|
56
|
-
"html5ever",
|
|
57
|
-
"tendril",
|
|
58
|
-
"thiserror",
|
|
59
|
-
"unicode-width",
|
|
60
|
-
]
|
|
61
|
-
|
|
62
45
|
[[package]]
|
|
63
46
|
name = "html5ever"
|
|
64
|
-
version = "0.
|
|
47
|
+
version = "0.40.1"
|
|
65
48
|
dependencies = [
|
|
66
49
|
"log",
|
|
67
50
|
"markup5ever",
|
|
68
|
-
|
|
69
|
-
|
|
70
|
-
[[package]]
|
|
71
|
-
name = "indoc"
|
|
72
|
-
version = "2.0.8"
|
|
73
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
74
|
-
checksum = "a37b2691796cffeb8a8cd305ac66e65841559f147f4e63231d0eafa4db5384d1"
|
|
75
|
-
dependencies = [
|
|
76
|
-
"rustversion",
|
|
51
|
+
"memchr",
|
|
77
52
|
]
|
|
78
53
|
|
|
79
54
|
[[package]]
|
|
80
55
|
name = "libc"
|
|
81
|
-
version = "0.2.
|
|
56
|
+
version = "0.2.191"
|
|
82
57
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
83
|
-
checksum = "
|
|
58
|
+
checksum = "65cd87c9a6d60536580b316ad985726104730a5151c99845f2490428ef91240a"
|
|
84
59
|
|
|
85
60
|
[[package]]
|
|
86
61
|
name = "libmimalloc-sys"
|
|
@@ -108,9 +83,9 @@ checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
|
|
|
108
83
|
|
|
109
84
|
[[package]]
|
|
110
85
|
name = "markup5ever"
|
|
111
|
-
version = "0.
|
|
86
|
+
version = "0.40.0"
|
|
112
87
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
113
|
-
checksum = "
|
|
88
|
+
checksum = "0ab3dc68ac4a0f5719e560136778c1ee716e296030d75dbd4484e37e39e3a842"
|
|
114
89
|
dependencies = [
|
|
115
90
|
"log",
|
|
116
91
|
"tendril",
|
|
@@ -123,15 +98,6 @@ version = "2.8.3"
|
|
|
123
98
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
124
99
|
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
|
|
125
100
|
|
|
126
|
-
[[package]]
|
|
127
|
-
name = "memoffset"
|
|
128
|
-
version = "0.9.1"
|
|
129
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
130
|
-
checksum = "488016bfae457b036d996092f6cb448677611ce4449e970ceaf42695203f218a"
|
|
131
|
-
dependencies = [
|
|
132
|
-
"autocfg",
|
|
133
|
-
]
|
|
134
|
-
|
|
135
101
|
[[package]]
|
|
136
102
|
name = "mimalloc"
|
|
137
103
|
version = "0.1.52"
|
|
@@ -178,9 +144,9 @@ dependencies = [
|
|
|
178
144
|
|
|
179
145
|
[[package]]
|
|
180
146
|
name = "phf"
|
|
181
|
-
version = "0.
|
|
147
|
+
version = "0.14.0"
|
|
182
148
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
183
|
-
checksum = "
|
|
149
|
+
checksum = "010378780309880b08997fae13be7834dba947d36393bd372f2b1556deb2a2f6"
|
|
184
150
|
dependencies = [
|
|
185
151
|
"phf_shared",
|
|
186
152
|
"serde",
|
|
@@ -188,9 +154,9 @@ dependencies = [
|
|
|
188
154
|
|
|
189
155
|
[[package]]
|
|
190
156
|
name = "phf_codegen"
|
|
191
|
-
version = "0.
|
|
157
|
+
version = "0.14.0"
|
|
192
158
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
193
|
-
checksum = "
|
|
159
|
+
checksum = "41b585a510fb76fdebead6897982ef2a03a21d8e6cbcca904999742a4afc6ffe"
|
|
194
160
|
dependencies = [
|
|
195
161
|
"phf_generator",
|
|
196
162
|
"phf_shared",
|
|
@@ -198,9 +164,9 @@ dependencies = [
|
|
|
198
164
|
|
|
199
165
|
[[package]]
|
|
200
166
|
name = "phf_generator"
|
|
201
|
-
version = "0.
|
|
167
|
+
version = "0.14.0"
|
|
202
168
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
203
|
-
checksum = "
|
|
169
|
+
checksum = "aeb62e0959d5a1bebc965f4d15d9e2b7cea002b6b0f5ba8cde6cc26738467100"
|
|
204
170
|
dependencies = [
|
|
205
171
|
"fastrand",
|
|
206
172
|
"phf_shared",
|
|
@@ -208,9 +174,9 @@ dependencies = [
|
|
|
208
174
|
|
|
209
175
|
[[package]]
|
|
210
176
|
name = "phf_shared"
|
|
211
|
-
version = "0.
|
|
177
|
+
version = "0.14.0"
|
|
212
178
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
213
|
-
checksum = "
|
|
179
|
+
checksum = "c6fd9027e2d9319be6349febd1db4e8d02aa544921200c9b777720ac34a3aa89"
|
|
214
180
|
dependencies = [
|
|
215
181
|
"siphasher",
|
|
216
182
|
]
|
|
@@ -238,9 +204,9 @@ dependencies = [
|
|
|
238
204
|
|
|
239
205
|
[[package]]
|
|
240
206
|
name = "pulldown-cmark"
|
|
241
|
-
version = "0.13.
|
|
207
|
+
version = "0.13.5"
|
|
242
208
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
243
|
-
checksum = "
|
|
209
|
+
checksum = "278e9268c93b9e498a2da46c34d90ba93a3255eb99009646398f34bc73ab36eb"
|
|
244
210
|
dependencies = [
|
|
245
211
|
"bitflags",
|
|
246
212
|
"memchr",
|
|
@@ -249,46 +215,43 @@ dependencies = [
|
|
|
249
215
|
|
|
250
216
|
[[package]]
|
|
251
217
|
name = "pyhtml2text"
|
|
252
|
-
version = "0.
|
|
218
|
+
version = "0.4.1"
|
|
253
219
|
dependencies = [
|
|
254
|
-
"html2text",
|
|
255
220
|
"html5ever",
|
|
256
221
|
"mimalloc",
|
|
257
222
|
"pulldown-cmark",
|
|
258
223
|
"pyo3",
|
|
224
|
+
"unicode-width",
|
|
259
225
|
]
|
|
260
226
|
|
|
261
227
|
[[package]]
|
|
262
228
|
name = "pyo3"
|
|
263
|
-
version = "0.
|
|
229
|
+
version = "0.30.1"
|
|
264
230
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
265
|
-
checksum = "
|
|
231
|
+
checksum = "229ca13b8a15bb0d1487b81ab975bfcd97513416499a8312011b6036e42e0de4"
|
|
266
232
|
dependencies = [
|
|
267
|
-
"indoc",
|
|
268
233
|
"libc",
|
|
269
|
-
"memoffset",
|
|
270
234
|
"once_cell",
|
|
271
235
|
"portable-atomic",
|
|
272
236
|
"pyo3-build-config",
|
|
273
237
|
"pyo3-ffi",
|
|
274
238
|
"pyo3-macros",
|
|
275
|
-
"unindent",
|
|
276
239
|
]
|
|
277
240
|
|
|
278
241
|
[[package]]
|
|
279
242
|
name = "pyo3-build-config"
|
|
280
|
-
version = "0.
|
|
243
|
+
version = "0.30.1"
|
|
281
244
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
282
|
-
checksum = "
|
|
245
|
+
checksum = "70130e2b6dd1ee589a8788026a429561ca5c40fa7a2fa7914c2282eddae24700"
|
|
283
246
|
dependencies = [
|
|
284
247
|
"target-lexicon",
|
|
285
248
|
]
|
|
286
249
|
|
|
287
250
|
[[package]]
|
|
288
251
|
name = "pyo3-ffi"
|
|
289
|
-
version = "0.
|
|
252
|
+
version = "0.30.1"
|
|
290
253
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
291
|
-
checksum = "
|
|
254
|
+
checksum = "5e7ff73c555c286d212d4ce7e5fbda845a973c4f61bd5f160f5f2c3c2a04d1c4"
|
|
292
255
|
dependencies = [
|
|
293
256
|
"libc",
|
|
294
257
|
"pyo3-build-config",
|
|
@@ -296,27 +259,26 @@ dependencies = [
|
|
|
296
259
|
|
|
297
260
|
[[package]]
|
|
298
261
|
name = "pyo3-macros"
|
|
299
|
-
version = "0.
|
|
262
|
+
version = "0.30.1"
|
|
300
263
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
301
|
-
checksum = "
|
|
264
|
+
checksum = "c46d2e5862c7bb8207e3b639f713dda3c499933444784154c11d013c5a691b60"
|
|
302
265
|
dependencies = [
|
|
303
266
|
"proc-macro2",
|
|
304
267
|
"pyo3-macros-backend",
|
|
305
268
|
"quote",
|
|
306
|
-
"syn
|
|
269
|
+
"syn",
|
|
307
270
|
]
|
|
308
271
|
|
|
309
272
|
[[package]]
|
|
310
273
|
name = "pyo3-macros-backend"
|
|
311
|
-
version = "0.
|
|
274
|
+
version = "0.30.1"
|
|
312
275
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
313
|
-
checksum = "
|
|
276
|
+
checksum = "38b89f066f2ae72a2aef52d1537ca2fcd104440ebfadee06e97aa90738a25b9b"
|
|
314
277
|
dependencies = [
|
|
315
278
|
"heck",
|
|
316
279
|
"proc-macro2",
|
|
317
|
-
"pyo3-build-config",
|
|
318
280
|
"quote",
|
|
319
|
-
"syn
|
|
281
|
+
"syn",
|
|
320
282
|
]
|
|
321
283
|
|
|
322
284
|
[[package]]
|
|
@@ -337,12 +299,6 @@ dependencies = [
|
|
|
337
299
|
"bitflags",
|
|
338
300
|
]
|
|
339
301
|
|
|
340
|
-
[[package]]
|
|
341
|
-
name = "rustversion"
|
|
342
|
-
version = "1.0.23"
|
|
343
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
344
|
-
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
|
|
345
|
-
|
|
346
302
|
[[package]]
|
|
347
303
|
name = "scopeguard"
|
|
348
304
|
version = "1.2.0"
|
|
@@ -375,7 +331,7 @@ checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
|
|
|
375
331
|
dependencies = [
|
|
376
332
|
"proc-macro2",
|
|
377
333
|
"quote",
|
|
378
|
-
"syn
|
|
334
|
+
"syn",
|
|
379
335
|
]
|
|
380
336
|
|
|
381
337
|
[[package]]
|
|
@@ -392,15 +348,15 @@ checksum = "33f4fe9184a62d842c9ef383018f3306d8ba224fd9d836f56d7288308847c256"
|
|
|
392
348
|
|
|
393
349
|
[[package]]
|
|
394
350
|
name = "smallvec"
|
|
395
|
-
version = "1.16.
|
|
351
|
+
version = "1.16.4"
|
|
396
352
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
397
|
-
checksum = "
|
|
353
|
+
checksum = "d0d5d4b1981b8bbbb7aed204cae5726d683d5d6ac7e74ec54122ea28f315e7c8"
|
|
398
354
|
|
|
399
355
|
[[package]]
|
|
400
356
|
name = "string_cache"
|
|
401
|
-
version = "0.
|
|
357
|
+
version = "0.11.0"
|
|
402
358
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
403
|
-
checksum = "
|
|
359
|
+
checksum = "ffa8a5dbe8b3f0bbe29d4c3225daafaeead63afdc1b65fc4c01a1384166038e6"
|
|
404
360
|
dependencies = [
|
|
405
361
|
"new_debug_unreachable",
|
|
406
362
|
"parking_lot",
|
|
@@ -410,9 +366,9 @@ dependencies = [
|
|
|
410
366
|
|
|
411
367
|
[[package]]
|
|
412
368
|
name = "string_cache_codegen"
|
|
413
|
-
version = "0.
|
|
369
|
+
version = "0.11.2"
|
|
414
370
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
415
|
-
checksum = "
|
|
371
|
+
checksum = "928dcdf75e47626b3617a976ec205d9f057584c371c1f23b782129268d0e6edc"
|
|
416
372
|
dependencies = [
|
|
417
373
|
"phf_generator",
|
|
418
374
|
"phf_shared",
|
|
@@ -420,17 +376,6 @@ dependencies = [
|
|
|
420
376
|
"quote",
|
|
421
377
|
]
|
|
422
378
|
|
|
423
|
-
[[package]]
|
|
424
|
-
name = "syn"
|
|
425
|
-
version = "2.0.119"
|
|
426
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
427
|
-
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
|
|
428
|
-
dependencies = [
|
|
429
|
-
"proc-macro2",
|
|
430
|
-
"quote",
|
|
431
|
-
"unicode-ident",
|
|
432
|
-
]
|
|
433
|
-
|
|
434
379
|
[[package]]
|
|
435
380
|
name = "syn"
|
|
436
381
|
version = "3.0.9"
|
|
@@ -457,26 +402,6 @@ dependencies = [
|
|
|
457
402
|
"new_debug_unreachable",
|
|
458
403
|
]
|
|
459
404
|
|
|
460
|
-
[[package]]
|
|
461
|
-
name = "thiserror"
|
|
462
|
-
version = "2.0.21"
|
|
463
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
464
|
-
checksum = "09e52cb86a36cede5cb101bf8908837b3e4c6e5e59fe7fd85c23fb56200d189e"
|
|
465
|
-
dependencies = [
|
|
466
|
-
"thiserror-impl",
|
|
467
|
-
]
|
|
468
|
-
|
|
469
|
-
[[package]]
|
|
470
|
-
name = "thiserror-impl"
|
|
471
|
-
version = "2.0.21"
|
|
472
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
473
|
-
checksum = "fe5197923287db20a58125f0bc85c062f7f2c892de97b18c356f9efb14b28524"
|
|
474
|
-
dependencies = [
|
|
475
|
-
"proc-macro2",
|
|
476
|
-
"quote",
|
|
477
|
-
"syn 3.0.9",
|
|
478
|
-
]
|
|
479
|
-
|
|
480
405
|
[[package]]
|
|
481
406
|
name = "unicase"
|
|
482
407
|
version = "2.10.0"
|
|
@@ -491,21 +416,15 @@ checksum = "a2c754d6c33795a1c324727428e5a7dedb5b06195f9890bdbcba760d3e246563"
|
|
|
491
416
|
|
|
492
417
|
[[package]]
|
|
493
418
|
name = "unicode-width"
|
|
494
|
-
version = "0.2.
|
|
495
|
-
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
496
|
-
checksum = "b4ac048d71ede7ee76d585517add45da530660ef4390e49b098733c6e897f254"
|
|
497
|
-
|
|
498
|
-
[[package]]
|
|
499
|
-
name = "unindent"
|
|
500
|
-
version = "0.2.4"
|
|
419
|
+
version = "0.2.3"
|
|
501
420
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
502
|
-
checksum = "
|
|
421
|
+
checksum = "ca718471876992059f251e33aafc334346ada08042b7661dd1fe313a9bced557"
|
|
503
422
|
|
|
504
423
|
[[package]]
|
|
505
424
|
name = "web_atoms"
|
|
506
|
-
version = "0.
|
|
425
|
+
version = "0.3.0"
|
|
507
426
|
source = "registry+https://github.com/rust-lang/crates.io-index"
|
|
508
|
-
checksum = "
|
|
427
|
+
checksum = "7572660c8890448ba236b7376f27e389c6a7e1c70195622faced601f855c0ada"
|
|
509
428
|
dependencies = [
|
|
510
429
|
"phf",
|
|
511
430
|
"phf_codegen",
|
|
@@ -1,6 +1,6 @@
|
|
|
1
1
|
[package]
|
|
2
2
|
name = "pyhtml2text"
|
|
3
|
-
version = "0.
|
|
3
|
+
version = "0.4.1"
|
|
4
4
|
edition = "2021"
|
|
5
5
|
license = "MIT"
|
|
6
6
|
publish = false
|
|
@@ -11,16 +11,17 @@ name = "_pyhtml2text"
|
|
|
11
11
|
crate-type = ["cdylib"]
|
|
12
12
|
|
|
13
13
|
[dependencies]
|
|
14
|
-
pyo3 = { version = "0.
|
|
15
|
-
#
|
|
16
|
-
|
|
17
|
-
|
|
18
|
-
#
|
|
19
|
-
|
|
14
|
+
pyo3 = { version = "0.30", features = ["abi3-py310"] }
|
|
15
|
+
# The HTML parser; patched below.
|
|
16
|
+
html5ever = "0.40"
|
|
17
|
+
# Display width of characters, for wrapping. Pinned, since newer Unicode data
|
|
18
|
+
# can change the output.
|
|
19
|
+
unicode-width = "=0.2.3"
|
|
20
20
|
# Reads the Markdown back to check that emphasis parses as intended.
|
|
21
21
|
pulldown-cmark = { version = "0.13", default-features = false }
|
|
22
22
|
|
|
23
|
-
# Much faster than the system allocators for
|
|
23
|
+
# Much faster than the system allocators for the many small allocations of
|
|
24
|
+
# parsing and rendering.
|
|
24
25
|
# Not on glibc armv7/ppc64le/s390x: their manylinux2014 cross compilers are too
|
|
25
26
|
# old to build it (keep in sync with the cfg in src/lib.rs).
|
|
26
27
|
[target.'cfg(not(all(target_env = "gnu", any(target_arch = "arm", target_arch = "powerpc64", target_arch = "s390x"))))'.dependencies]
|
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
MIT License
|
|
2
2
|
|
|
3
3
|
Copyright (c) 2026 Jared Lynskey
|
|
4
|
-
Copyright (c) 2016 Chris Emerson (rust-html2text, which this package embeds)
|
|
5
4
|
|
|
6
5
|
Permission is hereby granted, free of charge, to any person obtaining a copy
|
|
7
6
|
of this software and associated documentation files (the "Software"), to deal
|
|
@@ -1,23 +1,31 @@
|
|
|
1
1
|
Metadata-Version: 2.4
|
|
2
2
|
Name: pyhtml2text
|
|
3
|
-
Version: 0.
|
|
3
|
+
Version: 0.4.1
|
|
4
4
|
Classifier: Programming Language :: Rust
|
|
5
5
|
Classifier: Programming Language :: Python :: 3
|
|
6
|
+
Classifier: Programming Language :: Python :: 3 :: Only
|
|
7
|
+
Classifier: Programming Language :: Python :: 3.10
|
|
8
|
+
Classifier: Programming Language :: Python :: 3.11
|
|
9
|
+
Classifier: Programming Language :: Python :: 3.12
|
|
10
|
+
Classifier: Programming Language :: Python :: 3.13
|
|
11
|
+
Classifier: Programming Language :: Python :: 3.14
|
|
12
|
+
Classifier: Programming Language :: Python :: 3.15
|
|
13
|
+
Classifier: Programming Language :: Python :: Free Threading :: 3 - Stable
|
|
6
14
|
Classifier: Topic :: Text Processing :: Markup :: HTML
|
|
15
|
+
Classifier: Topic :: Text Processing :: Markup :: Markdown
|
|
16
|
+
Classifier: Typing :: Typed
|
|
7
17
|
License-File: LICENSE
|
|
8
|
-
Summary: Fast HTML to plain text, with relative link resolution
|
|
18
|
+
Summary: Fast HTML to plain text and Markdown, with relative link resolution
|
|
9
19
|
Keywords: html,text,markdown,html2text
|
|
10
20
|
Author-email: Jared Lynskey <jared@lynskey.co.nz>
|
|
11
21
|
License-Expression: MIT
|
|
12
22
|
Requires-Python: >=3.10
|
|
13
23
|
Description-Content-Type: text/markdown; charset=UTF-8; variant=GFM
|
|
14
24
|
Project-URL: Source, https://github.com/jaredlynskey/pyhtml2text
|
|
15
|
-
Project-URL: Upstream, https://github.com/jugglerchris/rust-html2text
|
|
16
25
|
|
|
17
26
|
# pyhtml2text
|
|
18
27
|
|
|
19
|
-
Fast HTML to plain text and to Markdown for Python,
|
|
20
|
-
[rust-html2text](https://github.com/jugglerchris/rust-html2text).
|
|
28
|
+
Fast HTML to plain text and to Markdown for Python, written in Rust.
|
|
21
29
|
|
|
22
30
|
Relative links can be resolved against the page's URL, so the output's links
|
|
23
31
|
work on their own.
|
|
@@ -35,6 +43,8 @@ html = """<h2>Before you start</h2>
|
|
|
35
43
|
print(html2markdown(html, base_url="https://example.com/docs/intro.html"))
|
|
36
44
|
```
|
|
37
45
|
|
|
46
|
+
Output:
|
|
47
|
+
|
|
38
48
|
```markdown
|
|
39
49
|
## Before you start
|
|
40
50
|
|
|
@@ -114,28 +124,29 @@ To plain text:
|
|
|
114
124
|
|---|---|---|
|
|
115
125
|
| [html2text](https://pypi.org/project/html2text/) (pure Python) | 71.5 ms | 14.9 ms |
|
|
116
126
|
| [html2text-rs](https://pypi.org/project/html2text-rs/) | 15.2 ms | 2.5 ms |
|
|
117
|
-
| **pyhtml2text** | **
|
|
127
|
+
| **pyhtml2text** | **7.0 ms** | **1.3 ms** |
|
|
118
128
|
|
|
119
129
|
- The GIL is released while converting, so threads scale across cores. On
|
|
120
|
-
free-threaded Python (3.14t) the module runs without re-enabling
|
|
130
|
+
free-threaded Python (3.14t and 3.15t) the module runs without re-enabling
|
|
131
|
+
the GIL.
|
|
121
132
|
- Wheels for Linux (x86_64, aarch64), macOS and Windows x64 are built with
|
|
122
133
|
profile-guided optimisation, which makes them about 25% faster. Other
|
|
123
134
|
platforms get the same code without it.
|
|
124
135
|
- Deeply nested HTML is parsed in linear time, so hostile input can't stall a
|
|
125
|
-
worker: 40,000 nested lists take 0.
|
|
136
|
+
worker: 40,000 nested lists take 0.02 s, against 18 s with stock html5ever,
|
|
126
137
|
and 100,000 levels of nesting followed by unmatched end tags take under
|
|
127
138
|
0.1 s.
|
|
128
139
|
|
|
129
140
|
## Credits
|
|
130
141
|
|
|
131
|
-
|
|
132
|
-
|
|
133
|
-
that adds `base_url` and some performance changes, and a copy of
|
|
134
|
-
[html5ever](https://github.com/servo/html5ever) (MIT/Apache-2.0) with
|
|
142
|
+
HTML is parsed by a copy of [html5ever](https://github.com/servo/html5ever)
|
|
143
|
+
(MIT/Apache-2.0) with
|
|
135
144
|
[servo/html5ever#800](https://github.com/servo/html5ever/pull/800) by Josef
|
|
136
145
|
Guenther applied, which makes parsing deeply nested HTML linear (see
|
|
137
|
-
`vendor/`).
|
|
138
|
-
[
|
|
139
|
-
|
|
140
|
-
|
|
146
|
+
`vendor/`). Text is measured with
|
|
147
|
+
[unicode-width](https://github.com/unicode-rs/unicode-width) (MIT/Apache-2.0)
|
|
148
|
+
and Markdown output is checked with
|
|
149
|
+
[pulldown-cmark](https://github.com/raphlinus/pulldown-cmark) (MIT). This
|
|
150
|
+
package isn't affiliated with or endorsed by any of these projects, so please
|
|
151
|
+
report issues here rather than upstream.
|
|
141
152
|
|
|
@@ -1,7 +1,6 @@
|
|
|
1
1
|
# pyhtml2text
|
|
2
2
|
|
|
3
|
-
Fast HTML to plain text and to Markdown for Python,
|
|
4
|
-
[rust-html2text](https://github.com/jugglerchris/rust-html2text).
|
|
3
|
+
Fast HTML to plain text and to Markdown for Python, written in Rust.
|
|
5
4
|
|
|
6
5
|
Relative links can be resolved against the page's URL, so the output's links
|
|
7
6
|
work on their own.
|
|
@@ -19,6 +18,8 @@ html = """<h2>Before you start</h2>
|
|
|
19
18
|
print(html2markdown(html, base_url="https://example.com/docs/intro.html"))
|
|
20
19
|
```
|
|
21
20
|
|
|
21
|
+
Output:
|
|
22
|
+
|
|
22
23
|
```markdown
|
|
23
24
|
## Before you start
|
|
24
25
|
|
|
@@ -98,27 +99,28 @@ To plain text:
|
|
|
98
99
|
|---|---|---|
|
|
99
100
|
| [html2text](https://pypi.org/project/html2text/) (pure Python) | 71.5 ms | 14.9 ms |
|
|
100
101
|
| [html2text-rs](https://pypi.org/project/html2text-rs/) | 15.2 ms | 2.5 ms |
|
|
101
|
-
| **pyhtml2text** | **
|
|
102
|
+
| **pyhtml2text** | **7.0 ms** | **1.3 ms** |
|
|
102
103
|
|
|
103
104
|
- The GIL is released while converting, so threads scale across cores. On
|
|
104
|
-
free-threaded Python (3.14t) the module runs without re-enabling
|
|
105
|
+
free-threaded Python (3.14t and 3.15t) the module runs without re-enabling
|
|
106
|
+
the GIL.
|
|
105
107
|
- Wheels for Linux (x86_64, aarch64), macOS and Windows x64 are built with
|
|
106
108
|
profile-guided optimisation, which makes them about 25% faster. Other
|
|
107
109
|
platforms get the same code without it.
|
|
108
110
|
- Deeply nested HTML is parsed in linear time, so hostile input can't stall a
|
|
109
|
-
worker: 40,000 nested lists take 0.
|
|
111
|
+
worker: 40,000 nested lists take 0.02 s, against 18 s with stock html5ever,
|
|
110
112
|
and 100,000 levels of nesting followed by unmatched end tags take under
|
|
111
113
|
0.1 s.
|
|
112
114
|
|
|
113
115
|
## Credits
|
|
114
116
|
|
|
115
|
-
|
|
116
|
-
|
|
117
|
-
that adds `base_url` and some performance changes, and a copy of
|
|
118
|
-
[html5ever](https://github.com/servo/html5ever) (MIT/Apache-2.0) with
|
|
117
|
+
HTML is parsed by a copy of [html5ever](https://github.com/servo/html5ever)
|
|
118
|
+
(MIT/Apache-2.0) with
|
|
119
119
|
[servo/html5ever#800](https://github.com/servo/html5ever/pull/800) by Josef
|
|
120
120
|
Guenther applied, which makes parsing deeply nested HTML linear (see
|
|
121
|
-
`vendor/`).
|
|
122
|
-
[
|
|
123
|
-
|
|
124
|
-
|
|
121
|
+
`vendor/`). Text is measured with
|
|
122
|
+
[unicode-width](https://github.com/unicode-rs/unicode-width) (MIT/Apache-2.0)
|
|
123
|
+
and Markdown output is checked with
|
|
124
|
+
[pulldown-cmark](https://github.com/raphlinus/pulldown-cmark) (MIT). This
|
|
125
|
+
package isn't affiliated with or endorsed by any of these projects, so please
|
|
126
|
+
report issues here rather than upstream.
|