smol-html 0.1.5__py3-none-any.whl → 0.1.7__py3-none-any.whl

{smol_html-0.1.5.dist-info → smol_html-0.1.7.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: smol-html
-Version: 0.1.5
+Version: 0.1.7
 Summary: Small, dependable HTML cleaner/minifier with sensible defaults
 Project-URL: Homepage, https://github.com/NosibleAI/smol-html
 Project-URL: Repository, https://github.com/NosibleAI/smol-html
@@ -35,6 +35,21 @@ Description-Content-Type: text/markdown
 
 Small, dependable HTML cleaner/minifier with sensible defaults.
 
+---
+
+## Credits
+
+This package is built on top of excellent open-source projects:
+
+- [**minify-html**](https://github.com/wilsonzlin/minify-html) (MIT License) – High-performance HTML minifier.
+- [**lxml**](https://lxml.de/) (BSD-like License) – Fast and feature-rich XML/HTML parsing and cleaning.
+- [**BeautifulSoup4**](https://www.crummy.com/software/BeautifulSoup/) (MIT License) – Python library for parsing and navigating HTML/XML.
+- [**Brotli**](https://github.com/google/brotli) (MIT License) – Compression algorithm developed by Google.
+
+We are grateful to the maintainers and contributors of these projects — `smol-html` wouldn’t be possible without their work.
+
+---
+
 ## Motivation
 
 Nosible is a search engine, which means we need to store and process a very large number of webpages. To make this tractable, we strip out visual chrome and other non-essential components that don’t matter for downstream tasks (indexing, ranking, retrieval, and LLM pipelines) while preserving the important content and structure. This package cleans and minifies HTML, greatly reducing size on disk; combined with Brotli compression (by Google), the savings are even larger.
@@ -125,11 +140,17 @@ out = cleaner.make_smol(raw_html="<p>Hi</p>")
 
 ## Compressed Bytes Output
 
-Produce compressed bytes using Brotli with `make_smol_bytes`
+Produce compressed bytes using Brotli with `make_smol_bytes`.
+
+- By default, the compressed bytes are URL-safe Base64 encoded (`base64_encode=True`).
+- If you enable Base64, you must decode before Brotli-decompressing.
+- You can disable Base64 by passing `base64_encode=False` and decompress directly.
 
+Default (Base64-encoded) output:
 
 ```python
 from smol_html import SmolHtmlCleaner
+import base64
 import brotli  # only needed if you want to decompress here in the example
 
 html = """
@@ -145,15 +166,31 @@ cleaner = SmolHtmlCleaner()
 # Get compressed bytes (quality 11 is strong compression)
 compressed = cleaner.make_smol_bytes(raw_html=html, compression_level=11)
 
-# Example: decompress back to text to inspect (optional)
-decompressed = brotli.decompress(compressed).decode("utf-8")
+# Because Base64 is enabled by default, decode before decompressing
+decoded = base64.urlsafe_b64decode(compressed)
+decompressed = brotli.decompress(decoded).decode("utf-8")
 print(decompressed)
 
-# Or write compressed output directly to a file
-with open("page.html.br", "wb") as f:
+# Or write Base64-encoded compressed output directly to a file
+with open("page.html.br.b64", "wb") as f:
     f.write(compressed)
 ```
 
+Disable Base64 and decompress directly:
+
+```python
+from smol_html import SmolHtmlCleaner
+import brotli
+
+cleaner = SmolHtmlCleaner()
+compressed_raw = cleaner.make_smol_bytes(
+    raw_html="<p>Hi</p>",
+    compression_level=11,
+    base64_encode=False,
+)
+print(brotli.decompress(compressed_raw).decode("utf-8"))
+```
+
 ## Parameter Reference
 
 To improve readability, the reference is split into two tables:
@@ -213,3 +250,10 @@ To improve readability, the reference is split into two tables:
 | `remove_unknown_tags` | `bool` | `True` |
 | `safe_attrs_only` | `bool` | `True` |
 | `safe_attrs` | `set[str]` | curated set |
+
+### `make_smol_bytes` Options
+
+| Parameter | Type | Default |
+|---|---|---|
+| `compression_level` | `int` | `4` |
+| `base64_encode` | `bool` | `True` |

smol_html-0.1.7.dist-info/RECORD

@@ -0,0 +1,4 @@
+smol_html-0.1.7.dist-info/METADATA,sha256=z62fJR48fIqXyaCKEFyX6_N5D9GOFf2G8JQukboEXFg,10058
+smol_html-0.1.7.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+smol_html-0.1.7.dist-info/licenses/LICENSE,sha256=88yg3BujRGq8MYlWhbrzB2YMNWJaXnBck3c7l23labs,1089
+smol_html-0.1.7.dist-info/RECORD,,

smol_html-0.1.5.dist-info/RECORD

@@ -1,4 +0,0 @@
-smol_html-0.1.5.dist-info/METADATA,sha256=jriTNIRVbdSkr7EXyEa1ssdm_rZmwo4IV_FLVEJTJrE,8539
-smol_html-0.1.5.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
-smol_html-0.1.5.dist-info/licenses/LICENSE,sha256=88yg3BujRGq8MYlWhbrzB2YMNWJaXnBck3c7l23labs,1089
-smol_html-0.1.5.dist-info/RECORD,,