selectolax 0.3.33__tar.gz → 0.3.34__tar.gz

selectolax-0.3.34/CHANGES.md

@@ -0,0 +1,330 @@
+# selectolax Changelog
+
+## Version 0.3.34
+
+Released
+
+- Lexbor backend now supports `:lexbor-contains("abc" i)` CSS pseudo-class to match text nodes.
+
+## Version 0.3.33
+
+Released
+
+- Add `merge_text_nodes` to lexbor backend. Fixes #170. @amirshukayev
+- Performance improvements in Cython code. @Vizonex
+
+## Version 0.3.32
+
+Released
+
+- Update lexbor. New version of lexbor fixes bugs with CSS selectors.
+
+## Version 0.3.31
+
+Released
+
+- Improve type hints, add docstrings to type hints
+- Prevent decomposing of the root node
+- Unpin Cython version and make it Optional
+- Allow empty attribute values. Fixes #165.
+
+## Version 0.3.30
+
+Released
+
+- Update lexbor
+- Expose `SelectolaxError` exception in lexbor.pyi
+
+## Version 0.3.29
+
+Released
+
+- Feat: Add unwrap empty tags functionality. Fixes #159.
+
+## Version 0.3.28
+
+Released
+
+- Fix: Update lexbor and improve HTML serialization speed. Fixes #153.
+- Fix: typo in type annotations. Fixes #147.
+- Fix: Fix incorrect type annotations for `LexborHTMLParser.__init__`. Fixes #144.
+
+## Version 0.3.27
+
+Released
+
+- Fix: Header detected as head
+
+## Version 0.3.26
+
+Released
+
+- Improve type hints
+
+## Version 0.3.25
+
+Released
+
+- Feat: Add `parse_fragment()` and `create_tag()`
+- Add missing typing for `Node.insert_child()`
+- Add `Node.parser` to access the `HTMLParser` to which the node belongs
+
+## Version 0.3.24
+
+Released
+
+- Add `Node.insert_child` method to lexbor and modest backends
+
+## Version 0.3.23
+
+Released
+
+- Add Python 3.13 wheels
+- Update lexbor
+
+## Version 0.3.21
+
+Released
+
+- ***Breaking change***: `lexbor` backend now includes the root node when querying CSS selectors. Same as `Modest` backend.
+- Fix `css_matches` and `any_css_matches` methods for `Modest` backend on some compilers
+
+## Version 0.3.20
+
+Released
+
+- Fixup for 0.3.19 release
+- Fix tag order for `lexbor` backend
+
+## Version 0.3.19
+
+Released
+
+- Increase maximum HTML size to 2.4GB
+
+## Version 0.3.18
+
+Released
+
+- Fix memory leak when using CSS selectors, `lexbor` backend
+
+## Version 0.3.17
+
+Released
+
+- Update lexbor
+- Add Python 3.12 wheels
+
+## Version 0.3.16
+
+Released
+
+- Make HTML nodes hashable
+- Pin Cython version
+
+## Version 0.3.15
+
+Released
+
+- Improve typing. Thanks to @nesb1
+
+## Version 0.3.14
+
+Released
+
+- Fix memory leak for `lexbor` backend
+
+## Version 0.3.13
+
+Released
+
+- Update `lexbor`
+
+## Version 0.3.12
+
+Released
+
+- Update `lexbor`
+- Add Python 3.11 wheels
+
+## Version 0.3.11
+
+Released
+
+- Fix out-of-bounds bug for `merge_text_nodes` method.
+
+## Version 0.3.10
+
+Released
+
+This release does not contain any changes.
+Due to a typo in the version number ([#70](https://github.com/rushter/selectolax/issues/70)), we need to make a new release.
+
+## Version 0.3.9
+
+Released
+
+- Remove trailing separator when using `text(deep=True, separator='x')`.
+- Add a new `merge_text_nodes` method for Modest backend.
+
+## Version 0.3.8
+
+Released
+
+- Fix incorrect text handling when using `text(deep=True)` on a text node.
+
+## Version 0.3.7
+
+Released
+
+- Fix return type of HTMLParser.tags
+
+## Version 0.3.6
+
+Released
+
+- Improve text handling
+- Add binary builds for Python 3.10 and ARM on MacOS and Linux
+
+## Version 0.3.5
+
+Released
+
+- Add type annotations
+
+## Version 0.3.4
+
+Released
+
+- Fix `HTMLParser.html`
+
+## Version 0.3.3
+
+Released
+
+- Use `document` for the `HTMLParser.html`, `LexborHTMLParser.html` root properties
+
+## Version 0.3.2
+
+Released
+
+- Fix `selector` method for lexbor
+- Improve text extraction for lexbor
+
+## Version 0.3.1
+
+Released
+
+- Fix `setup.py` for Windows
+
+## Version 0.3.0
+
+Released
+
+- Added `lexbor` backend
+- Fix cloning for `Modest` backend
+
+## Version 0.2.14
+
+Released
+
+- Added advanced Selector (the `select` method)
+- Improved speed of `strip_tags`
+- Added `clone` method for the `HtmlParser` object
+- Exposed `detect_encoding`, `decode_errors`, `use_meta_tags`, `raw_html` attributes for `HtmlParser`
+- Added `sget` method to the `attrs` property
+
+## Version 0.2.13
+
+Released
+
+- Don't throw exception when encoding text as UTF-8 bytes fails ([#40](https://github.com/rushter/selectolax/issues/40)).
+- Fix Node.attrs.items() causes ([#39](https://github.com/rushter/selectolax/issues/39)).
+
+## Version 0.2.12
+
+Released
+
+- Build wheels Apple Silicon
+
+## Version 0.2.11
+
+Released
+
+- Fix strip argument is ignored for the root node ([#35](https://github.com/rushter/selectolax/issues/35)).
+- Fix CSS parser hangs on a bad CSS selector ([#36](https://github.com/rushter/selectolax/issues/36)).
+
+## Version 0.2.10
+
+Released
+
+- Fix root node property ([#32](https://github.com/rushter/selectolax/issues/32)). The `root` property now points to the html tag.
+
+## Version 0.2.9
+
+Released
+
+- Fix README for PyPI
+
+## Version 0.2.8
+
+Released
+
+- Add wheels for Python 3.9
+
+## Version 0.2.7
+
+Released
+
+- Add `raw_value` attribute for `Node` objects ([#22](https://github.com/rushter/selectolax/issues/22))
+- Improve node modification operations
+
+## Version 0.2.6
+
+Released
+
+- Fix dependency on the source `Node` when inserting to or modifying destination `Node`
+
+## Version 0.2.5
+
+Released
+
+- Allow to pass Node instances to `replace_with`, `insert_before` and `insert_after` methods
+- Added `insert_before` and `insert_after` methods
+
+## Version 0.2.4
+
+Released
+
+- Set maximum input size to 80MB
+- Update modest
+
+## Version 0.2.3
+
+Released
+
+- Rebuild PyPi wheels to support Python 3.8 and manylinux2010
+
+## Version 0.2.2
+
+Released
+
+- Fix node comparison
+
+## Version 0.2.1
+
+Released
+
+- Add optional `include_text` parameter for the `iter` and `traverse` methods
+
+## Version 0.2.0
+
+Released
+
+- Fix `iter()` does not yield text nodes
+- Switch from TravisCI to Github Actions
+- Build and ship wheels for Windows, MacOS and Linux using Azure Pipelines
+- Add `unwrap` and `unwrap_tags` method ([#7](https://github.com/rushter/selectolax/issues/7))
+- Add `replace_with` method ([#13](https://github.com/rushter/selectolax/issues/13))
+- Add `attrs` property
+- Add `traverse` method

{selectolax-0.3.33 → selectolax-0.3.34}/MANIFEST.in

@@ -1,8 +1,8 @@
 include CONTRIBUTING.rst
 include HISTORY.rst
 include LICENSE
-include README.rst
-include CHANGES.rst
+include README.md
+include CHANGES.md
 
 include selectolax/*
 include selectolax/lexbor/*

selectolax-0.3.34/PKG-INFO

@@ -0,0 +1,32 @@
+Metadata-Version: 2.4
+Name: selectolax
+Version: 0.3.34
+Summary: Fast HTML5 parser with CSS selectors.
+Home-page: https://github.com/rushter/selectolax
+Author: Artem Golubin
+Author-email: Artem Golubin <me@rushter.com>
+License-Expression: MIT
+Project-URL: Repository, https://github.com/rushter/selectolax
+Project-URL: Documentation, https://selectolax.readthedocs.io/en/latest/parser.html
+Project-URL: Changelog, https://github.com/rushter/selectolax/blob/main/CHANGES.md
+Keywords: selectolax,html,parser,css,fast
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: Topic :: Text Processing :: Markup :: HTML
+Classifier: Topic :: Internet
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Intended Audience :: Developers
+Classifier: Natural Language :: English
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Requires-Python: >=3.9
+Description-Content-Type: text/x-rst
+License-File: LICENSE
+Provides-Extra: cython
+Requires-Dist: Cython; extra == "cython"
+Dynamic: author
+Dynamic: home-page
+Dynamic: license-file

selectolax-0.3.34/README.md

@@ -0,0 +1,153 @@
+![selectolax logo](docs/logo.png)
+
+---
+
+[![PyPI version](https://img.shields.io/pypi/v/selectolax.svg)](https://pypi.python.org/pypi/selectolax)
+
+A fast HTML5 parser with CSS selectors using [Modest](https://github.com/lexborisov/Modest/) and [Lexbor](https://github.com/lexbor/lexbor) engines.
+
+## Installation
+
+From PyPI using pip:
+
+```bash
+pip install selectolax
+```
+
+If installation fails due to compilation errors, you may need to install [Cython](https://github.com/cython/cython):
+
+```bash
+pip install selectolax[cython]
+```
+
+This usually happens when you try to install an outdated version of selectolax on a newer version of Python.
+
+Development version from GitHub:
+
+```bash
+git clone --recursive  https://github.com/rushter/selectolax
+cd selectolax
+pip install -r requirements_dev.txt
+python setup.py install
+```
+
+How to compile selectolax while developing:
+
+```bash
+make clean
+make dev
+```
+
+## Basic examples
+
+Here are some basic examples to get you started with selectolax:
+
+Parsing HTML and extracting text:
+
+```python
+In [1]: from selectolax.parser import HTMLParser
+   ...:
+   ...: html = """
+   ...: <h1 id="title" data-updated="20201101">Hi there</h1>
+   ...: <div class="post">Lorem Ipsum is simply dummy text of the printing and typesetting industry. </div>
+   ...: <div class="post">Lorem ipsum dolor sit amet, consectetur adipiscing elit.</div>
+   ...: """
+   ...: tree = HTMLParser(html)
+
+In [2]: tree.css_first('h1#title').text()
+Out[2]: 'Hi there'
+
+In [3]: tree.css_first('h1#title').attributes
+Out[3]: {'id': 'title', 'data-updated': '20201101'}
+
+In [4]: [node.text() for node in tree.css('.post')]
+Out[4]:
+['Lorem Ipsum is simply dummy text of the printing and typesetting industry. ',
+ 'Lorem ipsum dolor sit amet, consectetur adipiscing elit.']
+```
+
+### Using advanced CSS selectors
+
+```python
+In [1]: html = "<div><p id=p1><p id=p2><p id=p3><a>link</a><p id=p4><p id=p5>text<p id=p6></div>"
+   ...: selector = "div > :nth-child(2n+1):not(:has(a))"
+
+In [2]: for node in HTMLParser(html).css(selector):
+   ...:     print(node.attributes, node.text(), node.tag)
+   ...:     print(node.parent.tag)
+   ...:     print(node.html)
+   ...:
+{'id': 'p1'}  p
+div
+<p id="p1"></p>
+{'id': 'p5'} text p
+div
+<p id="p5">text</p>
+```
+
+#### Using `lexbor-contains` CSS pseudo-class to match text
+
+```python
+from selectolax.lexbor import LexborHTMLParser
+html = "<div><p>hello </p><p id='main'>lexbor is AwesOme</p></div>"
+parser = LexborHTMLParser(html)
+# Case-insensitive search
+results = parser.css('p:lexbor-contains("awesome" i)')
+# Case-sensitive search
+results = parser.css('p:lexbor-contains("AwesOme")')
+assert len(results) == 1
+assert results[0].text() == "lexbor is AwesOme"
+```
+
+* [Detailed overview](https://github.com/rushter/selectolax/blob/master/examples/walkthrough.ipynb)
+
+### Available backends
+
+Selectolax supports two backends: `Modest` and `Lexbor`. By default, all examples use the Modest backend.
+Most of the features between backends are almost identical, but there are still some differences.
+
+As of 2024, the preferred backend is `Lexbor`. The `Modest` backend is still available for compatibility reasons
+and the underlying C library that selectolax uses is not maintained anymore.
+
+To use `lexbor`, just import the parser and use it in the similar way to the `HTMLParser`.
+
+```python
+In [1]: from selectolax.lexbor import LexborHTMLParser
+
+In [2]: html = """
+   ...: <title>Hi there</title>
+   ...: <div id="updated">2021-08-15</div>
+   ...: """
+
+In [3]: parser = LexborHTMLParser(html)
+In [4]: parser.root.css_first("#updated").text()
+Out[4]: '2021-08-15'
+```
+
+## Simple Benchmark
+
+* Extract title, links, scripts and a meta tag from main pages of top 754 domains. See `examples/benchmark.py` for more information.
+
+| Package                       | Time      |
+|-------------------------------|-----------|
+| Beautiful Soup (html.parser)  | 61.02 sec.|
+| lxml / Beautiful Soup (lxml)  | 9.09 sec. |
+| html5_parser                  | 16.10 sec.|
+| selectolax (Modest)           | 2.94 sec. |
+| selectolax (Lexbor)           | 2.39 sec. |
+
+## Links
+
+* [selectolax API reference](https://selectolax.readthedocs.io/en/latest/index.html)
+* [Video introduction to web scraping using selectolax](https://youtu.be/HpRsfpPuUzE)
+* [How to Scrape 7k Products with Python using selectolax and httpx](https://www.youtube.com/watch?v=XpGvq755J2U)
+* [Detailed overview](https://github.com/rushter/selectolax/blob/master/examples/walkthrough.ipynb)
+* [Modest introduction](https://lexborisov.github.io/Modest/)
+* [Modest benchmark](https://lexborisov.github.io/benchmark-html-parsers/)
+* [Python benchmark](https://rushter.com/blog/python-fast-html-parser/)
+* [Another Python benchmark](https://www.peterbe.com/plog/selectolax-or-pyquery)
+
+## License
+
+* Modest engine — [LGPL2.1](https://github.com/lexborisov/Modest/blob/master/LICENSE)
+* selectolax - [MIT](https://github.com/rushter/selectolax/blob/master/LICENSE)

{selectolax-0.3.33 → selectolax-0.3.34}/lexbor/CHANGELOG.md

@@ -2,6 +2,39 @@
 
 ## [Unreleased]
 
+## [2.5.0] - 2025-08-13
+### Added
+- Added new module Engine.
+- Added benchmarks.
+- Added LEXBOR_BUILD_WITH_MSAN definition for building with msan.
+- CSS::Syntax: added UNICODE-RANGE token.
+- Added link to external bindings for Julia.
+
+### Fixed
+- Core: Fix undefined behavior in function lexbor_str_append().
+- Core: Fix undefined behavior in function lexbor_in_node_make().
+- Core: fixed slow realloc for large strings.
+- HTML: fixed error report for whitespace characters reference.
+- HTML: fixed error report on self-closing tag parsing.
+- HTML: fixed attribute cloning for bad HTML.
+- HTML: fixed duplicate attributes in svg namespace.
+- CSS::Syntax: fixed consume a numeric token.
+- Style: fixed use-after-poison when the element is destroyed.
+- URL: fixed hostname setter if port is specified.
+- Selectors: fixed selector :has(). It didn't always work properly.
+
+### Changed
+- HTML: allowed `<hr>` to be used inside `<select>`.
+- HTML: changing the serialization of attributes. The `<` and `>` characters will now be escaped in attributes.
+- Unicode: significant reduction tables size.
+- Encoding: size reduction and performance up of multibyte encodings.
+- CSS::Syntax: code refactoring.
+- Selectors: code refactoring. The approach to searching for nodes by selectors has been rewritten.
+
+Special thanks for patches, fixes and hints: Michael Hatherly @MichaelHatherly,
+Alex Peattie @alexpeattie, Niels Dossche @nielsdos, Máté Kocsis @kocsismate,
+Sebastian Pipping @hartwork.
+
 ## [2.4.0] - 2024-11-12
 ### Added
 - Core: improve performance using SWAR.

{selectolax-0.3.33 → selectolax-0.3.34}/lexbor/CMakeLists.txt

@@ -17,6 +17,7 @@ cmake_minimum_required(VERSION 2.8.12...3.27)
 #    LEXBOR_BUILD_BENCHMARKS             default: OFF; Build benchmarks.
 #    LEXBOR_BUILD_UTILS                  default: OFF; Build utils/helpers for project.
 #    LEXBOR_BUILD_WITH_ASAN              default: OFF; Build with address sanitizer if possible
+#    LEXBOR_BUILD_WITH_MSAN              default: OFF; Build with memory sanitizer if possible
 #    LEXBOR_BUILD_WITH_FUZZER            default: OFF; Build with fuzzer tests if possible
 #    LEXBOR_INSTALL_HEADERS              default: ON; The header files will be installed
 #                                         if set to ON
@@ -53,6 +54,7 @@ option(LEXBOR_BUILD_TESTS_CPP "Build C++ tests" OFF)
 option(LEXBOR_BUILD_UTILS "Build utils" OFF)
 option(LEXBOR_BUILD_BENCHMARKS "Build with benchmarks" OFF)
 option(LEXBOR_BUILD_WITH_ASAN "Build with address sanitizer" OFF)
+option(LEXBOR_BUILD_WITH_MSAN "Build with memory sanitizer" OFF)
 option(LEXBOR_BUILD_WITH_FUZZER "Build with fuzzer" OFF)
 option(LEXBOR_INSTALL_HEADERS "Install header files" ON)
 option(LEXBOR_WITH_PERF "Enables support for rdtsc." OFF)
@@ -298,6 +300,11 @@ IF(${LEXBOR_HAVE_ASAN})
     add_definitions(-DLEXBOR_HAVE_ADDRESS_SANITIZER)
 ENDIF()
 
+FEATURE_CHECK_MSAN(LEXBOR_HAVE_MSAN)
+IF(${LEXBOR_HAVE_MSAN})
+    add_definitions(-DLEXBOR_HAVE_MEMORY_SANITIZER)
+ENDIF()
+
 FEATURE_CHECK_FUZZER(LEXBOR_HAVE_FUZZER)
 IF(${LEXBOR_HAVE_FUZZER})
     set(LEXBOR_BUILD_EXAMPLES OFF)

{selectolax-0.3.33 → selectolax-0.3.34}/lexbor/README.md

@@ -179,7 +179,7 @@ make
 make test
 ```
 
-Please, see more information in [documentation](https://lexbor.com/docs/lexbor/#source_code).
+Please, see more information in [documentation](https://lexbor.com/documentation/#source-code).
 
 ## Single or separately
 
@@ -220,13 +220,18 @@ Please, see [roadmap](https://lexbor.com/roadmap/) on [lexbor.com](https://lexbo
 
 * E-mail [support@lexbor.com](mailto:support@lexbor.com)
 
-## SAST Tools
+## Our Sponsors
 
-[PVS-Studio](https://pvs-studio.com/en/pvs-studio/?utm_source=website&utm_medium=github&utm_campaign=open_source) - static analyzer for C, C++, C#, and Java code.
+[<img src="images/neural-logo.png" alt="goneural.ai" width="320">](https://goneural.ai/)
+
+## Sponsorship
 
-## AUTHOR
+You can help sponsor the maintainers of this software through the following organization:
+[github.com/sponsors/toxypi](https://github.com/sponsors/toxypi)
 
-Alexander Borisov <borisov@lexbor.com>
+## SAST Tools
+
+[PVS-Studio](https://pvs-studio.com/en/pvs-studio/?utm_source=website&utm_medium=github&utm_campaign=open_source) - static analyzer for C, C++, C#, and Java code.
 
 ## COPYRIGHT AND LICENSE