scrapling 0.3.1__py3-none-any.whl → 0.3.2__py3-none-any.whl

{scrapling-0.3.1.dist-info → scrapling-0.3.2.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scrapling
-Version: 0.3.1
+Version: 0.3.2
 Summary: Scrapling is an undetectable, powerful, flexible, high-performance Python library that makes Web Scraping easy and effortless as it should be!
 Home-page: https://github.com/D4Vinci/Scrapling
 Author: Karim Shoair
@@ -64,23 +64,26 @@ Classifier: Typing :: Typed
 Requires-Python: >=3.10
 Description-Content-Type: text/markdown
 License-File: LICENSE
-Requires-Dist: lxml>=6.0.0
+Requires-Dist: lxml>=6.0.1
 Requires-Dist: cssselect>=1.3.0
-Requires-Dist: click>=8.2.1
-Requires-Dist: orjson>=3.11.2
+Requires-Dist: orjson>=3.11.3
 Requires-Dist: tldextract>=5.3.0
-Requires-Dist: curl_cffi>=0.13.0
-Requires-Dist: playwright>=1.52.0
-Requires-Dist: rebrowser-playwright>=1.52.0
-Requires-Dist: camoufox>=0.4.11
-Requires-Dist: geoip2>=5.1.0
-Requires-Dist: msgspec>=0.19.0
+Provides-Extra: fetchers
+Requires-Dist: click>=8.2.1; extra == "fetchers"
+Requires-Dist: curl_cffi>=0.13.0; extra == "fetchers"
+Requires-Dist: playwright>=1.52.0; extra == "fetchers"
+Requires-Dist: rebrowser-playwright>=1.52.0; extra == "fetchers"
+Requires-Dist: camoufox>=0.4.11; extra == "fetchers"
+Requires-Dist: geoip2>=5.1.0; extra == "fetchers"
+Requires-Dist: msgspec>=0.19.0; extra == "fetchers"
 Provides-Extra: ai
-Requires-Dist: mcp>=1.13.0; extra == "ai"
+Requires-Dist: mcp>=1.14.0; extra == "ai"
 Requires-Dist: markdownify>=1.2.0; extra == "ai"
+Requires-Dist: scrapling[fetchers]; extra == "ai"
 Provides-Extra: shell
 Requires-Dist: IPython>=8.37; extra == "shell"
 Requires-Dist: markdownify>=1.2.0; extra == "shell"
+Requires-Dist: scrapling[fetchers]; extra == "shell"
 Provides-Extra: all
 Requires-Dist: scrapling[ai,shell]; extra == "all"
 Dynamic: license-file
@@ -158,6 +161,7 @@ Built for the modern Web, Scrapling has its own rapid parsing engine and its fet
 <a href="https://petrosky.io/d4vinci" target="_blank" title="PetroSky delivers cutting-edge VPS hosting."><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/petrosky.png"></a>
 <a href="https://www.swiftproxy.net/" target="_blank" title="Unlock Reliable Proxy Services with Swiftproxy!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/swiftproxy.png"></a>
 <a href="https://serpapi.com/?utm_source=scrapling" target="_blank" title="Scrape Google and other search engines with SerpApi"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/SerpApi.png"></a>
+<a href="https://www.nstproxy.com/?type=flow&utm_source=scrapling" target="_blank" title="One Proxy Service, Infinite Solutions at Unbeatable Prices!"><img src="https://raw.githubusercontent.com/D4Vinci/Scrapling/main/images/NSTproxy.png"></a>
 
 <!-- /sponsors -->
 
@@ -178,7 +182,7 @@ Built for the modern Web, Scrapling has its own rapid parsing engine and its fet
 - 🔄 **Smart Element Tracking**: Relocate elements after website changes using intelligent similarity algorithms.
 - 🎯 **Smart Flexible Selection**: CSS selectors, XPath selectors, filter-based search, text search, regex search, and more. 
 - 🔍 **Find Similar Elements**: Automatically locate elements similar to found elements.
-- 🤖 **MCP Server to be used with AI**: Built-in MCP server for AI-assisted Web Scraping and data extraction. The MCP server features custom, powerful capabilities that utilize Scrapling to extract targeted content before passing it to the AI (Claude/Cursor/etc), thereby speeding up operations and reducing costs by minimizing token usage.
+- 🤖 **MCP Server to be used with AI**: Built-in MCP server for AI-assisted Web Scraping and data extraction. The MCP server features custom, powerful capabilities that utilize Scrapling to extract targeted content before passing it to the AI (Claude/Cursor/etc), thereby speeding up operations and reducing costs by minimizing token usage. ([demo video](https://www.youtube.com/watch?v=qyFk3ZNwOxE))
 
 ### High-Performance & battle-tested Architecture
 - 🚀 **Lightning Fast**: Optimized performance outperforming most Python scraping libraries.
@@ -220,7 +224,7 @@ quotes = page.css('.quote .text::text')
 
 # Advanced stealth mode (Keep the browser open until you finish)
 with StealthySession(headless=True, solve_cloudflare=True) as session:
-    page = session.fetch('https://nopecha.com/demo/cloudflare')
+    page = session.fetch('https://nopecha.com/demo/cloudflare', google_search=False)
     data = page.css('#padded_content a')
 
 # Or use one-off request style, it opens the browser for this request, then closes it after finishing
@@ -229,7 +233,7 @@ data = page.css('#padded_content a')
     
 # Full browser automation (Keep the browser open until you finish)
 with DynamicSession(headless=True, disable_resources=False, network_idle=True) as session:
-    page = session.fetch('https://quotes.toscrape.com/')
+    page = session.fetch('https://quotes.toscrape.com/', load_dom=False)
     data = page.xpath('//span[@class="text"]/text()')  # XPath selector if you prefer it
 
 # Or use one-off request style, it opens the browser for this request, then closes it after finishing
@@ -273,7 +277,7 @@ from scrapling.parser import Selector
 
 page = Selector("<html>...</html>")
 ```
-And it works exactly the same way!
+And it works precisely the same way!
 
 ### Async Session Management Examples
 ```python
@@ -322,20 +326,20 @@ scrapling extract stealthy-fetch 'https://nopecha.com/demo/cloudflare' captchas.
 
 ## Performance Benchmarks
 
-Scrapling isn't just powerful—it's also blazing fast, and version 0.3 delivers exceptional performance improvements across all operations!
+Scrapling isn't just powerful—it's also blazing fast, and the updates since version 0.3 deliver exceptional performance improvements across all operations!
 
 ### Text Extraction Speed Test (5000 nested elements)
 
 | # |      Library      | Time (ms) | vs Scrapling | 
 |---|:-----------------:|:---------:|:------------:|
-| 1 |     Scrapling     |   1.88    |     1.0x     |
-| 2 |   Parsel/Scrapy   |   1.96    |    1.043x    |
-| 3 |     Raw Lxml      |   2.32    |    1.234x    |
-| 4 |      PyQuery      |   20.2    |     ~11x     |
-| 5 |    Selectolax     |   85.2    |     ~45x     |
-| 6 |  MechanicalSoup   |  1305.84  |    ~695x     |
-| 7 |   BS4 with Lxml   |  1307.92  |    ~696x     |
-| 8 | BS4 with html5lib |  3336.28  |    ~1775x    |
+| 1 |     Scrapling     |   1.92    |     1.0x     |
+| 2 |   Parsel/Scrapy   |   1.99    |    1.036x    |
+| 3 |     Raw Lxml      |   2.33    |    1.214x    |
+| 4 |      PyQuery      |   20.61   |     ~11x     |
+| 5 |    Selectolax     |   80.65   |     ~42x     |
+| 6 |   BS4 with Lxml   |  1283.21  |    ~698x     |
+| 7 |  MechanicalSoup   |  1304.57  |    ~679x     |
+| 8 | BS4 with html5lib |  3331.96  |    ~1735x    |
 
 ### Element Similarity & Text Search Performance
 
@@ -343,8 +347,8 @@ Scrapling's adaptive element finding capabilities significantly outperform alter
 
 |   Library   | Time (ms) | vs Scrapling |
 |-------------|:---------:|:------------:|
-|  Scrapling  |   2.02    |     1.0x     |
-| AutoScraper |   10.26   |    5.08x     |
+|  Scrapling  |   1.87    |     1.0x     |
+| AutoScraper |   10.24   |    5.476x    |
 
 
 > All benchmarks represent averages of 100+ runs. See [benchmarks.py](https://github.com/D4Vinci/Scrapling/blob/main/benchmarks.py) for methodology.
@@ -357,29 +361,33 @@ Scrapling requires Python 3.10 or higher:
 pip install scrapling
 ```
 
-#### Fetchers Setup
-
-If you are going to use any of the fetchers or their classes, then install browser dependencies with
-```bash
-scrapling install
-```
-
-This downloads all browsers with their system dependencies and fingerprint manipulation dependencies.
+Starting with v0.3.2, this installation only includes the parser engine and its dependencies, without any fetchers or commandline dependencies.
 
 ### Optional Dependencies
 
-- Install the MCP server feature:
-```bash
-pip install "scrapling[ai]"
-```
-- Install shell features (Web Scraping shell and the `extract` command): 
-```bash
-pip install "scrapling[shell]"
-```
-- Install everything: 
-```bash
-pip install "scrapling[all]"
-```
+1. If you are going to use any of the extra features below, the fetchers, or their classes, then you need to install fetchers' dependencies, and then install their browser dependencies with
+    ```bash
+    pip install "scrapling[fetchers]"
+    
+    scrapling install
+    ```
+
+    This downloads all browsers with their system dependencies and fingerprint manipulation dependencies.
+
+2. Extra features:
+   - Install the MCP server feature:
+       ```bash
+       pip install "scrapling[ai]"
+       ```
+   - Install shell features (Web Scraping shell and the `extract` command): 
+       ```bash
+       pip install "scrapling[shell]"
+       ```
+   - Install everything: 
+       ```bash
+       pip install "scrapling[all]"
+       ```
+   Don't forget that you need to install the browser dependencies with `scrapling install` after any of these extras (if you didn't already)
 
 ## Contributing
 

scrapling-0.3.2.dist-info/RECORD

@@ -0,0 +1,44 @@
+scrapling/__init__.py,sha256=5TF141QF33L868x11G7suZB4cwAH-H73jKrO7eFxpBE,1236
+scrapling/cli.py,sha256=ooObP0VoYGxnskEJB6xFp23NREI_XDPJpsMSr9Sv8nk,26355
+scrapling/fetchers.py,sha256=aYQUxp-0i-OBucdpdG6zjWCafTCgpXJdnJ0GIrm5GfA,26523
+scrapling/parser.py,sha256=aJRqfuOxBHrM_Co9XHeuL6qYHgQTyi7zD1DoCA3mROY,57321
+scrapling/py.typed,sha256=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN_XKdLCPjaYaY,2
+scrapling/core/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+scrapling/core/_html_utils.py,sha256=ki47v54SsTL5-khi1jcLkJqAHqEq19cuex-dqzXdbEI,20328
+scrapling/core/_types.py,sha256=iXhi8LFkU4wjkGOjITdY1IDBEbn5rOxsl7xwEKT1L3I,895
+scrapling/core/ai.py,sha256=v3wjtXJgBRUtImE6Q_Bf_FruOArJyraQk4kqsqhlU8k,35474
+scrapling/core/custom_types.py,sha256=3kLrNDVm1vP3IziyyNjFVVQO_2bacwvm3hiK7h3gWjU,13634
+scrapling/core/mixins.py,sha256=2iUVcN2XSAKGEvNmAM2Rr9axpZoxu0M2gIFEaFTO_Dg,3206
+scrapling/core/shell.py,sha256=Um_CukPuX8K1VgEZsu_cXdsnoJTC_lnv4co2DBZXICU,21956
+scrapling/core/storage.py,sha256=8lWMPut6lPpvn9iOkgy9ao11_g8FNkXq67wHKtU4uuM,6290
+scrapling/core/translator.py,sha256=HLJngeRRw2M0eNe_f8AfQD64a49OECIEm5Df_WELVG4,5135
+scrapling/core/utils/__init__.py,sha256=7B14TcrDVwSaH6BQrMnzb1NtFa4Om237dJcF9oe-lM0,204
+scrapling/core/utils/_shell.py,sha256=zes71MmFTs7V9f0JFstaWcjQhKNZN6xvspu29YVQtRc,1707
+scrapling/core/utils/_utils.py,sha256=ATy-wwz00U-alOGH-NGK-VoPNr1qYmUwEoWuqAHjDkg,3143
+scrapling/engines/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
+scrapling/engines/constants.py,sha256=DP7yVRK1w1W4B1eXGeeKKQNDSo163FFXdPaxTy4adqw,4088
+scrapling/engines/static.py,sha256=779pVJvCCjFSCKGN7a_9rAH55oO2SyO88cnVrE2DQy8,33204
+scrapling/engines/_browsers/__init__.py,sha256=lu5RgcV4zYacRaKm28ph5TzjqAovTaQNNfXSgQGwDOU,123
+scrapling/engines/_browsers/_base.py,sha256=vX75atq2QpsiCCpT4P2W4HQqs3P2RZWYHvDevh271kQ,12238
+scrapling/engines/_browsers/_camoufox.py,sha256=oHXEaboA8G_Z6MoipeKuYNOFW2UgH93IAH0VUYu649Q,35134
+scrapling/engines/_browsers/_config_tools.py,sha256=mEPA5SGrWq0dl15cDOT6sOsm5NHMD0vI0fuPttGpw-U,4610
+scrapling/engines/_browsers/_controllers.py,sha256=6tlAsPtATLlaPq43oWILiHlwJ0yPEcN_8FsUugZLmwQ,28490
+scrapling/engines/_browsers/_page.py,sha256=ixwI5d-AIzfUGekRSCbPLJAckf673B7QCyaWO-xJa84,3688
+scrapling/engines/_browsers/_validators.py,sha256=knkGvgpGeqtOWx4Us3pln1o4mJXfG4M-SWII080I9AE,6117
+scrapling/engines/toolbelt/__init__.py,sha256=AbpHGcgLb-kRsJGnwFEktk7uzpZOCcBY74-YBdrKVGs,1
+scrapling/engines/toolbelt/convertor.py,sha256=e_rMcW8ScdfxKO-V5Mk61blVzwuDgd82CpRds0Z2tMQ,13102
+scrapling/engines/toolbelt/custom.py,sha256=uhMXa_LNcvvG3wZXBRKHXvqLqShMR9SHwc3bBv4UaQs,7664
+scrapling/engines/toolbelt/fingerprints.py,sha256=hCxKUTwo8sy7iN9wk8OA5vGo9XOn6E365zvC1C6zWDE,2212
+scrapling/engines/toolbelt/navigation.py,sha256=CWvM0KDuLPzvM6T8_yBq05nmB4scXshoEKVbhR4zEBk,3561
+scrapling/engines/toolbelt/bypasses/navigator_plugins.js,sha256=tbnnk3nCXB6QEQnOhDlu3n-s7lnUTAkrUsjP6FDQIQg,2104
+scrapling/engines/toolbelt/bypasses/notification_permission.js,sha256=poPM3o5WYgEX-EdiUfDCllpWfc3Umvw4jr2u6O6elus,237
+scrapling/engines/toolbelt/bypasses/playwright_fingerprint.js,sha256=clzuf7KYcvDWYaKKxT_bkAoCT2fGsOcUw47948CHjAc,267
+scrapling/engines/toolbelt/bypasses/screen_props.js,sha256=fZEuHMQ1-fYuxxUMoQXUvVWYUkPUbblkfMfpiLvBY7w,599
+scrapling/engines/toolbelt/bypasses/webdriver_fully.js,sha256=hdJw4clRAJQqIdq5gIFC_eC-x7C1i2ab01KV5ylmOBs,728
+scrapling/engines/toolbelt/bypasses/window_chrome.js,sha256=D7hqzNGGDorh8JVlvm2YIv7Bk2CoVkG55MDIdyqhT1w,6808
+scrapling-0.3.2.dist-info/licenses/LICENSE,sha256=XHgu8DRuT7_g3Hb9Q18YGg8eShp6axPBacbnQxT_WWQ,1499
+scrapling-0.3.2.dist-info/METADATA,sha256=NDQwyt5_1fQfDjizTLO5jUc2-Ww5KvrcdHCRoOuNERw,21948
+scrapling-0.3.2.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
+scrapling-0.3.2.dist-info/entry_points.txt,sha256=DHyt2Blxy0P5OE2HRcP95Wz9_xo2ERCDcNqrJjYS3o8,49
+scrapling-0.3.2.dist-info/top_level.txt,sha256=Ud-yF-PC2U5HQ3nc5QwT7HSPdIpF1RuwQ_mYgBzHHIM,10
+scrapling-0.3.2.dist-info/RECORD,,

scrapling-0.3.1.dist-info/RECORD

@@ -1,41 +0,0 @@
-scrapling/__init__.py,sha256=XOnMVykdniMkfYjaVULcCsKRa4rkNrjQm8NWpSOBGzI,1236
-scrapling/cli.py,sha256=jd1R-1ASOtZhV5D2qcYXoyfWYkQt11lIhuTKzET3XRU,26009
-scrapling/fetchers.py,sha256=l-N3tDfpM3sbCAvUNZdePMTbYnwnDFXZ3XxOFLqWmPE,26030
-scrapling/parser.py,sha256=jwr9QC-L3BLpoOoZ15tSv7FBKMXybpE-ekcXsfvDW_c,58357
-scrapling/py.typed,sha256=frcCV1k9oG9oKj3dpUqdJg1PxRT2RSN_XKdLCPjaYaY,2
-scrapling/core/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
-scrapling/core/_html_utils.py,sha256=0YfzgIV13cd_RxWAHpyaTrCZ5vv9t_Vmkrh43CZC2p0,20386
-scrapling/core/_types.py,sha256=iXhi8LFkU4wjkGOjITdY1IDBEbn5rOxsl7xwEKT1L3I,895
-scrapling/core/ai.py,sha256=6Cf-3AG8KtuGLxe_N8ruHau-8NWYgkylDGp3KNxjWbA,35515
-scrapling/core/custom_types.py,sha256=kYBNz8xVj9XmK2n8w0nuQY2gECM7gGSW6uUH8ZAjG1c,14038
-scrapling/core/mixins.py,sha256=AtOzY7Wp5vLBqnH4Ur-Id_0__sxrgsDsPOammogMyCY,3684
-scrapling/core/shell.py,sha256=IM6dOtc3diUcg_nU5q-lcX8JTLzaA_E_9k4oz7UDGhQ,23664
-scrapling/core/storage.py,sha256=NFRIM-i7lllpwPQWH_Lfyrb-mk52liRFkLOoiyMBRbI,6376
-scrapling/core/translator.py,sha256=lxX6h5I5pN0K0yLyfYAlMPNgToroRnEDD5qL-h9IU2c,5301
-scrapling/core/utils.py,sha256=Fi_Lv8rOhWhjXWV8Xccl39fyxri8A5oXfE1iqTBdHFU,3325
-scrapling/engines/__init__.py,sha256=Vru520eSiG1PeBlWE2LCFZoE0BaWkAhE1RKTfXjX8tI,414
-scrapling/engines/constants.py,sha256=w-mzpCQKHSQt0SwfeDY7ztdvKp9StKhp3AqerkRYRJ0,4186
-scrapling/engines/static.py,sha256=eDTMUUP1Vqrw_kMhIykUgcGLQvPa83-ojakIkTfDJSQ,33867
-scrapling/engines/_browsers/__init__.py,sha256=lu5RgcV4zYacRaKm28ph5TzjqAovTaQNNfXSgQGwDOU,123
-scrapling/engines/_browsers/_camoufox.py,sha256=dYlKQULv_1DvVSOHcQC0rfE20wegO_jmrs1TshUvkW4,33530
-scrapling/engines/_browsers/_config_tools.py,sha256=Jg4BkvWdRBh6VeMTfKJkjkuOSPe7VFvQ114Uj-ZXssU,4548
-scrapling/engines/_browsers/_controllers.py,sha256=TtQYiL8fsh9-mrnQkjHruyNuyUozqklfryukFjojxAw,26912
-scrapling/engines/_browsers/_page.py,sha256=VlHQsUTMpZhpM_0jLPdGy4JEiJMQ0YV8eE37kJUXpDk,2797
-scrapling/engines/_browsers/_validators.py,sha256=uQRd6Rk2PHccxwxdmYGKHWRG3Z5WUrebmjRdAc-fwwk,6170
-scrapling/engines/toolbelt/__init__.py,sha256=d21D8tuu2vpGY91o4AzMrNK3nKzy0JiK3pz7jjfLIMg,407
-scrapling/engines/toolbelt/convertor.py,sha256=kAM8Sffq3PBK6i3VTbn50uidTZmqTQZVcLzq18-YdJ0,12191
-scrapling/engines/toolbelt/custom.py,sha256=aWcxDUDZ5qogWHu7W2F2rUFMEDFiHO0QslTRW0MLPDA,10790
-scrapling/engines/toolbelt/fingerprints.py,sha256=MH5-xBmhtcjLPm0G1q0iXQFXGcUOvmY061RniXTsoTA,2288
-scrapling/engines/toolbelt/navigation.py,sha256=bw6-rfZPMxtjLyAm58KcQizrxEHNW4zvLi65-ymc6Dw,5145
-scrapling/engines/toolbelt/bypasses/navigator_plugins.js,sha256=tbnnk3nCXB6QEQnOhDlu3n-s7lnUTAkrUsjP6FDQIQg,2104
-scrapling/engines/toolbelt/bypasses/notification_permission.js,sha256=poPM3o5WYgEX-EdiUfDCllpWfc3Umvw4jr2u6O6elus,237
-scrapling/engines/toolbelt/bypasses/playwright_fingerprint.js,sha256=clzuf7KYcvDWYaKKxT_bkAoCT2fGsOcUw47948CHjAc,267
-scrapling/engines/toolbelt/bypasses/screen_props.js,sha256=fZEuHMQ1-fYuxxUMoQXUvVWYUkPUbblkfMfpiLvBY7w,599
-scrapling/engines/toolbelt/bypasses/webdriver_fully.js,sha256=hdJw4clRAJQqIdq5gIFC_eC-x7C1i2ab01KV5ylmOBs,728
-scrapling/engines/toolbelt/bypasses/window_chrome.js,sha256=D7hqzNGGDorh8JVlvm2YIv7Bk2CoVkG55MDIdyqhT1w,6808
-scrapling-0.3.1.dist-info/licenses/LICENSE,sha256=XHgu8DRuT7_g3Hb9Q18YGg8eShp6axPBacbnQxT_WWQ,1499
-scrapling-0.3.1.dist-info/METADATA,sha256=DE-JEFGhRVgFmpUOEZP_4JTBhd8ALWhO71-PI-uU068,20809
-scrapling-0.3.1.dist-info/WHEEL,sha256=_zCd3N1l69ArxyTb8rzEoP9TpbYXkqRFSNOD5OuxnTs,91
-scrapling-0.3.1.dist-info/entry_points.txt,sha256=DHyt2Blxy0P5OE2HRcP95Wz9_xo2ERCDcNqrJjYS3o8,49
-scrapling-0.3.1.dist-info/top_level.txt,sha256=Ud-yF-PC2U5HQ3nc5QwT7HSPdIpF1RuwQ_mYgBzHHIM,10
-scrapling-0.3.1.dist-info/RECORD,,