webchanges 3.34.2__tar.gz → 3.35.0__tar.gz

{webchanges-3.34.2/webchanges.egg-info → webchanges-3.35.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: webchanges
-Version: 3.34.2
+Version: 3.35.0
 Summary: Web Changes Delivered. AI-Summarized. Totally Anonymous.
 Author-email: Mike Borsetti <mike+webchanges@borsetti.com>
 Maintainer-email: Mike Borsetti <mike+webchanges@borsetti.com>
@@ -89,6 +89,8 @@ Provides-Extra: pushover
 Requires-Dist: chump; extra == "pushover"
 Provides-Extra: xmpp
 Requires-Dist: aioxmpp; extra == "xmpp"
+Provides-Extra: curl-cffi
+Requires-Dist: curl_cffi; extra == "curl-cffi"
 Provides-Extra: redis
 Requires-Dist: redis; extra == "redis"
 Provides-Extra: requests
@@ -96,7 +98,7 @@ Requires-Dist: requests; extra == "requests"
 Provides-Extra: safe-password
 Requires-Dist: keyring; extra == "safe-password"
 Provides-Extra: all
-Requires-Dist: webchanges[beautify,bs4,deepdiff_xml,html5lib,ical2text,imagediff,jq,matrix,ocr,pdf2text,pushbullet,pushover,pypdf_crypto,redis,requests,safe_password,use_browser,xmpp]; extra == "all"
+Requires-Dist: webchanges[beautify,bs4,curl_cffi,deepdiff_xml,html5lib,ical2text,imagediff,jq,matrix,ocr,pdf2text,pushbullet,pushover,pypdf_crypto,redis,requests,safe_password,use_browser,xmpp]; extra == "all"
 Dynamic: license-file
 
 .. role:: underline
@@ -139,7 +141,7 @@ Install **webchanges**  with ``uv`` (recommended):
 .. code-block:: bash
 
    uv pip install webchanges
-   
+
 or with ``pip``:
 
 .. code-block:: bash
@@ -154,13 +156,14 @@ implementation (no browser) `here <https://github.com/yubiuser/webchanges-docker
 
 As a GitHub Action
 ------------------
-**webchanges** can easily run as a `GitHub Action <https://www.docker.com/>`__! You will find an implementation 
+**webchanges** can easily run as a `GitHub Action <https://www.docker.com/>`__! You will find an implementation
 `here <https://github.com/swimmwatch/webchanges-action>`__.
 
 
 Documentation |readthedocs|
 ===========================
-The documentation is hosted on `Read the Docs <https://webchanges.readthedocs.io/>`__.
+The documentation is hosted on `Read the Docs <https://webchanges.readthedocs.io/>`__.  A (very good) AI-generated
+high-level architectural overview is available on `DeepWiki <https://deepwiki.com/mborsetti/webchanges>`__.
 
 
 Quick Start
@@ -208,7 +211,7 @@ Schedule
 
 Code
 ====
-|coveralls| |snyk| |issues| |code_style| |OpenSSF_Scorecard|
+|coveralls| |issues| |code_style| |OpenSSF_Scorecard|
 
 The code, issues tracker, and discussions are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
 
@@ -240,7 +243,7 @@ Compatibility with **urlwatch**
 This project is based on code from `urlwatch 2.21
 <https://github.com/thp/urlwatch/tree/346b25914b0418342ffe2fb0529bed702fddc01f>`__ dated 30 July 2020.
 For upgrading from **urlwatch** to **webchanges** (using the same job and
-configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upgrading.html>`__. 
+configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upgrading.html>`__.
 
 
 .. |support| image:: https://img.shields.io/pypi/pyversions/webchanges.svg
@@ -285,9 +288,9 @@ configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upg
 .. .. |security| image:: https://img.shields.io/badge/security-bandit-green.svg
 ..     :target: https://github.com/PyCQA/bandit
 ..     :alt: Security Status
-.. |snyk| image:: https://snyk.io/advisor/python/holidays/badge.svg
-    :target: https://snyk.io/advisor/python/holidays
-    :alt: Snyk Package Health Score
+.. .. |snyk| image:: https://snyk.io/advisor/python/webchanges/badge.svg
+..     :target: https://security.snyk.io/package/pip/webchanges
+..     :alt: Snyk Package Health Score
 .. |OpenSSF_Scorecard| image:: https://api.scorecard.dev/projects/github.com/mborsetti/webchanges/badge
     :target: https://scorecard.dev/viewer/?uri=github.com/mborsetti/webchanges
     :alt: OpenSSF Scoreard

{webchanges-3.34.2 → webchanges-3.35.0}/README.rst

@@ -38,7 +38,7 @@ Install **webchanges**  with ``uv`` (recommended):
 .. code-block:: bash
 
    uv pip install webchanges
-   
+
 or with ``pip``:
 
 .. code-block:: bash
@@ -53,13 +53,14 @@ implementation (no browser) `here <https://github.com/yubiuser/webchanges-docker
 
 As a GitHub Action
 ------------------
-**webchanges** can easily run as a `GitHub Action <https://www.docker.com/>`__! You will find an implementation 
+**webchanges** can easily run as a `GitHub Action <https://www.docker.com/>`__! You will find an implementation
 `here <https://github.com/swimmwatch/webchanges-action>`__.
 
 
 Documentation |readthedocs|
 ===========================
-The documentation is hosted on `Read the Docs <https://webchanges.readthedocs.io/>`__.
+The documentation is hosted on `Read the Docs <https://webchanges.readthedocs.io/>`__.  A (very good) AI-generated
+high-level architectural overview is available on `DeepWiki <https://deepwiki.com/mborsetti/webchanges>`__.
 
 
 Quick Start
@@ -107,7 +108,7 @@ Schedule
 
 Code
 ====
-|coveralls| |snyk| |issues| |code_style| |OpenSSF_Scorecard|
+|coveralls| |issues| |code_style| |OpenSSF_Scorecard|
 
 The code, issues tracker, and discussions are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
 
@@ -139,7 +140,7 @@ Compatibility with **urlwatch**
 This project is based on code from `urlwatch 2.21
 <https://github.com/thp/urlwatch/tree/346b25914b0418342ffe2fb0529bed702fddc01f>`__ dated 30 July 2020.
 For upgrading from **urlwatch** to **webchanges** (using the same job and
-configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upgrading.html>`__. 
+configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upgrading.html>`__.
 
 
 .. |support| image:: https://img.shields.io/pypi/pyversions/webchanges.svg
@@ -184,9 +185,9 @@ configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upg
 .. .. |security| image:: https://img.shields.io/badge/security-bandit-green.svg
 ..     :target: https://github.com/PyCQA/bandit
 ..     :alt: Security Status
-.. |snyk| image:: https://snyk.io/advisor/python/holidays/badge.svg
-    :target: https://snyk.io/advisor/python/holidays
-    :alt: Snyk Package Health Score
+.. .. |snyk| image:: https://snyk.io/advisor/python/webchanges/badge.svg
+..     :target: https://security.snyk.io/package/pip/webchanges
+..     :alt: Snyk Package Health Score
 .. |OpenSSF_Scorecard| image:: https://api.scorecard.dev/projects/github.com/mborsetti/webchanges/badge
     :target: https://scorecard.dev/viewer/?uri=github.com/mborsetti/webchanges
     :alt: OpenSSF Scoreard

{webchanges-3.34.2 → webchanges-3.35.0}/pyproject.toml

@@ -85,12 +85,13 @@ pushbullet = ['pushbullet.py']
 pushover = ['chump']
 xmpp = ['aioxmpp']
 # other
+curl_cffi = ['curl_cffi']
 redis = ['redis']
 requests = ['requests']
 safe_password = ['keyring']
 # all
 all = [
-    'webchanges[use_browser,beautify,bs4,html5lib,ical2text,jq,ocr,pdf2text,pypdf_crypto,deepdiff_xml,imagediff,matrix,pushbullet,pushover,xmpp,redis,requests,safe_password]',
+    'webchanges[use_browser,beautify,bs4,html5lib,ical2text,jq,ocr,pdf2text,pypdf_crypto,deepdiff_xml,imagediff,matrix,pushbullet,pushover,xmpp,curl_cffi,redis,requests,safe_password]',
 ]
 
 

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/__init__.py

@@ -22,7 +22,7 @@ __project_name__ = str(__package__)
 # * MINOR version when you add functionality in a backwards compatible manner, and
 # * MICRO or PATCH version when you make backwards compatible bug fixes. We no longer use '0'
 # If unsure on increments, use pkg_resources.parse_version to parse
-__version__ = '3.34.2'
+__version__ = '3.35.0'
 __description__ = (
     'Check web (or command output) for changes since last run and notify.\n\nAnonymously alerts you of web changes.'
 )

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/_vendored/headers.py

@@ -36,11 +36,7 @@ from collections.abc import Mapping
 
 # from https://github.com/encode/httpx/blob/master/httpx/_types.py
 HeaderTypes: typing.TypeAlias = (
-    'Headers'
-    | typing.Mapping[str, str]
-    | typing.Mapping[bytes, bytes]
-    | typing.Sequence[tuple[str, str]]
-    | typing.Sequence[tuple[bytes, bytes]]
+    'Headers | typing.Mapping[str, str] | typing.Mapping[bytes, bytes] | typing.Sequence[tuple[str, str]] | typing.Sequence[tuple[bytes, bytes]]'
 )
 
 

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/command.py

@@ -195,6 +195,7 @@ class UrlwatchCommand:
                 'cryptography',
                 'cssbeautifier',
                 'cssselect',
+                'curl_cffi',
                 'deepdiff',
                 'h2',
                 'html2text',
@@ -512,6 +513,8 @@ class UrlwatchCommand:
             return
         self.urlwatcher.urlwatch_config.joblist = list(self.urlwatcher.urlwatch_config.joblist) + new_jobs
         self.urlwatcher.run_jobs()
+        if self.urlwatcher.report.config['display']['new']:
+            self.urlwatcher.report.finish(self.urlwatcher.jobs_storage.filename)
         return
 
     def test_differ(self, arg_test_differ: list[str]) -> int:
@@ -713,12 +716,12 @@ class UrlwatchCommand:
                 jobs_to_run = [job for job in jobs if not job.__is_browser__]
                 if jobs_to_run:
                     logger.debug(
-                        "Running jobs that do not require Chrome (without 'use_browser: true') in parallel with "
+                        "Running jobs that do not require a browser (without 'use_browser') in parallel with "
                         "Python's default max_workers."
                     )
                     yield from job_runner(stack, jobs_to_run, self.urlwatch_config.max_workers)
                 else:
-                    logger.debug("Found no jobs that do not require Chrome (i.e. without 'use_browser: true').")
+                    logger.debug("Found no jobs that do not require a browser (i.e. without 'use_browser').")
 
                 # run BrowserJob jobs after
                 jobs_to_run = [job for job in jobs if job.__is_browser__]
@@ -732,12 +735,12 @@ class UrlwatchCommand:
                         max_workers = max(int(virt_mem / 800), 1)
                         max_workers = min(max_workers, os.cpu_count() or 1)
                     logger.debug(
-                        f"Running jobs that require Chrome (i.e. with 'use_browser: true') in parallel with "
+                        f"Running jobs that require a browser (i.e. with 'use_browser') in parallel with "
                         f'{max_workers} max_workers.'
                     )
                     yield from job_runner(stack, jobs_to_run, max_workers)
                 else:
-                    logger.debug("Found no jobs that require Chrome (i.e. with 'use_browser: true').")
+                    logger.debug("Found no jobs that require a browser (i.e. with 'use_browser').")
 
         start = time.perf_counter()
 
@@ -1071,7 +1074,7 @@ class UrlwatchCommand:
         cfg: _ConfigReportersList = self.urlwatcher.config_storage.config['report'][reporter_name]  # ty:ignore[invalid-key]
         if job_state:  # we want a full report
             cfg['enabled'] = True
-            self.urlwatcher.config_storage.config['display'][label] = True
+            self.urlwatcher.config_storage.config['display'][label] = True  # ty:ignore[invalid-key]
             self.urlwatcher.config_storage.config['report']['text']['details'] = True
             self.urlwatcher.config_storage.config['report']['text']['footer'] = True
             self.urlwatcher.config_storage.config['report']['text']['minimal'] = False

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/differs.py

@@ -1735,9 +1735,9 @@ class AIGoogleDiffer(DifferBase):
         if directives.get('tools'):
             data['tools'] = directives['tools']
         if directives.get('thinking_level'):
-            data['generationConfig'].update({'thinkingConfig': {'thinkingLevel': directives['thinking_level']}})
+            data['generationConfig'].update({'thinkingConfig': {'thinkingLevel': directives['thinking_level']}})  # ty:ignore[no-matching-overload]
         elif directives.get('thinking_budget'):
-            data['generationConfig'].update({'thinkingConfig': {'thinkingBudget': directives['thinking_budget']}})
+            data['generationConfig'].update({'thinkingConfig': {'thinkingBudget': directives['thinking_budget']}})  # ty:ignore[no-matching-overload]
         logger.info(f'Job {job.index_number}: Making the content generation request to Google AI model {model}')
         model_version = model  # default
         with httpx.Client(http2=h2 is not None) as http_client:
@@ -1806,7 +1806,7 @@ class AIGoogleDiffer(DifferBase):
         report_kind: ReportKind,
         _unfiltered_diff: dict[ReportKind, str] | None = None,
         tz: ZoneInfo | None = None,
-    ) -> dict[ReportKind, str]:
+    ) -> dict[ReportKind, str]:  # ty:ignore[invalid-method-override]
         logger.info(f'Job {self.job.index_number}: Running the {self.__kind__} differ from hooks.py')
         # warnings.warn(
         #     f'Job {self.job.index_number}: Using differ {self.__kind__}, which is BETA, may have bugs, and may '

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/filters/_html.py

@@ -529,12 +529,12 @@ class LxmlParser:
         if isinstance(element, etree._ElementUnicodeResult):
             parent = element.getparent()
             if (
-                (element.is_tail and parent.tail is None)
-                or (element.is_text and parent.text is None)
-                or (element.is_attribute and parent.attrib.get(element.attrname) is None)
+                (element.is_tail and parent.tail is None)  # ty:ignore[unresolved-attribute]
+                or (element.is_text and parent.text is None)  # ty:ignore[unresolved-attribute]
+                or (element.is_attribute and parent.attrib.get(element.attrname) is None)  # ty:ignore[unresolved-attribute]
             ):
                 return True
-            element = parent
+            element = parent  # ty:ignore[invalid-assignment]
         try:
             tree = element.getroottree()
             path = tree.getpath(element)

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/filters/_shell.py

@@ -60,7 +60,10 @@ def _pipe_filter(f_cls: FilterBase, data: str | bytes, subfilter: dict[str, Any]
             env=env,
         ).stdout
     except subprocess.CalledProcessError as e:
-        logger.error(f"The '{f_cls.__kind__}' filter returned error ({f_cls.job.get_indexed_location()}):\n{e.stderr}")
+        logger.error(
+            f"The '{f_cls.__kind__}' filter returned error code {e.returncode} ({f_cls.job.get_indexed_location()}):\n"
+            f'{e.stderr}\n---\n{e.stdout}'
+        )
         raise e
     except FileNotFoundError as e:
         logger.error(f"The '{f_cls.__kind__}' filter returned error ({f_cls.job.get_indexed_location()}):\n{e}")

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/handler.py

@@ -65,7 +65,7 @@ ErrorData = TypedDict('ErrorData', {'type': str, 'message': str}, total=False)
 class JobState(ContextManager):
     """The JobState class, which contains run information about a job."""
 
-    _http_client_used: str | None = None
+    _http_client_used: Literal['httpx', 'requests', 'curl_cffi', 'playwright'] | None = None
     error_ignored: bool
     exception: Exception | None = None
     generated_diff: dict[ReportKind, str]

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/jobs/_base.py

@@ -78,7 +78,7 @@ class JobBase(metaclass=TrackSubClasses):
     # __required__ in derived classes
     url: str = ''
     command: str = ''
-    use_browser: bool | None = False
+    use_browser: bool | str | None = False
 
     # __optional__ in derived classes
     _delay: float | None = None  # TODO: WIP experiment
@@ -98,9 +98,11 @@ class JobBase(metaclass=TrackSubClasses):
     encoding: str | None = None  # UrlJobBase
     evaluate: str | None = None  # BrowserJob
     filters: FiltersList | list[FiltersList | dict[str, Any]] | None = None
+    fingerprints: dict[str, str | dict[str, Any]] | None = None  # UrlJob (curl_cffi backend only)
     guid: str = ''
     headers = Headers(encoding='utf-8')  # UrlJobBase
-    http_client: Literal['httpx', 'requests'] | None = None  # UrlJob
+    http_client: Literal['httpx', 'requests', 'curl_cffi'] | None = None  # UrlJob
+    http_version: Literal['v1', 'v2', 'v2tls', 'v2_prior_knowledge', 'v3', 'v3only'] | None = None  # UrlJob
     http_credentials: str | None = None  # BrowserJob
     ignore_cached: bool | None = None  # UrlJobBase
     ignore_connection_errors: bool | None = None  # UrlJobBase
@@ -110,9 +112,10 @@ class JobBase(metaclass=TrackSubClasses):
     ignore_https_errors: bool | None = None  # BrowserJob
     ignore_timeout_errors: bool | None = None  # UrlJobBase
     ignore_too_many_redirects: bool | None = None  # UrlJobBase
+    impersonate: str | None = None  # UrlJob (curl_cffi backend only)
     init_script: str | None = None  # BrowserJob
     initialization_js: str | None = None  # BrowserJob
-    initialization_url: str | None = None  # BrowserJob
+    initialization_url: str | None = None  # UrlJob, BrowserJob
     is_markdown: bool | None = None
     kind: str | None = None  # hooks.py
     loop: asyncio.AbstractEventLoop | None = None

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/jobs/_browser.py

@@ -406,12 +406,27 @@ class BrowserJob(UrlJobBase):
         with ExitStack() as stack:
             p = stack.enter_context(sync_playwright())
             executable_path = os.getenv('WEBCHANGES_BROWSER_PATH')
-            channel = None if executable_path else 'chrome'
-            browser_name = executable_path or 'Chrome'
+            value = self.use_browser if isinstance(self.use_browser, str) else 'chrome'
+            if value.startswith(('chrome', 'msedge')):
+                browser_type = p.chromium
+                channel = None if executable_path else value
+            elif value == 'firefox':
+                browser_type = p.firefox
+                channel = None
+            elif value == 'webkit':
+                browser_type = p.webkit
+                channel = None
+            else:
+                raise ValueError(
+                    f"Job {job_state.job.index_number}: Directive 'use_browser' value {value!r} must be "
+                    f"'firefox', 'webkit', or start with 'chrome' or 'msedge' "
+                    f'( {self.get_indexed_location()} ).'
+                )
+            browser_name = executable_path or value
             no_viewport = False if not self.switches else any('--window-size' in switch for switch in self.switches)
             if not self.user_data_dir:
                 browser = stack.enter_context(
-                    p.chromium.launch(
+                    browser_type.launch(
                         executable_path=executable_path,
                         channel=channel,
                         args=args,
@@ -422,11 +437,14 @@ class BrowserJob(UrlJobBase):
                     )
                 )
                 browser_version = browser.version
-                user_agent = headers.pop(
-                    'User-Agent',
-                    f'Mozilla/5.0 ({self.get_user_agent_platform()}) AppleWebKit/537.36 (KHTML, like Gecko) '
-                    f'Chrome/{browser_version.split(".", maxsplit=1)[0]}.0.0.0 Safari/537.36',
-                )
+                if browser_type is p.chromium:
+                    default_user_agent = (
+                        f'Mozilla/5.0 ({self.get_user_agent_platform()}) AppleWebKit/537.36 (KHTML, like Gecko) '
+                        f'Chrome/{browser_version.split(".", maxsplit=1)[0]}.0.0.0 Safari/537.36'
+                    )
+                    user_agent = headers.pop('User-Agent', default_user_agent)
+                else:
+                    user_agent = headers.pop('User-Agent', None)
                 context = stack.enter_context(
                     browser.new_context(
                         no_viewport=no_viewport,
@@ -445,7 +463,7 @@ class BrowserJob(UrlJobBase):
                 user_agent = headers.pop('User-Agent', '')
 
                 context = stack.enter_context(
-                    p.chromium.launch_persistent_context(
+                    browser_type.launch_persistent_context(
                         user_data_dir=self.user_data_dir,
                         channel=channel,
                         executable_path=executable_path,
@@ -607,10 +625,10 @@ class BrowserJob(UrlJobBase):
             try:
                 if return_data is not None:
                     logger.info(f"Job {self.index_number}: Using the 'return_data' Callable")
-                    return return_data(page, url, self.wait_until, self.referer)
+                    return return_data(page, url, self.wait_until, self.referer)  # ty:ignore[invalid-argument-type]
                 if response_handler is not None:
                     logger.info(f"Job {self.index_number}: Using the 'response_handler' Callable")
-                    response = response_handler(page, url, self.wait_until, self.referer)
+                    response = response_handler(page, url, self.wait_until, self.referer)  # ty:ignore[invalid-argument-type]
                 else:
                     response = page.goto(url, wait_until=self.wait_until, referer=self.referer)
 

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/jobs/_url.py

@@ -13,8 +13,8 @@ import sys
 import time
 from ftplib import FTP
 from pathlib import Path
-from typing import TYPE_CHECKING, Mapping, Sequence
-from urllib.parse import urlencode, urlparse, urlsplit
+from typing import TYPE_CHECKING, Any, Mapping, Sequence
+from urllib.parse import parse_qsl, urlencode, urlparse, urlsplit
 
 import html2text
 
@@ -45,8 +45,24 @@ except ImportError as e:  # pragma: no cover
     requests = str(e)  # ty:ignore[invalid-assignment]
     urllib3 = str(e)  # ty:ignore[invalid-assignment]
 
+try:
+    from curl_cffi import requests as curl_cffi_requests
+    from curl_cffi.requests import exceptions as curl_cffi_exceptions
+except ImportError as e:  # pragma: no cover
+    curl_cffi_requests = str(e)  # ty:ignore[invalid-assignment]
+    curl_cffi_exceptions = str(e)  # ty:ignore[invalid-assignment]
+
 logger = logging.getLogger(__name__)
 
+# HTTP-version directive mappings.
+# httpx supports only HTTP/1.1 and HTTP/2; curl_cffi (via libcurl) supports the full set.
+_HTTPX_HTTP_VERSIONS: dict[str, tuple[bool, bool]] = {
+    # value -> (http1, http2)
+    'v1': (True, False),
+    'v2': (True, True),
+}
+_CURL_CFFI_HTTP_VERSIONS: tuple[str, ...] = ('v1', 'v2', 'v2tls', 'v2_prior_knowledge', 'v3', 'v3only')
+
 
 class UrlJob(UrlJobBase):
     """Retrieve a URL from a web server."""
@@ -55,8 +71,12 @@ class UrlJob(UrlJobBase):
 
     __optional__: tuple[str, ...] = (
         'empty_as_transient',
+        'fingerprints',
         'http_client',
+        'http_version',
         'ignore_dh_key_too_small',
+        'impersonate',
+        'initialization_url',
         'no_redirects',
         'retries',
         'ssl_no_verify',
@@ -77,6 +97,26 @@ class UrlJob(UrlJobBase):
         self.url = location
         self.guid = self.get_guid()
 
+    def _resolve_initialization_url(self, final_url: str) -> str:
+        """Resolves the main URL by substituting parameters extracted from the initialization URL's final redirect.
+
+        Mirrors the behavior of ``initialization_url`` in BrowserJob: parses the URL params (;-separated) from the
+        final redirected URL and substitutes them into ``self.url`` via str.format().
+
+        :param final_url: The URL after the initialization request completed (after redirects).
+        :returns: The resolved URL to use for the main request.
+        """
+        init_url_params = dict(parse_qsl(urlparse(final_url).params))
+        try:
+            url = self.url.format(**init_url_params)
+        except KeyError as e:
+            raise ValueError(
+                f"Job {self.index_number}: Directive 'initialization_url' did not find key {e} to substitute in 'url'."
+            ) from None
+        if url != self.url:
+            logger.info(f'Job {self.index_number}: URL updated to {url}')
+        return url
+
     def _retrieve_httpx(
         self,
         headers: (
@@ -89,19 +129,43 @@ class UrlJob(UrlJobBase):
         :return: The data retrieved and the ETag.
         :raises NotModifiedError: If an HTTP 304 response is received.
         """
-        http2: bool = h2 is not None
-        if http2:
-            logger.info(f'Job {self.index_number}: Using the HTTPX HTTP client library with HTTP/2 support')
-        else:
+        http1: bool = True
+        if self.http_version is not None:
+            if self.http_version not in _HTTPX_HTTP_VERSIONS:
+                raise ValueError(
+                    f"Job {self.index_number}: 'http_version: {self.http_version}' is not supported by the httpx "
+                    f"HTTP client; httpx only supports 'v1' and 'v2'. Use 'http_client: curl_cffi' for other "
+                    f'HTTP versions ( {self.get_indexed_location()} ).'
+                )
+            http1, http2 = _HTTPX_HTTP_VERSIONS[self.http_version]
+            if http2 and h2 is None:
+                raise ImportError(
+                    f"Job {self.index_number}: 'http_version: v2' requires the 'h2' package, which is not installed. "
+                    f"Please install it using e.g. 'pip install h2' ( {self.get_indexed_location()} )."
+                )
             logger.info(
-                f'Job {self.index_number}: Using the HTTPX HTTP client library (HTTP/2 support is not available since '
-                f'h2 is not installed)'
+                f'Job {self.index_number}: Using the HTTPX HTTP client library with http_version={self.http_version}'
             )
+        else:
+            http2: bool = h2 is not None
+            if http2:
+                logger.info(f'Job {self.index_number}: Using the HTTPX HTTP client library with HTTP/2 support')
+            else:
+                logger.info(
+                    f'Job {self.index_number}: Using the HTTPX HTTP client library (HTTP/2 support is not available '
+                    f'since h2 is not installed)'
+                )
 
         proxy = self.get_proxy()
         if proxy is not None:
             logger.debug(f'Job {self.index_number}: Proxy: {proxy}')
 
+        if self.fingerprints:
+            logger.warning(
+                f"Job {self.index_number}: 'fingerprints' is only supported with 'http_client: curl_cffi'; "
+                f'ignoring the directive.'
+            )
+
         if self.ignore_dh_key_too_small:
             logger.debug(
                 'Setting default cipher list to ciphers that do not make any use of Diffie Hellman Key Exchange '
@@ -116,15 +180,22 @@ class UrlJob(UrlJobBase):
             headers=headers,
             cookies=self.cookies,
             verify=context,
+            http1=http1,
             http2=http2,
             proxy=proxy,
             timeout=timeout,
             follow_redirects=(not self.no_redirects),
         ) as http_client:
+            url = self.url
+            if self.initialization_url:
+                logger.info(f'Job {self.index_number}: Initializing by navigating to {self.initialization_url}')
+                init_response = http_client.get(self.initialization_url)
+                url = self._resolve_initialization_url(str(init_response.url))
+
             try:
                 response = http_client.request(
                     method=self.method,  # ty:ignore[invalid-argument-type]
-                    url=self.url,
+                    url=url,
                     data=self.data,  # ty:ignore[invalid-argument-type]
                     params=self.params,
                 )
@@ -148,15 +219,19 @@ class UrlJob(UrlJobBase):
                     parsed_json = json.loads(response.text)
                     error_message = json.dumps(parsed_json, ensure_ascii=False, separators=(',', ':'))
                 except json.JSONDecodeError:
-                    html_text = (
-                        response.text.split('<title', maxsplit=1)[0] + response.text.split('</title>', maxsplit=1)[-1]
-                    )
-                    parser = html2text.HTML2Text()
-                    parser.unicode_snob = True
-                    parser.body_width = 0
-                    parser.single_line_break = True
-                    parser.ignore_images = True
-                    error_message = parser.handle(html_text).strip()
+                    if '<title' in response.text:
+                        html_text = (
+                            response.text.split('<title', maxsplit=1)[0]
+                            + response.text.split('</title>', maxsplit=1)[-1]
+                        )
+                        parser = html2text.HTML2Text()
+                        parser.unicode_snob = True
+                        parser.body_width = 0
+                        parser.single_line_break = True
+                        parser.ignore_images = True
+                        error_message = parser.handle(html_text).strip()
+                    else:
+                        error_message = response.text
                 http_error_msg += f'\n{error_message}'
 
             if response.status_code in (429, 500, 502, 503, 504):
@@ -223,6 +298,12 @@ class UrlJob(UrlJobBase):
             # required to suppress warnings with 'ssl_no_verify: true'
             urllib3.disable_warnings(urllib3.exceptions.InsecureRequestWarning)
 
+        if self.fingerprints:
+            logger.warning(
+                f"Job {self.index_number}: 'fingerprints' is only supported with 'http_client: curl_cffi'; "
+                f'ignoring the directive.'
+            )
+
         if self.ignore_dh_key_too_small:
             # https://stackoverflow.com/questions/38015537
             logger.debug(
@@ -238,18 +319,29 @@ class UrlJob(UrlJobBase):
                 )
                 logger.error('See https://github.com/psf/requests/issues/6443')
 
-        response = requests.request(
-            method=self.method,  # ty:ignore[invalid-argument-type]
-            url=self.url,
-            params=self.params,
-            data=self.data,
-            headers=headers,
-            cookies=self.cookies,
-            timeout=timeout,
-            allow_redirects=(not self.no_redirects),
-            proxies=proxies,
-            verify=(not self.ssl_no_verify),
-        )
+        with requests.Session() as session:
+            if headers:
+                session.headers.update(headers)
+            if self.cookies:
+                session.cookies.update(self.cookies)
+            if proxies:
+                session.proxies.update(proxies)
+            session.verify = not self.ssl_no_verify
+
+            url = self.url
+            if self.initialization_url:
+                logger.info(f'Job {self.index_number}: Initializing by navigating to {self.initialization_url}')
+                init_response = session.get(self.initialization_url, timeout=timeout)
+                url = self._resolve_initialization_url(str(init_response.url))
+
+            response = session.request(
+                method=self.method,  # ty:ignore[invalid-argument-type]
+                url=url,
+                params=self.params,
+                data=self.data,
+                timeout=timeout,
+                allow_redirects=(not self.no_redirects),
+            )
 
         if 400 <= response.status_code < 600:
             # Custom version of request.raise_for_status() to include returned text.
@@ -274,15 +366,19 @@ class UrlJob(UrlJobBase):
                     parsed_json = json.loads(response.text)
                     error_message = json.dumps(parsed_json, ensure_ascii=False, separators=(',', ':'))
                 except json.JSONDecodeError:
-                    html_text = (
-                        response.text.split('<title', maxsplit=1)[0] + response.text.split('</title>', maxsplit=1)[-1]
-                    )
-                    parser = html2text.HTML2Text()
-                    parser.unicode_snob = True
-                    parser.body_width = 0
-                    parser.single_line_break = True
-                    parser.ignore_images = True
-                    error_message = parser.handle(html_text).strip()
+                    if '<title' in response.text:
+                        html_text = (
+                            response.text.split('<title', maxsplit=1)[0]
+                            + response.text.split('</title>', maxsplit=1)[-1]
+                        )
+                        parser = html2text.HTML2Text()
+                        parser.unicode_snob = True
+                        parser.body_width = 0
+                        parser.single_line_break = True
+                        parser.ignore_images = True
+                        error_message = parser.handle(html_text).strip()
+                    else:
+                        error_message = response.text
                 http_error_msg += f'\n{error_message}'
 
             if response.status_code in (429, 500, 502, 503, 504):
@@ -337,8 +433,164 @@ class UrlJob(UrlJobBase):
 
         return data, etag, mime_type
 
-    def retrieve(self, job_state: JobState, headless: bool = True) -> tuple[str | bytes, str, str]:
-        """Runs job to retrieve the data, and returns data and ETag.
+    def _retrieve_curl_cffi(
+        self, headers: Mapping[str, str | bytes] | None, timeout: float | None
+    ) -> tuple[str | bytes, str, str]:
+        """Retrieves the data, Etag, and media type using the curl_cffi library (browser TLS/JA3 impersonation).
+
+        :return: The data retrieved, the ETag and media type.
+        :raises NotModifiedError: If an HTTP 304 response is received.
+        """
+        impersonate = self.impersonate or 'chrome'
+        logger.info(
+            f'Job {self.index_number}: Using the curl_cffi HTTP client library '
+            f'(browser TLS impersonation: {impersonate})'
+        )
+        proxy_str = self.get_proxy()
+        if proxy_str is not None:
+            scheme = urlsplit(self.url).scheme
+            proxies: dict[str, str] | None = {scheme: proxy_str}
+            logger.debug(f'Job {self.index_number}: Proxies: {proxies}')
+        else:
+            proxies = None
+
+        if self.ignore_dh_key_too_small:
+            logger.warning(
+                f'Job {self.index_number}: ignore_dh_key_too_small is not supported with the curl_cffi HTTP client '
+                f'library (TLS ciphers are fixed by the impersonated browser profile); ignoring the directive.'
+            )
+
+        session_kwargs: dict[str, Any] = {
+            'headers': headers,
+            'cookies': self.cookies,
+            'proxies': proxies,
+            'verify': (not self.ssl_no_verify),
+            'timeout': timeout,
+            'impersonate': impersonate,
+        }
+        if self.http_version is not None:
+            if self.http_version not in _CURL_CFFI_HTTP_VERSIONS:
+                raise ValueError(
+                    f"Job {self.index_number}: 'http_version: {self.http_version}' is not a valid value; expected one "
+                    f'of {_CURL_CFFI_HTTP_VERSIONS} ( {self.get_indexed_location()} ).'
+                )
+            session_kwargs['http_version'] = self.http_version
+            logger.info(f'Job {self.index_number}: curl_cffi http_version={self.http_version}')
+
+        if self.fingerprints:
+            allowed_fp_keys = {'ja3', 'akamai', 'extra_fp'}
+            unknown_fp_keys = set(self.fingerprints) - allowed_fp_keys
+            if unknown_fp_keys:
+                raise ValueError(
+                    f"Job {self.index_number}: 'fingerprints' contains unknown key(s) {sorted(unknown_fp_keys)}; "
+                    f'expected any of {sorted(allowed_fp_keys)} ( {self.get_indexed_location()} ).'
+                )
+            for key in allowed_fp_keys:
+                if key in self.fingerprints:
+                    session_kwargs[key] = self.fingerprints[key]
+            logger.info(f'Job {self.index_number}: curl_cffi fingerprints applied: {sorted(self.fingerprints)}')
+
+        with curl_cffi_requests.Session(**session_kwargs) as session:
+            url = self.url
+            if self.initialization_url:
+                logger.info(f'Job {self.index_number}: Initializing by navigating to {self.initialization_url}')
+                init_response = session.get(self.initialization_url)
+                url = self._resolve_initialization_url(str(init_response.url))
+
+            response = session.request(
+                method=self.method,
+                url=url,
+                params=self.params,
+                data=self.data,
+                allow_redirects=(not self.no_redirects),
+            )
+
+        if 400 <= response.status_code < 600:
+            # Custom version of request.raise_for_status() to include returned text.
+            if isinstance(response.reason, bytes):
+                try:
+                    reason = response.reason.decode()
+                except UnicodeDecodeError:
+                    reason = response.reason.decode('iso-8859-1')
+            else:
+                reason = response.reason
+
+            if response.status_code < 500:
+                http_error_msg = f'{response.status_code} Client Error: {reason} for url: {response.url}'
+            else:
+                http_error_msg = f'{response.status_code} Server Error: {reason} for url: {response.url}'
+
+            if response.status_code != 404:
+                try:
+                    parsed_json = json.loads(response.text)
+                    error_message = json.dumps(parsed_json, ensure_ascii=False, separators=(',', ':'))
+                except json.JSONDecodeError:
+                    if '<title' in response.text:
+                        html_text = (
+                            response.text.split('<title', maxsplit=1)[0]
+                            + response.text.split('</title>', maxsplit=1)[-1]
+                        )
+                        parser = html2text.HTML2Text()
+                        parser.unicode_snob = True
+                        parser.body_width = 0
+                        parser.single_line_break = True
+                        parser.ignore_images = True
+                        error_message = parser.handle(html_text).strip()
+                    else:
+                        error_message = response.text
+                http_error_msg += f'\n{error_message}'
+
+            if response.status_code in (429, 500, 502, 503, 504):
+                logger.debug(f'Job {self.index_number}: Response error is transient.')
+                raise TransientHTTPError(http_error_msg, status_code=response.status_code)
+
+            raise curl_cffi_exceptions.HTTPError(http_error_msg, response=response)
+
+        if response.status_code == 304:
+            logger.debug(f'Job {self.index_number}: Intercepted response with {response.status_code} status')
+            raise NotModifiedError(response.status_code)
+
+        # Save ETag from response to be used as If-None-Match header in future requests
+        if not response.history:  # no redirects
+            etag = response.headers.get('ETag', '')
+        else:
+            logger.info(f'Job {self.index_number}: ETag not captured as response was redirected to {response.url}')
+            etag = ''
+        # Save the media type (fka MIME type)
+        mime_type = response.headers.get('Content-Type', '').split(';')[0]
+
+        if FilterBase.filter_chain_needs_bytes(self.filters):  # ty:ignore[invalid-argument-type]
+            return response.content, etag, mime_type
+
+        if self.encoding:
+            response.encoding = self.encoding
+        # Note: curl_cffi's Response has no apparent_encoding, so we can't replicate the requests-library
+        # ISO-8859-1 RFC 2616 override here. curl_cffi's encoding detection is libcurl-driven and reliable enough
+        # in practice.
+
+        is_redirect = 300 <= response.status_code < 400 and 'Location' in response.headers
+        if self.no_redirects and is_redirect:
+            new_location = response.headers['Location']
+            if mime_type == 'text/plain':
+                data = f'Redirect {response.status_code} {response.reason} to {new_location}:\n{response.text}'
+            elif mime_type == 'text/html':
+                data = (
+                    f'Redirect <b>{response.status_code} {response.reason}</b> to <a href="{new_location}">'
+                    f'{new_location}</a>:<br>{response.text}'
+                )
+            elif mime_type == 'application/json':
+                data = f'{{"Redirect {response.status_code} {response.reason}":"{new_location}"}}'
+            else:
+                data = f'Redirect {response.status_code} {response.reason} to {new_location}.'
+        else:
+            data = response.text
+
+        return data, etag, mime_type
+
+    def retrieve(  # noqa: C901 mccabe complexity too high
+        self, job_state: JobState, headless: bool = True
+    ) -> tuple[str | bytes, str, str]:
+        """Runs job to retrieve the data, and returns data, ETag and media type.
 
         :param job_state: The JobState object, to keep track of the state of the retrieval.
         :param headless: For browser-based jobs, whether headless mode should be used.
@@ -434,7 +686,19 @@ class UrlJob(UrlJobBase):
         logger.debug(f'Job {self.index_number}: Headers: {headers}')
         logger.debug(f'Job {self.index_number}: Cookies: {self.cookies}')
 
-        if self.http_client == 'requests' or not httpx:
+        if self.http_client == 'curl_cffi' or (not self.http_client and not httpx and isinstance(requests, str)):
+            if isinstance(curl_cffi_requests, str):
+                message = f'Job {job_state.job.index_number} cannot be run '
+                if self.http_client == 'curl_cffi':
+                    message += "with 'http_client: curl_cffi' "
+                message += (
+                    f'( {self.get_indexed_location()} ):\n{curl_cffi_requests}\n'
+                    f"Please install module using e.g. 'pip install --upgrade webchanges[curl_cffi]'."
+                )
+                raise ImportError(message)
+            job_state._http_client_used = 'curl_cffi'
+            data, etag, mime_type = self._retrieve_curl_cffi(headers=headers, timeout=timeout)
+        elif self.http_client == 'requests' or not httpx:
             if isinstance(requests, str):
                 message = f'Job {job_state.job.index_number} cannot be run '
                 if self.http_client == 'requests':
@@ -444,6 +708,12 @@ class UrlJob(UrlJobBase):
                     f"Please install module using e.g. 'pip install --upgrade webchanges[requests]'."
                 )
                 raise ImportError(message)
+            if self.http_version is not None:
+                raise ValueError(
+                    f"Job {job_state.job.index_number}: 'http_version' is not supported by the 'requests' HTTP "
+                    f"client (HTTP/1.1 only). Use 'http_client: httpx' or 'http_client: curl_cffi' "
+                    f'( {self.get_indexed_location()} ).'
+                )
             job_state._http_client_used = 'requests'
             data, etag, mime_type = self._retrieve_requests(headers=headers, timeout=timeout)
         elif not self.http_client or self.http_client == 'httpx':
@@ -456,7 +726,7 @@ class UrlJob(UrlJobBase):
                     f"Please install module using e.g. 'pip install --upgrade httpx[http2,zstd]'."
                 )
                 raise ImportError(message)
-            job_state._http_client_used = 'HTTPX'
+            job_state._http_client_used = 'httpx'
             data, etag, mime_type = self._retrieve_httpx(headers=headers, timeout=timeout)
         else:
             raise ValueError(
@@ -495,9 +765,16 @@ class UrlJob(UrlJobBase):
         :returns: A string to display and/or use in reports.
         """
         if (
-            httpx
-            and isinstance(exception, (httpx.HTTPError, httpx.InvalidURL, httpx.CookieConflict, httpx.StreamError))
-        ) or (not isinstance(requests, str) and isinstance(exception, requests.exceptions.RequestException)):
+            (
+                httpx
+                and isinstance(exception, (httpx.HTTPError, httpx.InvalidURL, httpx.CookieConflict, httpx.StreamError))
+            )
+            or (not isinstance(requests, str) and isinstance(exception, requests.exceptions.RequestException))
+            or (
+                not isinstance(curl_cffi_exceptions, str)
+                and isinstance(exception, curl_cffi_exceptions.RequestException)
+            )
+        ):
             # Instead of a full traceback, just show the error
             exception_str = str(exception).strip()
             if self.proxy and (
@@ -572,4 +849,24 @@ class UrlJob(UrlJobBase):
                 if isinstance(exception, TransientHTTPError):
                     return self._ignore_http_error_code(exception.status_code)
 
+        elif not isinstance(curl_cffi_exceptions, str) and isinstance(
+            exception, (curl_cffi_exceptions.RequestException, TransientHTTPError)
+        ):
+            if self.ignore_timeout_errors and (
+                isinstance(exception, curl_cffi_exceptions.Timeout)
+                or (isinstance(exception, TransientHTTPError) and exception.status_code == 504)
+            ):
+                return True
+            if self.ignore_connection_errors and (
+                isinstance(exception, (curl_cffi_exceptions.ConnectionError, TransientHTTPError))
+            ):
+                return True
+            if self.ignore_too_many_redirects and isinstance(exception, curl_cffi_exceptions.TooManyRedirects):
+                return True
+            if self.ignore_http_error_codes:
+                if isinstance(exception, curl_cffi_exceptions.HTTPError) and exception.response is not None:
+                    return self._ignore_http_error_code(exception.response.status_code)
+                if isinstance(exception, TransientHTTPError):
+                    return self._ignore_http_error_code(exception.status_code)
+
         return False

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges/reporters/_base.py

@@ -145,7 +145,7 @@ class ReporterBase(metaclass=TrackSubClasses):
         else:
             config = {}
 
-        return othercls(self.report, config, self.job_states, self.duration, self.jobs_files, self.differ_defaults)
+        return othercls(self.report, config, self.job_states, self.duration, self.jobs_files, self.differ_defaults)  # ty:ignore[invalid-argument-type]
 
     @classmethod
     def get_base_config(cls, report: Report) -> dict[str, Any]:
@@ -236,7 +236,7 @@ class ReporterBase(metaclass=TrackSubClasses):
 
         any_enabled = False
         for name, subclass in cls.__subclasses__.items():
-            cfg: _ConfigReportersList = report.config['report'].get(name, {})
+            cfg: _ConfigReportersList = report.config['report'].get(name, {})  # ty:ignore[invalid-assignment]
             if cfg.get('enabled', False):
                 any_enabled = True
                 logger.info(f'Submitting with {name} ({subclass})')

{webchanges-3.34.2 → webchanges-3.35.0/webchanges.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: webchanges
-Version: 3.34.2
+Version: 3.35.0
 Summary: Web Changes Delivered. AI-Summarized. Totally Anonymous.
 Author-email: Mike Borsetti <mike+webchanges@borsetti.com>
 Maintainer-email: Mike Borsetti <mike+webchanges@borsetti.com>
@@ -89,6 +89,8 @@ Provides-Extra: pushover
 Requires-Dist: chump; extra == "pushover"
 Provides-Extra: xmpp
 Requires-Dist: aioxmpp; extra == "xmpp"
+Provides-Extra: curl-cffi
+Requires-Dist: curl_cffi; extra == "curl-cffi"
 Provides-Extra: redis
 Requires-Dist: redis; extra == "redis"
 Provides-Extra: requests
@@ -96,7 +98,7 @@ Requires-Dist: requests; extra == "requests"
 Provides-Extra: safe-password
 Requires-Dist: keyring; extra == "safe-password"
 Provides-Extra: all
-Requires-Dist: webchanges[beautify,bs4,deepdiff_xml,html5lib,ical2text,imagediff,jq,matrix,ocr,pdf2text,pushbullet,pushover,pypdf_crypto,redis,requests,safe_password,use_browser,xmpp]; extra == "all"
+Requires-Dist: webchanges[beautify,bs4,curl_cffi,deepdiff_xml,html5lib,ical2text,imagediff,jq,matrix,ocr,pdf2text,pushbullet,pushover,pypdf_crypto,redis,requests,safe_password,use_browser,xmpp]; extra == "all"
 Dynamic: license-file
 
 .. role:: underline
@@ -139,7 +141,7 @@ Install **webchanges**  with ``uv`` (recommended):
 .. code-block:: bash
 
    uv pip install webchanges
-   
+
 or with ``pip``:
 
 .. code-block:: bash
@@ -154,13 +156,14 @@ implementation (no browser) `here <https://github.com/yubiuser/webchanges-docker
 
 As a GitHub Action
 ------------------
-**webchanges** can easily run as a `GitHub Action <https://www.docker.com/>`__! You will find an implementation 
+**webchanges** can easily run as a `GitHub Action <https://www.docker.com/>`__! You will find an implementation
 `here <https://github.com/swimmwatch/webchanges-action>`__.
 
 
 Documentation |readthedocs|
 ===========================
-The documentation is hosted on `Read the Docs <https://webchanges.readthedocs.io/>`__.
+The documentation is hosted on `Read the Docs <https://webchanges.readthedocs.io/>`__.  A (very good) AI-generated
+high-level architectural overview is available on `DeepWiki <https://deepwiki.com/mborsetti/webchanges>`__.
 
 
 Quick Start
@@ -208,7 +211,7 @@ Schedule
 
 Code
 ====
-|coveralls| |snyk| |issues| |code_style| |OpenSSF_Scorecard|
+|coveralls| |issues| |code_style| |OpenSSF_Scorecard|
 
 The code, issues tracker, and discussions are hosted on `GitHub <https://github.com/mborsetti/webchanges>`__.
 
@@ -240,7 +243,7 @@ Compatibility with **urlwatch**
 This project is based on code from `urlwatch 2.21
 <https://github.com/thp/urlwatch/tree/346b25914b0418342ffe2fb0529bed702fddc01f>`__ dated 30 July 2020.
 For upgrading from **urlwatch** to **webchanges** (using the same job and
-configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upgrading.html>`__. 
+configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upgrading.html>`__.
 
 
 .. |support| image:: https://img.shields.io/pypi/pyversions/webchanges.svg
@@ -285,9 +288,9 @@ configuration files), see `here <https://webchanges.readthedocs.io/en/stable/upg
 .. .. |security| image:: https://img.shields.io/badge/security-bandit-green.svg
 ..     :target: https://github.com/PyCQA/bandit
 ..     :alt: Security Status
-.. |snyk| image:: https://snyk.io/advisor/python/holidays/badge.svg
-    :target: https://snyk.io/advisor/python/holidays
-    :alt: Snyk Package Health Score
+.. .. |snyk| image:: https://snyk.io/advisor/python/webchanges/badge.svg
+..     :target: https://security.snyk.io/package/pip/webchanges
+..     :alt: Snyk Package Health Score
 .. |OpenSSF_Scorecard| image:: https://api.scorecard.dev/projects/github.com/mborsetti/webchanges/badge
     :target: https://scorecard.dev/viewer/?uri=github.com/mborsetti/webchanges
     :alt: OpenSSF Scoreard

{webchanges-3.34.2 → webchanges-3.35.0}/webchanges.egg-info/requires.txt

@@ -16,7 +16,7 @@ colorama
 tzdata
 
 [all]
-webchanges[beautify,bs4,deepdiff_xml,html5lib,ical2text,imagediff,jq,matrix,ocr,pdf2text,pushbullet,pushover,pypdf_crypto,redis,requests,safe_password,use_browser,xmpp]
+webchanges[beautify,bs4,curl_cffi,deepdiff_xml,html5lib,ical2text,imagediff,jq,matrix,ocr,pdf2text,pushbullet,pushover,pypdf_crypto,redis,requests,safe_password,use_browser,xmpp]
 
 [beautify]
 beautifulsoup4
@@ -26,6 +26,9 @@ cssbeautifier
 [bs4]
 beautifulsoup4
 
+[curl_cffi]
+curl_cffi
+
 [deepdiff]
 deepdiff