flaresolverr-cli 0.2.2__tar.gz → 0.3.0__tar.gz

{flaresolverr_cli-0.2.2 → flaresolverr_cli-0.3.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: flaresolverr-cli
-Version: 0.2.2
+Version: 0.3.0
 Summary: A requests.Session that proxies through a FlareSolverr instance.
 Home-page: https://github.com/Xavier-Lam/FlareSolverrSession
 Author: Xavier-Lam
@@ -50,7 +50,9 @@ Dynamic: summary
 
 A [`requests.Session`](https://docs.python-requests.org/) that transparently routes all HTTP requests through a [FlareSolverr](https://github.com/FlareSolverr/FlareSolverr) instance, allowing you to bypass Cloudflare protection with a familiar Python API.
 
-The project ships with a RPC client for direct access to the FlareSolverr JSON API, and a command-line interface (CLI) for quick requests and session management.
+The package also provides a more powerful [Adapter](#adapter) to handle complex requests if the `Session` is not sufficient.
+
+The project ships with a command-line interface (CLI) for requests and session management, and an RPC client for direct access to the FlareSolverr JSON API.
 
 This project is not responsible for solving challenges itself, it only forwards requests to *FlareSolverr*. If *FlareSolverr* fails to solve a challenge, it will raise an exception. Any issues related to challenge solving should be reported to the *FlareSolverr* project.
 
@@ -98,7 +100,7 @@ session = Session(
 ```
 
 #### Response Object
-A `FlareSolverr` object is attached to the `response` as `response.flaresolverr`. It contains metadata about the request and challenge solving process returned by *FlareSolverr*.
+A `FlareSolverr` metadata object is attached to the `response` as `response.flaresolverr`. It contains details about the request and the challenge solving process returned by *FlareSolverr*.
 
 | Attribute | Description |
 |---|---|
@@ -109,26 +111,36 @@ A `FlareSolverr` object is attached to the `response` as `response.flaresolverr`
 | `flaresolverr.version` | FlareSolverr server version |
 
 #### Exception Handling
-All exceptions defined in the module based on `FlareSolverrError`, which inherits from `requests.RequestException`. The inheritance hierarchy is as follows:
+If `FlareSolverr` returns an error response, the session will raise a `FlareSolverrResponseError` exception.
+
+All exceptions defined in the module are based on `FlareSolverrError`, which inherits from `requests.RequestException`. The hierarchy is as follows:
 
     requests.RequestException
     └── FlareSolverrError
         ├── FlareSolverrResponseError
-        │   ├── FlareSolverrChallengeError
+        │   └── FlareSolverrChallengeError
         └── FlareSolverrUnsupportedMethodError
 
+Exception Details:
 
 | Exception | Description |
 |---|---|
 | `FlareSolverrResponseError` | FlareSolverr returned an error response. The response dict is available as `response_data` attribute. |
-| `FlareSolverrChallengeError` | Challenge solving failed. |
+| `FlareSolverrChallengeError` | Challenge solving failed, raised only in `Session`. |
 | `FlareSolverrUnsupportedMethodError` | Unsupported HTTP method or content type. |
 
+#### Limitations
+
+- **Only GET and  `application/x-www-form-urlencoded` POST** are supported. Otherwise, it will raise `FlareSolverrUnsupportedMethodError`. 
+- **Headers returned by FlareSolverr may be empty** for some sites, depending on the FlareSolverr version and configuration. An empty HTTP status will be treated as `200`. See [FlareSolverr#1162](https://github.com/FlareSolverr/FlareSolverr/issues/1162).
+
+> If you need more control over the requests or want to use unsupported methods/content types, consider using the [Adapter](#adapter) instead.
+
 ### Command-Line Interface
 
 After installation, you can use the `flaresolverr-cli` command. It is a convenient CLI tool to send HTTP requests through FlareSolverr and manage sessions.
 
-It will output json response from FlareSolverr. If the FlareSolverr URL is not provided via `-f`, it will use the `FLARESOLVERR_URL` environment variable (defaulting to `http://localhost:8191/v1`).
+It prints the json response from FlareSolverr. If the FlareSolverr URL is not provided via `-f`, it will use the `FLARESOLVERR_URL` environment variable (defaulting to `http://localhost:8191/v1`).
 
 #### Sending requests
 
@@ -163,9 +175,45 @@ flaresolverr-cli session destroy my-session
 flaresolverr-cli session clear
 ```
 
+### Adapter
+If your requests are more complex than standard `GET` or form `POST`, the module provides an adapter to retrieve Cloudflare challenge solutions from *FlareSolverr* and apply them to your requests without modifying your existing codebase.
+
+```python
+import requests
+from flaresolverr_session import Adapter
+
+adapter = Adapter("http://localhost:8191/v1")
+
+session = requests.Session()
+session.mount("https://nowsecure.nl", adapter)
+
+response = session.get("https://protected-site.com/page")
+print(response.text)
+```
+
+It is recommended only mount the adapter to specific origins that require Cloudflare bypass. Read the [caveats section](#caveats) before using it.
+
+> Don't use the `Session` provided by `flaresolverr_session` here.
+
+#### Caveats
+
+* The *FlareSolverr* instance and the machine running the adapter **must share the same public IP** (or use the same proxy with a consistent public IP). Otherwise the cookies obtained from *FlareSolverr* will not be accepted by Cloudflare.
+* The proxy used for the original request is automatically applied to the *FlareSolverr* request for the reason mentioned above.
+* The adapter automatically sends a `GET` request to the original URL to solve the challenge. You can provide a custom `challenge_url` to override this behavior.
+* Cloudflare cookies are tied to the `User-Agent` used during challenge solving. The adapter automatically sets the `User-Agent` returned by FlareSolverr.
+* The adapter is less reliable than using the [Session](#basic-usage) directly.
+
+#### How It Works
+
+1. The adapter first attempts to send the request normally through its base adapter.
+2. If it detects a Cloudflare challenge, the adapter forwards the URL to a FlareSolverr instance.
+3. FlareSolverr solves the challenge and returns cookies and a `User-Agent`.
+4. The adapter retries the original request using the returned credentials.
+
+
 ### RPC Tool
 
-The `flaresolverr_rpc` module provides a programmatic interface to the FlareSolverr JSON API, useful when you need low-level access to the raw API responses.
+The `flaresolverr_rpc` module provides a programmatic interface to the FlareSolverr JSON API, ideal for low-level access to raw API responses.
 
 ```python
 from flaresolverr_session import RPC
@@ -192,9 +240,3 @@ with RPC("http://localhost:8191/v1") as rpc:
 ```
 
 All methods return the raw JSON response dict from FlareSolverr.
-
-
-## Limitations
-
-- **Only GET and  `application/x-www-form-urlencoded` POST** are supported. Otherwise, it will raise `FlareSolverrUnsupportedMethodError`. 
-- **Headers returned by FlareSolverr** may be empty for some sites, depending on the FlareSolverr version and configuration. Empty HTTP status will be regarded as `200`. See [FlareSolverr#1162](https://github.com/FlareSolverr/FlareSolverr/issues/1162).

{flaresolverr_cli-0.2.2 → flaresolverr_cli-0.3.0}/README.md

@@ -6,7 +6,9 @@
 
 A [`requests.Session`](https://docs.python-requests.org/) that transparently routes all HTTP requests through a [FlareSolverr](https://github.com/FlareSolverr/FlareSolverr) instance, allowing you to bypass Cloudflare protection with a familiar Python API.
 
-The project ships with a RPC client for direct access to the FlareSolverr JSON API, and a command-line interface (CLI) for quick requests and session management.
+The package also provides a more powerful [Adapter](#adapter) to handle complex requests if the `Session` is not sufficient.
+
+The project ships with a command-line interface (CLI) for requests and session management, and an RPC client for direct access to the FlareSolverr JSON API.
 
 This project is not responsible for solving challenges itself, it only forwards requests to *FlareSolverr*. If *FlareSolverr* fails to solve a challenge, it will raise an exception. Any issues related to challenge solving should be reported to the *FlareSolverr* project.
 
@@ -54,7 +56,7 @@ session = Session(
 ```
 
 #### Response Object
-A `FlareSolverr` object is attached to the `response` as `response.flaresolverr`. It contains metadata about the request and challenge solving process returned by *FlareSolverr*.
+A `FlareSolverr` metadata object is attached to the `response` as `response.flaresolverr`. It contains details about the request and the challenge solving process returned by *FlareSolverr*.
 
 | Attribute | Description |
 |---|---|
@@ -65,26 +67,36 @@ A `FlareSolverr` object is attached to the `response` as `response.flaresolverr`
 | `flaresolverr.version` | FlareSolverr server version |
 
 #### Exception Handling
-All exceptions defined in the module based on `FlareSolverrError`, which inherits from `requests.RequestException`. The inheritance hierarchy is as follows:
+If `FlareSolverr` returns an error response, the session will raise a `FlareSolverrResponseError` exception.
+
+All exceptions defined in the module are based on `FlareSolverrError`, which inherits from `requests.RequestException`. The hierarchy is as follows:
 
     requests.RequestException
     └── FlareSolverrError
         ├── FlareSolverrResponseError
-        │   ├── FlareSolverrChallengeError
+        │   └── FlareSolverrChallengeError
         └── FlareSolverrUnsupportedMethodError
 
+Exception Details:
 
 | Exception | Description |
 |---|---|
 | `FlareSolverrResponseError` | FlareSolverr returned an error response. The response dict is available as `response_data` attribute. |
-| `FlareSolverrChallengeError` | Challenge solving failed. |
+| `FlareSolverrChallengeError` | Challenge solving failed, raised only in `Session`. |
 | `FlareSolverrUnsupportedMethodError` | Unsupported HTTP method or content type. |
 
+#### Limitations
+
+- **Only GET and  `application/x-www-form-urlencoded` POST** are supported. Otherwise, it will raise `FlareSolverrUnsupportedMethodError`. 
+- **Headers returned by FlareSolverr may be empty** for some sites, depending on the FlareSolverr version and configuration. An empty HTTP status will be treated as `200`. See [FlareSolverr#1162](https://github.com/FlareSolverr/FlareSolverr/issues/1162).
+
+> If you need more control over the requests or want to use unsupported methods/content types, consider using the [Adapter](#adapter) instead.
+
 ### Command-Line Interface
 
 After installation, you can use the `flaresolverr-cli` command. It is a convenient CLI tool to send HTTP requests through FlareSolverr and manage sessions.
 
-It will output json response from FlareSolverr. If the FlareSolverr URL is not provided via `-f`, it will use the `FLARESOLVERR_URL` environment variable (defaulting to `http://localhost:8191/v1`).
+It prints the json response from FlareSolverr. If the FlareSolverr URL is not provided via `-f`, it will use the `FLARESOLVERR_URL` environment variable (defaulting to `http://localhost:8191/v1`).
 
 #### Sending requests
 
@@ -119,9 +131,45 @@ flaresolverr-cli session destroy my-session
 flaresolverr-cli session clear
 ```
 
+### Adapter
+If your requests are more complex than standard `GET` or form `POST`, the module provides an adapter to retrieve Cloudflare challenge solutions from *FlareSolverr* and apply them to your requests without modifying your existing codebase.
+
+```python
+import requests
+from flaresolverr_session import Adapter
+
+adapter = Adapter("http://localhost:8191/v1")
+
+session = requests.Session()
+session.mount("https://nowsecure.nl", adapter)
+
+response = session.get("https://protected-site.com/page")
+print(response.text)
+```
+
+It is recommended only mount the adapter to specific origins that require Cloudflare bypass. Read the [caveats section](#caveats) before using it.
+
+> Don't use the `Session` provided by `flaresolverr_session` here.
+
+#### Caveats
+
+* The *FlareSolverr* instance and the machine running the adapter **must share the same public IP** (or use the same proxy with a consistent public IP). Otherwise the cookies obtained from *FlareSolverr* will not be accepted by Cloudflare.
+* The proxy used for the original request is automatically applied to the *FlareSolverr* request for the reason mentioned above.
+* The adapter automatically sends a `GET` request to the original URL to solve the challenge. You can provide a custom `challenge_url` to override this behavior.
+* Cloudflare cookies are tied to the `User-Agent` used during challenge solving. The adapter automatically sets the `User-Agent` returned by FlareSolverr.
+* The adapter is less reliable than using the [Session](#basic-usage) directly.
+
+#### How It Works
+
+1. The adapter first attempts to send the request normally through its base adapter.
+2. If it detects a Cloudflare challenge, the adapter forwards the URL to a FlareSolverr instance.
+3. FlareSolverr solves the challenge and returns cookies and a `User-Agent`.
+4. The adapter retries the original request using the returned credentials.
+
+
 ### RPC Tool
 
-The `flaresolverr_rpc` module provides a programmatic interface to the FlareSolverr JSON API, useful when you need low-level access to the raw API responses.
+The `flaresolverr_rpc` module provides a programmatic interface to the FlareSolverr JSON API, ideal for low-level access to raw API responses.
 
 ```python
 from flaresolverr_session import RPC
@@ -148,9 +196,3 @@ with RPC("http://localhost:8191/v1") as rpc:
 ```
 
 All methods return the raw JSON response dict from FlareSolverr.
-
-
-## Limitations
-
-- **Only GET and  `application/x-www-form-urlencoded` POST** are supported. Otherwise, it will raise `FlareSolverrUnsupportedMethodError`. 
-- **Headers returned by FlareSolverr** may be empty for some sites, depending on the FlareSolverr version and configuration. Empty HTTP status will be regarded as `200`. See [FlareSolverr#1162](https://github.com/FlareSolverr/FlareSolverr/issues/1162).

{flaresolverr_cli-0.2.2 → flaresolverr_cli-0.3.0}/flaresolverr_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: flaresolverr-cli
-Version: 0.2.2
+Version: 0.3.0
 Summary: A requests.Session that proxies through a FlareSolverr instance.
 Home-page: https://github.com/Xavier-Lam/FlareSolverrSession
 Author: Xavier-Lam
@@ -50,7 +50,9 @@ Dynamic: summary
 
 A [`requests.Session`](https://docs.python-requests.org/) that transparently routes all HTTP requests through a [FlareSolverr](https://github.com/FlareSolverr/FlareSolverr) instance, allowing you to bypass Cloudflare protection with a familiar Python API.
 
-The project ships with a RPC client for direct access to the FlareSolverr JSON API, and a command-line interface (CLI) for quick requests and session management.
+The package also provides a more powerful [Adapter](#adapter) to handle complex requests if the `Session` is not sufficient.
+
+The project ships with a command-line interface (CLI) for requests and session management, and an RPC client for direct access to the FlareSolverr JSON API.
 
 This project is not responsible for solving challenges itself, it only forwards requests to *FlareSolverr*. If *FlareSolverr* fails to solve a challenge, it will raise an exception. Any issues related to challenge solving should be reported to the *FlareSolverr* project.
 
@@ -98,7 +100,7 @@ session = Session(
 ```
 
 #### Response Object
-A `FlareSolverr` object is attached to the `response` as `response.flaresolverr`. It contains metadata about the request and challenge solving process returned by *FlareSolverr*.
+A `FlareSolverr` metadata object is attached to the `response` as `response.flaresolverr`. It contains details about the request and the challenge solving process returned by *FlareSolverr*.
 
 | Attribute | Description |
 |---|---|
@@ -109,26 +111,36 @@ A `FlareSolverr` object is attached to the `response` as `response.flaresolverr`
 | `flaresolverr.version` | FlareSolverr server version |
 
 #### Exception Handling
-All exceptions defined in the module based on `FlareSolverrError`, which inherits from `requests.RequestException`. The inheritance hierarchy is as follows:
+If `FlareSolverr` returns an error response, the session will raise a `FlareSolverrResponseError` exception.
+
+All exceptions defined in the module are based on `FlareSolverrError`, which inherits from `requests.RequestException`. The hierarchy is as follows:
 
     requests.RequestException
     └── FlareSolverrError
         ├── FlareSolverrResponseError
-        │   ├── FlareSolverrChallengeError
+        │   └── FlareSolverrChallengeError
         └── FlareSolverrUnsupportedMethodError
 
+Exception Details:
 
 | Exception | Description |
 |---|---|
 | `FlareSolverrResponseError` | FlareSolverr returned an error response. The response dict is available as `response_data` attribute. |
-| `FlareSolverrChallengeError` | Challenge solving failed. |
+| `FlareSolverrChallengeError` | Challenge solving failed, raised only in `Session`. |
 | `FlareSolverrUnsupportedMethodError` | Unsupported HTTP method or content type. |
 
+#### Limitations
+
+- **Only GET and  `application/x-www-form-urlencoded` POST** are supported. Otherwise, it will raise `FlareSolverrUnsupportedMethodError`. 
+- **Headers returned by FlareSolverr may be empty** for some sites, depending on the FlareSolverr version and configuration. An empty HTTP status will be treated as `200`. See [FlareSolverr#1162](https://github.com/FlareSolverr/FlareSolverr/issues/1162).
+
+> If you need more control over the requests or want to use unsupported methods/content types, consider using the [Adapter](#adapter) instead.
+
 ### Command-Line Interface
 
 After installation, you can use the `flaresolverr-cli` command. It is a convenient CLI tool to send HTTP requests through FlareSolverr and manage sessions.
 
-It will output json response from FlareSolverr. If the FlareSolverr URL is not provided via `-f`, it will use the `FLARESOLVERR_URL` environment variable (defaulting to `http://localhost:8191/v1`).
+It prints the json response from FlareSolverr. If the FlareSolverr URL is not provided via `-f`, it will use the `FLARESOLVERR_URL` environment variable (defaulting to `http://localhost:8191/v1`).
 
 #### Sending requests
 
@@ -163,9 +175,45 @@ flaresolverr-cli session destroy my-session
 flaresolverr-cli session clear
 ```
 
+### Adapter
+If your requests are more complex than standard `GET` or form `POST`, the module provides an adapter to retrieve Cloudflare challenge solutions from *FlareSolverr* and apply them to your requests without modifying your existing codebase.
+
+```python
+import requests
+from flaresolverr_session import Adapter
+
+adapter = Adapter("http://localhost:8191/v1")
+
+session = requests.Session()
+session.mount("https://nowsecure.nl", adapter)
+
+response = session.get("https://protected-site.com/page")
+print(response.text)
+```
+
+It is recommended only mount the adapter to specific origins that require Cloudflare bypass. Read the [caveats section](#caveats) before using it.
+
+> Don't use the `Session` provided by `flaresolverr_session` here.
+
+#### Caveats
+
+* The *FlareSolverr* instance and the machine running the adapter **must share the same public IP** (or use the same proxy with a consistent public IP). Otherwise the cookies obtained from *FlareSolverr* will not be accepted by Cloudflare.
+* The proxy used for the original request is automatically applied to the *FlareSolverr* request for the reason mentioned above.
+* The adapter automatically sends a `GET` request to the original URL to solve the challenge. You can provide a custom `challenge_url` to override this behavior.
+* Cloudflare cookies are tied to the `User-Agent` used during challenge solving. The adapter automatically sets the `User-Agent` returned by FlareSolverr.
+* The adapter is less reliable than using the [Session](#basic-usage) directly.
+
+#### How It Works
+
+1. The adapter first attempts to send the request normally through its base adapter.
+2. If it detects a Cloudflare challenge, the adapter forwards the URL to a FlareSolverr instance.
+3. FlareSolverr solves the challenge and returns cookies and a `User-Agent`.
+4. The adapter retries the original request using the returned credentials.
+
+
 ### RPC Tool
 
-The `flaresolverr_rpc` module provides a programmatic interface to the FlareSolverr JSON API, useful when you need low-level access to the raw API responses.
+The `flaresolverr_rpc` module provides a programmatic interface to the FlareSolverr JSON API, ideal for low-level access to raw API responses.
 
 ```python
 from flaresolverr_session import RPC
@@ -192,9 +240,3 @@ with RPC("http://localhost:8191/v1") as rpc:
 ```
 
 All methods return the raw JSON response dict from FlareSolverr.
-
-
-## Limitations
-
-- **Only GET and  `application/x-www-form-urlencoded` POST** are supported. Otherwise, it will raise `FlareSolverrUnsupportedMethodError`. 
-- **Headers returned by FlareSolverr** may be empty for some sites, depending on the FlareSolverr version and configuration. Empty HTTP status will be regarded as `200`. See [FlareSolverr#1162](https://github.com/FlareSolverr/FlareSolverr/issues/1162).

{flaresolverr_cli-0.2.2 → flaresolverr_cli-0.3.0}/flaresolverr_cli.egg-info/SOURCES.txt

@@ -8,11 +8,15 @@ flaresolverr_cli.egg-info/entry_points.txt
 flaresolverr_cli.egg-info/requires.txt
 flaresolverr_cli.egg-info/top_level.txt
 flaresolverr_session/__init__.py
+flaresolverr_session/adapter.py
 flaresolverr_session/cli.py
+flaresolverr_session/detection.py
 flaresolverr_session/exceptions.py
 flaresolverr_session/rpc.py
 flaresolverr_session/session.py
+tests/test_adapter.py
+tests/test_challenge.py
 tests/test_cli.py
+tests/test_detection.py
 tests/test_rpc.py
-tests/test_session.py
-tests/testconf.py
+tests/test_session.py

{flaresolverr_cli-0.2.2 → flaresolverr_cli-0.3.0}/flaresolverr_session/__init__.py

@@ -1,5 +1,7 @@
 # -*- coding: utf-8 -*-
 
+from flaresolverr_session.adapter import Adapter
+from flaresolverr_session.detection import is_cloudflare_challenge
 from flaresolverr_session.exceptions import (
     FlareSolverrError,
     FlareSolverrResponseError,
@@ -12,11 +14,13 @@ from flaresolverr_session.session import Session, Response
 __title__ = "flaresolverr-session"
 __description__ = "A requests.Session that proxies through a FlareSolverr instance."
 __url__ = "https://github.com/Xavier-Lam/FlareSolverrSession"
-__version__ = "0.2.2"
+__version__ = "0.3.0"
 __author__ = "Xavier-Lam"
 __author_email__ = "xavierlam7@hotmail.com"
 
 __all__ = [
+    "Adapter",
+    "is_cloudflare_challenge",
     "Session",
     "Response",
     "RPC",

flaresolverr_cli-0.3.0/flaresolverr_session/adapter.py

@@ -0,0 +1,168 @@
+# -*- coding: utf-8 -*-
+
+import warnings
+
+try:
+    from urlparse import urlparse, urlunparse
+except ImportError:
+    from urllib.parse import urlparse, urlunparse
+
+from requests.adapters import BaseAdapter, HTTPAdapter
+from requests.cookies import RequestsCookieJar
+from requests.utils import select_proxy
+
+from flaresolverr_session.detection import is_cloudflare_challenge
+from flaresolverr_session.rpc import RPC
+
+
+class Adapter(BaseAdapter):
+    """A ``requests`` transport adapter that retries requests blocked by
+    Cloudflare by solving challenges through FlareSolverr.
+
+    Normal (non-challenged) requests are forwarded directly through the
+    *base_adapter*.  When a Cloudflare challenge is detected the adapter
+    automatically:
+
+    1. Sends the *challenge_url* (or the blocked URL) to FlareSolverr.
+    2. Caches the ``cf_clearance`` cookie and User-Agent per site.
+    3. Retries the original request with the obtained cookie and UA.
+
+    Parameters:
+        flaresolverr_url (str or None): The FlareSolverr API endpoint.
+            Ignored when *rpc* is provided.
+        rpc (flaresolverr_session.rpc.RPC or None):
+            A pre-configured :class:`~flaresolverr_session.rpc.RPC`
+            instance.  When provided, *flaresolverr_url* is ignored.
+        challenge_url (str or None): URL to use for solving challenges.
+            When *None*, the adapter sends the blocked URL itself for
+            challenge solving.  Can be an absolute path without domain
+            (e.g. ``"/"``) — it will be combined with the domain of
+            the blocked URL.
+        base_adapter (requests.adapters.BaseAdapter or None): The
+            adapter used to perform actual HTTP requests.
+    """
+
+    def __init__(
+        self,
+        flaresolverr_url=None,
+        rpc=None,
+        challenge_url=None,
+        base_adapter=None,
+    ):
+        super(Adapter, self).__init__()
+
+        if rpc is not None and flaresolverr_url is not None:
+            warnings.warn(
+                "Both 'rpc' and 'flaresolverr_url' are provided. "
+                "The 'rpc' instance will be used and "
+                "'flaresolverr_url' will be ignored.",
+                stacklevel=2,
+            )
+        if not rpc:
+            rpc = RPC(flaresolverr_url)
+        self._rpc = rpc
+        self._challenge_url = challenge_url
+
+        if base_adapter is None:
+            base_adapter = HTTPAdapter()
+        self._base_adapter = base_adapter
+
+        # Per-site caches keyed by hostname.
+        self._cf_cookies = {}
+        self._user_agents = {}
+
+    def send(
+        self, request, stream=False, timeout=None, verify=True, cert=None, proxies=None
+    ):
+        kwargs = dict(
+            stream=stream,
+            timeout=timeout,
+            verify=verify,
+            cert=cert,
+            proxies=proxies,
+        )
+
+        self._prepare_request(request)
+        response = self._base_adapter.send(request, **kwargs)
+
+        if is_cloudflare_challenge(response):
+            self._solve_challenge(request.url, proxies=proxies)
+            self._prepare_request(request)
+            response = self._base_adapter.send(request, **kwargs)
+
+        return response
+
+    def _prepare_request(self, request):
+        parsed = urlparse(request.url)
+        domain = parsed.hostname
+        path = parsed.path or "/"
+
+        jar = self._cf_cookies.get(domain)
+        if jar:
+            # Merge cookies into the Cookie header.
+            existing = RequestsCookieJar()
+            # Parse existing Cookie header if present.
+            cookie_header = request.headers.get("Cookie")
+            if cookie_header:
+                for pair in cookie_header.split(";"):
+                    pair = pair.strip()
+                    if "=" in pair:
+                        name, value = pair.split("=", 1)
+                        existing.set(name.strip(), value.strip())
+            # Overlay cached CF cookies that match the request path.
+            for cookie in jar:
+                if path.startswith(cookie.path):
+                    existing.set(cookie.name, cookie.value)
+
+            # Re-build Cookie header.
+            cookie_str = "; ".join("%s=%s" % (c.name, c.value) for c in existing)
+            if cookie_str:
+                request.headers["Cookie"] = cookie_str
+
+        user_agent = self._user_agents.get(domain)
+        if user_agent:
+            request.headers["User-Agent"] = user_agent
+
+    def _solve_challenge(self, original_url, proxies=None):
+        challenge_url = self._get_challenge_url(original_url)
+        rpc_kwargs = {"url": challenge_url}
+
+        if proxies:
+            proxy_url = select_proxy(challenge_url, proxies)
+            if proxy_url:
+                rpc_kwargs["proxy"] = {"url": proxy_url}
+
+        data = self._rpc.request.get(**rpc_kwargs)
+        solution = data.get("solution", {})
+
+        domain = urlparse(original_url).hostname
+
+        user_agent = solution.get("userAgent")
+        if user_agent:
+            self._user_agents[domain] = user_agent
+
+        jar = RequestsCookieJar()
+        for cd in solution.get("cookies", []):
+            if cd.get("name") == "cf_clearance":
+                jar.set(
+                    cd["name"],
+                    cd.get("value", ""),
+                    domain=cd.get("domain"),
+                    path=cd.get("path", "/"),
+                    secure=cd.get("secure", False),
+                    expires=cd.get("expiry"),
+                )
+                break
+        self._cf_cookies[domain] = jar
+
+    def _get_challenge_url(self, original_url):
+        if self._challenge_url is None:
+            return original_url
+
+        if self._challenge_url.startswith("/"):
+            parsed = urlparse(original_url)
+            return urlunparse(
+                (parsed.scheme, parsed.netloc, self._challenge_url, "", "", "")
+            )
+
+        return self._challenge_url