camoufox 0.1.0b0__tar.gz

camoufox-0.1.0b0/PKG-INFO

@@ -0,0 +1,156 @@
+Metadata-Version: 2.1
+Name: camoufox
+Version: 0.1.0b0
+Summary: Wrapper around Playwright to help launch Camoufox
+Home-page: https://github.com/daijro/camoufox
+License: MIT
+Keywords: client,fingerprint,browser,scraping,injector,firefox,playwright
+Author: daijro
+Author-email: daijro.dev@gmail.com
+Requires-Python: >=3.8,<4.0
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: Internet :: WWW/HTTP
+Classifier: Topic :: Internet :: WWW/HTTP :: Browsers
+Classifier: Topic :: Software Development :: Libraries :: Python Modules
+Requires-Dist: browserforge
+Requires-Dist: click
+Requires-Dist: numpy
+Requires-Dist: orjson
+Requires-Dist: platformdirs
+Requires-Dist: playwright
+Requires-Dist: pyyaml
+Requires-Dist: requests
+Requires-Dist: tqdm
+Requires-Dist: typing_extensions
+Requires-Dist: ua_parser
+Project-URL: Repository, https://github.com/daijro/camoufox
+Description-Content-Type: text/markdown
+
+# Camoufox Python Interface
+
+#### This is the Python library for Camoufox. 
+
+> [!WARNING]
+> This is still experimental and in active development!
+
+## Installation
+
+First, install the `camoufox` package:
+
+```bash
+pip install -U camoufox
+```
+
+Then, download the Camoufox browser:
+
+```bash
+camoufox fetch
+```
+
+To uninstall, run `camoufox remove`.
+
+<hr width=50>
+
+## Usage
+
+Camoufox is fully compatible with your existing Playwright code. You only have to change your browser initialization:
+
+#### Sync API
+
+```python
+from camoufox.sync_api import Camoufox
+
+with Camoufox(headless=False) as browser:
+    page = browser.new_page()
+    page.goto("https://example.com/")
+```
+
+#### Async API
+
+```python
+from camoufox.async_api import AsyncCamoufox
+
+async with AsyncCamoufox(headless=False) as browser:
+    page = await browser.new_page()
+    await page.goto("https://example.com")
+```
+<details>
+<summary>Parameters</summary>
+
+```
+Launches a new browser instance for Camoufox.
+
+Parameters:
+    playwright (Playwright):
+        The playwright instance to use.
+    config (Optional[Dict[str, Any]]):
+        The configuration to use.
+    addons (Optional[List[str]]):
+        The addons to use.
+    fingerprint (Optional[Fingerprint]):
+        The fingerprint to use.
+    exclude_addons (Optional[List[DefaultAddons]]):
+        The default addons to exclude, passed as a list of camoufox.DefaultAddons enums.
+    screen (Optional[browserforge.fingerprints.Screen]):
+        The screen constraints to use.
+    os (Optional[ListOrString]):
+        The operating system to use for the fingerprint. Either a string or a list of strings.
+    user_agent (Optional[ListOrString]):
+        The user agent to use for the fingerprint. Either a string or a list of strings.
+    fonts (Optional[List[str]]):
+        The fonts to load into Camoufox, in addition to the default fonts.
+    args (Optional[List[str]]):
+        The arguments to pass to the browser.
+    executable_path (Optional[str]):
+        The path to the Camoufox browser executable.
+    **launch_options (Dict[str, Any]):
+        Additional Firefox launch options.
+```
+</details>
+
+---
+
+### Config
+
+Camoufox [config data](https://github.com/daijro/camoufox?tab=readme-ov-file#fingerprint-injection) can be passed as a dictionary to the `config` parameter:
+
+```python
+from camoufox import Camoufox
+
+with Camoufox(
+    config={
+        'webrtc:ipv4': '123.45.67.89',
+        'webrtc:ipv6': 'e791:d37a:88f6:48d1:2cad:2667:4582:1d6d',
+    }
+) as browser:
+    page = browser.new_page()
+    page.goto("https://www.browserscan.net/")
+```
+
+<hr width=50>
+
+### BrowserForge Integration
+
+Camoufox is fully compatible with BrowserForge.
+
+By default, Camoufox will use a random fingerprint. You can also inject your own Firefox Browserforge fingerprint into Camoufox with the following example:
+
+```python
+from camoufox.sync_api import Camoufox
+from browserforge.fingerprints import FingerprintGenerator
+
+fg = FingerprintGenerator(browser='firefox')
+
+# Launch Camoufox with a random Firefox fingerprint
+with Camoufox(fingerprint=fg.generate()) as browser:
+    page = browser.new_page()
+    page.goto("https://www.browserscan.net/")
+```
+
+---

camoufox-0.1.0b0/README.md

@@ -0,0 +1,122 @@
+# Camoufox Python Interface
+
+#### This is the Python library for Camoufox. 
+
+> [!WARNING]
+> This is still experimental and in active development!
+
+## Installation
+
+First, install the `camoufox` package:
+
+```bash
+pip install -U camoufox
+```
+
+Then, download the Camoufox browser:
+
+```bash
+camoufox fetch
+```
+
+To uninstall, run `camoufox remove`.
+
+<hr width=50>
+
+## Usage
+
+Camoufox is fully compatible with your existing Playwright code. You only have to change your browser initialization:
+
+#### Sync API
+
+```python
+from camoufox.sync_api import Camoufox
+
+with Camoufox(headless=False) as browser:
+    page = browser.new_page()
+    page.goto("https://example.com/")
+```
+
+#### Async API
+
+```python
+from camoufox.async_api import AsyncCamoufox
+
+async with AsyncCamoufox(headless=False) as browser:
+    page = await browser.new_page()
+    await page.goto("https://example.com")
+```
+<details>
+<summary>Parameters</summary>
+
+```
+Launches a new browser instance for Camoufox.
+
+Parameters:
+    playwright (Playwright):
+        The playwright instance to use.
+    config (Optional[Dict[str, Any]]):
+        The configuration to use.
+    addons (Optional[List[str]]):
+        The addons to use.
+    fingerprint (Optional[Fingerprint]):
+        The fingerprint to use.
+    exclude_addons (Optional[List[DefaultAddons]]):
+        The default addons to exclude, passed as a list of camoufox.DefaultAddons enums.
+    screen (Optional[browserforge.fingerprints.Screen]):
+        The screen constraints to use.
+    os (Optional[ListOrString]):
+        The operating system to use for the fingerprint. Either a string or a list of strings.
+    user_agent (Optional[ListOrString]):
+        The user agent to use for the fingerprint. Either a string or a list of strings.
+    fonts (Optional[List[str]]):
+        The fonts to load into Camoufox, in addition to the default fonts.
+    args (Optional[List[str]]):
+        The arguments to pass to the browser.
+    executable_path (Optional[str]):
+        The path to the Camoufox browser executable.
+    **launch_options (Dict[str, Any]):
+        Additional Firefox launch options.
+```
+</details>
+
+---
+
+### Config
+
+Camoufox [config data](https://github.com/daijro/camoufox?tab=readme-ov-file#fingerprint-injection) can be passed as a dictionary to the `config` parameter:
+
+```python
+from camoufox import Camoufox
+
+with Camoufox(
+    config={
+        'webrtc:ipv4': '123.45.67.89',
+        'webrtc:ipv6': 'e791:d37a:88f6:48d1:2cad:2667:4582:1d6d',
+    }
+) as browser:
+    page = browser.new_page()
+    page.goto("https://www.browserscan.net/")
+```
+
+<hr width=50>
+
+### BrowserForge Integration
+
+Camoufox is fully compatible with BrowserForge.
+
+By default, Camoufox will use a random fingerprint. You can also inject your own Firefox Browserforge fingerprint into Camoufox with the following example:
+
+```python
+from camoufox.sync_api import Camoufox
+from browserforge.fingerprints import FingerprintGenerator
+
+fg = FingerprintGenerator(browser='firefox')
+
+# Launch Camoufox with a random Firefox fingerprint
+with Camoufox(fingerprint=fg.generate()) as browser:
+    page = browser.new_page()
+    page.goto("https://www.browserscan.net/")
+```
+
+---

camoufox-0.1.0b0/camoufox/__init__.py

@@ -0,0 +1,5 @@
+from .addons import DefaultAddons
+from .async_api import AsyncCamoufox, AsyncNewBrowser
+from .sync_api import Camoufox, NewBrowser
+
+__all__ = ["Camoufox", "NewBrowser", "AsyncCamoufox", "AsyncNewBrowser", "DefaultAddons"]

camoufox-0.1.0b0/camoufox/__main__.py

@@ -0,0 +1,112 @@
+"""
+Binary CLI manager for Camoufox.
+
+Adapted from https://github.com/daijro/hrequests/blob/main/hrequests/__main__.py
+"""
+
+from importlib.metadata import PackageNotFoundError
+from importlib.metadata import version as pkg_version
+
+import click
+
+from .pkgman import CamoufoxFetcher, installed_verstr, rprint
+
+
+class CamoufoxUpdate(CamoufoxFetcher):
+    """
+    Checks & updates Camoufox
+    """
+
+    def __init__(self) -> None:
+        """
+        Initializes the CamoufoxUpdate class
+        """
+        super().__init__()
+        try:
+            self.current_verstr = installed_verstr()
+        except FileNotFoundError:
+            self.current_verstr = None
+
+    def is_updated_needed(self) -> None:
+        if self.current_verstr is None:
+            return True
+        # If the installed version is not the latest version
+        if self.current_verstr != self.verstr:
+            return True
+        return False
+
+    def update(self) -> None:
+        """
+        Updates the library if needed
+        """
+        # Check if the version is the same as the latest available version
+        if not self.is_updated_needed():
+            rprint("Camoufox binaries up to date!", fg="green")
+            rprint(f"Current version: v{self.current_verstr}", fg="green")
+            return
+
+        # Download updated file
+        if self.current_verstr is not None:
+            # Display an updating message
+            rprint(
+                f"Updating Camoufox binaries from v{self.current_verstr} => v{self.verstr}",
+                fg="yellow",
+            )
+        else:
+            rprint(f"Fetching Camoufox binaries v{self.verstr}...", fg="yellow")
+        # Install the new version
+        self.install()
+
+
+@click.group()
+def cli() -> None:
+    pass
+
+
+@cli.command(name='fetch')
+def fetch() -> None:
+    """
+    Fetch the latest version of Camoufox
+    """
+    CamoufoxUpdate().update()
+
+
+@cli.command(name='remove')
+def remove() -> None:
+    """
+    Remove all library files
+    """
+    if not CamoufoxUpdate().cleanup():
+        rprint("Camoufox binaries not found!", fg="red")
+
+
+@cli.command(name='version')
+def version() -> None:
+    """
+    Display the current version
+    """
+    # python package version
+    try:
+        rprint(f"Pip package:\tv{pkg_version('camoufox')}", fg="green")
+    except PackageNotFoundError:
+        rprint("Pip package:\tNot installed!", fg="red")
+
+    updater = CamoufoxUpdate()
+    bin_ver = updater.current_verstr
+
+    # If binaries are not downloaded
+    if not bin_ver:
+        rprint("Camoufox:\tNot downloaded!", fg="red")
+        return
+    # Print the base version
+    rprint(f"Camoufox:\tv{bin_ver} ", fg="green", nl=False)
+
+    # Check for library updates
+    if updater.is_updated_needed():
+        rprint(f"(Latest: v{updater.verstr})", fg="red")
+    else:
+        rprint("(Up to date!)", fg="yellow")
+
+
+if __name__ == '__main__':
+    cli()

camoufox-0.1.0b0/camoufox/addons.py

@@ -0,0 +1,184 @@
+import asyncio
+import os
+import socket
+import threading
+import time
+from enum import Enum
+from typing import List
+
+import orjson
+
+from .exceptions import InvalidAddonPath, InvalidDebugPort, MissingDebugPort
+
+
+class DefaultAddons(Enum):
+    """
+    Default addons to be downloaded
+    """
+
+    uBO = "https://addons.mozilla.org/firefox/downloads/latest/ublock-origin/latest.xpi"
+    BPC = "https://gitflic.ru/project/magnolia1234/bpc_uploads/blob/raw?file=bypass_paywalls_clean-latest.xpi"
+
+
+def get_debug_port(args: List[str]) -> int:
+    """
+    Gets the debug port from the args, or creates a new one if not provided
+    """
+    for i, arg in enumerate(args):
+        # Search for debugger server port
+        if arg == "-start-debugger-server":
+            # If arg is found but no port is provided, raise an error
+            if i + 1 >= len(args):
+                raise MissingDebugPort(f"No debug port provided: {args}")
+            debug_port = args[i + 1]
+            # Try to parse the debug port as an integer
+            try:
+                return int(debug_port)
+            except ValueError as e:
+                raise InvalidDebugPort(
+                    f"Error parsing debug port. Must be an integer: {debug_port}"
+                ) from e
+
+    # Create new debugger server port
+    debug_port_int = get_open_port()
+    # Add -start-debugger-server {debug_port} to args
+    args.extend(["-start-debugger-server", str(debug_port_int)])
+
+    return debug_port_int
+
+
+def confirm_paths(paths: List[str]) -> None:
+    """
+    Confirms that the addon paths are valid
+    """
+    for path in paths:
+        if not os.path.exists(path):
+            raise InvalidAddonPath(path)
+
+
+def get_open_port() -> int:
+    """
+    Gets an open port
+    """
+    with socket.socket(socket.AF_INET, socket.SOCK_STREAM) as s:
+        s.bind(('localhost', 0))
+        return s.getsockname()[1]
+
+
+def threaded_try_load_addons(debug_port_int: int, addons_list: List[str]) -> None:
+    """
+    Tries to load addons (in a separate thread)
+    """
+    thread = threading.Thread(
+        target=try_load_addons, args=(debug_port_int, addons_list), daemon=True
+    )
+    thread.start()
+
+
+def try_load_addons(debug_port_int: int, addons_list: List[str]) -> None:
+    """
+    Tries to load addons
+    """
+    # Wait for the server to be open
+    while True:
+        try:
+            with socket.create_connection(("localhost", debug_port_int)):
+                break
+        except ConnectionRefusedError:
+            time.sleep(0.05)
+
+    # Load addons
+    asyncio.run(load_all_addons(debug_port_int, addons_list))
+
+
+async def load_all_addons(debug_port_int: int, addons_list: List[str]) -> None:
+    """
+    Loads all addons
+    """
+    addon_loaders = [LoadFirefoxAddon(debug_port_int, addon) for addon in addons_list]
+    await asyncio.gather(*[addon_loader.load() for addon_loader in addon_loaders])
+
+
+class LoadFirefoxAddon:
+    '''
+    Firefox addon loader
+    https://github.com/daijro/hrequests/blob/main/hrequests/extensions.py#L95
+    '''
+
+    def __init__(self, port, addon_path):
+        self.port: int = port
+        self.addon_path: str = addon_path
+        self.success: bool = False
+        self.buffers: list = []
+        self.remaining_bytes: int = 0
+
+    async def load(self):
+        reader, writer = await asyncio.open_connection('localhost', self.port)
+        writer.write(self._format_message({"to": "root", "type": "getRoot"}))
+        await writer.drain()
+
+        while True:
+            data = await reader.read(100)  # Adjust buffer size as needed
+            if not data:
+                break
+            await self._process_data(writer, data)
+
+        writer.close()
+        await writer.wait_closed()
+        return self.success
+
+    async def _process_data(self, writer, data):
+        while data:
+            if self.remaining_bytes == 0:
+                index = data.find(b':')
+                if index == -1:
+                    self.buffers.append(data)
+                    return
+
+                total_data = b''.join(self.buffers) + data
+                size, _, remainder = total_data.partition(b':')
+
+                try:
+                    self.remaining_bytes = int(size)
+                except ValueError as e:
+                    raise ValueError("Invalid state") from e
+
+                data = remainder
+
+            if len(data) < self.remaining_bytes:
+                self.remaining_bytes -= len(data)
+                self.buffers.append(data)
+                return
+            else:
+                self.buffers.append(data[: self.remaining_bytes])
+                message = orjson.loads(b''.join(self.buffers))
+                self.buffers.clear()
+
+                await self._on_message(writer, message)
+
+                data = data[self.remaining_bytes :]
+                self.remaining_bytes = 0
+
+    async def _on_message(self, writer, message):
+        if "addonsActor" in message:
+            writer.write(
+                self._format_message(
+                    {
+                        "to": message["addonsActor"],
+                        "type": "installTemporaryAddon",
+                        "addonPath": self.addon_path,
+                    }
+                )
+            )
+            await writer.drain()
+
+        if "addon" in message:
+            self.success = True
+            writer.write_eof()
+
+        if "error" in message:
+            writer.write_eof()
+
+    def _format_message(self, data):
+        raw = orjson.dumps(data)
+        return f"{len(raw)}:".encode() + raw

camoufox-0.1.0b0/camoufox/async_api.py

@@ -0,0 +1,88 @@
+from typing import Any, Dict, List, Optional
+
+from browserforge.fingerprints import Fingerprint, Screen
+from playwright.async_api import Browser, Playwright, PlaywrightContextManager
+
+from .addons import DefaultAddons
+from .utils import ListOrString, get_launch_options
+
+
+class AsyncCamoufox(PlaywrightContextManager):
+    """
+    Wrapper around playwright.async_api.PlaywrightContextManager that automatically
+    launches a browser and closes it when the context manager is exited.
+    """
+
+    def __init__(self, **launch_options):
+        super().__init__()
+        self.launch_options = launch_options
+        self.browser: Optional[Browser] = None
+
+    async def __aenter__(self) -> Browser:
+        await super().__aenter__()
+        self.browser = await AsyncNewBrowser(self._playwright, **self.launch_options)
+        return self.browser
+
+    async def __aexit__(self, *args: Any):
+        if self.browser:
+            await self.browser.close()
+        await super().__aexit__(*args)
+
+
+async def AsyncNewBrowser(
+    playwright: Playwright,
+    *,
+    config: Optional[Dict[str, Any]] = None,
+    addons: Optional[List[str]] = None,
+    fingerprint: Optional[Fingerprint] = None,
+    exclude_addons: Optional[List[DefaultAddons]] = None,
+    screen: Optional[Screen] = None,
+    os: Optional[ListOrString] = None,
+    user_agent: Optional[ListOrString] = None,
+    fonts: Optional[List[str]] = None,
+    args: Optional[List[str]] = None,
+    executable_path: Optional[str] = None,
+    **launch_options: Dict[str, Any]
+) -> Browser:
+    """
+    Launches a new browser instance for Camoufox.
+
+    Parameters:
+        playwright (Playwright):
+            The playwright instance to use.
+        config (Optional[Dict[str, Any]]):
+            The configuration to use.
+        addons (Optional[List[str]]):
+            The addons to use.
+        fingerprint (Optional[Fingerprint]):
+            The fingerprint to use.
+        exclude_addons (Optional[List[DefaultAddons]]):
+            The default addons to exclude, passed as a list of camoufox.DefaultAddons enums.
+        screen (Optional[browserforge.fingerprints.Screen]):
+            The screen constraints to use.
+        os (Optional[ListOrString]):
+            The operating system to use for the fingerprint. Either a string or a list of strings.
+        user_agent (Optional[ListOrString]):
+            The user agent to use for the fingerprint. Either a string or a list of strings.
+        fonts (Optional[List[str]]):
+            The fonts to load into Camoufox, in addition to the default fonts.
+        args (Optional[List[str]]):
+            The arguments to pass to the browser.
+        executable_path (Optional[str]):
+            The path to the Camoufox browser executable.
+        **launch_options (Dict[str, Any]):
+            Additional Firefox launch options.
+    """
+    opt = get_launch_options(
+        config=config,
+        addons=addons,
+        fingerprint=fingerprint,
+        exclude_addons=exclude_addons,
+        screen=screen,
+        os=os,
+        user_agent=user_agent,
+        fonts=fonts,
+        args=args,
+        executable_path=executable_path,
+    )
+    return await playwright.firefox.launch(**opt, **launch_options)