pydoll-python 2.10.0__tar.gz → 2.12.0__tar.gz

{pydoll_python-2.10.0 → pydoll_python-2.12.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: pydoll-python
-Version: 2.10.0
+Version: 2.12.0
 Summary: Pydoll is a library for automating chromium-based browsers without a WebDriver, offering realistic interactions.
 License-File: LICENSE
 Author: Thalison Fernandes
@@ -12,7 +12,7 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
-Requires-Dist: aiofiles (>=23.2.1,<24.0.0)
+Requires-Dist: aiofiles (>=25.1.0,<26.0.0)
 Requires-Dist: aiohttp (>=3.9.5,<4.0.0)
 Requires-Dist: typing_extensions (>=4.14.0,<5.0.0)
 Requires-Dist: websockets (>=14,<15)
@@ -82,6 +82,88 @@ await tab.scroll.by(ScrollPosition.UP, 300, smooth=False)
 
 Unlike `execute_script("window.scrollBy(...)")` which returns immediately, the scroll API uses CDP's `awaitPromise` to wait for the browser's `scrollend` event, ensuring your next actions only execute after scrolling completely finishes. Perfect for taking screenshots, loading lazy content, or creating realistic reading patterns.
 
+### Keyboard API: Complete Control Over Keyboard Input
+
+The new `KeyboardAPI` provides a clean, centralized interface for all keyboard interactions at the page level:
+
+```python
+from pydoll.constants import Key
+
+# Press individual keys
+await tab.keyboard.press(Key.ENTER)
+await tab.keyboard.press(Key.TAB)
+
+# Use hotkeys/shortcuts with up to 3 keys  
+await tab.keyboard.hotkey(Key.CONTROL, Key.A)  # Select all (works!)
+await tab.keyboard.hotkey(Key.CONTROL, Key.C)  # Copy (works!)
+await tab.keyboard.hotkey(Key.CONTROL, Key.SHIFT, Key.ARROWRIGHT)  # Select word right
+
+# Manual control for complex sequences
+await tab.keyboard.down(Key.SHIFT)
+await tab.keyboard.press(Key.ARROWRIGHT)  # Select text while holding Shift
+await tab.keyboard.up(Key.SHIFT)
+```
+
+**Key improvements:**
+- **Centralized**: All keyboard operations accessible via `tab.keyboard`
+- **Smart modifier detection**: Hotkeys automatically detect and apply modifiers (Ctrl, Shift, Alt, Meta)
+- **Complete key support**: 26 letters (A-Z), 10 digits (0-9), all function keys, numpad, and special keys
+- **Page-level shortcuts**: Works for Ctrl+C, Ctrl+V, Ctrl+A, etc.
+
+> **⚠️ CDP Limitation:** Browser UI shortcuts (like Ctrl+T for new tab, F12 for DevTools) don't work via CDP. Use Pydoll's methods instead: `await browser.new_tab()`, `await tab.close()`.
+
+### Retry Decorator: Production-Ready Error Recovery
+
+Transform fragile scripts into robust production scrapers with the `@retry` decorator. Automatically recover from network failures, timeouts, and transient errors with exponential backoff and custom recovery strategies:
+
+```python
+import asyncio
+from pydoll.browser.chromium import Chrome
+from pydoll.decorators import retry
+from pydoll.exceptions import ElementNotFound, NetworkError
+
+class ProductScraper:
+    def __init__(self):
+        self.tab = None
+        self.retry_count = 0
+    
+    # Recovery callback executed before each retry
+    async def recover_from_failure(self):
+        self.retry_count += 1
+        print(f"Attempt {self.retry_count} failed. Recovering...")
+        
+        # Refresh page and restore state
+        if self.tab:
+            await self.tab.refresh()
+            await asyncio.sleep(2)
+    
+    @retry(
+        max_retries=3,
+        exceptions=[ElementNotFound, NetworkError],
+        on_retry=recover_from_failure,  # Execute recovery logic
+        delay=2.0,
+        exponential_backoff=True
+    )
+    async def scrape_product(self, url: str):
+        if not self.tab:
+            browser = Chrome()
+            self.tab = await browser.start()
+        
+        await self.tab.go_to(url)
+        title = await self.tab.find(class_name='product-title', timeout=5)
+        return await title.text
+```
+
+**Powerful features:**
+- **Smart retry logic**: Only retry on specific exceptions you define
+- **Exponential backoff**: Progressively increase wait times (1s → 2s → 4s → 8s)
+- **Recovery callbacks**: Execute custom logic between retries (refresh page, switch proxy, restart browser)
+- **Production-tested**: Handle the chaos of real-world scraping with confidence
+
+Perfect for handling rate limits, network instability, dynamic content loading, and CAPTCHA detection. Turn unreliable scrapers into bulletproof automation.
+
+[**📖 Full Documentation**](https://pydoll.tech/docs/features/advanced/decorators/)
+
 ## 📦 Installation
 
 ```bash

{pydoll_python-2.10.0 → pydoll_python-2.12.0}/README.md

@@ -62,6 +62,88 @@ await tab.scroll.by(ScrollPosition.UP, 300, smooth=False)
 
 Unlike `execute_script("window.scrollBy(...)")` which returns immediately, the scroll API uses CDP's `awaitPromise` to wait for the browser's `scrollend` event, ensuring your next actions only execute after scrolling completely finishes. Perfect for taking screenshots, loading lazy content, or creating realistic reading patterns.
 
+### Keyboard API: Complete Control Over Keyboard Input
+
+The new `KeyboardAPI` provides a clean, centralized interface for all keyboard interactions at the page level:
+
+```python
+from pydoll.constants import Key
+
+# Press individual keys
+await tab.keyboard.press(Key.ENTER)
+await tab.keyboard.press(Key.TAB)
+
+# Use hotkeys/shortcuts with up to 3 keys  
+await tab.keyboard.hotkey(Key.CONTROL, Key.A)  # Select all (works!)
+await tab.keyboard.hotkey(Key.CONTROL, Key.C)  # Copy (works!)
+await tab.keyboard.hotkey(Key.CONTROL, Key.SHIFT, Key.ARROWRIGHT)  # Select word right
+
+# Manual control for complex sequences
+await tab.keyboard.down(Key.SHIFT)
+await tab.keyboard.press(Key.ARROWRIGHT)  # Select text while holding Shift
+await tab.keyboard.up(Key.SHIFT)
+```
+
+**Key improvements:**
+- **Centralized**: All keyboard operations accessible via `tab.keyboard`
+- **Smart modifier detection**: Hotkeys automatically detect and apply modifiers (Ctrl, Shift, Alt, Meta)
+- **Complete key support**: 26 letters (A-Z), 10 digits (0-9), all function keys, numpad, and special keys
+- **Page-level shortcuts**: Works for Ctrl+C, Ctrl+V, Ctrl+A, etc.
+
+> **⚠️ CDP Limitation:** Browser UI shortcuts (like Ctrl+T for new tab, F12 for DevTools) don't work via CDP. Use Pydoll's methods instead: `await browser.new_tab()`, `await tab.close()`.
+
+### Retry Decorator: Production-Ready Error Recovery
+
+Transform fragile scripts into robust production scrapers with the `@retry` decorator. Automatically recover from network failures, timeouts, and transient errors with exponential backoff and custom recovery strategies:
+
+```python
+import asyncio
+from pydoll.browser.chromium import Chrome
+from pydoll.decorators import retry
+from pydoll.exceptions import ElementNotFound, NetworkError
+
+class ProductScraper:
+    def __init__(self):
+        self.tab = None
+        self.retry_count = 0
+    
+    # Recovery callback executed before each retry
+    async def recover_from_failure(self):
+        self.retry_count += 1
+        print(f"Attempt {self.retry_count} failed. Recovering...")
+        
+        # Refresh page and restore state
+        if self.tab:
+            await self.tab.refresh()
+            await asyncio.sleep(2)
+    
+    @retry(
+        max_retries=3,
+        exceptions=[ElementNotFound, NetworkError],
+        on_retry=recover_from_failure,  # Execute recovery logic
+        delay=2.0,
+        exponential_backoff=True
+    )
+    async def scrape_product(self, url: str):
+        if not self.tab:
+            browser = Chrome()
+            self.tab = await browser.start()
+        
+        await self.tab.go_to(url)
+        title = await self.tab.find(class_name='product-title', timeout=5)
+        return await title.text
+```
+
+**Powerful features:**
+- **Smart retry logic**: Only retry on specific exceptions you define
+- **Exponential backoff**: Progressively increase wait times (1s → 2s → 4s → 8s)
+- **Recovery callbacks**: Execute custom logic between retries (refresh page, switch proxy, restart browser)
+- **Production-tested**: Handle the chaos of real-world scraping with confidence
+
+Perfect for handling rate limits, network instability, dynamic content loading, and CAPTCHA detection. Turn unreliable scrapers into bulletproof automation.
+
+[**📖 Full Documentation**](https://pydoll.tech/docs/features/advanced/decorators/)
+
 ## 📦 Installation
 
 ```bash

{pydoll_python-2.10.0 → pydoll_python-2.12.0}/pydoll/browser/tab.py

@@ -4,6 +4,7 @@ import asyncio
 import base64 as _b64
 import logging
 import shutil
+import warnings
 from contextlib import asynccontextmanager
 from functools import partial
 from pathlib import Path
@@ -50,14 +51,20 @@ from pydoll.exceptions import (
     TopLevelTargetRequired,
     WaitElementTimeout,
 )
-from pydoll.interactions.scroll import ScrollAPI
+from pydoll.interactions import KeyboardAPI, ScrollAPI
 from pydoll.protocol.browser.types import DownloadBehavior, DownloadProgressState
 from pydoll.protocol.page.events import PageEvent
 from pydoll.protocol.page.types import ScreenshotFormat
+from pydoll.protocol.runtime.methods import (
+    CallFunctionOnResponse,
+    EvaluateResponse,
+    SerializationOptions,
+)
+from pydoll.protocol.runtime.types import CallArgument
+from pydoll.protocol.storage.methods import GetCookiesResponse
 from pydoll.utils import (
     decode_base64_to_bytes,
     has_return_outside_function,
-    is_script_already_function,
 )
 
 if TYPE_CHECKING:
@@ -133,6 +140,7 @@ class Tab(FindElementsMixin):
         self._cloudflare_captcha_callback_id: Optional[int] = None
         self._request: Optional[Request] = None
         self._scroll: Optional[ScrollAPI] = None
+        self._keyboard: Optional[KeyboardAPI] = None
         logger.debug(
             (
                 f'Tab initialized: target_id={self._target_id}, '
@@ -190,6 +198,18 @@ class Tab(FindElementsMixin):
             self._scroll = ScrollAPI(self)
         return self._scroll
 
+    @property
+    def keyboard(self) -> KeyboardAPI:
+        """
+        Get the keyboard API for controlling keyboard input at page level.
+
+        Returns:
+            KeyboardAPI: An instance of the KeyboardAPI class for keyboard operations.
+        """
+        if self._keyboard is None:
+            self._keyboard = KeyboardAPI(self)
+        return self._keyboard
+
     @property
     def intercept_file_chooser_dialog_enabled(self) -> bool:
         """Whether file chooser dialog interception is active."""
@@ -749,37 +769,174 @@ class Tab(FindElementsMixin):
         )
 
     @overload
-    async def execute_script(self, script: str) -> EvaluateResponse: ...
+    async def execute_script(
+        self,
+        script: str,
+        *,
+        object_group: Optional[str] = None,
+        include_command_line_api: Optional[bool] = None,
+        silent: Optional[bool] = None,
+        context_id: Optional[int] = None,
+        return_by_value: Optional[bool] = None,
+        generate_preview: Optional[bool] = None,
+        user_gesture: Optional[bool] = None,
+        await_promise: Optional[bool] = None,
+        throw_on_side_effect: Optional[bool] = None,
+        timeout: Optional[float] = None,
+        disable_breaks: Optional[bool] = None,
+        repl_mode: Optional[bool] = None,
+        allow_unsafe_eval_blocked_by_csp: Optional[bool] = None,
+        unique_context_id: Optional[str] = None,
+        serialization_options: Optional[SerializationOptions] = None,
+    ) -> EvaluateResponse: ...
 
     @overload
     async def execute_script(
-        self, script: str, element: 'WebElement'
+        self,
+        script: str,
+        element: WebElement,
+        *,
+        arguments: Optional[list[CallArgument]] = None,
+        silent: Optional[bool] = None,
+        return_by_value: Optional[bool] = None,
+        generate_preview: Optional[bool] = None,
+        user_gesture: Optional[bool] = None,
+        await_promise: Optional[bool] = None,
+        execution_context_id: Optional[int] = None,
+        object_group: Optional[str] = None,
+        throw_on_side_effect: Optional[bool] = None,
+        unique_context_id: Optional[str] = None,
+        serialization_options: Optional[SerializationOptions] = None,
     ) -> CallFunctionOnResponse: ...
 
     async def execute_script(
-        self, script: str, element: Optional['WebElement'] = None
+        self,
+        script: str,
+        element: Optional[WebElement] = None,
+        *,
+        arguments: Optional[list[CallArgument]] = None,
+        object_group: Optional[str] = None,
+        include_command_line_api: Optional[bool] = None,
+        silent: Optional[bool] = None,
+        context_id: Optional[int] = None,
+        return_by_value: Optional[bool] = None,
+        generate_preview: Optional[bool] = None,
+        user_gesture: Optional[bool] = None,
+        await_promise: Optional[bool] = None,
+        execution_context_id: Optional[int] = None,
+        throw_on_side_effect: Optional[bool] = None,
+        timeout: Optional[float] = None,
+        disable_breaks: Optional[bool] = None,
+        repl_mode: Optional[bool] = None,
+        allow_unsafe_eval_blocked_by_csp: Optional[bool] = None,
+        unique_context_id: Optional[str] = None,
+        serialization_options: Optional[SerializationOptions] = None,
     ) -> Union[EvaluateResponse, CallFunctionOnResponse]:
         """
         Execute JavaScript in page context.
 
         Args:
-            script: JavaScript code to execute.
-            element: Element context (use 'argument' in script to reference).
+            script (str): JavaScript code to execute.
+            element (Optional[WebElement]): Optional WebElement to execute script on.
+            arguments (Optional[list[CallArgument]]): Arguments to pass to the function.
+            object_group (Optional[str]): Symbolic group name for the result (Runtime.evaluate).
+            include_command_line_api (Optional[bool]): Whether to include command line API
+                (Runtime.evaluate).
+            silent (Optional[bool]): Whether to silence exceptions (Runtime.evaluate).
+            context_id (Optional[int]): ID of the execution context to evaluate in
+                (Runtime.evaluate).
+            return_by_value (Optional[bool]): Whether to return the result by value instead of
+                reference (Runtime.evaluate).
+            generate_preview (Optional[bool]): Whether to generate a preview for the result
+                (Runtime.evaluate).
+            user_gesture (Optional[bool]): Whether to treat evaluation as initiated by user
+                gesture (Runtime.evaluate).
+            await_promise (Optional[bool]): Whether to await promise result (Runtime.evaluate).
+            execution_context_id (Optional[int]): ID of the execution context to call the
+                function in.
+            throw_on_side_effect (Optional[bool]): Whether to throw if side effect cannot be
+                ruled out (Runtime.evaluate).
+            timeout (Optional[float]): Timeout in milliseconds (Runtime.evaluate).
+            disable_breaks (Optional[bool]): Whether to disable breakpoints during evaluation
+                (Runtime.evaluate).
+            repl_mode (Optional[bool]): Whether to execute in REPL mode (Runtime.evaluate).
+            allow_unsafe_eval_blocked_by_csp (Optional[bool]): Allow unsafe evaluation
+                (Runtime.evaluate).
+            unique_context_id (Optional[str]): Unique context ID for evaluation
+                (Runtime.evaluate).
+            serialization_options (Optional[SerializationOptions]): Serialization options for
+                the result (Runtime.evaluate).
+
+        Returns:
+            Union[EvaluateResponse, CallFunctionOnResponse]: The result of the script execution.
+
+        Raises:
+            InvalidScriptWithElement: If script uses 'argument' keyword but no element is provided.
 
         Examples:
+            # Execute a simple script to log a message
+            await page.execute_script('console.log("Hello World")')
+
+            # Execute a script that returns the page title
+            await page.execute_script('return document.title')
+
+            # Execute a script on an element to click it
             await page.execute_script('argument.click()', element)
-            await page.execute_script('argument.value = "Hello"', element)
 
-        Raises:
-            InvalidScriptWithElement: If script contains 'argument' but no element is provided.
+            # Execute a script on an element to set its value
+            await page.execute_script('argument.value = "Hello"', element)
         """
-        if 'argument' in script and element is None:
-            raise InvalidScriptWithElement('Script contains "argument" but no element was provided')
-
         logger.debug(f'Executing script: with_element={bool(element)}, length={len(script)}')
-        if element:
-            return await self._execute_script_with_element(script, element)
-        return await self._execute_script_without_element(script)
+        if element is not None:
+            warnings.warn(
+                'Passing a WebElement to Tab.execute_script() is deprecated. '
+                'Use WebElement.execute_script() instead.',
+                DeprecationWarning,
+                stacklevel=2,
+            )
+
+            return await element.execute_script(
+                script,
+                arguments=arguments,
+                silent=silent,
+                return_by_value=return_by_value,
+                generate_preview=generate_preview,
+                user_gesture=user_gesture,
+                await_promise=await_promise,
+                execution_context_id=execution_context_id,
+                object_group=object_group,
+                throw_on_side_effect=throw_on_side_effect,
+                unique_context_id=unique_context_id,
+                serialization_options=serialization_options,
+            )
+
+        if has_return_outside_function(script):
+            script = f'(function(){{ {script} }})()'
+
+        command = self._get_evaluate_command(
+            script,
+            object_group=object_group,
+            include_command_line_api=include_command_line_api,
+            silent=silent,
+            context_id=context_id,
+            return_by_value=return_by_value,
+            generate_preview=generate_preview,
+            user_gesture=user_gesture,
+            await_promise=await_promise,
+            throw_on_side_effect=throw_on_side_effect,
+            timeout=timeout,
+            disable_breaks=disable_breaks,
+            repl_mode=repl_mode,
+            allow_unsafe_eval_blocked_by_csp=allow_unsafe_eval_blocked_by_csp,
+            unique_context_id=unique_context_id,
+            serialization_options=serialization_options,
+        )
+        logger.debug(f'Executing script without element: length={len(script)}')
+        result: Union[EvaluateResponse, CallFunctionOnResponse] = await self._execute_command(
+            command
+        )
+        self._validate_argument_error(result)
+        return result
 
     # TODO: think about how to remove these duplications with the base class
     async def continue_request(
@@ -1155,46 +1312,45 @@ class Tab(FindElementsMixin):
         )
         return ConnectionHandler(self._connection_port, self._target_id)
 
-    async def _execute_script_with_element(self, script: str, element: 'WebElement'):
-        """
-        Execute script with element context.
-
-        Args:
-            script: JavaScript code to execute.
-            element: Element context (use 'argument' in script to reference).
-
-        Returns:
-            The result of the script execution.
-        """
-        if 'argument' not in script:
-            raise InvalidScriptWithElement('Script does not contain "argument"')
-
-        script = script.replace('argument', 'this')
-
-        if not is_script_already_function(script):
-            script = f'function(){{ {script} }}'
-
-        command = RuntimeCommands.call_function_on(
-            object_id=element._object_id, function_declaration=script, return_by_value=True
+    @staticmethod
+    def _get_evaluate_command(
+        script: str,
+        *,
+        object_group: Optional[str] = None,
+        include_command_line_api: Optional[bool] = None,
+        silent: Optional[bool] = None,
+        context_id: Optional[int] = None,
+        return_by_value: Optional[bool] = None,
+        generate_preview: Optional[bool] = None,
+        user_gesture: Optional[bool] = None,
+        await_promise: Optional[bool] = None,
+        throw_on_side_effect: Optional[bool] = None,
+        timeout: Optional[float] = None,
+        disable_breaks: Optional[bool] = None,
+        repl_mode: Optional[bool] = None,
+        allow_unsafe_eval_blocked_by_csp: Optional[bool] = None,
+        unique_context_id: Optional[str] = None,
+        serialization_options: Optional[SerializationOptions] = None,
+    ):
+        """Create an evaluate command with the given parameters."""
+        return RuntimeCommands.evaluate(
+            expression=script,
+            object_group=object_group,
+            include_command_line_api=include_command_line_api,
+            silent=silent,
+            context_id=context_id,
+            return_by_value=return_by_value,
+            generate_preview=generate_preview,
+            user_gesture=user_gesture,
+            await_promise=await_promise,
+            throw_on_side_effect=throw_on_side_effect,
+            timeout=timeout,
+            disable_breaks=disable_breaks,
+            repl_mode=repl_mode,
+            allow_unsafe_eval_blocked_by_csp=allow_unsafe_eval_blocked_by_csp,
+            unique_context_id=unique_context_id,
+            serialization_options=serialization_options,
         )
-        return await self._execute_command(command)
-
-    async def _execute_script_without_element(self, script: str):
-        """
-        Execute script without element context.
-
-        Args:
-            script: JavaScript code to execute.
-
-        Returns:
-            The result of the script execution.
-        """
-        if has_return_outside_function(script):
-            script = f'(function(){{ {script} }})()'
-
-        command = RuntimeCommands.evaluate(expression=script)
-        logger.debug(f'Executing script without element: length={len(script)}')
-        return await self._execute_command(command)
 
     async def _refresh_if_url_not_changed(self, url: str) -> bool:
         """Refresh page if URL hasn't changed."""
@@ -1204,6 +1360,33 @@ class Tab(FindElementsMixin):
             return True
         return False
 
+    @staticmethod
+    def _validate_argument_error(response: EvaluateResponse) -> None:
+        """
+        Validate that script didn't fail with ReferenceError about 'argument' being undefined.
+
+        Raises:
+            InvalidScriptWithElement: If script uses 'argument' keyword but no element was provided.
+        """
+        evaluate_result = response.get('result')
+        if not isinstance(evaluate_result, dict):
+            return
+
+        remote_object = evaluate_result.get('result')
+        if not isinstance(remote_object, dict):
+            return
+
+        if not (
+            remote_object.get('type') == 'object'
+            and remote_object.get('subtype') == 'error'
+            and remote_object.get('className') == 'ReferenceError'
+        ):
+            return
+
+        description = remote_object.get('description', '')
+        if 'argument is not defined' in description:
+            raise InvalidScriptWithElement('Script contains "argument" but no element was provided')
+
     async def _wait_page_load(self, timeout: int = 300):
         """
         Wait for page to finish loading.
@@ -1240,7 +1423,7 @@ class Tab(FindElementsMixin):
             element = cast('WebElement', element)
             if element:
                 # adjust the external div size to shadow root width (usually 300px)
-                await self.execute_script('argument.style="width: 300px"', element)
+                await element.execute_script('this.style="width: 300px"')
                 await asyncio.sleep(time_before_click)
                 await element.click()
         except Exception as exc: