scrapely-python 0.1.42__tar.gz → 0.1.43__tar.gz

{scrapely_python-0.1.42 → scrapely_python-0.1.43}/CHANGELOG.md

@@ -4,6 +4,17 @@ All notable changes to this project will be documented in this file.
 
 ## [Unreleased]
 
+## [0.1.43] - 2026-11-05
+
+- **Python actor processes start without loading the SDK.** The base images import `scrapely.patch_browser_pool` (from `sitecustomize`) in every Python process. Until now that loaded the whole SDK at each interpreter start: `scrapely.actor`, charging, the `scrapely_client` models and Playwright. On `actor-python:3.11-0.1.42`, `python -c pass` took 0.89-0.95 s of CPU, which is 4.2 s of wall time at a 0.25-core quota. It now takes 0.016 s. Each import below gets the same start-up cut:
+  - `import scrapely` is lazy (PEP 562). Every public name and submodule attribute resolves on first use, so every `from scrapely import X` and `scrapely.X` that worked before still works.
+  - `scrapely_client` loads only when an API client is created (`Actor.apify_client`, `create_charging_client`). `USES_PUSH_DATA_INTERCEPTION` and `push_data_charging_context` are still the client's own objects.
+- **Browser patches are applied through an import hook.** `scrapely.patch_browser_pool` imports no browser library any more; it imports builtin modules only. Each patch now runs the moment its library finishes loading, before the import that loaded it returns. That covers the Crawlee BrowserPool / PlaywrightBrowserPlugin / PlaywrightCrawler, raw Playwright (`async_playwright` and `sync_playwright`, each when its module loads), Camoufox `AsyncNewBrowser` and pyppeteer `launch`. The patches behave exactly as before. The hook also:
+  - survives `importlib.util.find_spec()` availability checks and a failed first import;
+  - logs a warning when a patch fails instead of breaking the actor's import.
+- **Start-up measured on staging** (fresh builds of the same commit on the same base, old SDK vs 0.1.43; "boot" = container started to the SDK's "Initializing API storage" line; paired median): web-scraper -0.74 s, capterra-scraper (raw Playwright over the remote browser) -0.89 s, contact-info-scraper (512 MB) -0.51 s. All 88 scenario runs passed on both sides. Locally at 1 CPU, a full `from apify import Actor` + `Actor.init` + `push_data` + exit went from 1.19-1.36 s to 0.92-1.02 s of CPU.
+- Base images (released with this version): `sitecustomize` no longer imports `pathlib` (~70 ms per start), and the `apify` shim resolves names on first use instead of `from scrapely import *`.
+
 ## [0.1.42] - 2026-11-02
 
 - **Sessions that keep their identity: `http.session(keep_identity=True)`.** A pinned `scrapely.http` session used to retire its identity on the first block (a 403, a challenge page, a network error). The session then moved to a new exit, a new profile and an empty cookie jar. That breaks every challenge solve: the challenge page, the solver's script and submission requests and the page fetched again must all share one identity. With `keep_identity=True`:

{scrapely_python-0.1.42 → scrapely_python-0.1.43}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scrapely-python
-Version: 0.1.42
+Version: 0.1.43
 Summary: Scrapely Python SDK - A toolkit for building Actors on the Scrapely platform
 Author-email: Scrapely <hello@scrape.ly>
 License-Expression: MIT AND Apache-2.0

{scrapely_python-0.1.42 → scrapely_python-0.1.43}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scrapely-python"
-version = "0.1.42"
+version = "0.1.43"
 description = "Scrapely Python SDK - A toolkit for building Actors on the Scrapely platform"
 readme = "README.md"
 license = "MIT AND Apache-2.0"

{scrapely_python-0.1.42 → scrapely_python-0.1.43}/src/scrapely/__init__.py

@@ -10,95 +10,114 @@ Usage:
             await Actor.push_data({'url': actor_input['url']})
 """
 
-from typing import TYPE_CHECKING, Any
+from __future__ import annotations
+
+# ``typing`` is not imported at run time (see below): this package loads at
+# every interpreter start in actor images.
+TYPE_CHECKING = False
+if TYPE_CHECKING:
+    from typing import Any
 
 # Side-effect: patch browser launches (Crawlee BrowserPool, PlaywrightCrawler,
 # raw Playwright, Camoufox, pyppeteer) to use the remote CDP browser (CDP_URL).
 # Must run on package import so `from scrapely import Actor` enables the patch
-# before any browser is launched (Node SDK parity). It does not import Crawlee:
-# the Crawlee patches are applied when Crawlee's browser modules load.
+# before any browser is launched (Node SDK parity). It imports none of those
+# libraries: each patch is applied the moment its library finishes loading.
 from .patch_browser_pool import set_cdp_url  # noqa: F401
 
-# Crawlee is NOT imported here: it costs ~1 s per run on the platform and many
-# actors never touch it. Everything that needs Crawlee (the re-exported Crawlee
-# classes, storage clients, events, request loaders) is resolved on first
-# attribute access by ``__getattr__`` below; ``Actor`` imports what it needs on
-# first use. Every ``from scrapely import X`` that worked before still works.
-from .actor import Actor
-from .configuration import Configuration
-from .charging_client import (
-    USES_PUSH_DATA_INTERCEPTION,
-    create_charging_client,
-    push_data_charging_context,
-)
-from .proxy_configuration import ProxyConfiguration
-from .browser_configuration import BrowserConfiguration
-from .charging import (
-    ChargingManager,
-    ChargeUncertainError,
-    ChargeRejectedError,
-    has_default_dataset_item_event_in_pricing,
-    is_default_dataset_item_event,
-    merge_charge_results,
-    push_data_and_charge,
-    resolve_default_dataset_item_event_name,
-    APIFY_DEFAULT_DATASET_ITEM_EVENT,
-    SCRAPELY_DEFAULT_DATASET_ITEM_EVENT,
-    DEFAULT_DATASET_ITEM_EVENT,
-)
-
-from .types import (
-    ActorInput,
-    ActorPermissionLevel,
-    ActorRun,
-    ApifyEnv,
-    AbortOptions,
-    CallOptions,
-    ExitOptions,
-    InitOptions,
-    MetamorphOptions,
-    OpenStorageOptions,
-    ProxyInfo,
-    RebootOptions,
-    RecordOptions,
-    SetStatusMessageOptions,
-    StartOptions,
-    WebhookOptions,
-)
+# Nothing else is imported here. The base images' sitecustomize imports
+# ``scrapely.patch_browser_pool`` (and so this package) in EVERY Python process
+# of an actor image; loading the whole SDK there cost ~0.75 s per interpreter
+# start at 1 CPU (5.7 s at 0.25 CPU). Every public name is resolved on first
+# attribute access by ``__getattr__`` below and cached in the module namespace,
+# so every ``from scrapely import X`` that worked before still works.
 
-from .webhook import (
-    Webhook,
-    WebhookEventType,
-)
-from .pricing_types import (
-    PricingModel,
-    FreeActorPricingInfo,
-    FlatPricePerMonthActorPricingInfo,
-    PricePerDatasetItemActorPricingInfo,
-    PayPerEventActorPricingInfo,
-    PricingPerEvent,
-    ActorPricingInfo,
-    ActorChargeEvent,
-    ChargeResult,
-    ActorPricingInfoResult,
-)
-from .errors import (
-    ScrapelyClientError,
-    ScrapelyApiError,
-    InvalidResponseBodyError,
-    InvalidRequestError,
-    UnauthorizedError,
-    ForbiddenError,
-    NotFoundError,
-    ConflictError,
-    RateLimitError,
-    ServerError,
-    raise_for_status,
-)
-
-# Public name -> (module, attribute) for names that need Crawlee. Loaded on
-# first access and cached in the module namespace.
+# Public name -> (module, attribute). Loaded on first access.
 _LAZY_ATTRS = {
+    'Actor': ('scrapely.actor', 'Actor'),
+    'Configuration': ('scrapely.configuration', 'Configuration'),
+    'ProxyConfiguration': ('scrapely.proxy_configuration', 'ProxyConfiguration'),
+    'BrowserConfiguration': ('scrapely.browser_configuration', 'BrowserConfiguration'),
+    # Charging client
+    **{
+        name: ('scrapely.charging_client', name)
+        for name in ('USES_PUSH_DATA_INTERCEPTION', 'create_charging_client', 'push_data_charging_context')
+    },
+    # Charging
+    **{
+        name: ('scrapely.charging', name)
+        for name in (
+            'ChargingManager',
+            'ChargeUncertainError',
+            'ChargeRejectedError',
+            'has_default_dataset_item_event_in_pricing',
+            'is_default_dataset_item_event',
+            'merge_charge_results',
+            'push_data_and_charge',
+            'resolve_default_dataset_item_event_name',
+            'APIFY_DEFAULT_DATASET_ITEM_EVENT',
+            'SCRAPELY_DEFAULT_DATASET_ITEM_EVENT',
+            'DEFAULT_DATASET_ITEM_EVENT',
+        )
+    },
+    # Types
+    **{
+        name: ('scrapely.types', name)
+        for name in (
+            'ActorInput',
+            'ActorPermissionLevel',
+            'ActorRun',
+            'ApifyEnv',
+            'AbortOptions',
+            'CallOptions',
+            'ExitOptions',
+            'InitOptions',
+            'MetamorphOptions',
+            'OpenStorageOptions',
+            'ProxyInfo',
+            'RebootOptions',
+            'RecordOptions',
+            'SetStatusMessageOptions',
+            'StartOptions',
+            'WebhookOptions',
+        )
+    },
+    # Webhook
+    'Webhook': ('scrapely.webhook', 'Webhook'),
+    'WebhookEventType': ('scrapely.webhook', 'WebhookEventType'),
+    # Pricing
+    **{
+        name: ('scrapely.pricing_types', name)
+        for name in (
+            'PricingModel',
+            'FreeActorPricingInfo',
+            'FlatPricePerMonthActorPricingInfo',
+            'PricePerDatasetItemActorPricingInfo',
+            'PayPerEventActorPricingInfo',
+            'PricingPerEvent',
+            'ActorPricingInfo',
+            'ActorChargeEvent',
+            'ChargeResult',
+            'ActorPricingInfoResult',
+        )
+    },
+    # Errors
+    **{
+        name: ('scrapely.errors', name)
+        for name in (
+            'ScrapelyClientError',
+            'ScrapelyApiError',
+            'InvalidResponseBodyError',
+            'InvalidRequestError',
+            'UnauthorizedError',
+            'ForbiddenError',
+            'NotFoundError',
+            'ConflictError',
+            'RateLimitError',
+            'ServerError',
+            'raise_for_status',
+        )
+    },
     # Crawlee re-exports
     'Request': ('crawlee', 'Request'),
     'RequestLoader': ('crawlee.request_loaders', 'RequestLoader'),
@@ -150,15 +169,28 @@ _LAZY_ATTRS = {
 }
 
 # Submodules that used to be loaded (and so bound as attributes) by this
-# package's eager imports; ``scrapely.events`` etc. keep working without an
-# explicit ``import scrapely.events``.
+# package's eager imports; ``scrapely.actor``, ``scrapely.events`` etc. keep
+# working without an explicit ``import scrapely.<name>``.
 _LAZY_SUBMODULES = frozenset({
+    '_browser_locations',
+    'actor',
+    'browser_configuration',
+    'cdp_retry',
+    'charging',
+    'charging_client',
+    'configuration',
+    'crypto',
+    'errors',
     'events',
     'log',
+    'pricing_types',
+    'proxy_configuration',
     'request_loaders',
     'storage',
     'storage_clients',
     'storages',
+    'types',
+    'webhook',
 })
 
 
@@ -193,6 +225,40 @@ if TYPE_CHECKING:  # Static analysers and IDEs see the lazy names as plain impor
     from crawlee.storage_clients.models import DatasetItemsListPage
     from crawlee.storages import Dataset, KeyValueStore, RequestQueue
 
+    from .actor import Actor
+    from .browser_configuration import BrowserConfiguration
+    from .charging import (
+        APIFY_DEFAULT_DATASET_ITEM_EVENT,
+        DEFAULT_DATASET_ITEM_EVENT,
+        SCRAPELY_DEFAULT_DATASET_ITEM_EVENT,
+        ChargeRejectedError,
+        ChargeUncertainError,
+        ChargingManager,
+        has_default_dataset_item_event_in_pricing,
+        is_default_dataset_item_event,
+        merge_charge_results,
+        push_data_and_charge,
+        resolve_default_dataset_item_event_name,
+    )
+    from .charging_client import (
+        USES_PUSH_DATA_INTERCEPTION,
+        create_charging_client,
+        push_data_charging_context,
+    )
+    from .configuration import Configuration
+    from .errors import (
+        ConflictError,
+        ForbiddenError,
+        InvalidRequestError,
+        InvalidResponseBodyError,
+        NotFoundError,
+        RateLimitError,
+        ScrapelyApiError,
+        ScrapelyClientError,
+        ServerError,
+        UnauthorizedError,
+        raise_for_status,
+    )
     from .events import (
         AbortingEvent,
         ActorEventTypes,
@@ -216,6 +282,19 @@ if TYPE_CHECKING:  # Static analysers and IDEs see the lazy names as plain impor
         SystemInfoEventData,
         UnknownEvent,
     )
+    from .pricing_types import (
+        ActorChargeEvent,
+        ActorPricingInfo,
+        ActorPricingInfoResult,
+        ChargeResult,
+        FlatPricePerMonthActorPricingInfo,
+        FreeActorPricingInfo,
+        PayPerEventActorPricingInfo,
+        PricePerDatasetItemActorPricingInfo,
+        PricingModel,
+        PricingPerEvent,
+    )
+    from .proxy_configuration import ProxyConfiguration
     from .request_loaders import ScrapelyRequestList
     from .storage import (
         ApiStorageClient,
@@ -226,8 +305,27 @@ if TYPE_CHECKING:  # Static analysers and IDEs see the lazy names as plain impor
         StorageClient,
         set_api_storage,
     )
+    from .types import (
+        AbortOptions,
+        ActorInput,
+        ActorPermissionLevel,
+        ActorRun,
+        ApifyEnv,
+        CallOptions,
+        ExitOptions,
+        InitOptions,
+        MetamorphOptions,
+        OpenStorageOptions,
+        ProxyInfo,
+        RebootOptions,
+        RecordOptions,
+        SetStatusMessageOptions,
+        StartOptions,
+        WebhookOptions,
+    )
+    from .webhook import Webhook, WebhookEventType
 
-__version__ = '0.1.41'
+__version__ = '0.1.43'
 __all__ = [
     # Core
     'Actor',

{scrapely_python-0.1.42 → scrapely_python-0.1.43}/src/scrapely/charging_client.py

@@ -11,15 +11,29 @@ from __future__ import annotations
 
 from typing import TYPE_CHECKING, Any, Callable, Dict, Optional, Sequence
 
-from scrapely_client import ApifyClientAsync
+if TYPE_CHECKING:
+    from scrapely_client import USES_PUSH_DATA_INTERCEPTION, ApifyClientAsync, push_data_charging_context
+
+    from .charging import ChargingManager
 
+# ``scrapely_client`` (219 pydantic models) loads only when a client is created
+# or one of its names below is used: ``Actor`` imports this module, and most
+# runs never create an API client.
+#
 # Re-exported so the SDK and the client share one marker attribute and one
 # context var. Identity matters: the client's intercepted ``push_items()``
 # reads/writes these, and the SDK detects interception via the attribute.
-from scrapely_client import USES_PUSH_DATA_INTERCEPTION, push_data_charging_context
+_CLIENT_NAMES = frozenset({'ApifyClientAsync', 'USES_PUSH_DATA_INTERCEPTION', 'push_data_charging_context'})
 
-if TYPE_CHECKING:
-    from .charging import ChargingManager
+
+def __getattr__(name: str) -> Any:
+    if name in _CLIENT_NAMES:
+        import scrapely_client
+
+        value = getattr(scrapely_client, name)
+        globals()[name] = value
+        return value
+    raise AttributeError(f'module {__name__!r} has no attribute {name!r}')
 
 __all__ = [
     'USES_PUSH_DATA_INTERCEPTION',
@@ -81,7 +95,7 @@ def create_charging_client(
     charging_manager: Optional['ChargingManager'] = None,
     default_dataset_id: Optional[str] = None,
     **kwargs: Any,
-) -> ApifyClientAsync:
+) -> 'ApifyClientAsync':
     """Create an ``ApifyClientAsync`` configured with the SDK's charging hooks.
 
     Args:
@@ -101,6 +115,8 @@ def create_charging_client(
     if api_public_url:
         kwargs['api_public_url'] = api_public_url
 
+    from scrapely_client import ApifyClientAsync
+
     charging_hooks = _ChargingManagerHooks(charging_manager) if charging_manager is not None else None
 
     return ApifyClientAsync(

{scrapely_python-0.1.42 → scrapely_python-0.1.43}/src/scrapely/patch_browser_pool.py

@@ -30,11 +30,11 @@ Mirrors the Node SDK ``patch-browser-pool.ts``:
   sandbox-safe Chromium args when running on the platform.
 - Soft no-op when Playwright/Crawlee browsers are not installed (e.g. HTTP-only
   actors).
-- Crawlee is never imported by this module (it costs ~1 s on the platform). The
-  Crawlee patches are applied at once when Crawlee's browser / Playwright
-  crawler modules are already loaded, otherwise by a post-import hook the
-  moment those modules finish loading — before actor code can use the classes.
-  The Playwright, Camoufox and pyppeteer patches are applied eagerly on import.
+- Neither Crawlee, Playwright, Camoufox nor pyppeteer is imported by this
+  module (the base images import it at every interpreter start). Each patch is
+  applied at once when its module is already loaded, otherwise by a
+  post-import hook the moment that module finishes loading — before actor code
+  can use what it patches.
 
 Location and proxy handling (the remote browser never takes a proxy):
 
@@ -67,19 +67,79 @@ Location and proxy handling (the remote browser never takes a proxy):
 from __future__ import annotations
 
 import contextvars
-import functools
-import importlib.abc
-import logging
 import os
-import re
 import sys
-from typing import Any, List, Optional, Tuple
-from urllib.parse import SplitResult, parse_qsl, unquote, urlencode, urlsplit, urlunsplit
 
-from ._browser_locations import subdivision_to_state_name
-from .cdp_retry import connect_with_retry_async, wrap_connect_over_cdp
+# This module is imported at EVERY interpreter start in actor images (base
+# image sitecustomize), before any actor code: it imports only builtin modules
+# at load time. Everything else (typing, logging, re, urllib.parse, functools,
+# asyncio via cdp_retry, the browser location table) loads on first use.
+TYPE_CHECKING = False
+if TYPE_CHECKING:
+    from typing import Any, List, Optional, Tuple
+    from urllib.parse import SplitResult
 
-logger = logging.getLogger(__name__)
+
+class _LazyLogger:
+    """``logging.getLogger(__name__)``, imported on first use."""
+
+    def __getattr__(self, name: str) -> Any:
+        import logging
+
+        return getattr(logging.getLogger(__name__), name)
+
+
+logger = _LazyLogger()
+
+
+def urlsplit(*args: Any, **kwargs: Any) -> Any:
+    from urllib.parse import urlsplit as _urlsplit
+
+    return _urlsplit(*args, **kwargs)
+
+
+def urlunsplit(*args: Any, **kwargs: Any) -> Any:
+    from urllib.parse import urlunsplit as _urlunsplit
+
+    return _urlunsplit(*args, **kwargs)
+
+
+def parse_qsl(*args: Any, **kwargs: Any) -> Any:
+    from urllib.parse import parse_qsl as _parse_qsl
+
+    return _parse_qsl(*args, **kwargs)
+
+
+def urlencode(*args: Any, **kwargs: Any) -> Any:
+    from urllib.parse import urlencode as _urlencode
+
+    return _urlencode(*args, **kwargs)
+
+
+def unquote(*args: Any, **kwargs: Any) -> Any:
+    from urllib.parse import unquote as _unquote
+
+    return _unquote(*args, **kwargs)
+
+
+# ``cdp_retry`` pulls in asyncio and ``_browser_locations`` is a large table;
+# both are only needed once a browser is actually launched.
+def subdivision_to_state_name(country: Optional[str], subdivision: Optional[str]) -> Optional[str]:
+    from ._browser_locations import subdivision_to_state_name as _subdivision_to_state_name
+
+    return _subdivision_to_state_name(country, subdivision)
+
+
+def wrap_connect_over_cdp(*args: Any, **kwargs: Any) -> Any:
+    from .cdp_retry import wrap_connect_over_cdp as _wrap_connect_over_cdp
+
+    return _wrap_connect_over_cdp(*args, **kwargs)
+
+
+async def connect_with_retry_async(*args: Any, **kwargs: Any) -> Any:
+    from .cdp_retry import connect_with_retry_async as _connect_with_retry_async
+
+    return await _connect_with_retry_async(*args, **kwargs)
 
 _cdp_url: Optional[str] = None
 
@@ -219,7 +279,7 @@ def _redact_proxy_url(proxy_url: str) -> str:
     return f'{parts.scheme or "http"}://{userinfo}{parts.hostname}{port}'
 
 
-_COUNTRY_FRAGMENT_RE = re.compile(r'^country-([A-Za-z]{2})(?:_([A-Za-z0-9]{1,3}))?$')
+_COUNTRY_FRAGMENT_PATTERN = r'^country-([A-Za-z]{2})(?:_([A-Za-z0-9]{1,3}))?$'
 
 
 def _parse_proxy_url_location(proxy_url: Optional[str]) -> Tuple[Optional[str], Optional[str]]:
@@ -229,8 +289,10 @@ def _parse_proxy_url_location(proxy_url: Optional[str]) -> Tuple[Optional[str],
     parts = _parse_proxy_url(str(proxy_url))
     if not parts or not parts.username:
         return None, None
+    import re
+
     for fragment in unquote(parts.username).split(','):
-        match = _COUNTRY_FRAGMENT_RE.match(fragment.strip())
+        match = re.match(_COUNTRY_FRAGMENT_PATTERN, fragment.strip())
         if match:
             subdivision = match.group(2)
             return match.group(1).upper(), subdivision.upper() if subdivision else None
@@ -475,6 +537,8 @@ def _apply_crawler_patch() -> None:
     if getattr(PlaywrightCrawler, '_scrapely_browser_config_patched', False):
         return
 
+    import functools
+
     _orig_init = PlaywrightCrawler.__init__
 
     @functools.wraps(_orig_init)
@@ -784,39 +848,46 @@ class _AsyncPlaywrightContextManagerProxy:
         return await self._cm.__aexit__(*exc)
 
 
-def _apply_raw_playwright_patch() -> None:
-    """Wrap playwright's sync/async entry points to redirect raw launches to CDP."""
+def _apply_raw_playwright_async_patch() -> None:
+    """Wrap ``playwright.async_api.async_playwright`` to redirect raw launches to CDP."""
     try:
         import playwright.async_api as playwright_async_api
     except ImportError:
-        playwright_async_api = None  # type: ignore[assignment]
+        return
 
-    if playwright_async_api is not None and not getattr(
-        playwright_async_api, '_scrapely_raw_cdp_patched', False
-    ):
-        orig_async = playwright_async_api.async_playwright
+    if getattr(playwright_async_api, '_scrapely_raw_cdp_patched', False):
+        return
+    orig_async = playwright_async_api.async_playwright
+
+    def async_playwright_patched() -> Any:
+        return _AsyncPlaywrightContextManagerProxy(orig_async())
 
-        def async_playwright_patched() -> Any:
-            return _AsyncPlaywrightContextManagerProxy(orig_async())
+    playwright_async_api.async_playwright = async_playwright_patched
+    playwright_async_api._scrapely_raw_cdp_patched = True  # type: ignore[attr-defined]
 
-        playwright_async_api.async_playwright = async_playwright_patched
-        playwright_async_api._scrapely_raw_cdp_patched = True  # type: ignore[attr-defined]
 
+def _apply_raw_playwright_sync_patch() -> None:
+    """Wrap ``playwright.sync_api.sync_playwright`` to redirect raw launches to CDP."""
     try:
         import playwright.sync_api as playwright_sync_api
     except ImportError:
-        playwright_sync_api = None  # type: ignore[assignment]
+        return
 
-    if playwright_sync_api is not None and not getattr(
-        playwright_sync_api, '_scrapely_raw_cdp_patched', False
-    ):
-        orig_sync = playwright_sync_api.sync_playwright
+    if getattr(playwright_sync_api, '_scrapely_raw_cdp_patched', False):
+        return
+    orig_sync = playwright_sync_api.sync_playwright
 
-        def sync_playwright_patched() -> Any:
-            return _SyncPlaywrightContextManagerProxy(orig_sync())
+    def sync_playwright_patched() -> Any:
+        return _SyncPlaywrightContextManagerProxy(orig_sync())
 
-        playwright_sync_api.sync_playwright = sync_playwright_patched
-        playwright_sync_api._scrapely_raw_cdp_patched = True  # type: ignore[attr-defined]
+    playwright_sync_api.sync_playwright = sync_playwright_patched
+    playwright_sync_api._scrapely_raw_cdp_patched = True  # type: ignore[attr-defined]
+
+
+def _apply_raw_playwright_patch() -> None:
+    """Wrap playwright's sync/async entry points to redirect raw launches to CDP."""
+    _apply_raw_playwright_async_patch()
+    _apply_raw_playwright_sync_patch()
 
 
 def _apply_pyppeteer_patch() -> None:
@@ -874,31 +945,59 @@ def _apply_pyppeteer_patch() -> None:
     pyppeteer_mod._scrapely_cdp_patched = True
 
 
-# ─── Crawlee patches without importing Crawlee ───────────────────────────────
+# ─── Patches applied when their library loads ────────────────────────────────
 #
-# Importing Crawlee costs ~1 s per run on the platform, so this module never
-# imports it. The Crawlee patches mutate classes in place, so they only have to
-# run before actor code uses those classes: they are applied immediately when
-# the defining modules are already loaded, otherwise by a post-import hook the
-# moment they finish executing (``import crawlee.browsers``,
-# ``from crawlee.crawlers import PlaywrightCrawler``, a PlaywrightCrawler built
-# by any template...). Behaviour is the same as applying them eagerly.
+# This module is imported at every interpreter start in actor images (base
+# image sitecustomize), so it never imports Crawlee, Playwright, Camoufox or
+# pyppeteer itself: importing them up front cost ~1 s per process on the
+# platform even when the actor never opens a browser. Every patch mutates a
+# module or class in place, so it only has to run before actor code can use
+# what it patches: it is applied at once when the defining module is already
+# loaded, otherwise by a post-import hook the moment that module finishes
+# executing, before the import statement that loaded it returns
+# (``import crawlee.browsers``, ``from crawlee.crawlers import
+# PlaywrightCrawler``, ``from playwright.async_api import async_playwright``, a
+# PlaywrightCrawler built by any template...). Behaviour is the same as
+# applying them eagerly.
 
 # Module name -> patch functions to run once that module has finished loading.
 # ``crawlee.browsers`` imports BrowserPool, PlaywrightBrowserController and
 # PlaywrightBrowserPlugin in its ``__init__``; any import of one of those
-# submodules runs that ``__init__`` to completion first.
-_CRAWLEE_POST_IMPORT_PATCHES: dict = {
+# submodules runs that ``__init__`` to completion first. Likewise importing
+# ``pyppeteer.launcher`` runs the ``pyppeteer`` package first, and the
+# ``camoufox`` package defines the ``AsyncNewBrowser`` the patch replaces.
+_POST_IMPORT_PATCHES: dict = {
     'crawlee.browsers': (_apply_patch,),
     'crawlee.crawlers._playwright._playwright_crawler': (_apply_crawler_patch,),
+    'playwright.async_api': (_apply_raw_playwright_async_patch,),
+    'playwright.sync_api': (_apply_raw_playwright_sync_patch,),
+    'camoufox': (_apply_camoufox_patch,),
+    'pyppeteer': (_apply_pyppeteer_patch,),
 }
+# Kept under its old name for code that inspects it.
+_CRAWLEE_POST_IMPORT_PATCHES = _POST_IMPORT_PATCHES
+
+
+def _run_patches(module_name: str, patches: tuple) -> None:
+    """Run the patches for a loaded module; a failing patch never breaks the actor's import."""
+    for patch in patches:
+        try:
+            patch()
+        except Exception as exc:  # pragma: no cover - defensive (incompatible library version)
+            logger.warning(
+                '[scrapely] Could not patch %s for the remote browser (%s: %s)',
+                module_name,
+                type(exc).__name__,
+                exc,
+            )
 
 
-class _PostImportLoader(importlib.abc.Loader):
+class _PostImportLoader:
     """Delegating loader that runs the Scrapely patches after the module executes."""
 
-    def __init__(self, loader: Any, patches: tuple) -> None:
+    def __init__(self, loader: Any, fullname: str, patches: tuple) -> None:
         self._loader = loader
+        self._fullname = fullname
         self._patches = patches
 
     def create_module(self, spec: Any) -> Any:
@@ -914,22 +1013,25 @@ class _PostImportLoader(importlib.abc.Loader):
         except Exception:
             pass
         self._loader.exec_module(module)
-        for patch in self._patches:
-            patch()
+        # The module loaded: patch it now, once. A failed load (the exception
+        # above) keeps the hook, so a later successful import is patched too.
+        if _POST_IMPORT_PATCHES.get(self._fullname) is self._patches:
+            _POST_IMPORT_PATCHES.pop(self._fullname, None)
+        _run_patches(self._fullname, self._patches)
 
     def __getattr__(self, name: str) -> Any:
         return getattr(self._loader, name)
 
 
-class _CrawleePostImportFinder(importlib.abc.MetaPathFinder):
-    """Meta path finder that hooks the Crawlee modules in ``_CRAWLEE_POST_IMPORT_PATCHES``."""
+class _PostImportFinder:
+    """Meta path finder that hooks the modules in ``_POST_IMPORT_PATCHES``."""
 
     def find_spec(self, fullname: str, path: Any = None, target: Any = None) -> Any:
-        patches = _CRAWLEE_POST_IMPORT_PATCHES.get(fullname)
+        patches = _POST_IMPORT_PATCHES.get(fullname)
         if patches is None:
             return None
         for finder in sys.meta_path:
-            if finder is self or isinstance(finder, _CrawleePostImportFinder):
+            if finder is self or isinstance(finder, _PostImportFinder):
                 continue
             find_spec = getattr(finder, 'find_spec', None)
             if find_spec is None:
@@ -941,25 +1043,25 @@ class _CrawleePostImportFinder(importlib.abc.MetaPathFinder):
             return None
         if spec.loader is None or not hasattr(spec.loader, 'exec_module'):
             return spec
-        # One-shot: the patches are idempotent, but a module loads only once.
-        _CRAWLEE_POST_IMPORT_PATCHES.pop(fullname, None)
-        spec.loader = _PostImportLoader(spec.loader, patches)
+        # The hook stays registered until the module has really loaded
+        # (``importlib.util.find_spec`` availability checks do not consume it).
+        spec.loader = _PostImportLoader(spec.loader, fullname, patches)
         return spec
 
 
-def _install_crawlee_patches() -> None:
-    """Apply the Crawlee patches now if Crawlee is loaded, else when it loads."""
-    for module_name in list(_CRAWLEE_POST_IMPORT_PATCHES):
+# Kept under its old name for code that inspects ``sys.meta_path``.
+_CrawleePostImportFinder = _PostImportFinder
+
+
+def _install_post_import_patches() -> None:
+    """Apply each patch now if its module is loaded, else when it loads."""
+    for module_name in list(_POST_IMPORT_PATCHES):
         if module_name in sys.modules:
-            for patch in _CRAWLEE_POST_IMPORT_PATCHES.pop(module_name):
-                patch()
-    if _CRAWLEE_POST_IMPORT_PATCHES and not any(
-        isinstance(finder, _CrawleePostImportFinder) for finder in sys.meta_path
-    ):
-        sys.meta_path.insert(0, _CrawleePostImportFinder())
-
-
-_install_crawlee_patches()
-_apply_camoufox_patch()
-_apply_raw_playwright_patch()
-_apply_pyppeteer_patch()
+            _run_patches(module_name, _POST_IMPORT_PATCHES.pop(module_name))
+    if _POST_IMPORT_PATCHES and not any(isinstance(finder, _PostImportFinder) for finder in sys.meta_path):
+        sys.meta_path.insert(0, _PostImportFinder())
+
+
+_install_crawlee_patches = _install_post_import_patches
+
+_install_post_import_patches()

{scrapely_python-0.1.42 → scrapely_python-0.1.43}/src/scrapely_python.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scrapely-python
-Version: 0.1.42
+Version: 0.1.43
 Summary: Scrapely Python SDK - A toolkit for building Actors on the Scrapely platform
 Author-email: Scrapely <hello@scrape.ly>
 License-Expression: MIT AND Apache-2.0