scrapely-client-python 1.0.8__tar.gz → 1.0.9__tar.gz

{scrapely_client_python-1.0.8 → scrapely_client_python-1.0.9}/CHANGELOG.md

@@ -2,6 +2,11 @@
 
 All notable changes to this project will be documented in this file.
 
+## [1.0.9] - 2026-10-24
+
+- The `SCRAPELY_API_URL` environment variable sets the API URL when `api_url` is not passed to `ScrapelyClient`, `ScrapelyClientAsync` or `with_custom_http_client`; `api_url` still wins, and without either the client uses `https://api.scrape.ly`. An empty value is ignored. `api_public_url` is unchanged.
+- Version 1.0.3 defaulted to `https://api-staging.scrape.ly`; upgrade to 1.0.6 or later.
+
 ## [1.0.8] - 2026-10-20
 
 - Request ids are percent-encoded as one URL path segment in `get_request`, `update_request`, `delete_request`, `prolong_request_lock` and `delete_request_lock` (sync and async); `''`, `.` and `..` raise `ValueError`. `update_request` checks the id against the API's grammar (1-128 letters, digits, `-`, `_`, `~`) and refuses a request without an id instead of sending `requests/None`. Resource ids (`client.dataset(id)`, versions, env vars) and automation run ids are encoded the same way. An id such as `..` used to send the call to another resource.

{scrapely_client_python-1.0.8 → scrapely_client_python-1.0.9}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scrapely-client-python
-Version: 1.0.8
+Version: 1.0.9
 Summary: Scrapely API client for Python
 Keywords: scrapely,api,client,automation,crawling,scraping
 Author: TeraShift GmbH (Scrape.ly)
@@ -94,6 +94,8 @@ Description-Content-Type: text/markdown
 
 You'll need a Scrapely API token — find yours in the integrations section of the Scrapely dashboard. Pass it to the client and you're ready to go.
 
+The client sends requests to `https://api.scrape.ly` unless you pass another API origin as `api_url`. When `api_url` is not given, the `SCRAPELY_API_URL` environment variable sets it (for example `SCRAPELY_API_URL=https://api-staging.scrape.ly`). `api_public_url`, used for public storage links, keeps its own argument and default. The client does not read the token from the environment.
+
 ### Synchronous client
 
 ```python
@@ -190,6 +192,21 @@ for item in client.dataset('DATASET-ID').iterate_items(fields=['title', 'url']):
     process(item)
 ```
 
+### Create and run an automation
+
+```python
+automation = client.automations().create({
+    'title': 'Daily dentists',
+    'trigger': {'kind': 'schedule', 'cron': '0 9 * * *', 'timezone': 'Europe/Berlin'},
+    'steps': [{'kind': 'savedInput', 'savedInputId': 'SAVED-INPUT-ID'}],
+    'ai': {'mode': 'summarize', 'prompt': 'List the new practices with their phone numbers.'},
+    'deliveries': [{'channel': 'email'}],
+})
+run = client.automation(automation['id']).run()
+```
+
+Automations take and return the API's camelCase dictionaries (see the automations section of the API reference).
+
 ### Tune retries and timeouts
 
 ```python

{scrapely_client_python-1.0.8 → scrapely_client_python-1.0.9}/README.md

@@ -61,6 +61,8 @@
 
 You'll need a Scrapely API token — find yours in the integrations section of the Scrapely dashboard. Pass it to the client and you're ready to go.
 
+The client sends requests to `https://api.scrape.ly` unless you pass another API origin as `api_url`. When `api_url` is not given, the `SCRAPELY_API_URL` environment variable sets it (for example `SCRAPELY_API_URL=https://api-staging.scrape.ly`). `api_public_url`, used for public storage links, keeps its own argument and default. The client does not read the token from the environment.
+
 ### Synchronous client
 
 ```python
@@ -157,6 +159,21 @@ for item in client.dataset('DATASET-ID').iterate_items(fields=['title', 'url']):
     process(item)
 ```
 
+### Create and run an automation
+
+```python
+automation = client.automations().create({
+    'title': 'Daily dentists',
+    'trigger': {'kind': 'schedule', 'cron': '0 9 * * *', 'timezone': 'Europe/Berlin'},
+    'steps': [{'kind': 'savedInput', 'savedInputId': 'SAVED-INPUT-ID'}],
+    'ai': {'mode': 'summarize', 'prompt': 'List the new practices with their phone numbers.'},
+    'deliveries': [{'channel': 'email'}],
+})
+run = client.automation(automation['id']).run()
+```
+
+Automations take and return the API's camelCase dictionaries (see the automations section of the API reference).
+
 ### Tune retries and timeouts
 
 ```python

{scrapely_client_python-1.0.8 → scrapely_client_python-1.0.9}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "uv_build"
 
 [project]
 name = "scrapely-client-python"
-version = "1.0.8"
+version = "1.0.9"
 description = "Scrapely API client for Python"
 license = "Apache-2.0"
 license-files = [

{scrapely_client_python-1.0.8 → scrapely_client_python-1.0.9}/pyproject.toml.orig

@@ -4,7 +4,7 @@ build-backend = "uv_build"
 
 [project]
 name = "scrapely-client-python"
-version = "1.0.8"
+version = "1.0.9"
 description = "Scrapely API client for Python"
 authors = [{ name = "TeraShift GmbH (Scrape.ly)" }]
 license = "Apache-2.0"

{scrapely_client_python-1.0.8 → scrapely_client_python-1.0.9}/src/scrapely_client/_consts.py

@@ -7,6 +7,9 @@ from datetime import timedelta
 DEFAULT_API_URL = 'https://api.scrape.ly'
 """Default base URL for the Scrapely API."""
 
+API_URL_ENV_VAR = 'SCRAPELY_API_URL'
+"""Environment variable that sets the API URL when `api_url` is not passed to the client."""
+
 API_VERSION = 'v2'
 """Current Scrapely API version."""
 

{scrapely_client_python-1.0.8 → scrapely_client_python-1.0.9}/src/scrapely_client/_scrapely_client.py

@@ -2,12 +2,14 @@
 # Modified by TeraShift GmbH. See the LICENSE and NOTICE files.
 from __future__ import annotations
 
+import os
 from functools import cached_property
 from typing import TYPE_CHECKING
 
 from scrapely_client._charging import ChargingHooks, wrap_dataset_client_with_charging
 from scrapely_client._client_registry import ClientRegistry, ClientRegistryAsync
 from scrapely_client._consts import (
+    API_URL_ENV_VAR,
     API_VERSION,
     DEFAULT_API_URL,
     DEFAULT_MAX_RETRIES,
@@ -90,6 +92,13 @@ if TYPE_CHECKING:
     from scrapely_client.types import HttpCompressionAlgorithm
 
 
+def _resolve_api_url(api_url: str | None) -> str:
+    """Return `api_url` if given, else the `SCRAPELY_API_URL` environment variable, else the production API."""
+    if api_url is not None:
+        return api_url
+    return os.environ.get(API_URL_ENV_VAR, '').strip() or DEFAULT_API_URL
+
+
 @docs_group('Scrapely API clients')
 class ApifyClient:
     """Synchronous client for the Scrapely API.
@@ -124,7 +133,7 @@ class ApifyClient:
         self,
         token: str | None = None,
         *,
-        api_url: str = DEFAULT_API_URL,
+        api_url: str | None = None,
         api_public_url: str | None = DEFAULT_API_URL,
         max_retries: int = DEFAULT_MAX_RETRIES,
         min_delay_between_retries: timedelta = DEFAULT_MIN_DELAY_BETWEEN_RETRIES,
@@ -144,9 +153,9 @@ class ApifyClient:
         Args:
             token: The Scrapely API token. You can find your token on the
                 Integrations page in the Scrapely dashboard.
-            api_url: The URL of the Scrapely API server to connect to. Defaults to https://api.scrape.ly. It can
-                be an internal URL that is not globally accessible, in which case `api_public_url` should be set
-                as well.
+            api_url: The URL of the Scrapely API server to connect to. When not given, the `SCRAPELY_API_URL`
+                environment variable is used, then https://api.scrape.ly. It can be an internal URL that is not
+                globally accessible, in which case `api_public_url` should be set as well.
             api_public_url: The globally accessible URL of the Scrapely API server. Should be set only if `api_url`
                 is an internal URL that is not globally accessible. Defaults to https://api.scrape.ly.
             max_retries: How many times to retry a failed request at most.
@@ -165,7 +174,7 @@ class ApifyClient:
             default_dataset_id: ID of the actor run's default dataset, used to gate charging interception.
         """
         # We need to do this because of mocking in tests and default mutable arguments.
-        api_url = DEFAULT_API_URL if api_url is None else api_url
+        api_url = _resolve_api_url(api_url)
         api_public_url = DEFAULT_API_URL if api_public_url is None else api_public_url
 
         if headers:
@@ -234,7 +243,7 @@ class ApifyClient:
         cls,
         token: str | None = None,
         *,
-        api_url: str = DEFAULT_API_URL,
+        api_url: str | None = None,
         api_public_url: str | None = DEFAULT_API_URL,
         http_client: HttpClient,
     ) -> ApifyClient:
@@ -263,7 +272,8 @@ class ApifyClient:
         Args:
             token: The Scrapely API token. It is set as the `Authorization` header on the custom client,
                 unless the client already has one configured.
-            api_url: The URL of the Scrapely API server to connect to. Defaults to https://api.scrape.ly.
+            api_url: The URL of the Scrapely API server to connect to. When not given, the `SCRAPELY_API_URL`
+                environment variable is used, then https://api.scrape.ly.
             api_public_url: The globally accessible URL of the Scrapely API server. Defaults to https://api.scrape.ly.
             http_client: A custom HTTP client instance extending `HttpClient`.
         """
@@ -513,7 +523,7 @@ class ApifyClientAsync:
         self,
         token: str | None = None,
         *,
-        api_url: str = DEFAULT_API_URL,
+        api_url: str | None = None,
         api_public_url: str | None = DEFAULT_API_URL,
         max_retries: int = DEFAULT_MAX_RETRIES,
         min_delay_between_retries: timedelta = DEFAULT_MIN_DELAY_BETWEEN_RETRIES,
@@ -533,9 +543,9 @@ class ApifyClientAsync:
         Args:
             token: The Scrapely API token. You can find your token on the
                 Integrations page in the Scrapely dashboard.
-            api_url: The URL of the Scrapely API server to connect to. Defaults to https://api.scrape.ly. It can
-                be an internal URL that is not globally accessible, in which case `api_public_url` should be set
-                as well.
+            api_url: The URL of the Scrapely API server to connect to. When not given, the `SCRAPELY_API_URL`
+                environment variable is used, then https://api.scrape.ly. It can be an internal URL that is not
+                globally accessible, in which case `api_public_url` should be set as well.
             api_public_url: The globally accessible URL of the Scrapely API server. Should be set only if `api_url`
                 is an internal URL that is not globally accessible. Defaults to https://api.scrape.ly.
             max_retries: How many times to retry a failed request at most.
@@ -554,7 +564,7 @@ class ApifyClientAsync:
             default_dataset_id: ID of the actor run's default dataset, used to gate charging interception.
         """
         # We need to do this because of mocking in tests and default mutable arguments.
-        api_url = DEFAULT_API_URL if api_url is None else api_url
+        api_url = _resolve_api_url(api_url)
         api_public_url = DEFAULT_API_URL if api_public_url is None else api_public_url
 
         if headers:
@@ -623,7 +633,7 @@ class ApifyClientAsync:
         cls,
         token: str | None = None,
         *,
-        api_url: str = DEFAULT_API_URL,
+        api_url: str | None = None,
         api_public_url: str | None = DEFAULT_API_URL,
         http_client: HttpClientAsync,
     ) -> ApifyClientAsync:
@@ -652,7 +662,8 @@ class ApifyClientAsync:
         Args:
             token: The Scrapely API token. It is set as the `Authorization` header on the custom client,
                 unless the client already has one configured.
-            api_url: The URL of the Scrapely API server to connect to. Defaults to https://api.scrape.ly.
+            api_url: The URL of the Scrapely API server to connect to. When not given, the `SCRAPELY_API_URL`
+                environment variable is used, then https://api.scrape.ly.
             api_public_url: The globally accessible URL of the Scrapely API server. Defaults to https://api.scrape.ly.
             http_client: A custom HTTP client instance extending `HttpClientAsync`.
         """