apify 2.7.1b19__tar.gz → 2.7.1b20__tar.gz

{apify-2.7.1b19 → apify-2.7.1b20}/CHANGELOG.md

@@ -30,6 +30,7 @@ All notable changes to this project will be documented in this file.
 - [**breaking**] Replace `httpx` with `impit` ([#560](https://github.com/apify/apify-sdk-python/pull/560)) ([cca3869](https://github.com/apify/apify-sdk-python/commit/cca3869e85968865e56aafcdcb36fbccba27aef0)) by [@Mantisus](https://github.com/Mantisus), closes [#558](https://github.com/apify/apify-sdk-python/issues/558)
 - [**breaking**] Remove `Request.id` field ([#553](https://github.com/apify/apify-sdk-python/pull/553)) ([445ab5d](https://github.com/apify/apify-sdk-python/commit/445ab5d752b785fc2018b35c8adbe779253d7acd)) by [@Pijukatel](https://github.com/Pijukatel)
 - [**breaking**] Make `Actor` initialization stricter and more predictable ([#576](https://github.com/apify/apify-sdk-python/pull/576)) ([912222a](https://github.com/apify/apify-sdk-python/commit/912222a7a8123be66c94c50a2e461276fbfc50c4)) by [@Pijukatel](https://github.com/Pijukatel)
+- [**breaking**] Make default Apify storages use alias mechanism ([#606](https://github.com/apify/apify-sdk-python/pull/606)) ([dbea7d9](https://github.com/apify/apify-sdk-python/commit/dbea7d97fe7f25aa8658a32c5bb46a3800561df5)) by [@Pijukatel](https://github.com/Pijukatel), closes [#599](https://github.com/apify/apify-sdk-python/issues/599)
 
 
 <!-- git-cliff-unreleased-end -->
@@ -165,6 +166,18 @@ All notable changes to this project will be documented in this file.
 - Tagline overlap ([#501](https://github.com/apify/apify-sdk-python/pull/501)) ([bae8340](https://github.com/apify/apify-sdk-python/commit/bae8340c46fea756ea35ea4d591da84c09d478e2)) by [@katzino](https://github.com/katzino)
 
 
+## [2.7.0](https://github.com/apify/apify-sdk-python/releases/tag/v2.7.0) (2025-07-14)
+
+### 🚀 Features
+
+- **crypto:** Decrypt secret objects ([#482](https://github.com/apify/apify-sdk-python/pull/482)) ([ce9daf7](https://github.com/apify/apify-sdk-python/commit/ce9daf7381212b8dc194e8a643e5ca0dedbc0078)) by [@MFori](https://github.com/MFori)
+
+### 🐛 Bug Fixes
+
+- Sync `@docusaurus` theme version [internal] ([#500](https://github.com/apify/apify-sdk-python/pull/500)) ([a7485e7](https://github.com/apify/apify-sdk-python/commit/a7485e7d2276fde464ce862573d5b95e7d4d836a)) by [@katzino](https://github.com/katzino)
+- Tagline overlap ([#501](https://github.com/apify/apify-sdk-python/pull/501)) ([bae8340](https://github.com/apify/apify-sdk-python/commit/bae8340c46fea756ea35ea4d591da84c09d478e2)) by [@katzino](https://github.com/katzino)
+
+
 
 ## [2.7.3](https://github.com/apify/apify-sdk-python/releases/tag/v2.7.3) (2025-08-11)
 

{apify-2.7.1b19 → apify-2.7.1b20}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: apify
-Version: 2.7.1b19
+Version: 2.7.1b20
 Summary: Apify SDK for Python
 Project-URL: Apify Homepage, https://apify.com
 Project-URL: Changelog, https://docs.apify.com/sdk/python/docs/changelog

{apify-2.7.1b19 → apify-2.7.1b20}/docs/02_concepts/12_pay_per_event.mdx

@@ -13,6 +13,12 @@ Apify provides several [pricing models](https://docs.apify.com/platform/actors/p
 
 To use the pay-per-event pricing model, you first need to [set it up](https://docs.apify.com/platform/actors/running/actors-in-store#pay-per-event) for your Actor in the Apify console. After that, you're free to start charging for events.
 
+:::info How pay-per-event pricing works
+
+If you want more details about PPE pricing, please refer to our [PPE documentation](https://docs.apify.com/platform/actors/publishing/monetize/pay-per-event).
+
+:::
+
 ## Charging for events
 
 After monetization is set in the Apify console, you can add <ApiLink to="class/Actor#charge">`Actor.charge`</ApiLink> calls to your code and start monetizing!

apify-2.7.1b20/docs/04_upgrading/upgrading_to_v3.md

@@ -0,0 +1,121 @@
+---
+id: upgrading-to-v3
+title: Upgrading to v3
+---
+
+This page summarizes the breaking changes between Apify Python SDK v2.x and v3.0.
+
+## Python version support
+
+Support for Python 3.9 has been dropped. The Apify Python SDK v3.x now requires Python 3.10 or later. Make sure your environment is running a compatible version before upgrading.
+
+## Changes in storages
+
+Apify Python SDK v3.0 includes Crawlee v1.0, which brings significant changes to the storage APIs. In Crawlee v1.0, the `Dataset`, `KeyValueStore`, and `RequestQueue` storage APIs have been updated for consistency and simplicity. Below is a detailed overview of what's new, what's changed, and what's been removed.
+
+See the Crawlee's [Storages guide](https://crawlee.dev/python/docs/guides/storages) for more details.
+
+### Dataset
+
+The `Dataset` API now includes several new methods, such as:
+
+- `get_metadata` - retrieves metadata information for the dataset.
+- `purge` - completely clears the dataset, including all items (keeps the metadata only).
+- `list_items` - returns the dataset's items in a list format.
+
+Some older methods have been removed or replaced:
+
+- `from_storage_object` constructor has been removed. You should now use the `open` method with either a `name` or `id` parameter.
+- `get_info` method and the `storage_object` property have been replaced by the new `get_metadata` method.
+- `set_metadata` method has been removed.
+- `write_to_json` and `write_to_csv` methods have been removed; instead, use the `export_to` method for exporting data in different formats.
+
+### Key-value store
+
+The `KeyValueStore` API now includes several new methods, such as:
+
+- `get_metadata` - retrieves metadata information for the key-value store.
+- `purge` - completely clears the key-value store, removing all keys and values (keeps the metadata only).
+- `delete_value` - deletes a specific key and its associated value.
+- `list_keys` - lists all keys in the key-value store.
+
+Some older methods have been removed or replaced:
+
+- `from_storage_object` - removed; use the `open` method with either a `name` or `id` instead.
+- `get_info` and `storage_object` - replaced by the new `get_metadata` method.
+- `set_metadata` method has been removed.
+
+### Request queue
+
+The `RequestQueue` API now includes several new methods, such as:
+
+- `get_metadata` - retrieves metadata information for the request queue.
+- `purge` - completely clears the request queue, including all pending and processed requests (keeps the metadata only).
+- `add_requests` - replaces the previous `add_requests_batched` method, offering the same functionality under a simpler name.
+
+Some older methods have been removed or replaced:
+
+- `from_storage_object` - removed; use the `open` method with either a `name` or `id` instead.
+- `get_info` and `storage_object` - replaced by the new `get_metadata` method.
+- `get_request` has argument `unique_key` instead of `request_id` as the `id` field was removed from the `Request`.
+- `set_metadata` method has been removed.
+
+Some changes in the related model classes:
+
+- `resource_directory` in `RequestQueueMetadata` - removed; use the corresponding `path_to_*` property instead.
+- `stats` field in `RequestQueueMetadata` - removed as it was unused.
+- `RequestQueueHead` - replaced by `RequestQueueHeadWithLocks`.
+
+## Removed Actor.config property
+- `Actor.config` property has been removed. Use `Actor.configuration` instead.
+
+## Actor initialization and ServiceLocator changes
+
+`Actor` initialization and global `service_locator` services setup is more strict and predictable.
+- Services in `Actor` can't be changed after calling `Actor.init`, entering the `async with Actor` context manager or after requesting them from the `Actor`.
+- Services in `Actor` can be different from services in Crawler.
+
+
+**Now (v3.0):**
+
+```python
+from crawlee.crawlers import BasicCrawler
+from crawlee.storage_clients import MemoryStorageClient
+from crawlee.configuration import Configuration
+from crawlee.events import LocalEventManager
+from apify import Actor
+
+async def main():
+
+    async with Actor():
+        # This crawler will use same services as Actor and global service_locator
+        crawler_1 = BasicCrawler()
+
+        # This crawler will use custom services
+        custom_configuration = Configuration()
+        custom_event_manager = LocalEventManager.from_config(custom_configuration)
+        custom_storage_client = MemoryStorageClient()
+        crawler_2 = BasicCrawler(
+            configuration=custom_configuration,
+            event_manager=custom_event_manager,
+            storage_client=custom_storage_client,
+        )
+```
+
+## Removed Actor.config property
+- `Actor.config` property has been removed. Use `Actor.configuration` instead.
+
+## Default storage ids in configuration changed to None
+- `Configuration.default_key_value_store_id` changed from `'default'` to `None`.
+- `Configuration.default_dataset_id` changed from `'default'` to `None`.
+- `Configuration.default_request_queue_id` changed from `'default'` to `None`.
+
+Previously using the default storage without specifying its `id` in `Configuration` would lead to using specific storage with id `'default'`. Now it will use newly created unnamed storage with `'id'` assigned by the Apify platform, consecutive calls to get the default storage will return the same storage.
+
+## Storages
+
+<!-- TODO -->
+
+## Storage clients
+
+<!-- TODO -->

{apify-2.7.1b19 → apify-2.7.1b20}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
 
 [project]
 name = "apify"
-version = "2.7.1b19"
+version = "2.7.1b20"
 description = "Apify SDK for Python"
 authors = [{ name = "Apify Technologies s.r.o.", email = "support@apify.com" }]
 license = { file = "LICENSE" }

{apify-2.7.1b19 → apify-2.7.1b20}/src/apify/_actor.py

@@ -324,9 +324,6 @@ class _ActorType:
         self.log.info('Initializing Actor...')
         self.log.info('System info', extra=get_system_info())
 
-        # TODO: Print outdated SDK version warning (we need a new env var for this)
-        # https://github.com/apify/apify-sdk-python/issues/146
-
         await self.event_manager.__aenter__()
         self.log.debug('Event manager initialized')
 

{apify-2.7.1b19 → apify-2.7.1b20}/src/apify/_configuration.py

@@ -142,7 +142,7 @@ class Configuration(CrawleeConfiguration):
     ] = None
 
     default_dataset_id: Annotated[
-        str,
+        str | None,
         Field(
             validation_alias=AliasChoices(
                 'actor_default_dataset_id',
@@ -150,10 +150,10 @@ class Configuration(CrawleeConfiguration):
             ),
             description='Default dataset ID used by the Apify storage client when no ID or name is provided.',
         ),
-    ] = 'default'
+    ] = None
 
     default_key_value_store_id: Annotated[
-        str,
+        str | None,
         Field(
             validation_alias=AliasChoices(
                 'actor_default_key_value_store_id',
@@ -161,10 +161,10 @@ class Configuration(CrawleeConfiguration):
             ),
             description='Default key-value store ID for the Apify storage client when no ID or name is provided.',
         ),
-    ] = 'default'
+    ] = None
 
     default_request_queue_id: Annotated[
-        str,
+        str | None,
         Field(
             validation_alias=AliasChoices(
                 'actor_default_request_queue_id',
@@ -172,7 +172,7 @@ class Configuration(CrawleeConfiguration):
             ),
             description='Default request queue ID for the Apify storage client when no ID or name is provided.',
         ),
-    ] = 'default'
+    ] = None
 
     disable_outdated_warning: Annotated[
         bool,

{apify-2.7.1b19 → apify-2.7.1b20}/src/apify/storage_clients/_apify/_dataset_client.py

@@ -124,8 +124,10 @@ class ApifyDatasetClient(DatasetClient):
         )
         apify_datasets_client = apify_client_async.datasets()
 
-        # Normalize 'default' alias to None
-        alias = None if alias == 'default' else alias
+        # Normalize unnamed default storage in cases where not defined in `configuration.default_dataset_id` to unnamed
+        # storage aliased as `__default__`
+        if not any([alias, name, id, configuration.default_dataset_id]):
+            alias = '__default__'
 
         if alias:
             # Check if there is pre-existing alias mapping in the default KVS.
@@ -150,6 +152,11 @@ class ApifyDatasetClient(DatasetClient):
         # If none are provided, try to get the default storage ID from environment variables.
         elif id is None:
             id = configuration.default_dataset_id
+            if not id:
+                raise ValueError(
+                    'Dataset "id", "name", or "alias" must be specified, '
+                    'or a default dataset ID must be set in the configuration.'
+                )
 
         # Now create the client for the determined ID
         apify_dataset_client = apify_client_async.dataset(dataset_id=id)

{apify-2.7.1b19 → apify-2.7.1b20}/src/apify/storage_clients/_apify/_key_value_store_client.py

@@ -115,8 +115,10 @@ class ApifyKeyValueStoreClient(KeyValueStoreClient):
         )
         apify_kvss_client = apify_client_async.key_value_stores()
 
-        # Normalize 'default' alias to None
-        alias = None if alias == 'default' else alias
+        # Normalize unnamed default storage in cases where not defined in `configuration.default_key_value_store_id` to
+        # unnamed storage aliased as `__default__`
+        if not any([alias, name, id, configuration.default_key_value_store_id]):
+            alias = '__default__'
 
         if alias:
             # Check if there is pre-existing alias mapping in the default KVS.
@@ -142,6 +144,11 @@ class ApifyKeyValueStoreClient(KeyValueStoreClient):
         # If none are provided, try to get the default storage ID from environment variables.
         elif id is None:
             id = configuration.default_key_value_store_id
+            if not id:
+                raise ValueError(
+                    'KeyValueStore "id", "name", or "alias" must be specified, '
+                    'or a default KeyValueStore ID must be set in the configuration.'
+                )
 
         # Now create the client for the determined ID
         apify_kvs_client = apify_client_async.key_value_store(key_value_store_id=id)

{apify-2.7.1b19 → apify-2.7.1b20}/src/apify/storage_clients/_apify/_request_queue_client.py

@@ -200,8 +200,10 @@ class ApifyRequestQueueClient(RequestQueueClient):
         )
         apify_rqs_client = apify_client_async.request_queues()
 
-        # Normalize 'default' alias to None
-        alias = None if alias == 'default' else alias
+        # Normalize unnamed default storage in cases where not defined in `configuration.default_request_queue_id` to
+        # unnamed storage aliased as `__default__`
+        if not any([alias, name, id, configuration.default_request_queue_id]):
+            alias = '__default__'
 
         if alias:
             # Check if there is pre-existing alias mapping in the default KVS.
@@ -226,6 +228,11 @@ class ApifyRequestQueueClient(RequestQueueClient):
         # If none are provided, try to get the default storage ID from environment variables.
         elif id is None:
             id = configuration.default_request_queue_id
+            if not id:
+                raise ValueError(
+                    'RequestQueue "id", "name", or "alias" must be specified, '
+                    'or a default default_request_queue_id ID must be set in the configuration.'
+                )
 
         # Use suitable client_key to make `hadMultipleClients` response of Apify API useful.
         # It should persist across migrated or resurrected Actor runs on the Apify platform.

{apify-2.7.1b19 → apify-2.7.1b20}/src/apify/storage_clients/_apify/_utils.py

@@ -76,7 +76,7 @@ class AliasResolver:
         Returns:
             Map of aliases and storage ids.
         """
-        if not cls._alias_map:
+        if not cls._alias_map and Configuration.get_global_configuration().is_at_home:
             default_kvs_client = await _get_default_kvs_client()
 
             record = await default_kvs_client.get_record(cls._ALIAS_MAPPING_KEY)
@@ -156,7 +156,8 @@ async def _get_default_kvs_client() -> KeyValueStoreClientAsync:
         min_delay_between_retries_millis=500,
         timeout_secs=360,
     )
-
+    if not configuration.default_key_value_store_id:
+        raise ValueError("'Configuration.default_key_value_store_id' must be set.")
     return apify_client_async.key_value_store(key_value_store_id=configuration.default_key_value_store_id)
 
 

apify-2.7.1b20/tests/integration/test_apify_storages.py

@@ -0,0 +1,75 @@
+import asyncio
+
+import pytest
+
+from crawlee import service_locator
+from crawlee.storages import Dataset, KeyValueStore, RequestQueue
+
+from apify import Actor, Configuration
+from apify.storage_clients import ApifyStorageClient
+
+
+@pytest.mark.parametrize(
+    'storage_type',
+    [Dataset, KeyValueStore, RequestQueue],
+)
+async def test_alias_concurrent_creation_local(
+    storage_type: Dataset | KeyValueStore | RequestQueue, apify_token: str
+) -> None:
+    """Test that storages created with same alias are created only once even when created concurrently."""
+    service_locator.set_configuration(Configuration(token=apify_token))
+    service_locator.set_storage_client(ApifyStorageClient())
+    tasks = [asyncio.create_task(storage_type.open(alias='test')) for _ in range(2)]
+
+    storages = await asyncio.gather(*tasks)
+    unique_storage_ids = {storage.id for storage in storages}
+    try:
+        # Only one aliased storage should be created.
+        assert len(unique_storage_ids) == 1
+
+        # Clean up
+        await storages[0].drop()
+    except AssertionError:
+        for storage in storages:
+            await storage.drop()
+
+
+@pytest.mark.parametrize(
+    'storage_type',
+    [Dataset, KeyValueStore, RequestQueue],
+)
+async def test_unnamed_default_without_config(
+    storage_type: Dataset | KeyValueStore | RequestQueue, apify_token: str
+) -> None:
+    """Test that default Apify storage used locally is unnamed storage."""
+    service_locator.set_configuration(Configuration(token=apify_token))
+    service_locator.set_storage_client(ApifyStorageClient())
+
+    # Open storage and make sure it has no name and it has id
+    storage = await storage_type.open()
+    assert storage.name is None
+    assert storage.id
+
+    # Make sure the same instance is returned when opened again without name or alias
+    storage_again = await storage_type.open()
+    assert storage is storage_again
+
+    await storage.drop()
+
+
+@pytest.mark.parametrize(
+    'storage_type',
+    [Dataset, KeyValueStore, RequestQueue],
+)
+async def test_aliases_not_stored_on_platform_when_local(
+    storage_type: Dataset | KeyValueStore | RequestQueue, apify_token: str
+) -> None:
+    """Test that default Apify storage used locally is not persisting aliases to Apify based default KVS."""
+    service_locator.set_configuration(Configuration(token=apify_token))
+    service_locator.set_storage_client(ApifyStorageClient())
+    async with Actor(configure_logging=False):
+        await storage_type.open(alias='test')
+        default_kvs = await Actor.open_key_value_store(force_cloud=True)
+
+        # The default KVS should be empty
+        assert len(await default_kvs.list_keys()) == 0