async-durable-execution 2.2.0__tar.gz → 2.4.0__tar.gz

{async_durable_execution-2.2.0 → async_durable_execution-2.4.0}/PKG-INFO

@@ -1,6 +1,6 @@
-Metadata-Version: 2.4
+Metadata-Version: 2.5
 Name: async-durable-execution
-Version: 2.2.0
+Version: 2.4.0
 Summary: Community-maintained durable execution SDK for AWS Lambda in Python
 Project-URL: Documentation, https://github.com/zhongkechen/async-durable-execution#readme
 Project-URL: Issues, https://github.com/zhongkechen/async-durable-execution/issues
@@ -17,12 +17,15 @@ Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Programming Language :: Python :: 3.14
+Classifier: Programming Language :: Python :: 3.15
 Classifier: Programming Language :: Python :: Implementation :: CPython
 Classifier: Programming Language :: Python :: Implementation :: PyPy
 Requires-Python: >=3.10
 Requires-Dist: botocore<1.43.1,>=1.42.90
 Provides-Extra: aioboto
-Requires-Dist: aiobotocore; extra == 'aioboto'
+Requires-Dist: httpx<1,>=0.28.1; extra == 'aioboto'
+Provides-Extra: httpx
+Requires-Dist: httpx<1,>=0.28.1; extra == 'httpx'
 Provides-Extra: typing
 Requires-Dist: botocore-stubs<1.43.0,>=1.42.40; extra == 'typing'
 Description-Content-Type: text/markdown
@@ -72,11 +75,13 @@ composition, and APIs designed for modern Python applications.
 
 - **[Async-first durable code](https://zhongkechen.github.io/async-durable-execution/official-python-sdk-comparison.html#programming-model)** - Compared with the official AWS SDK, user-provided durable handlers, steps, child contexts, `flow` nodes, callback submitters, map item functions, parallel branches, and wait-for-condition checks are written with `async def`.
 - **[Operations not available in the official SDK](https://zhongkechen.github.io/async-durable-execution/api/extension/replay_safe.html)** - This SDK adds [replay-safe helpers](https://zhongkechen.github.io/async-durable-execution/api/extension/replay_safe.html) (`random()`, `now()`, `timestamp()`, and `uuid()`) and [durable self-invocation](https://zhongkechen.github.io/async-durable-execution/api/extension/recurse.html) (`recurse()`).
+- **[Stable custom operation SPI](https://zhongkechen.github.io/async-durable-execution/custom-operations.html)** - Third-party packages can reserve opaque deterministic primitive identities, use custom subtypes, and build stateful replay-safe operations without importing SDK internals.
 - **[Declarative DAG workflows](https://zhongkechen.github.io/async-durable-execution/api/extension/flow.html#quick-start)** - Define acyclic workflows with typed node inputs, inferred or conditional dependencies, failure routes, and durable operations inside each node. The SDK validates the graph before execution and skips nodes that are not required by the selected outputs.
 - **[Background operation tasks](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html#background-operation-tasks)** - Durable operations such as `step(...)`, `wait(...)`, `invoke(...)`, `recurse(...)`, `run_in_child_context(...)`, and `flow(...)` return `asyncio.Task` objects, so independent operations can run in the background and be awaited together with `asyncio.gather` without using `parallel()` or `map()`.
 - **[Pythonic operation parameters](https://zhongkechen.github.io/async-durable-execution/migrating-from-official-python-sdk.html#api-mapping)** - Operations use direct keyword arguments, standard Python types such as `datetime.timedelta`, and keyword-only names instead of configuration wrapper objects.
+- **[Composable SerDes pipelines](https://zhongkechen.github.io/async-durable-execution/serdes-pipelines.html)** - Chain async string transformations and offload large checkpoint payloads to EFS or S3 Files with bounded previews and validated immutable storage.
 - **[Integrated local and cloud runner](https://zhongkechen.github.io/async-durable-execution/api/runner.html)** - Runner functionality now ships through `async_durable_execution`, with separate local and cloud runner factories and typed test result helpers.
-- **[Async Lambda client support](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html#lambda-client-selection)** - Install the optional `aioboto` extra to use an async Lambda client; otherwise the SDK uses the bundled sync client through an async adapter.
+- **[Model-free Lambda clients](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html#lambda-client-selection)** - The SDK owns its Lambda REST wire format instead of depending on botocore service models. Install the optional `httpx` extra to send requests with HTTPX; otherwise the same requests use botocore's synchronous HTTP transport through an async adapter.
 - **[Replay-aware logging with stdlib logging](https://zhongkechen.github.io/async-durable-execution/migrating-from-official-python-sdk.html#logging)** - Standard `logging` loggers are enriched by durable context filtering so workflow logs remain replay safe.
 - **[Lambda layer packaging](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html#lambda-layer-packaging)** - The repo includes tooling and workflows to build and publish an SDK Lambda layer for functions that do not vendor dependencies directly.
 
@@ -88,13 +93,20 @@ Install the execution SDK:
 pip install async-durable-execution
 ```
 
-For an async Lambda service client, install the optional `aioboto` extra:
+For an async Lambda service client, install the optional `httpx` extra:
 
 ```console
-pip install "async-durable-execution[aioboto]"
+pip install "async-durable-execution[httpx]"
 ```
 
-The `aioboto` extra installs `aiobotocore`, which lets the SDK create an async Lambda client for durable checkpoint and state APIs. Without it, the SDK uses the bundled `botocore` dependency through a threaded async adapter.
+The `httpx` extra installs HTTPX, which the SDK uses for asynchronous
+model-free Lambda REST calls. Without it, the SDK sends the same signed
+requests with botocore's synchronous HTTP transport through a threaded async
+adapter. Botocore continues to provide AWS credentials, endpoint metadata, and
+SigV4 signing, but its generated Lambda service model is not used.
+
+The previous `aioboto` extra remains available as a backward-compatible alias
+for `httpx`.
 
 Create a durable Lambda handler:
 
@@ -247,6 +259,7 @@ For the developer workflow to run or deploy example integration tests, see the [
 - **[Deploy and Invoke](https://zhongkechen.github.io/async-durable-execution/deployment.html)** - Configure IAM, qualified function identifiers, invocations, CloudFormation, and SAM
 - **[Using Synchronous Code](https://zhongkechen.github.io/async-durable-execution/using-synchronous-code.html)** - Wrap existing synchronous business logic and blocking clients safely
 - **[Advanced Usage](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html)** - Explore background operation tasks, batch completion conditions, Lambda clients, and Lambda layers
+- **[Custom Durable Operations](https://zhongkechen.github.io/async-durable-execution/custom-operations.html)** - Build third-party durable operation libraries on the stable extension-author interface
 - **[Runner Architecture](https://zhongkechen.github.io/async-durable-execution/runner-architecture.html)** - Local and cloud runner execution flow, components, and diagrams
 - **[Contributing Guide](https://github.com/zhongkechen/async-durable-execution/blob/main/CONTRIBUTING.md)** - Development workflow, Hatch commands, testing, and pull request guidance
 

{async_durable_execution-2.2.0 → async_durable_execution-2.4.0}/README.md

@@ -43,11 +43,13 @@ composition, and APIs designed for modern Python applications.
 
 - **[Async-first durable code](https://zhongkechen.github.io/async-durable-execution/official-python-sdk-comparison.html#programming-model)** - Compared with the official AWS SDK, user-provided durable handlers, steps, child contexts, `flow` nodes, callback submitters, map item functions, parallel branches, and wait-for-condition checks are written with `async def`.
 - **[Operations not available in the official SDK](https://zhongkechen.github.io/async-durable-execution/api/extension/replay_safe.html)** - This SDK adds [replay-safe helpers](https://zhongkechen.github.io/async-durable-execution/api/extension/replay_safe.html) (`random()`, `now()`, `timestamp()`, and `uuid()`) and [durable self-invocation](https://zhongkechen.github.io/async-durable-execution/api/extension/recurse.html) (`recurse()`).
+- **[Stable custom operation SPI](https://zhongkechen.github.io/async-durable-execution/custom-operations.html)** - Third-party packages can reserve opaque deterministic primitive identities, use custom subtypes, and build stateful replay-safe operations without importing SDK internals.
 - **[Declarative DAG workflows](https://zhongkechen.github.io/async-durable-execution/api/extension/flow.html#quick-start)** - Define acyclic workflows with typed node inputs, inferred or conditional dependencies, failure routes, and durable operations inside each node. The SDK validates the graph before execution and skips nodes that are not required by the selected outputs.
 - **[Background operation tasks](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html#background-operation-tasks)** - Durable operations such as `step(...)`, `wait(...)`, `invoke(...)`, `recurse(...)`, `run_in_child_context(...)`, and `flow(...)` return `asyncio.Task` objects, so independent operations can run in the background and be awaited together with `asyncio.gather` without using `parallel()` or `map()`.
 - **[Pythonic operation parameters](https://zhongkechen.github.io/async-durable-execution/migrating-from-official-python-sdk.html#api-mapping)** - Operations use direct keyword arguments, standard Python types such as `datetime.timedelta`, and keyword-only names instead of configuration wrapper objects.
+- **[Composable SerDes pipelines](https://zhongkechen.github.io/async-durable-execution/serdes-pipelines.html)** - Chain async string transformations and offload large checkpoint payloads to EFS or S3 Files with bounded previews and validated immutable storage.
 - **[Integrated local and cloud runner](https://zhongkechen.github.io/async-durable-execution/api/runner.html)** - Runner functionality now ships through `async_durable_execution`, with separate local and cloud runner factories and typed test result helpers.
-- **[Async Lambda client support](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html#lambda-client-selection)** - Install the optional `aioboto` extra to use an async Lambda client; otherwise the SDK uses the bundled sync client through an async adapter.
+- **[Model-free Lambda clients](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html#lambda-client-selection)** - The SDK owns its Lambda REST wire format instead of depending on botocore service models. Install the optional `httpx` extra to send requests with HTTPX; otherwise the same requests use botocore's synchronous HTTP transport through an async adapter.
 - **[Replay-aware logging with stdlib logging](https://zhongkechen.github.io/async-durable-execution/migrating-from-official-python-sdk.html#logging)** - Standard `logging` loggers are enriched by durable context filtering so workflow logs remain replay safe.
 - **[Lambda layer packaging](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html#lambda-layer-packaging)** - The repo includes tooling and workflows to build and publish an SDK Lambda layer for functions that do not vendor dependencies directly.
 
@@ -59,13 +61,20 @@ Install the execution SDK:
 pip install async-durable-execution
 ```
 
-For an async Lambda service client, install the optional `aioboto` extra:
+For an async Lambda service client, install the optional `httpx` extra:
 
 ```console
-pip install "async-durable-execution[aioboto]"
+pip install "async-durable-execution[httpx]"
 ```
 
-The `aioboto` extra installs `aiobotocore`, which lets the SDK create an async Lambda client for durable checkpoint and state APIs. Without it, the SDK uses the bundled `botocore` dependency through a threaded async adapter.
+The `httpx` extra installs HTTPX, which the SDK uses for asynchronous
+model-free Lambda REST calls. Without it, the SDK sends the same signed
+requests with botocore's synchronous HTTP transport through a threaded async
+adapter. Botocore continues to provide AWS credentials, endpoint metadata, and
+SigV4 signing, but its generated Lambda service model is not used.
+
+The previous `aioboto` extra remains available as a backward-compatible alias
+for `httpx`.
 
 Create a durable Lambda handler:
 
@@ -218,6 +227,7 @@ For the developer workflow to run or deploy example integration tests, see the [
 - **[Deploy and Invoke](https://zhongkechen.github.io/async-durable-execution/deployment.html)** - Configure IAM, qualified function identifiers, invocations, CloudFormation, and SAM
 - **[Using Synchronous Code](https://zhongkechen.github.io/async-durable-execution/using-synchronous-code.html)** - Wrap existing synchronous business logic and blocking clients safely
 - **[Advanced Usage](https://zhongkechen.github.io/async-durable-execution/advanced-usage.html)** - Explore background operation tasks, batch completion conditions, Lambda clients, and Lambda layers
+- **[Custom Durable Operations](https://zhongkechen.github.io/async-durable-execution/custom-operations.html)** - Build third-party durable operation libraries on the stable extension-author interface
 - **[Runner Architecture](https://zhongkechen.github.io/async-durable-execution/runner-architecture.html)** - Local and cloud runner execution flow, components, and diagrams
 - **[Contributing Guide](https://github.com/zhongkechen/async-durable-execution/blob/main/CONTRIBUTING.md)** - Development workflow, Hatch commands, testing, and pull request guidance
 

{async_durable_execution-2.2.0 → async_durable_execution-2.4.0}/async_durable_execution/__about__.py

@@ -2,4 +2,4 @@
 #
 # SPDX-License-Identifier: Apache-2.0
 
-__version__ = "2.2.0"
+__version__ = "2.4.0"

{async_durable_execution-2.2.0 → async_durable_execution-2.4.0}/async_durable_execution/__init__.py

@@ -6,6 +6,7 @@ from .__about__ import __version__
 # Core runtime and supporting public APIs
 from ._core import (
     CallableRuntimeError,
+    ComposableSerDes,
     DurableContext,
     DurableExecutionsError,
     DurableServiceClient,
@@ -22,27 +23,46 @@ from ._core import (
     OperationSubType,
     OperationType,
     RetryStrategy,
+    RetryableSerDesError,
     SerDes,
     SerDesContext,
     SerDesError,
+    SerDesPipelineError,
+    SerDesStage,
     UserlandError,
     ValidationError,
     create_default_sync_client,
+    create_serdes_pipeline,
     durable_callable,
     durable_execution,
     get_current_context,
     get_durable_context,
     get_serdes_context,
+    is_composable_serdes,
+)
+from .filesystem_serdes import (
+    FileSystemPathEncoding,
+    FileSystemSerDesMode,
+    FileSystemSerDesStage,
+    FileSystemSerDesStageConfig,
+    create_file_system_serdes_stage,
+)
+from .preview import (
+    FieldMatchMode,
+    PreviewConfig,
+    PreviewField,
+    PreviewMode,
+    build_preview,
 )
 
 # Durable operations
-from ._extension.with_retry import (
+from ._operation.with_retry import (
     WithRetryContext,
     get_with_retry_context,
     with_retry,
 )
-from ._extension.map import MapItemContext, get_map_item_context, map
-from ._extension.flow import (
+from ._operation.map import MapItemContext, get_map_item_context, map
+from ._operation.flow import (
     FlowDefinitionError,
     FlowExecutionError,
     FlowNode,
@@ -56,7 +76,7 @@ from ._extension.flow import (
     get_node_context,
     node,
 )
-from ._extension.parallel import (
+from ._operation.parallel import (
     BatchItem,
     BatchItemStatus,
     BatchResult,
@@ -66,38 +86,46 @@ from ._extension.parallel import (
     CompletionStatus,
     NestingType,
 )
-from ._extension.wait_for_condition import (
+from ._operation.wait_for_condition import (
     PollingStrategy,
     WaitForConditionCheckContext,
     WaitForConditionError,
     get_wait_for_condition_check_context,
     wait_for_condition,
 )
-from ._primitive.invoke import invoke
-from ._extension.recurse import recurse
-from ._extension.parallel import (
+from ._operation.invoke import invoke
+from ._operation.recurse import recurse
+from ._operation.parallel import (
     parallel,
 )
-from ._primitive.callback import (
+from ._operation.callback import (
     Callback,
     CallbackError,
     create_callback,
 )
-from ._extension.wait_for_callback import (
+from ._operation.wait_for_callback import (
     WaitForCallbackContext,
     get_wait_for_callback_context,
     wait_for_callback,
 )
-from ._primitive.child import SummaryGenerator, run_in_child_context
-from ._primitive.step import (
+from ._operation.child import SummaryGenerator, run_in_child_context
+from ._operation.step import (
     StepContext,
     StepInterruptedError,
     StepSemantics,
     get_step_context,
     step,
 )
-from ._extension.replay_safe import now, random, timestamp, uuid
-from ._primitive.wait import wait
+from ._operation.replay_safe import now, random, timestamp, uuid
+from ._operation.wait import wait
+from .extension import (
+    ExtensionContext,
+    ExtensionOperation,
+    ExtensionStepFunction,
+    ExtensionStepResult,
+    ExtensionStepRetryStrategy,
+    get_extension_context,
+)
 from ._runner import (
     DurableFunctionCloudTestRunner,
     DurableFunctionLocalTestRunner,
@@ -108,10 +136,13 @@ from ._runner import (
 
 __all__ = [
     "__version__",
+    "build_preview",
     "create_callback",
     "create_cloud_runner",
     "create_default_sync_client",
+    "create_file_system_serdes_stage",
     "create_local_runner",
+    "create_serdes_pipeline",
     "durable_callable",
     "durable_dag",
     "durable_execution",
@@ -119,6 +150,7 @@ __all__ = [
     "flow",
     "get_current_context",
     "get_durable_context",
+    "get_extension_context",
     "get_map_item_context",
     "get_node_context",
     "get_serdes_context",
@@ -127,6 +159,7 @@ __all__ = [
     "get_wait_for_condition_check_context",
     "get_with_retry_context",
     "invoke",
+    "is_composable_serdes",
     "map",
     "now",
     "node",
@@ -147,6 +180,7 @@ __all__ = [
     "Callback",
     "CallbackError",
     "CallableRuntimeError",
+    "ComposableSerDes",
     "CompletionConfig",
     "CompletionDecision",
     "CompletionReason",
@@ -159,7 +193,17 @@ __all__ = [
     "DurableExecutionsError",
     "ErrorObject",
     "ExecutionError",
+    "ExtensionContext",
+    "ExtensionOperation",
+    "ExtensionStepFunction",
+    "ExtensionStepResult",
+    "ExtensionStepRetryStrategy",
     "ExtendedTypeSerDes",
+    "FieldMatchMode",
+    "FileSystemPathEncoding",
+    "FileSystemSerDesMode",
+    "FileSystemSerDesStage",
+    "FileSystemSerDesStageConfig",
     "FlowDefinitionError",
     "FlowExecutionError",
     "FlowNode",
@@ -178,10 +222,17 @@ __all__ = [
     "OperationStatus",
     "OperationSubType",
     "OperationType",
+    "PollingStrategy",
+    "PreviewConfig",
+    "PreviewField",
+    "PreviewMode",
     "RetryStrategy",
+    "RetryableSerDesError",
     "SerDes",
     "SerDesContext",
     "SerDesError",
+    "SerDesPipelineError",
+    "SerDesStage",
     "StepContext",
     "StepInterruptedError",
     "StepSemantics",
@@ -192,5 +243,4 @@ __all__ = [
     "WaitForConditionCheckContext",
     "WaitForConditionError",
     "WithRetryContext",
-    "PollingStrategy",
 ]

{async_durable_execution-2.2.0 → async_durable_execution-2.4.0}/async_durable_execution/_core/__init__.py

@@ -3,7 +3,9 @@
 from .client import (
     DurableServiceClient,
     aioboto_is_installed,
+    create_default_async_client,
     create_default_sync_client,
+    httpx_is_installed,
 )
 from .config import (
     Duration,
@@ -15,6 +17,7 @@ from .config import (
 from .context import (
     DurableContext,
     OperationContext,
+    SerDesContext,
     bind_current_context,
     bind_durable_definition,
     ensure_durable_operations_allowed,
@@ -30,6 +33,7 @@ from .exceptions import (
     InvalidStateError,
     InvocationError,
     OrphanedChildException,
+    RetryableSerDesError,
     SerDesError,
     SuspendExecution,
     TerminationReason,
@@ -50,6 +54,8 @@ from .execution import (
     durable_execution,
 )
 from .models import (
+    AwsApiModel,
+    BotoSerializableModel,
     CallbackDetails,
     CallbackOptions,
     CallbackTimeoutType,
@@ -64,34 +70,38 @@ from .models import (
     ExecutionDetails,
     InvocationStatus,
     LambdaContext,
+    MappingModel,
     Operation,
     OperationAction,
     OperationIdentifier,
     OperationPayload,
     OperationStatus,
     OperationSubType,
+    OperationSubTypeValue,
     OperationType,
     OperationUpdate,
-    SerializableModel,
     StateOutput,
     StepDetails,
     TimestampConverter,
     WaitDetails,
     WaitOptions,
-    _metadata,
 )
 from .serdes import (
+    ComposableSerDes,
     DEFAULT_JSON_SERDES,
     EncodedValue,
     ExtendedTypeSerDes,
     JsonSerDes,
     PassThroughSerDes,
     SerDes,
-    SerDesContext,
+    SerDesPipelineError,
+    SerDesStage,
     TypeCodecExtension,
     TypeTag,
+    create_serdes_pipeline,
     deserialize,
     get_serdes_context,
+    is_composable_serdes,
     serialize,
 )
 from .state import RECURSIVE_LEVEL_INPUT_FIELD, ExecutionState
@@ -99,6 +109,7 @@ from .task import create_eager_task
 
 __all__ = [
     "CallableRuntimeError",
+    "ComposableSerDes",
     "DurableContext",
     "DurableExecutionsError",
     "DurableServiceClient",
@@ -115,15 +126,20 @@ __all__ = [
     "OperationSubType",
     "OperationType",
     "RetryStrategy",
+    "RetryableSerDesError",
     "SerDes",
     "SerDesContext",
     "SerDesError",
+    "SerDesPipelineError",
+    "SerDesStage",
     "UserlandError",
     "ValidationError",
     "create_default_sync_client",
+    "create_serdes_pipeline",
     "durable_callable",
     "durable_execution",
     "get_current_context",
     "get_durable_context",
     "get_serdes_context",
+    "is_composable_serdes",
 ]