prefect-client 3.6.30.dev4__tar.gz → 3.7.1__tar.gz

{prefect_client-3.6.30.dev4 → prefect_client-3.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: prefect-client
-Version: 3.6.30.dev4
+Version: 3.7.1
 Summary: Workflow orchestration and management.
 Project-URL: Changelog, https://github.com/PrefectHQ/prefect/releases
 Project-URL: Documentation, https://docs.prefect.io

{prefect_client-3.6.30.dev4 → prefect_client-3.7.1}/src/prefect/AGENTS.md

@@ -23,7 +23,7 @@ There is no formal public/private boundary beyond the `_` prefix convention. Mod
 - **Engine ordering matters.** The engines apply features (retries, caching, result persistence, transactions) in a specific order. Changing the order or forgetting a feature in one engine path is the most common source of breakage.
 - **Set run metadata before state transitions.** Any run attribute that event subscribers observe at the moment of a state transition (e.g., custom `flow_run_name`) must be persisted to the server *before* `set_state()` is called for that transition. Metadata set after the call appears stale to subscribers of that state event.
 - **Sync and async must stay in sync.** Both `flow_engine.py` and `task_engine.py` have sync and async paths. Any behavior change must be applied to both.
-- **Use `whenever` compat helpers for all datetime conversions on Python 3.13+.** `types/_datetime.py` provides `now()`, `_whenever_to_stdlib()`, `_whenever_zdt_from_py()`, and `_whenever_pdt_from_py()` to abstract over API differences between whenever 0.7.x–0.9.x and ≥ 0.10.0. The `_WHENEVER_NEW_API` flag (True when whenever ≥ 0.10.0) guards version-specific code. Never call `DateTime.now()` or `pendulum.now()` directly — use `now()` from `types/_datetime.py` instead. Never call `ZonedDateTime.from_py_datetime()`, `PlainDateTime.from_py_datetime()`, or `.py_datetime()` directly — use the helpers instead. Violations will silently break on whichever whenever version the helpers weren't written for.
+- **Use `whenever` compat helpers for all datetime conversions on Python 3.13+.** `types/_datetime.py` provides `now()`, `_whenever_to_stdlib()`, `_whenever_zdt_from_py()`, and `_whenever_pdt_from_py()` to abstract over API differences between whenever 0.7.x–0.9.x and ≥ 0.10.0. The `_WHENEVER_NEW_API` flag (True when whenever ≥ 0.10.0) guards version-specific code. Never call `DateTime.now()` or `pendulum.now()` directly — use `now()` from `types/_datetime.py` instead. Avoid no-argument `datetime.datetime.now()` because it returns a tz-naive datetime that raises `TypeError` when compared to the tz-aware datetimes produced by `now()`. Explicitly timezone-aware stdlib calls such as `datetime.now(ZoneInfo("UTC"))` are allowed when needed to preserve timezone information. Never call `ZonedDateTime.from_py_datetime()`, `PlainDateTime.from_py_datetime()`, or `.py_datetime()` directly — use the helpers instead. Violations will silently break on whichever whenever version the helpers weren't written for.
 - **Flow and task engines advance state differently.** The flow engine makes blocking API calls to the server to propose and advance states. The task engine emits `prefect.task_run.*` events (delivered via WebSockets) but advances state locally through `set_state` calls with polling/backoff — do not assume the two engines work the same way.
 - **Flow state transitions go through the server.** The flow engine proposes states to the server, which accepts or rejects them via orchestration rules. The task engine, by contrast, manages state transitions locally via `set_state` and emits `prefect.task_run.*` events — it does not propose states to the server.
 - **`ProcessPoolTaskRunner` requires picklable data across subprocess boundaries.** `PrefectFuture` objects are not picklable and cannot be passed to worker subprocesses. Any `wait_for` futures must be waited on in the parent process and converted to `State` objects before submission. The subprocess task engine handles `State` objects via `resolve_to_final_result`, which raises `UpstreamTaskError` for non-completed upstreams.
@@ -31,6 +31,7 @@ There is no formal public/private boundary beyond the `_` prefix convention. Mod
 - **`ThreadPoolTaskRunner` is cloudpickled when a flow run is dispatched to a subprocess.** `threading.Lock` and other non-picklable thread primitives must be dropped in `__getstate__` and rebuilt in `__setstate__`. Any new instance state added to this class must be evaluated for picklability.
 - **Nested task submissions on a bounded `ThreadPoolTaskRunner` can deadlock.** When a worker task submits children and blocks on `.result()` while all `max_workers` threads are busy, the pool starves. `_warn_if_nested_submit_would_deadlock` detects this and emits a one-time warning — preserve this detection when changing pool management.
 - **`ResultRecordMetadata` tolerates unknown serializer types.** When loading persisted metadata, an unrecognized serializer `type` is converted to an `UnknownSerializer` placeholder rather than raising `ValidationError`. This allows inspecting result metadata when the serializer implementation is unavailable in the current environment. However, known serializer types with invalid fields still raise `ValidationError`. `UnknownSerializer.dumps()` and `UnknownSerializer.loads()` raise `RuntimeError` — the tolerance is for inspection only, not actual serialization/deserialization.
+- **Flow-run suspension is enforced at orchestration boundaries.** Suspension is delivered via `FlowRunSuspensionRequest` and raised with `raise_if_flow_run_suspension_requested()`. When changing engines, futures, task runners, or generator execution, check before returning control to flow user code or proposing a flow state that could leave `Suspended`; keep sync/async paths aligned and avoid per-boundary API reads.
 
 ## Logging
 
@@ -58,6 +59,8 @@ Use `get_logger()` from `prefect.logging` instead of raw `logging.getLogger()`
 - `utilities/` → Cross-cutting helpers: async utils, schema hydration, callables introspection, and more (see utilities/AGENTS.md)
 - `blocks/` → Server-persisted configuration objects for external service credentials and settings (see blocks/AGENTS.md)
 - `workers/` → Work-pool-based execution layer: polls for flow runs, dispatches to infrastructure (see workers/AGENTS.md)
+- `bundles/` → Flow bundling for remote execution (`create_bundle_for_flow_run`, cloudpickle+gzip); used by workers when dispatching `InfrastructureBoundFlow` runs; `execute_bundle_from_file` is lazy-loaded via `__getattr__` to avoid `runpy` warnings when invoked as `python -m prefect.bundles.execute`
+- `plugins.py` → Stable public surface for startup hooks and `prefect.collections` entrypoint loading; pluggy symbols are lazy-loaded so `prefect-client` builds (which don't ship pluggy) can still import `load_prefect_collections` without error
 - `docker/` → `DockerImage` class for building and pushing Docker images during deployment
 - `telemetry/` → OS-level resource metric collection and run telemetry
 - `testing/` → Test utilities shipped with the SDK: `prefect_test_harness`, assertion helpers, and reusable fixtures (see testing/AGENTS.md)

{prefect_client-3.6.30.dev4 → prefect_client-3.7.1}/src/prefect/__init__.py

@@ -78,25 +78,24 @@ del _build_info, pathlib
 
 def _initialize_plugins() -> None:
     """
-    Initialize the experimental plugin system if enabled.
+    Initialize the plugin system if enabled.
 
     This runs automatically when Prefect is imported and plugins are enabled
-    via experiments.plugins.enabled setting. Errors are logged but don't prevent
+    via the `plugins.enabled` setting. Errors are logged but don't prevent
     Prefect from loading.
     """
     try:
         # Import here to avoid circular imports and defer cost until needed
         from prefect.settings import get_current_settings
 
-        if not get_current_settings().experiments.plugins.enabled:
+        if not get_current_settings().plugins.enabled:
             return
 
         import anyio
 
-        from prefect._experimental.plugins import run_startup_hooks
-        from prefect._experimental.plugins.spec import HookContext
         from prefect.context import refresh_global_settings_context
         from prefect.logging import get_logger
+        from prefect.plugins import HookContext, run_startup_hooks
         from prefect.settings import get_current_settings
 
         ctx = HookContext(

prefect_client-3.7.1/src/prefect/_build_info.py

@@ -0,0 +1,5 @@
+# Generated by versioningit
+__version__ = "3.7.1"
+__build_date__ = "2026-05-16 02:18:55.839608+00:00"
+__git_commit__ = "e914068ebe6fb04552cfd9d68bf6ac12563bb4f1"
+__dirty__ = False

prefect_client-3.7.1/src/prefect/_experimental/_launchers.py

@@ -0,0 +1,31 @@
+"""
+Compatibility shim for the legacy `prefect._experimental._launchers` path.
+
+The launcher utilities graduated to GA at `prefect._internal.launchers` when the
+infrastructure decorators feature became stable. This module re-exports
+the public symbols from the new location and emits a
+`DeprecationWarning` so consumers know to migrate.
+"""
+
+import warnings
+
+from prefect._internal.launchers import (
+    get_launcher_for_side,
+    normalize_launcher,
+    resolve_bundle_step_with_launcher,
+    validate_bundle_step_launcher,
+)
+
+warnings.warn(
+    "`prefect._experimental._launchers` has moved to `prefect._internal.launchers`. "
+    "The old import path will be removed in a future release.",
+    DeprecationWarning,
+    stacklevel=2,
+)
+
+__all__ = [
+    "get_launcher_for_side",
+    "normalize_launcher",
+    "resolve_bundle_step_with_launcher",
+    "validate_bundle_step_launcher",
+]

prefect_client-3.7.1/src/prefect/_experimental/bundles/__init__.py

@@ -0,0 +1,68 @@
+"""
+Compatibility shim for the legacy `prefect._experimental.bundles` path.
+
+The bundle implementation graduated to GA at `prefect.bundles` when the
+infrastructure decorators feature became stable. This package keeps the
+old import path importable (and `python -m`-runnable for the legacy
+`...bundles.execute` entrypoint), but emits a `DeprecationWarning` so
+consumers know to migrate to `prefect.bundles`.
+"""
+
+import warnings
+from typing import Any
+
+from prefect.bundles import (
+    BundleCreationResult,
+    BundleLauncher,
+    BundleLauncherOverride,
+    BundleLauncherSide,
+    SerializedBundle,
+    aupload_bundle_to_storage,
+    convert_step_to_command,
+    create_bundle_for_flow_run,
+    execute_bundle_in_subprocess,
+    extract_flow_from_bundle,
+    upload_bundle_to_storage,
+)
+from prefect.bundles import _pin_prefect_in_bundle_step_requires  # noqa: F401
+
+warnings.warn(
+    "`prefect._experimental.bundles` has moved to `prefect.bundles`. "
+    "The old import path will be removed in a future release.",
+    DeprecationWarning,
+    stacklevel=2,
+)
+
+__all__ = [
+    "BundleCreationResult",
+    "BundleLauncher",
+    "BundleLauncherOverride",
+    "BundleLauncherSide",
+    "SerializedBundle",
+    "aupload_bundle_to_storage",
+    "convert_step_to_command",
+    "create_bundle_for_flow_run",
+    "execute_bundle_from_file",
+    "execute_bundle_in_subprocess",
+    "extract_flow_from_bundle",
+    "upload_bundle_to_storage",
+]
+
+
+def __getattr__(name: str) -> Any:
+    """Lazy-load `execute_bundle_from_file` from the `.execute` submodule
+    so importing this package does not pre-populate
+    `sys.modules["prefect._experimental.bundles.execute"]`. That would
+    make `python -m prefect._experimental.bundles.execute` emit a
+    `runpy` `RuntimeWarning` and hard-fail under
+    `PYTHONWARNINGS=error`. Accessing the legacy `.execute` submodule
+    by attribute (rather than importing it eagerly) still works because
+    Python binds it on the package after a separate `from . import
+    execute` (the legacy execute shim is reachable via plain
+    `import prefect._experimental.bundles.execute`).
+    """
+    if name == "execute_bundle_from_file":
+        from prefect.bundles.execute import execute_bundle_from_file
+
+        return execute_bundle_from_file
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")

prefect_client-3.7.1/src/prefect/_experimental/bundles/execute.py

@@ -0,0 +1,46 @@
+"""
+Compatibility shim for the legacy `prefect._experimental.bundles.execute`
+entrypoint.
+
+Existing work pool storage configurations may have stored the command
+`python -m prefect._experimental.bundles.execute --key ...` before the
+bundle implementation graduated to GA. This module re-exports the new
+`prefect.bundles.execute` symbols and forwards `python -m` execution to
+the new entrypoint, while emitting a `DeprecationWarning` so users know
+to migrate to `prefect.bundles.execute`.
+"""
+
+import warnings
+from typing import Any
+
+warnings.warn(
+    "`prefect._experimental.bundles.execute` has moved to "
+    "`prefect.bundles.execute`. The old import path will be removed in a "
+    "future release.",
+    DeprecationWarning,
+    stacklevel=2,
+)
+
+__all__ = ["execute_bundle", "execute_bundle_from_file"]  # noqa: F822
+
+
+def __getattr__(name: str) -> Any:
+    """Lazy-load symbols from `prefect.bundles.execute` so that running
+    `python -m prefect._experimental.bundles.execute` does not put
+    `prefect.bundles.execute` in `sys.modules` before the
+    `runpy.run_module` forward below executes — that would trigger a
+    runpy `RuntimeWarning` (and hard-fail under `PYTHONWARNINGS=error`).
+    """
+    if name in {"execute_bundle", "execute_bundle_from_file"}:
+        from prefect.bundles.execute import execute_bundle, execute_bundle_from_file
+
+        if name == "execute_bundle":
+            return execute_bundle
+        return execute_bundle_from_file
+    raise AttributeError(f"module {__name__!r} has no attribute {name!r}")
+
+
+if __name__ == "__main__":
+    import runpy
+
+    runpy.run_module("prefect.bundles.execute", run_name="__main__")

prefect_client-3.7.1/src/prefect/_experimental/plugins/__init__.py

@@ -0,0 +1,16 @@
+"""
+Deprecated. Use `prefect.plugins` instead.
+
+The plugin system has graduated from experimental to general availability.
+Imports from this path continue to work but emit a `DeprecationWarning`.
+"""
+
+from prefect._internal.compatibility.deprecated_paths import deprecated_module_attrs
+
+# Names are lazily resolved by `__getattr__` below; ruff's F822 (undefined
+# name in __all__) doesn't fit this deprecation-shim pattern.
+# fmt: off
+__all__ = ("run_startup_hooks", "HookContext", "SetupResult", "HookSpec", "SetupSummary", "PREFECT_PLUGIN_API_VERSION", "register_hook")  # noqa: F822
+# fmt: on
+
+__getattr__ = deprecated_module_attrs(__name__, "prefect.plugins", __all__)

prefect_client-3.7.1/src/prefect/_experimental/plugins/apply.py

@@ -0,0 +1,5 @@
+"""Deprecated. The contents of this module were never public API."""
+
+from prefect._internal.compatibility.deprecated_paths import deprecated_module_attrs
+
+__getattr__ = deprecated_module_attrs(__name__, "prefect.plugins", ())

prefect_client-3.7.1/src/prefect/_experimental/plugins/diagnostics.py

@@ -0,0 +1,7 @@
+"""Deprecated. Use `prefect.plugins` instead."""
+
+from prefect._internal.compatibility.deprecated_paths import deprecated_module_attrs
+
+__all__ = ("SetupSummary",)  # noqa: F822 - lazily resolved by `__getattr__`
+
+__getattr__ = deprecated_module_attrs(__name__, "prefect.plugins", __all__)

prefect_client-3.7.1/src/prefect/_experimental/plugins/manager.py

@@ -0,0 +1,7 @@
+"""Deprecated. Use `prefect.plugins` instead."""
+
+from prefect._internal.compatibility.deprecated_paths import deprecated_module_attrs
+
+__all__ = ("register_hook",)  # noqa: F822 - lazily resolved by `__getattr__`
+
+__getattr__ = deprecated_module_attrs(__name__, "prefect.plugins", __all__)

prefect_client-3.7.1/src/prefect/_experimental/plugins/spec.py

@@ -0,0 +1,7 @@
+"""Deprecated. Use `prefect.plugins` instead."""
+
+from prefect._internal.compatibility.deprecated_paths import deprecated_module_attrs
+
+__all__ = ("HookContext", "SetupResult", "HookSpec", "PREFECT_PLUGIN_API_VERSION")  # noqa: F822 - names lazily resolved by `__getattr__`
+
+__getattr__ = deprecated_module_attrs(__name__, "prefect.plugins", __all__)

prefect_client-3.7.1/src/prefect/_flow_run_suspension.py

@@ -0,0 +1,136 @@
+from __future__ import annotations
+
+import asyncio
+import contextvars
+import threading
+from contextlib import contextmanager
+from typing import Generator
+from uuid import UUID
+
+from prefect.client.schemas.objects import State
+from prefect.exceptions import Pause
+from prefect.logging.loggers import get_logger
+
+
+class FlowRunSuspensionRequest:
+    """
+    Mutable in-process suspension request shared by flow, child flow, and task contexts.
+
+    The stored state is the server-accepted `Suspended` state that should be raised
+    through Prefect's existing `Pause` control-flow path at the next orchestration
+    boundary.
+    """
+
+    def __init__(self) -> None:
+        self._suspended_state: State | None = None
+        self._lock = threading.Lock()
+
+    def mark_requested(self, state: State) -> None:
+        with self._lock:
+            self._suspended_state = state
+
+    def get_state(self) -> State | None:
+        with self._lock:
+            return self._suspended_state
+
+    def is_requested(self) -> bool:
+        return self.get_state() is not None
+
+    def raise_if_requested(self) -> None:
+        if state := self.get_state():
+            raise Pause(state=state)
+
+
+def is_suspended_flow_run_state(state: State | None) -> bool:
+    return bool(state and state.is_paused() and state.name == "Suspended")
+
+
+_active_flow_run_suspension_requests: dict[UUID, FlowRunSuspensionRequest] = {}
+_active_flow_run_suspension_requests_lock = threading.Lock()
+
+
+@contextmanager
+def register_flow_run_suspension_request(
+    flow_run_id: UUID,
+    suspension_request: FlowRunSuspensionRequest,
+) -> Generator[None, None, None]:
+    with _active_flow_run_suspension_requests_lock:
+        _active_flow_run_suspension_requests[flow_run_id] = suspension_request
+
+    try:
+        yield
+    finally:
+        with _active_flow_run_suspension_requests_lock:
+            if (
+                _active_flow_run_suspension_requests.get(flow_run_id)
+                is suspension_request
+            ):
+                _active_flow_run_suspension_requests.pop(flow_run_id, None)
+
+
+def mark_flow_run_suspension_requested(flow_run_id: UUID, state: State) -> bool:
+    with _active_flow_run_suspension_requests_lock:
+        suspension_request = _active_flow_run_suspension_requests.get(flow_run_id)
+
+    if suspension_request is None:
+        return False
+
+    suspension_request.mark_requested(state)
+    return True
+
+
+def raise_if_flow_run_suspension_requested() -> None:
+    from prefect.context import FlowRunContext
+
+    if flow_run_context := FlowRunContext.get():
+        flow_run_context.flow_run_suspension_request.raise_if_requested()
+
+
+@contextmanager
+def observe_flow_run_suspension(
+    flow_run_id: UUID,
+    suspension_request: FlowRunSuspensionRequest,
+    polling_interval: float = 10,
+) -> Generator[None, None, None]:
+    from prefect._internal.observers import FlowRunSuspendingObserver
+
+    logger = get_logger("flow_run_suspension")
+    stop_event = threading.Event()
+    ready_event = threading.Event()
+
+    def mark_suspended(flow_run_id: UUID, state: State) -> None:
+        if not mark_flow_run_suspension_requested(flow_run_id, state):
+            suspension_request.mark_requested(state)
+
+    async def run_observer() -> None:
+        async with FlowRunSuspendingObserver(
+            on_suspended=mark_suspended,
+            polling_interval=polling_interval,
+        ) as observer:
+            await observer.watch_flow_run_id(flow_run_id)
+            ready_event.set()
+            while not stop_event.is_set():
+                await asyncio.sleep(0.2)
+
+    context = contextvars.copy_context()
+
+    def observer_thread() -> None:
+        try:
+            context.run(lambda: asyncio.run(run_observer()))
+        except Exception:
+            logger.debug(
+                "Flow run suspension observer exited with an exception",
+                exc_info=True,
+            )
+        finally:
+            ready_event.set()
+
+    thread = threading.Thread(target=observer_thread, daemon=True)
+    thread.start()
+    ready_event.wait()
+
+    try:
+        yield
+    finally:
+        stop_event.set()
+        thread.join(timeout=2)

prefect_client-3.7.1/src/prefect/_internal/compatibility/deprecated_paths.py

@@ -0,0 +1,60 @@
+"""
+Generic helper for keeping a deprecated module path working as a re-export
+of a new location, emitting a DeprecationWarning per attribute access.
+
+Independent of any specific Prefect version migration. Intended for cases
+where a feature graduates between lifecycle stages (e.g., experimental to
+GA) and we want the old import path to continue working with a clear hand
+off message, but without coupling to the v2->v3 module migration helpers
+in `prefect._internal.compatibility.migration`.
+"""
+
+from __future__ import annotations
+
+import importlib
+import warnings
+from typing import Any, Callable, Iterable
+
+
+def deprecated_module_attrs(
+    old_module: str,
+    new_module: str,
+    names: Iterable[str],
+) -> Callable[[str], Any]:
+    """
+    Build a `__getattr__` for a deprecated module that re-exports public
+    symbols from a new location. Each access emits a DeprecationWarning
+    pointing the user at the new path.
+
+    Symbols not in `names` raise `AttributeError` (rather than being
+    silently re-exported) — this keeps the public surface explicit and
+    avoids leaking implementation details that the new location does not
+    intend to expose.
+
+    Usage at the bottom of the deprecated module:
+
+        from prefect._internal.compatibility.deprecated_paths import (
+            deprecated_module_attrs,
+        )
+
+        __getattr__ = deprecated_module_attrs(
+            __name__,
+            "prefect.plugins",
+            ("register_hook", "HookContext"),
+        )
+    """
+    name_set = frozenset(names)
+
+    def __getattr__(name: str) -> Any:
+        if name in name_set:
+            warnings.warn(
+                f"`{old_module}.{name}` has moved to `{new_module}.{name}`. "
+                f"Update your imports; the old path will be removed in a "
+                f"future release.",
+                DeprecationWarning,
+                stacklevel=2,
+            )
+            return getattr(importlib.import_module(new_module), name)
+        raise AttributeError(f"module {old_module!r} has no attribute {name!r}")
+
+    return __getattr__

prefect_client-3.6.30.dev4/src/prefect/_experimental/_launchers.py → prefect_client-3.7.1/src/prefect/_internal/launchers.py

@@ -4,7 +4,7 @@ from copy import deepcopy
 from typing import TYPE_CHECKING, Any
 
 if TYPE_CHECKING:
-    from prefect._experimental.bundles import (
+    from prefect.bundles import (
         BundleLauncher,
         BundleLauncherOverride,
         BundleLauncherSide,