prefect-client 3.9.1.dev7__tar.gz → 3.9.2__tar.gz

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.5
 Name: prefect-client
-Version: 3.9.1.dev7
+Version: 3.9.2
 Summary: Workflow orchestration and management.
 Project-URL: Changelog, https://github.com/PrefectHQ/prefect/releases
 Project-URL: Documentation, https://www.prefect.io/docs
@@ -163,7 +163,7 @@ describing the functionality you are interested in using and we will do our best
 ## Next steps
 
 There's lots more you can do to orchestrate and observe your workflows with Prefect!
-Start with our [friendly tutorial](https://www.prefect.io/docs/tutorials) or explore the [core concepts of Prefect workflows](https://www.prefect.io/docs/concepts/).
+Start with our [friendly tutorial](https://www.prefect.io/docs/v3/get-started/quickstart) or explore the [core concepts of Prefect workflows](https://www.prefect.io/docs/concepts/).
 
 ## Join the community
 
@@ -173,6 +173,6 @@ All community forums, including code contributions, issue discussions, and Slack
 
 ## Contribute
 
-See our [documentation on contributing to Prefect](https://www.prefect.io/docs/contributing/overview/).
+See our [documentation on contributing to Prefect](https://www.prefect.io/docs/contribute).
 
 Thanks for being part of the mission to build a new kind of workflow system and, of course, **happy engineering!**

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/README.md

@@ -94,7 +94,7 @@ describing the functionality you are interested in using and we will do our best
 ## Next steps
 
 There's lots more you can do to orchestrate and observe your workflows with Prefect!
-Start with our [friendly tutorial](https://www.prefect.io/docs/tutorials) or explore the [core concepts of Prefect workflows](https://www.prefect.io/docs/concepts/).
+Start with our [friendly tutorial](https://www.prefect.io/docs/v3/get-started/quickstart) or explore the [core concepts of Prefect workflows](https://www.prefect.io/docs/concepts/).
 
 ## Join the community
 
@@ -104,6 +104,6 @@ All community forums, including code contributions, issue discussions, and Slack
 
 ## Contribute
 
-See our [documentation on contributing to Prefect](https://www.prefect.io/docs/contributing/overview/).
+See our [documentation on contributing to Prefect](https://www.prefect.io/docs/contribute).
 
 Thanks for being part of the mission to build a new kind of workflow system and, of course, **happy engineering!**

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/AGENTS.md

@@ -29,7 +29,7 @@ Runtime flow/task parameter values must come from `utilities.callables.get_call_
 - **Sync and async must stay in sync.** Both `flow_engine.py` and `task_engine.py` have sync and async paths. Any behavior change must be applied to both.
 - **Event and log WebSocket subscribers are parallel implementations.** `events/clients.py::PrefectEventSubscriber` and `logging/clients.py::PrefectLogsSubscriber` duplicate the same reconnect/backoff/dedup logic, including the `reconnect_on_clean_close` option (used by `events/subscribers.py::FlowRunSubscriber` so a clean disconnect doesn't hide a terminal event). Mirror any reconnection-behavior change across both.
 - **Flow run observers own their stream recovery.** `_internal/observers.py` observers create subscribers with `reconnection_attempts=0` and delegate polling fallback, reconnect backoff, and setup deadlines to `_internal/_observation_delivery.py::ObservationDelivery`. Subscribers close any socket left by a failed or cancelled `__aenter__`, since callers never exit a context that failed to enter.
-- **Use `whenever` compat helpers for all datetime conversions on Python 3.13+.** `prefect-client` depends on `whenever` only on Python 3.13+, so modules it ships must keep `whenever` imports behind `sys.version_info >= (3, 13)`; only server code that the client build strips may import it unconditionally (see `client/AGENTS.md`). `types/_datetime.py` provides `now()`, `_whenever_to_stdlib()`, `_whenever_zdt_from_py()`, and `_whenever_pdt_from_py()` to abstract over API differences across supported whenever releases. Version-specific code is guarded by feature-detection flags (`_WHENEVER_NEW_API` for ≥ 0.10.0, `_SYSTEM_TZ` for ≥ 0.11), not version checks. Never call `DateTime.now()` or `pendulum.now()` directly — use `now()` from `types/_datetime.py` instead. Avoid no-argument `datetime.datetime.now()` because it returns a tz-naive datetime that raises `TypeError` when compared to the tz-aware datetimes produced by `now()`. Explicitly timezone-aware stdlib calls such as `datetime.now(ZoneInfo("UTC"))` are allowed when needed to preserve timezone information. Never call `ZonedDateTime.from_py_datetime()`, `PlainDateTime.from_py_datetime()`, or `.py_datetime()` directly — use the helpers instead. Likewise avoid `.assume_system_tz()`/`.to_system_tz()` (deprecated in 0.11; the warning fails tests under `filterwarnings = error`) — use `in_local_tz()` — and direct `DateTimeDelta` imports (removed in 0.11) — use the `Interval` alias. Violations will silently break on whichever whenever version the helpers weren't written for.
+- **Use `whenever` compat helpers for all datetime conversions on Python 3.13+.** `prefect-client` depends on `whenever` only on Python 3.13+, so modules it ships must keep `whenever` imports behind `sys.version_info >= (3, 13)`; only code that the client build strips may import it unconditionally (see the repo-root `client/AGENTS.md`). `types/_datetime.py` provides `now()`, `_whenever_to_stdlib()`, `_whenever_zdt_from_py()`, and `_whenever_pdt_from_py()` to abstract over API differences across supported whenever releases. Version-specific code is guarded by feature-detection flags (`_WHENEVER_NEW_API` for ≥ 0.10.0, `_SYSTEM_TZ` for ≥ 0.11), not version checks. These flags are only probed on Python 3.13+ (`_WHENEVER_NEW_API` is `False` and `_SYSTEM_TZ` is undefined on 3.11/3.12), so a helper that server-only code calls on every Python must feature-detect on the `whenever` class itself. Never call `DateTime.now()` or `pendulum.now()` directly — use `now()` from `types/_datetime.py` instead. Avoid no-argument `datetime.datetime.now()` because it returns a tz-naive datetime that raises `TypeError` when compared to the tz-aware datetimes produced by `now()`. Explicitly timezone-aware stdlib calls such as `datetime.now(ZoneInfo("UTC"))` are allowed when needed to preserve timezone information. Never call `ZonedDateTime.from_py_datetime()`, `PlainDateTime.from_py_datetime()`, or `.py_datetime()` directly — use the helpers instead. Likewise avoid `.assume_system_tz()`/`.to_system_tz()` (deprecated in 0.11; the warning fails tests under `filterwarnings = error`) — use `in_local_tz()` — and direct `DateTimeDelta` imports (removed in 0.11) — use the `Interval` alias. Violations will silently break on whichever whenever version the helpers weren't written for.
 - **Call Pendulum-only members via `types/_datetime.py::_pendulum_call` on Python 3.11/3.12.** Prefect's `DateTime`/`Date`/`Duration` wrap Pendulum-only members (and some shared stdlib methods like `replace`) with user-facing deprecation warnings; `_pendulum_call` bypasses them so users only see warnings for their own calls. `pyproject.toml` ignores these warnings in pytest, so direct calls from Prefect code pass tests silently unless covered by the guards in `tests/test_datetime.py`.
 - **Flow and task engines advance state differently.** The flow engine makes blocking API calls to the server to propose and advance states. The task engine emits `prefect.task_run.*` events (delivered via WebSockets) but advances state locally through `set_state` calls with polling/backoff — do not assume the two engines work the same way.
 - **Flow state transitions go through the server.** The flow engine proposes states to the server, which accepts or rejects them via orchestration rules. The task engine, by contrast, manages state transitions locally via `set_state` and emits `prefect.task_run.*` events — it does not propose states to the server.
@@ -41,6 +41,7 @@ Runtime flow/task parameter values must come from `utilities.callables.get_call_
 - **`ResultRecordMetadata` tolerates unknown serializer types.** When loading persisted metadata, an unrecognized serializer `type` is converted to an `UnknownSerializer` placeholder rather than raising `ValidationError`. This allows inspecting result metadata when the serializer implementation is unavailable in the current environment. However, known serializer types with invalid fields still raise `ValidationError`. `UnknownSerializer.dumps()` and `UnknownSerializer.loads()` raise `RuntimeError` — the tolerance is for inspection only, not actual serialization/deserialization.
 - **`ResultRecord` must be marked persisted after every storage write.** `ResultStore` calls `mark_persisted()` on every record it writes; `to_state_create` in `states.py` consults `is_persisted` as a fallback when `should_persist_result()` returns `False` (e.g., after run context teardown). Any new sync or async persist path that omits `mark_persisted()` will silently drop state metadata for already-written results.
 - **Flow-run suspension is enforced at orchestration boundaries.** Suspension is delivered via `FlowRunSuspensionRequest` and raised with `raise_if_flow_run_suspension_requested()`. When changing engines, futures, task runners, or generator execution, check before returning control to flow user code or proposing a flow state that could leave `Suspended`; keep sync/async paths aligned and avoid per-boundary API reads.
+- **Flow state hooks accept the same signatures on every execution path.** The sync and async flow engines and `runner/_hook_runner.py` share `_internal/engine.py::resolve_flow_hook_arguments`; `runner/_workspace_runtime_bootstrap.py` keeps its own copy because it may run under an older Prefect. Change the shared function and the copy together.
 - **Subflow tracking tasks use UUID dynamic keys when called from within a submitted task.** `flow_engine.py` marks the synthetic tracking `Task` with `_is_subflow_tracking_task=True`; `tasks.py` checks this flag to assign UUID-based (unstable) `dynamic_key` values when the tracking task runs inside a submitted task context. This prevents key collisions between concurrent sibling subflows on parent-flow retry. Direct subflow calls (no enclosing task context) keep stable sequential keys to preserve the retry optimization that reuses completed subflow results.
 
 ## Logging

prefect_client-3.9.2/src/prefect/_build_info.py

@@ -0,0 +1,5 @@
+# Generated by versioningit
+__version__ = "3.9.2"
+__build_date__ = "2026-11-06 21:19:45.400356+00:00"
+__git_commit__ = "b0479c0c26045b06321191bbad0cdfbb334d7380"
+__dirty__ = False

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/_internal/engine.py

@@ -1,5 +1,6 @@
 """Internal engine utilities"""
 
+import inspect
 from collections.abc import Callable
 from functools import partial
 from typing import TYPE_CHECKING, Any, Union
@@ -9,6 +10,9 @@ from prefect.context import FlowRunContext
 from prefect.flows import Flow
 from prefect.tasks import Task, TaskRunNameCallbackWithParameters
 
+if TYPE_CHECKING:
+    from prefect.client.schemas.objects import FlowRun, State
+
 
 def dynamic_key_for_task_run(
     context: FlowRunContext, task: "Task[..., Any]", stable: bool = True
@@ -93,3 +97,58 @@ def get_hook_name(hook: Callable[..., Any]) -> str:
             hook.func.__name__ if isinstance(hook, partial) else hook.__class__.__name__
         )
     )
+
+
+def resolve_flow_hook_arguments(
+    hook: Callable[..., Any],
+    flow: "Flow[..., Any]",
+    flow_run: "FlowRun",
+    state: "State",
+) -> tuple[tuple[Any, ...], dict[str, Any]]:
+    """
+    Return the positional and keyword arguments to call a flow state hook with,
+    checking its signature without calling it.
+
+    A hook that names its parameters `flow`, `flow_run` (or `run`) and `state`
+    gets them by name, in any order and including keyword-only parameters.
+    Otherwise, a hook that takes three positional arguments gets them in that
+    order, whatever their names, and a hook that only takes keyword arguments
+    gets them by name.
+
+    Raises:
+        TypeError: if the hook accepts none of these forms.
+    """
+    try:
+        signature = inspect.signature(hook)
+    except (TypeError, ValueError):
+        return (flow, flow_run, state), {}
+
+    def binds(*args: Any, **kwargs: Any) -> bool:
+        try:
+            signature.bind(*args, **kwargs)
+        except TypeError:
+            return False
+        return True
+
+    named = (inspect.Parameter.POSITIONAL_OR_KEYWORD, inspect.Parameter.KEYWORD_ONLY)
+    for flow_run_name in ("flow_run", "run"):
+        kwargs = {"flow": flow, flow_run_name: flow_run, "state": state}
+        declared = all(
+            name in signature.parameters and signature.parameters[name].kind in named
+            for name in kwargs
+        )
+        if declared and binds(**kwargs):
+            return (), kwargs
+
+    if binds(flow, flow_run, state):
+        return (flow, flow_run, state), {}
+
+    kwargs = {"flow": flow, "flow_run": flow_run, "state": state}
+    if binds(**kwargs):
+        return (), kwargs
+
+    raise TypeError(
+        f"Hook {get_hook_name(hook)!r} must accept the flow, the flow run and the"
+        " state as three positional arguments or as parameters named `flow`,"
+        " `flow_run` (or `run`) and `state`"
+    )

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/client/schemas/objects.py

@@ -1,6 +1,7 @@
 from __future__ import annotations
 
 import datetime
+import json
 import warnings
 from collections.abc import Callable, Mapping
 from enum import Enum
@@ -17,7 +18,6 @@ from typing import (
 )
 from uuid import UUID
 
-import orjson
 from pydantic import (
     AfterValidator,
     ConfigDict,
@@ -1768,7 +1768,7 @@ class FlowRunInput(ObjectBaseModel):
         Returns:
             Any: the decoded value
         """
-        return orjson.loads(self.value)
+        return json.loads(self.value)
 
 
 class GlobalConcurrencyLimit(ObjectBaseModel):

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/concurrency/AGENTS.md

@@ -18,7 +18,7 @@ Does NOT define concurrency limits (server-side in `server/`). Does NOT handle t
 
 **`raise_on_lease_renewal_failure` (public parameter):** Controls lease renewal failure behavior independently from `strict`. When `None` (default), falls back to the value of `strict` for backward compatibility. Set to `False` to let long-running tasks continue even if a transient renewal error occurs; set to `True` to terminate immediately on renewal failure regardless of `strict`. This means `strict=True, raise_on_lease_renewal_failure=False` gives strict slot acquisition but non-fatal renewal failures, and vice versa.
 
-**Lease renewal:** `_leases.py` starts a daemon thread that renews immediately on entry, then waits for 75% of `lease_duration` between renewals. The renewal thread uses the standard `get_client(sync_client=True)` path and retries each renewal up to 3 times with exponential backoff. If all attempts fail, a shared failure handler either cancels execution with the existing sync/async cancel scope or logs a warning, depending on `raise_on_lease_renewal_failure`.
+**Lease renewal:** `_leases.py` starts a daemon thread that renews immediately on entry, then waits for 75% of `lease_duration` between renewals. The renewal thread uses the standard `get_client(sync_client=True)` path and retries each renewal up to 3 times with exponential backoff. If all attempts fail, a shared failure handler either cancels execution with the existing sync/async cancel scope or logs a warning, depending on `raise_on_lease_renewal_failure`. Exception: when a `flow_run_id` is given and renewal fails with `410 Gone`, the thread reads the flow run and stops without cancelling if its server state is final, because the terminal transition releases the lease before the engine receives the proposal response.
 
 **Sync/async lockstep invariant:** `asyncio.py`/`sync.py` and `_asyncio.py`/`_sync.py` are parallel implementations. Any behavior change to one must be mirrored in the other.
 

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/concurrency/_leases.py

@@ -4,6 +4,9 @@ from contextlib import asynccontextmanager, contextmanager
 from typing import TYPE_CHECKING, AsyncGenerator, Generator
 from uuid import UUID
 
+import httpx
+
+from prefect._internal.compatibility.starlette import status
 from prefect._internal.concurrency.cancellation import (
     AsyncCancelScope,
     WatcherThreadCancelScope,
@@ -90,6 +93,45 @@ def _lease_renewal_loop(
             stop_event.wait(lease_duration * _RENEWAL_FRACTION)
 
 
+def _lease_released_by_final_flow_run(
+    exc: BaseException, lease_id: UUID, flow_run_id: UUID | None
+) -> bool:
+    """
+    Check whether a lease renewal failure is explained by the flow run having
+    reached a final state on the server, which releases the lease before the
+    engine has received the response to its terminal state proposal.
+    """
+    if flow_run_id is None or not (
+        isinstance(exc, httpx.HTTPStatusError)
+        and exc.response.status_code == status.HTTP_410_GONE
+    ):
+        return False
+
+    logger = get_logger("concurrency")
+    try:
+        with get_client(sync_client=True) as client:
+            flow_run = client.read_flow_run(flow_run_id)
+    except Exception:
+        logger.debug(
+            "Failed to read flow run %s after its concurrency lease was not found",
+            flow_run_id,
+            exc_info=True,
+        )
+        return False
+
+    if flow_run.state is None or not flow_run.state.is_final():
+        return False
+
+    logger.debug(
+        "Concurrency lease %s was released because flow run %s reached final "
+        "state %s; stopping lease renewal",
+        lease_id,
+        flow_run_id,
+        flow_run.state.type.value,
+    )
+    return True
+
+
 def _handle_lease_renewal_failure(
     exc: BaseException,
     lease_id: UUID,
@@ -139,6 +181,7 @@ def _start_lease_renewal_thread(
     raise_on_lease_renewal_failure: bool,
     suppress_warnings: bool,
     cancel_scope: AsyncCancelScope | WatcherThreadCancelScope,
+    flow_run_id: UUID | None = None,
 ) -> tuple[threading.Event, threading.Thread]:
     stop_event = threading.Event()
     renewal_ctx = contextvars.copy_context()
@@ -152,14 +195,17 @@ def _start_lease_renewal_thread(
         try:
             _lease_renewal_loop(lease_id, lease_duration, stop_event)
         except Exception as exc:
-            if not stop_event.is_set():
-                _handle_lease_renewal_failure(
-                    exc,
-                    lease_id,
-                    raise_on_lease_renewal_failure,
-                    suppress_warnings,
-                    cancel_scope,
-                )
+            if stop_event.is_set() or _lease_released_by_final_flow_run(
+                exc, lease_id, flow_run_id
+            ):
+                return
+            _handle_lease_renewal_failure(
+                exc,
+                lease_id,
+                raise_on_lease_renewal_failure,
+                suppress_warnings,
+                cancel_scope,
+            )
 
     def renewal_loop() -> None:
         renewal_ctx.run(_run_with_failure_handler)
@@ -179,6 +225,7 @@ def maintain_concurrency_lease(
     lease_duration: float,
     raise_on_lease_renewal_failure: bool = False,
     suppress_warnings: bool = False,
+    flow_run_id: UUID | None = None,
 ) -> Generator[None, None, None]:
     """
     Maintain a concurrency lease for the given lease ID.
@@ -187,6 +234,9 @@ def maintain_concurrency_lease(
         lease_id: The ID of the lease to maintain.
         lease_duration: The duration of the lease in seconds.
         raise_on_lease_renewal_failure: A boolean specifying whether to raise an error if the lease renewal fails.
+        flow_run_id: The ID of the flow run holding the lease. When the lease is
+            not found and this flow run is final on the server, the lease was
+            released by the terminal transition and execution is not terminated.
     """
     with WatcherThreadCancelScope() as cancel_scope:
         stop_event: threading.Event | None = None
@@ -198,6 +248,7 @@ def maintain_concurrency_lease(
                 raise_on_lease_renewal_failure,
                 suppress_warnings,
                 cancel_scope,
+                flow_run_id,
             )
             yield
         finally:
@@ -213,6 +264,7 @@ async def amaintain_concurrency_lease(
     lease_duration: float,
     raise_on_lease_renewal_failure: bool = False,
     suppress_warnings: bool = False,
+    flow_run_id: UUID | None = None,
 ) -> AsyncGenerator[None, None]:
     """
     Maintain a concurrency lease for the given lease ID.
@@ -221,6 +273,9 @@ async def amaintain_concurrency_lease(
         lease_id: The ID of the lease to maintain.
         lease_duration: The duration of the lease in seconds.
         raise_on_lease_renewal_failure: A boolean specifying whether to raise an error if the lease renewal fails.
+        flow_run_id: The ID of the flow run holding the lease. When the lease is
+            not found and this flow run is final on the server, the lease was
+            released by the terminal transition and execution is not terminated.
     """
     with AsyncCancelScope() as cancel_scope:
         stop_event: threading.Event | None = None
@@ -231,6 +286,7 @@ async def amaintain_concurrency_lease(
                 raise_on_lease_renewal_failure,
                 suppress_warnings,
                 cancel_scope,
+                flow_run_id,
             )
             yield
         finally:

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/deployments/runner.py

@@ -1671,14 +1671,14 @@ async def adeploy(
                 " or specify a remote storage location for the flow with `.from_source`."
                 " If you are attempting to deploy a flow to a local process work pool,"
                 " consider using `flow.serve` instead. See the documentation for more"
-                " information: https://www.prefect.io/docs/latest/how-to-guides/deployments/run-flows-in-local-processes"
+                " information: https://www.prefect.io/docs/v3/how-to-guides/deployment_infra/run-flows-in-local-processes"
             )
         elif work_pool.type == "process" and not ignore_warnings:
             console.print(
                 "Looks like you're deploying to a process work pool. If you're creating a"
                 " deployment for local development, calling `.serve` on your flow is a great"
                 " way to get started. See the documentation for more information:"
-                " https://www.prefect.io/docs/latest/how-to-guides/deployments/run-flows-in-local-processes "
+                " https://www.prefect.io/docs/v3/how-to-guides/deployment_infra/run-flows-in-local-processes "
                 " Set `ignore_warnings=True` to suppress this message.",
                 style="yellow",
             )
@@ -1912,14 +1912,14 @@ def deploy(
                 " or specify a remote storage location for the flow with `.from_source`."
                 " If you are attempting to deploy a flow to a local process work pool,"
                 " consider using `flow.serve` instead. See the documentation for more"
-                " information: https://www.prefect.io/docs/latest/how-to-guides/deployments/run-flows-in-local-processes"
+                " information: https://www.prefect.io/docs/v3/how-to-guides/deployment_infra/run-flows-in-local-processes"
             )
         elif work_pool.type == "process" and not ignore_warnings:
             console.print(
                 "Looks like you're deploying to a process work pool. If you're creating a"
                 " deployment for local development, calling `.serve` on your flow is a great"
                 " way to get started. See the documentation for more information:"
-                " https://www.prefect.io/docs/latest/how-to-guides/deployments/run-flows-in-local-processes "
+                " https://www.prefect.io/docs/v3/how-to-guides/deployment_infra/run-flows-in-local-processes "
                 " Set `ignore_warnings=True` to suppress this message.",
                 style="yellow",
             )

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/filesystems.py

@@ -547,7 +547,10 @@ class RemoteFileSystem(WritableFileSystem, WritableDeploymentStorage):
         counter = 0
         for f in Path(local_path).rglob("*"):
             relative_path = f.relative_to(local_path)
-            if included_files and relative_path.as_posix() not in included_files:
+            if (
+                included_files is not None
+                and relative_path.as_posix() not in included_files
+            ):
                 continue
 
             if to_path.endswith("/"):
@@ -603,7 +606,10 @@ class RemoteFileSystem(WritableFileSystem, WritableDeploymentStorage):
         counter = 0
         for f in Path(local_path).rglob("*"):
             relative_path = f.relative_to(local_path)
-            if included_files and relative_path.as_posix() not in included_files:
+            if (
+                included_files is not None
+                and relative_path.as_posix() not in included_files
+            ):
                 continue
 
             if to_path.endswith("/"):

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/flow_engine.py

@@ -11,7 +11,7 @@ import signal
 import sys
 import threading
 import time
-from collections.abc import Mapping
+from collections.abc import Callable, Mapping
 from contextlib import (
     AsyncExitStack,
     ExitStack,
@@ -59,7 +59,11 @@ from prefect._internal.control_listener import (
     get_intent,
     report_engine_outcome,
 )
-from prefect._internal.engine import get_hook_name, resolve_custom_flow_run_name
+from prefect._internal.engine import (
+    get_hook_name,
+    resolve_custom_flow_run_name,
+    resolve_flow_hook_arguments,
+)
 from prefect._internal.metrics import RunMetrics
 from prefect.client.orchestration import PrefectClient, SyncPrefectClient, get_client
 from prefect.client.schemas import FlowRun, TaskRun
@@ -330,28 +334,40 @@ def _send_heartbeats(
     resource, related = engine._build_heartbeat_event_template()
 
     stop_event = threading.Event()
+    # The heartbeat thread checks for a stop and emits while holding this lock,
+    # so once `stop_heartbeats` returns, no heartbeat is mid-emit and none will
+    # be emitted.
+    emit_lock = threading.Lock()
 
-    def heartbeat_loop() -> None:
-        while not stop_event.is_set():
-            # Check state before emitting - don't emit if final
-            if (
-                engine.flow_run
-                and engine.flow_run.state
-                and engine.flow_run.state.is_final()
-            ):
-                engine.logger.debug("Flow run in terminal state, stopping heartbeat")
-                return
+    def stop_heartbeats() -> None:
+        with emit_lock:
+            stop_event.set()
 
-            try:
-                engine._emit_flow_run_heartbeat(resource, related)
-            except Exception:
-                engine.logger.debug("Failed to emit heartbeat", exc_info=True)
-
-            # Sleep in increments to allow quick shutdown
-            for _ in range(heartbeat_seconds):
+    def heartbeat_loop() -> None:
+        while True:
+            with emit_lock:
                 if stop_event.is_set():
                     return
-                time.sleep(1)
+
+                # Check state before emitting - don't emit if final
+                if (
+                    engine.flow_run
+                    and engine.flow_run.state
+                    and engine.flow_run.state.is_final()
+                ):
+                    engine.logger.debug(
+                        "Flow run in terminal state, stopping heartbeat"
+                    )
+                    return
+
+                try:
+                    engine._emit_flow_run_heartbeat(resource, related)
+                except Exception:
+                    engine.logger.debug("Failed to emit heartbeat", exc_info=True)
+
+            # Waiting on the event wakes the thread as soon as heartbeats stop
+            if stop_event.wait(heartbeat_seconds):
+                return
 
     # Copy the current context so the heartbeat thread sees the same
     # `SettingsContext` (and therefore the same `PREFECT_API_URL`) as the
@@ -366,13 +382,14 @@ def _send_heartbeats(
     thread = threading.Thread(
         target=heartbeat_ctx.run, args=(heartbeat_loop,), daemon=True
     )
+    engine._stop_heartbeats = stop_heartbeats
     thread.start()
     engine.logger.debug("Started flow run heartbeat context")
 
     try:
         yield
     finally:
-        stop_event.set()
+        stop_heartbeats()
         if join_on_exit:
             thread.join(timeout=2)
         engine.logger.debug("Stopped flow run heartbeat context")
@@ -424,6 +441,15 @@ class BaseFlowRunEngine(Generic[P, R]):
         default_factory=FlowRunSuspensionRequest
     )
     _attempt_conclusion: EngineOutcomeReceipt | None = field(default=None, init=False)
+    # Stops the heartbeat thread started by the most recent `_send_heartbeats`.
+    # `set_state` calls it before proposing a terminal state: the server
+    # timestamps that state when it accepts it, but the heartbeat thread sees
+    # it only after the response arrives.  A heartbeat sent in that gap occurs
+    # after the terminal state, and an automation that expects every heartbeat
+    # to be followed by another event then fires on the finished run.
+    _stop_heartbeats: Callable[[], None] | None = field(
+        default=None, init=False, repr=False
+    )
 
     def __post_init__(self) -> None:
         if self.flow is None and self.flow_run_id is None:
@@ -757,6 +783,9 @@ class FlowRunEngine(BaseFlowRunEngine[P, R]):
         if self.short_circuit:
             return self.state
 
+        if state.is_final() and self._stop_heartbeats is not None:
+            self._stop_heartbeats()
+
         state = propose_state_sync(
             self.client, state, flow_run_id=self.flow_run.id, force=force
         )  # type: ignore
@@ -1072,7 +1101,8 @@ class FlowRunEngine(BaseFlowRunEngine[P, R]):
                     f"Running hook {hook_name!r} in response to entering state"
                     f" {state.name!r}"
                 )
-                result = hook(flow, flow_run, state)
+                args, kwargs = resolve_flow_hook_arguments(hook, flow, flow_run, state)
+                result = hook(*args, **kwargs)
                 if asyncio.iscoroutine(result):
                     run_coro_as_sync(result)
             except Exception:
@@ -1137,7 +1167,10 @@ class FlowRunEngine(BaseFlowRunEngine[P, R]):
             if lease_id := self.state.state_details.deployment_concurrency_lease_id:
                 stack.enter_context(
                     maintain_concurrency_lease(
-                        lease_id, 300, raise_on_lease_renewal_failure=True
+                        lease_id,
+                        300,
+                        raise_on_lease_renewal_failure=True,
+                        flow_run_id=self.flow_run.id,
                     )
                 )
 
@@ -1492,6 +1525,9 @@ class AsyncFlowRunEngine(BaseFlowRunEngine[P, R]):
         if self.short_circuit:
             return self.state
 
+        if state.is_final() and self._stop_heartbeats is not None:
+            self._stop_heartbeats()
+
         state = await propose_state(
             self.client, state, flow_run_id=self.flow_run.id, force=force
         )  # type: ignore
@@ -1800,7 +1836,8 @@ class AsyncFlowRunEngine(BaseFlowRunEngine[P, R]):
                     f"Running hook {hook_name!r} in response to entering state"
                     f" {state.name!r}"
                 )
-                result = hook(flow, flow_run, state)
+                args, kwargs = resolve_flow_hook_arguments(hook, flow, flow_run, state)
+                result = hook(*args, **kwargs)
                 if asyncio.iscoroutine(result):
                     await result
             except Exception:
@@ -1865,7 +1902,10 @@ class AsyncFlowRunEngine(BaseFlowRunEngine[P, R]):
             if lease_id := self.state.state_details.deployment_concurrency_lease_id:
                 await stack.enter_async_context(
                     amaintain_concurrency_lease(
-                        lease_id, 300, raise_on_lease_renewal_failure=True
+                        lease_id,
+                        300,
+                        raise_on_lease_renewal_failure=True,
+                        flow_run_id=self.flow_run.id,
                     )
                 )
 

{prefect_client-3.9.1.dev7 → prefect_client-3.9.2}/src/prefect/flow_runs.py

@@ -133,6 +133,7 @@ async def wait_for_flow_run(
     )
 
     with anyio.move_on_after(timeout):
+        received_final_event = False
         async with get_events_subscriber(filter=event_filter) as subscriber:
             flow_run = await client.read_flow_run(flow_run_id)
             if flow_run.state and flow_run.state.is_final():
@@ -151,7 +152,19 @@ async def wait_for_flow_run(
                     logger.info(f"Flow run is in state {state.name!r}")
 
                 if state.is_final():
-                    return await client.read_flow_run(flow_run_id)
+                    received_final_event = True
+                    break
+
+        if received_final_event:
+            # A final state event can be delivered before the state change is
+            # visible to API reads, so read until the run reports a final state
+            poll_delay = 0.1
+            while True:
+                flow_run = await client.read_flow_run(flow_run_id)
+                if flow_run.state and flow_run.state.is_final():
+                    return flow_run
+                await anyio.sleep(poll_delay)
+                poll_delay = min(poll_delay * 2, 5)
 
     raise FlowRunWaitTimeout(
         f"Flow run with ID {flow_run_id} exceeded watch timeout of {timeout} seconds"