prefect-client 2.16.1__tar.gz → 2.16.3__tar.gz

{prefect-client-2.16.1/src/prefect_client.egg-info → prefect-client-2.16.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: prefect-client
-Version: 2.16.1
+Version: 2.16.3
 Summary: Workflow orchestration and management.
 Home-page: https://www.prefect.io
 Author: Prefect Technologies, Inc.
@@ -13,12 +13,12 @@ Project-URL: Tracker, https://github.com/PrefectHQ/prefect/issues
 Description: <p align="center"><img src="https://github.com/PrefectHQ/prefect/assets/3407835/c654cbc6-63e8-4ada-a92a-efd2f8f24b85" width=1000></p>
         
         <p align="center">
-            <a href="https://pypi.python.org/pypi/prefect/" alt="PyPI version">
-                <img alt="PyPI" src="https://img.shields.io/pypi/v/prefect?color=0052FF&labelColor=090422"></a>
+            <a href="https://pypi.python.org/pypi/prefect-client/" alt="PyPI version">
+                <img alt="PyPI" src="https://img.shields.io/pypi/v/prefect-client?color=0052FF&labelColor=090422"></a>
             <a href="https://github.com/prefecthq/prefect/" alt="Stars">
                 <img src="https://img.shields.io/github/stars/prefecthq/prefect?color=0052FF&labelColor=090422" /></a>
-            <a href="https://pepy.tech/badge/prefect/" alt="Downloads">
-                <img src="https://img.shields.io/pypi/dm/prefect?color=0052FF&labelColor=090422" /></a>
+            <a href="https://pepy.tech/badge/prefect-client/" alt="Downloads">
+                <img src="https://img.shields.io/pypi/dm/prefect-client?color=0052FF&labelColor=090422" /></a>
             <a href="https://github.com/prefecthq/prefect/pulse" alt="Activity">
                 <img src="https://img.shields.io/github/commit-activity/m/prefecthq/prefect?color=0052FF&labelColor=090422" /></a>
             <br>
@@ -30,78 +30,80 @@ Description: <p align="center"><img src="https://github.com/PrefectHQ/prefect/as
                 <img src="https://img.shields.io/badge/youtube-watch_videos-red.svg?color=0052FF&labelColor=090422&logo=youtube" /></a>
         </p>
         
-        # Prefect
+        # prefect-client
         
-        Prefect is an orchestration and observability platform for building, observing, and triaging workflows. 
-        It's the simplest way to transform Python code into an interactive workflow application.
+        The `prefect-client` package is a minimal-installation of `prefect` which is designed for interacting with Prefect Cloud 
+        or remote any `prefect` server. It sheds some functionality and dependencies in exchange for a smaller installation size, 
+        making it ideal for use in lightweight or ephemeral environments. These characteristics make it ideal for use in lambdas 
+        or other resource-constrained environments. 
         
-        Prefect allows you to expose your workflows through an API so teams dependent on you can programmatically access your pipelines, business logic, and more.
-        Prefect also allows you to standardize workflow development and deployment across your organization.
-        
-        With Prefect, you can build resilient, dynamic workflows that react to the world around them and recover from unexpected changes.
-        With just a few decorators, Prefect supercharges your code with features like automatic retries, distributed execution, scheduling, caching, and much more.
-        
-        Every activity is tracked and can be monitored with a self-hosted [Prefect server](https://docs.prefect.io/latest/guides/host/) instance or managed [Prefect Cloud](https://www.prefect.io/cloud-vs-oss?utm_source=oss&utm_medium=oss&utm_campaign=oss_gh_repo&utm_term=none&utm_content=none) dashboard.
         
         ## Getting started
         
-        Prefect requires Python 3.8 or later. To [install Prefect](https://docs.prefect.io/getting-started/installation/), run the following command:
+        `prefect-client` shares the same installation requirements as prefect. To install, make sure you are on Python 3.8 or 
+        later and run the following command:
         
         ```bash
-        pip install prefect
+        pip install prefect-client
         ```
         
-        Then create and run a Python file that uses Prefect `flow` and `task` decorators to orchestrate and observe your workflow that fetches the number of GitHub stars from a repository.
+        Next, ensure that your `prefect-client` has access to a remote `prefect` server by exporting the `PREFECT_API_KEY` 
+        (if using Prefect Cloud) and `PREFECT_API_URL` environment variables. Once those are set, use the package in your code as 
+        you would normally use `prefect`! 
+        
+        
+        For example, to remotely trigger a run a deployment:
         
         ```python
-        from prefect import flow, task
-        from typing import List
-        import httpx
+        from prefect.deployments import run_deployment
         
         
-        @task(log_prints=True)
-        def get_stars(repo: str):
-            url = f"https://api.github.com/repos/{repo}"
-            count = httpx.get(url).json()["stargazers_count"]
-            print(f"{repo} has {count} stars!")
+        def my_lambda(event):
+            ...
+            run_deployment(
+                name="my-flow/my-deployment",
+                parameters={"foo": "bar"},
+                timeout=0,
+            )
         
+        my_lambda({})
+        ```
         
-        @flow(name="GitHub Stars")
-        def github_stars(repos: List[str]):
-            for repo in repos:
-                get_stars(repo)
+        To emit events in an event driven system:
         
+        ```python
+        from prefect.events import emit_event
         
-        # run the flow!
-        if __name__=="__main__":
-            github_stars(["PrefectHQ/Prefect"])
-        ```
         
-        Fire up the Prefect UI to see what happened:
+        def something_happened():
+            emit_event("my-event", resource={"prefect.resource.id": "foo.bar"})
         
-        ```bash
-        prefect server start
+        something_happened()
         ```
         
-        ![Prefect UI dashboard](/docs/img/ui/cloud-dashboard.png)
-        
-        To run your workflow on a schedule, turn it into a deployment and schedule it to run every minute by changing the last line of your script to the following:
         
+        Or just interact with a `prefect` API:
         ```python
-            github_stars.serve(name="first-deployment", cron="* * * * *")
-        ```
+        from prefect.client.orchestration import get_client
         
-        You now have a server running locally that is looking for scheduled deployments!
-        Additionally you can run your workflow manually from the UI or CLI - and if you're using Prefect Cloud, you can even run deployments in response to [events](https://docs.prefect.io/latest/concepts/automations/).
         
-        ## Prefect Cloud
+        async def query_api():
+            async with get_client() as client:
+                limits = await client.read_concurrency_limits(limit=10, offset=0)
+                print(limits)
+        
+        
+        query_api()
+        ```
         
-        Stop worrying about your workflows.
-        Prefect Cloud allows you to centrally deploy, monitor, and manage the data workflows you support. With managed orchestration, automations, and webhooks, all backed by enterprise-class security, build production-ready code quickly and reliably.
         
-        Read more about Prefect Cloud [here](https://www.prefect.io/cloud-vs-oss?utm_source=oss&utm_medium=oss&utm_campaign=oss_gh_repo&utm_term=none&utm_content=none) or sign up to [try it for yourself](https://app.prefect.cloud?utm_source=oss&utm_medium=oss&utm_campaign=oss_gh_repo&utm_term=none&utm_content=none).
+        ## Known limitations
+        By design, `prefect-client` omits all CLI and server components. This means that the CLI is not available for use 
+        and attempts to access server objects will fail. Furthermore, some classes, methods, and objects may be available 
+        for import in `prefect-client` but may not be "runnable" if they tap into server-oriented functionality. If you 
+        encounter such a limitation, feel free to [open an issue](https://github.com/PrefectHQ/prefect/issues/new/choose) 
+        describing the functionality you are interested in using and we will do our best to make it available.
         
-        ![Prefect Automations](/docs/img/ui/automations.png)
         
         ## Next steps
         

prefect-client-2.16.3/README.md

@@ -0,0 +1,109 @@
+<p align="center"><img src="https://github.com/PrefectHQ/prefect/assets/3407835/c654cbc6-63e8-4ada-a92a-efd2f8f24b85" width=1000></p>
+
+<p align="center">
+    <a href="https://pypi.python.org/pypi/prefect-client/" alt="PyPI version">
+        <img alt="PyPI" src="https://img.shields.io/pypi/v/prefect-client?color=0052FF&labelColor=090422"></a>
+    <a href="https://github.com/prefecthq/prefect/" alt="Stars">
+        <img src="https://img.shields.io/github/stars/prefecthq/prefect?color=0052FF&labelColor=090422" /></a>
+    <a href="https://pepy.tech/badge/prefect-client/" alt="Downloads">
+        <img src="https://img.shields.io/pypi/dm/prefect-client?color=0052FF&labelColor=090422" /></a>
+    <a href="https://github.com/prefecthq/prefect/pulse" alt="Activity">
+        <img src="https://img.shields.io/github/commit-activity/m/prefecthq/prefect?color=0052FF&labelColor=090422" /></a>
+    <br>
+    <a href="https://prefect.io/slack" alt="Slack">
+        <img src="https://img.shields.io/badge/slack-join_community-red.svg?color=0052FF&labelColor=090422&logo=slack" /></a>
+    <a href="https://discourse.prefect.io/" alt="Discourse">
+        <img src="https://img.shields.io/badge/discourse-browse_forum-red.svg?color=0052FF&labelColor=090422&logo=discourse" /></a>
+    <a href="https://www.youtube.com/c/PrefectIO/" alt="YouTube">
+        <img src="https://img.shields.io/badge/youtube-watch_videos-red.svg?color=0052FF&labelColor=090422&logo=youtube" /></a>
+</p>
+
+# prefect-client
+
+The `prefect-client` package is a minimal-installation of `prefect` which is designed for interacting with Prefect Cloud 
+or remote any `prefect` server. It sheds some functionality and dependencies in exchange for a smaller installation size, 
+making it ideal for use in lightweight or ephemeral environments. These characteristics make it ideal for use in lambdas 
+or other resource-constrained environments. 
+
+
+## Getting started
+
+`prefect-client` shares the same installation requirements as prefect. To install, make sure you are on Python 3.8 or 
+later and run the following command:
+
+```bash
+pip install prefect-client
+```
+
+Next, ensure that your `prefect-client` has access to a remote `prefect` server by exporting the `PREFECT_API_KEY` 
+(if using Prefect Cloud) and `PREFECT_API_URL` environment variables. Once those are set, use the package in your code as 
+you would normally use `prefect`! 
+
+
+For example, to remotely trigger a run a deployment:
+
+```python
+from prefect.deployments import run_deployment
+
+
+def my_lambda(event):
+    ...
+    run_deployment(
+        name="my-flow/my-deployment",
+        parameters={"foo": "bar"},
+        timeout=0,
+    )
+
+my_lambda({})
+```
+
+To emit events in an event driven system:
+
+```python
+from prefect.events import emit_event
+
+
+def something_happened():
+    emit_event("my-event", resource={"prefect.resource.id": "foo.bar"})
+
+something_happened()
+```
+
+
+Or just interact with a `prefect` API:
+```python
+from prefect.client.orchestration import get_client
+
+
+async def query_api():
+    async with get_client() as client:
+        limits = await client.read_concurrency_limits(limit=10, offset=0)
+        print(limits)
+
+
+query_api()
+```
+
+
+## Known limitations
+By design, `prefect-client` omits all CLI and server components. This means that the CLI is not available for use 
+and attempts to access server objects will fail. Furthermore, some classes, methods, and objects may be available 
+for import in `prefect-client` but may not be "runnable" if they tap into server-oriented functionality. If you 
+encounter such a limitation, feel free to [open an issue](https://github.com/PrefectHQ/prefect/issues/new/choose) 
+describing the functionality you are interested in using and we will do our best to make it available.
+
+
+## Next steps
+
+There's lots more you can do to orchestrate and observe your workflows with Prefect!
+Start with our [friendly tutorial](https://docs.prefect.io/tutorials) or explore the [core concepts of Prefect workflows](https://docs.prefect.io/concepts/).
+
+## Join the community
+
+Prefect is made possible by the fastest growing community of thousands of friendly data engineers. Join us in building a new kind of workflow system. The [Prefect Slack community](https://prefect.io/slack) is a fantastic place to learn more about Prefect, ask questions, or get help with workflow design. All community forums, including code contributions, issue discussions, and slack messages are subject to our [Code of Conduct](https://discourse.prefect.io/faq).
+
+## Contribute
+
+See our [documentation on contributing to Prefect](https://docs.prefect.io/contributing/overview/).
+
+Thanks for being part of the mission to build a new kind of workflow system and, of course, **happy engineering!**

{prefect-client-2.16.1 → prefect-client-2.16.3}/requirements-client.txt

@@ -23,6 +23,7 @@ pydantic[email]>=1.10.0,!=2.0.0,!=2.0.1,!=2.1.0,<3.0.0
 python_dateutil >= 2.8.2, < 3.0.0
 python-slugify >= 5.0, < 9.0
 pyyaml >= 5.4.1, < 7.0.0
+rfc3339-validator >= 0.1.4, < 0.2.0
 rich >= 11.0, < 14.0
 ruamel.yaml >= 0.17.0
 sniffio >=1.3.0, < 2.0.0

{prefect-client-2.16.1 → prefect-client-2.16.3}/src/prefect/_internal/concurrency/services.py

@@ -91,6 +91,11 @@ class QueueService(abc.ABC, Generic[T]):
             self._remove_instance()
             self._stopped = True
 
+            # Allow asyncio task to be garbage-collected. Its context may contain
+            # references to all Prefect Task calls made during a flow run, through
+            # EngineContext. Issue #10338.
+            self._task = None
+
             # Signal completion to the loop
             self._queue.put_nowait(None)
 

{prefect-client-2.16.1 → prefect-client-2.16.3}/src/prefect/_internal/concurrency/threads.py

@@ -214,6 +214,9 @@ class EventLoopThread(Portal):
         for call in self._on_shutdown:
             await self._run_call(call)
 
+        # Empty the list to allow calls to be garbage collected. Issue #10338.
+        self._on_shutdown = []
+
     async def _run_call(self, call: Call) -> None:
         task = call.run()
         if task is not None:

{prefect-client-2.16.1 → prefect-client-2.16.3}/src/prefect/client/orchestration.py

@@ -2215,6 +2215,24 @@ class PrefectClient:
         response = await self._client.post("/task_runs/filter", json=body)
         return pydantic.parse_obj_as(List[TaskRun], response.json())
 
+    async def delete_task_run(self, task_run_id: UUID) -> None:
+        """
+        Delete a task run by id.
+
+        Args:
+            task_run_id: the task run ID of interest
+        Raises:
+            prefect.exceptions.ObjectNotFound: If request returns 404
+            httpx.RequestError: If requests fails
+        """
+        try:
+            await self._client.delete(f"/task_runs/{task_run_id}")
+        except httpx.HTTPStatusError as e:
+            if e.response.status_code == 404:
+                raise prefect.exceptions.ObjectNotFound(http_exc=e) from e
+            else:
+                raise
+
     async def set_task_run_state(
         self,
         task_run_id: UUID,

{prefect-client-2.16.1 → prefect-client-2.16.3}/src/prefect/deployments/__init__.py

@@ -12,4 +12,9 @@ from prefect.deployments.deployments import (
     load_deployments_from_yaml,
     Deployment,
 )
-from prefect.deployments.runner import RunnerDeployment, deploy, DeploymentImage
+from prefect.deployments.runner import (
+    RunnerDeployment,
+    deploy,
+    DeploymentImage,
+    EntrypointType,
+)

{prefect-client-2.16.1 → prefect-client-2.16.3}/src/prefect/deployments/deployments.py

@@ -36,6 +36,10 @@ from prefect.client.schemas.objects import (
 from prefect.client.schemas.schedules import SCHEDULE_TYPES
 from prefect.client.utilities import inject_client
 from prefect.context import FlowRunContext, PrefectObjectRegistry, TaskRunContext
+from prefect.deployments.schedules import (
+    FlexibleScheduleList,
+    normalize_to_minimal_deployment_schedules,
+)
 from prefect.deployments.steps.core import run_steps
 from prefect.events.schemas import DeploymentTrigger
 from prefect.exceptions import (
@@ -222,12 +226,28 @@ async def load_flow_from_flow_run(
     is largely for testing, and assumes the flow is already available locally.
     """
     deployment = await client.read_deployment(flow_run.deployment_id)
+
+    if deployment.entrypoint is None:
+        raise ValueError(
+            f"Deployment {deployment.id} does not have an entrypoint and can not be run."
+        )
+
     run_logger = flow_run_logger(flow_run)
 
     runner_storage_base_path = storage_base_path or os.environ.get(
         "PREFECT__STORAGE_BASE_PATH"
     )
 
+    # If there's no colon, assume it's a module path
+    if ":" not in deployment.entrypoint:
+        run_logger.debug(
+            f"Importing flow code from module path {deployment.entrypoint}"
+        )
+        flow = await run_sync_in_worker_thread(
+            load_flow_from_entrypoint, deployment.entrypoint
+        )
+        return flow
+
     if not ignore_storage and not deployment.pull_steps:
         sys.path.insert(0, ".")
         if deployment.storage_document_id:
@@ -259,8 +279,6 @@ async def load_flow_from_flow_run(
             os.chdir(output["directory"])
 
     import_path = relative_path_to_current_platform(deployment.entrypoint)
-    run_logger.debug(f"Importing flow code from '{import_path}'")
-
     # for backwards compat
     if deployment.manifest_path:
         with open(deployment.manifest_path, "r") as f:
@@ -268,7 +286,10 @@ async def load_flow_from_flow_run(
             import_path = (
                 Path(deployment.manifest_path).parent / import_path
             ).absolute()
+    run_logger.debug(f"Importing flow code from '{import_path}'")
+
     flow = await run_sync_in_worker_thread(load_flow_from_entrypoint, str(import_path))
+
     return flow
 
 
@@ -632,11 +653,17 @@ class Deployment(BaseModel):
             cls._validate_schedule(value)
         return value
 
-    @validator("schedules")
-    def validate_schedules(cls, value):
-        for schedule in value:
-            cls._validate_schedule(schedule.schedule)
-        return value
+    @root_validator(pre=True)
+    def validate_schedules(cls, values):
+        if "schedules" in values:
+            values["schedules"] = normalize_to_minimal_deployment_schedules(
+                values["schedules"]
+            )
+
+            for schedule in values["schedules"]:
+                cls._validate_schedule(schedule.schedule)
+
+        return values
 
     @classmethod
     @sync_compatible
@@ -919,6 +946,7 @@ class Deployment(BaseModel):
         ignore_file: str = ".prefectignore",
         apply: bool = False,
         load_existing: bool = True,
+        schedules: Optional[FlexibleScheduleList] = None,
         **kwargs,
     ) -> "Deployment":
         """
@@ -938,6 +966,14 @@ class Deployment(BaseModel):
             load_existing: if True, load any settings that may already be configured for
                 the named deployment server-side (e.g., schedules, default parameter
                 values, etc.)
+            schedules: An optional list of schedules. Each item in the list can be:
+                  - An instance of `MinimalDeploymentSchedule`.
+                  - A dictionary with a `schedule` key, and optionally, an
+                    `active` key. The `schedule` key should correspond to a
+                    schedule type, and `active` is a boolean indicating whether
+                    the schedule is active or not.
+                  - An instance of one of the predefined schedule types:
+                    `IntervalSchedule`, `CronSchedule`, or `RRuleSchedule`.
             **kwargs: other keyword arguments to pass to the constructor for the
                 `Deployment` class
         """
@@ -946,7 +982,11 @@ class Deployment(BaseModel):
 
         # note that `deployment.load` only updates settings that were *not*
         # provided at initialization
-        deployment = cls(name=name, **kwargs)
+        deployment = cls(
+            name=name,
+            schedules=schedules,
+            **kwargs,
+        )
         deployment.flow_name = flow.name
         if not deployment.entrypoint:
             ## first see if an entrypoint can be determined