zenml-nightly 0.66.0.dev20240930__py3-none-any.whl → 0.67.0.dev20241002__py3-none-any.whl

README.md

@@ -327,7 +327,7 @@ the Apache License Version 2.0.
     <a href="https://github.com/zenml-io/zenml-projects">Projects Showcase</a>
     <br />
     <br />
-    🎉 Version 0.66.0 is out. Check out the release notes
+    🎉 Version 0.67.0 is out. Check out the release notes
     <a href="https://github.com/zenml-io/zenml/releases">here</a>.
     <br />
     🖥️ Download our VS Code Extension <a href="https://marketplace.visualstudio.com/items?itemName=ZenML.zenml-vscode">here</a>.

RELEASE_NOTES.md

@@ -1,5 +1,103 @@
 <!-- markdown-link-check-disable -->
 
+# 0.67.0
+
+## Highlights
+
+- **Improved Sagemaker Orchestrator:** Now supports warm pools for AWS Sagemaker, enhancing performance and reducing startup times for TrainingJobs.
+- **New DAG Visualizer:** Shipped major enhancements to the DAG Visualizer for Pipeline Runs:
+  - Preview of the actual DAG before pipeline completion
+  - Visual adjustments for improved clarity
+  - Real-time updates during pipeline execution
+- **Environment Variable References in Configurations:** Introduced the ability to reference environment variables in both code and configuration files using the syntax ${ENV_VARIABLE_NAME}, increasing flexibility in setups.
+- **Enhanced UX for Major Cloud Providers:** Displaying direct pipeline/log URL when working with major cloud platforms.
+- **Skypilot with Kubernetes Support:** Added compatibility for running Skypilot orchestrator on Kubernetes clusters.
+- **Updated Deepchecks Integration:** The Deepchecks integration has been refreshed with the latest features and improvements.
+
+
+## Features and Improvements
+
+- **AWS Integration:**
+  - Added permissions to workflow to enable assuming AWS role.
+  - Fixed expired credentials error when using the docker service connector.
+- **Error Handling:** Improved error messages for stack components of uninstalled integrations.
+- **API Key Management:** Added an option to write API keys to a file instead of using the CLI.
+
+## Pipeline Execution:
+
+- Implemented fixes for executing steps as single step pipelines.
+- Added filter option for templatable runs.
+- Added additional filtering options for pipeline runs.
+- MLflow Integration: Linked registered models in MLflow with the corresponding MLflow run.
+- Analytics: Added missing analytics event to improve user insights.
+
+## Documentation Updates
+
+- Updated documentation for various integrations including:
+  - Lightning AI orchestrator
+  - Kubeflow
+  - Comet experiment tracker
+  - Neptune
+  - Hugging Face deployer
+  - Weights & Biases (wandb)
+- Added documentation for run templates.
+- Fixed incorrect method name in Pigeon docs.
+- Various small documentation fixes and improvements.
+
+## Bug Fixes
+
+- Fixed YAML formatting issues.
+- Resolved RBAC issues for subpages in response models.
+- Fixed step output annotation in Discord test.
+- Addressed MLFlow integration requirements duplication.
+- Fixed Lightning orchestrator functionality.
+
+## What's Changed
+* Error message for stack components of uninstalled integrations by @bcdurak in https://github.com/zenml-io/zenml/pull/2996
+* Enable migration testing for version 0.66.0 by @schustmi in https://github.com/zenml-io/zenml/pull/2998
+* Add permissions to workflow to enable assuming AWS role by @schustmi in https://github.com/zenml-io/zenml/pull/2999
+* Add option to write api key to file instead of CLI by @schustmi in https://github.com/zenml-io/zenml/pull/3001
+* Fix yaml formatting by @schustmi in https://github.com/zenml-io/zenml/pull/3004
+* Update ZenML Pro links for consistency.- Update ZenML Pro links for c… by @htahir1 in https://github.com/zenml-io/zenml/pull/3007
+* Fix incorrect method name in Pigeon docs by @strickvl in https://github.com/zenml-io/zenml/pull/3008
+* Fixes for executing steps as single step pipelines by @schustmi in https://github.com/zenml-io/zenml/pull/3006
+* Add filter option for templatable runs by @schustmi in https://github.com/zenml-io/zenml/pull/3000
+* Add missing analytics event by @schustmi in https://github.com/zenml-io/zenml/pull/3009
+* Fix expired credentials error when using the docker service connector by @schustmi in https://github.com/zenml-io/zenml/pull/3002
+* Fix Lightning docs by @strickvl in https://github.com/zenml-io/zenml/pull/3013
+* Remove image builder warning by @htahir1 in https://github.com/zenml-io/zenml/pull/3014
+* Fixed kubeflow docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/3018
+* Update Comet experiment tracker docs by @htahir1 in https://github.com/zenml-io/zenml/pull/3019
+* Small docs fixes by @strickvl in https://github.com/zenml-io/zenml/pull/3022
+* Feature/cleanup unused file by @AlexejPenner in https://github.com/zenml-io/zenml/pull/3023
+* MLFlow integration requirements duplicate fix by @bcdurak in https://github.com/zenml-io/zenml/pull/3011
+* Fix Neptune docs by @htahir1 in https://github.com/zenml-io/zenml/pull/3026
+* Fix huggingface deployer docs by @htahir1 in https://github.com/zenml-io/zenml/pull/3024
+* Fix step output annotation in Discord test by @wjayesh in https://github.com/zenml-io/zenml/pull/3029
+* Fix RBAC for subpages in response models by @schustmi in https://github.com/zenml-io/zenml/pull/3031
+* Allow env variable placeholders in configurations by @schustmi in https://github.com/zenml-io/zenml/pull/3003
+* Leverage warm pools for AWS Sagemaker by @avishniakov in https://github.com/zenml-io/zenml/pull/3027
+* Updated wandb docs by @htahir1 in https://github.com/zenml-io/zenml/pull/3030
+* Add hyperlint by @htahir1 in https://github.com/zenml-io/zenml/pull/3035
+* Bump NLP template by @avishniakov in https://github.com/zenml-io/zenml/pull/3036
+* Add additional filtering options by @schustmi in https://github.com/zenml-io/zenml/pull/2951
+* Bump starter template version by @schustmi in https://github.com/zenml-io/zenml/pull/3038
+* Docs for run templates by @bcdurak in https://github.com/zenml-io/zenml/pull/3028
+* Update Lightning AI orchestrator documentation by @strickvl in https://github.com/zenml-io/zenml/pull/3016
+* Add default value for PipelineRun.is_templatable by @schustmi in https://github.com/zenml-io/zenml/pull/3040
+* Use a generic OAuth2 client credentials flow to login to the Cloud API by @stefannica in https://github.com/zenml-io/zenml/pull/3041
+* fix lightning orchestrator by @safoinme in https://github.com/zenml-io/zenml/pull/3010
+* Linking registered models in MLflow with the corresponding MLflow run by @aiakide in https://github.com/zenml-io/zenml/pull/3020
+* Bugfixing mlflow registry linting issue by @bcdurak in https://github.com/zenml-io/zenml/pull/3043
+* Enhancing the orchestrator UX for major cloud providers by @bcdurak in https://github.com/zenml-io/zenml/pull/3005
+* Skypilot with Kubernetes by @safoinme in https://github.com/zenml-io/zenml/pull/3033
+* Update deepchecks integration by @wjayesh in https://github.com/zenml-io/zenml/pull/2987
+
+## New Contributors
+* @aiakide made their first contribution in https://github.com/zenml-io/zenml/pull/3020
+
+**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.66.0...0.67.0
+
 # 0.66.0
 
 ## New Features and Improvements

zenml/VERSION

@@ -1 +1 @@
-0.66.0.dev20240930
+0.67.0.dev20241002

zenml/integrations/pytorch/materializers/base_pytorch_materializer.py

@@ -41,7 +41,10 @@ class BasePyTorchMaterializer(BaseMaterializer):
             The loaded PyTorch object.
         """
         with fileio.open(os.path.join(self.uri, self.FILENAME), "rb") as f:
-            return torch.load(f)
+            # NOTE (security): The `torch.load` function uses `pickle` as
+            # the default unpickler, which is NOT secure. This materializer
+            # is intended for use with trusted data sources.
+            return torch.load(f)  # nosec
 
     def save(self, obj: Any) -> None:
         """Uses `torch.save` to save a PyTorch object.
@@ -50,7 +53,10 @@ class BasePyTorchMaterializer(BaseMaterializer):
             obj: The PyTorch object to save.
         """
         with fileio.open(os.path.join(self.uri, self.FILENAME), "wb") as f:
-            torch.save(obj, f, pickle_module=cloudpickle)
+            # NOTE (security): The `torch.save` function uses `cloudpickle` as
+            # the default unpickler, which is NOT secure. This materializer
+            # is intended for use with trusted data sources.
+            torch.save(obj, f, pickle_module=cloudpickle)  # nosec
 
 
 # Alias for the BasePyTorchMaterializer class, allowing users that have already used

zenml/integrations/pytorch/materializers/pytorch_module_materializer.py

@@ -61,7 +61,10 @@ class PyTorchModuleMaterializer(BasePyTorchMaterializer):
             with fileio.open(
                 os.path.join(self.uri, CHECKPOINT_FILENAME), "wb"
             ) as f:
-                torch.save(model.state_dict(), f, pickle_module=cloudpickle)
+                # NOTE (security): The `torch.save` function uses `cloudpickle` as
+                # the default unpickler, which is NOT secure. This materializer
+                # is intended for use with trusted data sources.
+                torch.save(model.state_dict(), f, pickle_module=cloudpickle)  # nosec
 
     def extract_metadata(self, model: Module) -> Dict[str, "MetadataType"]:
         """Extract metadata from the given `Model` object.

zenml/zen_server/deploy/helm/Chart.yaml

@@ -1,6 +1,6 @@
 apiVersion: v2
 name: zenml
-version: "0.66.0"
+version: "0.67.0"
 description: Open source MLOps framework for portable production ready ML pipelines
 keywords:
 - mlops

zenml/zen_server/deploy/helm/README.md

@@ -20,8 +20,8 @@ ZenML is an open-source MLOps framework designed to help you create robust, main
 To install the ZenML chart directly from Amazon ECR, use the following command:
 
 ```bash
-# example command for version 0.66.0
-helm install my-zenml oci://public.ecr.aws/zenml/zenml --version 0.66.0
+# example command for version 0.67.0
+helm install my-zenml oci://public.ecr.aws/zenml/zenml --version 0.67.0
 ```
 
 Note: Ensure you have OCI support enabled in your Helm client and that you are authenticated with Amazon ECR.

zenml/zen_stores/migrations/versions/0.67.0_release.py

@@ -0,0 +1,23 @@
+"""Release [0.67.0].
+
+Revision ID: 0.67.0
+Revises: 0.66.0
+Create Date: 2024-09-25 10:39:36.711715
+
+"""
+
+# revision identifiers, used by Alembic.
+revision = "0.67.0"
+down_revision = "0.66.0"
+branch_labels = None
+depends_on = None
+
+
+def upgrade() -> None:
+    """Upgrade database schema and/or data, creating a new revision."""
+    pass
+
+
+def downgrade() -> None:
+    """Downgrade database schema and/or data back to the previous revision."""
+    pass

{zenml_nightly-0.66.0.dev20240930.dist-info → zenml_nightly-0.67.0.dev20241002.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: zenml-nightly
-Version: 0.66.0.dev20240930
+Version: 0.67.0.dev20241002
 Summary: ZenML: Write production-ready ML code.
 Home-page: https://zenml.io
 License: Apache-2.0
@@ -472,7 +472,7 @@ the Apache License Version 2.0.
     <a href="https://github.com/zenml-io/zenml-projects">Projects Showcase</a>
     <br />
     <br />
-    🎉 Version 0.66.0 is out. Check out the release notes
+    🎉 Version 0.67.0 is out. Check out the release notes
     <a href="https://github.com/zenml-io/zenml/releases">here</a>.
     <br />
     🖥️ Download our VS Code Extension <a href="https://marketplace.visualstudio.com/items?itemName=ZenML.zenml-vscode">here</a>.

{zenml_nightly-0.66.0.dev20240930.dist-info → zenml_nightly-0.67.0.dev20241002.dist-info}/RECORD

@@ -1,12 +1,12 @@
 CLA.md,sha256=_ZIvAMX21mdV8bzxQLOYGIDu5vw8WSG7zQwQ_ea27XM,11654
 CODE-OF-CONDUCT.md,sha256=9zBNLt5KHB64X_qwNWaKWl7BvXkFUEluWd5heLPYQBM,5496
 CONTRIBUTING.md,sha256=bls3RAZF76sz531nvErQM9EiyeLACOluusZ8-pyH9j4,11488
-README.md,sha256=yNXIkuhBE5B8yUE_lE0CUJvI4JMRXVpmCNM89dcWRGw,13741
-RELEASE_NOTES.md,sha256=GjuHlUd-5AcepXC8ETMkIRMxKVQtBekiuY_CW0oP4XQ,365106
+README.md,sha256=iw5cHwauKn5gqGAYm-n7UzLwTMAIzt6MjrcihXKUCJE,13741
+RELEASE_NOTES.md,sha256=2plIwWFgPFsVu4kQkwo3U6m8yn_Cps7nxheep56XQ0Y,371604
 ROADMAP.md,sha256=hiLSmr16BH8Dfx7SaQM4JcXCGCVl6mFZPFAwJeDTrJU,407
 SECURITY.md,sha256=9DepA8y03yvCZLHEfcXLTDH4lUyKHquAdukBsccNN7c,682
 zenml/README.md,sha256=827dekbOWAs1BpW7VF1a4d7EbwPbjwccX-2zdXBENZo,1777
-zenml/VERSION,sha256=TuTd-nG56z5qzBJOnqc-4wrrFW2mqNu84SM9mLBuMIY,19
+zenml/VERSION,sha256=sDTHhZELTMjA3ICAgBew5MrjwCglbSLYo6vTTCZb4uc,19
 zenml/__init__.py,sha256=X_JnjAYTBG2ZYgcM2_FDPssygdj7z7VTAmU9HEi17p0,2306
 zenml/actions/__init__.py,sha256=mrt6wPo73iKRxK754_NqsGyJ3buW7RnVeIGXr1xEw8Y,681
 zenml/actions/base_action.py,sha256=UcaHev6BTuLDwuswnyaPjdA8AgUqB5xPZ-lRtuvf2FU,25553
@@ -435,9 +435,9 @@ zenml/integrations/pycaret/materializers/__init__.py,sha256=7I9prC68U3-qE7j5iXSD
 zenml/integrations/pycaret/materializers/model_materializer.py,sha256=sIOISb82OpXkqjfumNaCG7mt09QqHAOCa6jE_IHFgEc,4750
 zenml/integrations/pytorch/__init__.py,sha256=B8xtFRfFgUqpC28KY_y_o9BMNWvlxqZlJiaJnw-rUpQ,1163
 zenml/integrations/pytorch/materializers/__init__.py,sha256=m4laxcEZpP-Po_s6ch9FHHyjLxZ8Ft8TBfU_MpXIy-Q,920
-zenml/integrations/pytorch/materializers/base_pytorch_materializer.py,sha256=LxflaFIYukrIb7UEUl5_yvs26AhrM_-MoJlprPGL9Uw,2012
+zenml/integrations/pytorch/materializers/base_pytorch_materializer.py,sha256=1mtQQ7j_JyqZn9BbYX9mrpRO27KkG54XUYJBVW-Qt3w,2457
 zenml/integrations/pytorch/materializers/pytorch_dataloader_materializer.py,sha256=XEw4P3VQS8uMKuQ24twR6N_1RA882yOeee8iInV6pUQ,2301
-zenml/integrations/pytorch/materializers/pytorch_module_materializer.py,sha256=ad9A657P7M0mhoHlLBbfYILk84D-1R1XOun5Ne9exwM,2720
+zenml/integrations/pytorch/materializers/pytorch_module_materializer.py,sha256=O5V4yUAoK6wZdJT1RVn6Ui2JoH4yaMrIoLDjDKgViGA,2957
 zenml/integrations/pytorch/utils.py,sha256=IbJTsMb5zYcGlxZj0xKIFQuPHdnizWMd_zmeM6B7v-M,1242
 zenml/integrations/pytorch_lightning/__init__.py,sha256=tk1Io7xI6HbwHkUauvXMs9xQRb1oploneHCmRIcbeAE,1177
 zenml/integrations/pytorch_lightning/materializers/__init__.py,sha256=zZsRri9X8YMJy4_h-8_Uss5RHjkFkd-RGF9zKnd713Q,814
@@ -1087,8 +1087,8 @@ zenml/zen_server/deploy/docker/docker_provider.py,sha256=SjGiYcE3U2riQn5SbhlFzz6
 zenml/zen_server/deploy/docker/docker_zen_server.py,sha256=jIkndpWSGDM2pv4RTXDyjlVwC-jBs3FBU3d3TysE7lA,7199
 zenml/zen_server/deploy/exceptions.py,sha256=tX0GNnLB_GMkeN7zGNlJRwtlrpZ5Slvyj_unVYVmGxk,1396
 zenml/zen_server/deploy/helm/.helmignore,sha256=u5h-ao70WpklXR1jKBJILV8PMlXqhBUgtNkDQt9f4rM,349
-zenml/zen_server/deploy/helm/Chart.yaml,sha256=BydefHIrV30REOUD5BtR5vDRbx4sU4Bl7EmX6Dw-3jo,333
-zenml/zen_server/deploy/helm/README.md,sha256=K20wD3WtudfDfL2Gwrks9wGZu3bjGF3wulPN6Km-97w,1777
+zenml/zen_server/deploy/helm/Chart.yaml,sha256=r_OFyz_VPSEQQMkoJUtflxrh8ar3fn790G2RA1YvFVw,333
+zenml/zen_server/deploy/helm/README.md,sha256=andm4bXFCk3IqIbQ3kWtjWtniCw8SlKACm910b3waAs,1777
 zenml/zen_server/deploy/helm/templates/NOTES.txt,sha256=1APbGHYNf5cKDyDXkTYE3ybVBSPrYiSJh4dKCUOtH2c,1977
 zenml/zen_server/deploy/helm/templates/_environment.tpl,sha256=4tPmEJZc2V1grGuxZQCPvfMUdn4iVt4sNnA8AZlxrEs,14494
 zenml/zen_server/deploy/helm/templates/_helpers.tpl,sha256=SuPztyigDEA00ftUfQtDCNK25RWZB4x58BXthNLJKJA,2042
@@ -1277,6 +1277,7 @@ zenml/zen_stores/migrations/versions/0.63.0_release.py,sha256=CIs6q7mzVbw0w-bgnT
 zenml/zen_stores/migrations/versions/0.64.0_release.py,sha256=ym80JerzU1vzgfQMtECCDG6KIhRjTWL32VXIG2daf2Q,462
 zenml/zen_stores/migrations/versions/0.65.0_release.py,sha256=_sDVoEsWpu2AbRwmRTf7hTlAPMcwe4g-X4sSR9gUw5c,462
 zenml/zen_stores/migrations/versions/0.66.0_release.py,sha256=0Ch5NxErsxtoYJgchwaZLYkUTGoIOtxi_dcrO-XpVYM,450
+zenml/zen_stores/migrations/versions/0.67.0_release.py,sha256=uUZfHxO7xQI_UABYeR9jVIgZkA-NUksq6QGhsG8GbD4,450
 zenml/zen_stores/migrations/versions/026d4577b6a0_add_code_path.py,sha256=hXLzvQcylNrbCVD6vha52PFkSPNC2klW9kA0vuQX_cE,1091
 zenml/zen_stores/migrations/versions/03742aa7fdd7_add_secrets.py,sha256=gewKqu1AnzvNTjVvK1eaAwP0hVneWDUyDRSLTvRCdpg,1587
 zenml/zen_stores/migrations/versions/0701da9951a0_added_service_table.py,sha256=dmMUbQhcQparYdV9nfxcgcJTwlEbUWJlzMQ9d0YQyrs,3399
@@ -1407,8 +1408,8 @@ zenml/zen_stores/secrets_stores/sql_secrets_store.py,sha256=Bq1djrUP9saoD7vECjS7
 zenml/zen_stores/sql_zen_store.py,sha256=h96CArBeXYj3iGEfccYg5FkhWjoSwUjGMWRTjPoOUyY,392556
 zenml/zen_stores/template_utils.py,sha256=UB_CBoMIfBequa8g3H7aTI6V_ke1SJr4IW1qwFCL1jc,8932
 zenml/zen_stores/zen_store_interface.py,sha256=kzR_i8vHjULld3MquSaMorcab8lJk1e9RZquw1VXjHY,93510
-zenml_nightly-0.66.0.dev20240930.dist-info/LICENSE,sha256=wbnfEnXnafPbqwANHkV6LUsPKOtdpsd-SNw37rogLtc,11359
-zenml_nightly-0.66.0.dev20240930.dist-info/METADATA,sha256=4xnAvnUKcDCzVxKUT4DuKAzwijhQ9WdPi3iFYyWMJ6U,21272
-zenml_nightly-0.66.0.dev20240930.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
-zenml_nightly-0.66.0.dev20240930.dist-info/entry_points.txt,sha256=QK3ETQE0YswAM2mWypNMOv8TLtr7EjnqAFq1br_jEFE,43
-zenml_nightly-0.66.0.dev20240930.dist-info/RECORD,,
+zenml_nightly-0.67.0.dev20241002.dist-info/LICENSE,sha256=wbnfEnXnafPbqwANHkV6LUsPKOtdpsd-SNw37rogLtc,11359
+zenml_nightly-0.67.0.dev20241002.dist-info/METADATA,sha256=AG_lpa_8Rm-9SOTZzb-M6aANYZ3ssRFSpR7_GiIq3Ww,21272
+zenml_nightly-0.67.0.dev20241002.dist-info/WHEEL,sha256=sP946D7jFCHeNz5Iq4fL4Lu-PrWrFsgfLXbbkciIZwg,88
+zenml_nightly-0.67.0.dev20241002.dist-info/entry_points.txt,sha256=QK3ETQE0YswAM2mWypNMOv8TLtr7EjnqAFq1br_jEFE,43
+zenml_nightly-0.67.0.dev20241002.dist-info/RECORD,,