zenml-nightly 0.66.0.dev20240928__py3-none-any.whl → 0.68.1.dev20241102__py3-none-any.whl

README.md

@@ -86,7 +86,7 @@ Thank you for your support! 🌟
 ## 🤸 Quickstart
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/zenml-io/zenml/blob/main/examples/quickstart/quickstart.ipynb)
 
-[Install ZenML](https://docs.zenml.io/getting-started/installation) via [PyPI](https://pypi.org/project/zenml/). Python 3.8 - 3.11 is required:
+[Install ZenML](https://docs.zenml.io/getting-started/installation) via [PyPI](https://pypi.org/project/zenml/). Python 3.9 - 3.12 is required:
 
 ```bash
 pip install "zenml[server]" notebook
@@ -228,22 +228,28 @@ And finally, here are some other examples and use cases for inspiration:
 5. [LLMops](https://github.com/zenml-io/zenml-projects/tree/main/llm-complete-guide): Complete guide to do LLM with ZenML
 
 
+## 📚 Learn from Books
+
+<div align="center">
+  <a href="https://www.amazon.com/LLM-Engineers-Handbook-engineering-production/dp/1836200072">
+    <img src="docs/book/.gitbook/assets/llm_engineering_handbook_cover.jpg" alt="LLM Engineer's Handbook Cover" width="200"/></img>
+  </a>&nbsp;&nbsp;&nbsp;&nbsp;
+  <a href="https://www.amazon.com/-/en/Andrew-McMahon/dp/1837631964">
+    <img src="docs/book/.gitbook/assets/ml_engineering_with_python.jpg" alt="Machine Learning Engineering with Python Cover" width="200"/></img>
+  </a>
+  </br></br>
+</div>
+
+ZenML is featured in these comprehensive guides to modern MLOps and LLM engineering. Learn how to build production-ready machine learning systems with real-world examples and best practices.
+
 ## 🔋 Deploy ZenML
 
 For full functionality ZenML should be deployed on the cloud to
 enable collaborative features as the central MLOps interface for teams.
 
-Currently, there are two main ways to deploy ZenML:
+Read more about various deployment options [here](https://docs.zenml.io/getting-started/deploying-zenml).
 
-- **ZenML Pro**: [ZenML Pro](cloud.zenml.io/?utm_source=readme&utm_medium=referral_link&utm_campaign=cloud_promotion&utm_content=signup_link),
-offers a SaaS version which comes with a control plane to create and manage multiple ZenML servers.
-These servers are managed and maintained by ZenML’s dedicated team, alleviating
-the burden of server management from your end. They also come with additional features like RBAC, a Model Control Plane
-and [much more](https://zenml-io.gitbook.io/zenml-documentation/getting-started/zenml-pro.
-- **Self-hosted deployment**: Alternatively, you have the flexibility to [deploy
-ZenML on your own self-hosted environment](https://docs.zenml.io/getting-started/deploying-zenml#deploying-a-zenml-server).
-This can be achieved through various methods, including using our CLI, Docker,
-Helm, or HuggingFace Spaces.
+Or, sign up for [ZenML Pro to get a fully managed server on a free trial](https://cloud.zenml.io/?utm_source=readme&utm_medium=referral_link&utm_campaign=cloud_promotion&utm_content=signup_link).
 
 ## Use ZenML with VS Code
 
@@ -327,7 +333,7 @@ the Apache License Version 2.0.
     <a href="https://github.com/zenml-io/zenml-projects">Projects Showcase</a>
     <br />
     <br />
-    🎉 Version 0.66.0 is out. Check out the release notes
+    🎉 Version 0.68.1 is out. Check out the release notes
     <a href="https://github.com/zenml-io/zenml/releases">here</a>.
     <br />
     🖥️ Download our VS Code Extension <a href="https://marketplace.visualstudio.com/items?itemName=ZenML.zenml-vscode">here</a>.

RELEASE_NOTES.md

@@ -1,5 +1,241 @@
 <!-- markdown-link-check-disable -->
 
+# 0.68.1
+
+Fixes an issue with some partially cached pipelines running on remote orchestrators.
+
+## What's Changed
+* Remove unavailable upstream steps during cache precomputation by @schustmi in https://github.com/zenml-io/zenml/pull/3146
+
+**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.68.0...0.68.1
+
+# 0.68.0
+
+## Highlights
+
+- **Stack Components on the Dashboard:** We're bringing back stack components. 
+With this release you will get access to the list of your stack components on 
+the ZenML dashboard. More functionality is going to follow in the next releases.
+- **Client-Side Caching:** Implemented client-side computation for cached steps, 
+significantly reducing time and costs associated with remote orchestrator 
+spin-up.
+- **Streamlined Onboarding Process:** Unified the starter and production setup 
+into a single sequential flow, providing a more intuitive user experience.
+- **BentoML Integration:** Updated to version 1.3.5 with enhanced 
+containerization support.
+- **Artifact Management:** Introduced `register_artifact` function enabling 
+direct linking of existing data in the artifact store, particularly useful 
+for tools like PyTorch-Lightning that manage their own checkpoints.
+- **Enhanced Error Handling:** Added Error Boundary to visualization components 
+for improved reliability and user experience.
+
+## Additional Features and Improvements
+
+- Added multiple access points for deleting pipeline runs
+- Improved pipeline detail view functionality
+- Improved service account handling for Kaniko image builder
+
+## Breaking Changes and Deprecations
+
+- Discontinued Python 3.8 support
+- Removed legacy pipeline and step interface
+- Removed legacy post execution workflow
+- Removed legacy dashboard option
+- Removed `zenml stack up/down` CLI commands
+- Removed `zenml deploy` and `zenml <stack-component> deploy`
+- Removed `StepEnvironment` class
+- Removed `ArtifactConfig` class for model version specification
+- Removed `ExternalArtifact` class
+- Deprecated `Client.list_runs` in favor of `Client.list_pipeline_runs`
+- Deprecated `ArtifactVersionResponse.read` in favor of `ArtifactVersionResponse.load`
+
+## Documentation Updates
+
+Added new guides for the following topics:
+
+- Kubernetes per-pod configuration
+- Factory generation of artifact names
+- Common stacks best practices
+- Azure 1-click dashboard deployment
+- ZenML server upgrade best practices
+- Custom Dataset classes and Materializers
+- Comprehensive ZenML Pro documentation
+- Image building optimization during pipeline runs
+- Enhanced BentoML integration documentation
+
+## What's Changed
+* Release 0.67.0 migration testing by @bcdurak in https://github.com/zenml-io/zenml/pull/3050
+* Prevent too large requests by @avishniakov in https://github.com/zenml-io/zenml/pull/3048
+* Fix Neptune linting after 1.12.0 release by @avishniakov in https://github.com/zenml-io/zenml/pull/3055
+* Fix Lightning Orchestrator (remove -y from pip install) by @wjayesh in https://github.com/zenml-io/zenml/pull/3058
+* Fix artifact pruning endpoint path by @schustmi in https://github.com/zenml-io/zenml/pull/3052
+* Update python versioning in docs by @avishniakov in https://github.com/zenml-io/zenml/pull/3059
+* Fix infinite loop while fetching artifact store in logs storage class by @avishniakov in https://github.com/zenml-io/zenml/pull/3061
+* Make sync a setting for sagemaker/azureml orchestrator by @schustmi in https://github.com/zenml-io/zenml/pull/3062
+* Remove some deprecated features by @schustmi in https://github.com/zenml-io/zenml/pull/2926
+* Fix MySQL warning when filtering pipelines by latest run by @schustmi in https://github.com/zenml-io/zenml/pull/3051
+* Remove more deprecated stuff by @schustmi in https://github.com/zenml-io/zenml/pull/3063
+* Remove log versions from versioned buckets in S3 by @avishniakov in https://github.com/zenml-io/zenml/pull/3060
+* add docs on k8s per pod settings by @wjayesh in https://github.com/zenml-io/zenml/pull/3066
+* Remove Python 3.8 support by @strickvl in https://github.com/zenml-io/zenml/pull/3034
+* `register_artifact ` function by @avishniakov in https://github.com/zenml-io/zenml/pull/3053
+* Fix bad link in docs by @avishniakov in https://github.com/zenml-io/zenml/pull/3069
+* Fix model linkage for the lazy loading scenarios by @avishniakov in https://github.com/zenml-io/zenml/pull/3054
+* Updating template versions after the Python 3.8 changes by @bcdurak in https://github.com/zenml-io/zenml/pull/3070
+* Add UUID materializer by @htahir1 in https://github.com/zenml-io/zenml/pull/3073
+* Fix pipeline and model URLs for ZenML Pro on-prem deployments by @stefannica in https://github.com/zenml-io/zenml/pull/3083
+* Update bentoml integration to 1.3.5 and add containerization by @wjayesh in https://github.com/zenml-io/zenml/pull/3045
+* Fix mlflow linting by @schustmi in https://github.com/zenml-io/zenml/pull/3085
+* Add docs for factory generation of artifact names by @strickvl in https://github.com/zenml-io/zenml/pull/3084
+* Remove unnecessary metadata fields in UUID materializer test by @htahir1 in https://github.com/zenml-io/zenml/pull/3088
+* Client-side computation of cached steps by @schustmi in https://github.com/zenml-io/zenml/pull/3068
+* Fix Kaniko image builder service account passing by @schustmi in https://github.com/zenml-io/zenml/pull/3081
+* Bugfix in GitLab Code Repository integration by @4gt-104 in https://github.com/zenml-io/zenml/pull/3076
+* Add docs on common stacks best practices by @strickvl in https://github.com/zenml-io/zenml/pull/3092
+* [docs] Update stacks page and add azure 1-click from dashboard docs by @wjayesh in https://github.com/zenml-io/zenml/pull/3082
+* Local development how-to section by @strickvl in https://github.com/zenml-io/zenml/pull/3090
+* [docs] best practices for upgrading zenml server by @wjayesh in https://github.com/zenml-io/zenml/pull/3087
+* Fix S3 ArtifactStore auth issue by @avishniakov in https://github.com/zenml-io/zenml/pull/3086
+* Reduce migration testing runtime by @avishniakov in https://github.com/zenml-io/zenml/pull/3078
+* [docs] Dedicated docs on how to skip building an image on pipeline run by @wjayesh in https://github.com/zenml-io/zenml/pull/3079
+* Fix filtering by tag for pipeline runs by @schustmi in https://github.com/zenml-io/zenml/pull/3097
+* Remove deprecated features: `zenml deploy` and `zenml <stack-component> deploy` by @stefannica in https://github.com/zenml-io/zenml/pull/3089
+* Do not tag model via `Model` class on creation by @avishniakov in https://github.com/zenml-io/zenml/pull/3098
+* Sagemaker add pipeline tags by @htahir1 in https://github.com/zenml-io/zenml/pull/3080
+* [docs] Add custom Dataset classes and Materializers in ZenML by @htahir1 in https://github.com/zenml-io/zenml/pull/3091
+* Delete Scarf related scripts and workflow files by @htahir1 in https://github.com/zenml-io/zenml/pull/3103
+* Add more detailed docs for ZenML Pro by @wjayesh in https://github.com/zenml-io/zenml/pull/3065
+* Add missing code hash filter in client method by @schustmi in https://github.com/zenml-io/zenml/pull/3094
+* Remove lineage graph and legacy dashboard support by @schustmi in https://github.com/zenml-io/zenml/pull/3064
+* Add unittest to cover gitlab CR regex. by @4gt-104 in https://github.com/zenml-io/zenml/pull/3102
+* Automating the release process using Github workflows by @bcdurak in https://github.com/zenml-io/zenml/pull/3101
+* Bugfix for release automation by @bcdurak in https://github.com/zenml-io/zenml/pull/3107
+* Bugfix for new version in the release automation by @bcdurak in https://github.com/zenml-io/zenml/pull/3108
+* using the right parent image name by @bcdurak in https://github.com/zenml-io/zenml/pull/3109
+* Making the new release automation scripts executable by @bcdurak in https://github.com/zenml-io/zenml/pull/3110
+* Fixing the env variables for the release automation by @bcdurak in https://github.com/zenml-io/zenml/pull/3111
+* Adding the right Github configuration before using the `gh` CLI to fetch the version by @bcdurak in https://github.com/zenml-io/zenml/pull/3112
+* Fixing the outputs of the first step in the release automation by @bcdurak in https://github.com/zenml-io/zenml/pull/3113
+* Handling github auth and release notes for release automation by @bcdurak in https://github.com/zenml-io/zenml/pull/3114
+* Fixing the cloudbuild call for release automation by @bcdurak in https://github.com/zenml-io/zenml/pull/3116
+* Fixing the update tenant call in the script by @bcdurak in https://github.com/zenml-io/zenml/pull/3118
+* Release automation with the new redeploy logic by @bcdurak in https://github.com/zenml-io/zenml/pull/3120
+* Fixing the automation triggers for other branches by @bcdurak in https://github.com/zenml-io/zenml/pull/3125
+* Update link for `llm-complete-guide` repository.- Updated link to poi… by @htahir1 in https://github.com/zenml-io/zenml/pull/3128
+* Fixing the migration testing for the release branches by @bcdurak in https://github.com/zenml-io/zenml/pull/3127
+* Update pipeline deletion docs by @strickvl in https://github.com/zenml-io/zenml/pull/3123
+* Disabling the cache for the quickstart tests by @bcdurak in https://github.com/zenml-io/zenml/pull/3133
+* Update Argilla integration for v2.x SDK by @sdiazlor in https://github.com/zenml-io/zenml/pull/2915
+* Using pip instead of `gh` CLI in the migration tests by @bcdurak in https://github.com/zenml-io/zenml/pull/3136
+* Adapting tags to work with older versions of Sagemaker by @bcdurak in https://github.com/zenml-io/zenml/pull/3135
+* Manual trigger for the `release_finalize` workflow by @bcdurak in https://github.com/zenml-io/zenml/pull/3137
+* Fixing the prepare trigger for the release automation by @bcdurak in https://github.com/zenml-io/zenml/pull/3138
+
+## New Contributors
+* @4gt-104 made their first contribution in https://github.com/zenml-io/zenml/pull/3076
+* @sdiazlor made their first contribution in https://github.com/zenml-io/zenml/pull/2915
+
+**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.67.0...0.68.0
+
+
+# 0.67.0
+
+## Highlights
+
+- **Improved Sagemaker Orchestrator:** Now supports warm pools for AWS Sagemaker, enhancing performance and reducing startup times for TrainingJobs.
+- **New DAG Visualizer:** Shipped major enhancements to the DAG Visualizer for Pipeline Runs:
+  - Preview of the actual DAG before pipeline completion
+  - Visual adjustments for improved clarity
+  - Real-time updates during pipeline execution
+- **Environment Variable References in Configurations:** Introduced the ability to reference environment variables in both code and configuration files using the syntax ${ENV_VARIABLE_NAME}, increasing flexibility in setups.
+- **Enhanced UX for Major Cloud Providers:** Displaying direct pipeline/log URL when working with major cloud platforms.
+- **Skypilot with Kubernetes Support:** Added compatibility for running Skypilot orchestrator on Kubernetes clusters.
+- **Updated Deepchecks Integration:** The Deepchecks integration has been refreshed with the latest features and improvements.
+
+
+## Features and Improvements
+
+- **AWS Integration:**
+  - Added permissions to workflow to enable assuming AWS role.
+  - Fixed expired credentials error when using the docker service connector.
+- **Error Handling:** Improved error messages for stack components of uninstalled integrations.
+- **API Key Management:** Added an option to write API keys to a file instead of using the CLI.
+
+## Pipeline Execution:
+
+- Implemented fixes for executing steps as single step pipelines.
+- Added filter option for templatable runs.
+- Added additional filtering options for pipeline runs.
+- MLflow Integration: Linked registered models in MLflow with the corresponding MLflow run.
+- Analytics: Added missing analytics event to improve user insights.
+
+## Documentation Updates
+
+- Updated documentation for various integrations including:
+  - Lightning AI orchestrator
+  - Kubeflow
+  - Comet experiment tracker
+  - Neptune
+  - Hugging Face deployer
+  - Weights & Biases (wandb)
+- Added documentation for run templates.
+- Fixed incorrect method name in Pigeon docs.
+- Various small documentation fixes and improvements.
+
+## Bug Fixes
+
+- Fixed YAML formatting issues.
+- Resolved RBAC issues for subpages in response models.
+- Fixed step output annotation in Discord test.
+- Addressed MLFlow integration requirements duplication.
+- Fixed Lightning orchestrator functionality.
+
+## What's Changed
+* Error message for stack components of uninstalled integrations by @bcdurak in https://github.com/zenml-io/zenml/pull/2996
+* Enable migration testing for version 0.66.0 by @schustmi in https://github.com/zenml-io/zenml/pull/2998
+* Add permissions to workflow to enable assuming AWS role by @schustmi in https://github.com/zenml-io/zenml/pull/2999
+* Add option to write api key to file instead of CLI by @schustmi in https://github.com/zenml-io/zenml/pull/3001
+* Fix yaml formatting by @schustmi in https://github.com/zenml-io/zenml/pull/3004
+* Update ZenML Pro links for consistency.- Update ZenML Pro links for c… by @htahir1 in https://github.com/zenml-io/zenml/pull/3007
+* Fix incorrect method name in Pigeon docs by @strickvl in https://github.com/zenml-io/zenml/pull/3008
+* Fixes for executing steps as single step pipelines by @schustmi in https://github.com/zenml-io/zenml/pull/3006
+* Add filter option for templatable runs by @schustmi in https://github.com/zenml-io/zenml/pull/3000
+* Add missing analytics event by @schustmi in https://github.com/zenml-io/zenml/pull/3009
+* Fix expired credentials error when using the docker service connector by @schustmi in https://github.com/zenml-io/zenml/pull/3002
+* Fix Lightning docs by @strickvl in https://github.com/zenml-io/zenml/pull/3013
+* Remove image builder warning by @htahir1 in https://github.com/zenml-io/zenml/pull/3014
+* Fixed kubeflow docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/3018
+* Update Comet experiment tracker docs by @htahir1 in https://github.com/zenml-io/zenml/pull/3019
+* Small docs fixes by @strickvl in https://github.com/zenml-io/zenml/pull/3022
+* Feature/cleanup unused file by @AlexejPenner in https://github.com/zenml-io/zenml/pull/3023
+* MLFlow integration requirements duplicate fix by @bcdurak in https://github.com/zenml-io/zenml/pull/3011
+* Fix Neptune docs by @htahir1 in https://github.com/zenml-io/zenml/pull/3026
+* Fix huggingface deployer docs by @htahir1 in https://github.com/zenml-io/zenml/pull/3024
+* Fix step output annotation in Discord test by @wjayesh in https://github.com/zenml-io/zenml/pull/3029
+* Fix RBAC for subpages in response models by @schustmi in https://github.com/zenml-io/zenml/pull/3031
+* Allow env variable placeholders in configurations by @schustmi in https://github.com/zenml-io/zenml/pull/3003
+* Leverage warm pools for AWS Sagemaker by @avishniakov in https://github.com/zenml-io/zenml/pull/3027
+* Updated wandb docs by @htahir1 in https://github.com/zenml-io/zenml/pull/3030
+* Add hyperlint by @htahir1 in https://github.com/zenml-io/zenml/pull/3035
+* Bump NLP template by @avishniakov in https://github.com/zenml-io/zenml/pull/3036
+* Add additional filtering options by @schustmi in https://github.com/zenml-io/zenml/pull/2951
+* Bump starter template version by @schustmi in https://github.com/zenml-io/zenml/pull/3038
+* Docs for run templates by @bcdurak in https://github.com/zenml-io/zenml/pull/3028
+* Update Lightning AI orchestrator documentation by @strickvl in https://github.com/zenml-io/zenml/pull/3016
+* Add default value for PipelineRun.is_templatable by @schustmi in https://github.com/zenml-io/zenml/pull/3040
+* Use a generic OAuth2 client credentials flow to login to the Cloud API by @stefannica in https://github.com/zenml-io/zenml/pull/3041
+* fix lightning orchestrator by @safoinme in https://github.com/zenml-io/zenml/pull/3010
+* Linking registered models in MLflow with the corresponding MLflow run by @aiakide in https://github.com/zenml-io/zenml/pull/3020
+* Bugfixing mlflow registry linting issue by @bcdurak in https://github.com/zenml-io/zenml/pull/3043
+* Enhancing the orchestrator UX for major cloud providers by @bcdurak in https://github.com/zenml-io/zenml/pull/3005
+* Skypilot with Kubernetes by @safoinme in https://github.com/zenml-io/zenml/pull/3033
+* Update deepchecks integration by @wjayesh in https://github.com/zenml-io/zenml/pull/2987
+
+## New Contributors
+* @aiakide made their first contribution in https://github.com/zenml-io/zenml/pull/3020
+
+**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.66.0...0.67.0
+
 # 0.66.0
 
 ## New Features and Improvements

zenml/VERSION

@@ -1 +1 @@
-0.66.0.dev20240928
+0.68.1.dev20241102

zenml/__init__.py

@@ -13,7 +13,6 @@
 #  permissions and limitations under the License.
 """Initialization for ZenML."""
 
-# Define ROOT_DIR
 import os
 
 ROOT_DIR = os.path.dirname(os.path.abspath(__file__))
@@ -32,25 +31,22 @@ init_logging()
 from zenml.models import *  # noqa: F401
 
 # Define public Python API
-from zenml.api import show
+from zenml.zen_server.utils import show_dashboard as show
 from zenml.artifacts.utils import (
     log_artifact_metadata,
     save_artifact,
     load_artifact,
+    register_artifact,
 )
 from zenml.model.utils import (
     log_model_metadata,
     link_artifact_to_model,
-    log_model_version_metadata,
 )
 from zenml.artifacts.artifact_config import ArtifactConfig
 from zenml.artifacts.external_artifact import ExternalArtifact
 from zenml.model.model import Model
-from zenml.model.model_version import ModelVersion # TODO: deprecate me
-from zenml.new.pipelines.pipeline_context import get_pipeline_context
-from zenml.new.pipelines.pipeline_decorator import pipeline
-from zenml.new.steps.step_decorator import step
-from zenml.new.steps.step_context import get_step_context
+from zenml.pipelines import get_pipeline_context, pipeline
+from zenml.steps import step, get_step_context
 from zenml.steps.utils import log_step_metadata
 from zenml.entrypoints import entrypoint
 
@@ -62,13 +58,12 @@ __all__ = [
     "load_artifact",
     "log_artifact_metadata",
     "log_model_metadata",
-    "log_model_version_metadata",
     "log_step_metadata",
     "Model",
-    "ModelVersion", # TODO: deprecate me
     "link_artifact_to_model",
     "pipeline",
     "save_artifact",
+    "register_artifact",
     "show",
     "step",
     "entrypoint",

zenml/analytics/context.py

@@ -25,6 +25,7 @@ from uuid import UUID
 
 from zenml import __version__
 from zenml.analytics.client import default_client
+from zenml.analytics.utils import is_analytics_disabled_internally
 from zenml.constants import (
     ENV_ZENML_SERVER,
     handle_bool_env_var,
@@ -73,14 +74,28 @@ class AnalyticsContext:
         Returns:
             The analytics context.
         """
+        if is_analytics_disabled_internally():
+            self.analytics_opt_in = False
+            return self
+
         # Fetch the analytics opt-in setting
         from zenml.config.global_config import GlobalConfiguration
 
         try:
             gc = GlobalConfiguration()
+
+            if not gc.is_initialized:
+                # If the global configuration is not initialized, using the
+                # zen store can lead to multiple initialization issues, because
+                # the analytics are triggered during the initialization of the
+                # zen store.
+                return self
+
             store_info = gc.zen_store.get_store_info()
 
-            if self.in_server:
+            # For local ZenML servers, we always use the client's analytics
+            # opt-in configuration.
+            if self.in_server and not store_info.is_local():
                 self.analytics_opt_in = store_info.analytics_enabled
             else:
                 self.analytics_opt_in = gc.analytics_opt_in

zenml/analytics/enums.py

@@ -80,24 +80,11 @@ class AnalyticsEvent(str, Enum):
     # Service account and API keys
     CREATED_SERVICE_ACCOUNT = "Service account created"
 
-    # Stack recipes
-    RUN_STACK_RECIPE = "Stack recipe ran"
-    DEPLOY_STACK = "Stack deployed"
-    DESTROY_STACK = "Stack destroyed"
-
-    # Stack component deploy
-    DEPLOY_STACK_COMPONENT = "Stack component deployed"
-    DESTROY_STACK_COMPONENT = "Stack component destroyed"
-
     # Full stack infrastructure deployment
     DEPLOY_FULL_STACK = "Full stack deployed"
 
     # Tag created
     CREATED_TAG = "Tag created"
 
-    # ZenML server events
-    ZENML_SERVER_DEPLOYED = "ZenML server deployed"
-    ZENML_SERVER_DESTROYED = "ZenML server destroyed"
-
     # Server Settings
     SERVER_SETTINGS_UPDATED = "Server Settings Updated"

zenml/analytics/utils.py

@@ -14,7 +14,6 @@
 """Utility functions and classes for ZenML analytics."""
 
 import json
-import os
 from functools import wraps
 from typing import Any, Callable, Dict, Optional, TypeVar, cast
 from uuid import UUID
@@ -22,13 +21,15 @@ from uuid import UUID
 from zenml.analytics import identify, track
 from zenml.analytics.enums import AnalyticsEvent
 from zenml.analytics.models import AnalyticsTrackedModelMixin
-from zenml.constants import ENV_ZENML_ANALYTICS_OPT_IN
 from zenml.logger import get_logger
 
 logger = get_logger(__name__)
 F = TypeVar("F", bound=Callable[..., Any])
 
 
+_INTERNAL_DISABLE_ANALYTICS = False
+
+
 class AnalyticsEncoder(json.JSONEncoder):
     """Helper encoder class for JSON serialization."""
 
@@ -101,11 +102,13 @@ class analytics_disabler:
 
     def __init__(self) -> None:
         """Initialization of the context manager."""
-        self.original_value = os.environ.get(ENV_ZENML_ANALYTICS_OPT_IN)
+        self.original_value: Optional[bool] = None
 
     def __enter__(self) -> None:
         """Disable the analytics."""
-        os.environ[ENV_ZENML_ANALYTICS_OPT_IN] = str(False)
+        global _INTERNAL_DISABLE_ANALYTICS
+        self.original_value = _INTERNAL_DISABLE_ANALYTICS
+        _INTERNAL_DISABLE_ANALYTICS = True
 
     def __exit__(
         self,
@@ -120,10 +123,18 @@ class analytics_disabler:
             exc_value: The instance of the exception
             traceback: The traceback of the exception
         """
+        global _INTERNAL_DISABLE_ANALYTICS
         if self.original_value is not None:
-            os.environ[ENV_ZENML_ANALYTICS_OPT_IN] = self.original_value
-        else:
-            del os.environ[ENV_ZENML_ANALYTICS_OPT_IN]
+            _INTERNAL_DISABLE_ANALYTICS = self.original_value
+
+
+def is_analytics_disabled_internally() -> bool:
+    """Whether analytics are disabled by an internal helper function.
+
+    Returns:
+        Whether analytics are disabled by an internal helper function.
+    """
+    return _INTERNAL_DISABLE_ANALYTICS
 
 
 def track_decorator(event: AnalyticsEvent) -> Callable[[F], F]:

zenml/artifact_stores/base_artifact_store.py

@@ -475,6 +475,17 @@ class BaseArtifactStore(StackComponent):
 
         default_filesystem_registry.register(filesystem_class)
 
+    def _remove_previous_file_versions(self, path: PathType) -> None:
+        """Remove all file versions but the latest in the given path.
+
+        Method is useful for logs stored in versioned file systems
+        like AWS S3.
+
+        Args:
+            path: The path to the file.
+        """
+        return
+
 
 class BaseArtifactStoreFlavor(Flavor):
     """Base class for artifact store flavors."""

zenml/artifacts/artifact_config.py

@@ -13,19 +13,13 @@
 #  permissions and limitations under the License.
 """Artifact Config classes to support Model Control Plane feature."""
 
-from typing import TYPE_CHECKING, Dict, List, Optional, Union
+from typing import Any, Dict, List, Optional, Union
 
-from pydantic import BaseModel, ConfigDict, Field, model_validator
+from pydantic import BaseModel, Field, model_validator
 
-from zenml.enums import ModelStages
-from zenml.exceptions import StepContextError
 from zenml.logger import get_logger
 from zenml.metadata.metadata_types import MetadataType
-from zenml.new.steps.step_context import get_step_context
-
-if TYPE_CHECKING:
-    from zenml.model.model import Model
-
+from zenml.utils.pydantic_utils import before_validator_handler
 
 logger = get_logger(__name__)
 
@@ -43,7 +37,6 @@ class ArtifactConfig(BaseModel):
             name="my_artifact",  # override the default artifact name
             version=42,  # set a custom version
             tags=["tag1", "tag2"],  # set custom tags
-            model_name="my_model",  # link the artifact to a model
         )
     ]:
         return ...
@@ -53,11 +46,7 @@ class ArtifactConfig(BaseModel):
         name: The name of the artifact.
         version: The version of the artifact.
         tags: The tags of the artifact.
-        model_name: The name of the model to link artifact to.
-        model_version: The identifier of a version of the model to link the artifact
-            to. It can be an exact version ("my_version"), exact version number
-            (42), stage (ModelStages.PRODUCTION or "production"), or
-            (ModelStages.LATEST or None) for the latest version (default).
+        run_metadata: Metadata to add to the artifact.
         is_model_artifact: Whether the artifact is a model artifact.
         is_deployment_artifact: Whether the artifact is a deployment artifact.
     """
@@ -69,63 +58,28 @@ class ArtifactConfig(BaseModel):
     tags: Optional[List[str]] = None
     run_metadata: Optional[Dict[str, MetadataType]] = None
 
-    model_name: Optional[str] = None
-    model_version: Optional[Union[ModelStages, str, int]] = Field(
-        default=None, union_mode="smart"
-    )
     is_model_artifact: bool = False
     is_deployment_artifact: bool = False
 
-    # TODO: In Pydantic v2, the `model_` is a protected namespaces for all
-    #  fields defined under base models. If not handled, this raises a warning.
-    #  It is possible to suppress this warning message with the following
-    #  configuration, however the ultimate solution is to rename these fields.
-    #  Even though they do not cause any problems right now, if we are not
-    #  careful we might overwrite some fields protected by pydantic.
-    model_config = ConfigDict(protected_namespaces=())
-
-    @model_validator(mode="after")
-    def artifact_config_validator(self) -> "ArtifactConfig":
-        """Model validator for the artifact config.
+    @model_validator(mode="before")
+    @classmethod
+    @before_validator_handler
+    def _remove_old_attributes(cls, data: Dict[str, Any]) -> Dict[str, Any]:
+        """Remove old attributes that are not used anymore.
 
-        Raises:
-            ValueError: If both model_name and model_version is set incorrectly.
+        Args:
+            data: The model data.
 
         Returns:
-            the validated instance.
+            Model data without the removed attributes.
         """
-        if self.model_name is not None and self.model_version is None:
-            raise ValueError(
-                f"Creation of new model version from {self.__class__.__name__} "
-                "is not allowed. Please either keep `model_name` and "
-                "`model_version` both `None` to get the model version from the "
-                "step context or specify both at the same time. You can use "
-                "`ModelStages.LATEST` as `model_version` when latest model "
-                "version is desired."
-            )
-        return self
-
-    @property
-    def _model(self) -> Optional["Model"]:
-        """The model linked to this artifact.
+        model_name = data.pop("model_name", None)
+        model_version = data.pop("model_version", None)
 
-        Returns:
-            The model or None if the model version cannot be determined.
-        """
-        try:
-            model_ = get_step_context().model
-        except (StepContextError, RuntimeError):
-            model_ = None
-        # Check if another model name was specified
-        if (self.model_name is not None) and (
-            model_ is None or model_.name != self.model_name
-        ):
-            # Create a new Model instance with the provided model name and version
-            from zenml.model.model import Model
-
-            on_the_fly_config = Model(
-                name=self.model_name, version=self.model_version
+        if model_name or model_version:
+            logger.warning(
+                "Specifying a model name or version for a step output "
+                "artifact is not supported anymore."
             )
-            return on_the_fly_config
 
-        return model_
+        return data