zenml-nightly 0.55.0.dev20240124__py3-none-any.whl → 0.72.0.dev20250116__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- zenml/VERSION +1 -1
- zenml/__init__.py +9 -12
- zenml/actions/__init__.py +14 -0
- zenml/actions/base_action.py +682 -0
- zenml/actions/pipeline_run/__init__.py +0 -0
- zenml/actions/pipeline_run/pipeline_run_action.py +223 -0
- zenml/analytics/context.py +70 -15
- zenml/analytics/enums.py +21 -17
- zenml/analytics/models.py +1 -0
- zenml/analytics/utils.py +19 -7
- zenml/annotators/base_annotator.py +4 -3
- zenml/artifact_stores/base_artifact_store.py +155 -72
- zenml/artifact_stores/local_artifact_store.py +3 -2
- zenml/artifacts/artifact_config.py +67 -58
- zenml/artifacts/external_artifact.py +24 -39
- zenml/artifacts/external_artifact_config.py +27 -37
- zenml/artifacts/preexisting_data_materializer.py +90 -0
- zenml/artifacts/unmaterialized_artifact.py +2 -12
- zenml/artifacts/utils.py +521 -220
- zenml/cli/__init__.py +1511 -456
- zenml/cli/annotator.py +48 -14
- zenml/cli/artifact.py +22 -4
- zenml/cli/authorized_device.py +1 -0
- zenml/cli/base.py +185 -22
- zenml/cli/code_repository.py +1 -0
- zenml/cli/formatter.py +1 -1
- zenml/cli/integration.py +119 -24
- zenml/cli/login.py +1053 -0
- zenml/cli/model.py +42 -15
- zenml/cli/model_registry.py +1 -2
- zenml/cli/pipeline.py +168 -82
- zenml/cli/secret.py +1 -2
- zenml/cli/served_model.py +56 -26
- zenml/cli/server.py +486 -619
- zenml/cli/service_accounts.py +55 -14
- zenml/cli/service_connectors.py +97 -6
- zenml/cli/stack.py +821 -531
- zenml/cli/stack_components.py +15 -598
- zenml/cli/tag.py +1 -0
- zenml/cli/text_utils.py +36 -2
- zenml/cli/user_management.py +204 -6
- zenml/cli/utils.py +411 -290
- zenml/client.py +1742 -298
- zenml/client_lazy_loader.py +224 -0
- zenml/code_repositories/base_code_repository.py +5 -4
- zenml/code_repositories/git/local_git_repository_context.py +1 -0
- zenml/code_repositories/local_repository_context.py +1 -0
- zenml/config/__init__.py +2 -0
- zenml/config/base_settings.py +6 -6
- zenml/config/build_configuration.py +43 -17
- zenml/config/compiler.py +82 -49
- zenml/config/docker_settings.py +139 -83
- zenml/config/global_config.py +260 -234
- zenml/config/pipeline_configurations.py +32 -11
- zenml/config/pipeline_run_configuration.py +15 -3
- zenml/config/pipeline_spec.py +6 -6
- zenml/config/resource_settings.py +8 -9
- zenml/config/retry_config.py +27 -0
- zenml/config/schedule.py +27 -18
- zenml/config/secret_reference_mixin.py +8 -4
- zenml/config/secrets_store_config.py +16 -24
- zenml/config/server_config.py +434 -51
- zenml/config/settings_resolver.py +2 -1
- zenml/config/source.py +97 -31
- zenml/config/step_configurations.py +83 -39
- zenml/config/step_run_info.py +3 -0
- zenml/config/store_config.py +20 -54
- zenml/config/strict_base_model.py +2 -6
- zenml/console.py +0 -1
- zenml/constants.py +232 -74
- zenml/container_registries/azure_container_registry.py +1 -0
- zenml/container_registries/base_container_registry.py +7 -3
- zenml/container_registries/default_container_registry.py +4 -3
- zenml/container_registries/dockerhub_container_registry.py +1 -0
- zenml/container_registries/gcp_container_registry.py +1 -0
- zenml/container_registries/github_container_registry.py +2 -10
- zenml/data_validators/base_data_validator.py +2 -2
- zenml/entrypoints/base_entrypoint_configuration.py +76 -17
- zenml/entrypoints/pipeline_entrypoint_configuration.py +1 -0
- zenml/entrypoints/step_entrypoint_configuration.py +21 -2
- zenml/enums.py +91 -9
- zenml/environment.py +52 -319
- zenml/{steps/base_parameters.py → event_hub/__init__.py} +5 -7
- zenml/event_hub/base_event_hub.py +196 -0
- zenml/event_hub/event_hub.py +181 -0
- zenml/event_sources/__init__.py +14 -0
- zenml/{lineage_graph/edge.py → event_sources/base_event.py} +5 -7
- zenml/event_sources/base_event_source.py +695 -0
- zenml/event_sources/webhooks/__init__.py +14 -0
- zenml/event_sources/webhooks/base_webhook_event_source.py +231 -0
- zenml/exceptions.py +40 -41
- zenml/feature_stores/base_feature_store.py +4 -6
- zenml/hooks/hook_validators.py +3 -11
- zenml/image_builders/base_image_builder.py +5 -2
- zenml/image_builders/build_context.py +24 -80
- zenml/image_builders/local_image_builder.py +14 -6
- zenml/integrations/__init__.py +16 -4
- zenml/integrations/airflow/__init__.py +3 -5
- zenml/integrations/airflow/flavors/airflow_orchestrator_flavor.py +15 -7
- zenml/integrations/airflow/orchestrators/airflow_orchestrator.py +15 -252
- zenml/integrations/airflow/orchestrators/dag_generator.py +5 -3
- zenml/integrations/argilla/__init__.py +46 -0
- zenml/integrations/argilla/annotators/__init__.py +20 -0
- zenml/integrations/argilla/annotators/argilla_annotator.py +443 -0
- zenml/{post_execution → integrations/argilla/flavors}/__init__.py +9 -13
- zenml/integrations/argilla/flavors/argilla_annotator_flavor.py +150 -0
- zenml/integrations/aws/__init__.py +7 -3
- zenml/integrations/aws/container_registries/aws_container_registry.py +44 -8
- zenml/integrations/aws/flavors/__init__.py +6 -0
- zenml/integrations/aws/flavors/aws_container_registry_flavor.py +3 -2
- zenml/integrations/aws/flavors/aws_image_builder_flavor.py +146 -0
- zenml/integrations/aws/flavors/sagemaker_orchestrator_flavor.py +104 -16
- zenml/integrations/aws/flavors/sagemaker_step_operator_flavor.py +6 -2
- zenml/integrations/aws/image_builders/__init__.py +20 -0
- zenml/integrations/aws/image_builders/aws_image_builder.py +307 -0
- zenml/integrations/aws/orchestrators/sagemaker_orchestrator.py +421 -133
- zenml/integrations/aws/service_connectors/aws_service_connector.py +175 -48
- zenml/integrations/aws/step_operators/sagemaker_step_operator.py +1 -1
- zenml/integrations/azure/__init__.py +14 -5
- zenml/integrations/azure/azureml_utils.py +201 -0
- zenml/integrations/azure/flavors/__init__.py +11 -0
- zenml/integrations/azure/flavors/azureml.py +139 -0
- zenml/integrations/azure/flavors/azureml_orchestrator_flavor.py +166 -0
- zenml/integrations/azure/flavors/azureml_step_operator_flavor.py +71 -18
- zenml/integrations/azure/orchestrators/__init__.py +19 -0
- zenml/integrations/azure/orchestrators/azureml_orchestrator.py +583 -0
- zenml/integrations/azure/orchestrators/azureml_orchestrator_entrypoint_config.py +82 -0
- zenml/integrations/azure/service_connectors/azure_service_connector.py +15 -6
- zenml/integrations/azure/step_operators/azureml_step_operator.py +78 -173
- zenml/integrations/bentoml/__init__.py +1 -1
- zenml/integrations/bentoml/constants.py +1 -1
- zenml/integrations/bentoml/materializers/bentoml_bento_materializer.py +19 -31
- zenml/integrations/bentoml/model_deployers/bentoml_model_deployer.py +128 -239
- zenml/integrations/bentoml/services/__init__.py +15 -4
- zenml/integrations/bentoml/services/bentoml_container_deployment.py +399 -0
- zenml/integrations/bentoml/services/{bentoml_deployment.py → bentoml_local_deployment.py} +85 -43
- zenml/integrations/bentoml/services/deployment_type.py +23 -0
- zenml/integrations/bentoml/steps/bento_builder.py +2 -0
- zenml/integrations/bentoml/steps/bentoml_deployer.py +97 -47
- zenml/integrations/bitbucket/__init__.py +42 -0
- zenml/integrations/bitbucket/plugins/__init__.py +20 -0
- zenml/integrations/bitbucket/plugins/bitbucket_webhook_event_source_flavor.py +43 -0
- zenml/integrations/bitbucket/plugins/event_sources/__init__.py +0 -0
- zenml/integrations/bitbucket/plugins/event_sources/bitbucket_webhook_event_source.py +486 -0
- zenml/integrations/comet/__init__.py +49 -0
- zenml/integrations/{kserve/services → comet/experiment_trackers}/__init__.py +5 -6
- zenml/integrations/comet/experiment_trackers/comet_experiment_tracker.py +175 -0
- zenml/integrations/comet/flavors/__init__.py +24 -0
- zenml/integrations/comet/flavors/comet_experiment_tracker_flavor.py +129 -0
- zenml/integrations/constants.py +14 -2
- zenml/integrations/databricks/__init__.py +70 -0
- zenml/{lineage_graph → integrations/databricks/flavors}/__init__.py +14 -18
- zenml/integrations/databricks/flavors/databricks_model_deployer_flavor.py +118 -0
- zenml/integrations/databricks/flavors/databricks_orchestrator_flavor.py +174 -0
- zenml/integrations/{kserve → databricks}/model_deployers/__init__.py +5 -5
- zenml/integrations/databricks/model_deployers/databricks_model_deployer.py +249 -0
- zenml/integrations/databricks/orchestrators/__init__.py +20 -0
- zenml/integrations/databricks/orchestrators/databricks_orchestrator.py +501 -0
- zenml/integrations/databricks/orchestrators/databricks_orchestrator_entrypoint_config.py +97 -0
- zenml/integrations/databricks/services/__init__.py +19 -0
- zenml/integrations/databricks/services/databricks_deployment.py +407 -0
- zenml/integrations/{gcp/orchestrators/vertex_scheduler → databricks/utils}/__init__.py +2 -2
- zenml/integrations/databricks/utils/databricks_utils.py +87 -0
- zenml/integrations/deepchecks/__init__.py +30 -10
- zenml/integrations/deepchecks/data_validators/deepchecks_data_validator.py +59 -18
- zenml/integrations/deepchecks/materializers/deepchecks_dataset_materializer.py +3 -1
- zenml/integrations/deepchecks/materializers/deepchecks_results_materializer.py +3 -3
- zenml/integrations/deepchecks/validation_checks.py +62 -35
- zenml/integrations/discord/__init__.py +1 -0
- zenml/integrations/discord/steps/discord_alerter_ask_step.py +1 -0
- zenml/integrations/discord/steps/discord_alerter_post_step.py +1 -0
- zenml/integrations/evidently/__init__.py +22 -4
- zenml/integrations/evidently/column_mapping.py +11 -3
- zenml/integrations/evidently/data_validators/evidently_data_validator.py +24 -6
- zenml/integrations/evidently/metrics.py +7 -8
- zenml/integrations/evidently/tests.py +7 -8
- zenml/integrations/facets/__init__.py +22 -5
- zenml/integrations/facets/models.py +2 -7
- zenml/integrations/feast/__init__.py +19 -3
- zenml/integrations/feast/feature_stores/feast_feature_store.py +13 -32
- zenml/integrations/gcp/__init__.py +6 -6
- zenml/integrations/gcp/artifact_stores/gcp_artifact_store.py +6 -0
- zenml/integrations/gcp/flavors/gcp_artifact_store_flavor.py +1 -0
- zenml/integrations/gcp/flavors/vertex_orchestrator_flavor.py +24 -2
- zenml/integrations/gcp/flavors/vertex_step_operator_flavor.py +14 -1
- zenml/integrations/gcp/google_credentials_mixin.py +1 -1
- zenml/integrations/gcp/orchestrators/vertex_orchestrator.py +519 -296
- zenml/integrations/gcp/service_connectors/gcp_service_connector.py +788 -113
- zenml/integrations/gcp/step_operators/vertex_step_operator.py +9 -1
- zenml/integrations/github/__init__.py +15 -0
- zenml/integrations/github/code_repositories/github_code_repository.py +3 -2
- zenml/integrations/github/plugins/__init__.py +19 -0
- zenml/integrations/github/plugins/event_sources/__init__.py +0 -0
- zenml/integrations/github/plugins/event_sources/github_webhook_event_source.py +510 -0
- zenml/integrations/github/plugins/github_webhook_event_source_flavor.py +43 -0
- zenml/integrations/gitlab/code_repositories/gitlab_code_repository.py +7 -1
- zenml/integrations/great_expectations/__init__.py +22 -10
- zenml/integrations/great_expectations/data_validators/ge_data_validator.py +61 -57
- zenml/integrations/great_expectations/flavors/great_expectations_data_validator_flavor.py +35 -2
- zenml/integrations/great_expectations/ge_store_backend.py +24 -11
- zenml/integrations/great_expectations/materializers/ge_materializer.py +9 -9
- zenml/integrations/great_expectations/utils.py +5 -5
- zenml/integrations/huggingface/__init__.py +52 -1
- zenml/integrations/huggingface/flavors/__init__.py +26 -0
- zenml/integrations/huggingface/flavors/huggingface_model_deployer_flavor.py +130 -0
- zenml/integrations/huggingface/materializers/__init__.py +3 -0
- zenml/integrations/huggingface/materializers/huggingface_datasets_materializer.py +102 -19
- zenml/integrations/huggingface/materializers/huggingface_pt_model_materializer.py +18 -19
- zenml/integrations/huggingface/materializers/huggingface_t5_materializer.py +104 -0
- zenml/integrations/huggingface/materializers/huggingface_tf_model_materializer.py +18 -19
- zenml/integrations/huggingface/materializers/huggingface_tokenizer_materializer.py +13 -15
- zenml/integrations/huggingface/model_deployers/__init__.py +20 -0
- zenml/integrations/huggingface/model_deployers/huggingface_model_deployer.py +247 -0
- zenml/integrations/huggingface/services/__init__.py +19 -0
- zenml/integrations/huggingface/services/huggingface_deployment.py +292 -0
- zenml/integrations/huggingface/steps/__init__.py +21 -0
- zenml/integrations/huggingface/steps/accelerate_runner.py +166 -0
- zenml/integrations/huggingface/steps/huggingface_deployer.py +110 -0
- zenml/integrations/hyperai/__init__.py +53 -0
- zenml/integrations/hyperai/flavors/__init__.py +20 -0
- zenml/integrations/hyperai/flavors/hyperai_orchestrator_flavor.py +170 -0
- zenml/integrations/hyperai/orchestrators/__init__.py +21 -0
- zenml/integrations/hyperai/orchestrators/hyperai_orchestrator.py +510 -0
- zenml/integrations/hyperai/service_connectors/__init__.py +20 -0
- zenml/integrations/hyperai/service_connectors/hyperai_service_connector.py +375 -0
- zenml/integrations/integration.py +101 -47
- zenml/integrations/kaniko/flavors/kaniko_image_builder_flavor.py +6 -44
- zenml/integrations/kaniko/image_builders/kaniko_image_builder.py +9 -6
- zenml/integrations/kubeflow/__init__.py +4 -1
- zenml/integrations/kubeflow/flavors/kubeflow_orchestrator_flavor.py +67 -82
- zenml/integrations/kubeflow/orchestrators/kubeflow_orchestrator.py +306 -248
- zenml/integrations/kubernetes/__init__.py +6 -3
- zenml/integrations/kubernetes/flavors/__init__.py +8 -0
- zenml/integrations/kubernetes/flavors/kubernetes_orchestrator_flavor.py +43 -2
- zenml/integrations/kubernetes/flavors/kubernetes_step_operator_flavor.py +166 -0
- zenml/integrations/kubernetes/orchestrators/kube_utils.py +67 -12
- zenml/integrations/kubernetes/orchestrators/kubernetes_orchestrator.py +89 -21
- zenml/integrations/kubernetes/orchestrators/kubernetes_orchestrator_entrypoint.py +39 -7
- zenml/integrations/kubernetes/orchestrators/manifest_utils.py +42 -19
- zenml/integrations/kubernetes/pod_settings.py +21 -31
- zenml/integrations/kubernetes/serialization_utils.py +3 -3
- zenml/integrations/kubernetes/service_connectors/kubernetes_service_connector.py +14 -11
- zenml/integrations/kubernetes/step_operators/__init__.py +22 -0
- zenml/integrations/kubernetes/step_operators/kubernetes_step_operator.py +236 -0
- zenml/integrations/label_studio/__init__.py +1 -3
- zenml/integrations/label_studio/annotators/label_studio_annotator.py +46 -14
- zenml/integrations/label_studio/flavors/__init__.py +2 -0
- zenml/integrations/label_studio/flavors/label_studio_annotator_flavor.py +20 -5
- zenml/integrations/langchain/__init__.py +7 -1
- zenml/integrations/langchain/materializers/document_materializer.py +44 -8
- zenml/integrations/langchain/materializers/openai_embedding_materializer.py +28 -2
- zenml/integrations/lightgbm/__init__.py +1 -0
- zenml/integrations/lightgbm/materializers/lightgbm_booster_materializer.py +8 -15
- zenml/integrations/lightgbm/materializers/lightgbm_dataset_materializer.py +11 -16
- zenml/integrations/lightning/__init__.py +48 -0
- zenml/integrations/lightning/flavors/__init__.py +23 -0
- zenml/integrations/lightning/flavors/lightning_orchestrator_flavor.py +168 -0
- zenml/integrations/lightning/orchestrators/__init__.py +23 -0
- zenml/integrations/lightning/orchestrators/lightning_orchestrator.py +617 -0
- zenml/integrations/lightning/orchestrators/lightning_orchestrator_entrypoint.py +303 -0
- zenml/integrations/lightning/orchestrators/lightning_orchestrator_entrypoint_configuration.py +77 -0
- zenml/integrations/lightning/orchestrators/utils.py +67 -0
- zenml/integrations/mlflow/__init__.py +55 -8
- zenml/integrations/mlflow/experiment_trackers/mlflow_experiment_tracker.py +31 -13
- zenml/integrations/mlflow/flavors/mlflow_experiment_tracker_flavor.py +32 -37
- zenml/integrations/mlflow/model_deployers/mlflow_model_deployer.py +42 -233
- zenml/integrations/mlflow/model_registries/mlflow_model_registry.py +42 -49
- zenml/integrations/mlflow/services/mlflow_deployment.py +30 -5
- zenml/integrations/mlflow/steps/mlflow_deployer.py +25 -27
- zenml/integrations/mlflow/steps/mlflow_registry.py +3 -1
- zenml/integrations/modal/__init__.py +46 -0
- zenml/integrations/modal/flavors/__init__.py +26 -0
- zenml/integrations/modal/flavors/modal_step_operator_flavor.py +125 -0
- zenml/integrations/modal/step_operators/__init__.py +22 -0
- zenml/integrations/modal/step_operators/modal_step_operator.py +242 -0
- zenml/integrations/neptune/experiment_trackers/neptune_experiment_tracker.py +7 -5
- zenml/integrations/neptune/experiment_trackers/run_state.py +71 -55
- zenml/integrations/neptune/flavors/neptune_experiment_tracker_flavor.py +1 -1
- zenml/integrations/neural_prophet/__init__.py +6 -1
- zenml/integrations/numpy/__init__.py +32 -0
- zenml/integrations/{kserve/constants.py → numpy/materializers/__init__.py} +5 -4
- zenml/integrations/numpy/materializers/numpy_materializer.py +246 -0
- zenml/integrations/openai/__init__.py +1 -1
- zenml/integrations/openai/hooks/open_ai_failure_hook.py +39 -14
- zenml/{steps/external_artifact.py → integrations/pandas/__init__.py} +17 -11
- zenml/integrations/{kserve/custom_deployer → pandas/materializers}/__init__.py +5 -5
- zenml/integrations/pandas/materializers/pandas_materializer.py +192 -0
- zenml/integrations/pigeon/__init__.py +44 -0
- zenml/integrations/pigeon/annotators/__init__.py +20 -0
- zenml/integrations/pigeon/annotators/pigeon_annotator.py +330 -0
- zenml/integrations/{kserve → pigeon}/flavors/__init__.py +7 -7
- zenml/integrations/pigeon/flavors/pigeon_annotator_flavor.py +104 -0
- zenml/integrations/pillow/materializers/pillow_image_materializer.py +17 -20
- zenml/integrations/polars/__init__.py +1 -0
- zenml/integrations/polars/materializers/dataframe_materializer.py +26 -39
- zenml/integrations/prodigy/__init__.py +48 -0
- zenml/integrations/prodigy/annotators/__init__.py +20 -0
- zenml/integrations/prodigy/annotators/prodigy_annotator.py +275 -0
- zenml/integrations/prodigy/flavors/__init__.py +24 -0
- zenml/integrations/prodigy/flavors/prodigy_annotator_flavor.py +101 -0
- zenml/integrations/pycaret/__init__.py +6 -0
- zenml/integrations/pycaret/materializers/model_materializer.py +7 -22
- zenml/integrations/pytorch/materializers/base_pytorch_materializer.py +8 -2
- zenml/integrations/pytorch/materializers/pytorch_module_materializer.py +4 -1
- zenml/integrations/registry.py +38 -1
- zenml/integrations/s3/__init__.py +2 -3
- zenml/integrations/s3/artifact_stores/s3_artifact_store.py +191 -9
- zenml/integrations/s3/flavors/s3_artifact_store_flavor.py +22 -45
- zenml/integrations/s3/utils.py +39 -0
- zenml/integrations/seldon/__init__.py +18 -2
- zenml/integrations/seldon/model_deployers/seldon_model_deployer.py +19 -141
- zenml/integrations/seldon/secret_schemas/secret_schemas.py +2 -2
- zenml/integrations/seldon/seldon_client.py +55 -70
- zenml/integrations/seldon/services/seldon_deployment.py +4 -5
- zenml/integrations/seldon/steps/seldon_deployer.py +21 -20
- zenml/integrations/sklearn/__init__.py +1 -1
- zenml/integrations/skypilot/flavors/skypilot_orchestrator_base_vm_config.py +29 -7
- zenml/integrations/skypilot/orchestrators/skypilot_base_vm_orchestrator.py +47 -28
- zenml/integrations/skypilot/orchestrators/skypilot_orchestrator_entrypoint.py +2 -2
- zenml/integrations/skypilot_aws/__init__.py +3 -2
- zenml/integrations/skypilot_aws/flavors/skypilot_orchestrator_aws_vm_flavor.py +1 -1
- zenml/integrations/skypilot_azure/__init__.py +2 -4
- zenml/integrations/skypilot_azure/flavors/skypilot_orchestrator_azure_vm_flavor.py +1 -1
- zenml/integrations/skypilot_gcp/__init__.py +3 -2
- zenml/integrations/skypilot_gcp/flavors/skypilot_orchestrator_gcp_vm_flavor.py +1 -1
- zenml/integrations/skypilot_kubernetes/__init__.py +52 -0
- zenml/integrations/skypilot_kubernetes/flavors/__init__.py +26 -0
- zenml/integrations/skypilot_kubernetes/flavors/skypilot_orchestrator_kubernetes_vm_flavor.py +125 -0
- zenml/integrations/skypilot_kubernetes/orchestrators/__init__.py +25 -0
- zenml/integrations/skypilot_kubernetes/orchestrators/skypilot_kubernetes_vm_orchestrator.py +74 -0
- zenml/integrations/skypilot_lambda/__init__.py +50 -0
- zenml/integrations/{kserve/secret_schemas → skypilot_lambda/flavors}/__init__.py +9 -12
- zenml/integrations/skypilot_lambda/flavors/skypilot_orchestrator_lambda_vm_flavor.py +130 -0
- zenml/{lineage_graph/node → integrations/skypilot_lambda/orchestrators}/__init__.py +8 -15
- zenml/integrations/skypilot_lambda/orchestrators/skypilot_lambda_vm_orchestrator.py +92 -0
- zenml/integrations/slack/__init__.py +1 -0
- zenml/integrations/slack/alerters/slack_alerter.py +22 -2
- zenml/integrations/slack/flavors/__init__.py +2 -0
- zenml/integrations/slack/flavors/slack_alerter_flavor.py +13 -4
- zenml/integrations/slack/steps/slack_alerter_ask_step.py +1 -0
- zenml/integrations/slack/steps/slack_alerter_post_step.py +1 -0
- zenml/integrations/spark/flavors/spark_step_operator_flavor.py +2 -39
- zenml/integrations/spark/step_operators/kubernetes_step_operator.py +1 -0
- zenml/integrations/spark/step_operators/spark_entrypoint_configuration.py +1 -0
- zenml/integrations/spark/step_operators/spark_step_operator.py +2 -0
- zenml/integrations/tekton/__init__.py +2 -1
- zenml/integrations/tekton/flavors/tekton_orchestrator_flavor.py +66 -23
- zenml/integrations/tekton/orchestrators/tekton_orchestrator.py +549 -235
- zenml/integrations/tensorboard/__init__.py +1 -13
- zenml/integrations/tensorboard/services/tensorboard_service.py +3 -4
- zenml/integrations/tensorboard/visualizers/tensorboard_visualizer.py +66 -59
- zenml/integrations/tensorflow/__init__.py +10 -15
- zenml/integrations/tensorflow/materializers/keras_materializer.py +24 -27
- zenml/integrations/tensorflow/materializers/tf_dataset_materializer.py +9 -16
- zenml/integrations/vllm/__init__.py +50 -0
- zenml/integrations/vllm/flavors/__init__.py +21 -0
- zenml/integrations/vllm/flavors/vllm_model_deployer_flavor.py +91 -0
- zenml/integrations/vllm/model_deployers/__init__.py +19 -0
- zenml/integrations/vllm/model_deployers/vllm_model_deployer.py +263 -0
- zenml/integrations/vllm/services/__init__.py +19 -0
- zenml/integrations/vllm/services/vllm_deployment.py +206 -0
- zenml/integrations/wandb/__init__.py +1 -0
- zenml/integrations/wandb/flavors/wandb_experiment_tracker_flavor.py +21 -18
- zenml/integrations/whylogs/__init__.py +18 -2
- zenml/integrations/whylogs/data_validators/whylogs_data_validator.py +3 -3
- zenml/integrations/whylogs/flavors/whylogs_data_validator_flavor.py +1 -1
- zenml/integrations/whylogs/materializers/whylogs_materializer.py +14 -21
- zenml/integrations/xgboost/materializers/xgboost_booster_materializer.py +11 -22
- zenml/integrations/xgboost/materializers/xgboost_dmatrix_materializer.py +10 -19
- zenml/io/fileio.py +1 -0
- zenml/io/filesystem.py +2 -2
- zenml/io/local_filesystem.py +3 -3
- zenml/logger.py +41 -17
- zenml/logging/__init__.py +5 -0
- zenml/logging/step_logging.py +280 -32
- zenml/{new/steps → login}/__init__.py +4 -1
- zenml/login/credentials.py +370 -0
- zenml/login/credentials_store.py +638 -0
- zenml/{new/pipelines → login/pro}/__init__.py +4 -1
- zenml/login/pro/client.py +492 -0
- zenml/login/pro/constants.py +28 -0
- zenml/{zen_stores/schemas/identity_schemas.py → login/pro/models.py} +9 -11
- zenml/login/pro/organization/__init__.py +14 -0
- zenml/login/pro/organization/client.py +79 -0
- zenml/{lineage_graph/node/base_node.py → login/pro/organization/models.py} +13 -12
- zenml/{_hub → login/pro/tenant}/__init__.py +2 -2
- zenml/login/pro/tenant/client.py +92 -0
- zenml/login/pro/tenant/models.py +176 -0
- zenml/login/pro/utils.py +107 -0
- zenml/login/server_info.py +52 -0
- zenml/{cli → login}/web_login.py +71 -21
- zenml/materializers/__init__.py +2 -4
- zenml/materializers/base_materializer.py +180 -51
- zenml/materializers/built_in_materializer.py +50 -23
- zenml/materializers/cloudpickle_materializer.py +4 -5
- zenml/materializers/numpy_materializer.py +23 -233
- zenml/materializers/pandas_materializer.py +22 -174
- zenml/materializers/pydantic_materializer.py +2 -2
- zenml/materializers/service_materializer.py +12 -10
- zenml/materializers/structured_string_materializer.py +12 -6
- zenml/materializers/uuid_materializer.py +79 -0
- zenml/metadata/lazy_load.py +33 -17
- zenml/metadata/metadata_types.py +112 -3
- zenml/model/lazy_load.py +85 -4
- zenml/model/model.py +236 -226
- zenml/model/utils.py +99 -141
- zenml/model_deployers/base_model_deployer.py +319 -47
- zenml/model_registries/base_model_registry.py +17 -15
- zenml/models/__init__.py +254 -175
- zenml/models/v2/base/base.py +254 -112
- zenml/models/v2/base/base_plugin_flavor.py +76 -0
- zenml/models/v2/base/filter.py +530 -199
- zenml/models/v2/base/page.py +2 -12
- zenml/models/v2/base/scoped.py +292 -22
- zenml/models/v2/core/action.py +276 -0
- zenml/models/v2/core/action_flavor.py +57 -0
- zenml/models/v2/core/api_key.py +35 -12
- zenml/models/v2/core/artifact.py +126 -6
- zenml/models/v2/core/artifact_version.py +289 -65
- zenml/models/v2/core/artifact_visualization.py +10 -4
- zenml/models/v2/core/code_reference.py +13 -4
- zenml/models/v2/core/code_repository.py +39 -16
- zenml/models/v2/core/component.py +113 -61
- zenml/models/v2/core/device.py +25 -5
- zenml/models/v2/core/event_source.py +244 -0
- zenml/models/v2/core/event_source_flavor.py +67 -0
- zenml/models/v2/core/flavor.py +90 -27
- zenml/models/v2/core/logs.py +81 -12
- zenml/models/v2/core/model.py +91 -42
- zenml/models/v2/core/model_version.py +100 -55
- zenml/models/v2/core/model_version_artifact.py +96 -89
- zenml/models/v2/core/model_version_pipeline_run.py +86 -53
- zenml/models/v2/core/pipeline.py +205 -80
- zenml/models/v2/core/pipeline_build.py +110 -21
- zenml/models/v2/core/pipeline_deployment.py +101 -36
- zenml/models/v2/core/pipeline_run.py +545 -33
- zenml/models/v2/core/run_metadata.py +23 -181
- zenml/models/v2/core/run_template.py +439 -0
- zenml/models/v2/core/schedule.py +66 -29
- zenml/models/v2/core/secret.py +33 -21
- zenml/models/v2/core/server_settings.py +224 -0
- zenml/models/v2/core/service.py +500 -0
- zenml/models/v2/core/service_account.py +40 -15
- zenml/models/v2/core/service_connector.py +247 -49
- zenml/models/v2/core/stack.py +163 -70
- zenml/models/v2/core/step_run.py +210 -48
- zenml/models/v2/core/tag.py +21 -8
- zenml/models/v2/core/tag_resource.py +13 -4
- zenml/models/v2/core/trigger.py +422 -0
- zenml/models/v2/core/trigger_execution.py +119 -0
- zenml/models/v2/core/user.py +136 -69
- zenml/models/v2/core/workspace.py +26 -7
- zenml/models/v2/misc/auth_models.py +11 -2
- zenml/models/v2/misc/build_item.py +3 -3
- zenml/models/v2/misc/external_user.py +3 -6
- zenml/models/v2/misc/info_models.py +78 -0
- zenml/models/v2/misc/loaded_visualization.py +2 -2
- zenml/models/v2/misc/run_metadata.py +38 -0
- zenml/models/v2/misc/server_models.py +100 -0
- zenml/models/v2/misc/service_connector_type.py +9 -17
- zenml/models/v2/misc/stack_deployment.py +96 -0
- zenml/models/v2/misc/user_auth.py +7 -9
- zenml/orchestrators/__init__.py +4 -0
- zenml/orchestrators/base_orchestrator.py +136 -25
- zenml/orchestrators/containerized_orchestrator.py +1 -0
- zenml/orchestrators/dag_runner.py +18 -3
- zenml/orchestrators/input_utils.py +109 -48
- zenml/orchestrators/local/local_orchestrator.py +10 -0
- zenml/orchestrators/local_docker/local_docker_orchestrator.py +14 -42
- zenml/orchestrators/output_utils.py +16 -6
- zenml/orchestrators/publish_utils.py +12 -5
- zenml/orchestrators/step_launcher.py +142 -194
- zenml/orchestrators/step_run_utils.py +386 -0
- zenml/orchestrators/step_runner.py +181 -270
- zenml/orchestrators/utils.py +219 -84
- zenml/orchestrators/wheeled_orchestrator.py +147 -0
- zenml/pipelines/__init__.py +3 -16
- zenml/{new/pipelines → pipelines}/build_utils.py +287 -47
- zenml/{new/pipelines → pipelines}/pipeline_context.py +6 -2
- zenml/pipelines/pipeline_decorator.py +40 -64
- zenml/{new/pipelines/pipeline.py → pipelines/pipeline_definition.py} +376 -440
- zenml/pipelines/run_utils.py +358 -0
- zenml/plugins/__init__.py +0 -0
- zenml/plugins/base_plugin_flavor.py +88 -0
- zenml/plugins/plugin_flavor_registry.py +342 -0
- zenml/secret/base_secret.py +7 -8
- zenml/secret/schemas/basic_auth_secret_schema.py +0 -1
- zenml/service_connectors/docker_service_connector.py +19 -4
- zenml/service_connectors/service_connector.py +12 -14
- zenml/service_connectors/service_connector_registry.py +71 -55
- zenml/service_connectors/service_connector_utils.py +418 -0
- zenml/services/__init__.py +0 -2
- zenml/services/container/container_service.py +9 -6
- zenml/services/container/container_service_endpoint.py +1 -1
- zenml/services/container/entrypoint.py +3 -2
- zenml/services/local/local_daemon_entrypoint.py +9 -6
- zenml/services/local/local_service.py +1 -1
- zenml/services/local/local_service_endpoint.py +1 -1
- zenml/services/service.py +222 -130
- zenml/services/service_status.py +2 -1
- zenml/services/service_type.py +6 -5
- zenml/stack/flavor.py +25 -18
- zenml/stack/flavor_registry.py +4 -4
- zenml/stack/stack.py +20 -131
- zenml/stack/stack_component.py +136 -110
- zenml/stack/utils.py +36 -15
- zenml/stack_deployments/__init__.py +14 -0
- zenml/stack_deployments/aws_stack_deployment.py +320 -0
- zenml/stack_deployments/azure_stack_deployment.py +315 -0
- zenml/stack_deployments/gcp_stack_deployment.py +315 -0
- zenml/stack_deployments/stack_deployment.py +232 -0
- zenml/stack_deployments/utils.py +48 -0
- zenml/step_operators/step_operator_entrypoint_configuration.py +2 -1
- zenml/steps/__init__.py +3 -9
- zenml/steps/base_step.py +172 -315
- zenml/{new/steps → steps}/decorated_step.py +1 -0
- zenml/steps/entrypoint_function_utils.py +33 -93
- zenml/{new/steps → steps}/step_context.py +70 -50
- zenml/steps/step_decorator.py +47 -93
- zenml/steps/step_invocation.py +22 -60
- zenml/steps/utils.py +161 -48
- zenml/types.py +14 -1
- zenml/utils/archivable.py +178 -0
- zenml/utils/callback_registry.py +71 -0
- zenml/utils/code_repository_utils.py +1 -0
- zenml/utils/code_utils.py +346 -0
- zenml/utils/cuda_utils.py +50 -0
- zenml/utils/dashboard_utils.py +67 -21
- zenml/utils/deprecation_utils.py +22 -24
- zenml/utils/dict_utils.py +22 -0
- zenml/utils/docker_utils.py +34 -5
- zenml/utils/downloaded_repository_context.py +1 -0
- zenml/utils/env_utils.py +55 -1
- zenml/utils/filesync_model.py +65 -28
- zenml/utils/function_utils.py +260 -0
- zenml/utils/integration_utils.py +1 -0
- zenml/utils/json_utils.py +131 -0
- zenml/utils/materializer_utils.py +1 -1
- zenml/utils/metadata_utils.py +368 -0
- zenml/utils/notebook_utils.py +136 -0
- zenml/utils/package_utils.py +89 -0
- zenml/utils/pagination_utils.py +9 -7
- zenml/utils/pipeline_docker_image_builder.py +152 -149
- zenml/utils/pydantic_utils.py +276 -66
- zenml/utils/requirements_utils.py +71 -0
- zenml/utils/secret_utils.py +66 -12
- zenml/utils/settings_utils.py +2 -1
- zenml/utils/singleton.py +15 -3
- zenml/utils/source_code_utils.py +1 -0
- zenml/utils/source_utils.py +236 -14
- zenml/utils/string_utils.py +140 -0
- zenml/utils/typed_model.py +5 -3
- zenml/utils/typing_utils.py +223 -0
- zenml/utils/visualization_utils.py +5 -3
- zenml/utils/yaml_utils.py +1 -1
- zenml/zen_server/auth.py +387 -55
- zenml/zen_server/cache.py +208 -0
- zenml/zen_server/cloud_utils.py +253 -0
- zenml/zen_server/csrf.py +91 -0
- zenml/zen_server/dashboard/assets/404-Dfq64Boz.js +1 -0
- zenml/zen_server/dashboard/assets/@radix-DeK6qiuw.js +85 -0
- zenml/zen_server/dashboard/assets/@react-router-B3Z5rLr2.js +29 -0
- zenml/zen_server/dashboard/assets/@reactflow-BUNIMFeC.js +17 -0
- zenml/zen_server/dashboard/assets/@reactflow-C26Olbza.css +1 -0
- zenml/zen_server/dashboard/assets/@tanstack-DT5WLu9C.js +22 -0
- zenml/zen_server/dashboard/assets/AlertDialogDropdownItem-B73Vs10T.js +1 -0
- zenml/zen_server/dashboard/assets/CodeSnippet-Bbx6fIb6.css +1 -0
- zenml/zen_server/dashboard/assets/CodeSnippet-DIJRT2NT.js +9 -0
- zenml/zen_server/dashboard/assets/CollapsibleCard-BzUHGZOU.js +1 -0
- zenml/zen_server/dashboard/assets/Commands-BEGyld4c.js +1 -0
- zenml/zen_server/dashboard/assets/ComponentBadge-xyKiek1s.js +1 -0
- zenml/zen_server/dashboard/assets/CopyButton-DhW-mapu.js +2 -0
- zenml/zen_server/dashboard/assets/CsvVizualization-D8oazBiE.js +15 -0
- zenml/zen_server/dashboard/assets/DeleteAlertDialog-WkSIIgfy.js +1 -0
- zenml/zen_server/dashboard/assets/DialogItem-Bgroeg29.js +1 -0
- zenml/zen_server/dashboard/assets/DisplayDate-CDMUcQHS.js +1 -0
- zenml/zen_server/dashboard/assets/EmptyState-BzdlCwp3.js +1 -0
- zenml/zen_server/dashboard/assets/Error-CY5tlu17.js +1 -0
- zenml/zen_server/dashboard/assets/ExecutionStatus-G8mjIaeA.js +1 -0
- zenml/zen_server/dashboard/assets/Helpbox-Bb1ed--O.js +1 -0
- zenml/zen_server/dashboard/assets/Infobox-Da6-76M2.js +1 -0
- zenml/zen_server/dashboard/assets/InlineAvatar-DqnZaBNq.js +1 -0
- zenml/zen_server/dashboard/assets/Lock-CYYy18Mm.js +1 -0
- zenml/zen_server/dashboard/assets/MarkdownVisualization-ylXaAxev.js +14 -0
- zenml/zen_server/dashboard/assets/NestedCollapsible-aK5ojKoF.js +1 -0
- zenml/zen_server/dashboard/assets/NumberBox-Dtp3J6g5.js +1 -0
- zenml/zen_server/dashboard/assets/Partials-CqZp5NMX.js +1 -0
- zenml/zen_server/dashboard/assets/PasswordChecker-B0nadgh6.js +1 -0
- zenml/zen_server/dashboard/assets/ProBadge-B4tRUYve.js +1 -0
- zenml/zen_server/dashboard/assets/ProCta-CZuP29Qz.js +1 -0
- zenml/zen_server/dashboard/assets/ProviderIcon-Bd7GUQ1_.js +1 -0
- zenml/zen_server/dashboard/assets/ProviderRadio-mstdqzsS.js +1 -0
- zenml/zen_server/dashboard/assets/RunSelector-CsruSB4i.js +1 -0
- zenml/zen_server/dashboard/assets/RunsBody-DxxtWVYz.js +1 -0
- zenml/zen_server/dashboard/assets/SearchField-D6tPxyqw.js +1 -0
- zenml/zen_server/dashboard/assets/SecretTooltip-CLzJIYW_.js +1 -0
- zenml/zen_server/dashboard/assets/SetPassword-Yn50ooBC.js +1 -0
- zenml/zen_server/dashboard/assets/StackList-U537qoYd.js +1 -0
- zenml/zen_server/dashboard/assets/Tabs-CNv-eTYM.js +1 -0
- zenml/zen_server/dashboard/assets/Tick-jEIevzVf.js +1 -0
- zenml/zen_server/dashboard/assets/UpdatePasswordSchemas-C16GW-kX.js +1 -0
- zenml/zen_server/dashboard/assets/UsageReason-Bf2tzhv1.js +1 -0
- zenml/zen_server/dashboard/assets/WizardFooter-D6i-AP1K.js +1 -0
- zenml/zen_server/dashboard/assets/acp-DOsXjFc7.webp +0 -0
- zenml/zen_server/dashboard/assets/adam-e-y0WnB_.webp +0 -0
- zenml/zen_server/dashboard/assets/alex-DcCuDHPg.webp +0 -0
- zenml/zen_server/dashboard/assets/all-pipeline-runs-query-DUti43aF.js +1 -0
- zenml/zen_server/dashboard/assets/baris-C0ZrZ10g.webp +0 -0
- zenml/zen_server/dashboard/assets/check-DloQpStc.js +1 -0
- zenml/zen_server/dashboard/assets/check-circle-jNbX5-sR.js +1 -0
- zenml/zen_server/dashboard/assets/chevron-down-6JyMkfjR.js +1 -0
- zenml/zen_server/dashboard/assets/chevron-right-double-D7ojK9Co.js +1 -0
- zenml/zen_server/dashboard/assets/cloud-squares-DeRLMopf.svg +43 -0
- zenml/zen_server/dashboard/assets/code-browser-CUFUIHfp.js +1 -0
- zenml/zen_server/dashboard/assets/code-snippets-CqONne41.js +13 -0
- zenml/zen_server/dashboard/assets/components-Br2ezRib.js +1 -0
- zenml/zen_server/dashboard/assets/connectors-video-C9qY4syJ.svg +21 -0
- zenml/zen_server/dashboard/assets/copy-C8XQA2Ug.js +1 -0
- zenml/zen_server/dashboard/assets/create-stack-Ch2WPs9U.js +1 -0
- zenml/zen_server/dashboard/assets/dates-3pMLCNrD.js +1 -0
- zenml/zen_server/dashboard/assets/delete-run-Byf9hTjA.js +1 -0
- zenml/zen_server/dashboard/assets/docker-BdA9vrnW.js +1 -0
- zenml/zen_server/dashboard/assets/dots-horizontal-otGBOSDJ.js +1 -0
- zenml/zen_server/dashboard/assets/flyte-Cj-xy_8I.svg +10 -0
- zenml/zen_server/dashboard/assets/form-schemas-BZqKBPBF.js +1 -0
- zenml/zen_server/dashboard/assets/gcp-CFtm4BA7.js +1 -0
- zenml/zen_server/dashboard/assets/hamza-NKKOZz1I.webp +0 -0
- zenml/zen_server/dashboard/assets/help-Cc9bBIJH.js +1 -0
- zenml/zen_server/dashboard/assets/index-CE0aQlv8.js +55 -0
- zenml/zen_server/dashboard/assets/index-CtdYkjUi.js +1 -0
- zenml/zen_server/dashboard/assets/index-CyBKZcpO.js +1 -0
- zenml/zen_server/dashboard/assets/index-DXvT1_Um.css +1 -0
- zenml/zen_server/dashboard/assets/index-Uu49AX48.js +1 -0
- zenml/zen_server/dashboard/assets/index-v6gQjDEo.js +1 -0
- zenml/zen_server/dashboard/assets/index.esm-Dy6Z9Ung.js +1 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-400-normal-BLGc9T1a.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-400-normal-ZzOtrSSW.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-500-normal-D4Vwzodn.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-500-normal-DH2hs3aW.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-600-normal-BGBWG807.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-600-normal-BuzJQFbW.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-ext-400-normal-BPnxn4xp.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-ext-400-normal-Dc4VJyIJ.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-ext-500-normal-BShVwWPj.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-ext-500-normal-CUiC4oBV.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-ext-600-normal-Bt9VVOA-.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-cyrillic-ext-600-normal-CaqZN2hq.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-400-normal-BZzXV7-1.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-400-normal-DxZsaF_h.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-500-normal-CeQXL5ds.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-500-normal-d_eO-yCQ.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-600-normal-CwicyhtI.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-600-normal-Dhlb-90d.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-ext-400-normal-Bput3-QP.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-ext-400-normal-DCpCPQOf.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-ext-500-normal-B6guLgqG.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-ext-500-normal-M2hEX8vc.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-ext-600-normal-C9WLioJ8.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-greek-ext-600-normal-Cnui8OiR.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-400-normal-BOOGhInR.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-400-normal-gitzw0hO.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-500-normal-D2bGa7uu.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-500-normal-deR1Tlfd.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-600-normal-B5cFAncS.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-600-normal-D273HNI0.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-ext-400-normal-C1t-h-pH.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-ext-400-normal-hnt3BR84.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-ext-500-normal-CIS2RHJS.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-ext-500-normal-UMdmhHu2.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-ext-600-normal-BnYJhD27.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-latin-ext-600-normal-CAF0vJDd.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-vietnamese-400-normal-BUNmGMP1.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-vietnamese-400-normal-DMkecbls.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-vietnamese-500-normal-DOriooB6.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-vietnamese-500-normal-DQPw2Hwd.woff +0 -0
- zenml/zen_server/dashboard/assets/inter-vietnamese-600-normal-Cc8MFFhd.woff2 +0 -0
- zenml/zen_server/dashboard/assets/inter-vietnamese-600-normal-Cm6aH8_k.woff +0 -0
- zenml/zen_server/dashboard/assets/key-icon-aH-QIa5R.js +1 -0
- zenml/zen_server/dashboard/assets/kubernetes-B2wmAJ1d.js +1 -0
- zenml/zen_server/dashboard/assets/layout-BtHBmE4w.js +1 -0
- zenml/zen_server/dashboard/assets/link-external-b9AXw_sW.js +1 -0
- zenml/zen_server/dashboard/assets/login-command-CkqxPtV3.js +1 -0
- zenml/zen_server/dashboard/assets/login-mutation-DNDVp_2H.js +1 -0
- zenml/zen_server/dashboard/assets/logs-WMSM52RF.js +1 -0
- zenml/zen_server/dashboard/assets/mcp-Cb1aMeoq.webp +0 -0
- zenml/zen_server/dashboard/assets/metaflow-weOkWNyT.svg +10 -0
- zenml/zen_server/dashboard/assets/not-found-Bmup4ctE.js +1 -0
- zenml/zen_server/dashboard/assets/package-C6uypY4h.js +1 -0
- zenml/zen_server/dashboard/assets/page--XLMzHrn.js +1 -0
- zenml/zen_server/dashboard/assets/page-ANYGfEUL.js +1 -0
- zenml/zen_server/dashboard/assets/page-B5Sr8pib.js +1 -0
- zenml/zen_server/dashboard/assets/page-BC27C_OI.js +2 -0
- zenml/zen_server/dashboard/assets/page-BNxYrN0q.js +1 -0
- zenml/zen_server/dashboard/assets/page-BYJfqgLN.js +1 -0
- zenml/zen_server/dashboard/assets/page-B_0XkV48.js +1 -0
- zenml/zen_server/dashboard/assets/page-BrmJp1Wt.js +1 -0
- zenml/zen_server/dashboard/assets/page-C2nU3Gxn.js +1 -0
- zenml/zen_server/dashboard/assets/page-C70wZtV2.js +1 -0
- zenml/zen_server/dashboard/assets/page-CHRn1fQm.js +1 -0
- zenml/zen_server/dashboard/assets/page-CWr96ZKN.js +1 -0
- zenml/zen_server/dashboard/assets/page-CXAbSyp9.js +1 -0
- zenml/zen_server/dashboard/assets/page-CaeI9ptC.js +1 -0
- zenml/zen_server/dashboard/assets/page-Cc8ZEuj4.js +1 -0
- zenml/zen_server/dashboard/assets/page-CltCNL0T.js +1 -0
- zenml/zen_server/dashboard/assets/page-CmlYj7Nl.js +1 -0
- zenml/zen_server/dashboard/assets/page-D6Ev5P8V.js +1 -0
- zenml/zen_server/dashboard/assets/page-D9Oh05fl.js +1 -0
- zenml/zen_server/dashboard/assets/page-DGlm1RVc.js +1 -0
- zenml/zen_server/dashboard/assets/page-DN4BVIOL.js +1 -0
- zenml/zen_server/dashboard/assets/page-Dif8CWyZ.js +1 -0
- zenml/zen_server/dashboard/assets/page-DlIi5ThM.js +1 -0
- zenml/zen_server/dashboard/assets/page-DoW7YxTu.js +1 -0
- zenml/zen_server/dashboard/assets/page-Dth9X1Ih.js +1 -0
- zenml/zen_server/dashboard/assets/page-DweqqCkF.js +1 -0
- zenml/zen_server/dashboard/assets/page-DyOJ_pq3.js +1 -0
- zenml/zen_server/dashboard/assets/page-Hn8q9iJZ.js +1 -0
- zenml/zen_server/dashboard/assets/page-IhckKFnD.js +6 -0
- zenml/zen_server/dashboard/assets/page-LyZ_l8vR.js +1 -0
- zenml/zen_server/dashboard/assets/page-PamGpk0j.js +1 -0
- zenml/zen_server/dashboard/assets/page-PxOWfKgF.js +2 -0
- zenml/zen_server/dashboard/assets/persist-DeXRG61d.js +1 -0
- zenml/zen_server/dashboard/assets/persist-vP0-Xl4f.js +1 -0
- zenml/zen_server/dashboard/assets/plus-tf1V2hTJ.js +1 -0
- zenml/zen_server/dashboard/assets/refresh-BjOeWlEq.js +1 -0
- zenml/zen_server/dashboard/assets/repos-video-D8kpu60k.svg +9 -0
- zenml/zen_server/dashboard/assets/rocket-DjT2cDvG.js +1 -0
- zenml/zen_server/dashboard/assets/service-DH_oUqQj.js +2 -0
- zenml/zen_server/dashboard/assets/settings_preview-0JLrRgHP.webp +0 -0
- zenml/zen_server/dashboard/assets/sharedSchema-Bw1_Wa7l.js +14 -0
- zenml/zen_server/dashboard/assets/stack-detail-query-B_0R_fd6.js +1 -0
- zenml/zen_server/dashboard/assets/stefan-B08Ftbba.webp +0 -0
- zenml/zen_server/dashboard/assets/templates-1S_8WeSK.webp +0 -0
- zenml/zen_server/dashboard/assets/tick-circle-BEX_Tp4v.js +1 -0
- zenml/zen_server/dashboard/assets/tour-cover-BYfeen6M.webp +0 -0
- zenml/zen_server/dashboard/assets/trash-arLUMWMS.js +1 -0
- zenml/zen_server/dashboard/assets/update-server-settings-mutation-D9qYhfaN.js +1 -0
- zenml/zen_server/dashboard/assets/upgrade-form-CwRHBuXB.webp +0 -0
- zenml/zen_server/dashboard/assets/url-Dh93fvh0.js +1 -0
- zenml/zen_server/dashboard/assets/zod-BwEbpOxH.js +1 -0
- zenml/zen_server/dashboard/index.html +19 -1
- zenml/zen_server/deploy/__init__.py +7 -16
- zenml/zen_server/deploy/base_provider.py +49 -78
- zenml/zen_server/deploy/{local → daemon}/__init__.py +3 -3
- zenml/zen_server/deploy/{local/local_provider.py → daemon/daemon_provider.py} +48 -66
- zenml/zen_server/deploy/{local/local_zen_server.py → daemon/daemon_zen_server.py} +78 -62
- zenml/zen_server/deploy/deployer.py +94 -175
- zenml/zen_server/deploy/deployment.py +23 -17
- zenml/zen_server/deploy/docker/docker_provider.py +15 -31
- zenml/zen_server/deploy/docker/docker_zen_server.py +30 -35
- zenml/zen_server/deploy/helm/Chart.yaml +1 -1
- zenml/zen_server/deploy/helm/README.md +3 -13
- zenml/zen_server/deploy/helm/templates/NOTES.txt +23 -7
- zenml/zen_server/deploy/helm/templates/_environment.tpl +175 -23
- zenml/zen_server/deploy/helm/templates/server-db-job.yaml +45 -18
- zenml/zen_server/deploy/helm/templates/server-db-pvc.yaml +25 -0
- zenml/zen_server/deploy/helm/templates/server-deployment.yaml +22 -6
- zenml/zen_server/deploy/helm/templates/server-secret.yaml +11 -10
- zenml/zen_server/deploy/helm/values.yaml +210 -28
- zenml/zen_server/exceptions.py +20 -1
- zenml/zen_server/feature_gate/__init__.py +13 -0
- zenml/zen_server/feature_gate/endpoint_utils.py +61 -0
- zenml/zen_server/feature_gate/feature_gate_interface.py +49 -0
- zenml/zen_server/feature_gate/zenml_cloud_feature_gate.py +125 -0
- zenml/zen_server/jwt.py +64 -32
- zenml/zen_server/rate_limit.py +200 -0
- zenml/zen_server/rbac/endpoint_utils.py +92 -9
- zenml/zen_server/rbac/models.py +21 -17
- zenml/zen_server/rbac/rbac_sql_zen_store.py +175 -0
- zenml/zen_server/rbac/utils.py +71 -30
- zenml/zen_server/rbac/zenml_cloud_rbac.py +13 -188
- zenml/zen_server/routers/actions_endpoints.py +324 -0
- zenml/zen_server/routers/artifact_version_endpoints.py +28 -2
- zenml/zen_server/routers/auth_endpoints.py +249 -131
- zenml/zen_server/routers/code_repositories_endpoints.py +1 -0
- zenml/zen_server/routers/devices_endpoints.py +56 -49
- zenml/zen_server/routers/event_source_endpoints.py +327 -0
- zenml/zen_server/routers/logs_endpoints.py +66 -0
- zenml/zen_server/routers/model_versions_endpoints.py +59 -0
- zenml/zen_server/routers/models_endpoints.py +7 -1
- zenml/zen_server/routers/pipeline_builds_endpoints.py +6 -1
- zenml/zen_server/routers/pipeline_deployments_endpoints.py +35 -0
- zenml/zen_server/routers/pipelines_endpoints.py +19 -32
- zenml/zen_server/routers/plugin_endpoints.py +107 -0
- zenml/zen_server/routers/run_templates_endpoints.py +212 -0
- zenml/zen_server/routers/runs_endpoints.py +91 -32
- zenml/zen_server/routers/schedule_endpoints.py +1 -0
- zenml/zen_server/routers/secrets_endpoints.py +4 -2
- zenml/zen_server/routers/server_endpoints.py +186 -4
- zenml/zen_server/routers/service_connectors_endpoints.py +56 -0
- zenml/zen_server/routers/service_endpoints.py +180 -0
- zenml/zen_server/routers/stack_components_endpoints.py +2 -1
- zenml/zen_server/routers/stack_deployment_endpoints.py +164 -0
- zenml/zen_server/routers/steps_endpoints.py +21 -12
- zenml/zen_server/routers/triggers_endpoints.py +336 -0
- zenml/zen_server/routers/users_endpoints.py +280 -45
- zenml/zen_server/routers/webhook_endpoints.py +127 -0
- zenml/zen_server/routers/workspaces_endpoints.py +220 -185
- zenml/zen_server/secure_headers.py +120 -0
- zenml/{new → zen_server/template_execution}/__init__.py +1 -1
- zenml/zen_server/template_execution/runner_entrypoint_configuration.py +42 -0
- zenml/zen_server/template_execution/utils.py +474 -0
- zenml/zen_server/template_execution/workload_manager_interface.py +92 -0
- zenml/zen_server/utils.py +374 -74
- zenml/zen_server/zen_server_api.py +299 -52
- zenml/zen_stores/base_zen_store.py +90 -58
- zenml/zen_stores/migrations/alembic.py +22 -9
- zenml/zen_stores/migrations/env.py +2 -2
- zenml/zen_stores/migrations/utils.py +731 -0
- zenml/zen_stores/migrations/versions/0.21.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.21.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.22.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.23.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.30.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.31.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.31.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.32.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.32.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.33.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.34.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.35.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.35.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.36.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.36.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.37.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.38.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.39.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.39.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.40.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.40.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.40.2_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.40.3_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.41.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.42.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.42.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.43.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.44.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.44.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.44.2_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.44.3_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.45.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.45.1_release_0_45_1.py +0 -1
- zenml/zen_stores/migrations/versions/0.45.2_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.45.3_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.45.4_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.45.5_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.45.6_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.46.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.46.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.47.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.50.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.51.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.52.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.53.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.53.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.54.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.54.1_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.55.0_release.py +0 -1
- zenml/zen_stores/migrations/versions/0.55.1_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.55.2_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.55.3_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.55.4_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.55.5_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.56.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.56.1_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.56.2_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.56.3_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.56.4_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.57.0.rc1_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.57.0.rc2_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.57.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.57.1_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.58.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.58.1_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.58.2_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.60.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.61.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.62.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.63.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.64.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.65.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.66.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.67.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.68.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.68.1_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.70.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.71.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/0.72.0_release.py +23 -0
- zenml/zen_stores/migrations/versions/026d4577b6a0_add_code_path.py +39 -0
- zenml/zen_stores/migrations/versions/03742aa7fdd7_add_secrets.py +1 -0
- zenml/zen_stores/migrations/versions/0701da9951a0_added_service_table.py +94 -0
- zenml/zen_stores/migrations/versions/0b06faa59c93_add_service_connectors.py +1 -0
- zenml/zen_stores/migrations/versions/0d707865f404_adding_labels_to_stacks.py +30 -0
- zenml/zen_stores/migrations/versions/0e4735b23577_increase_pipeline_spec_field_length.py +1 -0
- zenml/zen_stores/migrations/versions/1041bc644e0d_remove_secrets_manager.py +6 -3
- zenml/zen_stores/migrations/versions/10a907dad202_delete_mlmd_tables.py +2 -1
- zenml/zen_stores/migrations/versions/14d687c8fa1c_rename_model_config_to_model_version.py +1 -0
- zenml/zen_stores/migrations/versions/19f27d5b234e_add_build_and_deployment_tables.py +1 -0
- zenml/zen_stores/migrations/versions/1a9a9d2a836d_admin_users.py +56 -0
- zenml/zen_stores/migrations/versions/1ac1b9c04da1_make_secrets_values_optional.py +1 -0
- zenml/zen_stores/migrations/versions/1cb6477f72d6_move_artifact_save_type.py +99 -0
- zenml/zen_stores/migrations/versions/1d74e596abb8_add_run_once_start_time_to_schedule.py +36 -0
- zenml/zen_stores/migrations/versions/1d8f30c54477_migrate_to_new_.py +124 -0
- zenml/zen_stores/migrations/versions/248dfd320b68_update_size_of_flavor_config_schema.py +1 -0
- zenml/zen_stores/migrations/versions/25155145c545_separate_actions_and_triggers.py +228 -0
- zenml/zen_stores/migrations/versions/26351d482b9e_add_step_run_unique_constraint.py +37 -0
- zenml/zen_stores/migrations/versions/26b776ad583e_redesign_artifacts.py +9 -10
- zenml/zen_stores/migrations/versions/2d201872e23c_remove_db_dependency_loop.py +29 -0
- zenml/zen_stores/migrations/versions/37835ce041d2_optimizing_database.py +4 -3
- zenml/zen_stores/migrations/versions/389046140cad_data_versioning.py +1 -0
- zenml/zen_stores/migrations/versions/3944116bbd56_rename_project_to_workspace.py +1 -0
- zenml/zen_stores/migrations/versions/3b68abe58f44_add_model_watchtower_entities.py +1 -0
- zenml/zen_stores/migrations/versions/3c5a367730c2_add_environment_info_to_runs.py +1 -0
- zenml/zen_stores/migrations/versions/3dcc5d20e82f_add_last_user_activity.py +51 -0
- zenml/zen_stores/migrations/versions/43a86093b60e_add_labels_for_stack_components.py +1 -0
- zenml/zen_stores/migrations/versions/46506f72f0ed_add_server_settings.py +123 -0
- zenml/zen_stores/migrations/versions/479103df60b6_add_triggers.py +162 -0
- zenml/zen_stores/migrations/versions/4a3087070f4e_add_step_source_code.py +1 -0
- zenml/zen_stores/migrations/versions/4c41c0ca42db_add_code_repository_table.py +1 -0
- zenml/zen_stores/migrations/versions/4d688d8f7aff_rename_model_version_to_model.py +1 -0
- zenml/zen_stores/migrations/versions/4e1972485075_endpoint_artifact_deployment_artifact.py +1 -0
- zenml/zen_stores/migrations/versions/4f66af55fbb9_rename_model_config_model_to_model_.py +1 -0
- zenml/zen_stores/migrations/versions/5330ba58bf20_rename_tables_and_foreign_keys.py +8 -9
- zenml/zen_stores/migrations/versions/5994f9ad0489_introduce_role_permissions.py +4 -2
- zenml/zen_stores/migrations/versions/5cc3f41cf048_add_save_models_to_registry.py +1 -0
- zenml/zen_stores/migrations/versions/6119cd9b93c2_tags_table.py +1 -0
- zenml/zen_stores/migrations/versions/623a234c11f5_add_sdk_docs_url_to_flavors.py +1 -0
- zenml/zen_stores/migrations/versions/6917bce75069_add_pipeline_run_unique_constraint.py +5 -4
- zenml/zen_stores/migrations/versions/6a28c4fd0ef2_add_caching_info.py +1 -0
- zenml/zen_stores/migrations/versions/6f707b385dc1_fix_model_artifacts.py +1 -0
- zenml/zen_stores/migrations/versions/722392c91006_make_is_service_account_mandatory.py +1 -0
- zenml/zen_stores/migrations/versions/72675226b2de_unique_users.py +31 -0
- zenml/zen_stores/migrations/versions/72722dee4686_track_server_version.py +1 -0
- zenml/zen_stores/migrations/versions/7280c14811d6_use_text_type.py +1 -0
- zenml/zen_stores/migrations/versions/728c6369cfaa_add_name_column_to_input_artifact_pk.py +4 -2
- zenml/zen_stores/migrations/versions/729263e47b55_fix_external_input_artifacts.py +1 -0
- zenml/zen_stores/migrations/versions/743ec82b1b3c_update_size_of_build_images.py +3 -2
- zenml/zen_stores/migrations/versions/7500f434b71c_remove_shared_columns.py +4 -2
- zenml/zen_stores/migrations/versions/76a7b9451ccd_add_build_template_deployment_id.py +52 -0
- zenml/zen_stores/migrations/versions/7834208cc3f6_artifact_project_scoping.py +9 -7
- zenml/zen_stores/migrations/versions/7b651bf6822e_track_secrets_in_db.py +7 -7
- zenml/zen_stores/migrations/versions/7d1919bb1ef0_add_run_templates.py +100 -0
- zenml/zen_stores/migrations/versions/7e4a481d17f7_add_identity_table.py +3 -2
- zenml/zen_stores/migrations/versions/7f603e583dd7_fixed_migration.py +2 -1
- zenml/zen_stores/migrations/versions/86fa52918b54_remove_teams_and_roles.py +1 -0
- zenml/zen_stores/migrations/versions/8a64fbfecda0_add_num_outputs_to_run_step.py +1 -0
- zenml/zen_stores/migrations/versions/8ed03137cacc_polymorthic_run_metadata.py +1 -0
- zenml/zen_stores/migrations/versions/904464ea4041_add_pipeline_model_run_unique_constraints.py +192 -0
- zenml/zen_stores/migrations/versions/909550c7c4da_remove_user_hub_token.py +36 -0
- zenml/zen_stores/migrations/versions/93cbda80a732_add_service_accounts.py +1 -0
- zenml/zen_stores/migrations/versions/979eff8fc4b1_add_code_repo_description_and_logo_url.py +1 -0
- zenml/zen_stores/migrations/versions/9971237fa937_artifact_visualizations.py +1 -0
- zenml/zen_stores/migrations/versions/9d8020441014_increase_step_configuration_length.py +1 -0
- zenml/zen_stores/migrations/versions/a1237ba94fd8_add_model_version_producer_run_unique_.py +68 -0
- zenml/zen_stores/migrations/versions/a39c4184c8ce_remove_secrets_manager_flavors.py +3 -2
- zenml/zen_stores/migrations/versions/a91762e6be36_artifact_version_table.py +5 -4
- zenml/zen_stores/migrations/versions/ade72effebaf_added_logs_table.py +1 -0
- zenml/zen_stores/migrations/versions/alembic_start.py +2 -1
- zenml/zen_stores/migrations/versions/b4eccf34dfa3_add_hub_token_to_user_model.py +1 -0
- zenml/zen_stores/migrations/versions/b4fca5241eea_migrate_onboarding_state.py +167 -0
- zenml/zen_stores/migrations/versions/b557b2871693_update_step_run_input_types.py +33 -0
- zenml/zen_stores/migrations/versions/b59aa68fdb1f_simplify_pipelines.py +139 -0
- zenml/zen_stores/migrations/versions/b73bc71f1106_remove_component_spec_path.py +36 -0
- zenml/zen_stores/migrations/versions/bf2120261b5a_add_configured_model_version_id.py +74 -0
- zenml/zen_stores/migrations/versions/c1b18cec3a48_increase_length_on_flavor_config_schema.py +1 -0
- zenml/zen_stores/migrations/versions/c22561cbb3a9_add_artifact_unique_constraints.py +86 -0
- zenml/zen_stores/migrations/versions/cc269488e5a9_separate_run_metadata.py +135 -0
- zenml/zen_stores/migrations/versions/cc9894cb58aa_add_user_metadata.py +41 -0
- zenml/zen_stores/migrations/versions/ccd68b7825ae_add_status_to_pipeline_and_step_run.py +1 -0
- zenml/zen_stores/migrations/versions/d02b3d3464cf_add_orchestrator_run_id_column.py +1 -0
- zenml/zen_stores/migrations/versions/d26471b6fe8f_update_build_filtering.py +1 -0
- zenml/zen_stores/migrations/versions/d7b3acf9aa46_create_schedule_table.py +1 -0
- zenml/zen_stores/migrations/versions/e1d66d91a099_add_stack_and_component_spec_paths_to_.py +1 -0
- zenml/zen_stores/migrations/versions/e5225281b4d3_add_connector_skew_tolerance.py +1 -0
- zenml/zen_stores/migrations/versions/e65aa6708ff7_pipeline_versioning.py +1 -0
- zenml/zen_stores/migrations/versions/ec0d785ca296_create_run_metadata_table.py +1 -0
- zenml/zen_stores/migrations/versions/ec6307720f92_simplify_model_version_links.py +119 -0
- zenml/zen_stores/migrations/versions/f3b3964e3a0f_add_oauth_devices.py +1 -0
- zenml/zen_stores/migrations/versions/f49904a80aa7_increase_length_of_artifact_table_sources.py +1 -0
- zenml/zen_stores/migrations/versions/fbd7f18ced1e_increase_step_run_field_lengths.py +5 -4
- zenml/zen_stores/rest_zen_store.py +1326 -305
- zenml/zen_stores/schemas/__init__.py +22 -3
- zenml/zen_stores/schemas/action_schemas.py +192 -0
- zenml/zen_stores/schemas/api_key_schemas.py +23 -10
- zenml/zen_stores/schemas/artifact_schemas.py +112 -49
- zenml/zen_stores/schemas/artifact_visualization_schemas.py +17 -8
- zenml/zen_stores/schemas/base_schemas.py +27 -0
- zenml/zen_stores/schemas/code_repository_schemas.py +25 -10
- zenml/zen_stores/schemas/component_schemas.py +74 -11
- zenml/zen_stores/schemas/constants.py +16 -0
- zenml/zen_stores/schemas/device_schemas.py +29 -15
- zenml/zen_stores/schemas/event_source_schemas.py +188 -0
- zenml/zen_stores/schemas/flavor_schemas.py +19 -9
- zenml/zen_stores/schemas/logs_schemas.py +12 -6
- zenml/zen_stores/schemas/model_schemas.py +192 -139
- zenml/zen_stores/schemas/pipeline_build_schemas.py +16 -16
- zenml/zen_stores/schemas/pipeline_deployment_schemas.py +58 -17
- zenml/zen_stores/schemas/pipeline_run_schemas.py +170 -35
- zenml/zen_stores/schemas/pipeline_schemas.py +51 -33
- zenml/zen_stores/schemas/run_metadata_schemas.py +28 -78
- zenml/zen_stores/schemas/run_template_schemas.py +267 -0
- zenml/zen_stores/schemas/schedule_schema.py +15 -5
- zenml/zen_stores/schemas/secret_schemas.py +18 -10
- zenml/zen_stores/schemas/server_settings_schemas.py +129 -0
- zenml/zen_stores/schemas/service_connector_schemas.py +13 -6
- zenml/zen_stores/schemas/service_schemas.py +258 -0
- zenml/zen_stores/schemas/stack_schemas.py +23 -6
- zenml/zen_stores/schemas/step_run_schemas.py +132 -41
- zenml/zen_stores/schemas/tag_schemas.py +31 -50
- zenml/zen_stores/schemas/trigger_schemas.py +316 -0
- zenml/zen_stores/schemas/user_schemas.py +66 -23
- zenml/zen_stores/schemas/utils.py +112 -0
- zenml/zen_stores/schemas/workspace_schemas.py +36 -19
- zenml/zen_stores/secrets_stores/aws_secrets_store.py +41 -32
- zenml/zen_stores/secrets_stores/azure_secrets_store.py +20 -23
- zenml/zen_stores/secrets_stores/base_secrets_store.py +80 -12
- zenml/zen_stores/secrets_stores/gcp_secrets_store.py +42 -33
- zenml/zen_stores/secrets_stores/hashicorp_secrets_store.py +7 -11
- zenml/zen_stores/secrets_stores/secrets_store_interface.py +1 -0
- zenml/zen_stores/secrets_stores/service_connector_secrets_store.py +15 -8
- zenml/zen_stores/secrets_stores/sql_secrets_store.py +8 -9
- zenml/zen_stores/sql_zen_store.py +4062 -799
- zenml/zen_stores/template_utils.py +263 -0
- zenml/zen_stores/zen_store_interface.py +614 -44
- zenml_nightly-0.72.0.dev20250116.dist-info/METADATA +486 -0
- zenml_nightly-0.72.0.dev20250116.dist-info/RECORD +1294 -0
- {zenml_nightly-0.55.0.dev20240124.dist-info → zenml_nightly-0.72.0.dev20250116.dist-info}/WHEEL +1 -1
- CLA.md +0 -110
- CODE-OF-CONDUCT.md +0 -132
- CONTRIBUTING.md +0 -260
- README.md +0 -304
- RELEASE_NOTES.md +0 -3919
- ROADMAP.md +0 -5
- SECURITY.md +0 -15
- zenml/_hub/client.py +0 -285
- zenml/_hub/constants.py +0 -21
- zenml/_hub/utils.py +0 -80
- zenml/api.py +0 -61
- zenml/cli/hub.py +0 -1115
- zenml/cli/stack_recipes.py +0 -469
- zenml/integrations/gcp/google_cloud_function.py +0 -187
- zenml/integrations/gcp/google_cloud_scheduler.py +0 -83
- zenml/integrations/gcp/orchestrators/vertex_scheduler/main.py +0 -91
- zenml/integrations/gcp/orchestrators/vertex_scheduler/requirements.txt +0 -2
- zenml/integrations/kserve/__init__.py +0 -57
- zenml/integrations/kserve/custom_deployer/zenml_custom_model.py +0 -175
- zenml/integrations/kserve/flavors/kserve_model_deployer_flavor.py +0 -137
- zenml/integrations/kserve/model_deployers/kserve_model_deployer.py +0 -1003
- zenml/integrations/kserve/secret_schemas/secret_schemas.py +0 -65
- zenml/integrations/kserve/services/kserve_deployment.py +0 -596
- zenml/integrations/kserve/steps/__init__.py +0 -22
- zenml/integrations/kserve/steps/kserve_deployer.py +0 -472
- zenml/integrations/kserve/steps/kserve_step_utils.py +0 -293
- zenml/integrations/kubeflow/utils.py +0 -96
- zenml/lineage_graph/lineage_graph.py +0 -244
- zenml/lineage_graph/node/artifact_node.py +0 -52
- zenml/lineage_graph/node/step_node.py +0 -41
- zenml/models/v2/base/internal.py +0 -37
- zenml/models/v2/base/update.py +0 -40
- zenml/models/v2/misc/hub_plugin_models.py +0 -79
- zenml/new/pipelines/deserialization_utils.py +0 -291
- zenml/new/pipelines/model_utils.py +0 -72
- zenml/new/pipelines/pipeline_decorator.py +0 -109
- zenml/new/steps/step_decorator.py +0 -160
- zenml/pipelines/base_pipeline.py +0 -274
- zenml/post_execution/pipeline.py +0 -58
- zenml/post_execution/pipeline_run.py +0 -55
- zenml/services/service_registry.py +0 -214
- zenml/services/terraform/__init__.py +0 -14
- zenml/services/terraform/terraform_service.py +0 -441
- zenml/steps/step_environment.py +0 -108
- zenml/steps/step_output.py +0 -36
- zenml/utils/mlstacks_utils.py +0 -635
- zenml/utils/terraform_utils.py +0 -42
- zenml/zen_server/dashboard/_redirects +0 -1
- zenml/zen_server/dashboard/asset-manifest.json +0 -131
- zenml/zen_server/dashboard/manifest.json +0 -25
- zenml/zen_server/dashboard/precache-manifest.c139638dcc4d9d3425353266447a2fad.js +0 -462
- zenml/zen_server/dashboard/robots.txt +0 -2
- zenml/zen_server/dashboard/service-worker.js +0 -39
- zenml/zen_server/dashboard/static/css/2.5b37d44a.chunk.css +0 -16
- zenml/zen_server/dashboard/static/css/2.5b37d44a.chunk.css.map +0 -1
- zenml/zen_server/dashboard/static/css/main.77e46c35.chunk.css +0 -2
- zenml/zen_server/dashboard/static/css/main.77e46c35.chunk.css.map +0 -1
- zenml/zen_server/dashboard/static/js/2.bb4cef22.chunk.js +0 -3
- zenml/zen_server/dashboard/static/js/2.bb4cef22.chunk.js.LICENSE.txt +0 -95
- zenml/zen_server/dashboard/static/js/2.bb4cef22.chunk.js.map +0 -1
- zenml/zen_server/dashboard/static/js/main.270838b4.chunk.js +0 -2
- zenml/zen_server/dashboard/static/js/main.270838b4.chunk.js.map +0 -1
- zenml/zen_server/dashboard/static/js/runtime-main.bfca2edd.js +0 -2
- zenml/zen_server/dashboard/static/js/runtime-main.bfca2edd.js.map +0 -1
- zenml/zen_server/dashboard/static/media/AlertTriangle.28aee535.svg +0 -5
- zenml/zen_server/dashboard/static/media/ArrowSquareOut.abfb9bc7.svg +0 -5
- zenml/zen_server/dashboard/static/media/Back.86c23a22.svg +0 -4
- zenml/zen_server/dashboard/static/media/BookOpen.5cb101ff.svg +0 -4
- zenml/zen_server/dashboard/static/media/BoundingBox.1eb98717.svg +0 -10
- zenml/zen_server/dashboard/static/media/Burger.9b1c67d7.svg +0 -3
- zenml/zen_server/dashboard/static/media/Cached.2381fb8d.svg +0 -1
- zenml/zen_server/dashboard/static/media/Calendar.356e11c7.svg +0 -3
- zenml/zen_server/dashboard/static/media/ChartBarHorizontal.0247447b.svg +0 -6
- zenml/zen_server/dashboard/static/media/ChartLine.0d79e18d.svg +0 -4
- zenml/zen_server/dashboard/static/media/ChatDots.2e1c9211.svg +0 -6
- zenml/zen_server/dashboard/static/media/Check.dad6beb2.svg +0 -3
- zenml/zen_server/dashboard/static/media/CheckCircleFilled.c19566d0.svg +0 -3
- zenml/zen_server/dashboard/static/media/Checkbox.af50e31e.svg +0 -3
- zenml/zen_server/dashboard/static/media/ChevronDown.f860ce32.svg +0 -3
- zenml/zen_server/dashboard/static/media/ChevronDownLight.6642d756.svg +0 -3
- zenml/zen_server/dashboard/static/media/ChevronLeft.f6edfcdb.svg +0 -3
- zenml/zen_server/dashboard/static/media/CircleCheck.f98fd6ca.svg +0 -1
- zenml/zen_server/dashboard/static/media/Clock.ffc9de95.svg +0 -3
- zenml/zen_server/dashboard/static/media/Close.74e9efbc.svg +0 -5
- zenml/zen_server/dashboard/static/media/CloseWithBorder.6960930a.svg +0 -3
- zenml/zen_server/dashboard/static/media/CloseWithoutBorder.cd6f71df.svg +0 -3
- zenml/zen_server/dashboard/static/media/CloudArrowUp.0aecb235.svg +0 -6
- zenml/zen_server/dashboard/static/media/Code.ef0f33b5.svg +0 -3
- zenml/zen_server/dashboard/static/media/Config.0be63f8a.svg +0 -1
- zenml/zen_server/dashboard/static/media/Connector.9fd46ef1.svg +0 -10
- zenml/zen_server/dashboard/static/media/Copy.36e2112a.svg +0 -1
- zenml/zen_server/dashboard/static/media/Dashboard.d05787e0.svg +0 -3
- zenml/zen_server/dashboard/static/media/Data.b1c3b5f8.svg +0 -3
- zenml/zen_server/dashboard/static/media/Delete.3c361b28.svg +0 -8
- zenml/zen_server/dashboard/static/media/Docs.7541d478.svg +0 -7
- zenml/zen_server/dashboard/static/media/Download.fba04d87.svg +0 -5
- zenml/zen_server/dashboard/static/media/Edit.490eb294.svg +0 -6
- zenml/zen_server/dashboard/static/media/EmptyRightArrow.23749d01.svg +0 -3
- zenml/zen_server/dashboard/static/media/Example.6396cd37.svg +0 -5
- zenml/zen_server/dashboard/static/media/Extension.1394cd4a.svg +0 -3
- zenml/zen_server/dashboard/static/media/Eye.d9e4ee62.svg +0 -4
- zenml/zen_server/dashboard/static/media/Failed.0213c1a0.svg +0 -1
- zenml/zen_server/dashboard/static/media/FileText.1f15bacd.svg +0 -7
- zenml/zen_server/dashboard/static/media/Filter.ab6b9c0d.svg +0 -3
- zenml/zen_server/dashboard/static/media/Folders.12b29887.svg +0 -5
- zenml/zen_server/dashboard/static/media/FunnelFill.6df4c143.svg +0 -3
- zenml/zen_server/dashboard/static/media/GitCommit.7dd9c2aa.svg +0 -5
- zenml/zen_server/dashboard/static/media/GitHub_Logo.cefc2023.png +0 -0
- zenml/zen_server/dashboard/static/media/Graph.2c63a892.svg +0 -11
- zenml/zen_server/dashboard/static/media/History.08329240.svg +0 -3
- zenml/zen_server/dashboard/static/media/Home.0843b0d5.svg +0 -3
- zenml/zen_server/dashboard/static/media/ImageBuilder.ea762d9c.svg +0 -6
- zenml/zen_server/dashboard/static/media/InProgress.304a0edc.svg +0 -1
- zenml/zen_server/dashboard/static/media/Info.9fe10c5c.svg +0 -3
- zenml/zen_server/dashboard/static/media/KeyboardReturn.491afbe3.svg +0 -3
- zenml/zen_server/dashboard/static/media/Link.72bbb55d.svg +0 -4
- zenml/zen_server/dashboard/static/media/Lock.30f5e1fe.svg +0 -5
- zenml/zen_server/dashboard/static/media/Lock2.a769ea52.svg +0 -3
- zenml/zen_server/dashboard/static/media/LockKey.92f21621.svg +0 -6
- zenml/zen_server/dashboard/static/media/Logs.8bf4d005.svg +0 -5
- zenml/zen_server/dashboard/static/media/MinusCircle.4188f418.svg +0 -4
- zenml/zen_server/dashboard/static/media/ModelRegistry.f0de050a.svg +0 -6
- zenml/zen_server/dashboard/static/media/MultiUser.a2ba7c67.svg +0 -10
- zenml/zen_server/dashboard/static/media/PaginationFirst.92628634.svg +0 -4
- zenml/zen_server/dashboard/static/media/PaginationLast.00d3c732.svg +0 -4
- zenml/zen_server/dashboard/static/media/PaginationNext.86158845.svg +0 -3
- zenml/zen_server/dashboard/static/media/PaginationPrev.60c18a88.svg +0 -3
- zenml/zen_server/dashboard/static/media/Pen.f2d831d4.svg +0 -6
- zenml/zen_server/dashboard/static/media/PhotoCamera.179d6d4c.svg +0 -3
- zenml/zen_server/dashboard/static/media/Pipeline.30d298b0.svg +0 -7
- zenml/zen_server/dashboard/static/media/Plus.5aa1c16b.svg +0 -3
- zenml/zen_server/dashboard/static/media/PlusCircle.92d860dd.svg +0 -5
- zenml/zen_server/dashboard/static/media/Repositories.71a36b8c.svg +0 -3
- zenml/zen_server/dashboard/static/media/RightArrow.f30d3871.svg +0 -29
- zenml/zen_server/dashboard/static/media/Rocket.63bf7b9d.svg +0 -3
- zenml/zen_server/dashboard/static/media/RocketLaunch.1bff2b59.svg +0 -6
- zenml/zen_server/dashboard/static/media/Rubik-Medium.c87313aa.ttf +0 -0
- zenml/zen_server/dashboard/static/media/Rubik-Regular.b3d0902b.ttf +0 -0
- zenml/zen_server/dashboard/static/media/Run.daec4fb2.svg +0 -6
- zenml/zen_server/dashboard/static/media/Search.d1afcce5.svg +0 -4
- zenml/zen_server/dashboard/static/media/Settings.59ca73ae.svg +0 -4
- zenml/zen_server/dashboard/static/media/Share2.46c3ff66.svg +0 -3
- zenml/zen_server/dashboard/static/media/SignOut.6aa718c5.svg +0 -3
- zenml/zen_server/dashboard/static/media/SimplePlus.5cf7ec20.svg +0 -3
- zenml/zen_server/dashboard/static/media/SingleUser.bef3a095.svg +0 -4
- zenml/zen_server/dashboard/static/media/SourceCodePro-Regular.b484b32f.ttf +0 -0
- zenml/zen_server/dashboard/static/media/Stack.19b604ac.svg +0 -5
- zenml/zen_server/dashboard/static/media/StackComponent.b1ba90b5.svg +0 -4
- zenml/zen_server/dashboard/static/media/Star.f0c25022.svg +0 -9
- zenml/zen_server/dashboard/static/media/StarOutline.94ca8cd9.svg +0 -3
- zenml/zen_server/dashboard/static/media/Storefront.4b4796fe.svg +0 -3
- zenml/zen_server/dashboard/static/media/Stream.543e3039.svg +0 -3
- zenml/zen_server/dashboard/static/media/SupportAgent.510ddf1f.svg +0 -8
- zenml/zen_server/dashboard/static/media/Table.77033750.svg +0 -6
- zenml/zen_server/dashboard/static/media/Tool.d5785486.svg +0 -3
- zenml/zen_server/dashboard/static/media/UserPlus.741a99d7.svg +0 -6
- zenml/zen_server/dashboard/static/media/Verified.0625b2a0.svg +0 -3
- zenml/zen_server/dashboard/static/media/addNew.4fb6c939.svg +0 -8
- zenml/zen_server/dashboard/static/media/arrowClose.cbd53f3f.svg +0 -3
- zenml/zen_server/dashboard/static/media/arrowOpen.6ceef0af.svg +0 -3
- zenml/zen_server/dashboard/static/media/check_small.30bc0138.svg +0 -3
- zenml/zen_server/dashboard/static/media/circleArrowSideClose.98d6013e.svg +0 -18
- zenml/zen_server/dashboard/static/media/circleArrowSideOpen.63653df6.svg +0 -18
- zenml/zen_server/dashboard/static/media/image.104fd14b.png +0 -0
- zenml/zen_server/dashboard/static/media/imageAddIcon.e83004a9.svg +0 -7
- zenml/zen_server/dashboard/static/media/logo.93333e5c.svg +0 -1
- zenml/zen_server/dashboard/static/media/logo_small.4204397d.svg +0 -3
- zenml/zen_server/dashboard/static/media/logo_white.d4b4414e.svg +0 -20
- zenml/zen_server/dashboard/static/media/notConnected.5e2c8ea7.svg +0 -8
- zenml/zen_server/dashboard/static/media/plugin-fallback.72c294e6.svg +0 -6
- zenml/zen_server/dashboard/static/media/share.bcd998b0.svg +0 -5
- zenml/zen_server/dashboard/static/media/stars.08a9b19a.svg +0 -8
- zenml/zen_server/deploy/terraform/__init__.py +0 -41
- zenml/zen_server/deploy/terraform/providers/__init__.py +0 -14
- zenml/zen_server/deploy/terraform/providers/aws_provider.py +0 -61
- zenml/zen_server/deploy/terraform/providers/azure_provider.py +0 -59
- zenml/zen_server/deploy/terraform/providers/gcp_provider.py +0 -59
- zenml/zen_server/deploy/terraform/providers/terraform_provider.py +0 -332
- zenml/zen_server/deploy/terraform/recipes/aws/.gitignore +0 -8
- zenml/zen_server/deploy/terraform/recipes/aws/helm.tf +0 -20
- zenml/zen_server/deploy/terraform/recipes/aws/ingress.tf +0 -30
- zenml/zen_server/deploy/terraform/recipes/aws/outputs.tf +0 -14
- zenml/zen_server/deploy/terraform/recipes/aws/printf.cmd +0 -2
- zenml/zen_server/deploy/terraform/recipes/aws/sql.tf +0 -62
- zenml/zen_server/deploy/terraform/recipes/aws/terraform.tf +0 -44
- zenml/zen_server/deploy/terraform/recipes/aws/variables.tf +0 -179
- zenml/zen_server/deploy/terraform/recipes/aws/vpc.tf +0 -47
- zenml/zen_server/deploy/terraform/recipes/aws/zen_server.tf +0 -111
- zenml/zen_server/deploy/terraform/recipes/azure/.gitignore +0 -8
- zenml/zen_server/deploy/terraform/recipes/azure/helm.tf +0 -20
- zenml/zen_server/deploy/terraform/recipes/azure/ingress.tf +0 -30
- zenml/zen_server/deploy/terraform/recipes/azure/key_vault.tf +0 -73
- zenml/zen_server/deploy/terraform/recipes/azure/outputs.tf +0 -14
- zenml/zen_server/deploy/terraform/recipes/azure/printf.cmd +0 -2
- zenml/zen_server/deploy/terraform/recipes/azure/rg.tf +0 -36
- zenml/zen_server/deploy/terraform/recipes/azure/sql.tf +0 -65
- zenml/zen_server/deploy/terraform/recipes/azure/terraform.tf +0 -52
- zenml/zen_server/deploy/terraform/recipes/azure/variables.tf +0 -188
- zenml/zen_server/deploy/terraform/recipes/azure/zen_server.tf +0 -111
- zenml/zen_server/deploy/terraform/recipes/gcp/.gitignore +0 -8
- zenml/zen_server/deploy/terraform/recipes/gcp/helm.tf +0 -20
- zenml/zen_server/deploy/terraform/recipes/gcp/ingress.tf +0 -30
- zenml/zen_server/deploy/terraform/recipes/gcp/outputs.tf +0 -14
- zenml/zen_server/deploy/terraform/recipes/gcp/printf.cmd +0 -2
- zenml/zen_server/deploy/terraform/recipes/gcp/sql.tf +0 -64
- zenml/zen_server/deploy/terraform/recipes/gcp/terraform.tf +0 -44
- zenml/zen_server/deploy/terraform/recipes/gcp/variables.tf +0 -183
- zenml/zen_server/deploy/terraform/recipes/gcp/zen_server.tf +0 -122
- zenml/zen_server/deploy/terraform/terraform_zen_server.py +0 -255
- zenml/zen_server/routers/run_metadata_endpoints.py +0 -97
- zenml_nightly-0.55.0.dev20240124.dist-info/METADATA +0 -438
- zenml_nightly-0.55.0.dev20240124.dist-info/RECORD +0 -1072
- {zenml_nightly-0.55.0.dev20240124.dist-info → zenml_nightly-0.72.0.dev20250116.dist-info}/LICENSE +0 -0
- {zenml_nightly-0.55.0.dev20240124.dist-info → zenml_nightly-0.72.0.dev20250116.dist-info}/entry_points.txt +0 -0
RELEASE_NOTES.md
DELETED
@@ -1,3919 +0,0 @@
|
|
1
|
-
<!-- markdown-link-check-disable -->
|
2
|
-
|
3
|
-
# 0.55.0
|
4
|
-
|
5
|
-
This release comes with a range of new features, bug fixes and documentation updates. The most notable changes are the ability to do lazy loading of Artifact Version, Artifact Version Metadata, and Model Version Metadata inside the pipeline code, and the ability to link Artifacts to Model Versions implicitly via the `save_artifact` function.
|
6
|
-
|
7
|
-
Additionally, we've updated the documentation to include a new starter guide on how to manage artifacts, and a new production guide that walks you through how to configure your pipelines to run in production.
|
8
|
-
|
9
|
-
## What's Changed
|
10
|
-
* Remove --name from service account creation in docs by @christianversloot in https://github.com/zenml-io/zenml/pull/2295
|
11
|
-
* Secrets store hot backup and restore by @stefannica in https://github.com/zenml-io/zenml/pull/2277
|
12
|
-
* Updating the README of the e2e template by @bcdurak in https://github.com/zenml-io/zenml/pull/2299
|
13
|
-
* Add missing docstring for Skypilot setting by @schustmi in https://github.com/zenml-io/zenml/pull/2305
|
14
|
-
* Update Manage artifacts starter guide docs by @JonathanLoscalzo in https://github.com/zenml-io/zenml/pull/2301
|
15
|
-
* Add some tiny details and moved around a page by @htahir1 in https://github.com/zenml-io/zenml/pull/2297
|
16
|
-
* Model links lazy evaluation in pipeline code by @avishniakov in https://github.com/zenml-io/zenml/pull/2205
|
17
|
-
* Link artifact to MCP entity via function call or implicitly in `save_artifact` by @avishniakov in https://github.com/zenml-io/zenml/pull/2298
|
18
|
-
* Extend MCP/ACP listing capabilities by @avishniakov in https://github.com/zenml-io/zenml/pull/2285
|
19
|
-
* Add latest `zenml` version to migration testing scripts by @strickvl in https://github.com/zenml-io/zenml/pull/2294
|
20
|
-
* Remove Python 3.7 check for Langchain Integration by @strickvl in https://github.com/zenml-io/zenml/pull/2308
|
21
|
-
* Allow spellcheck to run for docs changes by @strickvl in https://github.com/zenml-io/zenml/pull/2307
|
22
|
-
* Add helper message for `zenml up --blocking` login by @strickvl in https://github.com/zenml-io/zenml/pull/2290
|
23
|
-
* Fix secret migration from external store in helm deployment by @stefannica in https://github.com/zenml-io/zenml/pull/2315
|
24
|
-
* Small docs fixes by @htahir1 in https://github.com/zenml-io/zenml/pull/2314
|
25
|
-
* Rename model version to a model by @avishniakov in https://github.com/zenml-io/zenml/pull/2267
|
26
|
-
* Updating the docs after the Skypilot tests by @bcdurak in https://github.com/zenml-io/zenml/pull/2311
|
27
|
-
* Remove unused Segment / Mixpanel generation workflow and script by @strickvl in https://github.com/zenml-io/zenml/pull/2319
|
28
|
-
* Add `log_step_metadata` utility function by @strickvl in https://github.com/zenml-io/zenml/pull/2322
|
29
|
-
* Add conditional checks to prevent scheduled actions running inside forked repositories by @strickvl in https://github.com/zenml-io/zenml/pull/2317
|
30
|
-
* RBAC resource sharing by @schustmi in https://github.com/zenml-io/zenml/pull/2320
|
31
|
-
* Fix typo in migration downgrade by @avishniakov in https://github.com/zenml-io/zenml/pull/2337
|
32
|
-
* Separate `skypilot` flavors into different folders by @safoinme in https://github.com/zenml-io/zenml/pull/2332
|
33
|
-
* Add warning for GCP integration when using Python >=3.11 by @strickvl in https://github.com/zenml-io/zenml/pull/2333
|
34
|
-
|
35
|
-
## New Contributors
|
36
|
-
* @JonathanLoscalzo made their first contribution in https://github.com/zenml-io/zenml/pull/2301
|
37
|
-
|
38
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.54.1...0.55.0
|
39
|
-
|
40
|
-
# 0.54.1
|
41
|
-
|
42
|
-
Release 0.54.1, includes a mix of updates and new additions and bug fixes. The most notable changes are the new production guide,
|
43
|
-
allowing multi step VMs for the Skypilot orchestrator which allows you to configure a step to run on a specific VM or run the entire pipeline on a single VM,
|
44
|
-
and some improvements to the Model Control Plane.
|
45
|
-
|
46
|
-
## What's Changed
|
47
|
-
* Bump aquasecurity/trivy-action from 0.16.0 to 0.16.1 by @dependabot in https://github.com/zenml-io/zenml/pull/2244
|
48
|
-
* Bump crate-ci/typos from 1.16.26 to 1.17.0 by @dependabot in https://github.com/zenml-io/zenml/pull/2245
|
49
|
-
* Add YAML formatting standardisation to formatting & linting scripts by @strickvl in https://github.com/zenml-io/zenml/pull/2224
|
50
|
-
* Remove text annotation by @strickvl in https://github.com/zenml-io/zenml/pull/2246
|
51
|
-
* Add MariaDB migration testing by @strickvl in https://github.com/zenml-io/zenml/pull/2170
|
52
|
-
* Delete artifact links from model version via Client, ModelVersion and API by @avishniakov in https://github.com/zenml-io/zenml/pull/2191
|
53
|
-
* Default/Non-Default step params produce conflict with yaml ones as defaults are set in code by @avishniakov in https://github.com/zenml-io/zenml/pull/2247
|
54
|
-
* Prune of unused artifacts links via client by @avishniakov in https://github.com/zenml-io/zenml/pull/2192
|
55
|
-
* Rename nlp example by @safoinme in https://github.com/zenml-io/zenml/pull/2221
|
56
|
-
* Support refreshing service connector credentials in the Vertex step operator to support long-running jobs by @stefannica in https://github.com/zenml-io/zenml/pull/2198
|
57
|
-
* Refactor secrets stores to store all secret metadata in the DB by @stefannica in https://github.com/zenml-io/zenml/pull/2193
|
58
|
-
* Add `latest_version_id` to the `ModelResponse` by @avishniakov in https://github.com/zenml-io/zenml/pull/2266
|
59
|
-
* Remove `link_artifact` from docs for MCP by @strickvl in https://github.com/zenml-io/zenml/pull/2272
|
60
|
-
* Improve action by adding advice to KeyError when configured steps are not present in pipeline by @christianversloot in https://github.com/zenml-io/zenml/pull/2265
|
61
|
-
* Allow multi step configuration for skypilot by @safoinme in https://github.com/zenml-io/zenml/pull/2166
|
62
|
-
* Reworking the examples by @bcdurak in https://github.com/zenml-io/zenml/pull/2259
|
63
|
-
* A docs update for incorrect import in docs/book/user-guide/starter-guide/track-ml-models.md by @yo-harsh in https://github.com/zenml-io/zenml/pull/2279
|
64
|
-
* Allow `sklearn` versions > 1.3 by @Vishal-Padia in https://github.com/zenml-io/zenml/pull/2271
|
65
|
-
* Free `sklearn` dependency to allow all versions by @strickvl in https://github.com/zenml-io/zenml/pull/2281
|
66
|
-
* Misc CI bugfixes by @strickvl in https://github.com/zenml-io/zenml/pull/2260
|
67
|
-
* Fix `yamlfix` script to use `--no-yamlfix` flag by @strickvl in https://github.com/zenml-io/zenml/pull/2280
|
68
|
-
* Fix dependabot settings autoformatting by `yamlfix` by @strickvl in https://github.com/zenml-io/zenml/pull/2282
|
69
|
-
* Add advice for next step to error on AuthorizationException by @christianversloot in https://github.com/zenml-io/zenml/pull/2264
|
70
|
-
* Allow skypilot to configure step or run full pipeline in one VM by @safoinme in https://github.com/zenml-io/zenml/pull/2276
|
71
|
-
* A docs update with production guide + restructured advanced guide by @htahir1 in https://github.com/zenml-io/zenml/pull/2232
|
72
|
-
|
73
|
-
## New Contributors
|
74
|
-
* @yo-harsh made their first contribution in https://github.com/zenml-io/zenml/pull/2279
|
75
|
-
* @Vishal-Padia made their first contribution in https://github.com/zenml-io/zenml/pull/2271
|
76
|
-
|
77
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.54.0...0.54.1
|
78
|
-
|
79
|
-
|
80
|
-
# 0.54.0
|
81
|
-
|
82
|
-
This release brings a range of new features, bug fixes and documentation
|
83
|
-
updates. The Model Control Plane has received a number of small bugfixes and
|
84
|
-
improvements, notably the ability to change model and model version names.
|
85
|
-
|
86
|
-
We've also added a whole new starter guide that walks you through
|
87
|
-
how to get started with ZenML, from creating your first pipeline to fetching
|
88
|
-
objects once your pipelines have run and much more. Be sure to [check it out](https://docs.zenml.io/user-guide/starter-guide) if
|
89
|
-
you're new to ZenML!
|
90
|
-
|
91
|
-
Speaking of documentation improvements, the Model Control Plane now has [its own
|
92
|
-
dedicated documentation section](https://docs.zenml.io/user-guide/advanced-guide/data-management/model-management) introducing the concepts and features of the
|
93
|
-
Model Control Plane.
|
94
|
-
|
95
|
-
As always, this release comes with number of bug fixes, docs additions and
|
96
|
-
smaller improvements to our internal processes.
|
97
|
-
|
98
|
-
## Breaking Change
|
99
|
-
|
100
|
-
This PR introduces breaking changes in the areas of the REST API concerning secrets and tags. As a consequence, the ZenML Client running the previous ZenML version is no longer compatible with a ZenML Server running the new version and vice-versa. To address this, simply ensure that all your ZenML clients use the same version as the server(s) they connect to.
|
101
|
-
|
102
|
-
## 🥳 Community Contributions 🥳
|
103
|
-
|
104
|
-
We'd like to give a special thanks to @christianversloot for two PRs he
|
105
|
-
contributed to this release. One of them [fixes a bug](https://github.com/zenml-io/zenml/pull/2195) that prevented ZenML from
|
106
|
-
running on Windows and the other one [adds a new materializer for the Polars library](https://github.com/zenml-io/zenml/pull/2229).
|
107
|
-
|
108
|
-
Also many thanks to @sean-hickey-wf for his contribution of [an improvement to
|
109
|
-
the Slack Alerter stack component](https://github.com/zenml-io/zenml/pull/2153)
|
110
|
-
which allows you to define custom blocks for the Slack message.
|
111
|
-
|
112
|
-
## What's Changed
|
113
|
-
* Completing the hydration story with the remaining models by @bcdurak in https://github.com/zenml-io/zenml/pull/2151
|
114
|
-
* Remove secrets manager flavors from DB by @stefannica in https://github.com/zenml-io/zenml/pull/2182
|
115
|
-
* Prepare 0.53.1 release by @stefannica in https://github.com/zenml-io/zenml/pull/2183
|
116
|
-
* Update package name for nightly build by @strickvl in https://github.com/zenml-io/zenml/pull/2172
|
117
|
-
* Remove space saver action + upgrade other actions by @strickvl in https://github.com/zenml-io/zenml/pull/2174
|
118
|
-
* mutable names in Model and MV by @avishniakov in https://github.com/zenml-io/zenml/pull/2185
|
119
|
-
* Fix image building for nightly container builds by @strickvl in https://github.com/zenml-io/zenml/pull/2189
|
120
|
-
* Test that artifacts not get linked to model version not from context by @avishniakov in https://github.com/zenml-io/zenml/pull/2188
|
121
|
-
* Warn if Model(Version) config fluctuates from DB state by @avishniakov in https://github.com/zenml-io/zenml/pull/2144
|
122
|
-
* Add blocks field to SlackAlerterParameters for custom slack blocks by @sean-hickey-wf in https://github.com/zenml-io/zenml/pull/2153
|
123
|
-
* Model control plane technical documentation by @strickvl in https://github.com/zenml-io/zenml/pull/2111
|
124
|
-
* Alembic branching issue fix by @avishniakov in https://github.com/zenml-io/zenml/pull/2197
|
125
|
-
* Bump github/codeql-action from 2 to 3 by @dependabot in https://github.com/zenml-io/zenml/pull/2201
|
126
|
-
* Bump google-github-actions/get-gke-credentials from 0 to 2 by @dependabot in https://github.com/zenml-io/zenml/pull/2202
|
127
|
-
* Bump google-github-actions/auth from 1 to 2 by @dependabot in https://github.com/zenml-io/zenml/pull/2203
|
128
|
-
* Bump aws-actions/amazon-ecr-login from 1 to 2 by @dependabot in https://github.com/zenml-io/zenml/pull/2200
|
129
|
-
* Bump crate-ci/typos from 1.16.25 to 1.16.26 by @dependabot in https://github.com/zenml-io/zenml/pull/2207
|
130
|
-
* Fix unreliable test behaviour when using hypothesis by @strickvl in https://github.com/zenml-io/zenml/pull/2208
|
131
|
-
* Added more pod spec properties for k8s orchestrator by @htahir1 in https://github.com/zenml-io/zenml/pull/2097
|
132
|
-
* Fix API docs environment setup by @strickvl in https://github.com/zenml-io/zenml/pull/2190
|
133
|
-
* Use placeholder runs to show pipeline runs in the dashboard without delay by @schustmi in https://github.com/zenml-io/zenml/pull/2048
|
134
|
-
* Update README and CONTRIBUTING.md docs with links to good first issues for contribution by @strickvl in https://github.com/zenml-io/zenml/pull/2220
|
135
|
-
* Bump supported `mlstacks` version to 0.8.0 by @strickvl in https://github.com/zenml-io/zenml/pull/2196
|
136
|
-
* Misc cleanup by @schustmi in https://github.com/zenml-io/zenml/pull/2126
|
137
|
-
* Refactor pipeline run updates by @schustmi in https://github.com/zenml-io/zenml/pull/2117
|
138
|
-
* Rename log_model_version_metadata to log_model_metadata by @htahir1 in https://github.com/zenml-io/zenml/pull/2215
|
139
|
-
* Update starter and create new production guide by @htahir1 in https://github.com/zenml-io/zenml/pull/2143
|
140
|
-
* Fix typo by @strickvl in https://github.com/zenml-io/zenml/pull/2223
|
141
|
-
* Consolidate Custom Filter Logic by @fa9r in https://github.com/zenml-io/zenml/pull/2116
|
142
|
-
* Force forward slashes when saving artifacts by @christianversloot in https://github.com/zenml-io/zenml/pull/2195
|
143
|
-
* Temporarily disable two MLflow tests for MacOS with Python 3.9 and 3.10 by @strickvl in https://github.com/zenml-io/zenml/pull/2186
|
144
|
-
* Disable template updates for forked repositories by @strickvl in https://github.com/zenml-io/zenml/pull/2222
|
145
|
-
* Remove Label Studio text annotation example by @strickvl in https://github.com/zenml-io/zenml/pull/2225
|
146
|
-
* Add scarf checker script and CI workflow by @strickvl in https://github.com/zenml-io/zenml/pull/2227
|
147
|
-
* Add `mlstacks` installation instructions to docs by @strickvl in https://github.com/zenml-io/zenml/pull/2228
|
148
|
-
* Adding the `hydrate` flag to the client methods by @bcdurak in https://github.com/zenml-io/zenml/pull/2120
|
149
|
-
* Fixing the remaining docs pages for `run_metadata` by @bcdurak in https://github.com/zenml-io/zenml/pull/2230
|
150
|
-
* Fix CI check to disallow template testing on forked repositories by @strickvl in https://github.com/zenml-io/zenml/pull/2231
|
151
|
-
* Fix fork check syntax by @strickvl in https://github.com/zenml-io/zenml/pull/2237
|
152
|
-
* Add missing annotations section to zenml service account by @wjayesh in https://github.com/zenml-io/zenml/pull/2234
|
153
|
-
* Allow filtering artifacts with/without custom names by @schustmi in https://github.com/zenml-io/zenml/pull/2226
|
154
|
-
* Adjust migration settings based on database engine by @strickvl in https://github.com/zenml-io/zenml/pull/2236
|
155
|
-
* Added one more chapter to starter guide by @htahir1 in https://github.com/zenml-io/zenml/pull/2238
|
156
|
-
* Add Polars materializer by @christianversloot in https://github.com/zenml-io/zenml/pull/2229
|
157
|
-
|
158
|
-
## New Contributors
|
159
|
-
* @sean-hickey-wf made their first contribution in https://github.com/zenml-io/zenml/pull/2153
|
160
|
-
* @dependabot 🤖 made their first contribution in https://github.com/zenml-io/zenml/pull/2201
|
161
|
-
|
162
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.53.1...0.54.0
|
163
|
-
|
164
|
-
# 0.53.1
|
165
|
-
|
166
|
-
This minor release contains a hot fix for a bug that was introduced in 0.53.0
|
167
|
-
where the secrets manager flavors were not removed from the database
|
168
|
-
properly. This release fixes that issue.
|
169
|
-
|
170
|
-
## What's Changed
|
171
|
-
* Remove secrets manager flavors from DB by @stefannica in https://github.com/zenml-io/zenml/pull/2182
|
172
|
-
|
173
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.53.0...0.53.1
|
174
|
-
|
175
|
-
# 0.53.0
|
176
|
-
|
177
|
-
This release is packed with a deeply reworked quickstart example and starter template, the removal of secret manager stack component, improved experience with Cloud Secret Stores, support for tags and metadata directly in Model Versions, some breaking changes for Model Control Plane and a few bugfixes.
|
178
|
-
|
179
|
-
## Breaking changes
|
180
|
-
|
181
|
-
### Secret Manager stack components sunset
|
182
|
-
|
183
|
-
Upon upgrading, all Secrets Manager stack components will be removed from the Stacks that still contain them and from the database. This also implies that access to any remaining secrets managed through Secrets Manager stack components will be lost. If you still have secrets configured and managed through Secrets Manager stack components, please consider migrating all your existing secrets to the centralized secrets store *before upgrading* by means of the `zenml secrets-manager secret migrate` CLI command. Also see the `zenml secret --help` command for more information.
|
184
|
-
|
185
|
-
### Renaming "endpoints" to "deployments" in Model Control Plane
|
186
|
-
|
187
|
-
This is just a renaming to provide better alignment with industry standards. Though, it will affect:
|
188
|
-
- `ArtifactConfig(..., is_endpoint_artifact=True)` now is `ArtifactConfig(..., is_deployment_artifact=True)`
|
189
|
-
- CLI command `zenml model endpoint_artifacts ...` now is `zenml model deployment_artifacts ...`
|
190
|
-
- `Client().list_model_version_artifact_links(..., only_endpoint_artifacts=True)` now is `Client().list_model_version_artifact_links(..., only_deployment_artifacts=True)`
|
191
|
-
- `ModelVersion(...).get_endpoint_artifact(...)` now is `ModelVersion(...).get_deployment_artifact(...)`
|
192
|
-
|
193
|
-
## Major bugfixes
|
194
|
-
* Fix various bugs by @stefannica in https://github.com/zenml-io/zenml/pull/2147
|
195
|
-
* Adding a link from pipeline runs to code repositories by @bcdurak in https://github.com/zenml-io/zenml/pull/2146
|
196
|
-
* Fix Client doesn't recover from remote connection resets by @avishniakov in https://github.com/zenml-io/zenml/pull/2129
|
197
|
-
* Bugfix: `run_metadata` value returns string instead of other types by @avishniakov in https://github.com/zenml-io/zenml/pull/2149
|
198
|
-
* `KubernetesSparkStepOperator` imports fails by @avishniakov in https://github.com/zenml-io/zenml/pull/2159
|
199
|
-
* Fix `get_pipeline_context().model_version.get_artifact(...)` flow by @avishniakov in https://github.com/zenml-io/zenml/pull/2162
|
200
|
-
|
201
|
-
## What's Changed
|
202
|
-
* Model Versions are taggable by @avishniakov in https://github.com/zenml-io/zenml/pull/2102
|
203
|
-
* Adding a condition to the PR template by @bcdurak in https://github.com/zenml-io/zenml/pull/2140
|
204
|
-
* trying local caching for custom runners by @safoinme in https://github.com/zenml-io/zenml/pull/2148
|
205
|
-
* make template tests runs on ubuntu latest instead of custom runners by @safoinme in https://github.com/zenml-io/zenml/pull/2150
|
206
|
-
* Fix various bugs by @stefannica in https://github.com/zenml-io/zenml/pull/2147
|
207
|
-
* Fix `importlib` calling to `importlib.metadata` by @safoinme in https://github.com/zenml-io/zenml/pull/2160
|
208
|
-
* Debugging `zenml clean` by @bcdurak in https://github.com/zenml-io/zenml/pull/2119
|
209
|
-
* Add metadata to model versions by @avishniakov in https://github.com/zenml-io/zenml/pull/2109
|
210
|
-
* Adding a link from pipeline runs to code repositories by @bcdurak in https://github.com/zenml-io/zenml/pull/2146
|
211
|
-
* Moving tags to the body for artifacts and artifact versions by @bcdurak in https://github.com/zenml-io/zenml/pull/2138
|
212
|
-
* Fix MLFlow test by @avishniakov in https://github.com/zenml-io/zenml/pull/2161
|
213
|
-
* Fix Client doesn't recover from remote connection resets by @avishniakov in https://github.com/zenml-io/zenml/pull/2129
|
214
|
-
* Bugfix: `run_metadata` value returns string instead of other types by @avishniakov in https://github.com/zenml-io/zenml/pull/2149
|
215
|
-
* `KubernetesSparkStepOperator` imports fails by @avishniakov in https://github.com/zenml-io/zenml/pull/2159
|
216
|
-
* Endpoint artifacts rename to deployment artifacts by @avishniakov in https://github.com/zenml-io/zenml/pull/2134
|
217
|
-
* Fix `get_pipeline_context().model_version.get_artifact(...)` flow by @avishniakov in https://github.com/zenml-io/zenml/pull/2162
|
218
|
-
* Add CodeRabbit config to repo base by @strickvl in https://github.com/zenml-io/zenml/pull/2165
|
219
|
-
* Feature: use service connectors to authenticate secrets stores. by @stefannica in https://github.com/zenml-io/zenml/pull/2154
|
220
|
-
* Add dependabot updates for Github Actions on CI by @strickvl in https://github.com/zenml-io/zenml/pull/2087
|
221
|
-
* Run DB migration testing using MySQL alongside SQLite by @strickvl in https://github.com/zenml-io/zenml/pull/2113
|
222
|
-
* Remove `precommit` by @strickvl in https://github.com/zenml-io/zenml/pull/2164
|
223
|
-
* Remove support for secrets managers by @stefannica in https://github.com/zenml-io/zenml/pull/2163
|
224
|
-
* Add MariaDB test harnesses by @christianversloot in https://github.com/zenml-io/zenml/pull/2155
|
225
|
-
* Feature/update quickstart from template by @AlexejPenner in https://github.com/zenml-io/zenml/pull/2157
|
226
|
-
* Bump MLFlow to 2.9.2 by @christianversloot in https://github.com/zenml-io/zenml/pull/2156
|
227
|
-
|
228
|
-
|
229
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.52.0...0.53.0
|
230
|
-
|
231
|
-
# 0.52.0
|
232
|
-
|
233
|
-
This adds the ability to pass in pipeline parameters as YAML configuration and fixes a couple of minor issues affecting the W&B integration and the way expiring credentials are refreshed when service connectors are used.
|
234
|
-
|
235
|
-
## Breaking Change
|
236
|
-
|
237
|
-
The current pipeline YAML configurations are now being validated to ensure that configured parameters match what is available in the code. This means that if you have a pipeline that is configured with a parameter that has a different value that what is provided through code, the pipeline will fail to run. This is a breaking change, but it is a good thing as it will help you catch errors early on.
|
238
|
-
|
239
|
-
This is an example of a pipeline configuration that will fail to run:
|
240
|
-
|
241
|
-
```yaml
|
242
|
-
parameters:
|
243
|
-
some_param: 24
|
244
|
-
|
245
|
-
steps:
|
246
|
-
my_step:
|
247
|
-
parameters:
|
248
|
-
input_2: 42
|
249
|
-
```
|
250
|
-
|
251
|
-
```python
|
252
|
-
# run.py
|
253
|
-
@step
|
254
|
-
def my_step(input_1: int, input_2: int) -> None:
|
255
|
-
pass
|
256
|
-
|
257
|
-
@pipeline
|
258
|
-
def my_pipeline(some_param: int):
|
259
|
-
# here an error will be raised since `input_2` is
|
260
|
-
# `42` in config, but `43` was provided in the code
|
261
|
-
my_step(input_1=42, input_2=43)
|
262
|
-
|
263
|
-
if __name__=="__main__":
|
264
|
-
# here an error will be raised since `some_param` is
|
265
|
-
# `24` in config, but `23` was provided in the code
|
266
|
-
my_pipeline(23)
|
267
|
-
```
|
268
|
-
|
269
|
-
## What's Changed
|
270
|
-
* Passing pipeline parameters as yaml config by @avishniakov in https://github.com/zenml-io/zenml/pull/2058
|
271
|
-
* Side-effect free tests by @avishniakov in https://github.com/zenml-io/zenml/pull/2065
|
272
|
-
* Fix various bugs by @stefannica in https://github.com/zenml-io/zenml/pull/2124
|
273
|
-
|
274
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.51.0...0.52.0
|
275
|
-
|
276
|
-
# 0.51.0
|
277
|
-
|
278
|
-
This release comes with a breaking change to the model version model, a new use-case example for NLP, and a range of bug fixes and enhancements to the artifact management and pipeline run management features.
|
279
|
-
|
280
|
-
## Breaking Change
|
281
|
-
* Artifact Version Table + Artifact Tagging by @fa9r in https://github.com/zenml-io/zenml/pull/2081
|
282
|
-
* Converting model models to use the new hydration paradigm by @bcdurak in https://github.com/zenml-io/zenml/pull/2101
|
283
|
-
|
284
|
-
## New Example
|
285
|
-
* NLP Template Example is a new example that demonstrates how to use ZenML for NLP tasks. by @safoinme in https://github.com/zenml-io/zenml/pull/2070
|
286
|
-
|
287
|
-
|
288
|
-
## What's Changed
|
289
|
-
* Updated to one quickstart again by @htahir1 in https://github.com/zenml-io/zenml/pull/2092
|
290
|
-
* Fix Nightly Build workflow files by @strickvl in https://github.com/zenml-io/zenml/pull/2090
|
291
|
-
* Make PyPi release depend on DB migration tests passing by @strickvl in https://github.com/zenml-io/zenml/pull/2088
|
292
|
-
* Bump `mlstacks` version in ZenML extra by @strickvl in https://github.com/zenml-io/zenml/pull/2091
|
293
|
-
* Fix SQL schema imports by @stefannica in https://github.com/zenml-io/zenml/pull/2098
|
294
|
-
* Fix migration for unowned stacks/components by @schustmi in https://github.com/zenml-io/zenml/pull/2099
|
295
|
-
* Polymorthic `run_metadata` by @avishniakov in https://github.com/zenml-io/zenml/pull/2064
|
296
|
-
* Update ruff formatter (for bugfixes) by @strickvl in https://github.com/zenml-io/zenml/pull/2106
|
297
|
-
* Lock in airflow version as higher versions will fail by @AlexejPenner in https://github.com/zenml-io/zenml/pull/2108
|
298
|
-
* Swap contents for HTMLString and MarkdownString in docs by @christianversloot in https://github.com/zenml-io/zenml/pull/2110
|
299
|
-
* Fix secrets list with cloud secrets stores and RBAC by @stefannica in https://github.com/zenml-io/zenml/pull/2107
|
300
|
-
* More track events by @htahir1 in https://github.com/zenml-io/zenml/pull/2112
|
301
|
-
* Fix pipeline run cascade deletion by @fa9r in https://github.com/zenml-io/zenml/pull/2104
|
302
|
-
* Take integrations tests out of unit tests folder by @safoinme in https://github.com/zenml-io/zenml/pull/2100
|
303
|
-
* Allow extra values when dehydrating response models by @schustmi in https://github.com/zenml-io/zenml/pull/2114
|
304
|
-
* Request optimizations by @schustmi in https://github.com/zenml-io/zenml/pull/2103
|
305
|
-
* Pagination in model versions by @avishniakov in https://github.com/zenml-io/zenml/pull/2115
|
306
|
-
* Add `StepContext.inputs` property by @fa9r in https://github.com/zenml-io/zenml/pull/2105
|
307
|
-
|
308
|
-
|
309
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.50.0...0.51.0
|
310
|
-
|
311
|
-
|
312
|
-
# 0.50.0
|
313
|
-
|
314
|
-
In this release, we introduce key updates aimed at improving user experience and security.
|
315
|
-
The `ModelConfig` object has been renamed to `ModelVersion` for a more intuitive interface.
|
316
|
-
Additionally, the release features enhancements such as optimized model hydration for better performance,
|
317
|
-
alongside a range of bug fixes and contributions from both new and returning community members.
|
318
|
-
|
319
|
-
## Breaking Change
|
320
|
-
- We have renamed the `ModelConfig` object to `ModelVersion` with other related changes to the model control plane,
|
321
|
-
the goal of this is to bring a simplified user-interface experience, so once ModelVersion is configured in
|
322
|
-
@pipeline or @step it will travel into all other user-facing places: step context, client, etc. by @avishniakov in [#2044](https://github.com/zenml-io/zenml/pull/2044)
|
323
|
-
- introducing RBAC for server endpoints, ensuring users have appropriate permissions for actions on resources.
|
324
|
-
Additionally, it improves data handling by dehydrating response models to redact inaccessible information, while
|
325
|
-
service accounts retain full permissions due to current database constraints. by @schustmi in [#1999](https://github.com/zenml-io/zenml/pull/1999)
|
326
|
-
|
327
|
-
## Enhancements
|
328
|
-
- Optimizing model hydration by @bcdurak in [#1971](https://github.com/zenml-io/zenml/pull/1971)
|
329
|
-
- Improve alembic migration safety by @fa9r in [#2073](https://github.com/zenml-io/zenml/pull/2073)
|
330
|
-
- Model Link Filtering by Artifact / Run Name by @fa9r in [#2074](https://github.com/zenml-io/zenml/pull/2074)
|
331
|
-
|
332
|
-
## Bug Fixes
|
333
|
-
- Fix tag<>resource ID generator to fix the issue of manipulating migrated tags properly [#2056](https://github.com/zenml-io/zenml/pull/2056)
|
334
|
-
- Fixes for `k3d` deployments via `mlstacks` using the ZenML CLI wrapper [#2059](https://github.com/zenml-io/zenml/pull/2059)
|
335
|
-
- Fix some filter options for pipeline runs by @schustmi [#2078](https://github.com/zenml-io/zenml/pull/2078)
|
336
|
-
- Fix Label Studio image annotation example by @strickvl [#2010](https://github.com/zenml-io/zenml/pull/2010)
|
337
|
-
- Alembic migration fix for databases with scheduled pipelines with 2+ runs by @bcdurak [#2072](https://github.com/zenml-io/zenml/pull/2072)
|
338
|
-
- Model version endpoint fixes by @schustmi in [#2060](https://github.com/zenml-io/zenml/pull/2060)
|
339
|
-
|
340
|
-
## ZenML Helm Chart Changes
|
341
|
-
- Make helm chart more robust to accidental secret deletions by @stefannica in [#2053](https://github.com/zenml-io/zenml/pull/2053)
|
342
|
-
- Separate helm hook resources from regular resources by @stefannica in [#2055](https://github.com/zenml-io/zenml/pull/2055)
|
343
|
-
|
344
|
-
## Other Changes
|
345
|
-
* Connectors docs small fixes by @strickvl in https://github.com/zenml-io/zenml/pull/2050
|
346
|
-
* Feature/configurable service account for seldon predictor service by @Johnyz21 in https://github.com/zenml-io/zenml/pull/1725
|
347
|
-
* Adding NLP Template Example by @safoinme in https://github.com/zenml-io/zenml/pull/2051
|
348
|
-
* Fix CI by @fa9r in https://github.com/zenml-io/zenml/pull/2069
|
349
|
-
* Depaginate step runs to allow running pipelines with arbitrary step count by @schustmi in https://github.com/zenml-io/zenml/pull/2068
|
350
|
-
* Remove user name from orchestrator run name by @schustmi in https://github.com/zenml-io/zenml/pull/2067
|
351
|
-
* Artifacts Tab by @fa9r in https://github.com/zenml-io/zenml/pull/1943
|
352
|
-
* Add warnings/updates to Huggingface Spaces deployment docs by @strickvl in https://github.com/zenml-io/zenml/pull/2052
|
353
|
-
* Nightly builds by @strickvl in https://github.com/zenml-io/zenml/pull/2031
|
354
|
-
* Allow for custom disk size and type when using VertexAI Step Operator by @strickvl in https://github.com/zenml-io/zenml/pull/2054
|
355
|
-
* Set nightly builds to run at half-past the hour by @strickvl in https://github.com/zenml-io/zenml/pull/2077
|
356
|
-
* Set DCP template tag by @avishniakov in https://github.com/zenml-io/zenml/pull/2076
|
357
|
-
* Add missing dehydration in get_service_connector endpoint by @schustmi in https://github.com/zenml-io/zenml/pull/2080
|
358
|
-
* Replace `black` with `ruff format` / bump `mypy` by @strickvl in https://github.com/zenml-io/zenml/pull/2082
|
359
|
-
* ModelVersion in pipeline context to pass in steps by @avishniakov in https://github.com/zenml-io/zenml/pull/2079
|
360
|
-
* Pin `bcrypt` by @strickvl in https://github.com/zenml-io/zenml/pull/2083
|
361
|
-
|
362
|
-
## New Contributors
|
363
|
-
* @Johnyz21 made their first contribution in https://github.com/zenml-io/zenml/pull/1725
|
364
|
-
|
365
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.47.0...0.50.0
|
366
|
-
|
367
|
-
|
368
|
-
# 0.47.0
|
369
|
-
This release fixes a bug that was introduced in 0.46.1 where the default user
|
370
|
-
was made inaccessible and was inadvertently duplicated. This release rescues
|
371
|
-
the original user and renames the duplicate.
|
372
|
-
|
373
|
-
## What's Changed
|
374
|
-
* Create tags table by @avishniakov in https://github.com/zenml-io/zenml/pull/2036
|
375
|
-
* Bring dashboard back to the release by @avishniakov in https://github.com/zenml-io/zenml/pull/2046
|
376
|
-
* Fix duplicate default user by @stefannica in https://github.com/zenml-io/zenml/pull/2045
|
377
|
-
|
378
|
-
|
379
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.46.1...tmp
|
380
|
-
|
381
|
-
# 0.46.1
|
382
|
-
|
383
|
-
The 0.46.1 release introduces support for Service Accounts and API Keys that
|
384
|
-
can be used to authenticate with the ZenML server from environments that do not
|
385
|
-
support the web login flow, such as CI/CD environments, for example.
|
386
|
-
|
387
|
-
Also included in this release are some documentation updates and bug fixes,
|
388
|
-
notably moving the database migration logic deployed with the Helm chart out of
|
389
|
-
the init containers and into a Kubernetes Job, which makes it possible to scale
|
390
|
-
out the ZenML server deployments without the risk of running into database
|
391
|
-
migration conflicts.
|
392
|
-
|
393
|
-
## What's Changed
|
394
|
-
* Small improvements to Hub docs page by @strickvl in https://github.com/zenml-io/zenml/pull/2015
|
395
|
-
* Pin OpenAI integration to `<1.0.0` by @strickvl in https://github.com/zenml-io/zenml/pull/2027
|
396
|
-
* Make error message nicer for when two artifacts that share a prefix are found by @strickvl in https://github.com/zenml-io/zenml/pull/2023
|
397
|
-
* Move db-migration to `job` instead of `init-container` to allow replicas by @safoinme in https://github.com/zenml-io/zenml/pull/2021
|
398
|
-
* Fix stuck/broken CI by @strickvl in https://github.com/zenml-io/zenml/pull/2032
|
399
|
-
* Increase `step.source_code` Cut-Off Limit by @fa9r in https://github.com/zenml-io/zenml/pull/2025
|
400
|
-
* Improve artifact linkage logging in MCP by @avishniakov in https://github.com/zenml-io/zenml/pull/2016
|
401
|
-
* Upgrade feast so apidocs don't fail no mo by @AlexejPenner in https://github.com/zenml-io/zenml/pull/2028
|
402
|
-
* Remove NumPy Visualizations for 2D Arrays by @fa9r in https://github.com/zenml-io/zenml/pull/2033
|
403
|
-
* Fix user activation bug by @stefannica in https://github.com/zenml-io/zenml/pull/2037
|
404
|
-
* Remove `create_new_model_version` arg of `ModelConfig` by @avishniakov in https://github.com/zenml-io/zenml/pull/2030
|
405
|
-
* Extend the wait period in between PyPi package publication and Docker image building for releases by @strickvl in https://github.com/zenml-io/zenml/pull/2029
|
406
|
-
* Make `zenml up` prefill username when launching dashboard by @strickvl in https://github.com/zenml-io/zenml/pull/2024
|
407
|
-
* Add warning when artifact store cannot be loaded by @strickvl in https://github.com/zenml-io/zenml/pull/2011
|
408
|
-
* Add extra config to `Kaniko` docs by @safoinme in https://github.com/zenml-io/zenml/pull/2019
|
409
|
-
* ZenML API Keys and Service Accounts by @stefannica in https://github.com/zenml-io/zenml/pull/1840
|
410
|
-
|
411
|
-
|
412
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.46.0..0.46.1
|
413
|
-
|
414
|
-
|
415
|
-
# 0.46.0
|
416
|
-
|
417
|
-
This release brings some upgrades, documentation updates and bug fixes. Notably,
|
418
|
-
our `langchain` integration now supports more modern versions and has been
|
419
|
-
upgraded to a new version at the lower edge of supported packages on account of
|
420
|
-
a security vulnerability.
|
421
|
-
|
422
|
-
Other fixes related to the Model Control Plane which was updated to support the
|
423
|
-
deletion of model versions via the CLI, for example.
|
424
|
-
|
425
|
-
## Breaking Change
|
426
|
-
|
427
|
-
We removed the `llama_index` integration in this release. This related to
|
428
|
-
unsolvable dependency clashes that relate to `sqlmodel` and our database. We
|
429
|
-
expect these clashes to be resolved in the future and then we will add our
|
430
|
-
integration back in. If you were using the `llama_index` materializer that was
|
431
|
-
part of the integration, you will have to use a custom materializer in the
|
432
|
-
meanwhile. We apologize for the inconvenience.
|
433
|
-
|
434
|
-
## What's Changed
|
435
|
-
* MCP-driven E2E template by @avishniakov in https://github.com/zenml-io/zenml/pull/2004
|
436
|
-
* Model scoped endpoints by @avishniakov in https://github.com/zenml-io/zenml/pull/2003
|
437
|
-
* Delete model version in cli by @avishniakov in https://github.com/zenml-io/zenml/pull/2006
|
438
|
-
* Add latest version to model list response by @avishniakov in https://github.com/zenml-io/zenml/pull/2007
|
439
|
-
* Fix `gcs bucket` docs error message by @safoinme in https://github.com/zenml-io/zenml/pull/2018
|
440
|
-
* Fix `Skypilot` docs configuration by @safoinme in https://github.com/zenml-io/zenml/pull/2017
|
441
|
-
* Bump `langchain`, disable `llama_index`, and fix Vector Store materializer by @strickvl in https://github.com/zenml-io/zenml/pull/2013
|
442
|
-
* Fix Build Options of `GCPImageBuilder` by @fa9r in https://github.com/zenml-io/zenml/pull/1992
|
443
|
-
* Fix the stack component describe CLI output by @stefannica in https://github.com/zenml-io/zenml/pull/2001
|
444
|
-
|
445
|
-
|
446
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.45.6...0.46.0
|
447
|
-
|
448
|
-
# 0.45.6
|
449
|
-
|
450
|
-
This release brings an array of enhancements and refinements. Notable improvements include
|
451
|
-
allowing for `disconnecting` service connectors from stack components, adding connector support to the
|
452
|
-
sagemaker step operator, turning synchronous mode on by default for all orchestrators, and enabling
|
453
|
-
server-side component config validation.
|
454
|
-
|
455
|
-
## What's Changed
|
456
|
-
* Updating `README.md` and update images by @znegrin in https://github.com/zenml-io/zenml/pull/1986
|
457
|
-
* Always set the active workspace to be the default workspace server side by @stefannica in https://github.com/zenml-io/zenml/pull/1989
|
458
|
-
* Update outdated CLI docs by @strickvl in https://github.com/zenml-io/zenml/pull/1990
|
459
|
-
* Turn synchronous mode on by default for all orchestrators by @stefannica in https://github.com/zenml-io/zenml/pull/1991
|
460
|
-
* Use docker credentials in the skypilot orchestrator by @stefannica in https://github.com/zenml-io/zenml/pull/1983
|
461
|
-
* Add missing space to `@step` warning message by @strickvl in https://github.com/zenml-io/zenml/pull/1994
|
462
|
-
* Fix sagemaker orchestrator and step operator env vars and other minor bugs by @stefannica in https://github.com/zenml-io/zenml/pull/1993
|
463
|
-
* fix: `BasePyTorchMaterliazer` -> `Materializer` by @cameronraysmith in https://github.com/zenml-io/zenml/pull/1969
|
464
|
-
* allow calling old base pytorch materilizzer by @safoinme in https://github.com/zenml-io/zenml/pull/1997
|
465
|
-
* Add connector support to sagemaker step operator. by @stefannica in https://github.com/zenml-io/zenml/pull/1996
|
466
|
-
* Server-Side Component Config Validation by @fa9r in https://github.com/zenml-io/zenml/pull/1988
|
467
|
-
* Allow disconnecting service-connector from stack component by @safoinme in https://github.com/zenml-io/zenml/pull/1864
|
468
|
-
|
469
|
-
## New Contributors
|
470
|
-
* @znegrin made their first contribution in https://github.com/zenml-io/zenml/pull/1986
|
471
|
-
|
472
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.45.5...0.45.6
|
473
|
-
|
474
|
-
|
475
|
-
# 0.45.5
|
476
|
-
|
477
|
-
This minor release contains bugfixes and documentation improvements. Notably,
|
478
|
-
our `sqlmodel` dependency has been pinned to 0.0.8 which fixes installation
|
479
|
-
errors following the release of 0.0.9.
|
480
|
-
|
481
|
-
## What's Changed
|
482
|
-
* Add a 'how do I...' section into docs by @strickvl in https://github.com/zenml-io/zenml/pull/1953
|
483
|
-
* Bump `mypy`, `ruff` and `black` by @strickvl in https://github.com/zenml-io/zenml/pull/1963
|
484
|
-
* Fix double slashes in weblogin by @schustmi in https://github.com/zenml-io/zenml/pull/1972
|
485
|
-
* SQLModel docs backport fixes by @strickvl in https://github.com/zenml-io/zenml/pull/1975
|
486
|
-
* Updated quickstart command in cloud quickstart by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1977
|
487
|
-
* Make sure vertex job id is only lower case letter, number or dash by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1978
|
488
|
-
* Fix DB initialization when using external authentication by @schustmi in https://github.com/zenml-io/zenml/pull/1965
|
489
|
-
* Pin SQLModel dependency to `0.0.8` by @strickvl in https://github.com/zenml-io/zenml/pull/1973
|
490
|
-
|
491
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.45.4...0.45.5
|
492
|
-
|
493
|
-
# 0.45.4
|
494
|
-
|
495
|
-
This minor update fixes a database migration bug that you could potentially
|
496
|
-
encounter while upgrading your ZenML version and relates to use of the
|
497
|
-
`ExternalArtifact` object.
|
498
|
-
If you are upgrading from <0.45.x version, this is the recommended release.
|
499
|
-
|
500
|
-
**PROBLEMS?**: If you upgraded to ZenML v0.45.2 or v0.45.3 and are experiencing
|
501
|
-
issues with your database, please consider upgrading to v0.45.4 instead.
|
502
|
-
|
503
|
-
## What's Changed
|
504
|
-
* Increase reuse of `ModelConfig` by @avishniakov in https://github.com/zenml-io/zenml/pull/1954
|
505
|
-
* resolve alembic branches by @avishniakov in https://github.com/zenml-io/zenml/pull/1964
|
506
|
-
* Fix corrupted migration for old dbs by @avishniakov in https://github.com/zenml-io/zenml/pull/1966
|
507
|
-
|
508
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.45.3...0.45.4
|
509
|
-
|
510
|
-
# 0.45.3
|
511
|
-
|
512
|
-
This minor update fixes a database migration bug that you could potentially
|
513
|
-
encounter while upgrading your ZenML version and relates to use of the
|
514
|
-
`ExternalArtifact` object.
|
515
|
-
|
516
|
-
**PROBLEMS?**: If you upgraded to ZenML v0.45.2 and are experiencing
|
517
|
-
issues with your database, please either [reach out to us on Slack directly](https://zenml.io/slack-invite/) or
|
518
|
-
feel free to [use this migration
|
519
|
-
script](https://gist.github.com/strickvl/2178d93c8693f068768a82587fd4db75) that will
|
520
|
-
manually fix the issue.
|
521
|
-
|
522
|
-
This release also includes a bugfix from @cameronraysmith relating to the
|
523
|
-
resolution of our Helm chart OCI location. Thank you!
|
524
|
-
|
525
|
-
## What's Changed
|
526
|
-
* fix: match chart name in docs to publish workflow by @cameronraysmith in https://github.com/zenml-io/zenml/pull/1942
|
527
|
-
* Evaluate YAML based config early + OSS-2511 by @avishniakov in https://github.com/zenml-io/zenml/pull/1876
|
528
|
-
* Fixing nullable parameter to avoid extra migrations by @bcdurak in https://github.com/zenml-io/zenml/pull/1955
|
529
|
-
* Pin Helm version to avoid 400 Bad Request error by @wjayesh in https://github.com/zenml-io/zenml/pull/1958
|
530
|
-
* `external_input_artifact` backward compatibility with alembic by @avishniakov in https://github.com/zenml-io/zenml/pull/1957
|
531
|
-
|
532
|
-
## New Contributors
|
533
|
-
* @cameronraysmith made their first contribution in https://github.com/zenml-io/zenml/pull/1942
|
534
|
-
|
535
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.45.2...0.45.3
|
536
|
-
|
537
|
-
# 0.45.2
|
538
|
-
|
539
|
-
This release replaces 0.45.0 and 0.45.1, and fixes the major migration bugs that were in
|
540
|
-
that yanked release. Please upgrade directly to 0.45.2 and avoid upgrading to
|
541
|
-
0.45.0 to avoid unexpected migration issues.
|
542
|
-
|
543
|
-
Note that 0.45.0 and 0.45.1 were removed from PyPI due to an issue with the
|
544
|
-
alembic versions + migration which could affect the database state. This release
|
545
|
-
fixes that issue.
|
546
|
-
|
547
|
-
If you have already upgraded to 0.45.0 please [let us know in Slack](https://zenml.io/slack-invite/) and we'll happy to assist in rollback and recovery.
|
548
|
-
|
549
|
-
This release introduces a major upgrade to ZenML, featuring a new authentication mechanism, performance improvements, the introduction of the model control plane, and internal enhancements.
|
550
|
-
|
551
|
-
## New Authentication Mechanism (#4303)
|
552
|
-
|
553
|
-
Our improved authentication mechanism offers a more secure way of connecting to the ZenML server. It initiates a device flow that prompts you to log in via the browser dashboard:
|
554
|
-
|
555
|
-
```
|
556
|
-
zenml connect --url <YOUR_SERVER_URL>
|
557
|
-
```
|
558
|
-
|
559
|
-
This eliminates the need for explicit credential input. The previous method (`zenml connect --url <URL> --username <USERNAME> --password <PASSWORD>`) remains operational but is less recommended due to security concerns.
|
560
|
-
|
561
|
-
**Critical** This change disrupts existing pipeline schedules. After upgrading, manually cancel and reschedule pipelines using the updated version of ZenML.
|
562
|
-
|
563
|
-
For more information, read about the device flow in our [documentation](https://docs.zenml.io/user-guide/starter-guide/switch-to-production).
|
564
|
-
|
565
|
-
## Performance enhancements (#3207)
|
566
|
-
|
567
|
-
Internal API adjustments have reduced the footprint of ZenML API objects by up to 35%. This will particularly benefit users with large step and pipeline configurations. Further reductions will be implemented in our next release.
|
568
|
-
|
569
|
-
## Model Control Plane debut (#5648)
|
570
|
-
|
571
|
-
ZenML now includes a preliminary version of the model control plane, a feature for registering models and their metadata on a single ZenML dashboard view. Future releases will provide more details. To test this early version, follow this [example](https://github.com/zenml-io/zenml-plugins/tree/main/model_control_plane).
|
572
|
-
|
573
|
-
## Breaking Changes
|
574
|
-
|
575
|
-
- Environment variables `ZENML_AUTH_TYPE` and `ZENML_JWT_SECRET_KEY` have been renamed to `ZENML_SERVER_AUTH_SCHEME` and `ZENML_SERVER_JWT_SECRET_KEY`, respectively.
|
576
|
-
- All ZenML server-issued JWT tokens now include an issuer and an audience. After the server update, current scheduled pipelines become invalidated. Reset your schedules and reconnect all clients to the server to obtain new tokens.
|
577
|
-
- `UnmaterializedArtifact` has been relocated to `zenml.artifacts`. Change your import statement from `from zenml.materializers import UnmaterializedArtifact` to `from zenml.artifacts.unmaterialized_artifact import UnmaterializedArtifact`.
|
578
|
-
|
579
|
-
## Deprecations
|
580
|
-
|
581
|
-
- `zenml.steps.external_artifact.ExternalArtifact` has moved to `zenml.artifacts.external_artifact.ExternalArtifact`.
|
582
|
-
|
583
|
-
|
584
|
-
## And the rest:
|
585
|
-
|
586
|
-
* Discord alerter integration by @bhatt-priyadutt in https://github.com/zenml-io/zenml/pull/1818. Huge shoutout to you priyadutt - we're sending some swag your way!
|
587
|
-
* Update Neptune dependency: `neptune-client` > `neptune` by @fa9r in https://github.com/zenml-io/zenml/pull/1837
|
588
|
-
* Disable codeql on pushes to `develop` by @strickvl in https://github.com/zenml-io/zenml/pull/1842
|
589
|
-
* Template not updating due to git diff misuse by @avishniakov in https://github.com/zenml-io/zenml/pull/1844
|
590
|
-
* Bump feast version to fix api docs generation by @fa9r in https://github.com/zenml-io/zenml/pull/1845
|
591
|
-
* CI Fixes / Improvements by @fa9r in https://github.com/zenml-io/zenml/pull/1848
|
592
|
-
* Fix MLflow registry methods with empty metadata by @fa9r in https://github.com/zenml-io/zenml/pull/1843
|
593
|
-
* Use configured template REF in CI by @avishniakov in https://github.com/zenml-io/zenml/pull/1851
|
594
|
-
* Fix template REF in CI by @avishniakov in https://github.com/zenml-io/zenml/pull/1852
|
595
|
-
* Fix AWS service connector installation requirements by @stefannica in https://github.com/zenml-io/zenml/pull/1850
|
596
|
-
* [Docs] Improvements to custom flavor and custom orchestrator pages by @htahir1 in https://github.com/zenml-io/zenml/pull/1747
|
597
|
-
* Optimizing the performance through database changes by @bcdurak in https://github.com/zenml-io/zenml/pull/1835
|
598
|
-
* Add `README` for `examples` folder by @strickvl in https://github.com/zenml-io/zenml/pull/1860
|
599
|
-
* Free up disk space in CI by @strickvl in https://github.com/zenml-io/zenml/pull/1863
|
600
|
-
* Make Terraform Optional Again by @fa9r in https://github.com/zenml-io/zenml/pull/1855
|
601
|
-
* Model watchtower becomes Model control plane by @strickvl in https://github.com/zenml-io/zenml/pull/1868
|
602
|
-
* Update documentation by @VishalKumar-S in https://github.com/zenml-io/zenml/pull/1872
|
603
|
-
* Fix CI by freeing up space on runner by @strickvl in https://github.com/zenml-io/zenml/pull/1866
|
604
|
-
* Allow for `user` param to be specified (successfully) in `DockerSettings` by @strickvl in https://github.com/zenml-io/zenml/pull/1857
|
605
|
-
* Add `get_pipeline_context` by @avishniakov in https://github.com/zenml-io/zenml/pull/1870
|
606
|
-
* [Helm] Use GCP creds directly instead of a file. by @wjayesh in https://github.com/zenml-io/zenml/pull/1874
|
607
|
-
* External authenticator support, authorized devices and web login by @stefannica in https://github.com/zenml-io/zenml/pull/1814
|
608
|
-
* Connect to Service-connector at component registration by @safoinme in https://github.com/zenml-io/zenml/pull/1858
|
609
|
-
* Fixing the `upgrade` migration script after the database changes by @bcdurak in https://github.com/zenml-io/zenml/pull/1877
|
610
|
-
* [Model Control Plane] v0.1 mega-branch by @avishniakov in https://github.com/zenml-io/zenml/pull/1816
|
611
|
-
* Update to templates by @htahir1 in https://github.com/zenml-io/zenml/pull/1878
|
612
|
-
* Docs for orgs, rbac and sso by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1875
|
613
|
-
* Convert network_config dict to NetworkConfig object in SageMaker orchestrator by @christianversloot in https://github.com/zenml-io/zenml/pull/1873
|
614
|
-
* Add missing Docker build options for GCP image builder by @strickvl in https://github.com/zenml-io/zenml/pull/1856
|
615
|
-
* Solve alembic branching issue by @avishniakov in https://github.com/zenml-io/zenml/pull/1879
|
616
|
-
* Fix typo for 0.45 release by @strickvl in https://github.com/zenml-io/zenml/pull/1881
|
617
|
-
* Only import ipinfo when necessary by @schustmi in https://github.com/zenml-io/zenml/pull/1888
|
618
|
-
* [Model Control Plane] Suppress excessive logging in model control plane by @avishniakov in https://github.com/zenml-io/zenml/pull/1885
|
619
|
-
* Add warning generation scripts for Gitbook docs by @strickvl in https://github.com/zenml-io/zenml/pull/1929
|
620
|
-
* Fix calling `click` decorator in model CLI command by @safoinme in https://github.com/zenml-io/zenml/pull/1932
|
621
|
-
* Lightweight template CI by @avishniakov in https://github.com/zenml-io/zenml/pull/1930
|
622
|
-
* Update `Skypilot` orchestrator setting docs section by @safoinme in https://github.com/zenml-io/zenml/pull/1931
|
623
|
-
|
624
|
-
### New Contributors
|
625
|
-
* @VishalKumar-S made their first contribution in https://github.com/zenml-io/zenml/pull/1872
|
626
|
-
|
627
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.44.3...0.45.0
|
628
|
-
|
629
|
-
|
630
|
-
# 0.44.3
|
631
|
-
|
632
|
-
## New Orchestrator: SkyPilot (#1765)
|
633
|
-
|
634
|
-
This release introduces a new orchestrator called SkyPilot. SkyPilot is a VM orchestrator
|
635
|
-
that can be used to run ZenML pipelines on a VM of choice in one of the three supported
|
636
|
-
cloud providers. It is a great choice for users who want to run ZenML pipelines on a GPU
|
637
|
-
instance, but don't want to use Kubernetes or serverless orchestrators like SageMaker.
|
638
|
-
|
639
|
-
## Fixes and Improvements
|
640
|
-
|
641
|
-
This release fixes several bugs and improves the user experience of the CLI and the
|
642
|
-
documentation. The most notable changes are:
|
643
|
-
* The new `connect` command that allows connecting all stack components within a stack to a
|
644
|
-
service connector with a single command.
|
645
|
-
* Adding an interactive flow to the `zenml stack deploy` command that allows users to
|
646
|
-
configure their stack in a guided manner.
|
647
|
-
* Add documentation on how to debug the SageMaker orchestrator, how to get started with
|
648
|
-
a quick cloud stack on GCP, and documentation on the use of service connectors with
|
649
|
-
enabled MFA.
|
650
|
-
|
651
|
-
## What's Changed
|
652
|
-
* Add support for empty API token in Kubernetes service connector. by @stefannica in https://github.com/zenml-io/zenml/pull/1808
|
653
|
-
* Use the container registry credentials to build images with the local image builder by @stefannica in https://github.com/zenml-io/zenml/pull/1804
|
654
|
-
* Fix CI by @fa9r in https://github.com/zenml-io/zenml/pull/1809
|
655
|
-
* Add documentation on how to debug the SageMaker orchestrator by @fa9r in https://github.com/zenml-io/zenml/pull/1810
|
656
|
-
* Bump `rich` and `uvicorn` by @jlopezpena in https://github.com/zenml-io/zenml/pull/1750
|
657
|
-
* SageMaker: Enable configuring authentication credentials explicitly by @fa9r in https://github.com/zenml-io/zenml/pull/1805
|
658
|
-
* Fix: ZenML DB migrations don't run if zenml is installed in path with spaces by @stefannica in https://github.com/zenml-io/zenml/pull/1815
|
659
|
-
* Fix mlflow 'run_name' variable overwriting by @iraadit in https://github.com/zenml-io/zenml/pull/1821
|
660
|
-
* Add `SECURITY.md` file for vulnerability disclosures. by @strickvl in https://github.com/zenml-io/zenml/pull/1824
|
661
|
-
* Add MFA limitation to service-connectors docs by @safoinme in https://github.com/zenml-io/zenml/pull/1827
|
662
|
-
* Improve `zenml stack describe` to show `mlstacks` outputs by @strickvl in https://github.com/zenml-io/zenml/pull/1826
|
663
|
-
* Documentation to get started with a quick cloud stack on GCP by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1807
|
664
|
-
* Fix missing text in git repo docs by @strickvl in https://github.com/zenml-io/zenml/pull/1831
|
665
|
-
* Handle irregular plural of `code_repository` for error message by @strickvl in https://github.com/zenml-io/zenml/pull/1832
|
666
|
-
* Connect stack to a service account by @safoinme in https://github.com/zenml-io/zenml/pull/1828
|
667
|
-
* SkyPilot Integration with VM Orchestrators by @htahir1 in https://github.com/zenml-io/zenml/pull/1765
|
668
|
-
* Add interactive CLI flow for `zenml stack deploy` by @strickvl in https://github.com/zenml-io/zenml/pull/1829
|
669
|
-
* Add `README` file for helm chart by @strickvl in https://github.com/zenml-io/zenml/pull/1830
|
670
|
-
* Fix slack environment variable in in `generative_chat` example README by @bhatt-priyadutt in https://github.com/zenml-io/zenml/pull/1836
|
671
|
-
|
672
|
-
## New Contributors
|
673
|
-
* @iraadit made their first contribution in https://github.com/zenml-io/zenml/pull/1821
|
674
|
-
|
675
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.44.2...0.44.3
|
676
|
-
|
677
|
-
|
678
|
-
# 0.44.2
|
679
|
-
|
680
|
-
This release contains updates for some of the most popular integrations, as well as several bug fixes and documentation improvements.
|
681
|
-
|
682
|
-
## Minor Default Behavior Changes
|
683
|
-
* The default page size for `zenml list` commands was reduced to 20 (from 50) to speed up the runtime of such commands.
|
684
|
-
* Simultaneous connection to local and remote ZenML servers is no longer possible since this caused several unexpected behaviors in the past.
|
685
|
-
|
686
|
-
## Integration Updates
|
687
|
-
- The `mlflow` integration now supports the newest MLflow version `2.6.0`.
|
688
|
-
- The `evidently` integration now supports the latest Evidently version `0.4.4`.
|
689
|
-
- The SageMaker orchestrator of the `aws` integration now supports authentication via service connectors.
|
690
|
-
|
691
|
-
## What's Changed
|
692
|
-
|
693
|
-
* Add `bandit` to CI for security linting by @strickvl in https://github.com/zenml-io/zenml/pull/1775
|
694
|
-
* Add `mlstacks` compatibility check to CI by @strickvl in https://github.com/zenml-io/zenml/pull/1767
|
695
|
-
* extend `StepContext` visibility to materializers by @avishniakov in https://github.com/zenml-io/zenml/pull/1769
|
696
|
-
* Revert GH changes to fix colima bug in macos gh by @safoinme in https://github.com/zenml-io/zenml/pull/1779
|
697
|
-
* Reduce CI runner count by @strickvl in https://github.com/zenml-io/zenml/pull/1777
|
698
|
-
* Add E2E template as example by @avishniakov in https://github.com/zenml-io/zenml/pull/1766
|
699
|
-
* Fix CI step names by @avishniakov in https://github.com/zenml-io/zenml/pull/1784
|
700
|
-
* Add vulnerability scanner by @strickvl in https://github.com/zenml-io/zenml/pull/1776
|
701
|
-
* Stop CI from running on push to `develop` by @strickvl in https://github.com/zenml-io/zenml/pull/1788
|
702
|
-
* Skip update templates outside PR by @avishniakov in https://github.com/zenml-io/zenml/pull/1786
|
703
|
-
* Fix azure service connector docs by @stefannica in https://github.com/zenml-io/zenml/pull/1778
|
704
|
-
* fix: use k8s V1CronJob instead of V1beta1CronJob (#1781) by @francoisserra in https://github.com/zenml-io/zenml/pull/1787
|
705
|
-
* Page limit adjustment by @bcdurak in https://github.com/zenml-io/zenml/pull/1791
|
706
|
-
* Prevent simultaneous connection to local and remote servers by @fa9r in https://github.com/zenml-io/zenml/pull/1792
|
707
|
-
* Update `MLflow` version to allow support for 2.6.0 by @safoinme in https://github.com/zenml-io/zenml/pull/1782
|
708
|
-
* Improve `ConnectionError` error message by @fa9r in https://github.com/zenml-io/zenml/pull/1783
|
709
|
-
* Stop old MLflow services when deploying new ones by @fa9r in https://github.com/zenml-io/zenml/pull/1793
|
710
|
-
* Prevent adding private components into shared stacks by @fa9r in https://github.com/zenml-io/zenml/pull/1794
|
711
|
-
* Publish server helm chart as part of CI by @wjayesh in https://github.com/zenml-io/zenml/pull/1740
|
712
|
-
* Docs on the use of ZenML-specific environment variables by @strickvl in https://github.com/zenml-io/zenml/pull/1796
|
713
|
-
* Add support for newer Evidently versions by @fa9r in https://github.com/zenml-io/zenml/pull/1780
|
714
|
-
* Link E2E example to docs by @avishniakov in https://github.com/zenml-io/zenml/pull/1790
|
715
|
-
* Copy step instance before applying configuration by @schustmi in https://github.com/zenml-io/zenml/pull/1798
|
716
|
-
* Fix AWS container registry image pushing with service connectors by @fa9r in https://github.com/zenml-io/zenml/pull/1797
|
717
|
-
* Make Sagemaker orchestrator work with connectors by @fa9r in https://github.com/zenml-io/zenml/pull/1799
|
718
|
-
* Add rebase Pre-requisite to PRs template by @safoinme in https://github.com/zenml-io/zenml/pull/1801
|
719
|
-
|
720
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.44.1...0.44.2
|
721
|
-
|
722
|
-
# 0.44.1
|
723
|
-
|
724
|
-
This release brings various improvements over the previous version, mainly
|
725
|
-
focusing on the usage of newly refactored `mlstacks` package, ZenML's `logging`
|
726
|
-
module and the changes in our analytics.
|
727
|
-
|
728
|
-
**Note:** *0.44.0 was removed from pypi due to an issue with the alembic versions which could affect the database state. A branch occurred in the versions: 0.42.1 -> [0.43.0, e1d66d91a099] -> 0.44.0. This release fixes the issue.<br>
|
729
|
-
The primary issue arises when deploying version 0.44.0 using a MySQL backend. Although the alembic migration executes all tasks up to 0.44.0, the alembic version represented in the database remains at 0.43.0. This issue persists irrespective of the measures taken, including trying various versions after 0.43.0.<br>
|
730
|
-
This imbalance leads to failure when running a second replica migration because the database's state is at 0.44.0 while the alembic version remains at 0.43.0. Similarly, attempts to run a second replica or restart the pod fail as the alembic tries to migrate from 0.43.0 to 0.44.0, which is not possible because these changes already exist in the database.<br>
|
731
|
-
Please note: If you encounter this problem, we recommend that you rollback to previous versions and then upgrade to 0.43.0. If you still experience difficulties, please join our Slack community at https://zenml.io/slack. We're ready to help you work through this issue.*
|
732
|
-
|
733
|
-
## What's Changed
|
734
|
-
|
735
|
-
* Remove e2e example and point to templates by @avishniakov in https://github.com/zenml-io/zenml/pull/1752
|
736
|
-
* Add cloud architecture docs by @htahir1 in https://github.com/zenml-io/zenml/pull/1751
|
737
|
-
* Update docs/docstrings following `mlstacks` repo name change by @strickvl in https://github.com/zenml-io/zenml/pull/1754
|
738
|
-
* Update Cloud deployment scenarios by @stefannica in https://github.com/zenml-io/zenml/pull/1757
|
739
|
-
* Fixing the logging message regarding caching by @bcdurak in https://github.com/zenml-io/zenml/pull/1748
|
740
|
-
* Improvements to the step logs storage functionality by @bcdurak in https://github.com/zenml-io/zenml/pull/1733
|
741
|
-
* Fix `qemu`/`colima` Github Actions bug by @safoinme in https://github.com/zenml-io/zenml/pull/1760
|
742
|
-
* Bump `ruff` and `mypy` by @strickvl in https://github.com/zenml-io/zenml/pull/1762
|
743
|
-
* Add Template Testing in Core by @avishniakov in https://github.com/zenml-io/zenml/pull/1745
|
744
|
-
* Removing analytics v1 and optimizing v2 by @bcdurak in https://github.com/zenml-io/zenml/pull/1753
|
745
|
-
* Update publish script to take a token by @strickvl in https://github.com/zenml-io/zenml/pull/1758
|
746
|
-
* Update variable name for release publication token by @strickvl in https://github.com/zenml-io/zenml/pull/1764
|
747
|
-
* Lock `MYSQL` Database during DB migrations by @safoinme in https://github.com/zenml-io/zenml/pull/1763
|
748
|
-
* `mlstacks` integration (and deprecation of old deployment logic) by @strickvl in https://github.com/zenml-io/zenml/pull/1721
|
749
|
-
* Upgrade typing extensions within api docs build workflow by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1741
|
750
|
-
* Fix branching alembic history by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1772
|
751
|
-
* Remove pinned `zenml` version specified in TOC for SDK docs by @strickvl in https://github.com/zenml-io/zenml/pull/1770
|
752
|
-
* Modified the track metadata for the opt-in event by @bcdurak in https://github.com/zenml-io/zenml/pull/1774
|
753
|
-
* Check alembic branch divergence in CI by @strickvl in https://github.com/zenml-io/zenml/pull/1773
|
754
|
-
* Remove the DB lock by @safoinme in https://github.com/zenml-io/zenml/pull/1771
|
755
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.43.0...0.44.1
|
756
|
-
|
757
|
-
|
758
|
-
# 0.44.0
|
759
|
-
|
760
|
-
This release brings various improvements over the previous version, mainly
|
761
|
-
focusing on the usage of newly refactored `mlstacks` package, ZenML's `logging`
|
762
|
-
module and the changes in our analytics.
|
763
|
-
|
764
|
-
## What's Changed
|
765
|
-
|
766
|
-
* Remove e2e example and point to templates by @avishniakov in https://github.com/zenml-io/zenml/pull/1752
|
767
|
-
* Add cloud architecture docs by @htahir1 in https://github.com/zenml-io/zenml/pull/1751
|
768
|
-
* Update docs/docstrings following `mlstacks` repo name change by @strickvl in https://github.com/zenml-io/zenml/pull/1754
|
769
|
-
* Update Cloud deployment scenarios by @stefannica in https://github.com/zenml-io/zenml/pull/1757
|
770
|
-
* Fixing the logging message regarding caching by @bcdurak in https://github.com/zenml-io/zenml/pull/1748
|
771
|
-
* Improvements to the step logs storage functionality by @bcdurak in https://github.com/zenml-io/zenml/pull/1733
|
772
|
-
* Fix `qemu`/`colima` Github Actions bug by @safoinme in https://github.com/zenml-io/zenml/pull/1760
|
773
|
-
* Bump `ruff` and `mypy` by @strickvl in https://github.com/zenml-io/zenml/pull/1762
|
774
|
-
* Add Template Testing in Core by @avishniakov in https://github.com/zenml-io/zenml/pull/1745
|
775
|
-
* Removing analytics v1 and optimizing v2 by @bcdurak in https://github.com/zenml-io/zenml/pull/1753
|
776
|
-
* Update publish script to take a token by @strickvl in https://github.com/zenml-io/zenml/pull/1758
|
777
|
-
* Update variable name for release publication token by @strickvl in https://github.com/zenml-io/zenml/pull/1764
|
778
|
-
* Lock `MYSQL` Database during DB migrations by @safoinme in https://github.com/zenml-io/zenml/pull/1763
|
779
|
-
* `mlstacks` integration (and deprecation of old deployment logic) by @strickvl in https://github.com/zenml-io/zenml/pull/1721
|
780
|
-
|
781
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.43.0...0.44.0
|
782
|
-
|
783
|
-
# 0.43.0
|
784
|
-
|
785
|
-
This release brings limited support for Python 3.11,
|
786
|
-
improves quickstart experience with the fully reworked flow,
|
787
|
-
enhances the user experience while dealing with ZenML docs,
|
788
|
-
offers new extended templates for projects,
|
789
|
-
and fixes GCP connector creation issue.
|
790
|
-
|
791
|
-
## Limited support for Python 3.11
|
792
|
-
This release adds limited support for Python 3.11.
|
793
|
-
The following integrations are currently not supported with Python 3.11:
|
794
|
-
- gcp
|
795
|
-
- kubeflow
|
796
|
-
- tekton
|
797
|
-
|
798
|
-
This is because:
|
799
|
-
- GCP packages that support Python 3.11 are not compatible with KFP 1
|
800
|
-
- Upgrade to KFP 2 is blocked by the fact that Tekton doesn't have any release compatible with KFP 2 yet (https://github.com/zenml-io/zenml/pull/1697)
|
801
|
-
|
802
|
-
## Breaking Changes
|
803
|
-
|
804
|
-
A minor breaking change in CLI for `zenml init`:
|
805
|
-
- previously supported flag `--starter`
|
806
|
-
- new flag `--template-with-defaults`
|
807
|
-
- behavior remains the same - flag is responsible for usage of default settings in the template
|
808
|
-
|
809
|
-
## What's Changed
|
810
|
-
* Disable implicit auth methods for service connectors by default by @stefannica in https://github.com/zenml-io/zenml/pull/1704
|
811
|
-
* New quickstart by @strickvl in https://github.com/zenml-io/zenml/pull/1692
|
812
|
-
* Set `MLflow` configuration as environment variables before deployment subprocess by @safoinme in https://github.com/zenml-io/zenml/pull/1705
|
813
|
-
* Fix Migration Guide Links by @fa9r in https://github.com/zenml-io/zenml/pull/1706
|
814
|
-
* Improve Input Validation Error Message by @fa9r in https://github.com/zenml-io/zenml/pull/1712
|
815
|
-
* Update link in cloudpickle_materializer.py by @duarteocarmo in https://github.com/zenml-io/zenml/pull/1713
|
816
|
-
* catch exceptions in `list_model_versions` by @avishniakov in https://github.com/zenml-io/zenml/pull/1703
|
817
|
-
* Rename `transition_model_stage` to `transition_model_version_stage` by @avishniakov in https://github.com/zenml-io/zenml/pull/1707
|
818
|
-
* pandas input to `predict` by @avishniakov in https://github.com/zenml-io/zenml/pull/1715
|
819
|
-
* Small fixes to global config docs page by @schustmi in https://github.com/zenml-io/zenml/pull/1714
|
820
|
-
* Allow specifying extra hosts for LocalDockerOrchestrator by @schustmi in https://github.com/zenml-io/zenml/pull/1709
|
821
|
-
* Flexible use of `ignore_cols` in `evidently_report_step` by @avishniakov in https://github.com/zenml-io/zenml/pull/1711
|
822
|
-
* Add external artifacts and direct links to run DAG by @fa9r in https://github.com/zenml-io/zenml/pull/1718
|
823
|
-
* E2E flow example for templates by @avishniakov in https://github.com/zenml-io/zenml/pull/1710
|
824
|
-
* Fix bug in service connector, Closes #1720 by @soubenz in https://github.com/zenml-io/zenml/pull/1726
|
825
|
-
* Document the namespace and service account k8s orchestrator settings by @stefannica in https://github.com/zenml-io/zenml/pull/1722
|
826
|
-
* Refactoring done and reduced some functions complexity and work-time by @thanseefpp in https://github.com/zenml-io/zenml/pull/1719
|
827
|
-
* Update custom orchestrator guide by @schustmi in https://github.com/zenml-io/zenml/pull/1728
|
828
|
-
* Improve error message when passing non-json serializable parameter by @schustmi in https://github.com/zenml-io/zenml/pull/1729
|
829
|
-
* Bump `ruff` to 0.0.282 by @strickvl in https://github.com/zenml-io/zenml/pull/1730
|
830
|
-
* Docs and README update for ZenML Cloud by @bcdurak in https://github.com/zenml-io/zenml/pull/1723
|
831
|
-
* bump `MLflow` to 2.5.0 by @safoinme in https://github.com/zenml-io/zenml/pull/1708
|
832
|
-
* Move Examples to Tests by @fa9r in https://github.com/zenml-io/zenml/pull/1673
|
833
|
-
* Add Error Handling for Empty Pipelines by @fa9r in https://github.com/zenml-io/zenml/pull/1734
|
834
|
-
* Revert "Add Error Handling for Empty Pipelines" by @fa9r in https://github.com/zenml-io/zenml/pull/1735
|
835
|
-
* Changing the links to the public roadmap by @bcdurak in https://github.com/zenml-io/zenml/pull/1737
|
836
|
-
* Add Error Handling for Empty Pipelines by @fa9r in https://github.com/zenml-io/zenml/pull/1736
|
837
|
-
* Revisit `init --template` CLI for new templates by @avishniakov in https://github.com/zenml-io/zenml/pull/1731
|
838
|
-
* Add Python 3.11 Support by @fa9r in https://github.com/zenml-io/zenml/pull/1702
|
839
|
-
* fix error on scheduled pipelines with KubernetesOrchestrator by @francoisserra in https://github.com/zenml-io/zenml/pull/1738
|
840
|
-
* Bugfix for identify calls with empty email strings by @bcdurak in https://github.com/zenml-io/zenml/pull/1739
|
841
|
-
|
842
|
-
## New Contributors
|
843
|
-
* @duarteocarmo made their first contribution in https://github.com/zenml-io/zenml/pull/1713
|
844
|
-
* @thanseefpp made their first contribution in https://github.com/zenml-io/zenml/pull/1719
|
845
|
-
|
846
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.42.0...0.43.0
|
847
|
-
|
848
|
-
# 0.42.1
|
849
|
-
|
850
|
-
This is a minor release that fixes a couple of minor issues and improves the
|
851
|
-
quickstart example.
|
852
|
-
|
853
|
-
## Breaking Changes
|
854
|
-
|
855
|
-
### Disable Implicit Auth Methods for Service Connectors by Default
|
856
|
-
|
857
|
-
The implicit authentication methods supported by cloud Service Connectors method
|
858
|
-
may constitute a security risk, because they can give users access to the same
|
859
|
-
cloud resources and services that the ZenML Server itself is allowed to access.
|
860
|
-
|
861
|
-
For this reason, the default behavior of ZenML Service Connectors has been
|
862
|
-
changed to disable implicit authentication methods by default. If you try to
|
863
|
-
configure any of the AWS, GCP or Azure Service Connectors using the implicit
|
864
|
-
authentication method, you will now receive an error message.
|
865
|
-
|
866
|
-
To enable implicit authentication methods, you have to set the
|
867
|
-
`ZENML_ENABLE_IMPLICIT_AUTH_METHODS` environment variable or the ZenML helm
|
868
|
-
chart `enableImplicitAuthMethods` configuration option to `true`.
|
869
|
-
|
870
|
-
## What's Changed
|
871
|
-
* Disable implicit auth methods for service connectors by default by @stefannica in https://github.com/zenml-io/zenml/pull/1704
|
872
|
-
* New quickstart by @strickvl in https://github.com/zenml-io/zenml/pull/1692
|
873
|
-
* Set `MLflow` configuration as environment variables before deployment subprocess by @safoinme in https://github.com/zenml-io/zenml/pull/1705
|
874
|
-
* Fix Migration Guide Links by @fa9r in https://github.com/zenml-io/zenml/pull/1706
|
875
|
-
|
876
|
-
|
877
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.42.0...0.42.1
|
878
|
-
|
879
|
-
# 0.42.0
|
880
|
-
|
881
|
-
This release brings major user experience improvements to how ZenML logs are
|
882
|
-
managed and displayed, removes Python 3.7 support, and fixes the Python 3.10
|
883
|
-
PyYAML issues caused by the Cython 3.0 release.
|
884
|
-
|
885
|
-
## Improved Logging UX
|
886
|
-
|
887
|
-
The log messages written by ZenML when running pipelines or executing ZenML CLI
|
888
|
-
commands are now more concise and easier to digest and the log message colors
|
889
|
-
were adjusted to be more intuitive. Additionally, all log messages, including
|
890
|
-
custom prints to stdout, now show up as step logs in the dashboard.
|
891
|
-
|
892
|
-
## Breaking Changes
|
893
|
-
|
894
|
-
### Python 3.7 Support Dropped
|
895
|
-
Python 3.7 reached its end of life on on June 27th, 2023. Since then, several
|
896
|
-
MLOps tools have stopped supporting Python 3.7. To prevent dependency issues
|
897
|
-
with our integrations and other open-source packages, ZenML will also no longer
|
898
|
-
support Python 3.7 starting from this release.
|
899
|
-
|
900
|
-
### Dependency and Integration Version Updates
|
901
|
-
ZenML now requires PyYAML 6 since older versions are broken under Python 3.10.
|
902
|
-
Subsequently, the following integrations now require a higher package version:
|
903
|
-
- Kubeflow now requires `kfp==1.8.22`
|
904
|
-
- Tekton now requires `kfk-tekton==1.7.1`
|
905
|
-
- Evidently now requires `evidently==0.2.7` or `evidently==0.2.8`
|
906
|
-
|
907
|
-
## What's Changed
|
908
|
-
* Add missing quote in docs by @schustmi in https://github.com/zenml-io/zenml/pull/1674
|
909
|
-
* Update Local Docker orchestrator docs by @strickvl in https://github.com/zenml-io/zenml/pull/1676
|
910
|
-
* Relax `fastapi` dependency version by @fa9r in https://github.com/zenml-io/zenml/pull/1675
|
911
|
-
* Improve flavor registration error message by @schustmi in https://github.com/zenml-io/zenml/pull/1671
|
912
|
-
* Simplified Page Iteration by @fa9r in https://github.com/zenml-io/zenml/pull/1679
|
913
|
-
* Document how to deploy ZenML with custom Docker image by @fa9r in https://github.com/zenml-io/zenml/pull/1672
|
914
|
-
* Document the ZenML Client and Models by @fa9r in https://github.com/zenml-io/zenml/pull/1678
|
915
|
-
* Add Label Studio text classification integration and example by @adamwawrzynski in https://github.com/zenml-io/zenml/pull/1658
|
916
|
-
* Improve yaml config docs page by @schustmi in https://github.com/zenml-io/zenml/pull/1680
|
917
|
-
* Catch correct exception when trying to access step context by @schustmi in https://github.com/zenml-io/zenml/pull/1681
|
918
|
-
* Add option to only export requirements for installed integrations by @schustmi in https://github.com/zenml-io/zenml/pull/1682
|
919
|
-
* Fix copy-paste error (Seldon / KServe docstring) by @strickvl in https://github.com/zenml-io/zenml/pull/1687
|
920
|
-
* Add avishniakov to `teams.yaml` by @avishniakov in https://github.com/zenml-io/zenml/pull/1688
|
921
|
-
* [NEW PR] Set contains_code to 1 instead of True by @kobiche in https://github.com/zenml-io/zenml/pull/1685
|
922
|
-
* Misc slack fixes by @schustmi in https://github.com/zenml-io/zenml/pull/1686
|
923
|
-
* Docs: Migration Guide by @fa9r in https://github.com/zenml-io/zenml/pull/1691
|
924
|
-
* fix: :card_file_box: Extend pipeline spec storage length by @francoisserra in https://github.com/zenml-io/zenml/pull/1694
|
925
|
-
* Make the workspace statistics endpoint more performant by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1689
|
926
|
-
* Deprecate examples CLI by @avishniakov in https://github.com/zenml-io/zenml/pull/1693
|
927
|
-
* Add cloud server deployment type by @schustmi in https://github.com/zenml-io/zenml/pull/1699
|
928
|
-
* Fix Python 3.10 PyYAML Installation Issues by @fa9r in https://github.com/zenml-io/zenml/pull/1695
|
929
|
-
* Remove Python 3.7 Support by @fa9r in https://github.com/zenml-io/zenml/pull/1652
|
930
|
-
* Improved logs for pipeline execution and CLI usage by @bcdurak in https://github.com/zenml-io/zenml/pull/1664
|
931
|
-
* Docs: Restructure Advanced Guide by @fa9r in https://github.com/zenml-io/zenml/pull/1698
|
932
|
-
|
933
|
-
## New Contributors
|
934
|
-
* @adamwawrzynski made their first contribution in https://github.com/zenml-io/zenml/pull/1658
|
935
|
-
* @avishniakov made their first contribution in https://github.com/zenml-io/zenml/pull/1688
|
936
|
-
* @kobiche made their first contribution in https://github.com/zenml-io/zenml/pull/1685
|
937
|
-
|
938
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.41.0...0.42.0
|
939
|
-
|
940
|
-
|
941
|
-
# 0.41.0
|
942
|
-
|
943
|
-
ZenML release 0.41.0 comes with a second round of updates to the pipeline and
|
944
|
-
step interface with major changes in how step outputs are defined, how
|
945
|
-
information about previous runs can be fetched programmatically, and how
|
946
|
-
information about the current run can be obtained.
|
947
|
-
|
948
|
-
See [this docs page](https://docs.zenml.io/user-guide/migration-guide/migration-zero-forty)
|
949
|
-
for an overview of all pipeline interface changes introduced since release
|
950
|
-
0.40.0 and for more information on how to migrate your existing ZenML pipelines
|
951
|
-
to the latest syntax.
|
952
|
-
|
953
|
-
## Fetching Runs Programmatically (#1635)
|
954
|
-
The entire syntax of fetching previous runs programmatically was majorly
|
955
|
-
redesigned. While the overall user flow is still almost identical, the new
|
956
|
-
approach does not contain pipeline-versioning-related inconsistencies, has a
|
957
|
-
more intuitive syntax, and is also easier for users to learn since the new
|
958
|
-
syntax uses the ZenML Client and response models natively instead of requiring
|
959
|
-
the `zenml.post_execution` util functions and corresponding `...View` wrapper
|
960
|
-
classes.
|
961
|
-
|
962
|
-
## Accessing Current Run Information (#1648)
|
963
|
-
How to fetch information about the current pipeline run from within the run has
|
964
|
-
been majorly redesigned:
|
965
|
-
- Instead of being an argument of the step function, the `StepContext` is now a
|
966
|
-
singleton that can be accessed via the new `zenml.get_step_context()` function.
|
967
|
-
- The `StepContext` is now decoupled from the `StepEnvironment` and the
|
968
|
-
`StepEnvironment` is deprecated.
|
969
|
-
- The `StepContext` now contains the full `PipelineRunResponseModel` and
|
970
|
-
`StepRunResponseModel` so all information about the run is accessible, not
|
971
|
-
just the name / id / params.
|
972
|
-
|
973
|
-
## Defining Step Outputs (#1653)
|
974
|
-
Instead of using the `zenml.steps.Output` class to annotate steps with multiple
|
975
|
-
outputs, ZenML can now handle `Tuple` annotations natively and output names can
|
976
|
-
now be assigned to any step output using `typing_extensions.Annotated`.
|
977
|
-
|
978
|
-
## What's Changed
|
979
|
-
* Remove remaining BaseParameters references by @schustmi in https://github.com/zenml-io/zenml/pull/1625
|
980
|
-
* Fix the s3 integration dependencies by @stefannica in https://github.com/zenml-io/zenml/pull/1641
|
981
|
-
* Don't run whylogs example on windows by @stefannica in https://github.com/zenml-io/zenml/pull/1644
|
982
|
-
* Adding the missing pages to our docs by @bcdurak in https://github.com/zenml-io/zenml/pull/1640
|
983
|
-
* Connectors startup guide and stack component references by @stefannica in https://github.com/zenml-io/zenml/pull/1632
|
984
|
-
* Fixing the listing functionality of several objects in our CLI by @bcdurak in https://github.com/zenml-io/zenml/pull/1616
|
985
|
-
* Revamp Post Execution by @fa9r in https://github.com/zenml-io/zenml/pull/1635
|
986
|
-
* Fix run configuration parameter merging by @schustmi in https://github.com/zenml-io/zenml/pull/1638
|
987
|
-
* Simplify email opt-in telemetry by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1637
|
988
|
-
* Fix Step Logs on Windows by @fa9r in https://github.com/zenml-io/zenml/pull/1645
|
989
|
-
* Improve config section of containerization docs page by @schustmi in https://github.com/zenml-io/zenml/pull/1649
|
990
|
-
* Validating slack alerter by @bhatt-priyadutt in https://github.com/zenml-io/zenml/pull/1609
|
991
|
-
* Added some error handling in gcp cloud function scheduling by @htahir1 in https://github.com/zenml-io/zenml/pull/1634
|
992
|
-
* CI: Disable Python 3.7 Mac Runners by @fa9r in https://github.com/zenml-io/zenml/pull/1650
|
993
|
-
* Redesign `StepContext` by @fa9r in https://github.com/zenml-io/zenml/pull/1648
|
994
|
-
* Fix output of dashboard url on pipeline run by @strickvl in https://github.com/zenml-io/zenml/pull/1629
|
995
|
-
* fix: use k8s orchestrator service account in step pod's manifest by @francoisserra in https://github.com/zenml-io/zenml/pull/1654
|
996
|
-
* Fix Image Builder Warning Message by @fa9r in https://github.com/zenml-io/zenml/pull/1659
|
997
|
-
* New step output annotations by @schustmi in https://github.com/zenml-io/zenml/pull/1653
|
998
|
-
* Add Python 3.10 to listed versions supported via PyPi by @strickvl in https://github.com/zenml-io/zenml/pull/1662
|
999
|
-
* Add DatabricksShell on list of notebooks allowed to show dashboard by @lucasbissaro in https://github.com/zenml-io/zenml/pull/1643
|
1000
|
-
* Fixing broken links in our examples folder by @bcdurak in https://github.com/zenml-io/zenml/pull/1661
|
1001
|
-
* Feature/frw 2013 docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1639
|
1002
|
-
* Update Pipeline Migration Page by @fa9r in https://github.com/zenml-io/zenml/pull/1667
|
1003
|
-
* Fix/set env variables before installing packages by @lopezco in https://github.com/zenml-io/zenml/pull/1665
|
1004
|
-
* Fix the `zenml deploy` story by @wjayesh in https://github.com/zenml-io/zenml/pull/1651
|
1005
|
-
* Always keep link to API docs pointed at the version of the release branch by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1636
|
1006
|
-
* Fix BentoML deployer by @safoinme in https://github.com/zenml-io/zenml/pull/1647
|
1007
|
-
* Corrected all mentions in docs from API docs to SDK docs. by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1669
|
1008
|
-
* Update outdated docs by @schustmi in https://github.com/zenml-io/zenml/pull/1668
|
1009
|
-
|
1010
|
-
## New Contributors
|
1011
|
-
* @lucasbissaro made their first contribution in https://github.com/zenml-io/zenml/pull/1643
|
1012
|
-
* @lopezco made their first contribution in https://github.com/zenml-io/zenml/pull/1665
|
1013
|
-
|
1014
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.40.3...0.41.0
|
1015
|
-
|
1016
|
-
# 0.40.3
|
1017
|
-
|
1018
|
-
This is a minor ZenML release that introduces a couple of new features:
|
1019
|
-
|
1020
|
-
* the [Azure Service Connector](https://docs.zenml.io/stacks-and-components/auth-management/azure-service-connector) is now available in addition to the AWS and GCP ones. It can be used to connect ZenML and Stack Components to Azure cloud infrastructure resources like Azure Blob Storage, Azure Container Registry and Azure Kubernetes Service.
|
1021
|
-
* Service Connectors can now also be managed through the ZenML Dashboard
|
1022
|
-
* adds `zenml secret export` CLI command to export secrets from the ZenML Secret Store to a local file
|
1023
|
-
* adds the ability to create/update ZenML secrets from JSON/YAML files or command line arguments (courtesy of @bhatt-priyadutt)
|
1024
|
-
|
1025
|
-
In addition to that, this release also contains a couple of bug fixes and improvements, including:
|
1026
|
-
|
1027
|
-
* better documentation and fixes for the ZenML [Vertex AI Orchestrator](https://docs.zenml.io/stacks-and-components/component-guide/orchestrators/vertex) and [Vertex AI Step Operator](https://docs.zenml.io/stacks-and-components/component-guide/step-operators/vertex)
|
1028
|
-
* adjust Seldon and BentoML Steps and Examples to new pipeline interface
|
1029
|
-
|
1030
|
-
## What's Changed
|
1031
|
-
* Add option to list all resources when verifying service connector config by @stefannica in https://github.com/zenml-io/zenml/pull/1573
|
1032
|
-
* Fix sandbox time limit by @schustmi in https://github.com/zenml-io/zenml/pull/1602
|
1033
|
-
* Secrets input structure change method by @bhatt-priyadutt in https://github.com/zenml-io/zenml/pull/1547
|
1034
|
-
* Implement Azure service connector by @stefannica in https://github.com/zenml-io/zenml/pull/1589
|
1035
|
-
* Adding the ability to tag the source of an event for the analytics by @bcdurak in https://github.com/zenml-io/zenml/pull/1599
|
1036
|
-
* Move all the logic into the script to make it as easy as possible to … by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1605
|
1037
|
-
* Only set mysql session variables when necessary by @schustmi in https://github.com/zenml-io/zenml/pull/1568
|
1038
|
-
* Bug in creating upstream_steps by @sidsaurb in https://github.com/zenml-io/zenml/pull/1601
|
1039
|
-
* Added logs endpoint to display on the dashboard by @htahir1 in https://github.com/zenml-io/zenml/pull/1526
|
1040
|
-
* Fix CI by @fa9r in https://github.com/zenml-io/zenml/pull/1612
|
1041
|
-
* Fix Azure Integration Imports and Improve Flavor Registration Error Handling by @fa9r in https://github.com/zenml-io/zenml/pull/1615
|
1042
|
-
* Deprecation Cleanup by @fa9r in https://github.com/zenml-io/zenml/pull/1608
|
1043
|
-
* Cleanup Local Logging Temp Files by @fa9r in https://github.com/zenml-io/zenml/pull/1621
|
1044
|
-
* Add cloud orchestrator warning message by @strickvl in https://github.com/zenml-io/zenml/pull/1418
|
1045
|
-
* Update custom code run in sandbox docs by @safoinme in https://github.com/zenml-io/zenml/pull/1610
|
1046
|
-
* Remove the GH Actions review reminder bot by @strickvl in https://github.com/zenml-io/zenml/pull/1624
|
1047
|
-
* Automatically optimize image sizes on PR creation by @strickvl in https://github.com/zenml-io/zenml/pull/1626
|
1048
|
-
* Deprecation Warning Improvements by @fa9r in https://github.com/zenml-io/zenml/pull/1620
|
1049
|
-
* Fix ZenML Installation when FastAPI is not Installed by @fa9r in https://github.com/zenml-io/zenml/pull/1627
|
1050
|
-
* Fix unnecessary / extra deprecation warnings by @strickvl in https://github.com/zenml-io/zenml/pull/1630
|
1051
|
-
* Add `zenml secret export` CLI command by @fa9r in https://github.com/zenml-io/zenml/pull/1607
|
1052
|
-
* Missing pipeline features docs by @schustmi in https://github.com/zenml-io/zenml/pull/1619
|
1053
|
-
* Fix for valid secret name by @bhatt-priyadutt in https://github.com/zenml-io/zenml/pull/1617
|
1054
|
-
* Fix and document Vertex AI orchestrator and step operator by @stefannica in https://github.com/zenml-io/zenml/pull/1606
|
1055
|
-
* Deprecate KServe Integration by @fa9r in https://github.com/zenml-io/zenml/pull/1631
|
1056
|
-
* Adjust Seldon Steps and Examples to New Pipeline Interface by @fa9r in https://github.com/zenml-io/zenml/pull/1560
|
1057
|
-
* Adjust BentoML Steps and Example to New Pipeline Interface by @fa9r in https://github.com/zenml-io/zenml/pull/1614
|
1058
|
-
* Moved kubernetes imports to inner function to avoid module not found error by @htahir1 in https://github.com/zenml-io/zenml/pull/1622
|
1059
|
-
|
1060
|
-
## New Contributors
|
1061
|
-
* @sidsaurb made their first contribution in https://github.com/zenml-io/zenml/pull/1601
|
1062
|
-
|
1063
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.40.2...test
|
1064
|
-
|
1065
|
-
# 0.40.2
|
1066
|
-
|
1067
|
-
Documentation and example updates.
|
1068
|
-
|
1069
|
-
## What's Changed
|
1070
|
-
* Update Example for sandbox by @safoinme in https://github.com/zenml-io/zenml/pull/1576
|
1071
|
-
* Document `zenml show` by @fa9r in https://github.com/zenml-io/zenml/pull/1570
|
1072
|
-
* Clean up for the new docs by @bcdurak in https://github.com/zenml-io/zenml/pull/1575
|
1073
|
-
* Add orchestrator outputs for sandbox examples by @strickvl in https://github.com/zenml-io/zenml/pull/1579
|
1074
|
-
* Docs: Added some adjustments to the code repository page. by @bcdurak in https://github.com/zenml-io/zenml/pull/1582
|
1075
|
-
* Sandbox documentation (and other docs updates) by @strickvl in https://github.com/zenml-io/zenml/pull/1574
|
1076
|
-
* Minor README update regarding the sandbox. by @bcdurak in https://github.com/zenml-io/zenml/pull/1586
|
1077
|
-
* Fix failing `mlflow_tracking` example test by @strickvl in https://github.com/zenml-io/zenml/pull/1581
|
1078
|
-
* Bump `ruff` and `mypy` by @strickvl in https://github.com/zenml-io/zenml/pull/1590
|
1079
|
-
* Remove `config.yaml` references in example docs by @strickvl in https://github.com/zenml-io/zenml/pull/1585
|
1080
|
-
* update mlflow tracking example and reduce number of epochs by @safoinme in https://github.com/zenml-io/zenml/pull/1598
|
1081
|
-
* Improve error message when requirements file does not exist by @schustmi in https://github.com/zenml-io/zenml/pull/1596
|
1082
|
-
* Fix build reuse for integrations with apt packages by @schustmi in https://github.com/zenml-io/zenml/pull/1594
|
1083
|
-
* make the `Github` repo token optional by @safoinme in https://github.com/zenml-io/zenml/pull/1593
|
1084
|
-
|
1085
|
-
|
1086
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.40.1...0.40.2
|
1087
|
-
|
1088
|
-
# 0.40.1
|
1089
|
-
|
1090
|
-
Small bug and docs fixes following the 0.40.0 release.
|
1091
|
-
|
1092
|
-
## What's Changed
|
1093
|
-
* Convert dict to tuple in ArtifactConfiguration validator by @schustmi in https://github.com/zenml-io/zenml/pull/1571
|
1094
|
-
* Docs cleanup by @schustmi in https://github.com/zenml-io/zenml/pull/1569
|
1095
|
-
* Fix `boto3<=1.24.59` by @safoinme in https://github.com/zenml-io/zenml/pull/1572
|
1096
|
-
|
1097
|
-
|
1098
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.40.0...0.40.1
|
1099
|
-
|
1100
|
-
# 0.40.0
|
1101
|
-
|
1102
|
-
ZenML release 0.40.0 introduces two big updates: a fresh and more flexible pipeline interface and a new way to connect and authenticate with external services in ZenML Connectors. See below for full details on these two major new sets of functionality.
|
1103
|
-
|
1104
|
-
The release also contains many bug fixes and quality-of-life improvements. Specifically, we reworked our documentation from the ground up with particular attention to the structure to help you find what you need. Our [Label Studio integration example](https://github.com/zenml-io/zenml/tree/main/examples/label_studio_annotation) is now working again and allows you to use more recent versions of the `label-studio` package that backs it.
|
1105
|
-
|
1106
|
-
## A Fresh Pipeline Interface
|
1107
|
-
|
1108
|
-
This release introduces a completely reworked interface for developing your ZenML steps and pipelines:
|
1109
|
-
|
1110
|
-
* Increased flexibility when defining steps: Steps can now have `Optional`, `Union`, and `Any` type annotations for their inputs and outputs. Additionally, default values are allowed for step inputs.
|
1111
|
-
|
1112
|
-
```python
|
1113
|
-
@step
|
1114
|
-
def trainer(data: pd.Dataframe, start_model: Union[svm.SVC, svm.SVR], coef0: Optional[int] = None) -> Any:
|
1115
|
-
pass
|
1116
|
-
```
|
1117
|
-
|
1118
|
-
* You can now easily run a step outside of a pipeline, making it easier to test and debug your code:
|
1119
|
-
|
1120
|
-
```python
|
1121
|
-
trainer(data=pd.Dataframe(...), start_model=svc.SVC(...))
|
1122
|
-
```
|
1123
|
-
|
1124
|
-
* External artifacts can be used to pass values to steps that are not produced by an upstream step. This provides more flexibility when working with external data or models:
|
1125
|
-
|
1126
|
-
```python
|
1127
|
-
from zenml.steps.external_artifact import ExternalArtifact
|
1128
|
-
|
1129
|
-
@pipeline
|
1130
|
-
def my_pipeline(lr: float):
|
1131
|
-
data = process_data()
|
1132
|
-
trainer(data=data, start_model=ExternalArtifact(svc.SVC(...)))
|
1133
|
-
```
|
1134
|
-
|
1135
|
-
* You can now call steps multiple times inside a pipeline, allowing you to create more complex workflows and reuse steps with different parameters:
|
1136
|
-
|
1137
|
-
```python
|
1138
|
-
@pipeline
|
1139
|
-
def my_pipeline(step_count: int) -> None:
|
1140
|
-
data = load_data_step()
|
1141
|
-
after = []
|
1142
|
-
for i in range(step_count):
|
1143
|
-
train_step(data, learning_rate=i * 0.0001, id=f"train_step_{i}")
|
1144
|
-
after.append(f"train_step_{i}")
|
1145
|
-
model = select_model_step(..., after=after)
|
1146
|
-
```
|
1147
|
-
|
1148
|
-
* Pipelines can now define inputs and outputs, providing a clearer interface for working with data and dependencies between pipelines:
|
1149
|
-
|
1150
|
-
```python
|
1151
|
-
@pipeline(enable_cache=False)
|
1152
|
-
def subpipeline(pipeline_param: int):
|
1153
|
-
out = step_1(k=None)
|
1154
|
-
step_2(a=3, b=pipeline_param)
|
1155
|
-
return 17
|
1156
|
-
```
|
1157
|
-
|
1158
|
-
* You can now call pipelines within other pipelines. This currently does not execute the inner pipeline but instead adds its steps to the parent pipeline, allowing you to create modular and reusable workflows:
|
1159
|
-
|
1160
|
-
```python
|
1161
|
-
@pipeline(enable_cache=False)
|
1162
|
-
def my_pipeline(a: int = 1):
|
1163
|
-
p1_output = subpipeline(pipeline_param=22)
|
1164
|
-
step_2(a=a, b=p1_output)
|
1165
|
-
```
|
1166
|
-
|
1167
|
-
To get started, simply import the new `@step` and `@pipeline` decorator and check out our new [starter guide](https://docs.zenml.io/user-guide/starter-guide) for more information.
|
1168
|
-
|
1169
|
-
```python
|
1170
|
-
from zenml import step, pipeline
|
1171
|
-
|
1172
|
-
@step
|
1173
|
-
def my_step(...):
|
1174
|
-
...
|
1175
|
-
|
1176
|
-
@pipeline
|
1177
|
-
def my_pipeline(...):
|
1178
|
-
...
|
1179
|
-
```
|
1180
|
-
|
1181
|
-
The old pipeline and step interface is still working using the imports from previous ZenML releases but is deprecated and will be removed in the future.
|
1182
|
-
|
1183
|
-
## 'Connectors' for authentication
|
1184
|
-
|
1185
|
-
In this update, we're pleased to present a new feature to ZenML: Service Connectors. The intention behind these connectors is to offer a reliable and more user-friendly method for integrating ZenML with external resources and services. We aim to simplify processes such as validating, storing, and generating security-sensitive data, along with the authentication and authorization of access to external services. We believe ZenML Service Connectors will be a useful tool to alleviate some of the common challenges in managing pipeline across various Stack Components.
|
1186
|
-
|
1187
|
-
Regardless of your background in infrastructure management - whether you're a beginner looking for quick cloud stack integration, or an experienced engineer focused on maintaining robust infrastructure security practices - our Service Connectors are designed to assist your work while maintaining high security standards.
|
1188
|
-
|
1189
|
-
Here are just a few ways you could use ZenML Service Connectors:
|
1190
|
-
|
1191
|
-
- Easy utilization of cloud resources: With ZenML's Service Connectors, you can use resources from AWS, GCP, and Azure without the need for extensive knowledge of cloud infrastructure or environment configuration. All you'll need is a ZenML Service Connector and a few Python libraries.
|
1192
|
-
- Assisted setup with security in mind: Our Service Connectors come with features for configuration validation and verification, the generation of temporary, low-privilege credentials, and pre-authenticated and pre-configured clients for Python libraries.
|
1193
|
-
- Easy local configuration transfer: ZenML's Service Connectors aim to resolve the reproducibility issue in ML pipelines. They do this by automatically transferring authentication configurations and credentials from your local machine, storing them securely, and allowing for effortless sharing across different environments.
|
1194
|
-
|
1195
|
-
[Visit our documentation pages](https://docs.zenml.io/stacks-and-components/auth-management) to learn more about ZenML Connectors and how you can use them in a way that supports your ML workflows.
|
1196
|
-
|
1197
|
-
## What's Changed
|
1198
|
-
|
1199
|
-
* Cleanup remaining references of `zenml.artifacts` by @fa9r in https://github.com/zenml-io/zenml/pull/1534
|
1200
|
-
* Upgrading the `black` version by @bcdurak in https://github.com/zenml-io/zenml/pull/1535
|
1201
|
-
* Remove dev breakpoints by @strickvl in https://github.com/zenml-io/zenml/pull/1540
|
1202
|
-
* Removing old option command from contribution doc by @bhatt-priyadutt in https://github.com/zenml-io/zenml/pull/1544
|
1203
|
-
* Improve CLI help text for `zenml integration install -i ...` by @strickvl in https://github.com/zenml-io/zenml/pull/1545
|
1204
|
-
* Fix RestZenStore error handling for list responses by @fa9r in https://github.com/zenml-io/zenml/pull/1539
|
1205
|
-
* Simplify Dashboard UX via `zenml.show()` by @fa9r in https://github.com/zenml-io/zenml/pull/1511
|
1206
|
-
* Removed hardcoded variable by @bhatt-priyadutt in https://github.com/zenml-io/zenml/pull/1543
|
1207
|
-
* Revert Quickstart Changes by @fa9r in https://github.com/zenml-io/zenml/pull/1546
|
1208
|
-
* Deprecate some long overdue functions by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1541
|
1209
|
-
* ZenML Connectors by @stefannica in https://github.com/zenml-io/zenml/pull/1514
|
1210
|
-
* Fix automatic dashboard opening after `zenml up` by @fa9r in https://github.com/zenml-io/zenml/pull/1551
|
1211
|
-
* Update Neptune README by @strickvl in https://github.com/zenml-io/zenml/pull/1554
|
1212
|
-
* Update example READMEs following deployment PR by @strickvl in https://github.com/zenml-io/zenml/pull/1555
|
1213
|
-
* Fix and update Label Studio example by @strickvl in https://github.com/zenml-io/zenml/pull/1542
|
1214
|
-
* Fix linter errors by @stefannica in https://github.com/zenml-io/zenml/pull/1557
|
1215
|
-
* Add Vertex as orchestrator and step operator to deploy CLI by @wjayesh in https://github.com/zenml-io/zenml/pull/1559
|
1216
|
-
* Fix dashboard secret references by @stefannica in https://github.com/zenml-io/zenml/pull/1561
|
1217
|
-
* New pipeline and step interface by @schustmi in https://github.com/zenml-io/zenml/pull/1466
|
1218
|
-
* Major Documentation Rehaul by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1562
|
1219
|
-
* Easy CSV Visualization by @fa9r in https://github.com/zenml-io/zenml/pull/1556
|
1220
|
-
|
1221
|
-
## New Contributors
|
1222
|
-
* @bhatt-priyadutt made their first contribution in https://github.com/zenml-io/zenml/pull/1544
|
1223
|
-
|
1224
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.39.1...0.40
|
1225
|
-
|
1226
|
-
# 0.39.1
|
1227
|
-
|
1228
|
-
Minor hotfix release for running ZenML in Google Colab environments.
|
1229
|
-
|
1230
|
-
## What's Changed
|
1231
|
-
* Fix Source Resolving in Colab by @fa9r in https://github.com/zenml-io/zenml/pull/1530
|
1232
|
-
|
1233
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.39.0...0.39.1
|
1234
|
-
|
1235
|
-
# 0.39.0
|
1236
|
-
|
1237
|
-
ZenML release 0.39.0 introduces several big new features:
|
1238
|
-
- The `zenml stack recipe` CLI commands now support fine-grained handling of
|
1239
|
-
individual stack components.
|
1240
|
-
- Artifacts are now automatically visualized in the dashboard.
|
1241
|
-
- Materializers received an overhaul: a new `cloudpickle` default materializer
|
1242
|
-
was added that works for arbitrary objects, and a `pycaret` materializer
|
1243
|
-
that can handle various modeling frameworks in a unified format.
|
1244
|
-
|
1245
|
-
The release also contains many bug fixes and quality-of-life improvements, such
|
1246
|
-
as new settings options for the SageMaker and Kubernetes orchestrators.
|
1247
|
-
|
1248
|
-
## Individual Stack Component Deployment
|
1249
|
-
|
1250
|
-
In this release, we've enhanced the ZenML stack recipe CLI to support
|
1251
|
-
conditional deployment, destruction, and configuration of individual stack
|
1252
|
-
components. Users can now quickly deploy and destroy components with options for
|
1253
|
-
each flavor, and pass a config file for custom variables. The new `output` CLI
|
1254
|
-
command allows users to retrieve outputs from their recipes. Overall, this
|
1255
|
-
update streamlines deploying and managing stack components by providing a more
|
1256
|
-
efficient and user-friendly experience.
|
1257
|
-
|
1258
|
-
## Artifact Visualization
|
1259
|
-
|
1260
|
-
Artifact visualizations are now automatically extracted by ZenML and embedded in
|
1261
|
-
the ZenML dashboard. Visualizations can now be defined by overriding the
|
1262
|
-
`save_visualizations` method of the materializer that handles an artifact.
|
1263
|
-
These visualizations are then automatically shown in the dashboard and can also
|
1264
|
-
be displayed in Jupyter notebooks using the new `visualize` post-execution
|
1265
|
-
method.
|
1266
|
-
|
1267
|
-
## Default Cloudpickle Materializer
|
1268
|
-
|
1269
|
-
ZenML now uses `cloudpickle` under the hood to save/load artifacts that other
|
1270
|
-
materializers cannot handle. This makes it even easier to get started with
|
1271
|
-
ZenML since you no longer need to define custom materializers if you just
|
1272
|
-
want to experiment with some new data types.
|
1273
|
-
|
1274
|
-
## What's Changed
|
1275
|
-
* Docs/zenml hub documentation by @bcdurak in https://github.com/zenml-io/zenml/pull/1490
|
1276
|
-
* Sort integration list before display by @strickvl in https://github.com/zenml-io/zenml/pull/1494
|
1277
|
-
* Update docs to surface CLI filtering syntax by @strickvl in https://github.com/zenml-io/zenml/pull/1496
|
1278
|
-
* ZenML Hub Tests & CLI Improvements by @fa9r in https://github.com/zenml-io/zenml/pull/1495
|
1279
|
-
* Delete Legacy Docs by @fa9r in https://github.com/zenml-io/zenml/pull/1497
|
1280
|
-
* Improve the REST API error handling by @stefannica in https://github.com/zenml-io/zenml/pull/1451
|
1281
|
-
* Fix circular import of PipelineRunConfiguration by @schustmi in https://github.com/zenml-io/zenml/pull/1501
|
1282
|
-
* Delete Deprecated Artifacts and Materializer Code by @fa9r in https://github.com/zenml-io/zenml/pull/1498
|
1283
|
-
* Allow filtering runs by code repo id by @schustmi in https://github.com/zenml-io/zenml/pull/1499
|
1284
|
-
* Add example to docs for passing stack component specific settings by @christianversloot in https://github.com/zenml-io/zenml/pull/1506
|
1285
|
-
* Increase step run field lengths by @schustmi in https://github.com/zenml-io/zenml/pull/1503
|
1286
|
-
* Fix Sagemaker orchestrator pipeline name bug by @strickvl in https://github.com/zenml-io/zenml/pull/1508
|
1287
|
-
* Generate unique SageMaker training job name based on pipeline and ste… by @christianversloot in https://github.com/zenml-io/zenml/pull/1505
|
1288
|
-
* [CI Fix] Pin Llama Index Version by @fa9r in https://github.com/zenml-io/zenml/pull/1516
|
1289
|
-
* Basic PyCaret integration and materializer by @christianversloot in https://github.com/zenml-io/zenml/pull/1512
|
1290
|
-
* Specify line endings for different operating systems by @strickvl in https://github.com/zenml-io/zenml/pull/1513
|
1291
|
-
* Extend SageMakerOrchestratorSettings with processor_args enabling step level configuration by @christianversloot in https://github.com/zenml-io/zenml/pull/1509
|
1292
|
-
* Fix post execution `get_pipeline()` and `pipeline.get_runs()` by @fa9r in https://github.com/zenml-io/zenml/pull/1510
|
1293
|
-
* Default `cloudpickle` Materializer & Materializer Inheritance by @fa9r in https://github.com/zenml-io/zenml/pull/1507
|
1294
|
-
* Artifact Visualization by @fa9r in https://github.com/zenml-io/zenml/pull/1472
|
1295
|
-
* Add Kubernetes Orchestrator Settings by @fa9r in https://github.com/zenml-io/zenml/pull/1518
|
1296
|
-
* Bump `ruff` to 0.0.265 by @strickvl in https://github.com/zenml-io/zenml/pull/1520
|
1297
|
-
* feat: Set cloud function service account to the one defined in Vertex… by @francoisserra in https://github.com/zenml-io/zenml/pull/1519
|
1298
|
-
* Fix Kubernetes Orchestrator Config Loading by @fa9r in https://github.com/zenml-io/zenml/pull/1523
|
1299
|
-
* Resolve path during module resolving by @schustmi in https://github.com/zenml-io/zenml/pull/1521
|
1300
|
-
* Fix `SO_REUSEPORT` issue by @fa9r in https://github.com/zenml-io/zenml/pull/1524
|
1301
|
-
* Add individual stack component deployment through recipes by @wjayesh in https://github.com/zenml-io/zenml/pull/1328
|
1302
|
-
* Raise 501 for Unauthenticated Artifact Stores by @fa9r in https://github.com/zenml-io/zenml/pull/1522
|
1303
|
-
* Fix Duplicate Step Error by @fa9r in https://github.com/zenml-io/zenml/pull/1527
|
1304
|
-
* Fix pulling of stack recipes on `zenml init` by @wjayesh in https://github.com/zenml-io/zenml/pull/1528
|
1305
|
-
* Store dockerfile and requirements for builds by @schustmi in https://github.com/zenml-io/zenml/pull/1525
|
1306
|
-
|
1307
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.38.0...0.39.0
|
1308
|
-
|
1309
|
-
# 0.38.0
|
1310
|
-
|
1311
|
-
The 0.38.0 ZenML release is a major milestone for the ZenML project. It marks
|
1312
|
-
the introduction of the ZenML Hub, a central platform that enables our users to
|
1313
|
-
search, share and discover community-contributed code, such as stack component
|
1314
|
-
flavors, materializers, and pipeline steps. The ZenML Hub allows our users to
|
1315
|
-
extend their ZenML experience by leveraging the community's diverse range of
|
1316
|
-
implementations and MLOps best practices.
|
1317
|
-
|
1318
|
-
If you're interested in learning more about our motivation for implementing the
|
1319
|
-
ZenML Hub and our plans for its future, we invite you to read
|
1320
|
-
[our new blog post](https://blog.zenml.io/zenml-hub-launch). In addition to
|
1321
|
-
this technical documentation, the blog post provides a comprehensive overview
|
1322
|
-
of the ZenML Hub's goals and objectives, as well as the features that
|
1323
|
-
we plan to introduce in the future.
|
1324
|
-
|
1325
|
-
Aside from this major new feature, the release also includes a number of small
|
1326
|
-
improvements and bug fixes.
|
1327
|
-
|
1328
|
-
## What's Changed
|
1329
|
-
* Fix broken ENV variable by @strickvl in https://github.com/zenml-io/zenml/pull/1458
|
1330
|
-
* fix screenshot size in code repo by @safoinme in https://github.com/zenml-io/zenml/pull/1467
|
1331
|
-
* Fix CI (Deepchecks integration tests) by @fa9r in https://github.com/zenml-io/zenml/pull/1470
|
1332
|
-
* chore: update teams.yml by @Cahllagerfeld in https://github.com/zenml-io/zenml/pull/1459
|
1333
|
-
* Fix `BuiltInContainerMaterializer` for subtypes and non-built-in types by @fa9r in https://github.com/zenml-io/zenml/pull/1464
|
1334
|
-
* Kubernetes Orchestrator Improvements by @fa9r in https://github.com/zenml-io/zenml/pull/1460
|
1335
|
-
* Fix flaky CLI tests by @schustmi in https://github.com/zenml-io/zenml/pull/1465
|
1336
|
-
* Fix circular import during type checking by @schustmi in https://github.com/zenml-io/zenml/pull/1463
|
1337
|
-
* Allow secret values replacement in REST API PUT by @stefannica in https://github.com/zenml-io/zenml/pull/1471
|
1338
|
-
* Fix two steps race condition by @safoinme in https://github.com/zenml-io/zenml/pull/1473
|
1339
|
-
* Downgrading ZenML Version in global config by @safoinme in https://github.com/zenml-io/zenml/pull/1474
|
1340
|
-
* Revert "Downgrading ZenML Version in global config" by @safoinme in https://github.com/zenml-io/zenml/pull/1476
|
1341
|
-
* Add metadata to stack components by @wjayesh in https://github.com/zenml-io/zenml/pull/1416
|
1342
|
-
* remove modules from the list output for stack recipes by @wjayesh in https://github.com/zenml-io/zenml/pull/1480
|
1343
|
-
* Pin `openai` integration to `>0.27.0` by @strickvl in https://github.com/zenml-io/zenml/pull/1461
|
1344
|
-
* Apply formatting fixes to `/scripts` by @strickvl in https://github.com/zenml-io/zenml/pull/1462
|
1345
|
-
* Move import outside of type checking by @schustmi in https://github.com/zenml-io/zenml/pull/1482
|
1346
|
-
* Delete extra word from `bentoml` docs by @strickvl in https://github.com/zenml-io/zenml/pull/1484
|
1347
|
-
* Remove top-level config from recommended repo structure by @schustmi in https://github.com/zenml-io/zenml/pull/1485
|
1348
|
-
* Bump `mypy` and `ruff` by @strickvl in https://github.com/zenml-io/zenml/pull/1481
|
1349
|
-
* ZenML Version Downgrade - Silence Warnning by @safoinme in https://github.com/zenml-io/zenml/pull/1477
|
1350
|
-
* Update ZenServer recipes to include secret stores by @wjayesh in https://github.com/zenml-io/zenml/pull/1483
|
1351
|
-
* Fix alembic order by @schustmi in https://github.com/zenml-io/zenml/pull/1487
|
1352
|
-
* Fix source resolving for classes in notebooks by @schustmi in https://github.com/zenml-io/zenml/pull/1486
|
1353
|
-
* fix: use pool_pre_ping to discard invalid SQL connections when borrow… by @francoisserra in https://github.com/zenml-io/zenml/pull/1489
|
1354
|
-
|
1355
|
-
## New Contributors
|
1356
|
-
* @Cahllagerfeld made their first contribution in https://github.com/zenml-io/zenml/pull/1459
|
1357
|
-
* @francoisserra made their first contribution in https://github.com/zenml-io/zenml/pull/1489
|
1358
|
-
|
1359
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.37.0...0.38.0
|
1360
|
-
|
1361
|
-
# 0.37.0
|
1362
|
-
|
1363
|
-
In this ZenML release, we are pleased to introduce a compelling new feature:
|
1364
|
-
[ZenML Code Repositories](https://docs.zenml.io/starter-guide/production-fundamentals/code-repositories).
|
1365
|
-
This innovative addition formalizes the principles of code versioning and
|
1366
|
-
tracking while consolidating their pivotal role in executing pipelines and
|
1367
|
-
caching pipeline steps. With Code Repositories, ZenML is equipped to maintain an
|
1368
|
-
accurate record of the code version employed in your pipeline runs. Furthermore,
|
1369
|
-
executing a pipeline that is monitored by a registered code repository can
|
1370
|
-
significantly accelerate the Docker image building process for containerized
|
1371
|
-
stack components.
|
1372
|
-
|
1373
|
-
As is the case with everything ZenML, we designed the ZenML Code Repository
|
1374
|
-
concept as a highly extensible abstraction. The update defines the basic Code
|
1375
|
-
Repository interface an includes two implementations integrating ZenML with
|
1376
|
-
two popular code repository flavors: GitHub and GitLab.
|
1377
|
-
|
1378
|
-
## Other Enhancements
|
1379
|
-
|
1380
|
-
We've updated the `pytorch-lightning` integration to support the `2.0` version.
|
1381
|
-
We also updated the `mlflow` integration to support the `2.2.2` version.
|
1382
|
-
|
1383
|
-
**IMPORTANT**: it is not recommended to continue using MLflow older than `2.2.1`
|
1384
|
-
as a model registry with ZenML, as [it is vulnerable to a security issue](https://github.com/advisories/GHSA-xg73-94fp-g449).
|
1385
|
-
|
1386
|
-
Last but not least, two stellar additions from our community members:
|
1387
|
-
|
1388
|
-
* `zenml stack delete` now supports a `--recursive` flag to delete all
|
1389
|
-
components in a stack. Many thanks to @KenmogneThimotee for the contribution!
|
1390
|
-
* the ZenML Sagemaker step operator has been expanded to support S3 input data
|
1391
|
-
and additional input arguments. Many thanks to @christianversloot for the
|
1392
|
-
contribution!
|
1393
|
-
|
1394
|
-
## Breaking Changes
|
1395
|
-
|
1396
|
-
The ZenML GitHub Orchestrator and GitHub Secrets Manager have been removed in
|
1397
|
-
this release. Given that their concerns overlapped with the new ZenML GitHub
|
1398
|
-
Code Repository and they didn't provide sufficient value on their own, we
|
1399
|
-
decided to discontinue them. If you were using these components, you can
|
1400
|
-
continue to use GitHub Actions to run your pipelines, in combination with the
|
1401
|
-
ZenML GitHub Code Repository.
|
1402
|
-
|
1403
|
-
## What's Changed
|
1404
|
-
* Test integration for seldon example by @safoinme in https://github.com/zenml-io/zenml/pull/1285
|
1405
|
-
* Update `pytorch-lightning` to support `2.0` by @safoinme in https://github.com/zenml-io/zenml/pull/1425
|
1406
|
-
* Code repository by @schustmi in https://github.com/zenml-io/zenml/pull/1344
|
1407
|
-
* Bump `ruff` to 0.259 by @strickvl in https://github.com/zenml-io/zenml/pull/1439
|
1408
|
-
* Change `pipeline_run_id` to `run_name` by @safoinme in https://github.com/zenml-io/zenml/pull/1390
|
1409
|
-
* Update `mypy>=1.1.1` and fix new errors by @safoinme in https://github.com/zenml-io/zenml/pull/1432
|
1410
|
-
* Add `--upgrade` option to ZenML integration install by @safoinme in https://github.com/zenml-io/zenml/pull/1435
|
1411
|
-
* Bump `MLflow` to 2.2.2 by @safoinme in https://github.com/zenml-io/zenml/pull/1441
|
1412
|
-
* HuggingFace Spaces server deployment option by @strickvl in https://github.com/zenml-io/zenml/pull/1427
|
1413
|
-
* Bugfix for server import by @bcdurak in https://github.com/zenml-io/zenml/pull/1442
|
1414
|
-
* Fix HF Spaces URL by @strickvl in https://github.com/zenml-io/zenml/pull/1444
|
1415
|
-
* Remove all `zenml.cli` imports outside of `zenml.cli` by @fa9r in https://github.com/zenml-io/zenml/pull/1447
|
1416
|
-
* Add recursive deletion of components for `zenml stack delete` by @KenmogneThimotee in https://github.com/zenml-io/zenml/pull/1437
|
1417
|
-
* Temporarily disable primary key requirement for newer mysql versions by @schustmi in https://github.com/zenml-io/zenml/pull/1450
|
1418
|
-
* Add step name suffix for sagemaker job name by @schustmi in https://github.com/zenml-io/zenml/pull/1452
|
1419
|
-
* Code repo docs by @schustmi in https://github.com/zenml-io/zenml/pull/1448
|
1420
|
-
* Allow resource settings for airflow kubernetes pod operators by @schustmi in https://github.com/zenml-io/zenml/pull/1378
|
1421
|
-
* SageMaker step operator: expand input arguments and add support for S3 input data by @christianversloot in https://github.com/zenml-io/zenml/pull/1381
|
1422
|
-
* Add Screenshots to Code Repo Token by @safoinme in https://github.com/zenml-io/zenml/pull/1454
|
1423
|
-
|
1424
|
-
## New Contributors
|
1425
|
-
* @KenmogneThimotee made their first contribution in https://github.com/zenml-io/zenml/pull/1437
|
1426
|
-
* @christianversloot made their first contribution in https://github.com/zenml-io/zenml/pull/1381
|
1427
|
-
|
1428
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.36.1...0.37.0
|
1429
|
-
|
1430
|
-
# 0.36.1
|
1431
|
-
|
1432
|
-
This minor release contains some small fixes and improvements.
|
1433
|
-
|
1434
|
-
- We fixed a bug with the way hooks were being parsed, which was causing
|
1435
|
-
pipelines to fail.
|
1436
|
-
- We brought various parts of the documentation up to date with features that
|
1437
|
-
had previously been added, notably the new image building functionality.
|
1438
|
-
- We added a failure hook that connects to OpenAI's ChatGPT API to allow you to
|
1439
|
-
receive a message when a pipeline fails that includes suggestions on how to
|
1440
|
-
fix the failing step.
|
1441
|
-
- We added a new integration with `langchain` and `llama_hub` to allow you to
|
1442
|
-
build on top of those libraries as part of a more robust MLOps workflow.
|
1443
|
-
- We made the first some bigger changes to our analytics system to make it more
|
1444
|
-
robust and secure. This release begins that migration. Users should expect no
|
1445
|
-
changes in behavior and all telemetry-related preferences will be preserved.
|
1446
|
-
|
1447
|
-
## What's Changed
|
1448
|
-
* Fix hook parser by @strickvl in https://github.com/zenml-io/zenml/pull/1428
|
1449
|
-
* Fix some pipeline bugs by @schustmi in https://github.com/zenml-io/zenml/pull/1426
|
1450
|
-
* Add image builders to Examples by @safoinme in https://github.com/zenml-io/zenml/pull/1434
|
1451
|
-
* ZenML Failure Hook for OpenAI ChatGPT fixes by @strickvl in https://github.com/zenml-io/zenml/pull/1430
|
1452
|
-
* Integrations with `langchain` and `llama_hub` by @fa9r in https://github.com/zenml-io/zenml/pull/1404
|
1453
|
-
* Add basic tests for the server and recipes CLI by @wjayesh in https://github.com/zenml-io/zenml/pull/1306
|
1454
|
-
* Add to our alembic migration guide by @strickvl in https://github.com/zenml-io/zenml/pull/1423
|
1455
|
-
* Analytics 2.0 by @bcdurak in https://github.com/zenml-io/zenml/pull/1411
|
1456
|
-
* Improve Slack Alerter by adding message blocks by @soubenz in https://github.com/zenml-io/zenml/pull/1402
|
1457
|
-
* Add HF deployment type by @strickvl in https://github.com/zenml-io/zenml/pull/1438
|
1458
|
-
|
1459
|
-
## New Contributors
|
1460
|
-
* @soubenz made their first contribution in https://github.com/zenml-io/zenml/pull/1402
|
1461
|
-
|
1462
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.36.0...0.36.1
|
1463
|
-
|
1464
|
-
# 0.36.0
|
1465
|
-
|
1466
|
-
Our latest release adds hooks to ZenML pipelines to handle custom logic that
|
1467
|
-
occurs on pipeline failure or success. This is a powerful feature that allows
|
1468
|
-
you to easily receive custom alerts, for example, when a pipeline fails or
|
1469
|
-
succeeds. (Check out our video showcasing the feature
|
1470
|
-
[here](https://www.youtube.com/watch?v=KUW2G3EsqF8).)
|
1471
|
-
|
1472
|
-
The release is also packed with bug fixes and documentation updates. Some
|
1473
|
-
smaller improvements include an increase of the `step_configurations` column
|
1474
|
-
size in the database to accommodate really large configurations and the ability
|
1475
|
-
to click through to orchestrator logs for the Sagemaker orchestrator directly
|
1476
|
-
from the ZenML dashboard.
|
1477
|
-
|
1478
|
-
## Breaking Changes
|
1479
|
-
|
1480
|
-
Secrets are now handled internally by ZenML. This changes some behaviors that
|
1481
|
-
you may have become used to with the (now-deprecated) Secrets Manager stack
|
1482
|
-
component. The default behavior for the KServe and Seldon Core Model Deployer if
|
1483
|
-
explicit credentials are not configured through the secret stack component
|
1484
|
-
attribute has changed. Now, the model deployer will attempt to reuse credentials
|
1485
|
-
configured for the Artifact Store in the same stack and may, in some cases, fail
|
1486
|
-
if it cannot use them. In most cases, if credentials are not configured for the
|
1487
|
-
active Artifact Store, the model deployer will assume some form of implicit
|
1488
|
-
in-cloud authentication is configured for the Kubernetes cluster where KServe /
|
1489
|
-
Seldon Core is installed and default to using that.
|
1490
|
-
|
1491
|
-
## What's Changed
|
1492
|
-
|
1493
|
-
* Add CLI utils tests by @strickvl in https://github.com/zenml-io/zenml/pull/1383
|
1494
|
-
* Don't use docker client when building images remotely by @schustmi in https://github.com/zenml-io/zenml/pull/1394
|
1495
|
-
* Fix zenml-quickstart-model typo by @safoinme in https://github.com/zenml-io/zenml/pull/1397
|
1496
|
-
* Ignore starting quotes from Artifact store path by @safoinme in https://github.com/zenml-io/zenml/pull/1388
|
1497
|
-
* CI speed improvements by @stefannica in https://github.com/zenml-io/zenml/pull/1384
|
1498
|
-
* Fix stack recipe link by @strickvl in https://github.com/zenml-io/zenml/pull/1393
|
1499
|
-
* Switch FastAPI response class to orjson so `NaN` values don't break the server by @fa9r in https://github.com/zenml-io/zenml/pull/1395
|
1500
|
-
* Numpy materializer metadata for arrays with strings by @safoinme in https://github.com/zenml-io/zenml/pull/1392
|
1501
|
-
* Fix last remaining runs index by @stefannica in https://github.com/zenml-io/zenml/pull/1399
|
1502
|
-
* Add failure (and success hooks) by @htahir1 in https://github.com/zenml-io/zenml/pull/1361
|
1503
|
-
* Replace `pyspelling` with `typos` by @strickvl in https://github.com/zenml-io/zenml/pull/1400
|
1504
|
-
* Fix the download nltk param for report step by @wjayesh in https://github.com/zenml-io/zenml/pull/1409
|
1505
|
-
* Add `build_timeout` attribute to `GCPImageBuilderConfig` by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/1408
|
1506
|
-
* Bump `ruff` to v0.255 by @strickvl in https://github.com/zenml-io/zenml/pull/1403
|
1507
|
-
* Update title of deployment docs page by @strickvl in https://github.com/zenml-io/zenml/pull/1412
|
1508
|
-
* Changed to debug log by @htahir1 in https://github.com/zenml-io/zenml/pull/1406
|
1509
|
-
* Fix incorrect `--sort_by` help text by @strickvl in https://github.com/zenml-io/zenml/pull/1413
|
1510
|
-
* Document CLI filtering query language by @strickvl in https://github.com/zenml-io/zenml/pull/1414
|
1511
|
-
* Fix GitHub pip download cache key by @stefannica in https://github.com/zenml-io/zenml/pull/1405
|
1512
|
-
* Add orchestrator logs link for Sagemaker by @strickvl in https://github.com/zenml-io/zenml/pull/1375
|
1513
|
-
* Phase out secrets managers from other stack components. by @stefannica in https://github.com/zenml-io/zenml/pull/1401
|
1514
|
-
* Add MLflow UI message to quickstart example and fix autolog spillage by @stefannica in https://github.com/zenml-io/zenml/pull/1421
|
1515
|
-
* Add tests for the model registry by @safoinme in https://github.com/zenml-io/zenml/pull/1415
|
1516
|
-
* Remove Aspell installation by @strickvl in https://github.com/zenml-io/zenml/pull/1419
|
1517
|
-
* Increase `step_configurations` column size to 2^24 by @strickvl in https://github.com/zenml-io/zenml/pull/1422
|
1518
|
-
* Add help text for `enable_service` option in recipe sub-command by @safoinme in https://github.com/zenml-io/zenml/pull/1424
|
1519
|
-
|
1520
|
-
|
1521
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.35.1...test
|
1522
|
-
|
1523
|
-
# 0.35.1
|
1524
|
-
|
1525
|
-
**Note:** *This release replaces the previous 0.35.0 release that was yanked from
|
1526
|
-
PyPI due to a bug. If you already installed 0.35.0 and are experiencing issues,
|
1527
|
-
we recommend you downgrade to 0.34.0 before installing and upgrading to 0.35.1.*
|
1528
|
-
|
1529
|
-
This release is packed with big features as well as documentation updates and
|
1530
|
-
some bug fixes.
|
1531
|
-
|
1532
|
-
The 0.35.1 release puts models front and center in ZenML with the addition of
|
1533
|
-
the **Model Registry** abstraction and Stack Component. You can now register,
|
1534
|
-
version and manage models as first class citizens in ZenML. This is a major
|
1535
|
-
milestone for ZenML and we are excited to see what you build with it!
|
1536
|
-
The introduction of Model Registries greatly simplifies the journey that the
|
1537
|
-
model takes from training to deployment and extends the ZenML ecosystem to
|
1538
|
-
include model registry tools and libraries. The first Model Registry integration
|
1539
|
-
included in this release is MLFlow, with many more to come in the future.
|
1540
|
-
|
1541
|
-
This release also continues the deprecation of Secrets Managers and the
|
1542
|
-
introduction of Secret Stores. You now have the option of configuring the ZenML
|
1543
|
-
server to use AWS, GCP, Azure or Hashicorp Vault directly as a centralized
|
1544
|
-
secrets store back-end. This is meant to replace all Secrets Manager flavors
|
1545
|
-
which were previously used to store secrets using the same cloud services.
|
1546
|
-
|
1547
|
-
Please be reminded that all Secrets Managers are now deprecated and will be
|
1548
|
-
removed in the near future. We recommend that you migrate all your secrets from
|
1549
|
-
the Secrets Manager stack components to the centralized secrets store by means
|
1550
|
-
of the included `zenml secrets-manager secret migrate` CLI command.
|
1551
|
-
|
1552
|
-
Last but not least, this release includes an updated Evidently integration that
|
1553
|
-
is compatible with the latest and greatest features from Evidently: reports and
|
1554
|
-
test suites. Check out the updated example to get a feel for the new features.
|
1555
|
-
|
1556
|
-
## Breaking Changes
|
1557
|
-
|
1558
|
-
This release introduces a few breaking changes. Please update your code to
|
1559
|
-
reflect the changes below:
|
1560
|
-
|
1561
|
-
* the order of pipelines and runs in the post-execution results has been
|
1562
|
-
reversed. This means that the most recent pipeline and pipeline run can be
|
1563
|
-
accessed using the first index of the respective lists instead of the last
|
1564
|
-
index. This change was made to make the post-execution results more intuitive
|
1565
|
-
and to allow returning multi-page results in the future. This is a code snippet
|
1566
|
-
outlining the changes that you need to make in your post-execution code:
|
1567
|
-
|
1568
|
-
```python
|
1569
|
-
from zenml.post_execution import get_pipelines, get_unlisted_runs
|
1570
|
-
|
1571
|
-
pipelines = get_pipelines()
|
1572
|
-
|
1573
|
-
# instead of calling this to get the pipeline last created
|
1574
|
-
latest_pipeline = pipelines[-1]
|
1575
|
-
|
1576
|
-
# you now have to call this
|
1577
|
-
latest_pipeline = pipelines[0]
|
1578
|
-
|
1579
|
-
# and instead of calling this to get the latest run of a pipeline
|
1580
|
-
latest_pipeline_run = latest_pipeline.get_runs()[-1]
|
1581
|
-
# or
|
1582
|
-
latest_pipeline_run = latest_pipeline.runs[-1]
|
1583
|
-
|
1584
|
-
# you now have to call this
|
1585
|
-
latest_pipeline_run = latest_pipeline.get_runs()[0]
|
1586
|
-
# or
|
1587
|
-
latest_pipeline_run = latest_pipeline.runs[0]
|
1588
|
-
|
1589
|
-
# the same applies to the unlisted runs; instead of
|
1590
|
-
last_unlisted_run = get_unlisted_runs()[-1]
|
1591
|
-
|
1592
|
-
# you now have to call this
|
1593
|
-
last_unlisted_run = get_unlisted_runs()[0]
|
1594
|
-
```
|
1595
|
-
|
1596
|
-
* if you were using the `StepEnvironment` to fetch the name of the active step
|
1597
|
-
in your step implementation, this name no longer reflects the name of the step
|
1598
|
-
function. Instead, it now reflects the name of the step used in the pipeline
|
1599
|
-
DAG, similar to what you would see in the ZenML dashboard when visualizing the
|
1600
|
-
pipeline. This is also implicitly reflected in the output of `zenml model-deployer model`
|
1601
|
-
CLI commands.
|
1602
|
-
|
1603
|
-
## What's Changed
|
1604
|
-
* Upgrade dev dependencies by @strickvl in https://github.com/zenml-io/zenml/pull/1334
|
1605
|
-
* Add warning when attempting server connection without user permissions by @strickvl in https://github.com/zenml-io/zenml/pull/1314
|
1606
|
-
* Keep CLI help text for `zenml pipeline` to a single line by @strickvl in https://github.com/zenml-io/zenml/pull/1338
|
1607
|
-
* Rename page attributes by @schustmi in https://github.com/zenml-io/zenml/pull/1266
|
1608
|
-
* Add missing docs for pipeline build by @schustmi in https://github.com/zenml-io/zenml/pull/1341
|
1609
|
-
* Sagemaker orchestrator docstring and example update by @strickvl in https://github.com/zenml-io/zenml/pull/1350
|
1610
|
-
* Fix `secret create` docs error for secret store by @strickvl in https://github.com/zenml-io/zenml/pull/1355
|
1611
|
-
* Update README for test environment provisioning by @strickvl in https://github.com/zenml-io/zenml/pull/1336
|
1612
|
-
* Disable name prefix matching when updating/deleting entities by @schustmi in https://github.com/zenml-io/zenml/pull/1345
|
1613
|
-
* Add Kubeflow Pipeline UI Port to deprecated config by @safoinme in https://github.com/zenml-io/zenml/pull/1358
|
1614
|
-
* Small clarifications for slack alerter by @htahir1 in https://github.com/zenml-io/zenml/pull/1365
|
1615
|
-
* update Neptune integration for v1.0 compatibility by @AleksanderWWW in https://github.com/zenml-io/zenml/pull/1335
|
1616
|
-
* Integrations conditional requirements by @safoinme in https://github.com/zenml-io/zenml/pull/1255
|
1617
|
-
* Fix fetching versioned pipelines in post execution by @schustmi in https://github.com/zenml-io/zenml/pull/1363
|
1618
|
-
* Load artifact store before loading artifact to register filesystem by @schustmi in https://github.com/zenml-io/zenml/pull/1367
|
1619
|
-
* Remove poetry from CI by @schustmi in https://github.com/zenml-io/zenml/pull/1346
|
1620
|
-
* Fix Sagemaker example readme by @strickvl in https://github.com/zenml-io/zenml/pull/1370
|
1621
|
-
* Update evidently to include reports and tests by @wjayesh in https://github.com/zenml-io/zenml/pull/1283
|
1622
|
-
* Fix neptune linting error on `develop` (and bump ruff) by @strickvl in https://github.com/zenml-io/zenml/pull/1372
|
1623
|
-
* Add pydantic materializer by @htahir1 in https://github.com/zenml-io/zenml/pull/1371
|
1624
|
-
* Registering GIFs added by @htahir1 in https://github.com/zenml-io/zenml/pull/1368
|
1625
|
-
* Refresh CLI cheat sheet by @strickvl in https://github.com/zenml-io/zenml/pull/1347
|
1626
|
-
* Add dependency resolution docs by @strickvl in https://github.com/zenml-io/zenml/pull/1337
|
1627
|
-
* [BUGFIX] Fix error while using an existing SQL server with GCP ZenServer by @wjayesh in https://github.com/zenml-io/zenml/pull/1353
|
1628
|
-
* Update step name assignment with the parameter name by @strickvl in https://github.com/zenml-io/zenml/pull/1310
|
1629
|
-
* Copy huggingface data directory to local before loading in materializers by @TimovNiedek in https://github.com/zenml-io/zenml/pull/1351
|
1630
|
-
* Update huggingface token classification example by @strickvl in https://github.com/zenml-io/zenml/pull/1369
|
1631
|
-
* Use the most specialized materializer based on MRO by @schustmi in https://github.com/zenml-io/zenml/pull/1376
|
1632
|
-
* Update Kserve to support 0.10.0 by @safoinme in https://github.com/zenml-io/zenml/pull/1373
|
1633
|
-
* Add more examples to integration tests by @schustmi in https://github.com/zenml-io/zenml/pull/1245
|
1634
|
-
* Fix order of runs and order of pipelines in post-execution by @stefannica in https://github.com/zenml-io/zenml/pull/1380
|
1635
|
-
* Add Cloud Secrets Store back-ends by @stefannica in https://github.com/zenml-io/zenml/pull/1348
|
1636
|
-
* Model Registry Stack Component + MLFlow integration by @safoinme in https://github.com/zenml-io/zenml/pull/1309
|
1637
|
-
* Fix broken docs URLs and add SDK docs url by @strickvl in https://github.com/zenml-io/zenml/pull/1349
|
1638
|
-
* Fix label studio `dataset delete` command by @strickvl in https://github.com/zenml-io/zenml/pull/1377
|
1639
|
-
* Add missing links to Quickstart by @strickvl in
|
1640
|
-
https://github.com/zenml-io/zenml/pull/1379
|
1641
|
-
* Fix PyPI readme logo display by @strickvl in https://github.com/zenml-io/zenml/pull/1382
|
1642
|
-
* Fixed broken migration for flavors by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1386
|
1643
|
-
* Add debug mode flag for `zenml info` by @strickvl in https://github.com/zenml-io/zenml/pull/1374
|
1644
|
-
* Update issue creation for bugs by @strickvl in https://github.com/zenml-io/zenml/pull/1387
|
1645
|
-
* Integration sdk docs generated correctly now by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1389
|
1646
|
-
|
1647
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.34.0...0.35.0
|
1648
|
-
|
1649
|
-
# 0.35.0 (YANKED)
|
1650
|
-
|
1651
|
-
This release is packed with big features as well as documentation updates and
|
1652
|
-
some bug fixes.
|
1653
|
-
|
1654
|
-
The 0.35.0 release puts models front and center in ZenML with the addition of
|
1655
|
-
the **Model Registry** abstraction and Stack Component. You can now register,
|
1656
|
-
version and manage models as first class citizens in ZenML. This is a major
|
1657
|
-
milestone for ZenML and we are excited to see what you build with it!
|
1658
|
-
The introduction of Model Registries greatly simplifies the journey that the
|
1659
|
-
model takes from training to deployment and extends the ZenML ecosystem to
|
1660
|
-
include model registry tools and libraries. The first Model Registry integration
|
1661
|
-
included in this release is MLFlow, with many more to come in the future.
|
1662
|
-
|
1663
|
-
This release also continues the deprecation of Secrets Managers and the
|
1664
|
-
introduction of Secret Stores. You now have the option of configuring the ZenML
|
1665
|
-
server to use AWS, GCP, Azure or Hashicorp Vault directly as a centralized
|
1666
|
-
secrets store back-end. This is meant to replace all Secrets Manager flavors
|
1667
|
-
which were previously used to store secrets using the same cloud services.
|
1668
|
-
|
1669
|
-
Please be reminded that all Secrets Managers are now deprecated and will be
|
1670
|
-
removed in the near future. We recommend that you migrate all your secrets from
|
1671
|
-
the Secrets Manager stack components to the centralized secrets store by means
|
1672
|
-
of the included `zenml secrets-manager secret migrate` CLI command.
|
1673
|
-
|
1674
|
-
Last but not least, this release includes an updated Evidently integration that
|
1675
|
-
is compatible with the latest and greatest features from Evidently: reports and
|
1676
|
-
test suites. Check out the updated example to get a feel for the new features.
|
1677
|
-
|
1678
|
-
## Breaking Changes
|
1679
|
-
|
1680
|
-
This release introduces a few breaking changes. Please update your code to
|
1681
|
-
reflect the changes below:
|
1682
|
-
|
1683
|
-
* the order of pipelines and runs in the post-execution results has been
|
1684
|
-
reversed. This means that the most recent pipeline and pipeline run can be
|
1685
|
-
accessed using the first index of the respective lists instead of the last
|
1686
|
-
index. This change was made to make the post-execution results more intuitive
|
1687
|
-
and to allow returning multi-page results in the future. This is a code snippet
|
1688
|
-
outlining the changes that you need to make in your post-execution code:
|
1689
|
-
|
1690
|
-
```python
|
1691
|
-
from zenml.post_execution import get_pipelines, get_unlisted_runs
|
1692
|
-
|
1693
|
-
pipelines = get_pipelines()
|
1694
|
-
|
1695
|
-
# instead of calling this to get the pipeline last created
|
1696
|
-
latest_pipeline = pipelines[-1]
|
1697
|
-
|
1698
|
-
# you now have to call this
|
1699
|
-
latest_pipeline = pipelines[0]
|
1700
|
-
|
1701
|
-
# and instead of calling this to get the latest run of a pipeline
|
1702
|
-
latest_pipeline_run = latest_pipeline.get_runs()[-1]
|
1703
|
-
# or
|
1704
|
-
latest_pipeline_run = latest_pipeline.runs[-1]
|
1705
|
-
|
1706
|
-
# you now have to call this
|
1707
|
-
latest_pipeline_run = latest_pipeline.get_runs()[0]
|
1708
|
-
# or
|
1709
|
-
latest_pipeline_run = latest_pipeline.runs[0]
|
1710
|
-
|
1711
|
-
# the same applies to the unlisted runs; instead of
|
1712
|
-
last_unlisted_run = get_unlisted_runs()[-1]
|
1713
|
-
|
1714
|
-
# you now have to call this
|
1715
|
-
last_unlisted_run = get_unlisted_runs()[0]
|
1716
|
-
```
|
1717
|
-
|
1718
|
-
* if you were using the `StepEnvironment` to fetch the name of the active step
|
1719
|
-
in your step implementation, this name no longer reflects the name of the step
|
1720
|
-
function. Instead, it now reflects the name of the step used in the pipeline
|
1721
|
-
DAG, similar to what you would see in the ZenML dashboard when visualizing the
|
1722
|
-
pipeline. This is also implicitly reflected in the output of `zenml model-deployer model`
|
1723
|
-
CLI commands.
|
1724
|
-
|
1725
|
-
## What's Changed
|
1726
|
-
* Upgrade dev dependencies by @strickvl in https://github.com/zenml-io/zenml/pull/1334
|
1727
|
-
* Add warning when attempting server connection without user permissions by @strickvl in https://github.com/zenml-io/zenml/pull/1314
|
1728
|
-
* Keep CLI help text for `zenml pipeline` to a single line by @strickvl in https://github.com/zenml-io/zenml/pull/1338
|
1729
|
-
* Rename page attributes by @schustmi in https://github.com/zenml-io/zenml/pull/1266
|
1730
|
-
* Add missing docs for pipeline build by @schustmi in https://github.com/zenml-io/zenml/pull/1341
|
1731
|
-
* Sagemaker orchestrator docstring and example update by @strickvl in https://github.com/zenml-io/zenml/pull/1350
|
1732
|
-
* Fix `secret create` docs error for secret store by @strickvl in https://github.com/zenml-io/zenml/pull/1355
|
1733
|
-
* Update README for test environment provisioning by @strickvl in https://github.com/zenml-io/zenml/pull/1336
|
1734
|
-
* Disable name prefix matching when updating/deleting entities by @schustmi in https://github.com/zenml-io/zenml/pull/1345
|
1735
|
-
* Add Kubeflow Pipeline UI Port to deprecated config by @safoinme in https://github.com/zenml-io/zenml/pull/1358
|
1736
|
-
* Small clarifications for slack alerter by @htahir1 in https://github.com/zenml-io/zenml/pull/1365
|
1737
|
-
* update Neptune integration for v1.0 compatibility by @AleksanderWWW in https://github.com/zenml-io/zenml/pull/1335
|
1738
|
-
* Integrations conditional requirements by @safoinme in https://github.com/zenml-io/zenml/pull/1255
|
1739
|
-
* Fix fetching versioned pipelines in post execution by @schustmi in https://github.com/zenml-io/zenml/pull/1363
|
1740
|
-
* Load artifact store before loading artifact to register filesystem by @schustmi in https://github.com/zenml-io/zenml/pull/1367
|
1741
|
-
* Remove poetry from CI by @schustmi in https://github.com/zenml-io/zenml/pull/1346
|
1742
|
-
* Fix Sagemaker example readme by @strickvl in https://github.com/zenml-io/zenml/pull/1370
|
1743
|
-
* Update evidently to include reports and tests by @wjayesh in https://github.com/zenml-io/zenml/pull/1283
|
1744
|
-
* Fix neptune linting error on `develop` (and bump ruff) by @strickvl in https://github.com/zenml-io/zenml/pull/1372
|
1745
|
-
* Add pydantic materializer by @htahir1 in https://github.com/zenml-io/zenml/pull/1371
|
1746
|
-
* Registering GIFs added by @htahir1 in https://github.com/zenml-io/zenml/pull/1368
|
1747
|
-
* Refresh CLI cheat sheet by @strickvl in https://github.com/zenml-io/zenml/pull/1347
|
1748
|
-
* Add dependency resolution docs by @strickvl in https://github.com/zenml-io/zenml/pull/1337
|
1749
|
-
* [BUGFIX] Fix error while using an existing SQL server with GCP ZenServer by @wjayesh in https://github.com/zenml-io/zenml/pull/1353
|
1750
|
-
* Update step name assignment with the parameter name by @strickvl in https://github.com/zenml-io/zenml/pull/1310
|
1751
|
-
* Copy huggingface data directory to local before loading in materializers by @TimovNiedek in https://github.com/zenml-io/zenml/pull/1351
|
1752
|
-
* Update huggingface token classification example by @strickvl in https://github.com/zenml-io/zenml/pull/1369
|
1753
|
-
* Use the most specialized materializer based on MRO by @schustmi in https://github.com/zenml-io/zenml/pull/1376
|
1754
|
-
* Update Kserve to support 0.10.0 by @safoinme in https://github.com/zenml-io/zenml/pull/1373
|
1755
|
-
* Add more examples to integration tests by @schustmi in https://github.com/zenml-io/zenml/pull/1245
|
1756
|
-
* Fix order of runs and order of pipelines in post-execution by @stefannica in https://github.com/zenml-io/zenml/pull/1380
|
1757
|
-
* Add Cloud Secrets Store back-ends by @stefannica in https://github.com/zenml-io/zenml/pull/1348
|
1758
|
-
* Model Registry Stack Component + MLFlow integration by @safoinme in https://github.com/zenml-io/zenml/pull/1309
|
1759
|
-
* Fix broken docs URLs and add SDK docs url by @strickvl in https://github.com/zenml-io/zenml/pull/1349
|
1760
|
-
* Fix label studio `dataset delete` command by @strickvl in https://github.com/zenml-io/zenml/pull/1377
|
1761
|
-
* Add missing links to Quickstart by @strickvl in https://github.com/zenml-io/zenml/pull/1379
|
1762
|
-
|
1763
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.34.0...0.35.0
|
1764
|
-
|
1765
|
-
# 0.34.0
|
1766
|
-
|
1767
|
-
This release comes with major upgrades to the python library as well as the dashboard:
|
1768
|
-
- You can now store you secrets in a centralized way instead of having them
|
1769
|
-
tied to a secrets manager stack component. The secrets manager component is deprecated but will still
|
1770
|
-
work while we continue migrating all secrets manager flavors to be available as a backend to store centralized
|
1771
|
-
secrets. Check out [the docs](https://docs.zenml.io/starter-guide/production-fundamentals/secrets-management)
|
1772
|
-
for more information.
|
1773
|
-
- Pipelines are now versioned: ZenML detects changes to your steps and structure of your pipelines and
|
1774
|
-
automatically creates new pipeline versions for you.
|
1775
|
-
- You can now build the required Docker images for your pipeline without actually running it with
|
1776
|
-
the `zenml pipeline build` command. This build can later be used to run the pipeline using the
|
1777
|
-
`zenml pipeline run` command or by passing it to `pipeline.run()` in python.
|
1778
|
-
- Metadata for runs and artifacts is now displayed in the dashboard: When viewing a pipeline run in the dashboard,
|
1779
|
-
click on a step or artifact to get useful metadata like the endpoint where your model is deployed or
|
1780
|
-
statistics about your training data.
|
1781
|
-
|
1782
|
-
|
1783
|
-
## What's Changed
|
1784
|
-
* Move inbuilt Flavors into the Database by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1187
|
1785
|
-
* Bump `ruff` version to 241 by @strickvl in https://github.com/zenml-io/zenml/pull/1289
|
1786
|
-
* Add docs for run name templates by @schustmi in https://github.com/zenml-io/zenml/pull/1290
|
1787
|
-
* Remove excess help text for `zenml connect` command by @strickvl in https://github.com/zenml-io/zenml/pull/1291
|
1788
|
-
* Increase default service timeout to 60 by @safoinme in https://github.com/zenml-io/zenml/pull/1294
|
1789
|
-
* increase timeout on quickstart example by @safoinme in https://github.com/zenml-io/zenml/pull/1296
|
1790
|
-
* Add warning about MacOS not being supported by @strickvl in https://github.com/zenml-io/zenml/pull/1303
|
1791
|
-
* Always include .zen in docker builds by @schustmi in https://github.com/zenml-io/zenml/pull/1292
|
1792
|
-
* Add warning and docs update for `label_studio` installation issue by @strickvl in https://github.com/zenml-io/zenml/pull/1299
|
1793
|
-
* Loosen version requirements for Great Expectations integration by @strickvl in https://github.com/zenml-io/zenml/pull/1302
|
1794
|
-
* Change zenml init --template to optionally prompt and track email by @stefannica in https://github.com/zenml-io/zenml/pull/1298
|
1795
|
-
* Update docs for Neptune experiment tracker integration by @strickvl in https://github.com/zenml-io/zenml/pull/1307
|
1796
|
-
* Fix the destroy function on the stack recipe CLI by @wjayesh in https://github.com/zenml-io/zenml/pull/1301
|
1797
|
-
* Add missing flavor migrations, make workspace ID optional by @schustmi in https://github.com/zenml-io/zenml/pull/1315
|
1798
|
-
* Bump ruff 246 by @strickvl in https://github.com/zenml-io/zenml/pull/1316
|
1799
|
-
* Remove tag from image name in gcp image builder by @schustmi in https://github.com/zenml-io/zenml/pull/1317
|
1800
|
-
* Fix docs typo by @strickvl in https://github.com/zenml-io/zenml/pull/1318
|
1801
|
-
* Fix step parameter merging by @schustmi in https://github.com/zenml-io/zenml/pull/1320
|
1802
|
-
* Increase timeout for mlflow deployment example by @strickvl in https://github.com/zenml-io/zenml/pull/1308
|
1803
|
-
* Workspace/projects fix for dashboard URL output when running pipeline by @strickvl in https://github.com/zenml-io/zenml/pull/1322
|
1804
|
-
* Component Metadata Tracking Docs by @fa9r in https://github.com/zenml-io/zenml/pull/1319
|
1805
|
-
* Add user environment `zenml info` command to CLI for debugging by @strickvl in https://github.com/zenml-io/zenml/pull/1312
|
1806
|
-
* Added caching to quickstart by @htahir1 in https://github.com/zenml-io/zenml/pull/1321
|
1807
|
-
* Renovation of the zenstore tests by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1275
|
1808
|
-
* Fixes GCP docs typo by @luckri13 in https://github.com/zenml-io/zenml/pull/1327
|
1809
|
-
* Remove deprecated CLI options by @strickvl in https://github.com/zenml-io/zenml/pull/1325
|
1810
|
-
* GCP Image Builder network by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/1323
|
1811
|
-
* improved flavor docs by @htahir1 in https://github.com/zenml-io/zenml/pull/1324
|
1812
|
-
* Commands to register, build and run pipelines from the CLI by @schustmi in https://github.com/zenml-io/zenml/pull/1293
|
1813
|
-
* Validate kserve model name by @strickvl in https://github.com/zenml-io/zenml/pull/1304
|
1814
|
-
* Fix post-execution run sorting by @schustmi in https://github.com/zenml-io/zenml/pull/1332
|
1815
|
-
* Secrets store with SQL back-end by @stefannica in https://github.com/zenml-io/zenml/pull/1313
|
1816
|
-
|
1817
|
-
## New Contributors
|
1818
|
-
* @luckri13 made their first contribution in https://github.com/zenml-io/zenml/pull/1327
|
1819
|
-
|
1820
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.33.0...0.34.0
|
1821
|
-
|
1822
|
-
# 0.33.0
|
1823
|
-
|
1824
|
-
This release introduces several big new features:
|
1825
|
-
- Docker images can now be built in GCP using the new
|
1826
|
-
[Google Cloud Image Builder](https://docs.zenml.io/component-gallery/image-builders/gcloud-build)
|
1827
|
-
integration. Special shoutout to @gabrielmbmb for this amazing contribution!
|
1828
|
-
- Getting started with ZenML has been made even easier. You can now use one of
|
1829
|
-
the new [ZenML Project Templates](https://github.com/zenml-io/zenml-project-templates)
|
1830
|
-
to initialize your ZenML repository with a basic project structure including a
|
1831
|
-
functional pipeline and basic scaffolding for materializers, parameters, and
|
1832
|
-
other classes you might want to extend.
|
1833
|
-
- Orchestrating runs on local Kubernetes has been made easier: The KubeFlow,
|
1834
|
-
Kubernetes, and Tekton orchestrators have been redesigned to be compatible with
|
1835
|
-
the [K3D modular stack recipe](https://github.com/zenml-io/mlops-stacks/tree/main/k3d-modular)
|
1836
|
-
that lets you spin up a local K3D Kubernetes cluster with a single line of code!
|
1837
|
-
- The MLflow integration has been updated and can now be used with the new
|
1838
|
-
MLflow 2.x!
|
1839
|
-
- You can now specify parameters and resources for your Seldon model deployers
|
1840
|
-
thanks to @d-lowl!
|
1841
|
-
|
1842
|
-
Furthermore, the internal `project` concept has been renamed to `workspace` to
|
1843
|
-
avoid confusion with the [zenml-projects](https://github.com/zenml-io/zenml-projects)
|
1844
|
-
repository. This should only be relevant to you if you have custom applications
|
1845
|
-
that are interacting with the REST API of the ZenML server directly since all
|
1846
|
-
models sent from/to the server need to contain a `workspace` instead of a
|
1847
|
-
`project` now.
|
1848
|
-
|
1849
|
-
## What's Changed
|
1850
|
-
* Renaming Project to Workspace by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1254
|
1851
|
-
* Integration tests for post execution functions by @fa9r in https://github.com/zenml-io/zenml/pull/1264
|
1852
|
-
* Introduce `post_execution.BaseView` by @fa9r in https://github.com/zenml-io/zenml/pull/1238
|
1853
|
-
* Make `/cloud` point to enterprise page by @strickvl in https://github.com/zenml-io/zenml/pull/1268
|
1854
|
-
* update mlflow to version greater than 2.0 by @safoinme in https://github.com/zenml-io/zenml/pull/1249
|
1855
|
-
* Store run start time by @schustmi in https://github.com/zenml-io/zenml/pull/1271
|
1856
|
-
* Relax pydantic dependency by @jlopezpena in https://github.com/zenml-io/zenml/pull/1262
|
1857
|
-
* Fix failing filter on stacks by component id by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1276
|
1858
|
-
* Track server version by @schustmi in https://github.com/zenml-io/zenml/pull/1265
|
1859
|
-
* Bump ruff, drop `autoflake`, add `darglint` back by @strickvl in https://github.com/zenml-io/zenml/pull/1279
|
1860
|
-
* Fixed startswith and endswith by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1278
|
1861
|
-
* Fix workspace scoping on `list_workspace_... endpoints` again by @fa9r in https://github.com/zenml-io/zenml/pull/1284
|
1862
|
-
* Custom Metadata Tracking by @fa9r in https://github.com/zenml-io/zenml/pull/1151
|
1863
|
-
* Bug: local ZenML server ignores ip-address CLI argument by @stefannica in https://github.com/zenml-io/zenml/pull/1282
|
1864
|
-
* Configure the zenml-server docker image and helm chart to run as non-privileged user by @stefannica in https://github.com/zenml-io/zenml/pull/1273
|
1865
|
-
* GCP Image Builder by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/1270
|
1866
|
-
* Disentangle K3D code from ZenML by @safoinme in https://github.com/zenml-io/zenml/pull/1185
|
1867
|
-
* Rework params / artifact docs by @strickvl in https://github.com/zenml-io/zenml/pull/1277
|
1868
|
-
* Always add active user to analytics by @stefannica in https://github.com/zenml-io/zenml/pull/1286
|
1869
|
-
* Fix step and pipeline run metadata in LineageGraph by @fa9r in https://github.com/zenml-io/zenml/pull/1288
|
1870
|
-
* add validator to endpoint url to replace hostname with k3d or docker … by @safoinme in https://github.com/zenml-io/zenml/pull/1189
|
1871
|
-
* Add option to use project templates to initialize a repository by @stefannica in https://github.com/zenml-io/zenml/pull/1287
|
1872
|
-
* Add example for Hyperparameter Tuning with ZenML by @nitay93 in https://github.com/zenml-io/zenml/pull/1206
|
1873
|
-
* Add seldon deployment predictor parameters and resource requirements by @d-lowl in https://github.com/zenml-io/zenml/pull/1280
|
1874
|
-
|
1875
|
-
## New Contributors
|
1876
|
-
* @jlopezpena made their first contribution in https://github.com/zenml-io/zenml/pull/1262
|
1877
|
-
* @nitay93 made their first contribution in https://github.com/zenml-io/zenml/pull/1206
|
1878
|
-
* @d-lowl made their first contribution in https://github.com/zenml-io/zenml/pull/1280
|
1879
|
-
|
1880
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.32.1...0.33.0
|
1881
|
-
|
1882
|
-
# 0.32.1
|
1883
|
-
|
1884
|
-
This release resolves several minor bugs and inconveniences introduced during
|
1885
|
-
the filtering and pagination overhaul in the last release. Additionally, the
|
1886
|
-
release includes new integration tests to improve future stability.
|
1887
|
-
|
1888
|
-
## What's Changed
|
1889
|
-
* Update and improve docker and helm deployment docs by @stefannica in https://github.com/zenml-io/zenml/pull/1246
|
1890
|
-
* Fixed broken link returned form pipeline runs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1257
|
1891
|
-
* Fix project scoping on `list_project_...` endpoints by @fa9r in https://github.com/zenml-io/zenml/pull/1256
|
1892
|
-
* Orchestrator tests by @schustmi in https://github.com/zenml-io/zenml/pull/1258
|
1893
|
-
* Add integration tests for lineage graph creation by @fa9r in https://github.com/zenml-io/zenml/pull/1253
|
1894
|
-
* Always instantiate a zen_store before startup. by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1261
|
1895
|
-
* Fix post execution run fetching by @schustmi in https://github.com/zenml-io/zenml/pull/1263
|
1896
|
-
* Implemented the option to choose between ascending and descending on list calls by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1260
|
1897
|
-
* Fix logger warning message by @strickvl in https://github.com/zenml-io/zenml/pull/1267
|
1898
|
-
|
1899
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.32.0...0.32.1
|
1900
|
-
|
1901
|
-
# 0.32.0
|
1902
|
-
|
1903
|
-
Release 0.32.0 introduces two big new features:
|
1904
|
-
* A new stack component, the "image builder", with a corresponding new Kaniko
|
1905
|
-
integration.
|
1906
|
-
* Logic for filtering and pagination of list requests.
|
1907
|
-
|
1908
|
-
## Image Builder Abstraction and Kaniko Integration
|
1909
|
-
ZenML stacks can now contain an image builder as additional optional stack
|
1910
|
-
component. The image builder defines how the Docker images are built that are
|
1911
|
-
required by many of the other stack components such as Airflow or Kubeflow.
|
1912
|
-
Previously, all image building was handled implicitly by ZenML using local
|
1913
|
-
Docker, which has now been refactored into the "local" image builder flavor.
|
1914
|
-
As an alternative, you can now install the new "kaniko" integration to build
|
1915
|
-
your images in Kubernetes using Kaniko.
|
1916
|
-
|
1917
|
-
## Filtering and Pagination
|
1918
|
-
All list commands in ZenML are now capable of advanced filtering such as
|
1919
|
-
`zenml stack list --created="gt:22-12-04 17:00:00" --name contains:def`.
|
1920
|
-
|
1921
|
-
Additionally, list commands now return pages of results, which significantly
|
1922
|
-
improves performance for power ZenML users that have already created many runs
|
1923
|
-
or other entities.
|
1924
|
-
|
1925
|
-
## What's Changed
|
1926
|
-
* UserResponseModel contains roles, block recursion properly on more Models, reduce amount of Runs on a PipelineResponseModel by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1180
|
1927
|
-
* Bump ruff version by @strickvl in https://github.com/zenml-io/zenml/pull/1232
|
1928
|
-
* Zenfile becomes project by @strickvl in https://github.com/zenml-io/zenml/pull/1235
|
1929
|
-
* Fix class resolution in notebooks under Python>=3.10 by @fa9r in https://github.com/zenml-io/zenml/pull/1234
|
1930
|
-
* Fix Sagemaker README images & pipeline addition by @strickvl in https://github.com/zenml-io/zenml/pull/1239
|
1931
|
-
* Step/Pipeline configuration tests by @schustmi in https://github.com/zenml-io/zenml/pull/1233
|
1932
|
-
* Removed gRPC from diagrams by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1242
|
1933
|
-
* Fix MLflow tracking example bug for Macs by @strickvl in https://github.com/zenml-io/zenml/pull/1237
|
1934
|
-
* Fix copy function to copyfile in registered filesystem by @safoinme in https://github.com/zenml-io/zenml/pull/1243
|
1935
|
-
* Image builder abstraction by @schustmi in https://github.com/zenml-io/zenml/pull/1198
|
1936
|
-
* Add support for modular recipes to the recipe CLI by @wjayesh in https://github.com/zenml-io/zenml/pull/1247
|
1937
|
-
* Add docs on upgrading and troubleshooting zenml server by @wjayesh in https://github.com/zenml-io/zenml/pull/1244
|
1938
|
-
* Improve Seldon and Kserve Docs by @wjayesh in https://github.com/zenml-io/zenml/pull/1236
|
1939
|
-
* Add Pagination to all List commands by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1113
|
1940
|
-
|
1941
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.31.1...0.32.0
|
1942
|
-
|
1943
|
-
# 0.31.1
|
1944
|
-
|
1945
|
-
This release includes several bug fixes and new additions under the hood such as
|
1946
|
-
testing for various internal utility functions. This should help keep ZenML more
|
1947
|
-
stable over time. Additionally, we added the ability to customize default
|
1948
|
-
materializers for custom artifact stores, and the ability to track system info
|
1949
|
-
and the Python version of pipeline runs (both where pipelines are initially
|
1950
|
-
executed as well as wherever they eventually run). We added better support for
|
1951
|
-
pipeline scheduling (particularly from within the CLI) and tracking of the
|
1952
|
-
source code of steps. The release also includes the addition of information
|
1953
|
-
about whether the pipeline is running on a stack created by the active user, and
|
1954
|
-
the ability to specify Kubernetes container resource requests and limits.
|
1955
|
-
Finally, we addressed issues with caching such that caching is enabled for steps
|
1956
|
-
that have explicit enable_cache=True specified (even when pipelines have it
|
1957
|
-
turned off).
|
1958
|
-
|
1959
|
-
|
1960
|
-
## What's Changed
|
1961
|
-
* Test for `enum_utils` by @strickvl in https://github.com/zenml-io/zenml/pull/1209
|
1962
|
-
* Add missing space in Azure docs by @strickvl in https://github.com/zenml-io/zenml/pull/1218
|
1963
|
-
* Test for `dashboard_utils` by @strickvl in https://github.com/zenml-io/zenml/pull/1202
|
1964
|
-
* Cloud version gets love by @htahir1 in https://github.com/zenml-io/zenml/pull/1219
|
1965
|
-
* ZenFiles to ZenML Projects by @htahir1 in https://github.com/zenml-io/zenml/pull/1220
|
1966
|
-
* Track System Info and Python Version of Pipeline Runs by @fa9r in https://github.com/zenml-io/zenml/pull/1215
|
1967
|
-
* Tests for `pydantic_utils` by @strickvl in https://github.com/zenml-io/zenml/pull/1207
|
1968
|
-
* Customizing Default Materializers for Custom Artifact Stores by @safoinme in https://github.com/zenml-io/zenml/pull/1224
|
1969
|
-
* Test `typed_model` utilities by @strickvl in https://github.com/zenml-io/zenml/pull/1208
|
1970
|
-
* Enable Airflow<2.4 by @schustmi in https://github.com/zenml-io/zenml/pull/1222
|
1971
|
-
* Fix `alembic_start` migration if tables exist by @fa9r in https://github.com/zenml-io/zenml/pull/1214
|
1972
|
-
* Tests for `network_utils` by @strickvl in https://github.com/zenml-io/zenml/pull/1201
|
1973
|
-
* Tests for `io_utils` and removal of duplicate code by @strickvl in https://github.com/zenml-io/zenml/pull/1199
|
1974
|
-
* Use `ruff` to replace our linting suite by @strickvl in https://github.com/zenml-io/zenml/pull/1211
|
1975
|
-
* Test `materializer` utilities by @safoinme in https://github.com/zenml-io/zenml/pull/1221
|
1976
|
-
* Add information whether pipeline is running on a stack created by the active user by @schustmi in https://github.com/zenml-io/zenml/pull/1229
|
1977
|
-
* Test `daemon` util functions by @strickvl in https://github.com/zenml-io/zenml/pull/1210
|
1978
|
-
* Test `filesync_model` utils by @strickvl in https://github.com/zenml-io/zenml/pull/1230
|
1979
|
-
* Track Source Code of Steps by @fa9r in https://github.com/zenml-io/zenml/pull/1216
|
1980
|
-
* Track Pipeline Run Schedules by @fa9r in https://github.com/zenml-io/zenml/pull/1227
|
1981
|
-
* Tests for analytics by @bcdurak in https://github.com/zenml-io/zenml/pull/1228
|
1982
|
-
* Allow specifying Kubernetes container resource requests and limits by @schustmi in https://github.com/zenml-io/zenml/pull/1223
|
1983
|
-
* Enable cache for all steps that have explicit `enable_cache=True` by @fa9r in https://github.com/zenml-io/zenml/pull/1217
|
1984
|
-
* Make shared stacks visible again by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1225
|
1985
|
-
|
1986
|
-
|
1987
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.31.0...0.31.1
|
1988
|
-
|
1989
|
-
|
1990
|
-
# 0.31.0
|
1991
|
-
|
1992
|
-
The highlights of this release are:
|
1993
|
-
|
1994
|
-
* our Materializers have been redesigned to be more flexible and easier to use
|
1995
|
-
* we have added a new integration test framework
|
1996
|
-
* the SageMaker orchestrator has been added to our list of supported orchestrators
|
1997
|
-
* pipeline runs and artifacts can now be deleted from the ZenML database via the CLI
|
1998
|
-
or the Client API
|
1999
|
-
* some integrations have been updated to a more recent version: Kubeflow, Seldon
|
2000
|
-
Core and Tekton
|
2001
|
-
|
2002
|
-
This release also includes a few bug fixes and other minor improvements to
|
2003
|
-
existing features.
|
2004
|
-
|
2005
|
-
## What's Changed
|
2006
|
-
* Fix installation instructions in readme and docs by @schustmi in https://github.com/zenml-io/zenml/pull/1167
|
2007
|
-
* Fix broken TOC for scheduling docs by @strickvl in https://github.com/zenml-io/zenml/pull/1169
|
2008
|
-
* Ensure model string fields have a max length by @strickvl in https://github.com/zenml-io/zenml/pull/1136
|
2009
|
-
* Integration test framework by @stefannica in https://github.com/zenml-io/zenml/pull/1099
|
2010
|
-
* Check if all ZenML server dependencies are installed for local zenml deployment using `zenml up` by @dnth in https://github.com/zenml-io/zenml/pull/1144
|
2011
|
-
* Persist the server ID in the database by @stefannica in https://github.com/zenml-io/zenml/pull/1173
|
2012
|
-
* Tiny docs improvements by @strickvl in https://github.com/zenml-io/zenml/pull/1179
|
2013
|
-
* Changing some interactions with analytics fields by @bcdurak in https://github.com/zenml-io/zenml/pull/1174
|
2014
|
-
* Fix `PyTorchDataLoaderMaterializer` for older torch versions by @fa9r in https://github.com/zenml-io/zenml/pull/1178
|
2015
|
-
* Redesign Materializers by @fa9r in https://github.com/zenml-io/zenml/pull/1154
|
2016
|
-
* Fixing the error messages when fetching entities by @bcdurak in https://github.com/zenml-io/zenml/pull/1171
|
2017
|
-
* Moved the active_user property onto the client, implemented get_myself as zenstore method by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1161
|
2018
|
-
* Bugfix/bump evidently version by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1183
|
2019
|
-
* Alembic migration to update size of flavor config schema by @fa9r in https://github.com/zenml-io/zenml/pull/1181
|
2020
|
-
* Deleting pipeline runs and artifacts by @fa9r in https://github.com/zenml-io/zenml/pull/1164
|
2021
|
-
* Signer email checked before setting in google cloud scheduler by @htahir1 in https://github.com/zenml-io/zenml/pull/1184
|
2022
|
-
* Fix zenml helm chart to not leak analytics events by @stefannica in https://github.com/zenml-io/zenml/pull/1190
|
2023
|
-
* Tests for `dict_utils` by @strickvl in https://github.com/zenml-io/zenml/pull/1196
|
2024
|
-
* Adding exception tracking to `zeml init` by @bcdurak in https://github.com/zenml-io/zenml/pull/1192
|
2025
|
-
* Prevent crashes during Airflow server forking on MacOS by @schustmi in https://github.com/zenml-io/zenml/pull/1186
|
2026
|
-
* add alpha as server deployment type by @wjayesh in https://github.com/zenml-io/zenml/pull/1197
|
2027
|
-
* Bugfix for custom flavor registration by @bcdurak in https://github.com/zenml-io/zenml/pull/1195
|
2028
|
-
* Tests for `uuid_utils` by @strickvl in https://github.com/zenml-io/zenml/pull/1200
|
2029
|
-
* Sagemaker orchestrator integration by @strickvl in https://github.com/zenml-io/zenml/pull/1177
|
2030
|
-
* Fix Pandas Materializer Index by @safoinme in https://github.com/zenml-io/zenml/pull/1193
|
2031
|
-
* Add support for deploying custom stack recipes using the ZenML CLI by @wjayesh in https://github.com/zenml-io/zenml/pull/1188
|
2032
|
-
* Add cloud CI environments by @stefannica in https://github.com/zenml-io/zenml/pull/1176
|
2033
|
-
* Fix project scoping for artifact list through ZenServer by @fa9r in https://github.com/zenml-io/zenml/pull/1203
|
2034
|
-
|
2035
|
-
|
2036
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.30.0...0.31.0
|
2037
|
-
|
2038
|
-
# 0.30.0
|
2039
|
-
|
2040
|
-
In this release, ZenML finally adds Mac M1 support, Python 3.10 support and much
|
2041
|
-
greater flexibility and configurability under the hood by deprecating some
|
2042
|
-
large dependencies like `ml-pipelines-sdk`.
|
2043
|
-
|
2044
|
-
## Scheduling
|
2045
|
-
|
2046
|
-
Based on some community feedback around scheduling, this release comes with
|
2047
|
-
improved docs concerning scheduling in general. Additionally, the Vertex AI
|
2048
|
-
orchestrator now also supports scheduling.
|
2049
|
-
|
2050
|
-
## Slimmer Dependencies
|
2051
|
-
|
2052
|
-
By removing dependencies on some of the packages that ZenML was built on, this
|
2053
|
-
version of ZenML is slimmer, faster and more configurable than ever. This also
|
2054
|
-
finally makes ZenML run natively on Macs with M1 processors without the need for
|
2055
|
-
Rosetta. This also finally enables ZenML to run on Python 3.10.
|
2056
|
-
|
2057
|
-
## Breaking Changes
|
2058
|
-
|
2059
|
-
* The removal of `ml-pipelines-sdk` and `tfx` leads to some larger changes in
|
2060
|
-
the database that is tracking your pipeline runs and artifacts. **Note**: There
|
2061
|
-
is an automatic migration to upgrade this automatically, However, please note
|
2062
|
-
that downgrading back down to 0.23.0 is not supported.
|
2063
|
-
* The CLI commands to export and import pipeline runs have been deprecated.
|
2064
|
-
Namely: `zenml pipeline runs export` and `zenml pipeline runs import`
|
2065
|
-
These commands were meant for migrating from `zenml<0.20.0` to
|
2066
|
-
`0.20.0<=zenml<0.30.0`.
|
2067
|
-
* The `azure-ml` integration dependency on `azureml-core` has been upgraded
|
2068
|
-
from `1.42` to `1.48`
|
2069
|
-
|
2070
|
-
|
2071
|
-
## What's Changed
|
2072
|
-
* Remove stack extra from installation, enable re-running the quickstart by @schustmi in https://github.com/zenml-io/zenml/pull/1133
|
2073
|
-
* Secrets manager support to experiment trackers docs by @safoinme in https://github.com/zenml-io/zenml/pull/1137
|
2074
|
-
* Updating the README files of our examples by @bcdurak in https://github.com/zenml-io/zenml/pull/1128
|
2075
|
-
* Prevent running with local ZenStore and remote code execution by @schustmi in https://github.com/zenml-io/zenml/pull/1134
|
2076
|
-
* Remove `ml-pipelines-sdk` dependency by @schustmi in https://github.com/zenml-io/zenml/pull/1103
|
2077
|
-
* Fix Huggingface dataset materializer by @safoinme in https://github.com/zenml-io/zenml/pull/1142
|
2078
|
-
* Disallow alembic downgrades for 0.30.0 release by @fa9r in https://github.com/zenml-io/zenml/pull/1140
|
2079
|
-
* Fix Client flavor-related methods by @schustmi in https://github.com/zenml-io/zenml/pull/1153
|
2080
|
-
* Replace User Password with Token in docker images by @safoinme in https://github.com/zenml-io/zenml/pull/1147
|
2081
|
-
* Remove zenml pipeline runs export / import CLI commands by @fa9r in https://github.com/zenml-io/zenml/pull/1150
|
2082
|
-
* Context manager to track events by @bcdurak in https://github.com/zenml-io/zenml/pull/1149
|
2083
|
-
* Made explicit `is not None` calls to allow for empty pwd again by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1159
|
2084
|
-
* Add Neptune exp tracker into flavors table by @dnth in https://github.com/zenml-io/zenml/pull/1156
|
2085
|
-
* Fix step operators by @schustmi in https://github.com/zenml-io/zenml/pull/1155
|
2086
|
-
* Display correct name when updating a stack component by @schustmi in https://github.com/zenml-io/zenml/pull/1160
|
2087
|
-
* Update mysql database creation by @schustmi in https://github.com/zenml-io/zenml/pull/1152
|
2088
|
-
* Adding component conditions to experiment tracker examples and adding to the environmental variable docs by @bcdurak in https://github.com/zenml-io/zenml/pull/1162
|
2089
|
-
* Increase dependency range for protobuf by @schustmi in https://github.com/zenml-io/zenml/pull/1163
|
2090
|
-
* Scheduling documentation by @strickvl in https://github.com/zenml-io/zenml/pull/1158
|
2091
|
-
* Adding scheduling for Vertex Pipelines by @htahir1 in https://github.com/zenml-io/zenml/pull/1148
|
2092
|
-
* Fix alembic migration for sqlite<3.25 by @fa9r in https://github.com/zenml-io/zenml/pull/1165
|
2093
|
-
* Fix pandas Series materializer by @jordandelbar in https://github.com/zenml-io/zenml/pull/1146
|
2094
|
-
|
2095
|
-
## New Contributors
|
2096
|
-
* @jordandelbar made their first contribution in https://github.com/zenml-io/zenml/pull/1146
|
2097
|
-
|
2098
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.23.0...0.30.0
|
2099
|
-
|
2100
|
-
# 0.23.0
|
2101
|
-
|
2102
|
-
This release comes with a brand-new Neptune integration to track your ML experiments
|
2103
|
-
as well as lots of performance improvements!
|
2104
|
-
|
2105
|
-
## Neptune integration
|
2106
|
-
|
2107
|
-
The new [Neptune integration](https://github.com/zenml-io/zenml/tree/main/examples/neptune_tracking)
|
2108
|
-
includes a Neptune experiment tracker component that allows you to track your machine learning experiments
|
2109
|
-
using Neptune.
|
2110
|
-
|
2111
|
-
## Performance Optimization
|
2112
|
-
|
2113
|
-
The 0.20.0 release introduced our new server but brought with it a few performance and scalability issues.
|
2114
|
-
Since then, we've made many improvements to it, and this release is the final and biggest boost in performance. We reduced the amount of server calls needed for almost all CLI commands and greatly improved the speed of the dashboard as well.
|
2115
|
-
|
2116
|
-
## PyArrow dependency removal
|
2117
|
-
|
2118
|
-
We've removed PyArrow as a dependency of the `zenml` python package.
|
2119
|
-
As a consequence of that, our NumPy and Pandas materializer no
|
2120
|
-
longer read and write their artifacts using PyArrow but instead use
|
2121
|
-
native formats instead. If you still want to use PyArrow to serialize
|
2122
|
-
your NumPy arrays and Pandas dataframes, you'll need to install it manually
|
2123
|
-
like this: `pip install pyarrow`
|
2124
|
-
|
2125
|
-
In future releases we'll get rid of other unnecessary dependencies to
|
2126
|
-
further slim down the `zenml` package.
|
2127
|
-
|
2128
|
-
## Breaking Changes
|
2129
|
-
|
2130
|
-
The following changes introduces with this release may require some manual
|
2131
|
-
intervention to update your current installations:
|
2132
|
-
|
2133
|
-
- If your code calls some methods of our `Client` class, it might need to be
|
2134
|
-
updated to the new model classes introduced by the performance optimization changes
|
2135
|
-
explained above
|
2136
|
-
- The CLI command to remove an attribute from a stack component now takes no more dashes
|
2137
|
-
in front of the attribute names:
|
2138
|
-
`zenml stack-component remove-attribute <COMPONENT_NAME> <ATTRIBUTE_NAME>`
|
2139
|
-
- If you're using a custom stack component and have overridden the `cleanup_step_run` method,
|
2140
|
-
you'll need to update the method signature to include a `step_failed` parameter.
|
2141
|
-
|
2142
|
-
## What's Changed
|
2143
|
-
* Docs regarding roles and permissions by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1081
|
2144
|
-
* Add global config dir to `zenml status` by @schustmi in https://github.com/zenml-io/zenml/pull/1084
|
2145
|
-
* Remove source pins and ignore source pins during step spec comparisons by @schustmi in https://github.com/zenml-io/zenml/pull/1083
|
2146
|
-
* Docs/links for roles permissions by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1091
|
2147
|
-
* Bugfix/eng 1485 fix api docs build by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1089
|
2148
|
-
* fix bento builder step parameters to match bentoml by @safoinme in https://github.com/zenml-io/zenml/pull/1096
|
2149
|
-
* Add bentoctl to BentoML docs and example by @safoinme in https://github.com/zenml-io/zenml/pull/1094
|
2150
|
-
* Fix BaseParameters sample code in docs by @jcarlosgarcia in https://github.com/zenml-io/zenml/pull/1098
|
2151
|
-
* zenml <stack-component> logs defaults to active stack without name_or_id by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1101
|
2152
|
-
* Fixed evidently docs by @htahir1 in https://github.com/zenml-io/zenml/pull/1111
|
2153
|
-
* Update sagemaker default instance type by @schustmi in https://github.com/zenml-io/zenml/pull/1112
|
2154
|
-
* The ultimate optimization for performance by @bcdurak in https://github.com/zenml-io/zenml/pull/1077
|
2155
|
-
* Update stack exporting and importing by @schustmi in https://github.com/zenml-io/zenml/pull/1114
|
2156
|
-
* Fix readme by @schustmi in https://github.com/zenml-io/zenml/pull/1116
|
2157
|
-
* Remove Pyarrow dependency by @safoinme in https://github.com/zenml-io/zenml/pull/1109
|
2158
|
-
* Bugfix for listing the runs filtered by a name by @bcdurak in https://github.com/zenml-io/zenml/pull/1118
|
2159
|
-
* Neptune.ai integration by @AleksanderWWW in https://github.com/zenml-io/zenml/pull/1082
|
2160
|
-
* Add YouTube video explaining Stack Components Settings vs Config by @dnth in https://github.com/zenml-io/zenml/pull/1120
|
2161
|
-
* Add failed Status to component when step fails by @safoinme in https://github.com/zenml-io/zenml/pull/1115
|
2162
|
-
* Add architecture diagrams to docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1119
|
2163
|
-
* Remove local orchestrator restriction from step operator docs by @schustmi in https://github.com/zenml-io/zenml/pull/1122
|
2164
|
-
* Validate Stack Before Provision by @safoinme in https://github.com/zenml-io/zenml/pull/1110
|
2165
|
-
* Bugfix/fix endpoints for dashboard development by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1125
|
2166
|
-
* Skip kubeflow UI daemon provisioning if a hostname is configured by @schustmi in https://github.com/zenml-io/zenml/pull/1126
|
2167
|
-
* Update Neptune Example by @safoinme in https://github.com/zenml-io/zenml/pull/1124
|
2168
|
-
* Add debugging guide to docs by @dnth in https://github.com/zenml-io/zenml/pull/1097
|
2169
|
-
* Fix stack component attribute removal CLI command by @schustmi in https://github.com/zenml-io/zenml/pull/1127
|
2170
|
-
* Improving error messages when fetching entities by @bcdurak in https://github.com/zenml-io/zenml/pull/1117
|
2171
|
-
* Introduce username and password to kubeflow for more native multi-tenant support by @htahir1 in https://github.com/zenml-io/zenml/pull/1123
|
2172
|
-
* Add support for Label Studio OCR config generation by @shivalikasingh95 in https://github.com/zenml-io/zenml/pull/1062
|
2173
|
-
* Misc doc updates by @schustmi in https://github.com/zenml-io/zenml/pull/1131
|
2174
|
-
* Fix Neptune run cleanup by @safoinme in https://github.com/zenml-io/zenml/pull/1130
|
2175
|
-
|
2176
|
-
## New Contributors
|
2177
|
-
* @jcarlosgarcia made their first contribution in https://github.com/zenml-io/zenml/pull/1098
|
2178
|
-
* @AleksanderWWW made their first contribution in https://github.com/zenml-io/zenml/pull/1082
|
2179
|
-
* @shivalikasingh95 made their first contribution in https://github.com/zenml-io/zenml/pull/1062
|
2180
|
-
|
2181
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.22.0...0.23.0
|
2182
|
-
|
2183
|
-
# 0.22.0
|
2184
|
-
|
2185
|
-
The 0.22.0 release comes with a new BentoML integration as well as a reworked
|
2186
|
-
Airflow orchestrator. Additionally, it greatly improves the server performance
|
2187
|
-
as well as other small fixes and updates to our docs!
|
2188
|
-
|
2189
|
-
## BentoML integration
|
2190
|
-
|
2191
|
-
The new [BentoML integration](https://github.com/zenml-io/zenml/tree/main/examples/bentoml_deployment)
|
2192
|
-
includes a BentoML model deployer component that allows you to deploy
|
2193
|
-
your models from any of the major machine learning frameworks on your local machine.
|
2194
|
-
## Airflow orchestrator v2
|
2195
|
-
|
2196
|
-
The previous Airflow orchestrator was limited to running locally and had many
|
2197
|
-
additional unpleasant constraints that made it hard to work with. This
|
2198
|
-
release includes a completely rewritten, new version of the Airflow orchestrator
|
2199
|
-
that now relies on Docker images to run your pipelines and works both locally
|
2200
|
-
and with remote Airflow deployments.
|
2201
|
-
|
2202
|
-
## Notable bugfixes
|
2203
|
-
|
2204
|
-
- Further improvements to the synchronization that transfers pipeline run information from
|
2205
|
-
the MLMD database to the ZenML Server.
|
2206
|
-
- The ZenML Label Studio integration can now be used with non-local (i.e.
|
2207
|
-
deployed) instances. For more information see [the Label Studiodocs](https://docs.zenml.io/component-gallery/annotators/label-studio).
|
2208
|
-
- The Spark example is fixed and now works again end-to-end.
|
2209
|
-
|
2210
|
-
## Breaking Changes
|
2211
|
-
|
2212
|
-
The following changes introduces with this release may require some manual
|
2213
|
-
intervention to update your current installations:
|
2214
|
-
|
2215
|
-
* the Airflow orchestrator now requires a newer version of Airflow
|
2216
|
-
(run `zenml integration install airflow` to upgrade) and Docker installed
|
2217
|
-
to work.
|
2218
|
-
|
2219
|
-
## What's Changed
|
2220
|
-
* Fix bug when running non-local annotator instance. by @sheikhomar in https://github.com/zenml-io/zenml/pull/1045
|
2221
|
-
* Introduce Permissions, Link Permissions to Roles, Restrict Access to endpoints based on Permission by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1007
|
2222
|
-
* Fix copy-pasted log message for annotator by @strickvl in https://github.com/zenml-io/zenml/pull/1049
|
2223
|
-
* Add warning message for client server version mismatch by @schustmi in https://github.com/zenml-io/zenml/pull/1047
|
2224
|
-
* Fix path to ingress values in ZenServer recipes by @wjayesh in https://github.com/zenml-io/zenml/pull/1053
|
2225
|
-
* Prevent deletion/update of default entities by @stefannica in https://github.com/zenml-io/zenml/pull/1046
|
2226
|
-
* Fix Publish API docs workflow by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1054
|
2227
|
-
* Fix multiple alembic heads warning by @fa9r in https://github.com/zenml-io/zenml/pull/1051
|
2228
|
-
* Fix Null Step Configuration/Parameters Error by @fa9r in https://github.com/zenml-io/zenml/pull/1050
|
2229
|
-
* Fix role permission migration by @schustmi in https://github.com/zenml-io/zenml/pull/1056
|
2230
|
-
* Made role assignment/revokation possible through zen_server by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1059
|
2231
|
-
* Bugfix/make role assignment work with enum by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1063
|
2232
|
-
* Manually set scoped for each endpoint by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1064
|
2233
|
-
* Add run args to local docker orchestrator settings by @schustmi in https://github.com/zenml-io/zenml/pull/1060
|
2234
|
-
* Docker ZenML deployment improvements and docs by @stefannica in https://github.com/zenml-io/zenml/pull/1061
|
2235
|
-
* Bugfix Mlflow service cleanup configuration by @safoinme in https://github.com/zenml-io/zenml/pull/1067
|
2236
|
-
* Rename DB Tables and Fix Foreign Keys by @fa9r in https://github.com/zenml-io/zenml/pull/1058
|
2237
|
-
* Paginate secrets in `AWSSecretsManager` by @chiragjn in https://github.com/zenml-io/zenml/pull/1057
|
2238
|
-
* Add explicit dashboard docs by @strickvl in https://github.com/zenml-io/zenml/pull/1052
|
2239
|
-
* Added GA and Gitlab to envs by @htahir1 in https://github.com/zenml-io/zenml/pull/1068
|
2240
|
-
* Add Inference Server Predictor to KServe and Seldon Docs by @safoinme in https://github.com/zenml-io/zenml/pull/1048
|
2241
|
-
* Rename project table to workspace by @fa9r in https://github.com/zenml-io/zenml/pull/1073
|
2242
|
-
* Airflow orchestrator v2 by @schustmi in https://github.com/zenml-io/zenml/pull/1042
|
2243
|
-
* Add get_or_create_run() ZenStore method by @fa9r in https://github.com/zenml-io/zenml/pull/1070
|
2244
|
-
* Fix the flaky fileio tests by @schustmi in https://github.com/zenml-io/zenml/pull/1072
|
2245
|
-
* BentoML Deployer Integration by @safoinme in https://github.com/zenml-io/zenml/pull/1044
|
2246
|
-
* Sync Speedup by @fa9r in https://github.com/zenml-io/zenml/pull/1055
|
2247
|
-
* Fixed broken links in docs and examples. by @dnth in https://github.com/zenml-io/zenml/pull/1076
|
2248
|
-
* Make additional stack component config options available as a setting by @schustmi in https://github.com/zenml-io/zenml/pull/1069
|
2249
|
-
* Rename `step_run_artifact` table to `step_run_input_artifact` by @fa9r in https://github.com/zenml-io/zenml/pull/1075
|
2250
|
-
* Update Spark Example to ZenML post 0.20.0 by @safoinme in https://github.com/zenml-io/zenml/pull/1071
|
2251
|
-
* Always set caching to false for all Kubeflow based orchestrators by @schustmi in https://github.com/zenml-io/zenml/pull/1079
|
2252
|
-
* Feature/eng 1402 consolidate stack sharing by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1036
|
2253
|
-
|
2254
|
-
## New Contributors
|
2255
|
-
* @sheikhomar made their first contribution in https://github.com/zenml-io/zenml/pull/1045
|
2256
|
-
* @chiragjn made their first contribution in https://github.com/zenml-io/zenml/pull/1057
|
2257
|
-
|
2258
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.21.1...0.22.0
|
2259
|
-
|
2260
|
-
# 0.21.1
|
2261
|
-
|
2262
|
-
This is an ad-hoc release to fix some bugs introduced the 0.21.0 release that
|
2263
|
-
made the local ZenML dashboard unusable.
|
2264
|
-
|
2265
|
-
## What's Changed
|
2266
|
-
* Include latest (not oldest) three runs in HydratedPipelineModel by @schustmi in https://github.com/zenml-io/zenml/pull/1039
|
2267
|
-
* Update docs to use `pip install [server]` by @strickvl in https://github.com/zenml-io/zenml/pull/1037
|
2268
|
-
* Docs fix for Deepchecks by @strickvl in https://github.com/zenml-io/zenml/pull/1040
|
2269
|
-
* Fix the pipeline run sync on sqlite and the --blocking zenml server deployment by @stefannica in https://github.com/zenml-io/zenml/pull/1041
|
2270
|
-
|
2271
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.21.0...0.21.1
|
2272
|
-
|
2273
|
-
# 0.21.0
|
2274
|
-
|
2275
|
-
This release primarily fixes a number of bugs that were introduced as part of
|
2276
|
-
the 0.20.0 ZenServer release. These significantly improve the stability when
|
2277
|
-
using ZenML with the ZenML Server.
|
2278
|
-
|
2279
|
-
Notable fixes include:
|
2280
|
-
|
2281
|
-
- Improved the synchronization that transfers pipeline run information from
|
2282
|
-
the MLMD database to the ZenML Server. This helps fix a number of issues with
|
2283
|
-
missing steps in the post-execution workflow, model deployment steps and other
|
2284
|
-
issues.
|
2285
|
-
- The Label Studio example is fixed and now works again end-to-end.
|
2286
|
-
- The ZenML Label Studio integration can now be used with non-local (i.e.
|
2287
|
-
deployed) instances. For more information see [the Label Studiodocs](https://docs.zenml.io/component-gallery/annotators/label-studio).
|
2288
|
-
|
2289
|
-
New features and other improvements:
|
2290
|
-
|
2291
|
-
- ZenML now uses [alembic](https://alembic.sqlalchemy.org/en/latest/) for
|
2292
|
-
automated database migrations. The migrations happen automatically after every
|
2293
|
-
ZenML update.
|
2294
|
-
- New `zenml pipeline runs export / import / migrate` CLI commands are now
|
2295
|
-
available to export, import and migrate pipeline runs from older, pre-0.20.0
|
2296
|
-
versions of ZenML. The ZenML server now also automatically picks up older
|
2297
|
-
pipeline runs that have been logged in the metadata store by ZenML prior to
|
2298
|
-
0.20.0.
|
2299
|
-
- An MLMD gRPC service can now be deployed with the ZenML Helm chart to act
|
2300
|
-
as a proxy between clients, orchestrators and the MySQL database. This
|
2301
|
-
significantly reduces the time it takes to run pipelines locally.
|
2302
|
-
- You can now specify affinity and tolerations and node selectors to all
|
2303
|
-
Kubernetes based orchestrators with the new Kubernetes Pod settings feature.
|
2304
|
-
|
2305
|
-
|
2306
|
-
## Breaking Changes
|
2307
|
-
|
2308
|
-
The following changes introduces with this release may require some manual
|
2309
|
-
intervention to update your current installations:
|
2310
|
-
|
2311
|
-
* the zenml server helm chart `values.yaml` file has been restructured to make
|
2312
|
-
it easier to configure and to clearly distinguish between the zenml server
|
2313
|
-
component and the newly introduced gRPC service component. Please update your
|
2314
|
-
`values.yaml` copies accordingly.
|
2315
|
-
* the Azure integration dependency versions have been updated. Please run
|
2316
|
-
`zenml integration install azure` to update your current installation, if
|
2317
|
-
you're using Azure.
|
2318
|
-
|
2319
|
-
|
2320
|
-
## What's Changed
|
2321
|
-
* Implement automatic alembic migration by @AlexejPenner in https://github.com/zenml-io/zenml/pull/990
|
2322
|
-
* Fix GCP Artifact Store listdir empty path by @safoinme in https://github.com/zenml-io/zenml/pull/998
|
2323
|
-
* Add flavors mini-video to docs by @strickvl in https://github.com/zenml-io/zenml/pull/999
|
2324
|
-
* Remove the Client() warning when used inside a step by @stefannica in https://github.com/zenml-io/zenml/pull/1000
|
2325
|
-
* Fix broken links caused by updated by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1002
|
2326
|
-
* Fix `FileNotFoundError` with remote path in HuggingFace Dataset materializer by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/995
|
2327
|
-
* Add `zenml pipeline runs export / import / migrate` CLI commands by @fa9r in https://github.com/zenml-io/zenml/pull/977
|
2328
|
-
* Log message when activating a stack as part of registration by @schustmi in https://github.com/zenml-io/zenml/pull/1005
|
2329
|
-
* Minor fixes in Migration to 0.20.0 documentation by @alvarobartt in https://github.com/zenml-io/zenml/pull/1009
|
2330
|
-
* Doc updates by @htahir1 in https://github.com/zenml-io/zenml/pull/1006
|
2331
|
-
* Fixing broken links in docs by @dnth in https://github.com/zenml-io/zenml/pull/1018
|
2332
|
-
* Label Studio example fix by @strickvl in https://github.com/zenml-io/zenml/pull/1021
|
2333
|
-
* Docs for using CUDA-enabled docker images by @strickvl in https://github.com/zenml-io/zenml/pull/1010
|
2334
|
-
* Add social media heading on docs page by @dnth in https://github.com/zenml-io/zenml/pull/1020
|
2335
|
-
* Add executing custom command for getting requirements by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/1012
|
2336
|
-
* Delay user instruction in dockerfile generation by @schustmi in https://github.com/zenml-io/zenml/pull/1004
|
2337
|
-
* Update link checker configs for faster, more accurate checks by @dnth in https://github.com/zenml-io/zenml/pull/1022
|
2338
|
-
* Add `pip install zenml[server]` to relevant examples by @dnth in https://github.com/zenml-io/zenml/pull/1027
|
2339
|
-
* Add Tolerations and NodeAffinity to Kubernetes executor by @wefner in https://github.com/zenml-io/zenml/pull/994
|
2340
|
-
* Support pydantic subclasses in BaseParameter attributes by @schustmi in https://github.com/zenml-io/zenml/pull/1023
|
2341
|
-
* Unify run names across orchestrators by @schustmi in https://github.com/zenml-io/zenml/pull/1025
|
2342
|
-
* Add gRPC metadata service to the ZenML helm chart by @stefannica in https://github.com/zenml-io/zenml/pull/1026
|
2343
|
-
* Make the MLMD pipeline run information transfer synchronous by @stefannica in https://github.com/zenml-io/zenml/pull/1032
|
2344
|
-
* Add console spinner back by @strickvl in https://github.com/zenml-io/zenml/pull/1034
|
2345
|
-
* Fix Azure CLI auth problem by @wjayesh in https://github.com/zenml-io/zenml/pull/1035
|
2346
|
-
* Allow non-local Label Studio instances for annotation by @strickvl in https://github.com/zenml-io/zenml/pull/1033
|
2347
|
-
* Before deleting the global zen_server files, spin it down by @AlexejPenner in https://github.com/zenml-io/zenml/pull/1029
|
2348
|
-
* Adding zenserver integration to stack recipe CLI by @wjayesh in https://github.com/zenml-io/zenml/pull/1017
|
2349
|
-
* Add support for Azure ZenServer by @wjayesh in https://github.com/zenml-io/zenml/pull/1024
|
2350
|
-
* Kubernetes Pod settings by @schustmi in https://github.com/zenml-io/zenml/pull/1008
|
2351
|
-
|
2352
|
-
## New Contributors
|
2353
|
-
* @alvarobartt made their first contribution in https://github.com/zenml-io/zenml/pull/1009
|
2354
|
-
* @wefner made their first contribution in https://github.com/zenml-io/zenml/pull/994
|
2355
|
-
|
2356
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.5...0.21.0
|
2357
|
-
|
2358
|
-
# 0.20.5
|
2359
|
-
|
2360
|
-
ZenML 0.20.5 fixes another series of minor bugs, significantly improves the performance of the CLI, and adds an option to specify APT packages in Docker images.
|
2361
|
-
|
2362
|
-
## What's Changed
|
2363
|
-
* Fix accessing local zen store and artifact store in containers by @stefannica in https://github.com/zenml-io/zenml/pull/976
|
2364
|
-
* K3d local registry pod spec updated by @wjayesh in https://github.com/zenml-io/zenml/pull/972
|
2365
|
-
* Update readme page by @dnth in https://github.com/zenml-io/zenml/pull/985
|
2366
|
-
* Remove beam dependency by @schustmi in https://github.com/zenml-io/zenml/pull/986
|
2367
|
-
* Fix error message when registering secret without secrets manager by @schustmi in https://github.com/zenml-io/zenml/pull/981
|
2368
|
-
* Update cheat sheet up to `zenml==0.20.4` by @dnth in https://github.com/zenml-io/zenml/pull/987
|
2369
|
-
* Example fixes (part 2) by @strickvl in https://github.com/zenml-io/zenml/pull/971
|
2370
|
-
* Allow duplicate step classes inside a pipeline by @schustmi in https://github.com/zenml-io/zenml/pull/989
|
2371
|
-
* Include deployment in azureml docker build by @schustmi in https://github.com/zenml-io/zenml/pull/984
|
2372
|
-
* Automatically open browser upon `zenml up` command by @dnth in https://github.com/zenml-io/zenml/pull/978
|
2373
|
-
* Add a `just_mine` flag for `zenml stack list` by @strickvl in https://github.com/zenml-io/zenml/pull/979
|
2374
|
-
* Add option to specify apt packages by @schustmi in https://github.com/zenml-io/zenml/pull/982
|
2375
|
-
* Replace old flavor references, fix the windows local ZenML server and other fixes by @stefannica in https://github.com/zenml-io/zenml/pull/988
|
2376
|
-
* Improve docker and k8s detection by @schustmi in https://github.com/zenml-io/zenml/pull/991
|
2377
|
-
* Update GH actions example by @schustmi in https://github.com/zenml-io/zenml/pull/993
|
2378
|
-
* Update `MissingStepParameterError` exception message by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/996
|
2379
|
-
* Separated code docs into `core` and `integration` docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/983
|
2380
|
-
* Add docs/mkdocstrings_helper.py to format script sources by @fa9r in https://github.com/zenml-io/zenml/pull/997
|
2381
|
-
* Further CLI optimization by @bcdurak in https://github.com/zenml-io/zenml/pull/992
|
2382
|
-
|
2383
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.4...0.20.5
|
2384
|
-
|
2385
|
-
|
2386
|
-
# 0.20.4
|
2387
|
-
|
2388
|
-
This release fixes another series of minor bugs that were introduced in 0.20.0.
|
2389
|
-
|
2390
|
-
## What's Changed
|
2391
|
-
* Detect failed executions by @schustmi in https://github.com/zenml-io/zenml/pull/964
|
2392
|
-
* Only build docker images for custom deployments by @schustmi in https://github.com/zenml-io/zenml/pull/960
|
2393
|
-
* M1 Mac Installation Tutorial by @fa9r in https://github.com/zenml-io/zenml/pull/966
|
2394
|
-
* Update ZenBytes links in docs by @fa9r in https://github.com/zenml-io/zenml/pull/968
|
2395
|
-
* Fix the API docs builder by @stefannica in https://github.com/zenml-io/zenml/pull/967
|
2396
|
-
* Fix `gpu_limit` condition in `VertexOrchestrator` by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/963
|
2397
|
-
* Add simple node affinity configurations by @schustmi in https://github.com/zenml-io/zenml/pull/973
|
2398
|
-
* First iteration of the CLI optimization by @bcdurak in https://github.com/zenml-io/zenml/pull/962
|
2399
|
-
|
2400
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.3...0.20.4
|
2401
|
-
|
2402
|
-
# 0.20.3
|
2403
|
-
|
2404
|
-
This release fixes another series of minor bugs that were introduced in 0.20.0.
|
2405
|
-
|
2406
|
-
## What's Changed
|
2407
|
-
* Fixed GitHub/Colab JSON formatting error on quickstart. by @fa9r in https://github.com/zenml-io/zenml/pull/947
|
2408
|
-
* Update YAML config template by @htahir1 in https://github.com/zenml-io/zenml/pull/952
|
2409
|
-
* correct code from merge and fix import by @wjayesh in https://github.com/zenml-io/zenml/pull/950
|
2410
|
-
* Check for active component using id instead of name by @schustmi in https://github.com/zenml-io/zenml/pull/956
|
2411
|
-
* Tekton fix by @htahir1 in https://github.com/zenml-io/zenml/pull/955
|
2412
|
-
* Improve zenml up/down UX and other fixes by @stefannica in https://github.com/zenml-io/zenml/pull/957
|
2413
|
-
* Update kubeflow docs for multi-tenant deployments by @htahir1 in https://github.com/zenml-io/zenml/pull/958
|
2414
|
-
* Update kubeflow.md by @abohmeed in https://github.com/zenml-io/zenml/pull/959
|
2415
|
-
* Add additional stack validation for step operators by @schustmi in https://github.com/zenml-io/zenml/pull/954
|
2416
|
-
* Fix pipeline run dashboard URL for unlisted runs by @fa9r in https://github.com/zenml-io/zenml/pull/951
|
2417
|
-
* Support subclasses of registered types in recursive materialization by @fa9r in https://github.com/zenml-io/zenml/pull/953
|
2418
|
-
|
2419
|
-
## New Contributors
|
2420
|
-
* @abohmeed made their first contribution in https://github.com/zenml-io/zenml/pull/959
|
2421
|
-
|
2422
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.2...0.20.3
|
2423
|
-
|
2424
|
-
# 0.20.2
|
2425
|
-
|
2426
|
-
After a successful release of the new ZenML server and dashboard paradigm, we set to ironing out some bugs that slipped through.
|
2427
|
-
|
2428
|
-
## What's Changed
|
2429
|
-
* Capitalize all docs page titles. by @fa9r in https://github.com/zenml-io/zenml/pull/937
|
2430
|
-
* Increase field sizes for docstrings and step parameters. by @fa9r in https://github.com/zenml-io/zenml/pull/940
|
2431
|
-
* Fixing the bug in the registration of custom flavors by @bcdurak in https://github.com/zenml-io/zenml/pull/938
|
2432
|
-
* Implemented `docstring` Attribute of StepModel by @fa9r in https://github.com/zenml-io/zenml/pull/936
|
2433
|
-
* Fix shared stack emoji by @strickvl in https://github.com/zenml-io/zenml/pull/941
|
2434
|
-
* Fix shared stacks not being allowed to be set as active. by @fa9r in https://github.com/zenml-io/zenml/pull/943
|
2435
|
-
* Typo fix by @strickvl in https://github.com/zenml-io/zenml/pull/944
|
2436
|
-
* Update Kubernetes Orchestrator Example by @fa9r in https://github.com/zenml-io/zenml/pull/942
|
2437
|
-
* Add code and instructions to run quickstart on Colab. by @fa9r in https://github.com/zenml-io/zenml/pull/939
|
2438
|
-
* Fixing the interaction in getting stacks/components by @bcdurak in https://github.com/zenml-io/zenml/pull/945
|
2439
|
-
* Fix Kubeflow run name by @safoinme in https://github.com/zenml-io/zenml/pull/946
|
2440
|
-
* `VertexOrchestrator` apply node selector constraint if `gpu_limit > 0` by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/935
|
2441
|
-
|
2442
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.20.1...0.20.2
|
2443
|
-
|
2444
|
-
# 0.20.0 / 0.20.1
|
2445
|
-
|
2446
|
-
The ZenML 0.20.0 release brings a number of big changes to its architecture and
|
2447
|
-
a lot of cool new features, some of which are not backwards compatible with
|
2448
|
-
previous versions.
|
2449
|
-
|
2450
|
-
These changes are only covered briefly in the release notes. For a detailed
|
2451
|
-
view on what happened and how you can get the most out of the 0.20.0 release,
|
2452
|
-
please head over to [our "ZenML 0.20.0: Our Biggest Release Yet" blog post](https://blog.zenml.io/zenml-revamped).
|
2453
|
-
|
2454
|
-
## Warning: Breaking Changes
|
2455
|
-
|
2456
|
-
Updating to ZenML 0.20.0 needs to be followed by a migration of your existing
|
2457
|
-
ZenML Stacks and you may also need to make changes to your current ZenML
|
2458
|
-
pipeline code. Please read [the migration guide](https://docs.zenml.io/guidelines/migration-zero-twenty) carefully and follow the
|
2459
|
-
instructions to ensure a smooth transition. The guide walks you through these
|
2460
|
-
changes and offers instructions on how to migrate your existing ZenML stacks and
|
2461
|
-
pipelines to the new version with minimal effort and disruption to your existing
|
2462
|
-
workloads.
|
2463
|
-
|
2464
|
-
If you have updated to ZenML 0.20.0 by mistake or are experiencing issues with
|
2465
|
-
the new version, you can always go back to the previous version by using
|
2466
|
-
`pip install zenml==0.13.2` instead of `pip install zenml` when installing
|
2467
|
-
ZenML manually or in your scripts.
|
2468
|
-
|
2469
|
-
## Overview of Changes
|
2470
|
-
|
2471
|
-
* [ZenML takes over the Metadata Store](https://docs.zenml.io/guidelines/migration-zero-twenty#zenml-takes-over-the-metadata-store-role)
|
2472
|
-
role. All information about your ZenML Stacks, pipelines, and artifacts is now
|
2473
|
-
tracked by ZenML itself directly. If you are currently using remote Metadata
|
2474
|
-
Stores (e.g. deployed in cloud) in your stacks, you will probably need to
|
2475
|
-
replace them with [ZenML cloud deployments](https://docs.zenml.io/guidelines/migration-zero-twenty/getting-started/deploying-zenml/deploying-zenml.md).
|
2476
|
-
* the [new ZenML Dashboard](https://docs.zenml.io/guidelines/migration-zero-twenty#the-zenml-dashboard-is-now-available) is now
|
2477
|
-
available with all ZenML deployments.
|
2478
|
-
* [ZenML Profiles have been removed](https://docs.zenml.io/guidelines/migration-zero-twenty#removal-of-profiles-and-the-local-yaml-database)
|
2479
|
-
in favor of ZenML Projects. You need to
|
2480
|
-
[manually migrate your existing ZenML Profiles](https://docs.zenml.io/guidelines/migration-zero-twenty#how-to-migrate-your-profiles)
|
2481
|
-
after the update.
|
2482
|
-
* the [configuration of Stack Components is now decoupled from their implementation](https://docs.zenml.io/guidelines/migration-zero-twenty#decoupling-stack-component-configuration-from-implementation).
|
2483
|
-
If you extended ZenML with custom stack component implementations, you may need
|
2484
|
-
to update the way they are registered in ZenML.
|
2485
|
-
* the updated ZenML server provides a new and improved collaborative experience.
|
2486
|
-
When connected to a ZenML server, you can now [share your ZenML Stacks and Stack Components](https://docs.zenml.io/guidelines/migration-zero-twenty#shared-zenml-stacks-and-stack-components) with other users. If you were
|
2487
|
-
previously using the ZenML Profiles or the ZenML server to share your ZenML
|
2488
|
-
Stacks, you should switch to the new ZenML server and Dashboard and update your
|
2489
|
-
existing workflows to reflect the new features.
|
2490
|
-
|
2491
|
-
## What's Changed
|
2492
|
-
* Fix error in checking Great Expectations results when exit_on_error=True by @TimovNiedek in https://github.com/zenml-io/zenml/pull/889
|
2493
|
-
* feat(user-dockerfile): Add user argument to DockerConfiguration by @cjidboon94 in https://github.com/zenml-io/zenml/pull/892
|
2494
|
-
* Minor doc updates for backporting by @htahir1 in https://github.com/zenml-io/zenml/pull/894
|
2495
|
-
* Removed feature request and replaced with hellonext board by @htahir1 in https://github.com/zenml-io/zenml/pull/897
|
2496
|
-
* Unit tests for (some) integrations by @strickvl in https://github.com/zenml-io/zenml/pull/880
|
2497
|
-
* Fixed integration installation command by @edshee in https://github.com/zenml-io/zenml/pull/900
|
2498
|
-
* Pipeline configuration and intermediate representation by @schustmi in https://github.com/zenml-io/zenml/pull/898
|
2499
|
-
* [Bugfix] Fix bug in auto-import of stack after recipe deploy by @wjayesh in https://github.com/zenml-io/zenml/pull/901
|
2500
|
-
* Update TOC on CONTRIBUTING.md by @strickvl in https://github.com/zenml-io/zenml/pull/907
|
2501
|
-
* ZenServer by @fa9r in https://github.com/zenml-io/zenml/pull/879
|
2502
|
-
* Update `kserve` README by @strickvl in https://github.com/zenml-io/zenml/pull/912
|
2503
|
-
* Confirmation prompts were not working by @htahir1 in https://github.com/zenml-io/zenml/pull/917
|
2504
|
-
* Stacks can be registered in `Click<8.0.0` now by @AlexejPenner in https://github.com/zenml-io/zenml/pull/920
|
2505
|
-
* Made Pipeline and Stack optional on the HydratedPipelineRunModel by @AlexejPenner in https://github.com/zenml-io/zenml/pull/919
|
2506
|
-
* Renamed all references from ZenServer to ZenML Server in logs and comments by @htahir1 in https://github.com/zenml-io/zenml/pull/915
|
2507
|
-
* Prettify pipeline runs list CLI output. by @fa9r in https://github.com/zenml-io/zenml/pull/921
|
2508
|
-
* Warn when registering non-local component with local ZenServer by @strickvl in https://github.com/zenml-io/zenml/pull/904
|
2509
|
-
* Fix duplicate results in pipeline run lists and unlisted flag. by @fa9r in https://github.com/zenml-io/zenml/pull/922
|
2510
|
-
* Fix error log by @htahir1 in https://github.com/zenml-io/zenml/pull/916
|
2511
|
-
* Update cli docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/913
|
2512
|
-
* Fix Pipeline Run Status by @fa9r in https://github.com/zenml-io/zenml/pull/923
|
2513
|
-
* Change the CLI emoji for whether a stack is shared or not. by @fa9r in https://github.com/zenml-io/zenml/pull/926
|
2514
|
-
* Fix running pipelines from different locations. by @fa9r in https://github.com/zenml-io/zenml/pull/925
|
2515
|
-
* Fix zenml stack-component describe CLI command. by @fa9r in https://github.com/zenml-io/zenml/pull/929
|
2516
|
-
* Update custom deployment to use ArtifactModel by @safoinme in https://github.com/zenml-io/zenml/pull/928
|
2517
|
-
* Fix the CI unit test and integration test failures by @stefannica in https://github.com/zenml-io/zenml/pull/924
|
2518
|
-
* Add gcp zenserver recipe by @wjayesh in https://github.com/zenml-io/zenml/pull/930
|
2519
|
-
* Extend Post Execution Class Properties by @fa9r in https://github.com/zenml-io/zenml/pull/931
|
2520
|
-
* Fixes for examples by @strickvl in https://github.com/zenml-io/zenml/pull/918
|
2521
|
-
* Update cheat sheet by @dnth in https://github.com/zenml-io/zenml/pull/932
|
2522
|
-
* Fix the docstring attribute of pipeline models. by @fa9r in https://github.com/zenml-io/zenml/pull/933
|
2523
|
-
* New docs post ZenML Server by @htahir1 in https://github.com/zenml-io/zenml/pull/927
|
2524
|
-
|
2525
|
-
## New Contributors
|
2526
|
-
* @TimovNiedek made their first contribution in https://github.com/zenml-io/zenml/pull/889
|
2527
|
-
* @cjidboon94 made their first contribution in https://github.com/zenml-io/zenml/pull/892
|
2528
|
-
* @edshee made their first contribution in https://github.com/zenml-io/zenml/pull/900
|
2529
|
-
|
2530
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.13.2...0.20.0
|
2531
|
-
|
2532
|
-
# 0.13.2
|
2533
|
-
|
2534
|
-
ZenML 0.13.2 comes with a new local Docker orchestrator and many other improvements and fixes:
|
2535
|
-
* You can now run your pipelines locally in isolated Docker containers per step
|
2536
|
-
* @gabrielmbmb updated our MLFlow experiment tracker to work with Databricks deployments 🎉
|
2537
|
-
* Documentation updates for cloud deployments and multi-tenancy Kubeflow support
|
2538
|
-
|
2539
|
-
## What's Changed
|
2540
|
-
* Update GitHub Actions by @fa9r in https://github.com/zenml-io/zenml/pull/864
|
2541
|
-
* Raise zenml exception when cyclic graph is detected by @schustmi in https://github.com/zenml-io/zenml/pull/866
|
2542
|
-
* Add source to segment identify call by @htahir1 in https://github.com/zenml-io/zenml/pull/868
|
2543
|
-
* Use default local paths/URIs for the local artifact and metadata stores by @stefannica in https://github.com/zenml-io/zenml/pull/873
|
2544
|
-
* Implement local docker orchestrator by @schustmi in https://github.com/zenml-io/zenml/pull/862
|
2545
|
-
* Update cheat sheet with latest CLI commands from 0.13.0 by @dnth in https://github.com/zenml-io/zenml/pull/867
|
2546
|
-
* Add a note about importing proper DockerConfiguration module by @jsuchome in https://github.com/zenml-io/zenml/pull/877
|
2547
|
-
* Bugfix/misc by @schustmi in https://github.com/zenml-io/zenml/pull/878
|
2548
|
-
* Fixed bug in tfx by @htahir1 in https://github.com/zenml-io/zenml/pull/883
|
2549
|
-
* Mlflow Databricks connection by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/882
|
2550
|
-
* Refactor cloud guide to stack deployment guide by @wjayesh in https://github.com/zenml-io/zenml/pull/861
|
2551
|
-
* Add cookie consent by @strickvl in https://github.com/zenml-io/zenml/pull/871
|
2552
|
-
* Stack recipe CLI improvements by @wjayesh in https://github.com/zenml-io/zenml/pull/872
|
2553
|
-
* Kubeflow workaround added by @htahir1 in https://github.com/zenml-io/zenml/pull/886
|
2554
|
-
|
2555
|
-
|
2556
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.13.1...0.13.2
|
2557
|
-
|
2558
|
-
# 0.13.1
|
2559
|
-
|
2560
|
-
ZenML 0.13.1 is here and it comes with several quality of life improvements:
|
2561
|
-
|
2562
|
-
* You can now specify the exact order in which your pipelines steps should be
|
2563
|
-
executed, e.g., via `step_b.after(step_a)`
|
2564
|
-
* TensorBoard was moved to a separate integration so you can use it with Pytorch
|
2565
|
-
and other modeling frameworks
|
2566
|
-
* You can now configure the Evidently integration to ignore specific columns in
|
2567
|
-
your datasets.
|
2568
|
-
|
2569
|
-
This release also contains a lot of documentation on how to deploy
|
2570
|
-
custom code (like preprocessing and postprocessing code) with our KServe and
|
2571
|
-
Seldon integrations.
|
2572
|
-
|
2573
|
-
## What's Changed
|
2574
|
-
* Fix flag info on recipes in docs by @wjayesh in https://github.com/zenml-io/zenml/pull/854
|
2575
|
-
* Fix some materializer issues by @schustmi in https://github.com/zenml-io/zenml/pull/852
|
2576
|
-
* Add ignore columns for evidently drift detection by @SangamSwadiK in https://github.com/zenml-io/zenml/pull/851
|
2577
|
-
* TensorBoard Integration by @fa9r in https://github.com/zenml-io/zenml/pull/850
|
2578
|
-
* Add option to specify task dependencies by @schustmi in https://github.com/zenml-io/zenml/pull/858
|
2579
|
-
* Custom code readme and docs by @safoinme in https://github.com/zenml-io/zenml/pull/853
|
2580
|
-
|
2581
|
-
## New Contributors
|
2582
|
-
* @SangamSwadiK made their first contribution in https://github.com/zenml-io/zenml/pull/851
|
2583
|
-
|
2584
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.13.0...0.13.1
|
2585
|
-
|
2586
|
-
|
2587
|
-
# 0.13.0
|
2588
|
-
|
2589
|
-
ZenML version 0.13.0 is chock-full with exciting features.
|
2590
|
-
|
2591
|
-
[Custom Code Deployment](https://github.com/zenml-io/zenml/tree/main/examples/custom_code_deployment) is the
|
2592
|
-
continuation of the Model Deployment story that we have been working on over the last few releases. Now
|
2593
|
-
it is possible to deploy custom code along with your models using Kserve or Seldon.
|
2594
|
-
|
2595
|
-
With [Spark](https://github.com/zenml-io/zenml/tree/main/examples/spark_distributed_programming) this
|
2596
|
-
release also brings distributed processing into the ZenML toolkit.
|
2597
|
-
|
2598
|
-
Spinning up and configuring infrastructure is a difficult part of the MLOps journey
|
2599
|
-
and can easily become a barrier to entry. Using our [mlops-stacks](https://github.com/zenml-io/mlops-stacks)
|
2600
|
-
repository, it is now possible to spin up perfectly configured infrastructure with
|
2601
|
-
the corresponding ZenML stack using the ZenML CLI.
|
2602
|
-
|
2603
|
-
As always, we've also included various bug fixes and lots of improvements to the documentation and our examples.
|
2604
|
-
|
2605
|
-
## Breaking Changes
|
2606
|
-
|
2607
|
-
This release introduces a breaking change to the CLI by adjusting the access to
|
2608
|
-
the stack component specific resources for `secret-managers` and
|
2609
|
-
`model-deployers` to be more explicitly linked to the component. Here is how:
|
2610
|
-
|
2611
|
-
```bash
|
2612
|
-
# `zenml secret register ...` becomes
|
2613
|
-
zenml secrets-manager secret register ...
|
2614
|
-
|
2615
|
-
# `zenml served_models list` becomes
|
2616
|
-
zenml model-deployer models list
|
2617
|
-
```
|
2618
|
-
|
2619
|
-
## What's Changed
|
2620
|
-
* Link checker by @dnth in https://github.com/zenml-io/zenml/pull/818
|
2621
|
-
* Update Readme with latest info from docs page by @dnth in https://github.com/zenml-io/zenml/pull/810
|
2622
|
-
* Typo on Readme by @dnth in https://github.com/zenml-io/zenml/pull/821
|
2623
|
-
* Update kserve installation to 0.9 on kserve deployment example by @safoinme in https://github.com/zenml-io/zenml/pull/823
|
2624
|
-
* Allow setting caching via the `config.yaml` by @strickvl in https://github.com/zenml-io/zenml/pull/827
|
2625
|
-
* Handle file-io with context manager by @aliabbasjaffri in https://github.com/zenml-io/zenml/pull/825
|
2626
|
-
* Add automated link check github actions by @dnth in https://github.com/zenml-io/zenml/pull/828
|
2627
|
-
* Fix the SQL zenstore to work with MySQL by @stefannica in https://github.com/zenml-io/zenml/pull/829
|
2628
|
-
* Improve label studio error messages if secrets are missing or of wrong schema by @schustmi in https://github.com/zenml-io/zenml/pull/832
|
2629
|
-
* Add secret scoping to the Azure Key Vault by @stefannica in https://github.com/zenml-io/zenml/pull/830
|
2630
|
-
* Unify CLI concepts (removing `secret`, `feature` and `served-models`) by @strickvl in https://github.com/zenml-io/zenml/pull/833
|
2631
|
-
* Put link checker as part of CI by @dnth in https://github.com/zenml-io/zenml/pull/838
|
2632
|
-
* Add missing requirement for step operators by @schustmi in https://github.com/zenml-io/zenml/pull/834
|
2633
|
-
* Fix broken links from link checker results by @dnth in https://github.com/zenml-io/zenml/pull/835
|
2634
|
-
* Fix served models logs formatting error by @safoinme in https://github.com/zenml-io/zenml/pull/836
|
2635
|
-
* New Docker build configuration by @schustmi in https://github.com/zenml-io/zenml/pull/811
|
2636
|
-
* Secrets references on stack component attributes by @schustmi in https://github.com/zenml-io/zenml/pull/817
|
2637
|
-
* Misc bugfixes by @schustmi in https://github.com/zenml-io/zenml/pull/842
|
2638
|
-
* Pillow Image materializer by @strickvl in https://github.com/zenml-io/zenml/pull/820
|
2639
|
-
* Add Tekton orchestrator by @schustmi in https://github.com/zenml-io/zenml/pull/844
|
2640
|
-
* Put Slack call to action at the top of README page. by @dnth in https://github.com/zenml-io/zenml/pull/846
|
2641
|
-
* Change Quickstart to Use Tabular Data by @fa9r in https://github.com/zenml-io/zenml/pull/843
|
2642
|
-
* Add sleep before docker builds in release GH action by @schustmi in https://github.com/zenml-io/zenml/pull/849
|
2643
|
-
* Implement Recursive Built-In Container Materializer by @fa9r in https://github.com/zenml-io/zenml/pull/812
|
2644
|
-
* Custom deployment with KServe and Seldon Core by @safoinme in https://github.com/zenml-io/zenml/pull/841
|
2645
|
-
* Spark Integration by @bcdurak in https://github.com/zenml-io/zenml/pull/837
|
2646
|
-
* Add zenml stack recipe CLI commands by @wjayesh in https://github.com/zenml-io/zenml/pull/807
|
2647
|
-
|
2648
|
-
## New Contributors
|
2649
|
-
* @aliabbasjaffri made their first contribution in https://github.com/zenml-io/zenml/pull/825
|
2650
|
-
|
2651
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.12.0...0.13.0
|
2652
|
-
|
2653
|
-
|
2654
|
-
# 0.12.0
|
2655
|
-
|
2656
|
-
The 0.12.0 release comes with the third implementation of the ZenML Model Deployer abstraction:
|
2657
|
-
The [KServe](https://github.com/zenml-io/zenml/tree/main/examples/kserve_deployment)
|
2658
|
-
integration allows you to deploy any PyTorch, TensorFlow or SKLearn from within your ZenML pipelines!
|
2659
|
-
|
2660
|
-
We also added functionality to specify hardware resources on a step level to control the amount of memory, CPUs and GPUs that
|
2661
|
-
each ZenML step has access to. This is currently limited to the Kubeflow and Vertex orchestrator but will be expanded in upcoming releases.
|
2662
|
-
|
2663
|
-
Additionally, we've added support for scoped secrets in our AWS, GCP and Vault Secrets Managers. These updated Secrets Managers allow
|
2664
|
-
you to configure a scope which determines if secrets are shared with other ZenML Secrets Managers using the same backend.
|
2665
|
-
|
2666
|
-
As always, we've also included various bug fixes and lots of improvements to the documentation and our examples.
|
2667
|
-
|
2668
|
-
## What's Changed
|
2669
|
-
* Fix Links on the examples by @safoinme in https://github.com/zenml-io/zenml/pull/782
|
2670
|
-
* Fix broken links in source code by @schustmi in https://github.com/zenml-io/zenml/pull/784
|
2671
|
-
* Invalidating artifact/metadata store if there is a change in one of them by @bcdurak in https://github.com/zenml-io/zenml/pull/719
|
2672
|
-
* Fixed broken link in README by @htahir1 in https://github.com/zenml-io/zenml/pull/785
|
2673
|
-
* Embed Cheat Sheet in a separate docs page by @fa9r in https://github.com/zenml-io/zenml/pull/790
|
2674
|
-
* Add data validation documentation by @stefannica in https://github.com/zenml-io/zenml/pull/789
|
2675
|
-
* Add local path for mlflow experiment tracker by @schustmi in https://github.com/zenml-io/zenml/pull/786
|
2676
|
-
* Improve Docker build logs. by @fa9r in https://github.com/zenml-io/zenml/pull/793
|
2677
|
-
* Allow standard library types in steps by @stefannica in https://github.com/zenml-io/zenml/pull/799
|
2678
|
-
* Added small description by @AlexejPenner in https://github.com/zenml-io/zenml/pull/801
|
2679
|
-
* Replace the restriction to use Repository inside step with a warning by @stefannica in https://github.com/zenml-io/zenml/pull/792
|
2680
|
-
* Adjust quickstart to data validators by @fa9r in https://github.com/zenml-io/zenml/pull/797
|
2681
|
-
* Add utility function to deprecate pydantic attributes by @schustmi in https://github.com/zenml-io/zenml/pull/778
|
2682
|
-
* Fix the mismatch KFP version between Kubeflow and GCP integration by @safoinme in https://github.com/zenml-io/zenml/pull/796
|
2683
|
-
* Made mlflow more verbose by @htahir1 in https://github.com/zenml-io/zenml/pull/802
|
2684
|
-
* Fix links by @dnth in https://github.com/zenml-io/zenml/pull/798
|
2685
|
-
* KServe model deployer integration by @stefannica in https://github.com/zenml-io/zenml/pull/655
|
2686
|
-
* retrieve pipeline requirement within running step by @safoinme in https://github.com/zenml-io/zenml/pull/805
|
2687
|
-
* Fix `--decouple_stores` error message by @strickvl in https://github.com/zenml-io/zenml/pull/814
|
2688
|
-
* Support subscripted generic step output types by @fa9r in https://github.com/zenml-io/zenml/pull/806
|
2689
|
-
* Allow empty kubeconfig when using local kubeflow orchestrator by @schustmi in https://github.com/zenml-io/zenml/pull/809
|
2690
|
-
* fix the secret register command in kserve docs page by @safoinme in https://github.com/zenml-io/zenml/pull/815
|
2691
|
-
* Annotation example (+ stack component update) by @strickvl in https://github.com/zenml-io/zenml/pull/813
|
2692
|
-
* Per-step resource configuration by @schustmi in https://github.com/zenml-io/zenml/pull/794
|
2693
|
-
* Scoped secrets by @stefannica in https://github.com/zenml-io/zenml/pull/803
|
2694
|
-
* Adjust examples and docs to new pipeline and step fetching syntax by @fa9r in https://github.com/zenml-io/zenml/pull/795
|
2695
|
-
|
2696
|
-
|
2697
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.11.0...0.12.0
|
2698
|
-
|
2699
|
-
|
2700
|
-
# 0.11.0
|
2701
|
-
|
2702
|
-
Our 0.11.0 release contains our new annotation workflow and stack component. We've been blogging about this for a few weeks, and even started maintaining our own repository of open-source annotation tools. With ZenML 0.11.0 you can bring data labeling into your MLOps pipelines and workflows as a first-class citizen. We've started our first iteration of this functionality by integrating with [Label Studio](https://labelstud.io/), a leader in the open-source annotation tool space.
|
2703
|
-
|
2704
|
-
This release also includes a ton of updates to our documentation. (Seriously, go check them out! We added tens of thousands of words since the last release.) We continued the work on our data validation story from the previous release: [Deepchecks](https://deepchecks.com/) is the newest data validator we support, and we updated our Evidently and Whylogs integrations to include all the latest and greatest from those tools.
|
2705
|
-
|
2706
|
-
Beyond this, as usual we included a number of smaller bugfixes and documentation changes to cumulatively improve experience of using ZenML as a user. For a detailed look at what's changed, give [our full release notes](https://github.com/zenml-io/zenml/releases/tag/0.11.0) a glance.
|
2707
|
-
|
2708
|
-
## Breaking Changes
|
2709
|
-
|
2710
|
-
The 0.11.0 release remodels the Evidently and whylogs integrations as Data Validator stack components, in an effort to converge all data profiling and validation libraries around the same abstraction. As a consequence, you now need to configure and add a Data Validator stack component to your stack if you wish to use Evidently or whylogs in your pipelines:
|
2711
|
-
|
2712
|
-
* for Evidently:
|
2713
|
-
|
2714
|
-
```shell
|
2715
|
-
zenml data-validator register evidently -f evidently
|
2716
|
-
zenml stack update -dv evidently
|
2717
|
-
```
|
2718
|
-
|
2719
|
-
* for whylogs:
|
2720
|
-
|
2721
|
-
```shell
|
2722
|
-
zenml data-validator register whylogs -f whylogs
|
2723
|
-
zenml stack update -dv whylogs
|
2724
|
-
```
|
2725
|
-
|
2726
|
-
In this release, we have also upgraded the Evidently and whylogs libraries to their latest and greatest versions (whylogs 1.0.6 and evidently 0.1.52). These versions introduce non-backwards compatible changes that are also reflected in the ZenML integrations:
|
2727
|
-
|
2728
|
-
* Evidently profiles are now materialized using their original `evidently.model_profile.Profile ` data type and the builtin `EvidentlyProfileStep` step now also returns a `Profile` instance instead of the previous dictionary representation. This may impact your existing pipelines as you may have to update your steps to take in `Profile` artifact instances instead of dictionaries.
|
2729
|
-
|
2730
|
-
* the whylogs `whylogs.DatasetProfile` data type was replaced by `whylogs.core.DatasetProfileView` in the builtin whylogs materializer and steps. This may impact your existing pipelines as you may have to update your steps to return and take in `whylogs.core.DatasetProfileView` artifact instances instead of `whylogs.DatasetProfile` objects.
|
2731
|
-
|
2732
|
-
* the whylogs library has gone through a major transformation that completely removed the session concept. As a result, the `enable_whylogs` step decorator was replaced by an `enable_whylabs` step decorator. You only need to use the step decorator if you wish to log your profiles to the Whylabs platform.
|
2733
|
-
|
2734
|
-
Pleaser refer to the examples provided for Evidently and whylogs to learn more about how to use the new integration versions:
|
2735
|
-
|
2736
|
-
* [Evidently](https://github.com/zenml-io/zenml/tree/main/examples/evidently_drift_detection)
|
2737
|
-
* [whylogs/Whylabs](https://github.com/zenml-io/zenml/tree/main/examples/whylogs_data_profiling)
|
2738
|
-
|
2739
|
-
## What's Changed
|
2740
|
-
* Changed PR template to reflect integrations flow by @htahir1 in https://github.com/zenml-io/zenml/pull/732
|
2741
|
-
* Fix broken Feast integration by @strickvl in https://github.com/zenml-io/zenml/pull/737
|
2742
|
-
* Describe args run.py application actually supports by @jsuchome in https://github.com/zenml-io/zenml/pull/740
|
2743
|
-
* Update kubernetes_orchestration example by @fa9r in https://github.com/zenml-io/zenml/pull/743
|
2744
|
-
* Fix some example links by @schustmi in https://github.com/zenml-io/zenml/pull/744
|
2745
|
-
* Fix broken links for docs and examples by @safoinme in https://github.com/zenml-io/zenml/pull/747
|
2746
|
-
* Update CONTRIBUTING.md by @strickvl in https://github.com/zenml-io/zenml/pull/748
|
2747
|
-
* Fix references to types when registering secrets managers by @strickvl in https://github.com/zenml-io/zenml/pull/738
|
2748
|
-
* Make examples conform to best practices guidance by @AlexejPenner in https://github.com/zenml-io/zenml/pull/734
|
2749
|
-
* API Docs with Cookies and Milk by @AlexejPenner in https://github.com/zenml-io/zenml/pull/758
|
2750
|
-
* Use correct region when trying to fetch ECR repositories by @schustmi in https://github.com/zenml-io/zenml/pull/761
|
2751
|
-
* Encode azure secrets manager secret names by @schustmi in https://github.com/zenml-io/zenml/pull/760
|
2752
|
-
* Add nested mlflow option to enable_mlflow decorator by @Val3nt-ML in https://github.com/zenml-io/zenml/pull/742
|
2753
|
-
* Combine all MLMD contexts by @schustmi in https://github.com/zenml-io/zenml/pull/759
|
2754
|
-
* Prevent extra attributes when initializing StackComponents by @schustmi in https://github.com/zenml-io/zenml/pull/763
|
2755
|
-
* New Docker images by @schustmi in https://github.com/zenml-io/zenml/pull/757
|
2756
|
-
* Fix facets magic display in Google Colab by @fa9r in https://github.com/zenml-io/zenml/pull/765
|
2757
|
-
* Allow fetching secrets from within a step by @schustmi in https://github.com/zenml-io/zenml/pull/766
|
2758
|
-
* Add notebook to great expectation example by @stefannica in https://github.com/zenml-io/zenml/pull/768
|
2759
|
-
* Module resolving and path fixes by @schustmi in https://github.com/zenml-io/zenml/pull/735
|
2760
|
-
* Fix step operator entrypoint by @schustmi in https://github.com/zenml-io/zenml/pull/771
|
2761
|
-
* Docs Revamp by @fa9r in https://github.com/zenml-io/zenml/pull/769
|
2762
|
-
* Allow fetching pipeline/step by name, class or instance by @AlexejPenner in https://github.com/zenml-io/zenml/pull/733
|
2763
|
-
* Data Validator abstraction and Deepchecks integration by @htahir1 in https://github.com/zenml-io/zenml/pull/553
|
2764
|
-
* rolling back seldon deployment example by @safoinme in https://github.com/zenml-io/zenml/pull/774
|
2765
|
-
* Added changes from 1062 and 1061 into the updated docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/775
|
2766
|
-
* Refresh Examples on `zenml examples pull` by @fa9r in https://github.com/zenml-io/zenml/pull/776
|
2767
|
-
* Annotation stack component and Label Studio integration by @strickvl in https://github.com/zenml-io/zenml/pull/764
|
2768
|
-
* Add optional machine specs to vertex orchestrator by @felixthebeard in https://github.com/zenml-io/zenml/pull/762
|
2769
|
-
|
2770
|
-
## New Contributors
|
2771
|
-
* @jsuchome made their first contribution in https://github.com/zenml-io/zenml/pull/740
|
2772
|
-
* @Val3nt-ML made their first contribution in https://github.com/zenml-io/zenml/pull/742
|
2773
|
-
* @felixthebeard made their first contribution in https://github.com/zenml-io/zenml/pull/762
|
2774
|
-
|
2775
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.10.0...0.11.0
|
2776
|
-
|
2777
|
-
# 0.10.0
|
2778
|
-
|
2779
|
-
The 0.10.0 release continues our streak of extending ZenML with support for new
|
2780
|
-
orchestrators, this time by adding
|
2781
|
-
[the Kubernetes Native Orchestrator](https://github.com/zenml-io/zenml/tree/main/examples/kubernetes_orchestration).
|
2782
|
-
This orchestrator is a lightweight alternative to other distributed orchestrators
|
2783
|
-
like Airflow or Kubeflow that gives our users the ability to run pipelines in any
|
2784
|
-
Kubernetes cluster without having to install and manage additional tools or
|
2785
|
-
components.
|
2786
|
-
|
2787
|
-
This release features another integration that we are really excited about: the
|
2788
|
-
popular data profiling and validation library [Great Expectations](https://greatexpectations.io/)
|
2789
|
-
is our first Data Validator, a new category of stack components that we are in
|
2790
|
-
the process of standardizing, that will make data quality a central feature of
|
2791
|
-
ZenML. [The ZenML Great Expectations integration](https://github.com/zenml-io/zenml/tree/main/examples/great_expectations_data_validation)
|
2792
|
-
eliminates the complexity associated with configuring the store backends for
|
2793
|
-
Great Expectations by reusing our Artifact Store concept for that purpose and
|
2794
|
-
gives ZenML users immediate access to Great Expectations in both local and cloud
|
2795
|
-
settings.
|
2796
|
-
|
2797
|
-
Last but not least, the release also includes a new secrets manager implementation,
|
2798
|
-
courtesy of our contributor @karimhabush, that integrates ZenML with the
|
2799
|
-
[Hashicorp Vault Server](https://www.vaultproject.io) as well as a few other bug
|
2800
|
-
fixes and improvements.
|
2801
|
-
|
2802
|
-
## What's Changed
|
2803
|
-
* Fix broken link by @strickvl in https://github.com/zenml-io/zenml/pull/707
|
2804
|
-
* Add stack component copy command by @schustmi in https://github.com/zenml-io/zenml/pull/705
|
2805
|
-
* Remove `force` flag from secrets managers' implementation by @strickvl in https://github.com/zenml-io/zenml/pull/708
|
2806
|
-
* Fixed wrong example README by @AlexejPenner in https://github.com/zenml-io/zenml/pull/712
|
2807
|
-
* Fix dead links in integrations docs. by @fa9r in https://github.com/zenml-io/zenml/pull/710
|
2808
|
-
* Fixing link to guide by @chethanuk-plutoflume in https://github.com/zenml-io/zenml/pull/716
|
2809
|
-
* Adding azure-keyvault-secrets to azure integration dependencies by @safoinme in https://github.com/zenml-io/zenml/pull/717
|
2810
|
-
* Fix MLflow repeated deployment error by @fa9r in https://github.com/zenml-io/zenml/pull/715
|
2811
|
-
* Replace alerter standard steps by Slack-specific steps to fix config issue. by @fa9r in https://github.com/zenml-io/zenml/pull/714
|
2812
|
-
* Fix broken links on README by @dnth in https://github.com/zenml-io/zenml/pull/722
|
2813
|
-
* Invalidate cache by @strickvl in https://github.com/zenml-io/zenml/pull/724
|
2814
|
-
* Skip Cleaning Trace on tests by @safoinme in https://github.com/zenml-io/zenml/pull/725
|
2815
|
-
* Kubernetes orchestrator by @fa9r in https://github.com/zenml-io/zenml/pull/688
|
2816
|
-
* Vault Secrets Manager integration - KV Secrets Engine by @karimhabush in https://github.com/zenml-io/zenml/pull/689
|
2817
|
-
* Add missing help text for CLI commands by @safoinme in https://github.com/zenml-io/zenml/pull/723
|
2818
|
-
* Misc bugfixes by @schustmi in https://github.com/zenml-io/zenml/pull/713
|
2819
|
-
* Great Expectations integration for data validation by @strickvl in https://github.com/zenml-io/zenml/pull/555
|
2820
|
-
* Fix GCP artifact store by @schustmi in https://github.com/zenml-io/zenml/pull/730
|
2821
|
-
|
2822
|
-
## New Contributors
|
2823
|
-
* @chethanuk-plutoflume made their first contribution in https://github.com/zenml-io/zenml/pull/716
|
2824
|
-
* @dnth made their first contribution in https://github.com/zenml-io/zenml/pull/722
|
2825
|
-
* @karimhabush made their first contribution in https://github.com/zenml-io/zenml/pull/689
|
2826
|
-
|
2827
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.9.0...0.10.0
|
2828
|
-
|
2829
|
-
# 0.9.0
|
2830
|
-
|
2831
|
-
It's been a couple of weeks, so it's time for a new release! 0.9.0 brings two whole new orchestrators, one of which was contributed by a community member just one day after we unveiled new documentation for orchestrator extensibility! The release also includes a new secrets manager, a Slack integration and a bunch of other smaller changes across the codebase. (Our new orchestrators are exciting enough that they'll get their own blog posts to showcase their strengths in due course.)
|
2832
|
-
|
2833
|
-
Beyond this, as usual we included a number of smaller bugfixes and documentation changes to cumulatively improve experience of using ZenML as a user.
|
2834
|
-
|
2835
|
-
## What's Changed
|
2836
|
-
* Pass secret to release linting workflow by @schustmi in https://github.com/zenml-io/zenml/pull/642
|
2837
|
-
* Fix typo in example by @anencore94 in https://github.com/zenml-io/zenml/pull/644
|
2838
|
-
* Added `SecretExistsError` in `register_secret()` method by @hectorLop in https://github.com/zenml-io/zenml/pull/648
|
2839
|
-
* Fix broken GCP Secrets example CLI command by @strickvl in https://github.com/zenml-io/zenml/pull/649
|
2840
|
-
* Upgrade to `ml-pipelines-sdk` v1.8.0 by @strickvl in https://github.com/zenml-io/zenml/pull/651
|
2841
|
-
* Fix example list CLI command name by @schustmi in https://github.com/zenml-io/zenml/pull/647
|
2842
|
-
* Fix README by @strickvl in https://github.com/zenml-io/zenml/pull/657
|
2843
|
-
* Fix broken links in docs by @safoinme in https://github.com/zenml-io/zenml/pull/652
|
2844
|
-
* Add `VertexOrchestrator` implementation by @gabrielmbmb in https://github.com/zenml-io/zenml/pull/640
|
2845
|
-
* Fix index page links and Heading links. by @safoinme in https://github.com/zenml-io/zenml/pull/661
|
2846
|
-
* Add docstring checks to `pre-commit` script by @strickvl in https://github.com/zenml-io/zenml/pull/481
|
2847
|
-
* Pin MLflow to <1.26.0 to prevent issues when matplotlib is not installed by @fa9r in https://github.com/zenml-io/zenml/pull/666
|
2848
|
-
* Making `utils` more consistent by @strickvl in https://github.com/zenml-io/zenml/pull/658
|
2849
|
-
* Fix linting failures on `develop` by @strickvl in https://github.com/zenml-io/zenml/pull/669
|
2850
|
-
* Add docstrings for `config` module by @strickvl in https://github.com/zenml-io/zenml/pull/668
|
2851
|
-
* Miscellaneous bugfixes by @schustmi in https://github.com/zenml-io/zenml/pull/660
|
2852
|
-
* Make ZenServer dependencies optional by @schustmi in https://github.com/zenml-io/zenml/pull/665
|
2853
|
-
* Implement Azure Secrets Manager integration by @strickvl in https://github.com/zenml-io/zenml/pull/654
|
2854
|
-
* Replace `codespell` with `pyspelling` by @strickvl in https://github.com/zenml-io/zenml/pull/663
|
2855
|
-
* Add Community Event to README by @htahir1 in https://github.com/zenml-io/zenml/pull/674
|
2856
|
-
* Fix failing integration tests by @strickvl in https://github.com/zenml-io/zenml/pull/677
|
2857
|
-
* Add `io` and `model_deployers` docstring checks by @strickvl in https://github.com/zenml-io/zenml/pull/675
|
2858
|
-
* Update `zenml stack down` to use --force flag by @schustmi in https://github.com/zenml-io/zenml/pull/673
|
2859
|
-
* Fix class resolving on windows by @schustmi in https://github.com/zenml-io/zenml/pull/678
|
2860
|
-
* Added `pipelines` docstring checks by @strickvl in https://github.com/zenml-io/zenml/pull/676
|
2861
|
-
* Docstring checks for `cli` module by @strickvl in https://github.com/zenml-io/zenml/pull/680
|
2862
|
-
* Docstring fixes for `entrypoints` and `experiment_trackers` modules by @strickvl in https://github.com/zenml-io/zenml/pull/672
|
2863
|
-
* Clearer Contributing.md by @htahir1 in https://github.com/zenml-io/zenml/pull/681
|
2864
|
-
* How to access secrets within step added to docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/653
|
2865
|
-
* FIX: Log a warning instead of raising an `AssertionError` by @ketangangal in https://github.com/zenml-io/zenml/pull/628
|
2866
|
-
* Reviewer Reminder by @htahir1 in https://github.com/zenml-io/zenml/pull/683
|
2867
|
-
* Fix some docs phrasings and headers by @strickvl in https://github.com/zenml-io/zenml/pull/670
|
2868
|
-
* Implement `SlackAlerter.ask()` by @fa9r in https://github.com/zenml-io/zenml/pull/662
|
2869
|
-
* Extending Alerters Docs by @fa9r in https://github.com/zenml-io/zenml/pull/690
|
2870
|
-
* Sane defaults for MySQL by @htahir1 in https://github.com/zenml-io/zenml/pull/691
|
2871
|
-
* pd.Series materializer by @Reed-Schimmel in https://github.com/zenml-io/zenml/pull/684
|
2872
|
-
* Add docstrings for `materializers` and `metadata_stores` by @strickvl in https://github.com/zenml-io/zenml/pull/694
|
2873
|
-
* Docstrings for the `integrations` module(s) by @strickvl in https://github.com/zenml-io/zenml/pull/692
|
2874
|
-
* Add remaining docstrings by @strickvl in https://github.com/zenml-io/zenml/pull/696
|
2875
|
-
* Allow enabling mlflow/wandb/whylogs with the class-based api by @schustmi in https://github.com/zenml-io/zenml/pull/697
|
2876
|
-
* GitHub Actions orchestrator by @schustmi in https://github.com/zenml-io/zenml/pull/685
|
2877
|
-
* Created MySQL docs, Vertex AI docs, and step.entrypoint() by @AlexejPenner in https://github.com/zenml-io/zenml/pull/698
|
2878
|
-
* Update ignored words by @strickvl in https://github.com/zenml-io/zenml/pull/701
|
2879
|
-
* Stack Component registering made easier by @AlexejPenner in https://github.com/zenml-io/zenml/pull/695
|
2880
|
-
* Cleaning up the docs after the revamp by @bcdurak in https://github.com/zenml-io/zenml/pull/699
|
2881
|
-
* Add model deployer to CLI docs by @safoinme in https://github.com/zenml-io/zenml/pull/702
|
2882
|
-
* Merge Cloud Integrations and create a Vertex AI Example by @AlexejPenner in https://github.com/zenml-io/zenml/pull/693
|
2883
|
-
* GitHub actions orchestrator example by @schustmi in https://github.com/zenml-io/zenml/pull/703
|
2884
|
-
|
2885
|
-
## New Contributors
|
2886
|
-
* @anencore94 made their first contribution in https://github.com/zenml-io/zenml/pull/644
|
2887
|
-
* @hectorLop made their first contribution in https://github.com/zenml-io/zenml/pull/648
|
2888
|
-
* @gabrielmbmb made their first contribution in https://github.com/zenml-io/zenml/pull/640
|
2889
|
-
* @ketangangal made their first contribution in https://github.com/zenml-io/zenml/pull/628
|
2890
|
-
* @Reed-Schimmel made their first contribution in https://github.com/zenml-io/zenml/pull/684
|
2891
|
-
|
2892
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.8.1...0.9.0
|
2893
|
-
|
2894
|
-
# 0.8.1
|
2895
|
-
|
2896
|
-
ZenML 0.8.1 is here and it comes with support for Python 3.9 🎉. It also includes major updates to our
|
2897
|
-
documentation, fixes some broken links in our examples and improves the `zenml go` command which helps
|
2898
|
-
you get started with ZenML.
|
2899
|
-
|
2900
|
-
## What's Changed
|
2901
|
-
* Hotfix/fix failing release by @AlexejPenner in https://github.com/zenml-io/zenml/pull/611
|
2902
|
-
* Remove autocomplete + alerter from documentation by @strickvl in https://github.com/zenml-io/zenml/pull/612
|
2903
|
-
* Support Python 3.9 by @htahir1 in https://github.com/zenml-io/zenml/pull/605
|
2904
|
-
* Revert README by @htahir1 in https://github.com/zenml-io/zenml/pull/624
|
2905
|
-
* Don't build cuda image on release by @schustmi in https://github.com/zenml-io/zenml/pull/623
|
2906
|
-
* Update quickstart for `zenml go` by @fa9r in https://github.com/zenml-io/zenml/pull/625
|
2907
|
-
* Improve kubeflow manual setup logs by @schustmi in https://github.com/zenml-io/zenml/pull/622
|
2908
|
-
* Added missing space to error message by @AlexejPenner in https://github.com/zenml-io/zenml/pull/614
|
2909
|
-
* Added --set flag to register stack command by @AlexejPenner in https://github.com/zenml-io/zenml/pull/613
|
2910
|
-
* Fixes for multiple examples by @schustmi in https://github.com/zenml-io/zenml/pull/626
|
2911
|
-
* Bring back the `served_model` format to the keras materializer by @stefannica in https://github.com/zenml-io/zenml/pull/629
|
2912
|
-
* Fix broken example links by @schustmi in https://github.com/zenml-io/zenml/pull/630
|
2913
|
-
* FAQ edits by @strickvl in https://github.com/zenml-io/zenml/pull/634
|
2914
|
-
* Fix version parsing by @schustmi in https://github.com/zenml-io/zenml/pull/633
|
2915
|
-
* Completed Best Practices Page by @AlexejPenner in https://github.com/zenml-io/zenml/pull/635
|
2916
|
-
* Comments on Issues should no longer trigger gh actions by @AlexejPenner in https://github.com/zenml-io/zenml/pull/636
|
2917
|
-
* Revise `CONTRIBUTING.md` by @strickvl in https://github.com/zenml-io/zenml/pull/615
|
2918
|
-
* Alerter Component for Slack Integration by @fa9r in https://github.com/zenml-io/zenml/pull/586
|
2919
|
-
* Update `zenml go` to open quickstart/notebooks. by @fa9r in https://github.com/zenml-io/zenml/pull/631
|
2920
|
-
* Update examples by @schustmi in https://github.com/zenml-io/zenml/pull/638
|
2921
|
-
* More detailed instructions on creating an integration by @AlexejPenner in https://github.com/zenml-io/zenml/pull/639
|
2922
|
-
* Added publish api docs to release workflow by @AlexejPenner in https://github.com/zenml-io/zenml/pull/641
|
2923
|
-
* Added *.md to ignore paths by @AlexejPenner in https://github.com/zenml-io/zenml/pull/637
|
2924
|
-
* Update README and Docs with new messaging and fix broken links by @htahir1 in https://github.com/zenml-io/zenml/pull/632
|
2925
|
-
|
2926
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.8.0...0.8.1
|
2927
|
-
|
2928
|
-
# 0.8.0
|
2929
|
-
|
2930
|
-
## 🧘♀️ Extensibility is our middle name
|
2931
|
-
|
2932
|
-
* The ability to register custom stack component flavors (and renaming types to
|
2933
|
-
flavor (Registering custom stack component flavors by @bcdurak in
|
2934
|
-
https://github.com/zenml-io/zenml/pull/541)
|
2935
|
-
* The ability to easily extend orchestrators
|
2936
|
-
* Documentation for stacks, stack components and flavors by @bcdurak in
|
2937
|
-
https://github.com/zenml-io/zenml/pull/607
|
2938
|
-
* Allow configuration of s3fs by @schustmi in
|
2939
|
-
https://github.com/zenml-io/zenml/pull/532
|
2940
|
-
* Ability to use SSL to connect to MySQL clients (That allows for connecting to
|
2941
|
-
Cloud based MYSQL deployments)
|
2942
|
-
* New MySQL metadata stores by @bcdurak in
|
2943
|
-
https://github.com/zenml-io/zenml/pull/580!
|
2944
|
-
* Docs and messaging change
|
2945
|
-
* Make Orchestrators more extensible and simplify the interface by @AlexejPenner
|
2946
|
-
in https://github.com/zenml-io/zenml/pull/581
|
2947
|
-
* S3 Compatible Artifact Store and materializers file handling by @safoinme in
|
2948
|
-
https://github.com/zenml-io/zenml/pull/598
|
2949
|
-
|
2950
|
-
## Manage your stacks
|
2951
|
-
|
2952
|
-
* Update stack and stack components via the CLI by @strickvl in
|
2953
|
-
https://github.com/zenml-io/zenml/pull/497
|
2954
|
-
* Add `stack delete` confirmation prompt by @strickvl in
|
2955
|
-
https://github.com/zenml-io/zenml/pull/548
|
2956
|
-
* Add `zenml stack export` and `zenml stack import` commands by @fa9r in
|
2957
|
-
https://github.com/zenml-io/zenml/pull/560
|
2958
|
-
|
2959
|
-
## Collaboration
|
2960
|
-
|
2961
|
-
* User management by @schustmi in https://github.com/zenml-io/zenml/pull/500
|
2962
|
-
|
2963
|
-
## CLI improvements
|
2964
|
-
|
2965
|
-
* CLI speed improvement by @bcdurak in
|
2966
|
-
https://github.com/zenml-io/zenml/pull/567
|
2967
|
-
* Ensure `rich` CLI displays full text and wraps table text by @strickvl in
|
2968
|
-
https://github.com/zenml-io/zenml/pull/577
|
2969
|
-
* Add CLI command to remove stack component attribute by @strickvl in
|
2970
|
-
https://github.com/zenml-io/zenml/pull/590
|
2971
|
-
* Beautify CLI by grouping commands list into tags by @safoinme in
|
2972
|
-
https://github.com/zenml-io/zenml/pull/546
|
2973
|
-
|
2974
|
-
## New integrations:
|
2975
|
-
|
2976
|
-
* Add PyTorch example by @htahir1 in https://github.com/zenml-io/zenml/pull/559
|
2977
|
-
* Added GCP as secret manager by @AlexejPenner in
|
2978
|
-
https://github.com/zenml-io/zenml/pull/556
|
2979
|
-
|
2980
|
-
## Documentation / ZenBytes etc
|
2981
|
-
|
2982
|
-
* ZenBytes update (and ZenML Projects)
|
2983
|
-
* Beautification of Examples by @AlexejPenner in
|
2984
|
-
https://github.com/zenml-io/zenml/pull/491
|
2985
|
-
* Document global configuration and repository by @stefannica in
|
2986
|
-
https://github.com/zenml-io/zenml/pull/579
|
2987
|
-
* ZenML Collaboration docs by @stefannica in
|
2988
|
-
https://github.com/zenml-io/zenml/pull/597
|
2989
|
-
|
2990
|
-
## ➕ Other Updates, Additions and Fixes
|
2991
|
-
|
2992
|
-
* Experiment tracker stack components by @htahir1 in
|
2993
|
-
https://github.com/zenml-io/zenml/pull/530
|
2994
|
-
* Secret Manager improvements and Seldon Core secret passing by @stefannica in
|
2995
|
-
https://github.com/zenml-io/zenml/pull/529
|
2996
|
-
* Pipeline run tracking by @schustmi in
|
2997
|
-
https://github.com/zenml-io/zenml/pull/601
|
2998
|
-
* Stream model deployer logs through CLI by @stefannica in
|
2999
|
-
https://github.com/zenml-io/zenml/pull/557
|
3000
|
-
* Fix various usability bugs by @stefannica in
|
3001
|
-
https://github.com/zenml-io/zenml/pull/561
|
3002
|
-
* Replace `-f` and `--force` with `-y` and `--yes` by @strickvl in
|
3003
|
-
https://github.com/zenml-io/zenml/pull/566
|
3004
|
-
* Make it easier to submit issues by @htahir1 in
|
3005
|
-
https://github.com/zenml-io/zenml/pull/571
|
3006
|
-
* Sync the repository and local store with the disk configuration files and
|
3007
|
-
other fixes by @stefannica in https://github.com/zenml-io/zenml/pull/588
|
3008
|
-
* Add ability to give in-line pip requirements for pipeline by @strickvl in
|
3009
|
-
https://github.com/zenml-io/zenml/pull/583
|
3010
|
-
* Fix evidently visualizer on Colab by @fa9r in
|
3011
|
-
https://github.com/zenml-io/zenml/pull/592
|
3012
|
-
|
3013
|
-
## 🙌 Community Contributions
|
3014
|
-
|
3015
|
-
* @Ankur3107 made their first contribution in
|
3016
|
-
https://github.com/zenml-io/zenml/pull/467
|
3017
|
-
* @MateusGheorghe made their first contribution in
|
3018
|
-
https://github.com/zenml-io/zenml/pull/523
|
3019
|
-
* Added support for scipy sparse matrices by @avramdj in
|
3020
|
-
https://github.com/zenml-io/zenml/pull/534
|
3021
|
-
|
3022
|
-
# 0.7.3
|
3023
|
-
|
3024
|
-
## 📊 Experiment Tracking Components
|
3025
|
-
|
3026
|
-
[PR #530](https://github.com/zenml-io/zenml/pull/530) adds a new stack component to ZenMLs ever-growing list: `experiment_trackers` allows users to configure your experiment tracking tools with ZenML. Examples of experiment tracking tools are [Weights&Biases](https://wandb.ai), [mlflow](https://mlflow.org), [Neptune](https://neptune.ai), amongst others.
|
3027
|
-
|
3028
|
-
Existing users might be confused, as ZenML has had MLflow and wandb support for a while now without such a component. However, this component allows uses more control over the configuration of MLflow and wandb with the new `MLFlowExperimentTracker` and
|
3029
|
-
`WandbExperimentTracker` components. This allows these tools to work in more scenarios than the currently limiting local use-cases.
|
3030
|
-
|
3031
|
-
## 🔎 XGBoost and LightGBM support
|
3032
|
-
|
3033
|
-
[XGBoost](https://xgboost.readthedocs.io/en/stable/) and [LightGBM](https://lightgbm.readthedocs.io/) are one of the most widely used boosting algorithm libraries out there. This release adds materializers for native objects for each library.
|
3034
|
-
|
3035
|
-
Check out [both examples here](https://github.com/zenml-io/zenml/tree/main/examples) and PR's [#544](https://github.com/zenml-io/zenml/pull/544) and [#538](https://github.com/zenml-io/zenml/pull/538) for more details.
|
3036
|
-
|
3037
|
-
## 📂 Parameterized S3FS support to enable non-AWS S3 storage (minio, ceph)
|
3038
|
-
|
3039
|
-
A big complaint of the [S3 Artifact Store](https://github.com/zenml-io/zenml/blob/main/src/zenml/integrations/s3/artifact_stores/s3_artifact_store.py) integration was that it was hard to parameterize it in a way that it supports non-AWS S3 storage like [minio](https://min.io/) and [ceph](https://docs.ceph.com/en/latest/radosgw/s3/). The latest release
|
3040
|
-
made this super simple! When you want to register an S3ArtifactStore from the CLI, you can now pass in `client_kwargs`, `config_kwargs` or `s3_additional_kwargs` as a JSON string. For example:
|
3041
|
-
|
3042
|
-
```shell
|
3043
|
-
zenml artifact-store register my_s3_store --type=s3 --path=s3://my_bucket \
|
3044
|
-
--client_kwargs='{"endpoint_url": "http://my-s3-endpoint"}'
|
3045
|
-
```
|
3046
|
-
|
3047
|
-
See PR [#532](https://github.com/zenml-io/zenml/pull/532) for more details.
|
3048
|
-
|
3049
|
-
## 🧱 New CLI commands to update stack components
|
3050
|
-
|
3051
|
-
We added functionality to allow users to update stacks that already exist. This shows the basic workflow:
|
3052
|
-
|
3053
|
-
```shell
|
3054
|
-
zenml orchestrator register local_orchestrator2 -t local
|
3055
|
-
zenml stack update default -o local_orchestrator2
|
3056
|
-
zenml stack describe default
|
3057
|
-
zenml container-registry register local_registry --type=default --uri=localhost:5000
|
3058
|
-
zenml container-registry update local --uri='somethingelse.com'
|
3059
|
-
zenml container-registry rename local local2
|
3060
|
-
zenml container-registry describe local2
|
3061
|
-
zenml stack rename default new_default
|
3062
|
-
zenml stack update new_default -c local2
|
3063
|
-
zenml stack describe new_default
|
3064
|
-
zenml stack remove-component -c
|
3065
|
-
```
|
3066
|
-
More details are in the [CLI docs](https://apidocs.zenml.io/0.7.3/cli/).
|
3067
|
-
Users can add new stack components to a pre-existing stack, or they can modify
|
3068
|
-
already-present stack components. They can also rename their stack and individual stack components.
|
3069
|
-
|
3070
|
-
## 🐛 Seldon Core authentication through ZenML secrets
|
3071
|
-
|
3072
|
-
The Seldon Core Model Deployer stack component was updated in this release to allow the configuration of ZenML secrets with credentials that authenticate Seldon to access the Artifact Store. The Seldon Core integration provides 3 different secret schemas for the 3 flavors of Artifact Store: AWS, GCP, and Azure, but custom secrets can be used as well. For more information on how to use this feature please refer to our [Seldon Core deployment example](https://github.com/zenml-io/zenml/tree/main/examples/seldon_deployment).
|
3073
|
-
|
3074
|
-
Lastly, we had numerous other changes such as ensuring the PyTorch materializer works across all artifact stores
|
3075
|
-
and the Kubeflow Metadata Store can be easily queried locally.
|
3076
|
-
|
3077
|
-
## Detailed Changelog
|
3078
|
-
* Fix caching & `mypy` errors by @strickvl in https://github.com/zenml-io/zenml/pull/524
|
3079
|
-
* Switch unit test from local_daemon to multiprocessing by @jwwwb in https://github.com/zenml-io/zenml/pull/508
|
3080
|
-
* Change Pytorch materializer to support remote storage by @safoinme in https://github.com/zenml-io/zenml/pull/525
|
3081
|
-
* Remove TODO from Feature Store `init` docstring by @strickvl in https://github.com/zenml-io/zenml/pull/527
|
3082
|
-
* Fixed typo predicter -> predictor by @MateusGheorghe in https://github.com/zenml-io/zenml/pull/523
|
3083
|
-
* Fix mypy errors by @strickvl in https://github.com/zenml-io/zenml/pull/528
|
3084
|
-
* Replaced old local_* logic by @htahir1 in https://github.com/zenml-io/zenml/pull/531
|
3085
|
-
* capitalize aws username in ECR docs by @wjayesh in https://github.com/zenml-io/zenml/pull/533
|
3086
|
-
* Build docker base images quicker after release by @schustmi in https://github.com/zenml-io/zenml/pull/537
|
3087
|
-
* Allow configuration of s3fs by @schustmi in https://github.com/zenml-io/zenml/pull/532
|
3088
|
-
* Update contributing and fix ci badge to main by @htahir1 in https://github.com/zenml-io/zenml/pull/536
|
3089
|
-
* Added XGboost integration by @htahir1 in https://github.com/zenml-io/zenml/pull/538
|
3090
|
-
* Added fa9r to .github/teams.yml. by @fa9r in https://github.com/zenml-io/zenml/pull/539
|
3091
|
-
* Secret Manager improvements and Seldon Core secret passing by @stefannica in https://github.com/zenml-io/zenml/pull/529
|
3092
|
-
* User management by @schustmi in https://github.com/zenml-io/zenml/pull/500
|
3093
|
-
* Update stack and stack components via the CLI by @strickvl in https://github.com/zenml-io/zenml/pull/497
|
3094
|
-
* Added lightgbm integration by @htahir1 in https://github.com/zenml-io/zenml/pull/544
|
3095
|
-
* Fix the Kubeflow metadata store and other stack management improvements by @stefannica in https://github.com/zenml-io/zenml/pull/542
|
3096
|
-
* Experiment tracker stack components by @htahir1 in https://github.com/zenml-io/zenml/pull/530
|
3097
|
-
|
3098
|
-
## New Contributors
|
3099
|
-
* @MateusGheorghe made their first contribution in https://github.com/zenml-io/zenml/pull/523
|
3100
|
-
* @fa9r made their first contribution in https://github.com/zenml-io/zenml/pull/539
|
3101
|
-
|
3102
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.7.2...0.7.3
|
3103
|
-
**Blog Post**: https://blog.zenml.io/zero-seven-two-three-release/
|
3104
|
-
|
3105
|
-
# 0.7.2
|
3106
|
-
|
3107
|
-
0.7.2 is a minor release which quickly patches some bugs found in the last
|
3108
|
-
release to do with Seldon and Mlflow deployment.
|
3109
|
-
|
3110
|
-
This release also features initial versions of two amazing new integrations:
|
3111
|
-
[HuggingFace](https://huggingface.co/) and [Weights&Biases](https://wandb.ai/site)!
|
3112
|
-
|
3113
|
-
- HuggingFace models are now supported to be passed through ZenML pipelines!
|
3114
|
-
- You can now track your pipeline runs with Weights&Biases with the new
|
3115
|
-
`enable_wandb` decorator!
|
3116
|
-
|
3117
|
-
Continuous model deployment with MLflow has been improved with ZenML 0.7.2. A new
|
3118
|
-
MLflow Model Deployer Stack component is now available and needs to be part of
|
3119
|
-
your stack to be able to deploy models:
|
3120
|
-
|
3121
|
-
```bash
|
3122
|
-
zenml integration install mlflow
|
3123
|
-
zenml model-deployer register mlflow --type=mlflow
|
3124
|
-
zenml stack register local_with_mlflow -m default -a default -o default -d mlflow
|
3125
|
-
zenml stack set local_with_mlflow
|
3126
|
-
```
|
3127
|
-
|
3128
|
-
The MLflow Model Deployer is yet another addition to the list of Model Deployers
|
3129
|
-
available in ZenML. You can read more on deploying models to production with MLflow
|
3130
|
-
in our [Continuous Training and Deployment documentation section](https://docs.zenml.io/advanced-guide/practical/deploying-models) and
|
3131
|
-
our [MLflow deployment example](https://github.com/zenml-io/zenml/tree/main/examples/mlflow_deployment).
|
3132
|
-
|
3133
|
-
## What's Changed
|
3134
|
-
* Fix the seldon deployment example by @htahir1 in https://github.com/zenml-io/zenml/pull/511
|
3135
|
-
* Create base deployer and refactor MLflow deployer implementation by @wjayesh in https://github.com/zenml-io/zenml/pull/489
|
3136
|
-
* Add nlp example by @Ankur3107 in https://github.com/zenml-io/zenml/pull/467
|
3137
|
-
* Fix typos by @strickvl in https://github.com/zenml-io/zenml/pull/515
|
3138
|
-
* Bugfix/hypothesis given does not work with fixture by @jwwwb in https://github.com/zenml-io/zenml/pull/513
|
3139
|
-
* Bug: fix long Kubernetes labels in Seldon deployments by @stefannica in https://github.com/zenml-io/zenml/pull/514
|
3140
|
-
* Change prediction_uri to prediction_url in MLflow deployer by @stefannica in https://github.com/zenml-io/zenml/pull/516
|
3141
|
-
* Simplify HuggingFace Integration by @AlexejPenner in https://github.com/zenml-io/zenml/pull/517
|
3142
|
-
* Weights & Biases Basic Integration by @htahir1 in https://github.com/zenml-io/zenml/pull/518
|
3143
|
-
|
3144
|
-
## New Contributors
|
3145
|
-
* @Ankur3107 made their first contribution in https://github.com/zenml-io/zenml/pull/467
|
3146
|
-
|
3147
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.7.1...0.7.2
|
3148
|
-
|
3149
|
-
# 0.7.1
|
3150
|
-
|
3151
|
-
The release introduces the [Seldon Core](https://github.com/SeldonIO/seldon-core) ZenML integration, featuring the
|
3152
|
-
*Seldon Core Model Deployer* and a *Seldon Core standard model deployer step*.
|
3153
|
-
The [*Model Deployer*](https://docs.zenml.io/component-gallery/model-deployers/model-deployers)
|
3154
|
-
is a new type of stack component that enables you to develop continuous
|
3155
|
-
model deployment pipelines that train models and continuously deploy them to an
|
3156
|
-
external model serving tool, service or platform. You can read more on deploying models
|
3157
|
-
to production with Seldon Core in our
|
3158
|
-
[Continuous Training and Deployment documentation section](https://docs.zenml.io/component-gallery/model-deployers/model-deployers) and our [Seldon Core deployment example](https://github.com/zenml-io/zenml/tree/main/examples/seldon_deployment).
|
3159
|
-
|
3160
|
-
We also see two new integrations with [Feast](https://feast.dev) as ZenML's first feature store integration. Feature stores allow data teams to serve data via an offline store and an online low-latency store where data is kept in sync between the two. It also offers a centralized registry where features (and feature schemas) are stored for use within a team or wider organization. ZenML now supports connecting to a Redis-backed Feast feature store as a stack component integration. Check out the [full example](https://github.com/zenml-io/zenml/tree/release/0.7.1/examples/feature_store) to see it in action!
|
3161
|
-
|
3162
|
-
0.7.1 also brings an addition to ZenML training library integrations with [NeuralProphet](https://neuralprophet.com/html/index.html). Check out the new [example](https://github.com/zenml-io/zenml/tree/main/examples) for more details, and the [docs](https://docs.zenml.io) for more further detail on all new features!
|
3163
|
-
|
3164
|
-
## What's Changed
|
3165
|
-
* Add linting of examples to `pre-commit` by @strickvl in https://github.com/zenml-io/zenml/pull/490
|
3166
|
-
* Remove dev-specific entries in `.gitignore` by @strickvl in https://github.com/zenml-io/zenml/pull/488
|
3167
|
-
* Produce periodic mocked data for Segment/Mixpanel by @AlexejPenner in https://github.com/zenml-io/zenml/pull/487
|
3168
|
-
* Abstractions for artifact stores by @bcdurak in https://github.com/zenml-io/zenml/pull/474
|
3169
|
-
* enable and disable cache from runtime config by @AlexejPenner in https://github.com/zenml-io/zenml/pull/492
|
3170
|
-
* Basic Seldon Core Deployment Service by @stefannica in https://github.com/zenml-io/zenml/pull/495
|
3171
|
-
* Parallelize our test suite and make errors more readable by @alex-zenml in https://github.com/zenml-io/zenml/pull/378
|
3172
|
-
* Provision local zenml service by @jwwwb in https://github.com/zenml-io/zenml/pull/496
|
3173
|
-
* bugfix/optional-secrets-manager by @safoinme in https://github.com/zenml-io/zenml/pull/493
|
3174
|
-
* Quick fix for copying folders by @bcdurak in https://github.com/zenml-io/zenml/pull/501
|
3175
|
-
* Pin exact ml-pipelines-sdk version by @schustmi in https://github.com/zenml-io/zenml/pull/506
|
3176
|
-
* Seldon Core model deployer stack component and standard step by @stefannica in https://github.com/zenml-io/zenml/pull/499
|
3177
|
-
* Fix datetime test / bug by @strickvl in https://github.com/zenml-io/zenml/pull/507
|
3178
|
-
* Added NeuralProphet integration by @htahir1 in https://github.com/zenml-io/zenml/pull/504
|
3179
|
-
* Feature Store (Feast with Redis) by @strickvl in https://github.com/zenml-io/zenml/pull/498
|
3180
|
-
|
3181
|
-
|
3182
|
-
# 0.7.0
|
3183
|
-
|
3184
|
-
With ZenML 0.7.0, a lot has been revamped under the hood about how things are stored. Importantly what this means is that ZenML now has system-wide profiles that let you register stacks to share across several of your projects! If you still want to manage your stacks for each project folder individually, profiles still let you do that as well.
|
3185
|
-
|
3186
|
-
Most projects of any complexity will require passwords or tokens to access data and infrastructure, and for this purpose ZenML 0.7.0 introduces [the Secrets Manager](https://docs.zenml.io/component-gallery/secrets-managers/secrets-managers) stack component to seamlessly pass around these values to your steps. Our AWS integration also allows you to use AWS Secrets Manager as a backend to handle all your secret persistence needs.
|
3187
|
-
|
3188
|
-
Finally, in addition to the new AzureML and Sagemaker Step Operators that version 0.6.3 brought, this release also adds the ability to [run individual steps on GCP's Vertex AI](https://docs.zenml.io/component-gallery/step-operators/gcloud-vertexai).
|
3189
|
-
|
3190
|
-
Beyond this, some smaller bugfixes and documentation changes combine to make ZenML 0.7.0 a more pleasant user experience.
|
3191
|
-
|
3192
|
-
## What's Changed
|
3193
|
-
* Added quick mention of how to use dockerignore by @AlexejPenner in https://github.com/zenml-io/zenml/pull/468
|
3194
|
-
* Made rich traceback optional with ENV variable by @htahir1 in https://github.com/zenml-io/zenml/pull/472
|
3195
|
-
* Separate stack persistence from repo implementation by @jwwwb in https://github.com/zenml-io/zenml/pull/462
|
3196
|
-
* Adding safoine username to github team by @safoinme in https://github.com/zenml-io/zenml/pull/475
|
3197
|
-
* Fix `zenml stack describe` bug by @strickvl in https://github.com/zenml-io/zenml/pull/476
|
3198
|
-
* ZenProfiles and centralized ZenML repositories by @stefannica in https://github.com/zenml-io/zenml/pull/471
|
3199
|
-
* Add `examples` folder to linting script by @strickvl in https://github.com/zenml-io/zenml/pull/482
|
3200
|
-
* Vertex AI integration and numerous other changes by @htahir1 in https://github.com/zenml-io/zenml/pull/477
|
3201
|
-
* Fix profile handing in the Azure ML step operator by @stefannica in https://github.com/zenml-io/zenml/pull/483
|
3202
|
-
* Copy the entire stack configuration into containers by @stefannica in https://github.com/zenml-io/zenml/pull/480
|
3203
|
-
* Improve some things with the Profiles CLI output by @stefannica in https://github.com/zenml-io/zenml/pull/484
|
3204
|
-
* Secrets manager stack component and interface by @AlexejPenner in https://github.com/zenml-io/zenml/pull/470
|
3205
|
-
* Update schedule.py (#485) by @avramdj in https://github.com/zenml-io/zenml/pull/485
|
3206
|
-
|
3207
|
-
## New Contributors
|
3208
|
-
* @avramdj in https://github.com/zenml-io/zenml/pull/485
|
3209
|
-
|
3210
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.6.3...0.7.0rc
|
3211
|
-
|
3212
|
-
|
3213
|
-
# 0.6.3
|
3214
|
-
|
3215
|
-
With ZenML 0.6.3, you can now run your ZenML steps on Sagemaker and AzureML! It's normal to have certain steps that require specific hardware on which to run model training, for example, and this latest release gives you the power to switch out hardware for individual steps to support this.
|
3216
|
-
|
3217
|
-
We added a new Tensorboard visualization that you can make use of when using our Kubeflow Pipelines integration. We handle the background processes needed to spin up this interactive web interface that you can use to visualize your model's performance over time.
|
3218
|
-
|
3219
|
-
Behind the scenes we gave our integration testing suite a massive upgrade, fixed a number of smaller bugs and made documentation updates. For a detailed look at what's changed, give [our full release notes](https://github.com/zenml-io/zenml/releases/tag/0.6.3) a glance.
|
3220
|
-
|
3221
|
-
## What's Changed
|
3222
|
-
* Fix typo by @wjayesh in https://github.com/zenml-io/zenml/pull/432
|
3223
|
-
* Remove tabulate dependency (replaced by rich) by @jwwwb in https://github.com/zenml-io/zenml/pull/436
|
3224
|
-
* Fix potential issue with local integration tests by @schustmi in https://github.com/zenml-io/zenml/pull/428
|
3225
|
-
* Remove support for python 3.6 by @schustmi in https://github.com/zenml-io/zenml/pull/437
|
3226
|
-
* Create clean test repos in separate folders by @michael-zenml in https://github.com/zenml-io/zenml/pull/430
|
3227
|
-
* Copy explicit materializers before modifying, log correct class by @schustmi in https://github.com/zenml-io/zenml/pull/434
|
3228
|
-
* Fix typo in mysql password parameter by @pafpixel in https://github.com/zenml-io/zenml/pull/438
|
3229
|
-
* Pytest-fixture for separate virtual environments for each integration test by @AlexejPenner in https://github.com/zenml-io/zenml/pull/405
|
3230
|
-
* Bugfix/fix failing tests due to comments step by @AlexejPenner in https://github.com/zenml-io/zenml/pull/444
|
3231
|
-
* Added --use-virtualenvs option to allow choosing envs to run by @AlexejPenner in https://github.com/zenml-io/zenml/pull/445
|
3232
|
-
* Log whether a step was cached by @strickvl in https://github.com/zenml-io/zenml/pull/435
|
3233
|
-
* Added basic integration tests for remaining examples by @strickvl in https://github.com/zenml-io/zenml/pull/439
|
3234
|
-
* Improve error message when provisioning local kubeflow resources with a non-local container registry. by @schustmi in https://github.com/zenml-io/zenml/pull/442
|
3235
|
-
* Enable generic step inputs and outputs by @schustmi in https://github.com/zenml-io/zenml/pull/440
|
3236
|
-
* Removed old reference to a step that no longer exists by @AlexejPenner in https://github.com/zenml-io/zenml/pull/452
|
3237
|
-
* Correctly use custom kubernetes context if specified by @schustmi in https://github.com/zenml-io/zenml/pull/451
|
3238
|
-
* Fix CLI stack component describe/list commands by @schustmi in https://github.com/zenml-io/zenml/pull/450
|
3239
|
-
* Ignore type of any tfx proto file by @schustmi in https://github.com/zenml-io/zenml/pull/453
|
3240
|
-
* Another boyscout pr on the gh actions by @AlexejPenner in https://github.com/zenml-io/zenml/pull/455
|
3241
|
-
* Upgrade TFX to 1.6.1 by @jwwwb in https://github.com/zenml-io/zenml/pull/441
|
3242
|
-
* Added ZenML Projects to README by @htahir1 in https://github.com/zenml-io/zenml/pull/457
|
3243
|
-
* Upgrade `rich` from 11.0 to 12.0 by @strickvl in https://github.com/zenml-io/zenml/pull/458
|
3244
|
-
* Add Kubeflow tensorboard viz and fix tensorflow file IO for cloud back-ends by @stefannica in https://github.com/zenml-io/zenml/pull/447
|
3245
|
-
* Implementing the `explain` subcommand by @bcdurak in https://github.com/zenml-io/zenml/pull/460
|
3246
|
-
* Implement AzureML and Sagemaker step operators by @schustmi in https://github.com/zenml-io/zenml/pull/456
|
3247
|
-
|
3248
|
-
## New Contributors
|
3249
|
-
* @pafpixel made their first contribution in https://github.com/zenml-io/zenml/pull/438
|
3250
|
-
|
3251
|
-
# 0.6.2
|
3252
|
-
|
3253
|
-
ZenML 0.6.2 brings you the ability to serve models using MLflow deployments as well as an updated CLI interface! For a real continuous deployment cycle, we know that ZenML pipelines should be able to handle everything — from pre-processing to training to serving to monitoring and then potentially re-training and re-serving. The interfaces we created in this release are the foundation on which all of this will build.
|
3254
|
-
|
3255
|
-
We also improved how you interact with ZenML through the CLI. Everything looks so much smarter and readable now with the popular `rich` library integrated into our dependencies.
|
3256
|
-
|
3257
|
-
Smaller changes that you'll notice include updates to our cloud integrations and bug fixes for Windows users. For a detailed look at what's changed, see below.
|
3258
|
-
|
3259
|
-
## What's Changed
|
3260
|
-
|
3261
|
-
* Updated notebook for quickstart by @htahir1 in https://github.com/zenml-io/zenml/pull/398
|
3262
|
-
* Update tensorflow base image by @schustmi in https://github.com/zenml-io/zenml/pull/396
|
3263
|
-
* Add cloud specific deployment guide + refactoring by @wjayesh in https://github.com/zenml-io/zenml/pull/400
|
3264
|
-
* add cloud sub page to toc.md by @wjayesh in https://github.com/zenml-io/zenml/pull/401
|
3265
|
-
* fix tab indent by @wjayesh in https://github.com/zenml-io/zenml/pull/402
|
3266
|
-
* Bugfix for workflows failing due to modules not being found by @bcdurak in https://github.com/zenml-io/zenml/pull/390
|
3267
|
-
* Improve github workflows by @schustmi in https://github.com/zenml-io/zenml/pull/406
|
3268
|
-
* Add plausible script to docs.zenml.io pages by @alex-zenml in https://github.com/zenml-io/zenml/pull/414
|
3269
|
-
* Add orchestrator and ECR docs by @wjayesh in https://github.com/zenml-io/zenml/pull/413
|
3270
|
-
* Richify the CLI by @alex-zenml in https://github.com/zenml-io/zenml/pull/392
|
3271
|
-
* Allow specification of required integrations for a pipeline by @schustmi in https://github.com/zenml-io/zenml/pull/408
|
3272
|
-
* Update quickstart in docs to conform to examples by @htahir1 in https://github.com/zenml-io/zenml/pull/410
|
3273
|
-
* Updated PR template with some more details by @htahir1 in https://github.com/zenml-io/zenml/pull/411
|
3274
|
-
* Bugfix on the CLI to work without a git installation by @bcdurak in https://github.com/zenml-io/zenml/pull/412
|
3275
|
-
* Added Ayush's Handle by @ayush714 in https://github.com/zenml-io/zenml/pull/417
|
3276
|
-
* Adding an info message on Windows if there is no application associated to .sh files by @bcdurak in https://github.com/zenml-io/zenml/pull/419
|
3277
|
-
* Catch `matplotlib` crash when running IPython in terminal by @strickvl in https://github.com/zenml-io/zenml/pull/416
|
3278
|
-
* Automatically activate integrations when unable to find stack component by @schustmi in https://github.com/zenml-io/zenml/pull/420
|
3279
|
-
* Fix some code inspections by @halvgaard in https://github.com/zenml-io/zenml/pull/422
|
3280
|
-
* Prepare integration tests on kubeflow by @schustmi in https://github.com/zenml-io/zenml/pull/423
|
3281
|
-
* Add concepts back into glossary by @strickvl in https://github.com/zenml-io/zenml/pull/425
|
3282
|
-
* Make guide easier to follow by @wjayesh in https://github.com/zenml-io/zenml/pull/427
|
3283
|
-
* Fix httplib to 0.19 and pyparsing to 2.4 by @jwwwb in https://github.com/zenml-io/zenml/pull/426
|
3284
|
-
* Wrap context serialization in try blocks by @jwwwb in https://github.com/zenml-io/zenml/pull/397
|
3285
|
-
* Track stack configuration when registering and running a pipeline by @schustmi in https://github.com/zenml-io/zenml/pull/429
|
3286
|
-
* MLflow deployment integration by @stefannica in https://github.com/zenml-io/zenml/pull/415
|
3287
|
-
|
3288
|
-
# 0.6.1
|
3289
|
-
|
3290
|
-
ZenML 0.6.1 is out and it's all about the cloud ☁️! We have improved AWS integration and a brand-new [Azure](https://github.com/zenml-io/zenml/tree/0.6.1/src/zenml/integrations/azure) integration! Run your pipelines on AWS and Azure now and let us know how it went on our [Slack](https://zenml.io/slack-invite).
|
3291
|
-
|
3292
|
-
Smaller changes that you'll notice include much-awaited updates and fixes, including the first iterations of scheduling pipelines and tracking more reproducibility-relevant data in the metadata store.
|
3293
|
-
|
3294
|
-
For a detailed look at what's changed, see below.
|
3295
|
-
|
3296
|
-
## What's changed
|
3297
|
-
|
3298
|
-
* Add MVP for scheduling by @htahir1 in https://github.com/zenml-io/zenml/pull/354
|
3299
|
-
* Add S3 artifact store and filesystem by @schustmi in https://github.com/zenml-io/zenml/pull/359
|
3300
|
-
* Update 0.6.0 release notes by @alex-zenml in https://github.com/zenml-io/zenml/pull/362
|
3301
|
-
* Fix cuda-dev base container image by @stefannica in https://github.com/zenml-io/zenml/pull/361
|
3302
|
-
* Mark ZenML as typed package by @schustmi in https://github.com/zenml-io/zenml/pull/360
|
3303
|
-
* Improve error message if ZenML repo is missing inside kubeflow container entrypoint by @schustmi in https://github.com/zenml-io/zenml/pull/363
|
3304
|
-
* Spell whylogs and WhyLabs correctly in our docs by @stefannica in https://github.com/zenml-io/zenml/pull/369
|
3305
|
-
* Feature/add readme for mkdocs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/372
|
3306
|
-
* Cleaning up the assets pushed by gitbook automatically by @bcdurak in https://github.com/zenml-io/zenml/pull/371
|
3307
|
-
* Turn codecov off for patch updates by @htahir1 in https://github.com/zenml-io/zenml/pull/376
|
3308
|
-
* Minor changes and fixes by @schustmi in https://github.com/zenml-io/zenml/pull/365
|
3309
|
-
* Only include python files when building local docs by @schustmi in https://github.com/zenml-io/zenml/pull/377
|
3310
|
-
* Prevent access to repo during step execution by @schustmi in https://github.com/zenml-io/zenml/pull/370
|
3311
|
-
* Removed duplicated Section within docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/379
|
3312
|
-
* Fixing the materializer registry to spot sub-classes of defined types by @bcdurak in https://github.com/zenml-io/zenml/pull/368
|
3313
|
-
* Computing hash of step and materializer works in notebooks by @htahir1 in https://github.com/zenml-io/zenml/pull/375
|
3314
|
-
* Sort requirements to improve docker build caching by @schustmi in https://github.com/zenml-io/zenml/pull/383
|
3315
|
-
* Make sure the s3 artifact store is registered when the integration is activated by @schustmi in https://github.com/zenml-io/zenml/pull/382
|
3316
|
-
* Make MLflow integration work with kubeflow and scheduled pipelines by @stefannica in https://github.com/zenml-io/zenml/pull/374
|
3317
|
-
* Reset _has_been_called to False ahead of pipeline.connect by @AlexejPenner in https://github.com/zenml-io/zenml/pull/385
|
3318
|
-
* Fix local airflow example by @schustmi in https://github.com/zenml-io/zenml/pull/366
|
3319
|
-
* Improve and extend base materializer error messages by @schustmi in https://github.com/zenml-io/zenml/pull/380
|
3320
|
-
* Windows CI issue by @schustmi in https://github.com/zenml-io/zenml/pull/389
|
3321
|
-
* Add the ability to attach custom properties to the Metadata Store by @bcdurak in https://github.com/zenml-io/zenml/pull/355
|
3322
|
-
* Handle case when return values do not match output by @AlexejPenner in https://github.com/zenml-io/zenml/pull/386
|
3323
|
-
* Quickstart code in docs fixed by @AlexejPenner in https://github.com/zenml-io/zenml/pull/387
|
3324
|
-
* Fix mlflow tracking example by @stefannica in https://github.com/zenml-io/zenml/pull/393
|
3325
|
-
* Implement azure artifact store and fileio plugin by @schustmi in https://github.com/zenml-io/zenml/pull/388
|
3326
|
-
* Create todo issues with separate issue type by @schustmi in https://github.com/zenml-io/zenml/pull/394
|
3327
|
-
* Log that steps are cached while running pipeline by @alex-zenml in https://github.com/zenml-io/zenml/pull/381
|
3328
|
-
* Schedule added to context for all orchestrators by @AlexejPenner in https://github.com/zenml-io/zenml/pull/391
|
3329
|
-
|
3330
|
-
# 0.6.0
|
3331
|
-
|
3332
|
-
ZenML 0.6.0 is out now. We've made some big changes under the hood, but our biggest public-facing addition is our new integration to support all your data logging needs: [`whylogs`](https://github.com/whylabs/whylogs). Our core architecture was [thoroughly reworked](https://github.com/zenml-io/zenml/pull/305) and is now in a much better place to support our ongoing development needs.
|
3333
|
-
|
3334
|
-
|
3335
|
-
Smaller changes that you'll notice include extensive documentation additions, updates and fixes. For a detailed look at what's changed, see below.
|
3336
|
-
|
3337
|
-
## 📊 Whylogs logging
|
3338
|
-
|
3339
|
-
[Whylogs](https://github.com/whylabs/whylogs) is an open source library that analyzes your data and creates statistical summaries called whylogs profiles. Whylogs profiles can be visualized locally or uploaded to the WhyLabs platform where more comprehensive analysis can be carried out.
|
3340
|
-
|
3341
|
-
ZenML integrates seamlessly with Whylogs and [WhyLabs](https://whylabs.ai/). This example shows how easy it is to enhance steps in an existing ML pipeline with Whylogs profiling features. Changes to the user code are minimal while ZenML takes care of all aspects related to Whylogs session initialization, profile serialization, versioning and persistence and even uploading generated profiles to [Whylabs](https://whylabs.ai/).
|
3342
|
-
|
3343
|
-

|
3344
|
-
|
3345
|
-
With our `WhylogsVisualizer`, as described in [the associated example notes](https://github.com/zenml-io/zenml/tree/0.6.0/examples/whylogs), you can visualize Whylogs profiles generated as part of a pipeline.
|
3346
|
-
|
3347
|
-
## ⛩ New Core Architecture
|
3348
|
-
|
3349
|
-
We implemented [some fundamental changes](https://github.com/zenml-io/zenml/pull/305) to the core architecture to solve some of the issues we previously had and provide a more extensible design to support quicker implementations of different stack components and integrations. The main change was to refactor the `Repository`, `Stack` and `StackComponent` architectures. These changes had a pretty wide impact so involved changes in many files throughout the codebase, especially in the CLI which makes calls to all these pieces.
|
3350
|
-
|
3351
|
-
We've already seen how it helps us move faster in building integrations and we hope it helps making contributions as pain-free as possible!
|
3352
|
-
|
3353
|
-
## 🗒 Documentation and Example Updates
|
3354
|
-
|
3355
|
-
As the codebase and functionality of ZenML grows, we always want to make sure our documentation is clear, up-to-date and easy to use. We made a number of changes in this release that will improve your experience in this regard:
|
3356
|
-
|
3357
|
-
- added a number of new explainers on key ZenML concepts and how to use them in your code, notably on [how to create a custom materializer](https://docs.zenml.io/v/0.6.0/guides/index/custom-materializer) and [how to fetch historic pipeline runs](https://docs.zenml.io/v/0.6.0/guides/index/historic-runs) using the `StepContext`
|
3358
|
-
- fixed a number of typos and broken links
|
3359
|
-
- [added versioning](https://github.com/zenml-io/zenml/pull/336) to our API documentation so you can choose to view the reference appropriate to the version that you're using. We now use `mkdocs` for this so you'll notice a slight visual refresh as well.
|
3360
|
-
- added new examples highlighting specific use cases and integrations:
|
3361
|
-
- how to create a custom materializer ([example](https://github.com/zenml-io/zenml/tree/0.6.0/examples/custom_materializer))
|
3362
|
-
- how to fetch historical pipeline runs ([example](https://github.com/zenml-io/zenml/tree/0.6.0/examples/fetch_historical_runs))
|
3363
|
-
- how to use standard interfaces for common ML patterns ([example](https://github.com/zenml-io/zenml/tree/0.6.0/examples/standard_interfaces))
|
3364
|
-
- `whylogs` logging ([example](https://github.com/zenml-io/zenml/tree/0.6.0/examples/whylogs))
|
3365
|
-
|
3366
|
-
## ➕ Other updates, additions and fixes
|
3367
|
-
|
3368
|
-
As with most releases, we made a number of small but significant fixes and additions. The most import of these were that you can now [access the metadata store](https://github.com/zenml-io/zenml/pull/338) via the step context. This enables a number of new possible workflows and pipeline patterns and we're really excited to have this in the release.
|
3369
|
-
|
3370
|
-
We [added in](https://github.com/zenml-io/zenml/pull/315) a markdown parser for the `zenml example info …` command, so now when you want to use our CLI to learn more about specific examples you will see beautifully parsed text and not markdown markup.
|
3371
|
-
|
3372
|
-
We improved a few of our error messages, too, like for when the return type of a step function [doesn’t match the expected type](https://github.com/zenml-io/zenml/pull/322), or if [step is called twice](https://github.com/zenml-io/zenml/pull/353). We hope this makes ZenML just that little bit easier to use.
|
3373
|
-
|
3374
|
-
# 0.5.7
|
3375
|
-
|
3376
|
-
ZenML 0.5.7 is here :100: and it brings not one, but :fire:TWO:fire: brand new integrations :rocket:! ZenML now support [MLFlow](https://www.mlflow.org/docs/latest/tracking.html) for tracking pipelines as experiments and [Evidently](https://github.com/evidentlyai/evidently) for detecting drift in your ML pipelines in production!
|
3377
|
-
|
3378
|
-
## New Features
|
3379
|
-
* Introducing the [MLFlow Tracking](https://www.mlflow.org/docs/latest/tracking.html) Integration, a first step towards
|
3380
|
-
our complete MLFlow Integration as described in the [#115 poll](https://github.com/zenml-io/zenml/discussions/115).
|
3381
|
-
Full example found [here](https://github.com/zenml-io/zenml/tree/0.5.7/examples/mlflow).
|
3382
|
-
* Introducing the [Evidently](https://github.com/evidentlyai/evidently) integration. Use the standard
|
3383
|
-
[Evidently drift detection step](https://github.com/zenml-io/zenml/blob/0.5.7/src/zenml/integrations/evidently/steps/evidently_profile.py)
|
3384
|
-
to calculate drift automatically in your pipeline. Full example found [here](https://github.com/zenml-io/zenml/tree/0.5.7/examples/drift_detection).
|
3385
|
-
|
3386
|
-
## Bugfixes
|
3387
|
-
* Prevent KFP install timeouts during `stack up` by @stefannica in https://github.com/zenml-io/zenml/pull/299
|
3388
|
-
* Prevent naming parameters same name as inputs/outputs to prevent kwargs-errors by @bcdurak in https://github.com/zenml-io/zenml/pull/300
|
3389
|
-
|
3390
|
-
|
3391
|
-
## What's Changed
|
3392
|
-
* Force pull overwrites local examples without user confirmation by @AlexejPenner in https://github.com/zenml-io/zenml/pull/278
|
3393
|
-
* Updated README with latest features by @htahir1 in https://github.com/zenml-io/zenml/pull/280
|
3394
|
-
* Integration test the examples within ci pipeline by @AlexejPenner in https://github.com/zenml-io/zenml/pull/282
|
3395
|
-
* Add exception for missing system requirements by @kamalesh0406 in https://github.com/zenml-io/zenml/pull/281
|
3396
|
-
* Examples are automatically pulled if not present before any example command is run by @AlexejPenner in https://github.com/zenml-io/zenml/pull/279
|
3397
|
-
* Add pipeline error for passing the same step object twice by @kamalesh0406 in https://github.com/zenml-io/zenml/pull/283
|
3398
|
-
* Create pytest fixture to use a temporary zenml repo in tests by @htahir1 in https://github.com/zenml-io/zenml/pull/287
|
3399
|
-
* Additional example run implementations for standard interfaces, functional and class based api by @AlexejPenner in https://github.com/zenml-io/zenml/pull/286
|
3400
|
-
* Make pull_request.yaml actually use os.runner instead of ubuntu by @htahir1 in https://github.com/zenml-io/zenml/pull/288
|
3401
|
-
* In pytest return to previous workdir before tearing down tmp_dir fixture by @AlexejPenner in https://github.com/zenml-io/zenml/pull/289
|
3402
|
-
* Don't raise an exception during integration installation if system requirement is not installed by @schustmi in https://github.com/zenml-io/zenml/pull/291
|
3403
|
-
* Update starting page for the API docs by @alex-zenml in https://github.com/zenml-io/zenml/pull/294
|
3404
|
-
* Add `stack up` failure prompts by @alex-zenml in https://github.com/zenml-io/zenml/pull/290
|
3405
|
-
* Spelling fixes by @alex-zenml in https://github.com/zenml-io/zenml/pull/295
|
3406
|
-
* Remove instructions to git init from docs by @bcdurak in https://github.com/zenml-io/zenml/pull/293
|
3407
|
-
* Fix the `stack up` and `orchestrator up` failure prompts by @stefannica in https://github.com/zenml-io/zenml/pull/297
|
3408
|
-
* Prevent KFP install timeouts during `stack up` by @stefannica in https://github.com/zenml-io/zenml/pull/299
|
3409
|
-
* Add stefannica to list of internal github users by @stefannica in https://github.com/zenml-io/zenml/pull/303
|
3410
|
-
* Improve KFP UI daemon error messages by @schustmi in https://github.com/zenml-io/zenml/pull/292
|
3411
|
-
* Replaced old diagrams with new ones in the docs by @AlexejPenner in https://github.com/zenml-io/zenml/pull/306
|
3412
|
-
* Fix broken links & text formatting in docs by @alex-zenml in https://github.com/zenml-io/zenml/pull/302
|
3413
|
-
* Run KFP container as local user/group if local by @stefannica in https://github.com/zenml-io/zenml/pull/304
|
3414
|
-
* Add james to github team by @jwwwb in https://github.com/zenml-io/zenml/pull/308
|
3415
|
-
* Implement integration of mlflow tracking by @AlexejPenner in https://github.com/zenml-io/zenml/pull/301
|
3416
|
-
* Bugfix integration tests on windows by @jwwwb in https://github.com/zenml-io/zenml/pull/296
|
3417
|
-
* Prevent naming parameters same name as inputs/outputs to prevent kwargs-errors by @bcdurak in https://github.com/zenml-io/zenml/pull/300
|
3418
|
-
* Add tests for `fileio` by @alex-zenml in https://github.com/zenml-io/zenml/pull/298
|
3419
|
-
* Evidently integration (standard steps and example) by @alex-zenml in https://github.com/zenml-io/zenml/pull/307
|
3420
|
-
* Implemented evidently integration by @stefannica in https://github.com/zenml-io/zenml/pull/310
|
3421
|
-
* Make mlflow example faster by @AlexejPenner in https://github.com/zenml-io/zenml/pull/312
|
3422
|
-
|
3423
|
-
## New Contributors
|
3424
|
-
* @kamalesh0406 made their first contribution in https://github.com/zenml-io/zenml/pull/281
|
3425
|
-
* @stefannica made their first contribution in https://github.com/zenml-io/zenml/pull/297
|
3426
|
-
* @jwwwb made their first contribution in https://github.com/zenml-io/zenml/pull/308
|
3427
|
-
|
3428
|
-
|
3429
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.5.6...0.5.7
|
3430
|
-
# 0.5.6
|
3431
|
-
|
3432
|
-
) * (
|
3433
|
-
( /( ( ` )\ )
|
3434
|
-
)\()) ( )\))( (()/(
|
3435
|
-
((_)\ ))\ ( ((_)()\ /(_))
|
3436
|
-
_((_) /((_) )\ ) (_()((_) (_))
|
3437
|
-
|_ / (_)) _(_/( | \/ | | |
|
3438
|
-
/ / / -_) | ' \)) | |\/| | | |__
|
3439
|
-
/___| \___| |_||_| |_| |_| |____|
|
3440
|
-
|
3441
|
-
This release fixes some known bugs from previous releases and especially 0.5.5. Therefore, upgrading to 0.5.6 is a **breaking change**. You must do the following in order to proceed with this version:
|
3442
|
-
|
3443
|
-
```
|
3444
|
-
cd zenml_enabled_repo
|
3445
|
-
rm -rf .zen/
|
3446
|
-
```
|
3447
|
-
|
3448
|
-
And then start again with ZenML init:
|
3449
|
-
|
3450
|
-
```
|
3451
|
-
pip install --upgrade zenml
|
3452
|
-
zenml init
|
3453
|
-
```
|
3454
|
-
|
3455
|
-
## New Features
|
3456
|
-
* Added `zenml example run [EXAMPLE_RUN_NAME]` feature: The ability to run an example with one command. In order to run this, do `zenml example pull` first and see all examples available by running `zenml example list`.
|
3457
|
-
* Added ability to specify a `.dockerignore` file before running pipelines on Kubeflow.
|
3458
|
-
* Kubeflow Orchestrator is now leaner and faster.
|
3459
|
-
* Added the `describe` command group to the CLI for groups `stack`, `orchestrator`, `artifact-store`, and `metadata-store`. E.g. `zenml stack describe`
|
3460
|
-
|
3461
|
-
## Bug fixes and minor improvements
|
3462
|
-
* Adding `StepContext` to a branch now invalidates caching by default. Disable explicitly with `enable_cache=True`.
|
3463
|
-
* Docs updated to reflect minor changes in CLI commands.
|
3464
|
-
* CLI `list` commands now mentions active component. Try `zenml stack list` to check this out.
|
3465
|
-
* `zenml version` now has cooler art.
|
3466
|
-
|
3467
|
-
## What's Changed
|
3468
|
-
* Delete blog reference from release notes by @alex-zenml in https://github.com/zenml-io/zenml/pull/228
|
3469
|
-
* Docs updates by @alex-zenml in https://github.com/zenml-io/zenml/pull/229
|
3470
|
-
* Update kubeflow guide by @schustmi in https://github.com/zenml-io/zenml/pull/230
|
3471
|
-
* Updated quickstart to reflect newest zenml version by @alexej-zenml in https://github.com/zenml-io/zenml/pull/231
|
3472
|
-
* Add KFP GCP example readme by @schustmi in https://github.com/zenml-io/zenml/pull/233
|
3473
|
-
* Baris/update docs with class api by @bcdurak in https://github.com/zenml-io/zenml/pull/232
|
3474
|
-
* fixing a small typo [ci skip] by @bcdurak in https://github.com/zenml-io/zenml/pull/236
|
3475
|
-
* Hamza/docs last min updates by @htahir1 in https://github.com/zenml-io/zenml/pull/234
|
3476
|
-
* fix broken links by @alex-zenml in https://github.com/zenml-io/zenml/pull/237
|
3477
|
-
* added one more page for standardized artifacts [ci skip] by @bcdurak in https://github.com/zenml-io/zenml/pull/238
|
3478
|
-
* Unified use of cli_utils.print_table for all table format cli printouts by @AlexejPenner in https://github.com/zenml-io/zenml/pull/240
|
3479
|
-
* Remove unused tfx kubeflow code by @schustmi in https://github.com/zenml-io/zenml/pull/239
|
3480
|
-
* Relaxed typing requirements for cli_utils.print_table by @AlexejPenner in https://github.com/zenml-io/zenml/pull/241
|
3481
|
-
* Pass input artifact types to kubeflow container entrypoint by @schustmi in https://github.com/zenml-io/zenml/pull/242
|
3482
|
-
* Catch duplicate run name error and throw custom exception by @schustmi in https://github.com/zenml-io/zenml/pull/243
|
3483
|
-
* Improved logs by @htahir1 in https://github.com/zenml-io/zenml/pull/244
|
3484
|
-
* CLI active component highlighting by @alex-zenml in https://github.com/zenml-io/zenml/pull/245
|
3485
|
-
* Baris/eng 244 clean up by @bcdurak in https://github.com/zenml-io/zenml/pull/246
|
3486
|
-
* CLI describe command by @alex-zenml in https://github.com/zenml-io/zenml/pull/248
|
3487
|
-
* Alexej/eng 35 run examples from cli by @AlexejPenner in https://github.com/zenml-io/zenml/pull/253
|
3488
|
-
* CLI argument and option flag consistency improvements by @alex-zenml in https://github.com/zenml-io/zenml/pull/250
|
3489
|
-
* Invalidate caching when a step requires a step context by @schustmi in https://github.com/zenml-io/zenml/pull/252
|
3490
|
-
* Implement better error messages for custom step output artifact types by @schustmi in https://github.com/zenml-io/zenml/pull/254
|
3491
|
-
* Small improvements by @schustmi in https://github.com/zenml-io/zenml/pull/251
|
3492
|
-
* Kubeflow dockerignore by @schustmi in https://github.com/zenml-io/zenml/pull/249
|
3493
|
-
* Rename container registry folder to be consistent with the other stack components by @schustmi in https://github.com/zenml-io/zenml/pull/257
|
3494
|
-
* Update todo script by @schustmi in https://github.com/zenml-io/zenml/pull/256
|
3495
|
-
* Update docs following CLI change by @alex-zenml in https://github.com/zenml-io/zenml/pull/255
|
3496
|
-
* Bump mypy version by @schustmi in https://github.com/zenml-io/zenml/pull/258
|
3497
|
-
* Kubeflow Windows daemon alternative by @schustmi in https://github.com/zenml-io/zenml/pull/259
|
3498
|
-
* Run pre commit in local environment by @schustmi in https://github.com/zenml-io/zenml/pull/260
|
3499
|
-
* Hamza/eng 269 move beam out by @htahir1 in https://github.com/zenml-io/zenml/pull/262
|
3500
|
-
* Update docs by @alex-zenml in https://github.com/zenml-io/zenml/pull/261
|
3501
|
-
* Hamza/update readme with contribitions by @htahir1 in https://github.com/zenml-io/zenml/pull/271
|
3502
|
-
* Hamza/eng 256 backoff analytics by @htahir1 in https://github.com/zenml-io/zenml/pull/270
|
3503
|
-
* Add spellcheck by @alex-zenml in https://github.com/zenml-io/zenml/pull/264
|
3504
|
-
* Using the pipeline run name to explicitly access when explaining the … by @AlexejPenner in https://github.com/zenml-io/zenml/pull/263
|
3505
|
-
* Import user main module in kubeflow entrypoint to make sure all components are registered by @schustmi in https://github.com/zenml-io/zenml/pull/273
|
3506
|
-
* Fix cli version command by @schustmi in https://github.com/zenml-io/zenml/pull/272
|
3507
|
-
* User is informed of version mismatch and example pull defaults to cod… by @AlexejPenner in https://github.com/zenml-io/zenml/pull/274
|
3508
|
-
* Hamza/eng 274 telemetry by @htahir1 in https://github.com/zenml-io/zenml/pull/275
|
3509
|
-
* Update docs with right commands and events by @htahir1 in https://github.com/zenml-io/zenml/pull/276
|
3510
|
-
* Fixed type annotation for some python versions by @AlexejPenner in https://github.com/zenml-io/zenml/pull/277
|
3511
|
-
|
3512
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.5.5...0.5.6
|
3513
|
-
|
3514
|
-
# 0.5.5
|
3515
|
-
|
3516
|
-
ZenML 0.5.5 is jam-packed with new features to take your ML pipelines to the next level. Our three biggest new features: Kubeflow Pipelines, CLI support for our integrations and Standard Interfaces. That’s right, Standard Interfaces are back!
|
3517
|
-
|
3518
|
-
## What's Changed
|
3519
|
-
* Implement base component tests by @schustmi in https://github.com/zenml-io/zenml/pull/211
|
3520
|
-
* Add chapter names by @alex-zenml in https://github.com/zenml-io/zenml/pull/212
|
3521
|
-
* Fix docstring error by @alex-zenml in https://github.com/zenml-io/zenml/pull/213
|
3522
|
-
* Hamza/add caching example by @htahir1 in https://github.com/zenml-io/zenml/pull/214
|
3523
|
-
* Update readme by @alex-zenml in https://github.com/zenml-io/zenml/pull/216
|
3524
|
-
* Hamza/add small utils by @htahir1 in https://github.com/zenml-io/zenml/pull/219
|
3525
|
-
* Update docs by @alex-zenml in https://github.com/zenml-io/zenml/pull/220
|
3526
|
-
* Docs fixes by @alex-zenml in https://github.com/zenml-io/zenml/pull/222
|
3527
|
-
* Baris/eng 182 standard interfaces by @bcdurak in https://github.com/zenml-io/zenml/pull/209
|
3528
|
-
* Fix naming error by @alex-zenml in https://github.com/zenml-io/zenml/pull/221
|
3529
|
-
* Remove framework design by @alex-zenml in https://github.com/zenml-io/zenml/pull/224
|
3530
|
-
* Alexej/eng 234 zenml integration install by @alexej-zenml in https://github.com/zenml-io/zenml/pull/223
|
3531
|
-
* Fix deployment section order by @alex-zenml in https://github.com/zenml-io/zenml/pull/225
|
3532
|
-
* the readme of the example by @bcdurak in https://github.com/zenml-io/zenml/pull/227
|
3533
|
-
* Kubeflow integration by @schustmi in https://github.com/zenml-io/zenml/pull/226
|
3534
|
-
|
3535
|
-
## New Contributors
|
3536
|
-
* @alexej-zenml made their first contribution in https://github.com/zenml-io/zenml/pull/223
|
3537
|
-
|
3538
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.5.4...0.5.5
|
3539
|
-
|
3540
|
-
# 0.5.4
|
3541
|
-
|
3542
|
-
0.5.4 adds a [lineage tracking](https://github.com/zenml-io/zenml/tree/main/examples/lineage) integration to visualize lineage of pipeline runs! It also includes numerous bug fixes and optimizations.
|
3543
|
-
|
3544
|
-
## What's Changed
|
3545
|
-
* Fix typos by @alex-zenml in https://github.com/zenml-io/zenml/pull/192
|
3546
|
-
* Fix Apache Beam bug by @alex-zenml in https://github.com/zenml-io/zenml/pull/194
|
3547
|
-
* Fix apache beam logging bug by @alex-zenml in https://github.com/zenml-io/zenml/pull/195
|
3548
|
-
* Add step context by @schustmi in https://github.com/zenml-io/zenml/pull/196
|
3549
|
-
* Init docstrings by @alex-zenml in https://github.com/zenml-io/zenml/pull/197
|
3550
|
-
* Hamza/small fixes by @htahir1 in https://github.com/zenml-io/zenml/pull/199
|
3551
|
-
* Fix writing to metadata store with airflow orchestrator by @schustmi in https://github.com/zenml-io/zenml/pull/198
|
3552
|
-
* Use pipeline parameter name as step name in post execution by @schustmi in https://github.com/zenml-io/zenml/pull/200
|
3553
|
-
* Add error message when step name is not in metadata store by @schustmi in https://github.com/zenml-io/zenml/pull/201
|
3554
|
-
* Add option to set repo location using an environment variable by @schustmi in https://github.com/zenml-io/zenml/pull/202
|
3555
|
-
* Run cloudbuild after pypi publish by @schustmi in https://github.com/zenml-io/zenml/pull/203
|
3556
|
-
* Refactor component generation by @schustmi in https://github.com/zenml-io/zenml/pull/204
|
3557
|
-
* Removed unnecessary panel dependency by @htahir1 in https://github.com/zenml-io/zenml/pull/206
|
3558
|
-
* Updated README to successively install requirements by @AlexejPenner in https://github.com/zenml-io/zenml/pull/205
|
3559
|
-
* Store active stack in local config by @schustmi in https://github.com/zenml-io/zenml/pull/208
|
3560
|
-
* Hamza/eng 125 lineage tracking vis by @htahir1 in https://github.com/zenml-io/zenml/pull/207
|
3561
|
-
|
3562
|
-
## New Contributors
|
3563
|
-
* @AlexejPenner made their first contribution in https://github.com/zenml-io/zenml/pull/205
|
3564
|
-
|
3565
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.5.3...0.5.4
|
3566
|
-
|
3567
|
-
# 0.5.3
|
3568
|
-
|
3569
|
-
Version 0.5.3 adds [statistics visualizations](https://github.com/zenml-io/zenml/blob/main/examples/visualizers/statistics/README.md), greatly improved speed for CLI commands as well as lots of small improvements to the pipeline and step interface.
|
3570
|
-
|
3571
|
-
## What's Changed
|
3572
|
-
* Make tests run in a random order by @alex-zenml in https://github.com/zenml-io/zenml/pull/160
|
3573
|
-
* Connect steps using *args by @schustmi in https://github.com/zenml-io/zenml/pull/162
|
3574
|
-
* Move location of repobeats image by @alex-zenml in https://github.com/zenml-io/zenml/pull/163
|
3575
|
-
* Hamza/add sam by @htahir1 in https://github.com/zenml-io/zenml/pull/165
|
3576
|
-
* Pipeline initialization with *args by @schustmi in https://github.com/zenml-io/zenml/pull/164
|
3577
|
-
* Improve detection of third party modules during class resolving by @schustmi in https://github.com/zenml-io/zenml/pull/167
|
3578
|
-
* Merge path_utils into fileio & refactor what was left by @alex-zenml in https://github.com/zenml-io/zenml/pull/168
|
3579
|
-
* Update docker files by @schustmi in https://github.com/zenml-io/zenml/pull/169
|
3580
|
-
* Hamza/deploy api reference by @htahir1 in https://github.com/zenml-io/zenml/pull/171
|
3581
|
-
* API Reference by @schustmi in https://github.com/zenml-io/zenml/pull/172
|
3582
|
-
* Add color back into our github actions by @alex-zenml in https://github.com/zenml-io/zenml/pull/176
|
3583
|
-
* Refactor tests not raising by @alex-zenml in https://github.com/zenml-io/zenml/pull/177
|
3584
|
-
* Improve step and pipeline interface by @schustmi in https://github.com/zenml-io/zenml/pull/175
|
3585
|
-
* Alex/eng 27 windows bug again by @htahir1 in https://github.com/zenml-io/zenml/pull/178
|
3586
|
-
* Automated todo tracking by @schustmi in https://github.com/zenml-io/zenml/pull/173
|
3587
|
-
* Fix mypy issues related to windows by @schustmi in https://github.com/zenml-io/zenml/pull/179
|
3588
|
-
* Include Github URL to TODO comment in issue by @schustmi in https://github.com/zenml-io/zenml/pull/181
|
3589
|
-
* Create Visualizers logic by @htahir1 in https://github.com/zenml-io/zenml/pull/182
|
3590
|
-
* Add README for visualizers examples by @alex-zenml in https://github.com/zenml-io/zenml/pull/184
|
3591
|
-
* Allow None as default value for BaseStep configs by @schustmi in https://github.com/zenml-io/zenml/pull/185
|
3592
|
-
* Baris/eng 37 standard import check by @bcdurak in https://github.com/zenml-io/zenml/pull/183
|
3593
|
-
* Replace duplicated code by call to source_utils.resolve_class by @schustmi in https://github.com/zenml-io/zenml/pull/186
|
3594
|
-
* Remove unused base enum cases by @schustmi in https://github.com/zenml-io/zenml/pull/187
|
3595
|
-
* Testing mocks for CLI `examples` command by @alex-zenml in https://github.com/zenml-io/zenml/pull/180
|
3596
|
-
* Set the correct module for steps created using our decorator by @schustmi in https://github.com/zenml-io/zenml/pull/188
|
3597
|
-
* Fix some cli commands by @schustmi in https://github.com/zenml-io/zenml/pull/189
|
3598
|
-
* Tag jira issues for which the todo was deleted by @schustmi in https://github.com/zenml-io/zenml/pull/190
|
3599
|
-
* Remove deadlinks by @alex-zenml in https://github.com/zenml-io/zenml/pull/191
|
3600
|
-
|
3601
|
-
|
3602
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.5.2...0.5.3
|
3603
|
-
|
3604
|
-
# 0.5.2
|
3605
|
-
|
3606
|
-
0.5.2 brings an improved post-execution workflow and lots of minor changes and upgrades for the developer experience when
|
3607
|
-
creating pipelines. It also improves the Airflow orchestrator logic to accommodate for more real world scenarios.
|
3608
|
-
|
3609
|
-
## What's Changed
|
3610
|
-
* Fix autocomplete for step and pipeline decorated functions by @schustmi in https://github.com/zenml-io/zenml/pull/144
|
3611
|
-
* Add reference docs for CLI example functionality by @alex-zenml in https://github.com/zenml-io/zenml/pull/145
|
3612
|
-
* Fix mypy integration by @schustmi in https://github.com/zenml-io/zenml/pull/147
|
3613
|
-
* Improve Post-Execution Workflow by @schustmi in https://github.com/zenml-io/zenml/pull/146
|
3614
|
-
* Fix CLI examples bug by @alex-zenml in https://github.com/zenml-io/zenml/pull/148
|
3615
|
-
* Update quickstart example notebook by @alex-zenml in https://github.com/zenml-io/zenml/pull/150
|
3616
|
-
* Add documentation images by @alex-zenml in https://github.com/zenml-io/zenml/pull/151
|
3617
|
-
* Add prettierignore to gitignore by @alex-zenml in https://github.com/zenml-io/zenml/pull/154
|
3618
|
-
* Airflow orchestrator improvements by @schustmi in https://github.com/zenml-io/zenml/pull/153
|
3619
|
-
* Google colab added by @htahir1 in https://github.com/zenml-io/zenml/pull/155
|
3620
|
-
* Tests for `core` and `cli` modules by @alex-zenml in https://github.com/zenml-io/zenml/pull/149
|
3621
|
-
* Add Paperspace environment check by @alex-zenml in https://github.com/zenml-io/zenml/pull/156
|
3622
|
-
* Step caching by @schustmi in https://github.com/zenml-io/zenml/pull/157
|
3623
|
-
* Add documentation for pipeline step parameter and run name configuration by @schustmi in https://github.com/zenml-io/zenml/pull/158
|
3624
|
-
* Automatically disable caching if the step function code has changed by @schustmi in https://github.com/zenml-io/zenml/pull/159
|
3625
|
-
|
3626
|
-
|
3627
|
-
**Full Changelog**: https://github.com/zenml-io/zenml/compare/0.5.1...0.5.2
|
3628
|
-
|
3629
|
-
# 0.5.1
|
3630
|
-
0.5.1 builds on top of Slack of the 0.5.0 release with quick bug updates.
|
3631
|
-
|
3632
|
-
|
3633
|
-
## Overview
|
3634
|
-
|
3635
|
-
* Pipeline can now be run via a YAML file. #132
|
3636
|
-
* CLI now let's you pull directly from GitHub examples folder. :fire: Amazing @alex-zenml with #141!
|
3637
|
-
* ZenML now has full [mypy](http://mypy-lang.org/) compliance. :tada: Thanks @schustmi for #140!
|
3638
|
-
* Numerous bugs and performance improvements. #136, @bcdurak great job with #142
|
3639
|
-
* Added new docs with a low level API guide. #143
|
3640
|
-
|
3641
|
-
[Our roadmap](https://zenml.hellonext.co/roadmap) goes into further detail on the timeline. Vote on the [next features now](https://github.com/zenml-io/zenml/discussions).
|
3642
|
-
|
3643
|
-
We encourage every user (old or new) to start afresh with this release. Please go over our latest [docs](https://docs.zenml.io) and [examples](examples) to get a hang of the new system.
|
3644
|
-
|
3645
|
-
|
3646
|
-
# 0.5.0
|
3647
|
-
This long-awaited ZenML release marks a seminal moment in the project's history. We present to you a complete
|
3648
|
-
revamp of the internals of ZenML, with a fresh new design and API. While these changes are significant, and have been months
|
3649
|
-
in the making, the original vision of ZenML has not wavered. We hope that the ZenML community finds the new
|
3650
|
-
design choices easier to grasp and use, and we welcome feedback on the [issues board](https://github.com/zenml-io/zenml/issues).
|
3651
|
-
|
3652
|
-
## Warning
|
3653
|
-
0.5.0 is a complete API change from the previous versions of ZenML, and is a *breaking* upgrade. Fundamental
|
3654
|
-
concepts have been changed, and therefore backwards compatibility is not maintained. Please use only this version
|
3655
|
-
with fresh projects.
|
3656
|
-
|
3657
|
-
With such significant changes, we expect this release to also be breaking. Please report any bugs in the issue board, and
|
3658
|
-
they should be addressed in upcoming releases.
|
3659
|
-
|
3660
|
-
## Overview
|
3661
|
-
|
3662
|
-
* Introducing a new functional API for creating pipelines and steps. This is now the default mechanism for building ZenML pipelines. [read more](https://docs.zenml.io/starter-guide/pipelines/pipelines)
|
3663
|
-
* Steps now use Materializers to handle artifact serialization/deserialization between steps. This is a powerful change, and will be expanded upon in the future. [read more](https://docs.zenml.io/pipelines/materializers)
|
3664
|
-
* Introducing the new `Stack` paradigm: Easily transition from one MLOps stack to the next with a few CLI commands [read more](https://docs.zenml.io/starter-guide/stacks/stacks)
|
3665
|
-
* Introducing a new `Artifact`, `Typing`, and `Annotation` system, with `pydantic` (and `dataclasses`) support [read more](https://docs.zenml.io/getting-started/core-concepts)
|
3666
|
-
* Deprecating the `pipelines_dir`: Now individual pipelines will be stored in their metadata stores, making the metadata store a single source of truth. [read more](https://docs.zenml.io/getting-started/core-concepts)
|
3667
|
-
* Deprecating the YAML config file: ZenML no longer natively compiles to an intermediate YAML-based representation. Instead, it compiles and deploys directly into the selected orchestrator's
|
3668
|
-
representation. While we do plan to support running pipelines directly through YAML in the future, it will no longer be
|
3669
|
-
the default route through which pipelines are run. [read more about orchestrators here](https://docs.zenml.io/component-gallery/orchestrators/orchestrators)
|
3670
|
-
|
3671
|
-
## Technical Improvements
|
3672
|
-
* A completely new system design, please refer to the [docs](https://docs.zenml.io/getting-started/core-concepts).
|
3673
|
-
* Better type hints and docstrings.
|
3674
|
-
* Auto-completion support.
|
3675
|
-
* Numerous performance improvements and bug fixes, including a smaller dependency footprint.
|
3676
|
-
|
3677
|
-
## What to expect in the next weeks and the new ZenML
|
3678
|
-
Currently, this release is bare bones. We are missing some basic features which used to be part of ZenML 0.3.8 (the previous release):
|
3679
|
-
|
3680
|
-
* Standard interfaces for `TrainingPipeline`.
|
3681
|
-
* Individual step interfaces like `PreprocessorStep`, `TrainerStep`, `DeployerStep` etc. need to be rewritten from within the new paradigm. They should
|
3682
|
-
be included in the non-RC version of this release.
|
3683
|
-
* A proper production setup with an orchestrator like Airflow.
|
3684
|
-
* A post-execution workflow to analyze and inspect pipeline runs.
|
3685
|
-
* The concept of `Backends` will evolve into a simple mechanism of transitioning individual steps into different runners.
|
3686
|
-
* Support for `KubernetesOrchestrator`, `KubeflowOrchestrator`, `GCPOrchestrator` and `AWSOrchestrator` are also planned.
|
3687
|
-
* Dependency management including Docker support is planned.
|
3688
|
-
|
3689
|
-
[Our roadmap](https://zenml.hellonext.co/roadmap) goes into further detail on the timeline.
|
3690
|
-
|
3691
|
-
We encourage every user (old or new) to start afresh with this release. Please go over our latest [docs](https://docs.zenml.io)
|
3692
|
-
and [examples](examples) to get a hang of the new system.
|
3693
|
-
|
3694
|
-
Onwards and upwards to 1.0.0!
|
3695
|
-
|
3696
|
-
# 0.5.0rc2
|
3697
|
-
This long-awaited ZenML release marks a seminal moment in the project's history. We present to you a complete
|
3698
|
-
revamp of the internals of ZenML, with a fresh new design and API. While these changes are significant, and have been months
|
3699
|
-
in the making, the original vision of ZenML has not wavered. We hope that the ZenML community finds the new
|
3700
|
-
design choices easier to grasp and use, and we welcome feedback on the [issues board](https://github.com/zenml-io/zenml/issues).
|
3701
|
-
|
3702
|
-
## Warning
|
3703
|
-
0.5.0rc0 is a complete API change from the previous versions of ZenML, and is a *breaking* upgrade. Fundamental
|
3704
|
-
concepts have been changed, and therefore backwards compatibility is not maintained. Please use only this version
|
3705
|
-
with fresh projects.
|
3706
|
-
|
3707
|
-
With such significant changes, we expect this release to also be breaking. Please report any bugs in the issue board, and
|
3708
|
-
they should be addressed in upcoming releases.
|
3709
|
-
|
3710
|
-
## Overview
|
3711
|
-
|
3712
|
-
* Introducing a new functional API for creating pipelines and steps. This is now the default mechanism for building ZenML pipelines. [read more](https://docs.zenml.io/starter-guide/pipelines/pipelines)
|
3713
|
-
* Introducing the new `Stack` paradigm: Easily transition from one MLOps stack to the next with a few CLI commands [read more](https://docs.zenml.io/starter-guide/stacks/stacks)
|
3714
|
-
* Introducing a new `Artifact`, `Typing`, and `Annotation` system, with `pydantic` (and `dataclasses`) support [read more](https://docs.zenml.io/getting-started/core-concepts)
|
3715
|
-
* Deprecating the `pipelines_dir`: Now individual pipelines will be stored in their metadata stores, making the metadata store a single source of truth. [read more](https://docs.zenml.io/starter-guide/stacks/stacks)
|
3716
|
-
* Deprecating the YAML config file: ZenML no longer natively compiles to an intermediate YAML-based representation. Instead, it compiles and deploys directly into the selected orchestrator's
|
3717
|
-
representation. While we do plan to support running pipelines directly through YAML in the future, it will no longer be
|
3718
|
-
the default route through which pipelines are run. [read more about orchestrators here](https://docs.zenml.io/core/stacks)
|
3719
|
-
|
3720
|
-
## Technical Improvements
|
3721
|
-
* A completely new system design, please refer to the [docs](https://docs.zenml.io/component-gallery/orchestrators/orchestrators).
|
3722
|
-
* Better type hints and docstrings.
|
3723
|
-
* Auto-completion support.
|
3724
|
-
* Numerous performance improvements and bug fixes, including a smaller dependency footprint.
|
3725
|
-
|
3726
|
-
## What to expect in the next weeks and the new ZenML
|
3727
|
-
Currently, this release is bare bones. We are missing some basic features which used to be part of ZenML 0.3.8 (the previous release):
|
3728
|
-
|
3729
|
-
* Standard interfaces for `TrainingPipeline`.
|
3730
|
-
* Individual step interfaces like `PreprocessorStep`, `TrainerStep`, `DeployerStep` etc. need to be rewritten from within the new paradigm. They should
|
3731
|
-
be included in the non-RC version of this release.
|
3732
|
-
* A proper production setup with an orchestrator like Airflow.
|
3733
|
-
* A post-execution workflow to analyze and inspect pipeline runs.
|
3734
|
-
* The concept of `Backends` will evolve into a simple mechanism of transitioning individual steps into different runners.
|
3735
|
-
* Support for `KubernetesOrchestrator`, `KubeflowOrchestrator`, `GCPOrchestrator` and `AWSOrchestrator` are also planned.
|
3736
|
-
* Dependency management including Docker support is planned.
|
3737
|
-
|
3738
|
-
[Our roadmap](https://zenml.hellonext.co/roadmap) goes into further detail on the timeline.
|
3739
|
-
|
3740
|
-
We encourage every user (old or new) to start afresh with this release. Please go over our latest [docs](https://docs.zenml.io)
|
3741
|
-
and [examples](examples) to get a hang of the new system.
|
3742
|
-
|
3743
|
-
Onwards and upwards to 1.0.0!
|
3744
|
-
|
3745
|
-
# 0.3.7.1
|
3746
|
-
This release fixes some known bugs from previous releases and especially 0.3.7. Same procedure as always, please delete existing pipelines, metadata, and artifact stores.
|
3747
|
-
|
3748
|
-
```
|
3749
|
-
cd zenml_enabled_repo
|
3750
|
-
rm -rf pipelines/
|
3751
|
-
rm -rf .zenml/
|
3752
|
-
```
|
3753
|
-
|
3754
|
-
And then another ZenML init:
|
3755
|
-
|
3756
|
-
```
|
3757
|
-
pip install --upgrade zenml
|
3758
|
-
cd zenml_enabled_repo
|
3759
|
-
zenml init
|
3760
|
-
```
|
3761
|
-
|
3762
|
-
## New Features
|
3763
|
-
* Introduced new `zenml example` CLI sub-group: Easily pull examples via zenml to check it out.
|
3764
|
-
|
3765
|
-
```bash
|
3766
|
-
zenml example pull # pulls all examples in `zenml_examples` directory
|
3767
|
-
zenml example pull EXAMPLE_NAME # pulls specific example
|
3768
|
-
zenml example info EXAMPLE_NAME # gives quick info regarding example
|
3769
|
-
```
|
3770
|
-
Thanks Michael Xu for the suggestion!
|
3771
|
-
|
3772
|
-
* Updated examples with new `zenml examples` paradigm for examples.
|
3773
|
-
|
3774
|
-
## Bug Fixes + Refactor
|
3775
|
-
|
3776
|
-
* ZenML now works on Windows -> Thank you @Franky007Bond for the heads up.
|
3777
|
-
* Updated numerous bugs in examples directory. Also updated README's.
|
3778
|
-
* Fixed remote orchestration logic -> Now remote orchestration works.
|
3779
|
-
* Changed datasource `to_config` to include reference to backend, metadata, and artifact store.
|
3780
|
-
|
3781
|
-
|
3782
|
-
# 0.3.7
|
3783
|
-
0.3.7 is a much-needed, long-awaited, big refactor of the Datasources paradigm of ZenML. There are also bug fixes, improvements, and more!
|
3784
|
-
|
3785
|
-
For those upgrading from an older version of ZenML, we ask to please delete their old `pipelines` dir and `.zenml` folders and start afresh with a `zenml init`.
|
3786
|
-
|
3787
|
-
If only working locally, this is as simple as:
|
3788
|
-
|
3789
|
-
```
|
3790
|
-
cd zenml_enabled_repo
|
3791
|
-
rm -rf pipelines/
|
3792
|
-
rm -rf .zenml/
|
3793
|
-
```
|
3794
|
-
|
3795
|
-
And then another ZenML init:
|
3796
|
-
|
3797
|
-
```
|
3798
|
-
pip install --upgrade zenml
|
3799
|
-
cd zenml_enabled_repo
|
3800
|
-
zenml init
|
3801
|
-
```
|
3802
|
-
|
3803
|
-
## New Features
|
3804
|
-
* The inner-workings of the `BaseDatasource` have been modified along with the concrete implementations. Now, there is no relation between a `DataStep` and a `Datasource`: A `Datasource` holds all the logic to version and track itself via the new `commit` paradigm.
|
3805
|
-
|
3806
|
-
* Introduced a new interface for datasources, the `process` method which is responsible for ingesting data and writing to TFRecords to be consumed by later steps.
|
3807
|
-
|
3808
|
-
* Datasource versions (snapshots) can be accessed directly via the `commits` paradigm: Every commit is a new version of data.
|
3809
|
-
|
3810
|
-
* Added `JSONDatasource` and `TFRecordsDatasource`.
|
3811
|
-
|
3812
|
-
## Bug Fixes + Refactor
|
3813
|
-
A big thanks to our new contributor @aak7912 for the help in this release with issue #71 and PR #75.
|
3814
|
-
|
3815
|
-
* Added an example for [regression](https://github.com/zenml-io/zenml/tree/main/examples/regression).
|
3816
|
-
* `compare_training_runs()` now takes an optional `datasource` parameter to filter by datasource.
|
3817
|
-
* `Trainer` interface refined to focus on `run_fn` rather than other helper functions.
|
3818
|
-
* New docs released with a streamlined vision and coherent storyline: https://docs.zenml.io
|
3819
|
-
* Got rid of unnecessary Torch dependency with base ZenML version.
|
3820
|
-
|
3821
|
-
|
3822
|
-
# 0.3.6
|
3823
|
-
0.3.6 is a more inwards-facing release as part of a bigger effort to create a more flexible ZenML. As a first step, ZenML now supports arbitrary splits for all components natively, freeing us from the `train/eval` split paradigm. Here is an overview of changes:
|
3824
|
-
|
3825
|
-
## New Features
|
3826
|
-
* The inner-workings of the `BaseTrainerStep`, `BaseEvaluatorStep` and the `BasePreprocessorStep` have been modified along with their respective components to work with the new split_mapping. Now, users can define arbitrary splits (not just train/eval). E.g. Doing a `train/eval/test` split is possible.
|
3827
|
-
|
3828
|
-
* Within the instance of a `TrainerStep`, the user has access to `input_patterns` and `output_patterns` which provide the required uris with respect to their splits for the input and output(test_results) examples.
|
3829
|
-
|
3830
|
-
* The built-in trainers are modified to work with the new changes.
|
3831
|
-
|
3832
|
-
## Bug Fixes + Refactor
|
3833
|
-
A big thanks to our new super supporter @zyfzjsc988 for most of the feedback that led to bug fixes and enhancements for this release:
|
3834
|
-
|
3835
|
-
* #63: Now one can specify which ports ZenML opens its add-on applications.
|
3836
|
-
* #64 Now there is a way to list integrations with the following code:
|
3837
|
-
```
|
3838
|
-
from zenml.utils.requirements_utils import list_integrations.
|
3839
|
-
list_integrations()
|
3840
|
-
```
|
3841
|
-
* Fixed #61: `view_anomalies()` breaking in the quickstart.
|
3842
|
-
* Analytics is now `opt-in` by default, to get rid of the unnecessary prompt at `zenml init`. Users can still freely `opt-out` by using the CLI:
|
3843
|
-
|
3844
|
-
```
|
3845
|
-
zenml config analytics opt-out
|
3846
|
-
```
|
3847
|
-
|
3848
|
-
Again, the telemetry data is fully anonymized and just used to improve the product. Read more [here](https://docs.zenml.io/misc/usage-analytics)
|
3849
|
-
|
3850
|
-
# 0.3.5
|
3851
|
-
|
3852
|
-
## New Features
|
3853
|
-
* Added a new interface into the trainer step called [`test_fn`]() which is utilized to produce model predictions and save them as test results
|
3854
|
-
|
3855
|
-
* Implemented a new evaluator step called [`AgnosticEvaluator`]() which is designed to work regardless of the model type as long as you run the `test_fn` in your trainer step
|
3856
|
-
|
3857
|
-
* The first two changes allow torch trainer steps to be followed by an agnostic evaluator step, see the example [here]().
|
3858
|
-
|
3859
|
-
* Proposed a new naming scheme, which is now integrated into the built-in steps, in order to make it easier to handle feature/label names
|
3860
|
-
|
3861
|
-
* Implemented a new adapted version of 2 TFX components, namely the [`Trainer`]() and the [`Evaluator`]() to allow the aforementioned changes to take place
|
3862
|
-
|
3863
|
-
* Modified the [`TorchFeedForwardTrainer`]() to showcase how to use TensorBoard in conjunction with PyTorch
|
3864
|
-
|
3865
|
-
|
3866
|
-
## Bug Fixes + Refactor
|
3867
|
-
* Refactored how ZenML treats relative imports for custom steps. Now:
|
3868
|
-
```python
|
3869
|
-
|
3870
|
-
```
|
3871
|
-
|
3872
|
-
* Updated the [Scikit Example](https://github.com/zenml-io/zenml/tree/main/examples/scikit), [PyTorch Lightning Example](https://github.com/zenml-io/zenml/tree/main/examples/pytorch_lightning), [GAN Example](https://github.com/zenml-io/zenml/tree/main/examples/gan) accordingly. Now they should work according to their README's.
|
3873
|
-
|
3874
|
-
Big shout out to @SarahKing92 in issue #34 for raising the above issues!
|
3875
|
-
|
3876
|
-
|
3877
|
-
# 0.3.4
|
3878
|
-
This release is a big design change and refactor. It involves a significant change in the Configuration file structure, meaning this is a **breaking upgrade**.
|
3879
|
-
For those upgrading from an older version of ZenML, we ask to please delete their old `pipelines` dir and `.zenml` folders and start afresh with a `zenml init`.
|
3880
|
-
|
3881
|
-
If only working locally, this is as simple as:
|
3882
|
-
|
3883
|
-
```
|
3884
|
-
cd zenml_enabled_repo
|
3885
|
-
rm -rf pipelines/
|
3886
|
-
rm -rf .zenml/
|
3887
|
-
```
|
3888
|
-
|
3889
|
-
And then another ZenML init:
|
3890
|
-
|
3891
|
-
```
|
3892
|
-
pip install --upgrade zenml
|
3893
|
-
cd zenml_enabled_repo
|
3894
|
-
zenml init
|
3895
|
-
```
|
3896
|
-
|
3897
|
-
## New Features
|
3898
|
-
* Introduced another higher-level pipeline: The [NLPPipeline](https://github.com/zenml-io/zenml/blob/main/zenml/pipelines/nlp_pipeline.py). This is a generic
|
3899
|
-
NLP pipeline for a text-datasource based training task. Full example of how to use the NLPPipeline can be found [here](https://github.com/zenml-io/zenml/tree/main/examples/nlp)
|
3900
|
-
* Introduced a [BaseTokenizerStep](https://github.com/zenml-io/zenml/blob/main/zenml/steps/tokenizer/base_tokenizer.py) as a simple mechanism to define how to train and encode using any generic
|
3901
|
-
tokenizer (again for NLP-based tasks).
|
3902
|
-
|
3903
|
-
## Bug Fixes + Refactor
|
3904
|
-
* Significant change to imports: Now imports are way simpler and user-friendly. E.g. Instead of:
|
3905
|
-
```python
|
3906
|
-
from zenml.core.pipelines.training_pipeline import TrainingPipeline
|
3907
|
-
```
|
3908
|
-
|
3909
|
-
A user can simple do:
|
3910
|
-
|
3911
|
-
```python
|
3912
|
-
from zenml.pipelines import TrainingPipeline
|
3913
|
-
```
|
3914
|
-
|
3915
|
-
The caveat is of course that this might involve a re-write of older ZenML code imports.
|
3916
|
-
|
3917
|
-
Note: Future releases are also expected to be breaking. Until announced, please expect that upgrading ZenML versions may cause older-ZenML
|
3918
|
-
generated pipelines to behave unexpectedly.
|
3919
|
-
<!-- -->
|