metaflow 2.15.3__tar.gz → 2.15.5__tar.gz

metaflow-2.15.5/PKG-INFO

@@ -0,0 +1,103 @@
+Metadata-Version: 2.2
+Name: metaflow
+Version: 2.15.5
+Summary: Metaflow: More AI and ML, Less Engineering
+Author: Metaflow Developers
+Author-email: help@metaflow.org
+License: Apache Software License
+Project-URL: Source, https://github.com/Netflix/metaflow
+Project-URL: Issues, https://github.com/Netflix/metaflow/issues
+Project-URL: Documentation, https://docs.metaflow.org
+Classifier: Development Status :: 5 - Production/Stable
+Classifier: License :: OSI Approved :: Apache Software License
+Classifier: Operating System :: MacOS :: MacOS X
+Classifier: Operating System :: POSIX :: Linux
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.6
+Classifier: Programming Language :: Python :: 3.7
+Classifier: Programming Language :: Python :: 3.8
+Classifier: Programming Language :: Python :: 3.9
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: requests
+Requires-Dist: boto3
+Provides-Extra: stubs
+Requires-Dist: metaflow-stubs==2.15.5; extra == "stubs"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: license
+Dynamic: project-url
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: summary
+
+![Metaflow_Logo_Horizontal_FullColor_Ribbon_Dark_RGB](https://user-images.githubusercontent.com/763451/89453116-96a57e00-d713-11ea-9fa6-82b29d4d6eff.png)
+
+# Metaflow
+
+[Metaflow](https://metaflow.org) is a human-centric framework designed to help scientists and engineers **build and manage real-life AI and ML systems**. Serving teams of all sizes and scale, Metaflow streamlines the entire development lifecycle—from rapid prototyping in notebooks to reliable, maintainable production deployments—enabling teams to iterate quickly and deliver robust systems efficiently.
+
+Originally developed at [Netflix](https://netflixtechblog.com/open-sourcing-metaflow-a-human-centric-framework-for-data-science-fa72e04a5d9) and now supported by [Outerbounds](https://outerbounds.com), Metaflow is designed to boost the productivity for research and engineering teams working on [a wide variety of projects](https://netflixtechblog.com/supporting-diverse-ml-systems-at-netflix-2d2e6b6d205d), from classical statistics to state-of-the-art deep learning and foundation models. By unifying code, data, and compute at every stage, Metaflow ensures seamless, end-to-end management of real-world AI and ML systems.
+
+Today, Metaflow powers thousands of AI and ML experiences across a diverse array of companies, large and small, including Amazon, Doordash, Dyson, Goldman Sachs, Ramp, and [many others](ADOPTERS.md). At Netflix alone, Metaflow supports over 3000 AI and ML projects, executes hundreds of millions of data-intensive high-performance compute jobs processing petabytes of data and manages tens of petabytes of models and artifacts for hundreds of users across its AI, ML, data science, and engineering teams.
+
+## From prototype to production (and back)
+
+Metaflow provides a simple and friendly pythonic [API](https://docs.metaflow.org) that covers foundational needs of AI and ML systems:
+<img src="./docs/prototype-to-prod.png" width="800px">
+
+1. [Rapid local prototyping](https://docs.metaflow.org/metaflow/basics), [support for notebooks](https://docs.metaflow.org/metaflow/managing-flows/notebook-runs), and built-in support for [experiment tracking, versioning](https://docs.metaflow.org/metaflow/client) and [visualization](https://docs.metaflow.org/metaflow/visualizing-results).
+2. [Effortlessly scale horizontally and vertically in your cloud](https://docs.metaflow.org/scaling/remote-tasks/introduction), utilizing both CPUs and GPUs, with [fast data access](https://docs.metaflow.org/scaling/data) for running [massive embarrassingly parallel](https://docs.metaflow.org/metaflow/basics#foreach) as well as [gang-scheduled](https://docs.metaflow.org/scaling/remote-tasks/distributed-computing) compute workloads [reliably](https://docs.metaflow.org/scaling/failures) and [efficiently](https://docs.metaflow.org/scaling/checkpoint/introduction).
+3. [Easily manage dependencies](https://docs.metaflow.org/scaling/dependencies) and [deploy with one-click](https://docs.metaflow.org/production/introduction) to highly available production orchestrators with built in support for [reactive orchestration](https://docs.metaflow.org/production/event-triggering).
+
+For full documentation, check out our [API Reference](https://docs.metaflow.org/api) or see our [Release Notes](https://github.com/Netflix/metaflow/releases) for the latest features and improvements. 
+
+
+## Getting started
+
+Getting up and running is easy. If you don't know where to start, [Metaflow sandbox](https://outerbounds.com/sandbox) will have you running and exploring in seconds.
+
+### Installing Metaflow
+
+To install Metaflow in your Python environment from [PyPI](https://pypi.org/project/metaflow/):
+
+```sh
+pip install metaflow
+```
+Alternatively, using [conda-forge](https://anaconda.org/conda-forge/metaflow):
+
+```sh
+conda install -c conda-forge metaflow
+```
+
+Once installed, a great way to get started is by following our [tutorial](https://docs.metaflow.org/getting-started/tutorials). It walks you through creating and running your first Metaflow flow step by step.  
+
+For more details on Metaflow’s features and best practices, check out:
+- [How Metaflow works](https://docs.metaflow.org/metaflow/basics)  
+- [Additional resources](https://docs.metaflow.org/introduction/metaflow-resources)  
+
+If you need help, don’t hesitate to reach out on our [Slack community](http://slack.outerbounds.co/)!
+
+
+### Deploying infrastructure for Metaflow in your cloud
+<img src="./docs/multicloud.png" width="800px">
+
+
+While you can get started with Metaflow easily on your laptop, the main benefits of Metaflow lie in its ability to [scale out to external compute clusters](https://docs.metaflow.org/scaling/remote-tasks/introduction) 
+and to [deploy to production-grade workflow orchestrators](https://docs.metaflow.org/production/introduction). To benefit from these features, follow this [guide](https://outerbounds.com/engineering/welcome/) to 
+configure Metaflow and the infrastructure behind it appropriately.
+
+
+## Get in touch
+We'd love to hear from you. Join our community [Slack workspace](http://slack.outerbounds.co/)!
+
+## Contributing
+We welcome contributions to Metaflow. Please see our [contribution guide](https://docs.metaflow.org/introduction/contributing-to-metaflow) for more details.
+

metaflow-2.15.5/README.md

@@ -0,0 +1,63 @@
+![Metaflow_Logo_Horizontal_FullColor_Ribbon_Dark_RGB](https://user-images.githubusercontent.com/763451/89453116-96a57e00-d713-11ea-9fa6-82b29d4d6eff.png)
+
+# Metaflow
+
+[Metaflow](https://metaflow.org) is a human-centric framework designed to help scientists and engineers **build and manage real-life AI and ML systems**. Serving teams of all sizes and scale, Metaflow streamlines the entire development lifecycle—from rapid prototyping in notebooks to reliable, maintainable production deployments—enabling teams to iterate quickly and deliver robust systems efficiently.
+
+Originally developed at [Netflix](https://netflixtechblog.com/open-sourcing-metaflow-a-human-centric-framework-for-data-science-fa72e04a5d9) and now supported by [Outerbounds](https://outerbounds.com), Metaflow is designed to boost the productivity for research and engineering teams working on [a wide variety of projects](https://netflixtechblog.com/supporting-diverse-ml-systems-at-netflix-2d2e6b6d205d), from classical statistics to state-of-the-art deep learning and foundation models. By unifying code, data, and compute at every stage, Metaflow ensures seamless, end-to-end management of real-world AI and ML systems.
+
+Today, Metaflow powers thousands of AI and ML experiences across a diverse array of companies, large and small, including Amazon, Doordash, Dyson, Goldman Sachs, Ramp, and [many others](ADOPTERS.md). At Netflix alone, Metaflow supports over 3000 AI and ML projects, executes hundreds of millions of data-intensive high-performance compute jobs processing petabytes of data and manages tens of petabytes of models and artifacts for hundreds of users across its AI, ML, data science, and engineering teams.
+
+## From prototype to production (and back)
+
+Metaflow provides a simple and friendly pythonic [API](https://docs.metaflow.org) that covers foundational needs of AI and ML systems:
+<img src="./docs/prototype-to-prod.png" width="800px">
+
+1. [Rapid local prototyping](https://docs.metaflow.org/metaflow/basics), [support for notebooks](https://docs.metaflow.org/metaflow/managing-flows/notebook-runs), and built-in support for [experiment tracking, versioning](https://docs.metaflow.org/metaflow/client) and [visualization](https://docs.metaflow.org/metaflow/visualizing-results).
+2. [Effortlessly scale horizontally and vertically in your cloud](https://docs.metaflow.org/scaling/remote-tasks/introduction), utilizing both CPUs and GPUs, with [fast data access](https://docs.metaflow.org/scaling/data) for running [massive embarrassingly parallel](https://docs.metaflow.org/metaflow/basics#foreach) as well as [gang-scheduled](https://docs.metaflow.org/scaling/remote-tasks/distributed-computing) compute workloads [reliably](https://docs.metaflow.org/scaling/failures) and [efficiently](https://docs.metaflow.org/scaling/checkpoint/introduction).
+3. [Easily manage dependencies](https://docs.metaflow.org/scaling/dependencies) and [deploy with one-click](https://docs.metaflow.org/production/introduction) to highly available production orchestrators with built in support for [reactive orchestration](https://docs.metaflow.org/production/event-triggering).
+
+For full documentation, check out our [API Reference](https://docs.metaflow.org/api) or see our [Release Notes](https://github.com/Netflix/metaflow/releases) for the latest features and improvements. 
+
+
+## Getting started
+
+Getting up and running is easy. If you don't know where to start, [Metaflow sandbox](https://outerbounds.com/sandbox) will have you running and exploring in seconds.
+
+### Installing Metaflow
+
+To install Metaflow in your Python environment from [PyPI](https://pypi.org/project/metaflow/):
+
+```sh
+pip install metaflow
+```
+Alternatively, using [conda-forge](https://anaconda.org/conda-forge/metaflow):
+
+```sh
+conda install -c conda-forge metaflow
+```
+
+Once installed, a great way to get started is by following our [tutorial](https://docs.metaflow.org/getting-started/tutorials). It walks you through creating and running your first Metaflow flow step by step.  
+
+For more details on Metaflow’s features and best practices, check out:
+- [How Metaflow works](https://docs.metaflow.org/metaflow/basics)  
+- [Additional resources](https://docs.metaflow.org/introduction/metaflow-resources)  
+
+If you need help, don’t hesitate to reach out on our [Slack community](http://slack.outerbounds.co/)!
+
+
+### Deploying infrastructure for Metaflow in your cloud
+<img src="./docs/multicloud.png" width="800px">
+
+
+While you can get started with Metaflow easily on your laptop, the main benefits of Metaflow lie in its ability to [scale out to external compute clusters](https://docs.metaflow.org/scaling/remote-tasks/introduction) 
+and to [deploy to production-grade workflow orchestrators](https://docs.metaflow.org/production/introduction). To benefit from these features, follow this [guide](https://outerbounds.com/engineering/welcome/) to 
+configure Metaflow and the infrastructure behind it appropriately.
+
+
+## Get in touch
+We'd love to hear from you. Join our community [Slack workspace](http://slack.outerbounds.co/)!
+
+## Contributing
+We welcome contributions to Metaflow. Please see our [contribution guide](https://docs.metaflow.org/introduction/contributing-to-metaflow) for more details.
+

{metaflow-2.15.3 → metaflow-2.15.5}/devtools/Makefile

@@ -22,6 +22,7 @@ DEVTOOLS_DIR := $(MKFILE_DIR).devtools
 PICK_SERVICES := $(MKFILE_DIR)pick_services.sh
 MINIKUBE_DIR := $(DEVTOOLS_DIR)/minikube
 MINIKUBE := $(MINIKUBE_DIR)/minikube
+HELM_DIR := $(DEVTOOLS_DIR)/helm
 TILT_DIR := $(DEVTOOLS_DIR)/tilt
 TILT := $(TILT_DIR)/tilt
 TILTFILE := $(MKFILE_DIR)/Tiltfile
@@ -51,9 +52,18 @@ endif
 
 install-helm:
 	@if ! command -v helm >/dev/null 2>&1; then \
-		echo "📥 Installing Helm $(HELM_VERSION) (may require sudo access)..."; \
-		curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | HELM_INSTALL_VERSION=$(HELM_VERSION) bash; \
+		echo "📥 Installing Helm $(HELM_VERSION)..."; \
+		mkdir -p "$(HELM_DIR)"; \
+		curl -fsSL https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 \
+			| HELM_INSTALL_VERSION="$(HELM_VERSION)" \
+			  USE_SUDO="false" \
+			  PATH="$(HELM_DIR):$$PATH" \
+			  HELM_INSTALL_DIR="$(HELM_DIR)" \
+			  bash; \
+		chmod +x "$(HELM_DIR)/helm"; \
 		echo "✅ Helm installation complete"; \
+	else \
+		echo "✅ Helm is already installed at $$(command -v helm)"; \
 	fi
 
 check-docker:
@@ -99,7 +109,7 @@ install-gum:
 	@if ! command -v gum >/dev/null 2>&1; then \
 		echo "📥 Installing gum..."; \
 		if [ "$(shell uname)" = "Darwin" ]; then \
-			HOMEBREW_NO_AUTO_UPDATE=1 brew install gum; \
+			HOMEBREW_NO_AUTO_UPDATE=1 brew install gum|| { echo "❌ Failed to install gum via Homebrew"; exit 1; }; \
 		elif command -v apt-get >/dev/null 2>&1; then \
 			curl -fsSL -o /tmp/gum.deb \
 			"https://github.com/charmbracelet/gum/releases/download/$(GUM_VERSION)/gum_$(GUM_VERSION:v%=%)_$(arch).deb"; \
@@ -182,7 +192,7 @@ up: install-brew check-docker install-curl install-gum setup-minikube install-he
 	@echo 'PATH="$(MINIKUBE_DIR):$(TILT_DIR):$$PATH" $(MINIKUBE) tunnel &' >> $(DEVTOOLS_DIR)/start.sh
 	@echo 'echo -e "🚀 Starting Tilt with selected services..."' >> $(DEVTOOLS_DIR)/start.sh
 	@echo 'echo -e "\033[1;38;5;46m\n🔥 \033[1;38;5;196mNext Steps:\033[0;38;5;46m Use \033[3mmetaflow-dev shell\033[23m to switch to the development\n   environment'\''s shell and start executing your Metaflow flows.\n\033[0m"' >> "$(DEVTOOLS_DIR)/start.sh"
-	@echo 'PATH="$(MINIKUBE_DIR):$(TILT_DIR):$$PATH" SERVICES="$$SERVICES" tilt up -f $(TILTFILE)' >> $(DEVTOOLS_DIR)/start.sh
+	@echo 'PATH="$(HELM_DIR):$(MINIKUBE_DIR):$(TILT_DIR):$$PATH" SERVICES="$$SERVICES" tilt up -f $(TILTFILE)' >> $(DEVTOOLS_DIR)/start.sh
 	@echo 'wait' >> $(DEVTOOLS_DIR)/start.sh
 	@chmod +x $(DEVTOOLS_DIR)/start.sh
 	@$(DEVTOOLS_DIR)/start.sh

{metaflow-2.15.3 → metaflow-2.15.5}/devtools/Tiltfile

@@ -529,8 +529,8 @@ if "metadata-service" in enabled_components:
             'metadatadb.password=metaflow123',
             'metadatadb.database=metaflow',
             'metadatadb.host=postgresql',
-            'image.repository=public.ecr.aws/p7g1e3j4/metaflow-service',
-            'image.tag=2.4.13-fbcc7d04',
+            'image.repository=public.ecr.aws/outerbounds/metaflow_metadata_service',
+            'image.tag=2.4.13-2-g70af4ed',
             'resources.requests.cpu=25m',
             'resources.requests.memory=64Mi',
             'resources.limits.cpu=50m',
@@ -567,8 +567,8 @@ if "ui" in enabled_components:
             'uiBackend.metadatadb.host=postgresql',
             'uiBackend.metaflowDatastoreSysRootS3=s3://metaflow-test',
             'uiBackend.metaflowS3EndpointURL=http://minio.default.svc.cluster.local:9000',
-            'uiBackend.image.name=public.ecr.aws/p7g1e3j4/metaflow-service',
-            'uiBackend.image.tag=2.4.13-fbcc7d04',
+            'uiBackend.image.name=public.ecr.aws/outerbounds/metaflow_metadata_service',
+            'uiBackend.image.tag=2.4.13-2-g70af4ed',
             'uiBackend.env[0].name=AWS_ACCESS_KEY_ID',
             'uiBackend.env[0].value=rootuser',
             'uiBackend.env[1].name=AWS_SECRET_ACCESS_KEY',

{metaflow-2.15.3 → metaflow-2.15.5}/metaflow/plugins/argo/argo_client.py

@@ -256,12 +256,19 @@ class ArgoClient(object):
                 json.loads(e.body)["message"] if e.body is not None else e.reason
             )
 
-    def trigger_workflow_template(self, name, parameters={}):
+    def trigger_workflow_template(self, name, usertype, username, parameters={}):
         client = self._client.get()
         body = {
             "apiVersion": "argoproj.io/v1alpha1",
             "kind": "Workflow",
-            "metadata": {"generateName": name + "-"},
+            "metadata": {
+                "generateName": name + "-",
+                "annotations": {
+                    "metaflow/triggered_by_user": json.dumps(
+                        {"type": usertype, "name": username}
+                    )
+                },
+            },
             "spec": {
                 "workflowTemplateRef": {"name": name},
                 "arguments": {

{metaflow-2.15.3 → metaflow-2.15.5}/metaflow/plugins/argo/argo_workflows.py

@@ -64,6 +64,7 @@ from metaflow.util import (
 )
 
 from .argo_client import ArgoClient
+from metaflow.util import resolve_identity
 
 
 class ArgoWorkflowsException(MetaflowException):
@@ -108,8 +109,7 @@ class ArgoWorkflows(object):
         notify_slack_webhook_url=None,
         notify_pager_duty_integration_key=None,
         notify_incident_io_api_key=None,
-        incident_io_success_severity_id=None,
-        incident_io_error_severity_id=None,
+        incident_io_alert_source_config_id=None,
         enable_heartbeat_daemon=True,
         enable_error_msg_capture=False,
     ):
@@ -160,8 +160,7 @@ class ArgoWorkflows(object):
         self.notify_slack_webhook_url = notify_slack_webhook_url
         self.notify_pager_duty_integration_key = notify_pager_duty_integration_key
         self.notify_incident_io_api_key = notify_incident_io_api_key
-        self.incident_io_success_severity_id = incident_io_success_severity_id
-        self.incident_io_error_severity_id = incident_io_error_severity_id
+        self.incident_io_alert_source_config_id = incident_io_alert_source_config_id
         self.enable_heartbeat_daemon = enable_heartbeat_daemon
         self.enable_error_msg_capture = enable_error_msg_capture
         self.parameters = self._process_parameters()
@@ -315,8 +314,16 @@ class ArgoWorkflows(object):
                     "Workflows before proceeding." % name
                 )
         try:
+            id_parts = resolve_identity().split(":")
+            parts_size = len(id_parts)
+            usertype = id_parts[0] if parts_size > 0 else "unknown"
+            username = id_parts[1] if parts_size > 1 else "unknown"
+
             return ArgoClient(namespace=KUBERNETES_NAMESPACE).trigger_workflow_template(
-                name, parameters
+                name,
+                usertype,
+                username,
+                parameters,
             )
         except Exception as e:
             raise ArgoWorkflowsException(str(e))
@@ -2505,25 +2512,49 @@ class ArgoWorkflows(object):
     def _incident_io_alert_template(self):
         if self.notify_incident_io_api_key is None:
             return None
-        if self.incident_io_error_severity_id is None:
+        if self.incident_io_alert_source_config_id is None:
             raise MetaflowException(
-                "Creating incidents for errors requires a severity id."
+                "Creating alerts for errors requires a alert source config ID."
             )
+        ui_links = self._incident_io_ui_urls_for_run()
         return Template("notify-incident-io-on-error").http(
             Http("POST")
-            .url("https://api.incident.io/v2/incidents")
+            .url(
+                "https://api.incident.io/v2/alert_events/http/%s"
+                % self.incident_io_alert_source_config_id
+            )
             .header("Content-Type", "application/json")
             .header("Authorization", "Bearer %s" % self.notify_incident_io_api_key)
             .body(
                 json.dumps(
                     {
                         "idempotency_key": "argo-{{workflow.name}}",  # use run id to deduplicate alerts.
-                        "visibility": "public",
-                        "severity_id": self.incident_io_error_severity_id,
-                        "name": "Flow %s has failed." % self.flow.name,
-                        "summary": "Metaflow run %s/argo-{{workflow.name}} failed! %s"
-                        % (self.flow.name, self._incident_io_ui_urls_for_run()),
-                        # TODO: Add support for custom field entries.
+                        "status": "firing",
+                        "title": "Flow %s has failed." % self.flow.name,
+                        "description": "Metaflow run {run_pathspec} failed!{urls}".format(
+                            run_pathspec="%s/argo-{{workflow.name}}" % self.flow.name,
+                            urls=(
+                                "\n\nSee details for the run at:\n\n"
+                                + "\n\n".join(ui_links)
+                                if ui_links
+                                else ""
+                            ),
+                        ),
+                        "source_url": (
+                            "%s/%s/%s"
+                            % (
+                                UI_URL.rstrip("/"),
+                                self.flow.name,
+                                "argo-{{workflow.name}}",
+                            )
+                            if UI_URL
+                            else None
+                        ),
+                        "metadata": {
+                            "run_status": "failed",
+                            "flow_name": self.flow.name,
+                            "run_id": "argo-{{workflow.name}}",
+                        },
                     }
                 )
             )
@@ -2532,27 +2563,49 @@ class ArgoWorkflows(object):
     def _incident_io_change_template(self):
         if self.notify_incident_io_api_key is None:
             return None
-        if self.incident_io_success_severity_id is None:
+        if self.incident_io_alert_source_config_id is None:
             raise MetaflowException(
-                "Creating incidents for successes requires a severity id."
+                "Creating alerts for successes requires an alert source config ID."
             )
+        ui_links = self._incident_io_ui_urls_for_run()
         return Template("notify-incident-io-on-success").http(
             Http("POST")
-            .url("https://api.incident.io/v2/incidents")
+            .url(
+                "https://api.incident.io/v2/alert_events/http/%s"
+                % self.incident_io_alert_source_config_id
+            )
             .header("Content-Type", "application/json")
             .header("Authorization", "Bearer %s" % self.notify_incident_io_api_key)
             .body(
                 json.dumps(
                     {
                         "idempotency_key": "argo-{{workflow.name}}",  # use run id to deduplicate alerts.
-                        "visibility": "public",
-                        "severity_id": self.incident_io_success_severity_id,
-                        # TODO: Do we need to make incident type configurable for successes? otherwise they are created as 'investigating'
-                        # "incident_type_id": ""
-                        "name": "Flow %s has succeeded." % self.flow.name,
-                        "summary": "Metaflow run %s/argo-{{workflow.name}} succeeded!%s"
-                        % (self.flow.name, self._incident_io_ui_urls_for_run()),
-                        # TODO: Add support for custom field entries.
+                        "status": "firing",
+                        "title": "Flow %s has succeeded." % self.flow.name,
+                        "description": "Metaflow run {run_pathspec} succeeded!{urls}".format(
+                            run_pathspec="%s/argo-{{workflow.name}}" % self.flow.name,
+                            urls=(
+                                "\n\nSee details for the run at:\n\n"
+                                + "\n\n".join(ui_links)
+                                if ui_links
+                                else ""
+                            ),
+                        ),
+                        "source_url": (
+                            "%s/%s/%s"
+                            % (
+                                UI_URL.rstrip("/"),
+                                self.flow.name,
+                                "argo-{{workflow.name}}",
+                            )
+                            if UI_URL
+                            else None
+                        ),
+                        "metadata": {
+                            "run_status": "succeeded",
+                            "flow_name": self.flow.name,
+                            "run_id": "argo-{{workflow.name}}",
+                        },
                     }
                 )
             )
@@ -2574,9 +2627,7 @@ class ArgoWorkflows(object):
                 "{{workflow.name}}",
             )
             links.append(url)
-        if links:
-            links = ["See details for the run at: ", *links]
-        return "\n\n".join(links)
+        return links
 
     def _pager_duty_change_template(self):
         # https://developer.pagerduty.com/docs/ZG9jOjExMDI5NTgy-send-a-change-event

{metaflow-2.15.3 → metaflow-2.15.5}/metaflow/plugins/argo/argo_workflows_cli.py

@@ -130,6 +130,7 @@ def argo_workflows(obj, name=None):
     is_flag=True,
     default=False,
     help="Only print out JSON sent to Argo Workflows. Do not deploy anything.",
+    hidden=True,
 )
 @click.option(
     "--max-workers",
@@ -182,14 +183,9 @@ def argo_workflows(obj, name=None):
     help="Incident.io API V2 key for workflow success/failure notifications.",
 )
 @click.option(
-    "--incident-io-success-severity-id",
-    default=None,
-    help="Incident.io severity id for success alerts.",
-)
-@click.option(
-    "--incident-io-error-severity-id",
+    "--incident-io-alert-source-config-id",
     default=None,
-    help="Incident.io severity id for error alerts.",
+    help="Incident.io Alert source config ID. Example '01GW2G3V0S59R238FAHPDS1R66'",
 )
 @click.option(
     "--enable-heartbeat-daemon/--no-enable-heartbeat-daemon",
@@ -229,8 +225,7 @@ def create(
     notify_slack_webhook_url=None,
     notify_pager_duty_integration_key=None,
     notify_incident_io_api_key=None,
-    incident_io_success_severity_id=None,
-    incident_io_error_severity_id=None,
+    incident_io_alert_source_config_id=None,
     enable_heartbeat_daemon=True,
     deployer_attribute_file=None,
     enable_error_msg_capture=False,
@@ -287,8 +282,7 @@ def create(
         notify_slack_webhook_url,
         notify_pager_duty_integration_key,
         notify_incident_io_api_key,
-        incident_io_success_severity_id,
-        incident_io_error_severity_id,
+        incident_io_alert_source_config_id,
         enable_heartbeat_daemon,
         enable_error_msg_capture,
     )
@@ -464,8 +458,7 @@ def make_flow(
     notify_slack_webhook_url,
     notify_pager_duty_integration_key,
     notify_incident_io_api_key,
-    incident_io_success_severity_id,
-    incident_io_error_severity_id,
+    incident_io_alert_source_config_id,
     enable_heartbeat_daemon,
     enable_error_msg_capture,
 ):
@@ -488,19 +481,18 @@ def make_flow(
             "https://api.slack.com/messaging/webhooks to generate a webhook url.\n"
             " For notifications through PagerDuty, generate an integration key by following the instructions at "
             "https://support.pagerduty.com/docs/services-and-integrations#create-a-generic-events-api-integration\n"
-            " For notifications through Incident.io, generate an API key with a permission to create incidents."
+            " For notifications through Incident.io, generate an alert source config."
         )
 
-    if notify_incident_io_api_key:
-        if notify_on_error and incident_io_error_severity_id is None:
-            raise MetaflowException(
-                "Incident.io error notifications require a severity id. Please set one with --incident-io-error-severity-id"
-            )
+    if (
+        (notify_on_error or notify_on_success)
+        and notify_incident_io_api_key
+        and incident_io_alert_source_config_id is None
+    ):
+        raise MetaflowException(
+            "Incident.io alerts require an alert source configuration ID. Please set one with --incident-io-alert-source-config-id"
+        )
 
-        if notify_on_success and incident_io_success_severity_id is None:
-            raise MetaflowException(
-                "Incident.io success notifications require a severity id. Please set one with --incident-io-success-severity-id"
-            )
     # Attach @kubernetes and @environment decorator to the flow to
     # ensure that the related decorator hooks are invoked.
     decorators._attach_decorators(
@@ -545,8 +537,7 @@ def make_flow(
         notify_slack_webhook_url=notify_slack_webhook_url,
         notify_pager_duty_integration_key=notify_pager_duty_integration_key,
         notify_incident_io_api_key=notify_incident_io_api_key,
-        incident_io_success_severity_id=incident_io_success_severity_id,
-        incident_io_error_severity_id=incident_io_error_severity_id,
+        incident_io_alert_source_config_id=incident_io_alert_source_config_id,
         enable_heartbeat_daemon=enable_heartbeat_daemon,
         enable_error_msg_capture=enable_error_msg_capture,
     )

{metaflow-2.15.3 → metaflow-2.15.5}/metaflow/plugins/argo/argo_workflows_deployer_objects.py

@@ -171,12 +171,16 @@ class ArgoWorkflowsTriggeredRun(TriggeredRun):
         command_obj.sync_wait()
         return command_obj.process.returncode == 0
 
-    def wait_for_completion(self, timeout: Optional[int] = None):
+    def wait_for_completion(
+        self, check_interval: int = 5, timeout: Optional[int] = None
+    ):
         """
         Wait for the workflow to complete or timeout.
 
         Parameters
         ----------
+        check_interval: int, default: 5
+            Frequency of checking for workflow completion, in seconds.
         timeout : int, optional, default None
             Maximum time in seconds to wait for workflow completion.
             If None, waits indefinitely.
@@ -187,7 +191,6 @@ class ArgoWorkflowsTriggeredRun(TriggeredRun):
             If the workflow does not complete within the specified timeout period.
         """
         start_time = time.time()
-        check_interval = 5
         while self.is_running:
             if timeout is not None and (time.time() - start_time) > timeout:
                 raise TimeoutError(