deployml-core 0.0.28__tar.gz

deployml_core-0.0.28/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2025 Drew Hoang
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

deployml_core-0.0.28/PKG-INFO

@@ -0,0 +1,111 @@
+Metadata-Version: 2.4
+Name: deployml-core
+Version: 0.0.28
+Summary: Infra for academia
+License-File: LICENSE
+Author: Drew Hoang
+Author-email: codentell@gmail.com
+Requires-Python: >=3.11
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Requires-Dist: google-cloud-storage (>=3.1.1,<4.0.0)
+Requires-Dist: ipython (>=9.0.0)
+Requires-Dist: jinja2
+Requires-Dist: jupyter (>=1.0.0)
+Requires-Dist: pandas (>=2.2.1)
+Requires-Dist: pyyaml
+Requires-Dist: requests (>=2.28.0)
+Requires-Dist: rich
+Requires-Dist: typer
+Description-Content-Type: text/markdown
+
+# deployml
+Infrastructure for academia with cost analysis
+
+## Features
+
+- 🏗️ **Infrastructure as Code**: Deploy ML infrastructure using Terraform
+- 💰 **Cost Analysis**: Integrated infracost analysis before deployment
+- ☁️ **Multi-Cloud Support**: GCP, AWS, and more
+- 🔬 **ML-Focused**: Pre-configured for MLflow, experiment tracking, and model registry
+- 🛡️ **Production Ready**: Security best practices and service account management
+
+## Instructions
+
+```bash
+poetry install
+poetry run deployml doctor
+poetry run deployml deploy --config-path your-config.yaml
+```
+
+docker build --platform=linux/amd64 -t gcr.io/mlops-intro-461805/mlflow/mlflow:latest .
+
+gcloud auth configure-docker docker push gcr.io/PROJECT_ID/mlflow-app:latest
+
+## Cost Analysis Integration
+
+deployml integrates with [infracost](https://www.infracost.io/) to provide cost estimates before deployment:
+
+### Installation
+```bash
+brew install infracost
+```
+Once installed you will need to create a free [infracost](https://www.infracost.io/) account before creating your API key. 
+
+To generate your infracost API key run the following command:
+
+```bash
+infracost auth login
+```
+If you want to retrieve your API key use:
+```bash
+infracost configure get api_key
+```
+
+### Cost Analysis Configuration
+Add cost analysis settings to your YAML configuration:
+
+```yaml
+name: "my-mlops-stack"
+cost_analysis:
+  enabled: true              # Enable/disable cost analysis (default: true)
+  warning_threshold: 100.0   # Warn if monthly cost exceeds this amount
+  currency: "USD"            # Currency for cost display
+```
+
+## Cloud Run Service Account Setup
+
+When deploying MLflow, you must specify the service account email that Cloud Run will use. This service account must have permission to access the artifact bucket.
+
+### How to get a service account email
+
+1. List your service accounts:
+   ```sh
+   gcloud iam service-accounts list
+   ```
+2. (Recommended) Create a dedicated service account for MLflow if you don't have one:
+   ```sh
+   gcloud iam service-accounts create mlflow-cloudrun --display-name "MLflow Cloud Run Service Account"
+   ```
+3. Find the email for your service account (it will look like `mlflow-cloudrun@YOUR_PROJECT.iam.gserviceaccount.com`).
+
+### Grant the service account permissions
+
+The Terraform module will automatically grant the service account the required permissions on the artifact bucket.
+
+### Deploying with Terraform
+
+Pass the service account email as a variable:
+
+```sh
+terraform apply -var="cloud_run_service_account=mlflow-cloudrun@YOUR_PROJECT.iam.gserviceaccount.com"
+```
+
+Or, if using the CLI, ensure it passes this variable to Terraform.
+
+### Why this is needed
+
+The MLflow server (running on Cloud Run) needs permission to list and read artifacts in the GCS bucket. This setup ensures the MLflow UI works for all users without manual permission fixes.

deployml_core-0.0.28/README.md

@@ -0,0 +1,87 @@
+# deployml
+Infrastructure for academia with cost analysis
+
+## Features
+
+- 🏗️ **Infrastructure as Code**: Deploy ML infrastructure using Terraform
+- 💰 **Cost Analysis**: Integrated infracost analysis before deployment
+- ☁️ **Multi-Cloud Support**: GCP, AWS, and more
+- 🔬 **ML-Focused**: Pre-configured for MLflow, experiment tracking, and model registry
+- 🛡️ **Production Ready**: Security best practices and service account management
+
+## Instructions
+
+```bash
+poetry install
+poetry run deployml doctor
+poetry run deployml deploy --config-path your-config.yaml
+```
+
+docker build --platform=linux/amd64 -t gcr.io/mlops-intro-461805/mlflow/mlflow:latest .
+
+gcloud auth configure-docker docker push gcr.io/PROJECT_ID/mlflow-app:latest
+
+## Cost Analysis Integration
+
+deployml integrates with [infracost](https://www.infracost.io/) to provide cost estimates before deployment:
+
+### Installation
+```bash
+brew install infracost
+```
+Once installed you will need to create a free [infracost](https://www.infracost.io/) account before creating your API key. 
+
+To generate your infracost API key run the following command:
+
+```bash
+infracost auth login
+```
+If you want to retrieve your API key use:
+```bash
+infracost configure get api_key
+```
+
+### Cost Analysis Configuration
+Add cost analysis settings to your YAML configuration:
+
+```yaml
+name: "my-mlops-stack"
+cost_analysis:
+  enabled: true              # Enable/disable cost analysis (default: true)
+  warning_threshold: 100.0   # Warn if monthly cost exceeds this amount
+  currency: "USD"            # Currency for cost display
+```
+
+## Cloud Run Service Account Setup
+
+When deploying MLflow, you must specify the service account email that Cloud Run will use. This service account must have permission to access the artifact bucket.
+
+### How to get a service account email
+
+1. List your service accounts:
+   ```sh
+   gcloud iam service-accounts list
+   ```
+2. (Recommended) Create a dedicated service account for MLflow if you don't have one:
+   ```sh
+   gcloud iam service-accounts create mlflow-cloudrun --display-name "MLflow Cloud Run Service Account"
+   ```
+3. Find the email for your service account (it will look like `mlflow-cloudrun@YOUR_PROJECT.iam.gserviceaccount.com`).
+
+### Grant the service account permissions
+
+The Terraform module will automatically grant the service account the required permissions on the artifact bucket.
+
+### Deploying with Terraform
+
+Pass the service account email as a variable:
+
+```sh
+terraform apply -var="cloud_run_service_account=mlflow-cloudrun@YOUR_PROJECT.iam.gserviceaccount.com"
+```
+
+Or, if using the CLI, ensure it passes this variable to Terraform.
+
+### Why this is needed
+
+The MLflow server (running on Cloud Run) needs permission to list and read artifacts in the GCS bucket. This setup ensures the MLflow UI works for all users without manual permission fixes.

deployml_core-0.0.28/pyproject.toml

@@ -0,0 +1,36 @@
+
+[project]
+name = "deployml-core"
+version = "0.0.28"
+description = "Infra for academia"
+authors = [
+    {name = "Drew Hoang", email = "codentell@gmail.com"},
+    {name = "Jarvin Bayona", email = "jarvin.bayona@gmail.com"},
+    {name = "Grant Nitta", email = "gtnitta@gmail.com"},
+    {name = "Robert Clements", email = "rclements@usfca.edu"}
+]
+readme = "README.md"
+requires-python = ">=3.11"
+dependencies = [
+    "typer",
+    "pyyaml", 
+    "jinja2",
+    "rich",
+    "google-cloud-storage>=3.1.1,<4.0.0",
+    "ipython>=9.0.0",
+    "jupyter>=1.0.0",
+    "requests>=2.28.0",
+    "pandas>=2.2.1"
+]
+
+[project.scripts]
+deployml = "deployml.cli.cli:main"
+
+[tool.poetry]
+packages = [{include = "deployml", from = "src"}]
+
+
+
+[build-system]
+requires = ["poetry-core>=2.0.0,<3.0.0"]
+build-backend = "poetry.core.masonry.api"

deployml_core-0.0.28/src/deployml/__init__.py

@@ -0,0 +1,15 @@
+# Import notebook functionality for easy access
+from .notebook import deploy, load, DeploymentStack, ServiceURLs
+
+# Import diagnostics for easy access
+from .diagnostics import run_doctor, check_system, DeployMLDoctor
+
+__all__ = [
+    'deploy', 
+    'load', 
+    'DeploymentStack',
+    'ServiceURLs',
+    'run_doctor',
+    'check_system', 
+    'DeployMLDoctor'
+]

deployml_core-0.0.28/src/deployml/api.py

@@ -0,0 +1,262 @@
+"""
+DeployML Python API
+
+Programmatic interface for managing DeployML deployments and teardown schedules.
+
+Example usage:
+    import deployml.api as api
+    
+    # Get teardown status
+    status = api.get_teardown_status(
+        project_id="my-project",
+        region="us-west1",
+        workspace_name="my-stack"
+    )
+    
+    # Update teardown schedule
+    result = api.update_teardown_schedule(
+        project_id="my-project",
+        region="us-west1",
+        workspace_name="my-stack",
+        duration_hours=6
+    )
+    
+    # Cancel teardown
+    api.cancel_teardown(
+        project_id="my-project",
+        region="us-west1",
+        workspace_name="my-stack"
+    )
+"""
+import json
+import subprocess
+from pathlib import Path
+from datetime import datetime, timedelta, timezone
+from typing import Optional, Dict, Any
+
+from .utils.teardown import (
+    calculate_cron_from_timestamp,
+    load_deployment_metadata,
+    save_deployment_metadata
+)
+
+
+def get_teardown_status(
+    project_id: str,
+    region: str,
+    workspace_name: str,
+    deployml_dir: Optional[Path] = None
+) -> Dict[str, Any]:
+    """
+    Get teardown status from Cloud Scheduler.
+    
+    Args:
+        project_id: GCP project ID
+        region: GCP region
+        workspace_name: Workspace name
+        deployml_dir: Optional path to deployment directory for metadata
+    
+    Returns:
+        Dictionary with status information including:
+        - exists: bool - Whether the scheduler job exists
+        - state: str - Job state (ENABLED, PAUSED, etc.)
+        - schedule: str - Cron schedule
+        - schedule_time: datetime - Next scheduled execution time
+        - time_zone: str - Timezone
+        - last_attempt_time: Optional[datetime] - Last execution time
+        - metadata: Optional[Dict] - Local metadata if available
+    """
+    scheduler_job_name = f"deployml-teardown-{workspace_name}"
+    
+    result = subprocess.run(
+        ["gcloud", "scheduler", "jobs", "describe", scheduler_job_name,
+         "--project", project_id, "--location", region, "--format", "json"],
+        capture_output=True,
+        text=True,
+    )
+    
+    status = {
+        "exists": False,
+        "scheduler_job_name": scheduler_job_name,
+    }
+    
+    # Load local metadata if available
+    if deployml_dir:
+        metadata = load_deployment_metadata(deployml_dir)
+        status["metadata"] = metadata
+    
+    if result.returncode != 0:
+        return status
+    
+    try:
+        job_info = json.loads(result.stdout)
+        status["exists"] = True
+        status["state"] = job_info.get("state", "UNKNOWN")
+        status["schedule"] = job_info.get("schedule", "")
+        status["time_zone"] = job_info.get("timeZone", "UTC")
+        
+        # Parse schedule time
+        schedule_time = job_info.get("scheduleTime", "")
+        if schedule_time:
+            time_str = schedule_time.replace('Z', '+00:00')
+            status["schedule_time"] = datetime.fromisoformat(time_str)
+        
+        # Parse last attempt time
+        last_attempt_time = job_info.get("lastAttemptTime", "")
+        if last_attempt_time and last_attempt_time != "1970-01-01T00:00:00Z":
+            time_str = last_attempt_time.replace('Z', '+00:00')
+            status["last_attempt_time"] = datetime.fromisoformat(time_str)
+        
+        return status
+    except json.JSONDecodeError:
+        return status
+
+
+def update_teardown_schedule(
+    project_id: str,
+    region: str,
+    workspace_name: str,
+    duration_hours: int,
+    deployml_dir: Optional[Path] = None,
+    time_zone: str = "UTC"
+) -> Dict[str, Any]:
+    """
+    Update teardown schedule programmatically.
+    
+    Args:
+        project_id: GCP project ID
+        region: GCP region
+        workspace_name: Workspace name
+        duration_hours: Hours until teardown
+        deployml_dir: Optional path to deployment directory for metadata
+        time_zone: Timezone (default: UTC)
+    
+    Returns:
+        Dictionary with update result:
+        - success: bool - Whether update succeeded
+        - scheduled_time: datetime - New scheduled time
+        - cron_schedule: str - Cron expression
+        - error: Optional[str] - Error message if failed
+    """
+    scheduler_job_name = f"deployml-teardown-{workspace_name}"
+    
+    # Check if job exists and get current timezone
+    result = subprocess.run(
+        ["gcloud", "scheduler", "jobs", "describe", scheduler_job_name,
+         "--project", project_id, "--location", region, "--format", "json"],
+        capture_output=True,
+        text=True,
+    )
+    
+    if result.returncode != 0:
+        return {
+            "success": False,
+            "error": f"Cloud Scheduler job not found: {scheduler_job_name}"
+        }
+    
+    # Get timezone from existing job if not provided
+    try:
+        job_info = json.loads(result.stdout)
+        time_zone = job_info.get("timeZone", time_zone)
+    except json.JSONDecodeError:
+        pass
+    
+    # Calculate new teardown time
+    now = datetime.now(timezone.utc)
+    teardown_at = now + timedelta(hours=duration_hours)
+    teardown_scheduled_timestamp = int(teardown_at.timestamp())
+    new_cron_schedule = calculate_cron_from_timestamp(teardown_scheduled_timestamp)
+    
+    # Update Cloud Scheduler job
+    update_result = subprocess.run(
+        [
+            "gcloud", "scheduler", "jobs", "update", "http", scheduler_job_name,
+            "--location", region,
+            "--schedule", new_cron_schedule,
+            "--time-zone", time_zone,
+            "--project", project_id,
+            "--quiet"
+        ],
+        capture_output=True,
+        text=True,
+    )
+    
+    if update_result.returncode != 0:
+        return {
+            "success": False,
+            "error": update_result.stderr,
+            "scheduled_time": teardown_at,
+            "cron_schedule": new_cron_schedule
+        }
+    
+    # Update local metadata if directory provided
+    if deployml_dir:
+        metadata = load_deployment_metadata(deployml_dir) or {}
+        metadata.update({
+            "deployed_at": metadata.get("deployed_at", now.isoformat()),
+            "teardown_scheduled_at": teardown_at.isoformat(),
+            "teardown_enabled": True,
+            "duration_hours": duration_hours,
+            "scheduler_job_name": scheduler_job_name
+        })
+        save_deployment_metadata(deployml_dir, metadata)
+    
+    return {
+        "success": True,
+        "scheduled_time": teardown_at,
+        "cron_schedule": new_cron_schedule,
+        "time_zone": time_zone
+    }
+
+
+def cancel_teardown(
+    project_id: str,
+    region: str,
+    workspace_name: str,
+    deployml_dir: Optional[Path] = None
+) -> Dict[str, Any]:
+    """
+    Cancel scheduled teardown programmatically.
+    
+    Args:
+        project_id: GCP project ID
+        region: GCP region
+        workspace_name: Workspace name
+        deployml_dir: Optional path to deployment directory for metadata
+    
+    Returns:
+        Dictionary with cancellation result:
+        - success: bool - Whether cancellation succeeded
+        - error: Optional[str] - Error message if failed
+    """
+    scheduler_job_name = f"deployml-teardown-{workspace_name}"
+    
+    result = subprocess.run(
+        ["gcloud", "scheduler", "jobs", "delete", scheduler_job_name,
+         "--project", project_id, "--location", region, "--quiet"],
+        capture_output=True,
+        text=True,
+    )
+    
+    if result.returncode != 0:
+        return {
+            "success": False,
+            "error": result.stderr
+        }
+    
+    # Update metadata if directory provided
+    if deployml_dir:
+        metadata = load_deployment_metadata(deployml_dir)
+        if metadata:
+            metadata["teardown_enabled"] = False
+            save_deployment_metadata(deployml_dir, metadata)
+    
+    return {"success": True}
+
+
+__all__ = [
+    'get_teardown_status',
+    'update_teardown_schedule',
+    'cancel_teardown'
+]
+