gmicloud 0.1.5__tar.gz → 0.1.6__tar.gz

gmicloud-0.1.6/PKG-INFO

@@ -0,0 +1,147 @@
+Metadata-Version: 2.2
+Name: gmicloud
+Version: 0.1.6
+Summary: GMI Cloud Python SDK
+Author-email: GMI <support@gmicloud.ai>
+License: MIT
+Classifier: Programming Language :: Python :: 3
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Requires-Python: >=3.6
+Description-Content-Type: text/markdown
+
+# GMICloud SDK (Beta)
+
+## Overview
+Before you start: Our service and GPU resource is currenly invite-only so please contact our team (getstarted@gmicloud.ai) to get invited if you don't have one yet.
+
+The GMI Inference Engine SDK provides a Python interface for deploying and managing machine learning models in production environments. It allows users to create model artifacts, schedule tasks for serving models, and call inference APIs easily.
+
+This SDK streamlines the process of utilizing GMI Cloud capabilities such as deploying models with Kubernetes-based Ray services, managing resources automatically, and accessing model inference endpoints. With minimal setup, developers can focus on building ML solutions instead of infrastructure.
+
+## Features
+
+- Artifact Management: Easily create, update, and manage ML model artifacts.
+- Task Management: Quickly create, schedule, and manage deployment tasks for model inference.
+- Usage Data Retrieval : Fetch and analyze usage data to optimize resource allocation.
+
+## Installation
+
+To install the SDK, use pip:
+
+```bash
+pip install gmicloud
+```
+
+## Setup
+
+You must configure authentication credentials for accessing the GMI Cloud API. 
+To create account and get log in info please visit **GMI inference platform: https://inference-engine.gmicloud.ai/**.
+
+There are two ways to configure the SDK:
+
+### Option 1: Using Environment Variables
+
+Set the following environment variables:
+
+```shell
+export GMI_CLOUD_CLIENT_ID=<YOUR_CLIENT_ID>
+export GMI_CLOUD_EMAIL=<YOUR_EMAIL>
+export GMI_CLOUD_PASSWORD=<YOUR_PASSWORD>
+```
+
+### Option 2: Passing Credentials as Parameters
+
+Pass `client_id`, `email`, and `password` directly to the Client object when initializing it in your script:
+
+```python
+from gmicloud import Client
+
+client = Client(client_id="<YOUR_CLIENT_ID>", email="<YOUR_EMAIL>", password="<YOUR_PASSWORD>")
+```
+
+## Quick Start
+
+### 1. How to run the code in the example folder
+```bash
+cd path/to/gmicloud-sdk
+# Create a virtual environment
+python -m venv venv
+source venv/bin/activate
+
+pip install -r requirements.txt
+python -m examples.create_task_from_artifact_template.py
+```
+
+### 2. Create an inference task from an artifact template
+
+This is the simplest example to deploy an inference task using an existing artifact template:
+
+Up-to-date code in /examples/create_task_from_artifact_template.py
+
+```python
+from datetime import datetime
+import os
+import sys
+
+from gmicloud import *
+from examples.completion import call_chat_completion
+
+cli = Client()
+
+# List templates offered by GMI cloud 
+templates = cli.list_templates()
+print(f"Found {len(templates)} templates: {templates}")
+
+# Pick a template from the list
+pick_template = "Llama-3.1-8B"
+
+# Create Artifact from template
+artifact_id, recommended_replica_resources = cli.create_artifact_from_template(templates[0])
+print(f"Created artifact {artifact_id} with recommended replica resources: {recommended_replica_resources}")
+
+# Create Task based on Artifact
+task_id = cli.create_task(artifact_id, recommended_replica_resources, TaskScheduling(
+    scheduling_oneoff=OneOffScheduling(
+        trigger_timestamp=int(datetime.now().timestamp()),
+        min_replicas=1,
+        max_replicas=1,
+    )
+))
+task = cli.task_manager.get_task(task_id)
+print(f"Task created: {task.config.task_name}. You can check details at https://inference-engine.gmicloud.ai/user-console/task")
+
+# Start Task and wait for it to be ready
+cli.start_task_and_wait(task.task_id)
+
+# Testing with calling chat completion
+print(call_chat_completion(cli, task.task_id))
+
+```
+
+## API Reference
+
+### Client
+
+Represents the entry point to interact with GMI Cloud APIs.
+Client(
+client_id: Optional[str] = "",
+email: Optional[str] = "",
+password: Optional[str] = ""
+)
+
+### Artifact Management
+
+* get_artifact_templates(): Fetch a list of available artifact templates.
+* create_artifact_from_template(template_id: str): Create a model artifact from a given template.
+* get_artifact(artifact_id: str): Get details of a specific artifact.
+
+### Task Management
+
+* create_task_from_artifact_template(template_id: str, scheduling: TaskScheduling): Create and schedule a task using an
+  artifact template.
+* start_task(task_id: str): Start a task.
+* get_task(task_id: str): Retrieve the status and details of a specific task.
+
+## Notes & Troubleshooting
+k

gmicloud-0.1.6/README.md

@@ -0,0 +1,135 @@
+# GMICloud SDK (Beta)
+
+## Overview
+Before you start: Our service and GPU resource is currenly invite-only so please contact our team (getstarted@gmicloud.ai) to get invited if you don't have one yet.
+
+The GMI Inference Engine SDK provides a Python interface for deploying and managing machine learning models in production environments. It allows users to create model artifacts, schedule tasks for serving models, and call inference APIs easily.
+
+This SDK streamlines the process of utilizing GMI Cloud capabilities such as deploying models with Kubernetes-based Ray services, managing resources automatically, and accessing model inference endpoints. With minimal setup, developers can focus on building ML solutions instead of infrastructure.
+
+## Features
+
+- Artifact Management: Easily create, update, and manage ML model artifacts.
+- Task Management: Quickly create, schedule, and manage deployment tasks for model inference.
+- Usage Data Retrieval : Fetch and analyze usage data to optimize resource allocation.
+
+## Installation
+
+To install the SDK, use pip:
+
+```bash
+pip install gmicloud
+```
+
+## Setup
+
+You must configure authentication credentials for accessing the GMI Cloud API. 
+To create account and get log in info please visit **GMI inference platform: https://inference-engine.gmicloud.ai/**.
+
+There are two ways to configure the SDK:
+
+### Option 1: Using Environment Variables
+
+Set the following environment variables:
+
+```shell
+export GMI_CLOUD_CLIENT_ID=<YOUR_CLIENT_ID>
+export GMI_CLOUD_EMAIL=<YOUR_EMAIL>
+export GMI_CLOUD_PASSWORD=<YOUR_PASSWORD>
+```
+
+### Option 2: Passing Credentials as Parameters
+
+Pass `client_id`, `email`, and `password` directly to the Client object when initializing it in your script:
+
+```python
+from gmicloud import Client
+
+client = Client(client_id="<YOUR_CLIENT_ID>", email="<YOUR_EMAIL>", password="<YOUR_PASSWORD>")
+```
+
+## Quick Start
+
+### 1. How to run the code in the example folder
+```bash
+cd path/to/gmicloud-sdk
+# Create a virtual environment
+python -m venv venv
+source venv/bin/activate
+
+pip install -r requirements.txt
+python -m examples.create_task_from_artifact_template.py
+```
+
+### 2. Create an inference task from an artifact template
+
+This is the simplest example to deploy an inference task using an existing artifact template:
+
+Up-to-date code in /examples/create_task_from_artifact_template.py
+
+```python
+from datetime import datetime
+import os
+import sys
+
+from gmicloud import *
+from examples.completion import call_chat_completion
+
+cli = Client()
+
+# List templates offered by GMI cloud 
+templates = cli.list_templates()
+print(f"Found {len(templates)} templates: {templates}")
+
+# Pick a template from the list
+pick_template = "Llama-3.1-8B"
+
+# Create Artifact from template
+artifact_id, recommended_replica_resources = cli.create_artifact_from_template(templates[0])
+print(f"Created artifact {artifact_id} with recommended replica resources: {recommended_replica_resources}")
+
+# Create Task based on Artifact
+task_id = cli.create_task(artifact_id, recommended_replica_resources, TaskScheduling(
+    scheduling_oneoff=OneOffScheduling(
+        trigger_timestamp=int(datetime.now().timestamp()),
+        min_replicas=1,
+        max_replicas=1,
+    )
+))
+task = cli.task_manager.get_task(task_id)
+print(f"Task created: {task.config.task_name}. You can check details at https://inference-engine.gmicloud.ai/user-console/task")
+
+# Start Task and wait for it to be ready
+cli.start_task_and_wait(task.task_id)
+
+# Testing with calling chat completion
+print(call_chat_completion(cli, task.task_id))
+
+```
+
+## API Reference
+
+### Client
+
+Represents the entry point to interact with GMI Cloud APIs.
+Client(
+client_id: Optional[str] = "",
+email: Optional[str] = "",
+password: Optional[str] = ""
+)
+
+### Artifact Management
+
+* get_artifact_templates(): Fetch a list of available artifact templates.
+* create_artifact_from_template(template_id: str): Create a model artifact from a given template.
+* get_artifact(artifact_id: str): Get details of a specific artifact.
+
+### Task Management
+
+* create_task_from_artifact_template(template_id: str, scheduling: TaskScheduling): Create and schedule a task using an
+  artifact template.
+* start_task(task_id: str): Start a task.
+* get_task(task_id: str): Retrieve the status and details of a specific task.
+
+## Notes & Troubleshooting
+k

{gmicloud-0.1.5 → gmicloud-0.1.6}/gmicloud/_internal/_enums.py

@@ -31,3 +31,8 @@ class TaskStatus(str, Enum):
     RUNNING = "running"
     NEEDSTOP = "needstop"
     ARCHIVED = "archived"
+
+class ModelParameterType(str, Enum):
+    NUMERIC = "numeric"
+    TEXT = "text"
+    BOOLEAN = "boolean"

{gmicloud-0.1.5 → gmicloud-0.1.6}/gmicloud/_internal/_manager/_artifact_manager.py

@@ -1,4 +1,5 @@
 import os
+import time
 from typing import List
 import mimetypes
 
@@ -7,6 +8,9 @@ from .._client._artifact_client import ArtifactClient
 from .._client._file_upload_client import FileUploadClient
 from .._models import *
 
+import logging
+
+logger = logging.getLogger(__name__)
 
 class ArtifactManager:
     """
@@ -86,6 +90,41 @@ class ArtifactManager:
             raise ValueError("Failed to create artifact from template.")
 
         return resp.artifact_id
+    
+    def create_artifact_from_template_name(self, artifact_template_name: str) -> tuple[str, ReplicaResource]:
+        """
+        Create an artifact from a template.
+        :param artifact_template_name: The name of the template to use.
+        :return: A tuple containing the artifact ID and the recommended replica resources.
+        :rtype: tuple[str, ReplicaResource]
+        """
+
+        recommended_replica_resources = None
+        template_id = None
+        try:
+            templates = self.get_public_templates()
+        except Exception as e:
+            logger.error(f"Failed to get artifact templates, Error: {e}")
+        for template in templates:
+            if template.template_data and template.template_data.name == artifact_template_name:
+                resources_template = template.template_data.resources
+                recommended_replica_resources = ReplicaResource(
+                    cpu=resources_template.cpu,
+                    ram_gb=resources_template.memory,
+                    gpu=resources_template.gpu,
+                    gpu_name=resources_template.gpu_name,
+                )
+                template_id = template.template_id
+                break
+        if not template_id:
+            raise ValueError(f"Template with name {artifact_template_name} not found.")
+        try: 
+            artifact_id = self.create_artifact_from_template(template_id)
+            self.wait_for_artifact_ready(artifact_id)
+            return artifact_id, recommended_replica_resources
+        except Exception as e:
+            logger.error(f"Failed to create artifact from template, Error: {e}")
+            raise e
 
     def rebuild_artifact(self, artifact_id: str) -> RebuildArtifactResponse:
         """
@@ -240,7 +279,31 @@ class ArtifactManager:
             FileUploadClient.upload_large_file(bigfile_upload_url_resp.upload_link, model_file_path)
 
         return artifact_id
-
+    
+
+    def wait_for_artifact_ready(self, artifact_id: str, timeout_s: int = 900) -> None:
+        """
+        Wait for an artifact to be ready.
+
+        :param artifact_id: The ID of the artifact to wait for.
+        :param timeout_s: The timeout in seconds.
+        :return: None
+        """
+        start_time = time.time()
+        while True:
+            try:
+                artifact = self.get_artifact(artifact_id)
+                if artifact.build_status == BuildStatus.SUCCESS:
+                    return
+                elif artifact.build_status in [BuildStatus.FAILED, BuildStatus.TIMEOUT, BuildStatus.CANCELLED]:
+                    raise Exception(f"Artifact build failed, status: {artifact.build_status}")
+            except Exception as e:
+                logger.error(f"Failed to get artifact, Error: {e}")
+            if time.time() - start_time > timeout_s:
+                raise Exception(f"Artifact build takes more than {timeout_s // 60} minutes. Testing aborted.")
+            time.sleep(10)
+
+    
     def get_public_templates(self) -> List[ArtifactTemplate]:
         """
         Fetch all artifact templates.
@@ -249,6 +312,26 @@ class ArtifactManager:
         :rtype: List[ArtifactTemplate]
         """
         return self.artifact_client.get_public_templates()
+        
+
+    def list_public_template_names(self) -> list[str]:
+        """
+        List all public templates.
+
+        :return: A list of template names.
+        :rtype: list[str]
+        """
+        template_names = []
+        try: 
+            templates = self.get_public_templates()
+            for template in templates:
+                if template.template_data and template.template_data.name:
+                    template_names.append(template.template_data.name)
+            return template_names
+        except Exception as e:
+            logger.error(f"Failed to get artifact templates, Error: {e}")
+            return []
+
 
     @staticmethod
     def _validate_file_name(file_name: str) -> None:

{gmicloud-0.1.5 → gmicloud-0.1.6}/gmicloud/_internal/_manager/_task_manager.py

@@ -4,6 +4,10 @@ from .._client._iam_client import IAMClient
 from .._client._task_client import TaskClient
 from .._models import *
 
+import time
+import logging
+
+logger = logging.getLogger(__name__)
 
 class TaskManager:
     """
@@ -132,6 +136,50 @@ class TaskManager:
         self._validate_not_empty(task_id, "Task ID")
 
         return self.task_client.start_task(task_id)
+    
+
+    def start_task_and_wait(self, task_id: str, timeout_s: int = 900) -> Task:
+        """
+        Start a task and wait for it to be ready.
+
+        :param task_id: The ID of the task to start.
+        :param timeout_s: The timeout in seconds.
+        :return: The task object.
+        :rtype: Task
+        """
+        # trigger start task
+        try:
+            self.start_task(task_id)
+            logger.info(f"Started task ID: {task_id}")
+        except Exception as e:
+            logger.error(f"Failed to start task, Error: {e}")
+            raise e
+        
+        start_time = time.time()
+        while True:
+            try:
+                task = self.get_task(task_id)
+                if task.task_status == TaskStatus.RUNNING:
+                    return task
+                elif task.task_status in [TaskStatus.NEEDSTOP, TaskStatus.ARCHIVED]:
+                    raise Exception(f"Unexpected task status after starting: {task.task_status}")
+                # Also check endpoint status. 
+                elif task.task_status == TaskStatus.RUNNING:
+                    if task.endpoint_info and task.endpoint_info.endpoint_status == TaskEndpointStatus.RUNNING:
+                        return task
+                    elif task.endpoint_info and task.endpoint_info.endpoint_status in [TaskEndpointStatus.UNKNOWN, TaskEndpointStatus.ARCHIVED]:
+                        raise Exception(f"Unexpected endpoint status after starting: {task.endpoint_info.endpoint_status}")
+                    else:
+                        logger.info(f"Pending endpoint starting. endpoint status: {task.endpoint_info.endpoint_status}")
+                else:
+                    logger.info(f"Pending task starting. Task status: {task.task_status}")
+
+            except Exception as e:
+                logger.error(f"Failed to get task, Error: {e}")
+            if time.time() - start_time > timeout_s:
+                raise Exception(f"Task creation takes more than {timeout_s // 60} minutes. Testing aborted.")
+            time.sleep(10)
+
 
     def stop_task(self, task_id: str) -> bool:
         """
@@ -143,6 +191,27 @@ class TaskManager:
         """
         self._validate_not_empty(task_id, "Task ID")
 
+        
+    def stop_task_and_wait(self, task_id: str, timeout_s: int = 900):
+        task_manager = self.task_manager
+        try:
+            self.task_manager.stop_task(task_id)
+            logger.info(f"Stopping task ID: {task_id}")
+        except Exception as e:
+            logger.error(f"Failed to stop task, Error: {e}")
+        task_manager = self.task_manager
+        start_time = time.time()
+        while True:
+            try:
+                task = self.get_task(task_id)
+                if task.task_status == TaskStatus.IDLE:
+                    break
+            except Exception as e:
+                logger.error(f"Failed to get task, Error: {e}")
+            if time.time() - start_time > timeout_s:
+                raise Exception(f"Task stopping takes more than {timeout_s // 60} minutes. Testing aborted.")
+            time.sleep(10)
+
         return self.task_client.stop_task(task_id)
 
     def get_usage_data(self, start_timestamp: str, end_timestamp: str) -> GetUsageDataResponse:

{gmicloud-0.1.5 → gmicloud-0.1.6}/gmicloud/_internal/_models.py

@@ -1,8 +1,8 @@
-from typing import Optional, List
+from typing import Optional, List, Union
 from datetime import datetime
 
 from pydantic import BaseModel
-from gmicloud._internal._enums import BuildStatus, TaskStatus, TaskEndpointStatus
+from gmicloud._internal._enums import BuildStatus, TaskStatus, TaskEndpointStatus, ModelParameterType
 
 
 class BigFileMetadata(BaseModel):
@@ -70,6 +70,7 @@ class CreateArtifactRequest(BaseModel):
     artifact_name: str  # The name of the artifact to create.
     artifact_description: Optional[str] = ""  # Description of the artifact.
     artifact_tags: Optional[List[str]] = None  # Tags for the artifact, separated by commas.
+    model_parameters: Optional[List["ModelParameter"]] = None  # Parameters for the artifact.
 
 
 class CreateArtifactResponse(BaseModel):
@@ -158,6 +159,7 @@ class TemplateMetadata(BaseModel):
     update_at: Optional[str] = None  # Timestamp when the template was last updated.
     update_by: Optional[str] = ""  # ID of the user who last updated the template.
 
+
 class TemplateData(BaseModel):
     """
     Data for an artifact template.
@@ -165,6 +167,7 @@ class TemplateData(BaseModel):
     description: Optional[str] = ""  # Description of the artifact template.
     icon_link: Optional[str] = ""  # Link to the icon for the artifact template.
     image_link: Optional[str] = ""  # Link to the image for the artifact template.
+    model_parameters: Optional[List["ModelParameter"]] = None  # Parameters for the artifact template.
     name: Optional[str] = ""  # Name of the artifact template.
     ray: Optional["RayContent"] = None  # Template for Ray-based artifacts.
     resources: Optional["ResourcesTemplate"] = None  # Resource allocation template.
@@ -172,6 +175,19 @@ class TemplateData(BaseModel):
     volume_path: Optional[str] = ""  # Path to the volume where the artifact is stored.
 
 
+class ModelParameter(BaseModel):
+    """
+    Parameter for an artifact template.
+    """
+    category: Optional[str] = ""  # Category of the parameter.
+    display_name: Optional[str] = ""  # Display name of the parameter.
+    key: Optional[str] = ""  # Key for the parameter.
+    max: Optional[float] = 0  # Maximum value for the parameter.
+    min: Optional[float] = 0  # Minimum value for the parameter.
+    step: Optional[float] = 0  # Step value for the parameter.
+    type: Optional[ModelParameterType] = ModelParameterType.TEXT  # Type of the parameter (e.g., numeric, bool, text).
+    value: Optional[Union[int, float, bool, str]] = ""  # Default value for the parameter.
+
 class RayContent(BaseModel):
     deployment_name: Optional[str] = ""  # Name of the deployment.
     file_path: Optional[str] = ""  # Path to the task file in storage.
@@ -234,7 +250,6 @@ class RayTaskConfig(BaseModel):
     Configuration settings for Ray tasks.
     """
     artifact_id: Optional[str] = ""  # Associated artifact ID.
-    ray_version: Optional[str] = ""  # Version of Ray used.
     ray_cluster_image: Optional[str] = ""  # Docker image for the Ray cluster.
     file_path: Optional[str] = ""  # Path to the task file in storage.
     deployment_name: Optional[str] = ""  # Name of the deployment.
@@ -282,6 +297,7 @@ class TaskConfig(BaseModel):
     """
     Configuration data for a task.
     """
+    task_name: Optional[str] = ""  # Name of the task.
     ray_task_config: Optional[RayTaskConfig] = None  # Configuration for a Ray-based task.
     task_scheduling: Optional[TaskScheduling] = None  # Scheduling configuration for the task.
     create_timestamp: Optional[int] = 0  # Timestamp when the task was created.