cloudos-cli 2.33.0__tar.gz → 2.35.0__tar.gz

{cloudos_cli-2.33.0 → cloudos_cli-2.35.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.33.0
+Version: 2.35.0
 Summary: Python package for interacting with CloudOS
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -1007,6 +1007,55 @@ Please, be aware that renaming can only happen in files and folders that are pre
 
 Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
 
+```bash
+    --cloudos-url $CLOUDOS \
+    --apikey $MY_API_KEY \
+    --workspace-id $WORKSPACE_ID \
+    --project-name $PROJEC_NAME
+```
+#### Create a (virtual) folder
+
+New folders can be created within the `Data` dataset and its subfolders using the following command
+
+```
+cloudos datasets mkdir <new_folder_path>
+```
+where ` new_folder_path` is the path to the folder to be created.
+
+Please, be aware that the folder creation can only happen in `Data` or its sub-folders.
+
+Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
+
+```bash
+    --cloudos-url $CLOUDOS \
+    --apikey $MY_API_KEY \
+    --workspace-id $WORKSPACE_ID \
+    --project-name $PROJEC_NAME
+```
+
+#### Copying files and folders
+
+Files and folders can be copied **from** anywhere in the project **to** `Data` or any of its subfolders programmatically (i.e `Data`, `Data/folder/file.txt`).
+
+1. The copy can happen **within the same project** running the following command:
+```
+cloudos datasets cp <souce_path> <destination_path> --profile <profile name>
+```
+where the source project as well as the destination one is the one defined in the profile.
+
+2. The move can also happen **across different projects**  within the same workspace by running the following command
+```
+cloudos datasets cp <source_path> <destiantion_path> --profile <profile_name> --destination-project-name <project_name>
+```
+In this case, only the source project is the one specified in the profile.
+
+Any of the `source_path` must be a full path; any `destination_path` must be a path starting with `Data` and finishing with the folder where to move the file/folder. An example of such command is:
+
+```
+cloudos datasets cp AnalysesResults/my_analysis/results/my_plot.png Data/plots 
+```
+
+Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
 ```bash
     --cloudos-url $CLOUDOS \
     --apikey $MY_API_KEY \

{cloudos_cli-2.33.0 → cloudos_cli-2.35.0}/README.md

@@ -972,6 +972,55 @@ Please, be aware that renaming can only happen in files and folders that are pre
 
 Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
 
+```bash
+    --cloudos-url $CLOUDOS \
+    --apikey $MY_API_KEY \
+    --workspace-id $WORKSPACE_ID \
+    --project-name $PROJEC_NAME
+```
+#### Create a (virtual) folder
+
+New folders can be created within the `Data` dataset and its subfolders using the following command
+
+```
+cloudos datasets mkdir <new_folder_path>
+```
+where ` new_folder_path` is the path to the folder to be created.
+
+Please, be aware that the folder creation can only happen in `Data` or its sub-folders.
+
+Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
+
+```bash
+    --cloudos-url $CLOUDOS \
+    --apikey $MY_API_KEY \
+    --workspace-id $WORKSPACE_ID \
+    --project-name $PROJEC_NAME
+```
+
+#### Copying files and folders
+
+Files and folders can be copied **from** anywhere in the project **to** `Data` or any of its subfolders programmatically (i.e `Data`, `Data/folder/file.txt`).
+
+1. The copy can happen **within the same project** running the following command:
+```
+cloudos datasets cp <souce_path> <destination_path> --profile <profile name>
+```
+where the source project as well as the destination one is the one defined in the profile.
+
+2. The move can also happen **across different projects**  within the same workspace by running the following command
+```
+cloudos datasets cp <source_path> <destiantion_path> --profile <profile_name> --destination-project-name <project_name>
+```
+In this case, only the source project is the one specified in the profile.
+
+Any of the `source_path` must be a full path; any `destination_path` must be a path starting with `Data` and finishing with the folder where to move the file/folder. An example of such command is:
+
+```
+cloudos datasets cp AnalysesResults/my_analysis/results/my_plot.png Data/plots 
+```
+
+Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
 ```bash
     --cloudos-url $CLOUDOS \
     --apikey $MY_API_KEY \

{cloudos_cli-2.33.0 → cloudos_cli-2.35.0}/cloudos_cli/__main__.py

@@ -88,7 +88,9 @@ def run_cloudos_cli(ctx):
             'datasets': {
                 'ls': shared_config,
                 'mv': shared_config,
-                'rename': shared_config
+                'rename': shared_config,
+                'cp': shared_config,
+                'mkdir': shared_config
             }
         })
     else:
@@ -135,7 +137,9 @@ def run_cloudos_cli(ctx):
             'datasets': {
                 'ls': shared_config,
                 'mv': shared_config,
-                'rename': shared_config
+                'rename': shared_config,
+                'cp': shared_config,
+                'mkdir': shared_config
             }
         })
 
@@ -2960,5 +2964,247 @@ def renaming_item(ctx, source_path, new_name, apikey, cloudos_url,
         sys.exit(1)
 
 
+@datasets.command(name="cp")
+@click.argument("source_path", required=True)
+@click.argument("destination_path", required=True)
+@click.option('-k', '--apikey', required=True, help='Your CloudOS API key.')
+@click.option('-c', '--cloudos-url', default=CLOUDOS_URL, required=True, help='The CloudOS URL.')
+@click.option('--workspace-id', required=True, help='The CloudOS workspace ID.')
+@click.option('--project-name', required=True, help='The source project name.')
+@click.option('--destination-project-name', required=False, help='The destination project name. Defaults to the source project.')
+@click.option('--disable-ssl-verification', is_flag=True, help='Disable SSL certificate verification.')
+@click.option('--ssl-cert', help='Path to your SSL certificate file.')
+@click.option('--profile', default=None, help='Profile to use from the config file.')
+@click.pass_context
+def copy_item_cli(ctx, source_path, destination_path, apikey, cloudos_url,
+                  workspace_id, project_name, destination_project_name,
+                  disable_ssl_verification, ssl_cert, profile):
+    """
+    Copy a file or folder (S3 or virtual) from SOURCE_PATH to DESTINATION_PATH.
+
+    SOURCE_PATH [path]: the full path to the file or folder to copy.
+     E.g.: AnalysesResults/my_analysis/results/my_plot.png\n
+    DESTINATION_PATH [path]: the full path to the destination folder. It must be a 'Data' folder path.
+     E.g.: Data/plots
+    """
+    click.echo("Loading configuration profile...")
+    config_manager = ConfigurationProfile()
+    required_dict = {
+        'apikey': True,
+        'workspace_id': True,
+        'workflow_name': False,
+        'project_name': True
+    }
+    apikey, cloudos_url, workspace_id, workflow_name, _, _, project_name = config_manager.load_profile_and_validate_data(
+        ctx, INIT_PROFILE, CLOUDOS_URL, profile=profile,
+        required_dict=required_dict,
+        apikey=apikey,
+        cloudos_url=cloudos_url,
+        workspace_id=workspace_id,
+        workflow_name=None,
+        repository_platform=None,
+        execution_platform=None,
+        project_name=project_name
+    )
+    destination_project_name = destination_project_name or project_name
+    verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
+    # Initialize clients
+    source_client = Datasets(
+        cloudos_url=cloudos_url,
+        apikey=apikey,
+        workspace_id=workspace_id,
+        project_name=project_name,
+        verify=verify_ssl,
+        cromwell_token=None
+    )
+    dest_client = Datasets(
+        cloudos_url=cloudos_url,
+        apikey=apikey,
+        workspace_id=workspace_id,
+        project_name=destination_project_name,
+        verify=verify_ssl,
+        cromwell_token=None
+    )
+    # Validate paths
+    dest_parts = destination_path.strip("/").split("/")
+    if not dest_parts or dest_parts[0] != "Data":
+        click.echo("[ERROR] DESTINATION_PATH must start with 'Data/'.", err=True)
+        sys.exit(1)
+    # Parse source and destination
+    source_parts = source_path.strip("/").split("/")
+    source_parent = "/".join(source_parts[:-1]) if len(source_parts) > 1 else ""
+    source_name = source_parts[-1]
+    dest_folder_name = dest_parts[-1]
+    dest_parent = "/".join(dest_parts[:-1]) if len(dest_parts) > 1 else ""
+    try:
+        source_content = source_client.list_folder_content(source_parent)
+        dest_content = dest_client.list_folder_content(dest_parent)
+    except Exception as e:
+        click.echo(f"[ERROR] Could not access paths: {str(e)}", err=True)
+        sys.exit(1)
+    # Find the source item
+    source_item = None
+    for item in source_content.get('files' or 'folders', {}):
+        if item.get("name") == source_name:
+            source_item = item
+            break
+    if not source_item:
+        click.echo(f"[ERROR] Item '{source_name}' not found in '{source_parent or '[project root]'}'", err=True)
+        sys.exit(1)
+    # Find the destination folder
+    destination_folder = None
+    for folder in dest_content.get("folders", []):
+        if folder.get("name") == dest_folder_name:
+            destination_folder = folder
+            break
+    if not destination_folder:
+        click.echo(f"[ERROR] Destination folder '{destination_path}' not found.", err=True)
+        sys.exit(1)
+    try:
+        # Determine item type
+        if "fileType" in source_item:
+            item_type = "file"
+        elif source_item.get("folderType") == "VirtualFolder":
+            item_type = "virtual_folder"
+        elif "s3BucketName" in source_item and source_item.get("folderType") == "S3Folder":
+            item_type = "s3_folder"
+        else:
+            click.echo("[ERROR] Could not determine item type.", err=True)
+            sys.exit(1)
+        click.echo(f"Copying {item_type.replace('_', ' ')} '{source_name}' to '{destination_path}'...")
+        if destination_folder.get("folderType") is True and destination_folder.get("kind") in ("Data", "Cohorts", "AnalysesResults"):
+            destination_kind = "Dataset"
+        else:
+            destination_kind = "Folder"
+        response = source_client.copy_item(
+            item=source_item,
+            destination_id=destination_folder["_id"],
+            destination_kind=destination_kind
+        )
+        if response.ok:
+            click.secho("[SUCCESS] Item copied successfully.", fg="green", bold=True)
+        else:
+            click.echo(f"[ERROR] Copy failed: {response.status_code} - {response.text}", err=True)
+            sys.exit(1)
+    except Exception as e:
+        click.echo(f"[ERROR] Copy operation failed: {str(e)}", err=True)
+        sys.exit(1)
+
+
+@datasets.command(name="mkdir")
+@click.argument("new_folder_path", required=True)
+@click.option('-k', '--apikey', required=True, help='Your CloudOS API key.')
+@click.option('-c', '--cloudos-url', default=CLOUDOS_URL, required=True, help='The CloudOS URL.')
+@click.option('--workspace-id', required=True, help='The CloudOS workspace ID.')
+@click.option('--project-name', required=True, help='The project name.')
+@click.option('--disable-ssl-verification', is_flag=True, help='Disable SSL certificate verification.')
+@click.option('--ssl-cert', help='Path to your SSL certificate file.')
+@click.option('--profile', default=None, help='Profile to use from the config file.')
+@click.pass_context
+def mkdir_item(ctx, new_folder_path, apikey, cloudos_url,
+               workspace_id, project_name,
+               disable_ssl_verification, ssl_cert, profile):
+    """
+    Create a virtual folder in a CloudOS project.
+
+    NEW_FOLDER_PATH [path]: Full path to the new folder including its name. Must start with 'Data'.
+    """
+    new_folder_path = new_folder_path.strip("/")
+    if not new_folder_path.startswith("Data"):
+        click.echo("[ERROR] NEW_FOLDER_PATH must start with 'Data'.", err=True)
+        sys.exit(1)
+
+    path_parts = new_folder_path.split("/")
+    if len(path_parts) < 2:
+        click.echo("[ERROR] NEW_FOLDER_PATH must include at least a parent folder and the new folder name.", err=True)
+        sys.exit(1)
+
+    parent_path = "/".join(path_parts[:-1])
+    folder_name = path_parts[-1]
+
+    click.echo("Loading configuration profile...")
+    config_manager = ConfigurationProfile()
+    required_dict = {
+        'apikey': True,
+        'workspace_id': True,
+        'workflow_name': False,
+        'project_name': True
+    }
+
+    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+        config_manager.load_profile_and_validate_data(
+            ctx,
+            INIT_PROFILE,
+            CLOUDOS_URL,
+            profile=profile,
+            required_dict=required_dict,
+            apikey=apikey,
+            cloudos_url=cloudos_url,
+            workspace_id=workspace_id,
+            workflow_name=None,
+            repository_platform=None,
+            execution_platform=None,
+            project_name=project_name
+        )
+    )
+
+    verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
+
+    client = Datasets(
+        cloudos_url=cloudos_url,
+        apikey=apikey,
+        workspace_id=workspace_id,
+        project_name=project_name,
+        verify=verify_ssl,
+        cromwell_token=None
+    )
+
+    # Split parent path to get its parent + name
+    parent_parts = parent_path.split("/")
+    parent_name = parent_parts[-1]
+    parent_of_parent_path = "/".join(parent_parts[:-1])
+
+    # List the parent of the parent
+    try:
+        contents = client.list_folder_content(parent_of_parent_path)
+    except Exception as e:
+        click.echo(f"[ERROR] Could not list contents at '{parent_of_parent_path}': {str(e)}", err=True)
+        sys.exit(1)
+
+    # Find the parent folder in the contents
+    folder_info = next(
+        (f for f in contents.get("folders", []) if f.get("name") == parent_name),
+        None
+    )
+
+    if not folder_info:
+        click.echo(f"[ERROR] Could not find folder '{parent_name}' in '{parent_of_parent_path}'.", err=True)
+        sys.exit(1)
+
+    parent_id = folder_info.get("_id")
+    folder_type = folder_info.get("folderType")
+
+    if folder_type is True:
+        parent_kind = "Dataset"
+    elif isinstance(folder_type, str):
+        parent_kind = "Folder"
+    else:
+        click.echo(f"[ERROR] Unrecognized folderType for '{parent_path}'.", err=True)
+        sys.exit(1)
+
+    # Create the folder
+    click.echo(f"Creating folder '{folder_name}' under '{parent_path}' ({parent_kind})...")
+    try:
+        response = client.create_virtual_folder(name=folder_name, parent_id=parent_id, parent_kind=parent_kind)
+        if response.ok:
+            click.secho(f"[SUCCESS] Folder '{folder_name}' created under '{parent_path}'", fg="green", bold=True)
+        else:
+            click.echo(f"[ERROR] Folder creation failed: {response.status_code} - {response.text}", err=True)
+            sys.exit(1)
+    except Exception as e:
+        click.echo(f"[ERROR] Folder creation failed: {str(e)}", err=True)
+        sys.exit(1)
+
+
 if __name__ == "__main__":
     run_cloudos_cli()

cloudos_cli-2.35.0/cloudos_cli/_version.py

@@ -0,0 +1 @@
+__version__ = '2.35.0'

{cloudos_cli-2.33.0 → cloudos_cli-2.35.0}/cloudos_cli/datasets/datasets.py

@@ -5,7 +5,7 @@ This is the main class for file explorer (datasets).
 from dataclasses import dataclass
 from typing import Union
 from cloudos_cli.clos import Cloudos
-from cloudos_cli.utils.requests import retry_requests_get, retry_requests_put
+from cloudos_cli.utils.requests import retry_requests_get, retry_requests_put, retry_requests_post
 import json
 
 @dataclass
@@ -403,4 +403,88 @@ class Datasets(Cloudos):
         }
 
         response = retry_requests_put(url, headers=headers, data=json.dumps(payload), verify=self.verify)
+        return response
+    
+    def copy_item(self, item, destination_id, destination_kind):
+        """Copy a file or folder (S3 or Virtual) to a destination in CloudOS."""
+        headers = {
+            "accept": "application/json",
+            "content-type": "application/json",
+            "ApiKey": self.apikey
+        }
+        parent = {"kind": destination_kind, "id": destination_id}
+
+        # Virtual folder
+        if item.get("folderType") == "VirtualFolder":
+            payload = {
+                "copyContentsFrom": item["_id"],
+                "name": item["name"],
+                "parent": parent
+            }
+            url = f"{self.cloudos_url}/api/v1/folders/virtual?teamId={self.workspace_id}"
+        # S3 folder
+        elif item.get("folderType") == "S3Folder":
+            payload = {
+                "s3BucketName": item["s3BucketName"],
+                "s3ObjectKey": item.get("s3ObjectKey") or item.get("s3Prefix"),
+                "name": item["name"],
+                "parent": parent,
+                "isManagedByLifebit": item.get("isManagedByLifebit", False)
+            }
+            url = f"{self.cloudos_url}/api/v1/folders/s3?teamId={self.workspace_id}"
+        # S3 file
+        elif item.get("fileType") == "S3File":
+            payload = {
+                "s3BucketName": item["s3BucketName"],
+                "s3ObjectKey": item["s3ObjectKey"],
+                "name": item["name"],
+                "parent": parent,
+                "isManagedByLifebit": item.get("isManagedByLifebit", False),
+                "sizeInBytes": item.get("sizeInBytes", 0)
+            }
+            url = f"{self.cloudos_url}/api/v1/files/s3?teamId={self.workspace_id}"
+        else:
+            raise ValueError(f"Unknown item type for copy: {item.get('name')}")
+        response = retry_requests_post(url, headers=headers, json=payload)
+
+        return response
+    
+    def create_virtual_folder(self, name: str, parent_id: str, parent_kind: str):
+        """
+        Create a new virtual folder in CloudOS under a given parent.
+
+        Parameters
+        ----------
+        name : str
+            The name of the new folder.
+        parent_id : str
+            The ID of the parent (can be a Dataset or a Folder).
+        parent_kind : str
+            The type of the parent: either "Dataset" or "Folder".
+
+        Returns
+        -------
+        response : requests.Response
+            The response object from the CloudOS API.
+        """
+        if parent_kind not in ("Dataset", "Folder"):
+            raise ValueError("Invalid parent_kind. Must be 'Dataset' or 'Folder'.")
+
+        url = f"{self.cloudos_url}/api/v1/folders/virtual?teamId={self.workspace_id}"
+
+        headers = {
+            "accept": "application/json",
+            "content-type": "application/json",
+            "ApiKey": self.apikey
+        }
+
+        payload = {
+            "name": name,
+            "parent": {
+                "kind": parent_kind,
+                "id": parent_id
+            }
+        }
+
+        response = retry_requests_post(url, headers=headers, json=payload, verify=self.verify)
         return response

{cloudos_cli-2.33.0 → cloudos_cli-2.35.0}/cloudos_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.33.0
+Version: 2.35.0
 Summary: Python package for interacting with CloudOS
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -1007,6 +1007,55 @@ Please, be aware that renaming can only happen in files and folders that are pre
 
 Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
 
+```bash
+    --cloudos-url $CLOUDOS \
+    --apikey $MY_API_KEY \
+    --workspace-id $WORKSPACE_ID \
+    --project-name $PROJEC_NAME
+```
+#### Create a (virtual) folder
+
+New folders can be created within the `Data` dataset and its subfolders using the following command
+
+```
+cloudos datasets mkdir <new_folder_path>
+```
+where ` new_folder_path` is the path to the folder to be created.
+
+Please, be aware that the folder creation can only happen in `Data` or its sub-folders.
+
+Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
+
+```bash
+    --cloudos-url $CLOUDOS \
+    --apikey $MY_API_KEY \
+    --workspace-id $WORKSPACE_ID \
+    --project-name $PROJEC_NAME
+```
+
+#### Copying files and folders
+
+Files and folders can be copied **from** anywhere in the project **to** `Data` or any of its subfolders programmatically (i.e `Data`, `Data/folder/file.txt`).
+
+1. The copy can happen **within the same project** running the following command:
+```
+cloudos datasets cp <souce_path> <destination_path> --profile <profile name>
+```
+where the source project as well as the destination one is the one defined in the profile.
+
+2. The move can also happen **across different projects**  within the same workspace by running the following command
+```
+cloudos datasets cp <source_path> <destiantion_path> --profile <profile_name> --destination-project-name <project_name>
+```
+In this case, only the source project is the one specified in the profile.
+
+Any of the `source_path` must be a full path; any `destination_path` must be a path starting with `Data` and finishing with the folder where to move the file/folder. An example of such command is:
+
+```
+cloudos datasets cp AnalysesResults/my_analysis/results/my_plot.png Data/plots 
+```
+
+Please, note that in the above example a preconfigured profile has been used. If no profile is provided and there is no default profile, the user will need to also provide the following flags
 ```bash
     --cloudos-url $CLOUDOS \
     --apikey $MY_API_KEY \

cloudos_cli-2.33.0/cloudos_cli/_version.py

@@ -1 +0,0 @@
-__version__ = '2.33.0'