cloudos-cli 2.37.0__tar.gz → 2.38.0__tar.gz

{cloudos_cli-2.37.0 → cloudos_cli-2.38.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.37.0
+Version: 2.38.0
 Summary: Python package for interacting with CloudOS
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -106,7 +106,7 @@ In order to generate a profile called `default`, the following command can be us
 cloudos configure
 ```
 
-This will bring in prompts for API, platform URL, project name, platform executor, repository provider, workflow name (if any). This will be the default profile if no other was explicitly set. The default profile allows running all subcommands without adding `--profile` option in the command line.
+This will bring in prompts for API, platform URL, project name, platform executor, repository provider, workflow name (if any) and session ID for interactive analysis. This will be the default profile if no other was explicitly set. The default profile allows running all subcommands without adding `--profile` option in the command line.
 
 ### Configure named profile
 
@@ -1089,6 +1089,32 @@ Please, note that in the above example a preconfigured profile has been used. If
     --project-name $PROJEC_NAME
 ```
 
+#### Linking S3 folders to Interactive Analysis
+
+The CLI can link S3 folders into an interactive session, which will appear in the left panel of the session as "mounted". This subcommand is using the option `--session-id` to access the correct interactive session. This option can be added to the CLI or defined in a profile, for convenience.
+
+The usage of the subcommand is as follows:
+
+```console
+cloudos datasets link <S3 FOLDER COMPLETE PATH> <options>
+```
+
+For example:
+
+```console
+cloudos datasets link s3://bucket/path/folder --profile test
+```
+where profile `test` has already defined the `--session-id`.
+
+Or specifying explicitly:
+
+```console
+cloudos datasets link s3://bucket/path/folder --profile test --session-id 1234
+```
+where profile `test` is used for all other necessary variables, but `--session-id` is overwritten from the command line.
+
+> [!NOTE]
+> If running the CLI inside a jupyter session, the pre-configured CLI installation will have the session ID already installed and only the `--apikey` needs to be added.
 
 #### Create a (virtual) folder
 

{cloudos_cli-2.37.0 → cloudos_cli-2.38.0}/README.md

@@ -71,7 +71,7 @@ In order to generate a profile called `default`, the following command can be us
 cloudos configure
 ```
 
-This will bring in prompts for API, platform URL, project name, platform executor, repository provider, workflow name (if any). This will be the default profile if no other was explicitly set. The default profile allows running all subcommands without adding `--profile` option in the command line.
+This will bring in prompts for API, platform URL, project name, platform executor, repository provider, workflow name (if any) and session ID for interactive analysis. This will be the default profile if no other was explicitly set. The default profile allows running all subcommands without adding `--profile` option in the command line.
 
 ### Configure named profile
 
@@ -1054,6 +1054,32 @@ Please, note that in the above example a preconfigured profile has been used. If
     --project-name $PROJEC_NAME
 ```
 
+#### Linking S3 folders to Interactive Analysis
+
+The CLI can link S3 folders into an interactive session, which will appear in the left panel of the session as "mounted". This subcommand is using the option `--session-id` to access the correct interactive session. This option can be added to the CLI or defined in a profile, for convenience.
+
+The usage of the subcommand is as follows:
+
+```console
+cloudos datasets link <S3 FOLDER COMPLETE PATH> <options>
+```
+
+For example:
+
+```console
+cloudos datasets link s3://bucket/path/folder --profile test
+```
+where profile `test` has already defined the `--session-id`.
+
+Or specifying explicitly:
+
+```console
+cloudos datasets link s3://bucket/path/folder --profile test --session-id 1234
+```
+where profile `test` is used for all other necessary variables, but `--session-id` is overwritten from the command line.
+
+> [!NOTE]
+> If running the CLI inside a jupyter session, the pre-configured CLI installation will have the session ID already installed and only the `--apikey` needs to be added.
 
 #### Create a (virtual) folder
 

{cloudos_cli-2.37.0 → cloudos_cli-2.38.0}/cloudos_cli/__main__.py

@@ -18,6 +18,7 @@ from cloudos_cli.utils.resources import ssl_selector, format_bytes
 from rich.style import Style
 from cloudos_cli.utils.array_job import generate_datasets_for_project
 from cloudos_cli.utils.details import get_path
+from cloudos_cli.link import Link
 
 
 # GLOBAL VARS
@@ -46,7 +47,10 @@ def run_cloudos_cli(ctx):
     config_manager = ConfigurationProfile()
     profile_to_use = config_manager.determine_default_profile()
     if profile_to_use is None:
-        print('[Warning] No profile found. Please create one with "cloudos configure".\n')
+        console = Console()
+        console.print(
+            "[bold yellow][Warning] No profile found. Please create one with \"cloudos configure\"."
+        )
         shared_config = dict({
             'apikey': '',
             'cloudos_url': CLOUDOS_URL,
@@ -55,7 +59,8 @@ def run_cloudos_cli(ctx):
             'workflow_name': '',
             'repository_platform': 'github',
             'execution_platform': 'aws',
-            'profile': INIT_PROFILE
+            'profile': INIT_PROFILE,
+            'session_id': '',
         })
         ctx.default_map = dict({
             'job': {
@@ -91,6 +96,7 @@ def run_cloudos_cli(ctx):
                 'mv': shared_config,
                 'rename': shared_config,
                 'cp': shared_config,
+                'link': shared_config,
                 'mkdir': shared_config,
                 'rm': shared_config
             }
@@ -105,7 +111,8 @@ def run_cloudos_cli(ctx):
             'workflow_name': profile_data.get('workflow_name', ""),
             'repository_platform': profile_data.get('repository_platform', ""),
             'execution_platform': profile_data.get('execution_platform', ""),
-            'profile': profile_to_use
+            'profile': profile_to_use,
+            'session_id': profile_data.get('session_id', "")
         })
         ctx.default_map = dict({
             'job': {
@@ -141,6 +148,7 @@ def run_cloudos_cli(ctx):
                 'mv': shared_config,
                 'rename': shared_config,
                 'cp': shared_config,
+                'link': shared_config,
                 'mkdir': shared_config,
                 'rm': shared_config
             }
@@ -211,7 +219,6 @@ def configure(ctx, profile, make_default):
     if make_default:
         config_manager.make_default_profile(profile_name=profile)
 
-
 @job.command('run')
 @click.option('-k',
               '--apikey',
@@ -220,7 +227,8 @@ def configure(ctx, profile, make_default):
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -407,7 +415,7 @@ def run(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -423,6 +431,13 @@ def run(ctx,
             project_name=project_name
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    workflow_name = user_options['workflow_name']
+    repository_platform = user_options['repository_platform']
+    execution_platform = user_options['execution_platform']
+    project_name = user_options['project_name']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     if do_not_save_logs:
@@ -655,7 +670,8 @@ def run(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--job-id',
               help='The job id in CloudOS to search for.',
               required=True)
@@ -689,7 +705,7 @@ def job_status(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -700,6 +716,8 @@ def job_status(ctx,
             cloudos_url=cloudos_url
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
 
     print('Executing status...')
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
@@ -726,7 +744,8 @@ def job_status(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -764,7 +783,7 @@ def job_logs(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -776,6 +795,9 @@ def job_logs(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     print('Executing logs...')
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
@@ -799,7 +821,8 @@ def job_logs(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -837,7 +860,7 @@ def job_results(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -849,6 +872,9 @@ def job_results(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     print('Executing results...')
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
@@ -872,7 +898,8 @@ def job_results(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--job-id',
               help='The job id in CloudOS to search for.',
               required=True)
@@ -923,7 +950,7 @@ def job_details(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -934,6 +961,9 @@ def job_details(ctx,
             cloudos_url=cloudos_url
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    execution_platform = user_options['execution_platform']
 
     print('Executing details...')
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
@@ -1096,7 +1126,8 @@ def job_details(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1162,7 +1193,7 @@ def list_jobs(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1174,6 +1205,9 @@ def list_jobs(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     outfile = output_basename + '.' + output_format
@@ -1226,7 +1260,8 @@ def list_jobs(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1266,7 +1301,7 @@ def abort_jobs(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1278,6 +1313,9 @@ def abort_jobs(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     print('Aborting jobs...')
@@ -1319,7 +1357,8 @@ def abort_jobs(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1370,7 +1409,7 @@ def list_workflows(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1382,6 +1421,9 @@ def list_workflows(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     outfile = output_basename + '.' + output_format
@@ -1417,7 +1459,8 @@ def list_workflows(ctx,
               '--cloudos-url',
               help=('The CloudOS url you are trying to access to. ' +
                     f'Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1463,7 +1506,7 @@ def import_wf(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1477,6 +1520,11 @@ def import_wf(ctx,
             repository_platform=repository_platform
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    workflow_name = user_options['workflow_name']
+    repository_platform = user_options['repository_platform']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     repo_import = ImportWorflow(
@@ -1497,7 +1545,8 @@ def import_wf(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1553,7 +1602,7 @@ def list_projects(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1565,6 +1614,9 @@ def list_projects(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     outfile = output_basename + '.' + output_format
@@ -1619,7 +1671,8 @@ def list_projects(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1654,7 +1707,7 @@ def cromwell_status(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1665,6 +1718,9 @@ def cromwell_status(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     if apikey is None and cromwell_token is None:
         raise ValueError("Please, use one of the following tokens: '--apikey', '--cromwell_token'")
@@ -1694,7 +1750,8 @@ def cromwell_status(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1734,7 +1791,7 @@ def cromwell_restart(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1745,6 +1802,9 @@ def cromwell_restart(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     if apikey is None and cromwell_token is None:
         raise ValueError("Please, use one of the following tokens: '--apikey', '--cromwell_token'")
@@ -1796,7 +1856,8 @@ def cromwell_restart(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1831,7 +1892,7 @@ def cromwell_stop(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1842,6 +1903,9 @@ def cromwell_stop(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     if apikey is None and cromwell_token is None:
         raise ValueError("Please, use one of the following tokens: '--apikey', '--cromwell_token'")
@@ -1869,7 +1933,8 @@ def cromwell_stop(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -1916,7 +1981,7 @@ def list_queues(ctx,
     }
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -1928,6 +1993,9 @@ def list_queues(ctx,
             workspace_id=workspace_id
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     outfile = output_basename + '.' + output_format
@@ -1977,7 +2045,8 @@ def remove_profile(ctx, profile):
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -2098,7 +2167,7 @@ def run_bash_job(ctx,
 
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -2114,6 +2183,13 @@ def run_bash_job(ctx,
             project_name=project_name
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    workflow_name = user_options['workflow_name']
+    repository_platform = user_options['repository_platform']
+    execution_platform = user_options['execution_platform']
+    project_name = user_options['project_name']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
 
@@ -2204,7 +2280,8 @@ def run_bash_job(ctx,
 @click.option('-c',
               '--cloudos-url',
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
+              default=CLOUDOS_URL,
+              required=True)
 @click.option('--workspace-id',
               help='The specific CloudOS workspace id.',
               required=True)
@@ -2373,7 +2450,7 @@ def run_bash_array_job(ctx,
 
     # determine if the user provided all required parameters
     config_manager = ConfigurationProfile()
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -2389,6 +2466,14 @@ def run_bash_array_job(ctx,
             project_name=project_name
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    workflow_name = user_options['workflow_name']
+    repository_platform = user_options['repository_platform']
+    execution_platform = user_options['execution_platform']
+    project_name = user_options['project_name']
+
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
 
     if not list_columns and not (command or custom_script_path):
@@ -2594,7 +2679,7 @@ def list_files(ctx,
         'project_name': False
     }
 
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -2610,6 +2695,11 @@ def list_files(ctx,
             project_name=project_name
         )
     )
+    # Setup only the required parameters
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    project_name = user_options['project_name']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
 
@@ -2696,7 +2786,7 @@ def list_files(ctx,
 @click.argument("source_path", required=True)
 @click.argument("destination_path", required=True)
 @click.option('-k', '--apikey', required=True, help='Your CloudOS API key.')
-@click.option('-c', '--cloudos-url', default=CLOUDOS_URL, required=False, help='The CloudOS URL.')
+@click.option('-c', '--cloudos-url', default=CLOUDOS_URL, required=True, help='The CloudOS URL.')
 @click.option('--workspace-id', required=True, help='The CloudOS workspace ID.')
 @click.option('--project-name', required=True, help='The source project name.')
 @click.option('--destination-project-name', required=False,
@@ -2736,7 +2826,7 @@ def move_files(ctx, source_path, destination_path, apikey, cloudos_url, workspac
         'project_name': True
     }
 
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -2752,6 +2842,10 @@ def move_files(ctx, source_path, destination_path, apikey, cloudos_url, workspac
             project_name=project_name
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    project_name = user_options['project_name']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
 
@@ -2880,7 +2974,7 @@ def renaming_item(ctx, source_path, new_name, apikey, cloudos_url,
         'project_name': True
     }
 
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -2896,6 +2990,10 @@ def renaming_item(ctx, source_path, new_name, apikey, cloudos_url,
             project_name=project_name
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    project_name = user_options['project_name']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     # Initialize Datasets clients
@@ -2984,7 +3082,7 @@ def copy_item_cli(ctx, source_path, destination_path, apikey, cloudos_url,
         'workflow_name': False,
         'project_name': True
     }
-    apikey, cloudos_url, workspace_id, workflow_name, _, _, project_name = config_manager.load_profile_and_validate_data(
+    user_options = config_manager.load_profile_and_validate_data(
         ctx, INIT_PROFILE, CLOUDOS_URL, profile=profile,
         required_dict=required_dict,
         apikey=apikey,
@@ -2995,6 +3093,11 @@ def copy_item_cli(ctx, source_path, destination_path, apikey, cloudos_url,
         execution_platform=None,
         project_name=project_name
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    project_name = user_options['project_name']
+
     destination_project_name = destination_project_name or project_name
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     # Initialize clients
@@ -3120,7 +3223,7 @@ def mkdir_item(ctx, new_folder_path, apikey, cloudos_url,
         'project_name': True
     }
 
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -3136,6 +3239,10 @@ def mkdir_item(ctx, new_folder_path, apikey, cloudos_url,
             project_name=project_name
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    project_name = user_options['project_name']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
 
@@ -3226,7 +3333,7 @@ def rm_item(ctx, target_path, apikey, cloudos_url,
         'project_name': True
     }
 
-    apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name = (
+    user_options = (
         config_manager.load_profile_and_validate_data(
             ctx,
             INIT_PROFILE,
@@ -3242,6 +3349,10 @@ def rm_item(ctx, target_path, apikey, cloudos_url,
             project_name=project_name
         )
     )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    project_name = user_options['project_name']
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
 
@@ -3294,5 +3405,67 @@ def rm_item(ctx, target_path, apikey, cloudos_url,
         sys.exit(1)
 
 
+@datasets.command(name="link")
+@click.argument("s3_path", required=True)
+@click.option('-k', '--apikey', help='Your CloudOS API key', required=True)
+@click.option('-c', '--cloudos-url',
+              help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
+              default=CLOUDOS_URL)
+@click.option('--workspace-id', help='The specific CloudOS workspace id.', required=True)
+@click.option('--session-id', help='The specific CloudOS interactive session id.', required=True)
+@click.option('--disable-ssl-verification', is_flag=True, help='Disable SSL certificate verification.')
+@click.option('--ssl-cert', help='Path to your SSL certificate file.')
+@click.option('--profile', help='Profile to use from the config file', default='default')
+@click.pass_context
+def link(ctx, s3_path, apikey, cloudos_url, workspace_id, session_id, disable_ssl_verification, ssl_cert, profile):
+    """
+    Link a S3 folder to an active interactive analysis.
+
+    S3_PATH [path]: the full path to the S3 folder to link. E.g.: 's3://bucket-name/folder/subfolder'\n
+    """
+    print(link.__doc__ + '\n')
+
+    profile = profile or ctx.default_map['datasets']['link']['profile']
+
+    # Create a dictionary with required and non-required params
+    required_dict = {
+        'apikey': True,
+        'workspace_id': True,
+        'workflow_name': False,
+        'project_name': False
+    }
+    # determine if the user provided all required parameters
+    config_manager = ConfigurationProfile()
+    user_options = (
+        config_manager.load_profile_and_validate_data(
+            ctx,
+            INIT_PROFILE,
+            CLOUDOS_URL,
+            profile=profile,
+            required_dict=required_dict,
+            apikey=apikey,
+            cloudos_url=cloudos_url,
+            workspace_id=workspace_id,
+            session_id=session_id
+        )
+    )
+    # Unpack the user options
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+    session_id = user_options['session_id']
+
+    verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
+    link_p = Link(
+        cloudos_url=cloudos_url,
+        apikey=apikey,
+        workspace_id=workspace_id,
+        cromwell_token=None,
+        project_name=None,
+        verify=verify_ssl
+    )
+    link_p.link_S3_folder(s3_path, session_id)
+
+
 if __name__ == "__main__":
     run_cloudos_cli()

cloudos_cli-2.38.0/cloudos_cli/_version.py

@@ -0,0 +1 @@
+__version__ = '2.38.0'

{cloudos_cli-2.37.0 → cloudos_cli-2.38.0}/cloudos_cli/configure/configure.py

@@ -68,6 +68,7 @@ class ConfigurationProfile:
             shared_config['workflow_name'] = profile_data.get('workflow_name', None)
             shared_config['repository_platform'] = profile_data.get('repository_platform', None)
             shared_config['execution_platform'] = profile_data.get('execution_platform', None)
+            shared_config['session_id'] = profile_data.get('session_id', None)
             shared_config['profile'] = profile_name
             print(f"Profile '{profile_name}' already exists. You can update single parameters or all.")
 
@@ -183,6 +184,18 @@ class ConfigurationProfile:
         else:
             workflow_name = workflow_name
 
+        # Interactive Analysis ID
+        session_id = input(
+            f"Interactive Analysis ID [{shared_config.get('session_id', profile_name)}]: "
+        ).strip()
+        # If the user presses Enter, keep the existing value
+        if session_id == "" and shared_config.get('session_id', None) is not None:
+            session_id = shared_config['session_id']
+        elif session_id == "":
+            session_id = None
+        else:
+            session_id = session_id
+
         # Make the profile the default if it is the first one
         if number_of_profiles >= 1:
             default_profile = self.determine_default_profile()
@@ -227,6 +240,9 @@ class ConfigurationProfile:
             config[profile_name]['workflow_name'] = workflow_name
         if default_profile is not None:
             config[profile_name]['default'] = str(default_profile)
+        if session_id is not None:
+            config[profile_name]['session_id'] = session_id
+        
 
         with open(self.config_file, 'w') as conf_file:
             config.write(conf_file)
@@ -360,6 +376,7 @@ class ConfigurationProfile:
             'workflow_name': config[profile_name].get('workflow_name', ""),
             'execution_platform': config[profile_name].get('execution_platform', ""),
             'repository_platform': config[profile_name].get('repository_platform', ""),
+            'session_id': config[profile_name].get('session_id', "")
         }
 
     def check_if_profile_exists(self, profile_name):
@@ -427,7 +444,7 @@ class ConfigurationProfile:
 
     def load_profile_and_validate_data(self, ctx, init_profile, cloudos_url_default, profile, required_dict, apikey=None,
                                        cloudos_url=None, workspace_id=None, project_name=None, workflow_name=None,
-                                       execution_platform=None, repository_platform=None):
+                                       execution_platform=None, repository_platform=None, session_id=None):
         """
         Load profile data and validate required parameters.
 
@@ -443,7 +460,7 @@ class ConfigurationProfile:
             The profile name to load.
         required_dict : dict
             A dictionary with param name as key and whether is required or not (as bool) as value.
-        apikey, cloudos_url, workspace_id, project_name, workflow_name, execution_platform, repository_platform : string
+        apikey, cloudos_url, workspace_id, project_name, workflow_name, execution_platform, repository_platform, session_id : string
             The values coming from the CLI to be compared with the profile
 
         Returns
@@ -457,8 +474,7 @@ class ConfigurationProfile:
             profile_data = self.load_profile(profile_name=profile)
             apikey = self.get_param_value(ctx, apikey, 'apikey', profile_data['apikey'],
                                           required=required_dict['apikey'], missing_required_params=missing)
-            cloudos_url = self.get_param_value(ctx, cloudos_url, 'cloudos_url',
-                                               profile_data['cloudos_url']) or cloudos_url_default
+            resolved_cloudos_url = self.get_param_value(ctx, cloudos_url, 'cloudos_url', profile_data['cloudos_url'])
             workspace_id = self.get_param_value(ctx, workspace_id, 'workspace_id', profile_data['workspace_id'],
                                                 required=required_dict['workspace_id'], missing_required_params=missing)
             workflow_name = self.get_param_value(ctx, workflow_name, 'workflow_name', profile_data['workflow_name'],
@@ -469,11 +485,13 @@ class ConfigurationProfile:
                                                       profile_data['execution_platform'])
             project_name = self.get_param_value(ctx, project_name, 'project_name', profile_data['project_name'],
                                                 required=required_dict['project_name'], missing_required_params=missing)
+            session_id = self.get_param_value(ctx, session_id, 'session_id', profile_data.get('session_id', None))
         else:
             # when no profile is used, we need to check if the user provided all required parameters
             apikey = self.get_param_value(ctx, apikey, 'apikey', apikey, required=required_dict['apikey'],
                                           missing_required_params=missing)
-            cloudos_url = self.get_param_value(ctx, cloudos_url, 'cloudos_url', cloudos_url) or cloudos_url_default
+            resolved_cloudos_url = self.get_param_value(ctx, cloudos_url, 'cloudos_url', cloudos_url,
+                                          missing_required_params=missing) 
             workspace_id = self.get_param_value(ctx, workspace_id, 'workspace_id', workspace_id,
                                                 required=required_dict['workspace_id'],
                                                 missing_required_params=missing)
@@ -486,10 +504,29 @@ class ConfigurationProfile:
             project_name = self.get_param_value(ctx, project_name, 'project_name', project_name,
                                                 required=required_dict['project_name'],
                                                 missing_required_params=missing)
+            session_id = self.get_param_value(ctx, session_id, 'session_id', session_id)
+        if not resolved_cloudos_url:
+            click.secho(
+                f"[Warning] No CloudOS URL provided via CLI or profile. Falling back to default: {cloudos_url_default}",
+                fg="yellow",
+                bold=True
+            )
+            cloudos_url = cloudos_url_default
+        else:
+            cloudos_url = resolved_cloudos_url
         cloudos_url = cloudos_url.rstrip('/')
 
         # Raise once, after all checks
         if missing:
             formatted = ', '.join(p for p in missing)
-            raise click.UsageError(f"Missing required option/s: {formatted}")
-        return apikey, cloudos_url, workspace_id, workflow_name, repository_platform, execution_platform, project_name
+            raise click.UsageError(f"Missing required option/s: {formatted} \nYou can configure the following parameters persistently by running cloudos configure:\n  --apikey,\n  --cloudos-url,\n  --workspace-id,\n  --workflow-name,\n  --repository-platform,\n  --execution-platform,\n  --project-name\n For more information on the usage of the command, please run cloudos configure --help")
+        return {
+            'apikey': apikey,
+            'cloudos_url': cloudos_url,
+            'workspace_id': workspace_id,
+            'workflow_name': workflow_name,
+            'repository_platform': repository_platform,
+            'execution_platform': execution_platform,
+            'project_name': project_name,
+            'session_id': session_id
+        }

cloudos_cli-2.38.0/cloudos_cli/link/__init__.py

@@ -0,0 +1,8 @@
+"""
+Functions and classes related to datasets.
+"""
+
+from .link import Link
+
+
+__all__ = ['link']

cloudos_cli-2.38.0/cloudos_cli/link/link.py

@@ -0,0 +1,125 @@
+"""
+This is the main class for linking files to interactive sessions.
+"""
+
+from dataclasses import dataclass
+from typing import Union
+from cloudos_cli.clos import Cloudos
+from cloudos_cli.utils.requests import retry_requests_post
+from urllib.parse import urlparse
+
+
+@dataclass
+class Link(Cloudos):
+    """Class for linking folders/files to interactive sessions.
+
+    Parameters
+    ----------
+    cloudos_url : string
+        The CloudOS service url.
+    apikey : string
+        Your CloudOS API key.
+    workspace_id : string
+        The specific Cloudos workspace id.
+    verify: [bool|string]
+        Whether to use SSL verification or not. Alternatively, if
+        a string is passed, it will be interpreted as the path to
+        the SSL certificate file.
+    """
+    workspace_id: str
+    project_name: str
+    verify: Union[bool, str] = True
+
+    def link_S3_folder(self,
+                       s3_folder: str,
+                       session_id: str) -> dict:
+        """Link an S3 folder to an interactive session.
+
+        Parameters
+        ----------
+        s3_folder : str
+            The S3 folder to link.
+        session_id : str
+            The interactive session ID.
+
+        Raises
+        ------
+        ValueError
+            If the S3 URL already exists with 'mounted' status
+            If the API key is invalid or permissions are insufficient
+            If the S3 URL is invalid or the session is not active.
+        """
+        url = (
+            f"{self.cloudos_url}/api/v1/"
+            f"interactive-sessions/{session_id}/fuse-filesystem/mount"
+            f"?teamId={self.workspace_id}"
+        )
+        headers = {
+            "Content-type": "application/json",
+            "apikey": self.apikey
+        }
+        data = self.parse_s3_path(s3_folder)
+        r = retry_requests_post(url, headers=headers, json=data, verify=self.verify)
+
+        if r.status_code == 403:
+            raise ValueError(f"Provided folder already exists with 'mounted' status")
+        elif r.status_code == 401:
+            raise ValueError(f"Forbidden: Invalid API key or insufficient permissions")
+        elif r.status_code == 400:
+            raise ValueError("Bad request: please make sure the S3 URL is valid, and the session is active")
+        elif r.status_code == 204:
+            full_path = (
+                f"s3://{data['dataItem']['data']['s3BucketName']}/"
+                f"{data['dataItem']['data']['s3Prefix']}"
+            )
+            print(f"Succesfully linked S3 folder: {full_path}")
+
+
+    def parse_s3_path(self, s3_url):
+        """
+        Parses an S3 URL and extracts the bucket name, prefix, and base name.
+
+        Parameters
+        ----------
+        s3_url : str
+            The S3 URL to parse. Must start with "s3://".
+
+        Returns
+        -------
+        dict: A dictionary containing the parsed S3 information structured as:
+                "dataItem": {
+                    "type": "S3Folder",
+                    "data": {
+                        "name": str,          # The base name (last segment of the prefix).
+                        "s3BucketName": str,  # The name of the S3 bucket.
+                        "s3Prefix": str       # The full prefix path in the bucket.
+
+        Raises
+        ------
+        ValueError
+            If the S3 URL does not start with "s3://".
+            If the S3 URL does not include a key after the bucket.
+        """
+        if not s3_url.startswith("s3://"):
+            raise ValueError("Invalid S3 URL. Link must start with 's3://'")
+
+        parsed = urlparse(s3_url)
+        bucket = parsed.netloc
+        prefix = parsed.path.lstrip('/') # Remove leading slash
+
+        if not prefix:
+            raise ValueError("S3 URL must include a key after the bucket")
+
+        parts = prefix.rstrip('/').split('/')
+        base = parts[-1] # Last segment (file or folder)
+        return {
+            "dataItem": {
+            "type": "S3Folder",
+            "data": {
+                "name": base,
+                "s3BucketName": bucket,
+                "s3Prefix": prefix
+            }
+            }
+        }
+

{cloudos_cli-2.37.0 → cloudos_cli-2.38.0}/cloudos_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.37.0
+Version: 2.38.0
 Summary: Python package for interacting with CloudOS
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -106,7 +106,7 @@ In order to generate a profile called `default`, the following command can be us
 cloudos configure
 ```
 
-This will bring in prompts for API, platform URL, project name, platform executor, repository provider, workflow name (if any). This will be the default profile if no other was explicitly set. The default profile allows running all subcommands without adding `--profile` option in the command line.
+This will bring in prompts for API, platform URL, project name, platform executor, repository provider, workflow name (if any) and session ID for interactive analysis. This will be the default profile if no other was explicitly set. The default profile allows running all subcommands without adding `--profile` option in the command line.
 
 ### Configure named profile
 
@@ -1089,6 +1089,32 @@ Please, note that in the above example a preconfigured profile has been used. If
     --project-name $PROJEC_NAME
 ```
 
+#### Linking S3 folders to Interactive Analysis
+
+The CLI can link S3 folders into an interactive session, which will appear in the left panel of the session as "mounted". This subcommand is using the option `--session-id` to access the correct interactive session. This option can be added to the CLI or defined in a profile, for convenience.
+
+The usage of the subcommand is as follows:
+
+```console
+cloudos datasets link <S3 FOLDER COMPLETE PATH> <options>
+```
+
+For example:
+
+```console
+cloudos datasets link s3://bucket/path/folder --profile test
+```
+where profile `test` has already defined the `--session-id`.
+
+Or specifying explicitly:
+
+```console
+cloudos datasets link s3://bucket/path/folder --profile test --session-id 1234
+```
+where profile `test` is used for all other necessary variables, but `--session-id` is overwritten from the command line.
+
+> [!NOTE]
+> If running the CLI inside a jupyter session, the pre-configured CLI installation will have the session ID already installed and only the `--apikey` needs to be added.
 
 #### Create a (virtual) folder
 

{cloudos_cli-2.37.0 → cloudos_cli-2.38.0}/cloudos_cli.egg-info/SOURCES.txt

@@ -19,6 +19,8 @@ cloudos_cli/import_wf/__init__.py
 cloudos_cli/import_wf/import_wf.py
 cloudos_cli/jobs/__init__.py
 cloudos_cli/jobs/job.py
+cloudos_cli/link/__init__.py
+cloudos_cli/link/link.py
 cloudos_cli/queue/__init__.py
 cloudos_cli/queue/queue.py
 cloudos_cli/utils/__init__.py

cloudos_cli-2.37.0/cloudos_cli/_version.py

@@ -1 +0,0 @@
-__version__ = '2.37.0'