cloudos-cli 2.62.0__tar.gz → 2.64.1__tar.gz

{cloudos_cli-2.62.0/cloudos_cli.egg-info → cloudos_cli-2.64.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.62.0
+Version: 2.64.1
 Summary: Python package for interacting with CloudOS
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -643,6 +643,12 @@ Nextflow standard output: s3://path/to/location/of/logs/stdout.txt
 Trace file: s3://path/to/location/of/logs/trace.txt
 ```
 
+You can also link the logs directory to an interactive session using the `--link` flag. This will mount the entire logs directory, providing access to all log files in your interactive session:
+
+```bash
+cloudos job logs --profile my_profile --job-id "12345678910" --link --session-id your_session_id
+```
+
 #### Get Job Results
 
 The following command allows you to get the path where CloudOS stores the output files for a job. This can be used only on your user's jobs and for jobs with "completed" status.
@@ -656,6 +662,12 @@ Executing results...
 results: s3://path/to/location/of/results/results/
 ```
 
+You can also link all result directories to an interactive session using the `--link` flag. This will mount all result directories from the job, providing direct access to output files in your interactive session:
+
+```bash
+cloudos job results --profile my_profile --job-id "12345678910" --link --session-id your_session_id
+```
+
 
 #### Clone or Resume job
 
@@ -908,6 +920,15 @@ Finding working directory path...
 Working directory for job 68747bac9e7fe38ec6e022ad: az://123456789000.blob.core.windows.net/cloudos-987652349087/projects/455654676/jobs/54678856765/work
 ```
 
+You can also link the working directory to an interactive session using the `--link` flag. This requires specifying a session ID either through the `--session-id` option or from a configured profile:
+
+```shell
+cloudos job workdir \
+    --profile profile-name \
+    --job-id 62c83a1191fe06013b7ef355 \
+    --link --session-id your_session_id
+```
+
 #### List Jobs
 
 You can get a summary of workspace jobs in two different formats:
@@ -1387,7 +1408,7 @@ The output of this command is a list of files and folders present in the specifi
 > If the `<path>` is left empty, the command will return the list of folders present in the selected project.
 
 If you require more information on the files and folder listed, you can use the `--details` flag that will output a table containing the following columns:
-- Type (folder or file)
+- Type (s3 folder, azure folder , virtual folder, file (user uploaded) or file (virtual copy))
 - Owner
 - Size (in human readable format)
 - Last updated
@@ -1776,7 +1797,7 @@ j_id = j.send_job(job_config)
 To check the status:
 
 ```python
-j_status = j.get_job_status(j_id)
+j_status = j.get_job_status(j_id, workspace_id)
 j_status_h = json.loads(j_status.content)["status"]
 print(j_status_h)
 ```
@@ -1851,7 +1872,7 @@ print(c_status_h)
 j = jb.Job(cloudos_url, apikey, None, workspace_id, project_name, workflow_name, True, mainfile,
            importsfile)
 j_id = j.send_job(job_config, workflow_type='wdl', cromwell_id=json.loads(c_status.content)["_id"])
-j_status = j.get_job_status(j_id)
+j_status = j.get_job_status(j_id, workspace_id)
 j_status_h = json.loads(j_status.content)["status"]
 print(j_status_h)
 

cloudos_cli-2.62.0/PKG-INFO → cloudos_cli-2.64.1/README.md

@@ -1,38 +1,3 @@
-Metadata-Version: 2.4
-Name: cloudos_cli
-Version: 2.62.0
-Summary: Python package for interacting with CloudOS
-Home-page: https://github.com/lifebit-ai/cloudos-cli
-Author: David Piñeyro
-Author-email: david.pineyro@lifebit.ai
-Classifier: Programming Language :: Python :: 3
-Classifier: Operating System :: POSIX :: Linux
-Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
-Requires-Python: >=3.9
-Description-Content-Type: text/markdown
-License-File: LICENSE
-Requires-Dist: click>=8.0.1
-Requires-Dist: rich-click>=1.8.2
-Requires-Dist: pandas>=1.3.4
-Requires-Dist: numpy>=1.26.4
-Requires-Dist: requests>=2.26.0
-Provides-Extra: test
-Requires-Dist: pytest; extra == "test"
-Requires-Dist: mock; extra == "test"
-Requires-Dist: responses; extra == "test"
-Requires-Dist: requests_mock; extra == "test"
-Dynamic: author
-Dynamic: author-email
-Dynamic: classifier
-Dynamic: description
-Dynamic: description-content-type
-Dynamic: home-page
-Dynamic: license-file
-Dynamic: provides-extra
-Dynamic: requires-dist
-Dynamic: requires-python
-Dynamic: summary
-
 # cloudos-cli
 
 [![CI_tests](https://github.com/lifebit-ai/cloudos-cli/actions/workflows/ci.yml/badge.svg)](https://github.com/lifebit-ai/cloudos-cli/actions/workflows/ci.yml)
@@ -643,6 +608,12 @@ Nextflow standard output: s3://path/to/location/of/logs/stdout.txt
 Trace file: s3://path/to/location/of/logs/trace.txt
 ```
 
+You can also link the logs directory to an interactive session using the `--link` flag. This will mount the entire logs directory, providing access to all log files in your interactive session:
+
+```bash
+cloudos job logs --profile my_profile --job-id "12345678910" --link --session-id your_session_id
+```
+
 #### Get Job Results
 
 The following command allows you to get the path where CloudOS stores the output files for a job. This can be used only on your user's jobs and for jobs with "completed" status.
@@ -656,6 +627,12 @@ Executing results...
 results: s3://path/to/location/of/results/results/
 ```
 
+You can also link all result directories to an interactive session using the `--link` flag. This will mount all result directories from the job, providing direct access to output files in your interactive session:
+
+```bash
+cloudos job results --profile my_profile --job-id "12345678910" --link --session-id your_session_id
+```
+
 
 #### Clone or Resume job
 
@@ -908,6 +885,15 @@ Finding working directory path...
 Working directory for job 68747bac9e7fe38ec6e022ad: az://123456789000.blob.core.windows.net/cloudos-987652349087/projects/455654676/jobs/54678856765/work
 ```
 
+You can also link the working directory to an interactive session using the `--link` flag. This requires specifying a session ID either through the `--session-id` option or from a configured profile:
+
+```shell
+cloudos job workdir \
+    --profile profile-name \
+    --job-id 62c83a1191fe06013b7ef355 \
+    --link --session-id your_session_id
+```
+
 #### List Jobs
 
 You can get a summary of workspace jobs in two different formats:
@@ -1387,7 +1373,7 @@ The output of this command is a list of files and folders present in the specifi
 > If the `<path>` is left empty, the command will return the list of folders present in the selected project.
 
 If you require more information on the files and folder listed, you can use the `--details` flag that will output a table containing the following columns:
-- Type (folder or file)
+- Type (s3 folder, azure folder , virtual folder, file (user uploaded) or file (virtual copy))
 - Owner
 - Size (in human readable format)
 - Last updated
@@ -1776,7 +1762,7 @@ j_id = j.send_job(job_config)
 To check the status:
 
 ```python
-j_status = j.get_job_status(j_id)
+j_status = j.get_job_status(j_id, workspace_id)
 j_status_h = json.loads(j_status.content)["status"]
 print(j_status_h)
 ```
@@ -1851,7 +1837,7 @@ print(c_status_h)
 j = jb.Job(cloudos_url, apikey, None, workspace_id, project_name, workflow_name, True, mainfile,
            importsfile)
 j_id = j.send_job(job_config, workflow_type='wdl', cromwell_id=json.loads(c_status.content)["_id"])
-j_status = j.get_job_status(j_id)
+j_status = j.get_job_status(j_id, workspace_id)
 j_status_h = json.loads(j_status.content)["status"]
 print(j_status_h)
 

{cloudos_cli-2.62.0 → cloudos_cli-2.64.1}/cloudos_cli/__main__.py

@@ -780,6 +780,7 @@ def run(ctx,
         print('\tPlease, wait until job completion (max wait time of ' +
               f'{wait_time} seconds).\n')
         j_status = j.wait_job_completion(job_id=j_id,
+                                         workspace_id=workspace_id,
                                          wait_time=wait_time,
                                          request_interval=request_interval,
                                          verbose=verbose,
@@ -793,7 +794,7 @@ def run(ctx,
             print(f'\nJob status for job "{j_name}" (ID: {j_id}): {j_final_s}')
             sys.exit(1)
     else:
-        j_status = j.get_job_status(j_id, verify_ssl)
+        j_status = j.get_job_status(j_id, workspace_id, verify_ssl)
         j_status_h = json.loads(j_status.content)["status"]
         print(f'\tYour current job status is: {j_status_h}')
         print('\tTo further check your job status you can either go to ' +
@@ -813,6 +814,9 @@ def run(ctx,
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
               default=CLOUDOS_URL,
               required=True)
+@click.option('--workspace-id',
+              help='The specific CloudOS workspace id.',
+              required=True)
 @click.option('--job-id',
               help='The job id in CloudOS to search for.',
               required=True)
@@ -830,6 +834,7 @@ def run(ctx,
 def job_status(ctx,
                apikey,
                cloudos_url,
+               workspace_id,
                job_id,
                verbose,
                disable_ssl_verification,
@@ -841,7 +846,7 @@ def job_status(ctx,
     # Create a dictionary with required and non-required params
     required_dict = {
         'apikey': True,
-        'workspace_id': False,
+        'workspace_id': True,
         'workflow_name': False,
         'project_name': False,
         'session_id': False,
@@ -855,6 +860,7 @@ def job_status(ctx,
             INIT_PROFILE,
             CLOUDOS_URL,
             profile=profile,
+            workspace_id=workspace_id,
             required_dict=required_dict,
             apikey=apikey,
             cloudos_url=cloudos_url
@@ -862,6 +868,7 @@ def job_status(ctx,
     )
     apikey = user_options['apikey']
     cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     print('Executing status...')
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
@@ -873,7 +880,7 @@ def job_status(ctx,
         print('\t' + str(cl) + '\n')
         print(f'\tSearching for job id: {job_id}')
     try:
-        j_status = cl.get_job_status(job_id, verify_ssl)
+        j_status = cl.get_job_status(job_id, workspace_id, verify_ssl)
         j_status_h = json.loads(j_status.content)["status"]
         print(f'\tYour current job status is: {j_status_h}\n')
         j_url = f'{cloudos_url}/app/advanced-analytics/analyses/{job_id}'
@@ -901,6 +908,12 @@ def job_status(ctx,
 @click.option('--job-id',
               help='The job id in CloudOS to search for.',
               required=True)
+@click.option('--link',
+              help='Link the working directory to an interactive session.',
+              is_flag=True)
+@click.option('--session-id',
+              help='The specific CloudOS interactive session id. Required when using --link flag.',
+              required=False)
 @click.option('--verbose',
               help='Whether to print information messages or not.',
               is_flag=True)
@@ -917,6 +930,8 @@ def job_workdir(ctx,
                 cloudos_url,
                 workspace_id,
                 job_id,
+                link,
+                session_id,
                 verbose,
                 disable_ssl_verification,
                 ssl_cert,
@@ -944,17 +959,30 @@ def job_workdir(ctx,
             required_dict=required_dict,
             apikey=apikey,
             cloudos_url=cloudos_url,
-            workspace_id=workspace_id
+            workspace_id=workspace_id,
+            session_id=session_id
         )
     )
     apikey = user_options['apikey']
     cloudos_url = user_options['cloudos_url']
     workspace_id = user_options['workspace_id']
+    # Get session_id from user_options (which includes profile data)
+    session_id = user_options.get('session_id') or session_id
+
+    # Validate link flag requirements AFTER loading profile
+    if link and not session_id:
+        raise click.ClickException("--session-id is required when using --link flag")
 
     print('Finding working directory path...')
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     if verbose:
         print('\t...Preparing objects')
+        print('\tUsing the following parameters:')
+        print(f'\t\tCloudOS url: {cloudos_url}')
+        print(f'\t\tWorkspace ID: {workspace_id}')
+        print(f'\t\tJob ID: {job_id}')
+        if link:
+            print(f'\t\tSession ID: {session_id}')
     cl = Cloudos(cloudos_url, apikey, None)
     if verbose:
         print('\tThe following Cloudos object was created:')
@@ -963,6 +991,24 @@ def job_workdir(ctx,
     try:
         workdir = cl.get_job_workdir(job_id, workspace_id, verify_ssl)
         print(f"Working directory for job {job_id}: {workdir}")
+        
+        # Link to interactive session if requested
+        if link:
+            if verbose:
+                print(f'\tLinking working directory to interactive session {session_id}...')
+            
+            # Use Link class to perform the linking
+            link_client = Link(
+                cloudos_url=cloudos_url,
+                apikey=apikey,
+                cromwell_token=None,  # Not needed for linking operations
+                workspace_id=workspace_id,
+                project_name=None,  # Not needed for S3 paths
+                verify=verify_ssl
+            )
+            
+            link_client.link_folder(workdir.strip(), session_id)
+            
     except BadRequestException as e:
         raise ValueError(f"Job '{job_id}' not found or not accessible: {str(e)}")
     except Exception as e:
@@ -985,6 +1031,12 @@ def job_workdir(ctx,
 @click.option('--job-id',
               help='The job id in CloudOS to search for.',
               required=True)
+@click.option('--link',
+              help='Link the logs directories to an interactive session.',
+              is_flag=True)
+@click.option('--session-id',
+              help='The specific CloudOS interactive session id. Required when using --link flag.',
+              required=False)
 @click.option('--verbose',
               help='Whether to print information messages or not.',
               is_flag=True)
@@ -1001,6 +1053,8 @@ def job_logs(ctx,
              cloudos_url,
              workspace_id,
              job_id,
+             link,
+             session_id,
              verbose,
              disable_ssl_verification,
              ssl_cert,
@@ -1028,17 +1082,30 @@ def job_logs(ctx,
             required_dict=required_dict,
             apikey=apikey,
             cloudos_url=cloudos_url,
-            workspace_id=workspace_id
+            workspace_id=workspace_id,
+            session_id=session_id
         )
     )
     apikey = user_options['apikey']
     cloudos_url = user_options['cloudos_url']
     workspace_id = user_options['workspace_id']
+    # Get session_id from user_options (which includes profile data)
+    session_id = user_options.get('session_id') or session_id
+
+    # Validate link flag requirements AFTER loading profile
+    if link and not session_id:
+        raise click.ClickException("--session-id is required when using --link flag")
 
     print('Executing logs...')
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     if verbose:
         print('\t...Preparing objects')
+        print('\tUsing the following parameters:')
+        print(f'\t\tCloudOS url: {cloudos_url}')
+        print(f'\t\tWorkspace ID: {workspace_id}')
+        print(f'\t\tJob ID: {job_id}')
+        if link:
+            print(f'\t\tSession ID: {session_id}')
     cl = Cloudos(cloudos_url, apikey, None)
     if verbose:
         print('\tThe following Cloudos object was created:')
@@ -1047,7 +1114,37 @@ def job_logs(ctx,
     try:
         logs = cl.get_job_logs(job_id, workspace_id, verify_ssl)
         for name, path in logs.items():
-            print(f"{name}: {path}\n")
+            print(f"{name}: {path}")
+        
+        # Link to interactive session if requested
+        if link:
+            if logs:
+                # Extract the parent logs directory from any log file path
+                # All log files should be in the same logs directory
+                first_log_path = next(iter(logs.values()))
+                # Remove the filename to get the logs directory
+                # e.g., "s3://bucket/path/to/logs/filename.txt" -> "s3://bucket/path/to/logs"
+                logs_dir = '/'.join(first_log_path.split('/')[:-1])
+                
+                if verbose:
+                    print(f'\tLinking logs directory to interactive session {session_id}...')
+                    print(f'\t\tLogs directory: {logs_dir}')
+                
+                # Use Link class to perform the linking
+                link_client = Link(
+                    cloudos_url=cloudos_url,
+                    apikey=apikey,
+                    cromwell_token=None,  # Not needed for linking operations
+                    workspace_id=workspace_id,
+                    project_name=None,  # Not needed for S3 paths
+                    verify=verify_ssl
+                )
+                
+                link_client.link_folder(logs_dir, session_id)
+            else:
+                if verbose:
+                    print('\tNo logs found to link.')
+            
     except BadRequestException as e:
         raise ValueError(f"Job '{job_id}' not found or not accessible: {str(e)}")
     except Exception as e:
@@ -1070,6 +1167,12 @@ def job_logs(ctx,
 @click.option('--job-id',
               help='The job id in CloudOS to search for.',
               required=True)
+@click.option('--link',
+              help='Link the results directories to an interactive session.',
+              is_flag=True)
+@click.option('--session-id',
+              help='The specific CloudOS interactive session id. Required when using --link flag.',
+              required=False)
 @click.option('--verbose',
               help='Whether to print information messages or not.',
               is_flag=True)
@@ -1086,6 +1189,8 @@ def job_results(ctx,
                 cloudos_url,
                 workspace_id,
                 job_id,
+                link,
+                session_id,
                 verbose,
                 disable_ssl_verification,
                 ssl_cert,
@@ -1113,17 +1218,30 @@ def job_results(ctx,
             required_dict=required_dict,
             apikey=apikey,
             cloudos_url=cloudos_url,
-            workspace_id=workspace_id
+            workspace_id=workspace_id,
+            session_id=session_id
         )
     )
     apikey = user_options['apikey']
     cloudos_url = user_options['cloudos_url']
     workspace_id = user_options['workspace_id']
+    # Get session_id from user_options (which includes profile data)
+    session_id = user_options.get('session_id') or session_id
+
+    # Validate link flag requirements AFTER loading profile
+    if link and not session_id:
+        raise click.ClickException("--session-id is required when using --link flag")
 
     print('Executing results...')
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
     if verbose:
         print('\t...Preparing objects')
+        print('\tUsing the following parameters:')
+        print(f'\t\tCloudOS url: {cloudos_url}')
+        print(f'\t\tWorkspace ID: {workspace_id}')
+        print(f'\t\tJob ID: {job_id}')
+        if link:
+            print(f'\t\tSession ID: {session_id}')
     cl = Cloudos(cloudos_url, apikey, None)
     if verbose:
         print('\tThe following Cloudos object was created:')
@@ -1132,7 +1250,28 @@ def job_results(ctx,
     try:
         results = cl.get_job_results(job_id, workspace_id, verify_ssl)
         for name, path in results.items():
-            print(f"{name}: {path}\n")
+            print(f"{name}: {path}")
+        
+        # Link to interactive session if requested
+        if link:
+            if verbose:
+                print(f'\tLinking {len(results)} result directories to interactive session {session_id}...')
+            
+            # Use Link class to perform the linking
+            link_client = Link(
+                cloudos_url=cloudos_url,
+                apikey=apikey,
+                cromwell_token=None,  # Not needed for linking operations
+                workspace_id=workspace_id,
+                project_name=None,  # Not needed for S3 paths
+                verify=verify_ssl
+            )
+            
+            for name, path in results.items():
+                if verbose:
+                    print(f'\t\tLinking {name} ({path})...')
+                link_client.link_folder(path, session_id)
+            
     except BadRequestException as e:
         raise ValueError(f"Job '{job_id}' not found or not accessible: {str(e)}")
     except Exception as e:
@@ -1149,6 +1288,9 @@ def job_results(ctx,
               help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
               default=CLOUDOS_URL,
               required=True)
+@click.option('--workspace-id',
+              help='The specific CloudOS workspace id.',
+              required=True)
 @click.option('--job-id',
               help='The job id in CloudOS to search for.',
               required=True)
@@ -1179,6 +1321,7 @@ def job_results(ctx,
 def job_details(ctx,
                 apikey,
                 cloudos_url,
+                workspace_id,
                 job_id,
                 output_format,
                 output_basename,
@@ -1193,7 +1336,7 @@ def job_details(ctx,
     # Create a dictionary with required and non-required params
     required_dict = {
         'apikey': True,
-        'workspace_id': False,
+        'workspace_id': True,
         'workflow_name': False,
         'project_name': False,
         'session_id': False,
@@ -1207,6 +1350,7 @@ def job_details(ctx,
             INIT_PROFILE,
             CLOUDOS_URL,
             profile=profile,
+            workspace_id=workspace_id,
             required_dict=required_dict,
             apikey=apikey,
             cloudos_url=cloudos_url
@@ -1214,6 +1358,7 @@ def job_details(ctx,
     )
     apikey = user_options['apikey']
     cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
 
     if ctx.get_parameter_source('output_basename') == click.core.ParameterSource.DEFAULT:
         output_basename = f"{job_id}_details"
@@ -1230,7 +1375,7 @@ def job_details(ctx,
 
     # check if the API gives a 403 error/forbidden error
     try:
-        j_details = cl.get_job_status(job_id, verify_ssl)
+        j_details = cl.get_job_status(job_id, workspace_id, verify_ssl)
     except BadRequestException as e:
         if '403' in str(e) or 'Forbidden' in str(e):
             raise ValueError("API can only show job details of your own jobs, cannot see other user's job details.")
@@ -1518,7 +1663,7 @@ def abort_jobs(ctx,
 
     for job in jobs:
         try:
-            j_status = cl.get_job_status(job, verify_ssl)
+            j_status = cl.get_job_status(job, None, verify_ssl)
         except Exception as e:
             click.secho(f"Failed to get status for job {job}, please make sure it exists in the workspace: {e}", fg='yellow', bold=True)
             continue
@@ -2823,6 +2968,7 @@ def run_bash_job(ctx,
         print('\tPlease, wait until job completion (max wait time of ' +
               f'{wait_time} seconds).\n')
         j_status = j.wait_job_completion(job_id=j_id,
+                                         workspace_id=workspace_id,
                                          wait_time=wait_time,
                                          request_interval=request_interval,
                                          verbose=False,
@@ -2836,7 +2982,7 @@ def run_bash_job(ctx,
             print(f'\nJob status for job "{j_name}" (ID: {j_id}): {j_final_s}')
             sys.exit(1)
     else:
-        j_status = j.get_job_status(j_id, verify_ssl)
+        j_status = j.get_job_status(j_id, workspace_id, verify_ssl)
         j_status_h = json.loads(j_status.content)["status"]
         print(f'\tYour current job status is: {j_status_h}')
         print('\tTo further check your job status you can either go to ' +
@@ -3191,6 +3337,7 @@ def run_bash_array_job(ctx,
         print('\tPlease, wait until job completion (max wait time of ' +
               f'{wait_time} seconds).\n')
         j_status = j.wait_job_completion(job_id=j_id,
+                                         workspace_id=workspace_id,
                                          wait_time=wait_time,
                                          request_interval=request_interval,
                                          verbose=False,
@@ -3204,7 +3351,7 @@ def run_bash_array_job(ctx,
             print(f'\nJob status for job "{j_name}" (ID: {j_id}): {j_final_s}')
             sys.exit(1)
     else:
-        j_status = j.get_job_status(j_id, verify_ssl)
+        j_status = j.get_job_status(j_id, workspace_id, verify_ssl)
         j_status_h = json.loads(j_status.content)["status"]
         print(f'\tYour current job status is: {j_status_h}')
         print('\tTo further check your job status you can either go to ' +
@@ -3317,6 +3464,25 @@ def list_files(ctx,
                 is_folder = "folderType" in item or item.get("isDir", False)
                 type_ = "folder" if is_folder else "file"
 
+                # Enhanced type information
+                if is_folder:
+                    folder_type = item.get("folderType")
+                    if folder_type == "VirtualFolder":
+                        type_ = "virtual folder"
+                    elif folder_type == "S3Folder":
+                        type_ = "s3 folder"
+                    elif folder_type == "AzureBlobFolder":
+                        type_ = "azure folder"
+                    else:
+                        type_ = "folder"
+                else:
+                    # Check if file is managed by Lifebit (user uploaded)
+                    is_managed_by_lifebit = item.get("isManagedByLifebit", False)
+                    if is_managed_by_lifebit:
+                        type_ = "file (user uploaded)"
+                    else:
+                        type_ = "file (virtual copy)"
+                        
                 user = item.get("user", {})
                 if isinstance(user, dict):
                     name = user.get("name", "").strip()

cloudos_cli-2.64.1/cloudos_cli/_version.py

@@ -0,0 +1 @@
+__version__ = '2.64.1'

{cloudos_cli-2.62.0 → cloudos_cli-2.64.1}/cloudos_cli/clos.py

@@ -37,13 +37,15 @@ class Cloudos:
     apikey: str
     cromwell_token: str
 
-    def get_job_status(self, j_id, verify=True):
+    def get_job_status(self, j_id, workspace_id=None, verify=True):
         """Get job status from CloudOS.
 
         Parameters
         ----------
         j_id : string
             The CloudOS job id of the job just launched.
+        workspace_id : string
+            The CloudOS workspace id from to check the job status.
         verify: [bool|string]
             Whether to use SSL verification or not. Alternatively, if
             a string is passed, it will be interpreted as the path to
@@ -60,21 +62,30 @@ class Cloudos:
             "Content-type": "application/json",
             "apikey": apikey
         }
-        r = retry_requests_get("{}/api/v1/jobs/{}".format(cloudos_url,
-                                                          j_id),
-                               headers=headers, verify=verify)
-        if r.status_code >= 400:
+        if workspace_id is not None:
+            url = f"{cloudos_url}/api/v1/jobs/{j_id}?teamId={workspace_id}"
+        else:
+            url = f"{cloudos_url}/api/v1/jobs/{j_id}"
+        r = retry_requests_get(url, headers=headers, verify=verify)
+        if r.status_code == 401:
+            raise NotAuthorisedException
+        elif r.status_code == 403:
+            # Handle 403 with more informative error message
+            self._handle_job_access_denied(j_id, workspace_id, verify)
+        elif r.status_code >= 400:
             raise BadRequestException(r)
         return r
 
-    def wait_job_completion(self, job_id, wait_time=3600, request_interval=30, verbose=False,
+    def wait_job_completion(self, job_id, workspace_id, wait_time=3600, request_interval=30, verbose=False,
                             verify=True):
         """Checks job status from CloudOS and wait for its complation.
 
         Parameters
         ----------
-        j_id : string
+        job_id : string
             The CloudOS job id of the job just launched.
+        workspace_id : string
+            The CloudOS workspace id from to check the job status.
         wait_time : int
             Max time to wait (in seconds) to job completion.
         request_interval : int
@@ -98,7 +109,7 @@ class Cloudos:
         if request_interval > wait_time:
             request_interval = wait_time
         while elapsed < wait_time:
-            j_status = self.get_job_status(job_id, verify)
+            j_status = self.get_job_status(job_id, workspace_id, verify)
             j_status_content = json.loads(j_status.content)
             j_status_h = j_status_content["status"]
             j_name = j_status_content["name"]
@@ -122,7 +133,7 @@ class Cloudos:
                         print(f'\tYour current job "{j_name}" (ID: {job_id}) status is: {j_status_h}.')
                     j_status_h_old = j_status_h
                 time.sleep(request_interval)
-        j_status = self.get_job_status(job_id, verify)
+        j_status = self.get_job_status(job_id, workspace_id, verify)
         j_status_content = json.loads(j_status.content)
         j_status_h = j_status_content["status"]
         j_name = j_status_content["name"]
@@ -202,21 +213,17 @@ class Cloudos:
             "Content-type": "application/json",
             "apikey": apikey
         }
-        r = retry_requests_get(f"{cloudos_url}/api/v1/jobs/{j_id}", headers=headers, verify=verify)
-        if r.status_code == 401:
-            raise NotAuthorisedException
-        elif r.status_code == 403:
-            # Handle 403 with more informative error message
-            self._handle_job_access_denied(j_id, workspace_id, verify)
-        elif r.status_code >= 400:
-            raise BadRequestException(r)
+        r = self.get_job_status(j_id, workspace_id, verify)
         r_json = r.json()
         job_workspace = r_json["team"]
         if job_workspace != workspace_id:
             raise ValueError("Workspace provided or configured is different from workspace where the job was executed")
+        if r_json["status"] =='initializing' or r_json["status"] =='scheduled':
+            raise ValueError("Working directories are not yet available. The job is still initializing.")
 
         if "resumeWorkDir" not in r_json:
-            print("Working directories are not available. This may be because the analysis was run without resumable mode enabled, or because intermediate results have since been removed.")
+            raise ValueError("Working directories are not available. This may be because the analysis was run without resumable mode enabled, or because intermediate results have since been removed.")
+        
         # Check if logs field exists, if not fall back to original folder-based approach
         elif "logs" in r_json:
             # Get workdir information from logs object using the same pattern as get_job_logs
@@ -322,38 +329,40 @@ class Cloudos:
             "Content-type": "application/json",
             "apikey": apikey
         }
-        r = retry_requests_get(f"{cloudos_url}/api/v1/jobs/{j_id}", headers=headers, verify=verify)
-        if r.status_code == 401:
-            raise NotAuthorisedException
-        elif r.status_code >= 400:
-            raise BadRequestException(r)
+        r = self.get_job_status(j_id, workspace_id, verify)
         r_json = r.json()
-        logs_obj = r_json["logs"]
+        
         job_workspace = r_json["team"]
         if job_workspace != workspace_id:
             raise ValueError("Workspace provided or configured is different from workspace where the job was executed")
-        cloud_name, cloud_meta, cloud_storage = find_cloud(self.cloudos_url, self.apikey, workspace_id, logs_obj)
-        container_name = cloud_storage["container"]
-        prefix_name = cloud_storage["prefix"]
-        logs_bucket = logs_obj[container_name]
-        logs_path = logs_obj[prefix_name]
-        contents_obj = self.get_storage_contents(cloud_name, cloud_meta, logs_bucket, logs_path, workspace_id, verify)
-        logs = {}
-        cloude_scheme = cloud_storage["scheme"]
-        storage_account_prefix = ''
-        if cloude_scheme == 'az':
-            storage_account_prefix = f'{workspace_id}.blob.core.windows.net/'
-        for item in contents_obj:
-            if not item["isDir"]:
-                filename = item["name"]
-                if filename == "stdout.txt":
-                    filename = "Nextflow standard output"
-                if filename == ".nextflow.log":
-                    filename = "Nextflow log"
-                if filename == "trace.txt":
-                    filename = "Trace file"
-                logs[filename] = f"{cloude_scheme}://{storage_account_prefix}{logs_bucket}/{item['path']}"
-        return logs
+        if r_json["status"] =='initializing' or r_json["status"] =='scheduled':
+            raise ValueError("Logs are not yet available. The job is still initializing.")
+        if "logs" not in r_json:
+            raise ValueError("ERROR: Logs are not available.")
+        else:
+            logs_obj = r_json["logs"]
+            cloud_name, cloud_meta, cloud_storage = find_cloud(self.cloudos_url, self.apikey, workspace_id, logs_obj)
+            container_name = cloud_storage["container"]
+            prefix_name = cloud_storage["prefix"]
+            logs_bucket = logs_obj[container_name]
+            logs_path = logs_obj[prefix_name]
+            contents_obj = self.get_storage_contents(cloud_name, cloud_meta, logs_bucket, logs_path, workspace_id, verify)
+            logs = {}
+            cloude_scheme = cloud_storage["scheme"]
+            storage_account_prefix = ''
+            if cloude_scheme == 'az':
+                storage_account_prefix = f'{workspace_id}.blob.core.windows.net/'
+            for item in contents_obj:
+                if not item["isDir"]:
+                    filename = item["name"]
+                    if filename == "stdout.txt":
+                        filename = "Nextflow standard output"
+                    if filename == ".nextflow.log":
+                        filename = "Nextflow log"
+                    if filename == "trace.txt":
+                        filename = "Trace file"
+                    logs[filename] = f"{cloude_scheme}://{storage_account_prefix}{logs_bucket}/{item['path']}"
+            return logs
 
     def get_job_results(self, j_id, workspace_id, verify=True):
         """
@@ -365,16 +374,11 @@ class Cloudos:
             "Content-type": "application/json",
             "apikey": apikey
         }
-        status = self.get_job_status(j_id, verify).json()["status"]
+        status = self.get_job_status(j_id, workspace_id, verify).json()["status"]
         if status != JOB_COMPLETED:
             raise JoBNotCompletedException(j_id, status)
 
-        r = retry_requests_get(f"{cloudos_url}/api/v1/jobs/{j_id}",
-                               headers=headers, verify=verify)
-        if r.status_code == 401:
-            raise NotAuthorisedException
-        if r.status_code >= 400:
-            raise BadRequestException(r)
+        r = self.get_job_status(j_id, workspace_id, verify)
         req_obj = r.json()
         job_workspace = req_obj["team"]
         if job_workspace != workspace_id:

{cloudos_cli-2.62.0 → cloudos_cli-2.64.1}/cloudos_cli/datasets/datasets.py

@@ -9,6 +9,7 @@ from cloudos_cli.utils.errors import BadRequestException
 from cloudos_cli.utils.requests import retry_requests_get, retry_requests_put, retry_requests_post, retry_requests_delete
 import json
 
+
 @dataclass
 class Datasets(Cloudos):
     """Class for file explorer.
@@ -105,7 +106,7 @@ class Datasets(Cloudos):
         #  Normalize response
         for item in datasets:
             item["folderType"] = True
-        response ={
+        response = {
                 "folders": datasets,
                 "files": []
             }
@@ -144,7 +145,7 @@ class Datasets(Cloudos):
         r = retry_requests_get("{}/api/v1/datasets/{}/items?teamId={}".format(self.cloudos_url,
                                                                               folder_id,
                                                                               self.workspace_id),
-                                headers=headers, verify=self.verify)
+                               headers=headers, verify=self.verify)
         if r.status_code >= 400:
             raise BadRequestException(r)
         return r.json()
@@ -175,7 +176,7 @@ class Datasets(Cloudos):
                                                                                                              s3_bucket_name,
                                                                                                              s3_relative_path,
                                                                                                              self.workspace_id),
-                                headers=headers, verify=self.verify)
+                               headers=headers, verify=self.verify)
         if r.status_code >= 400:
             raise BadRequestException(r)
         raw = r.json()
@@ -218,11 +219,11 @@ class Datasets(Cloudos):
         r = retry_requests_get("{}/api/v1/folders/virtual/{}/items?teamId={}".format(self.cloudos_url,
                                                                                      folder_id,
                                                                                      self.workspace_id),
-                                headers=headers, verify=self.verify)
+                               headers=headers, verify=self.verify)
         if r.status_code >= 400:
             raise BadRequestException(r)
         return r.json()
-    
+
     def list_azure_container_content(self, container_name: str, storage_account_name: str, path: str):
         """
         List contents of an Azure Blob container path.
@@ -259,7 +260,7 @@ class Datasets(Cloudos):
                     "blobPrefix": path_str,
                     "blobContainerName": container_name,
                     "blobStorageAccountName": storage_account_name,
-                    "kind": "Folder" 
+                    "kind": "Folder"
                 })
             else:
                 normalized["files"].append({
@@ -271,7 +272,7 @@ class Datasets(Cloudos):
                     "blobStorageAccountName": storage_account_name,
                     "sizeInBytes": item.get("size", 0),
                     "updatedAt": item.get("lastModified"),
-                    "kind": "File" 
+                    "kind": "File"
                 })
 
         return normalized
@@ -337,7 +338,7 @@ class Datasets(Cloudos):
                         container_name = job_folder['blobContainerName']
                         storage_account_name = job_folder['blobStorageAccountName']
                         blob_prefix = job_folder['blobPrefix']
-                        # trailing slash is mandatory for azure, otherwise it will not list the content of thefolde, just the folder 
+                        # trailing slash is mandatory for azure, otherwise it will not list the content of thefolde, just the folder
                         if not blob_prefix.endswith('/'):
                             blob_prefix += '/'
 
@@ -355,7 +356,7 @@ class Datasets(Cloudos):
                 raise ValueError(f"Folder '{job_name}' not found under dataset '{dataset_name}'")
 
         return folder_content
-    
+
     def move_files_and_folders(self, source_id: str, source_kind: str, target_id: str, target_kind: str):
         """
         Move a file to another dataset in CloudOS.
@@ -432,7 +433,7 @@ class Datasets(Cloudos):
         if response.status_code >= 400:
             raise BadRequestException(response)
         return response
-    
+
     def copy_item(self, item, destination_id, destination_kind):
         """Copy a file or folder (S3, Azure or Virtual) to a destination in CloudOS."""
         headers = {
@@ -454,7 +455,7 @@ class Datasets(Cloudos):
         elif item.get("folderType") == "S3Folder":
             payload = {
                 "s3BucketName": item["s3BucketName"],
-                "s3ObjectKey": item.get("s3ObjectKey") or item.get("s3Prefix"),
+                "s3Prefix": item.get("s3Prefix"),
                 "name": item["name"],
                 "parent": parent,
                 "isManagedByLifebit": item.get("isManagedByLifebit", False)
@@ -500,7 +501,7 @@ class Datasets(Cloudos):
         if response.status_code >= 400:
             raise BadRequestException(response)
         return response
-    
+
     def create_virtual_folder(self, name: str, parent_id: str, parent_kind: str):
         """
         Create a new virtual folder in CloudOS under a given parent.
@@ -542,7 +543,7 @@ class Datasets(Cloudos):
         if response.status_code >= 400:
             raise BadRequestException(response)
         return response
-    
+
     def delete_item(self, item_id: str, kind: str):
         """
         Delete a file or folder in CloudOS.
@@ -573,4 +574,4 @@ class Datasets(Cloudos):
         response = retry_requests_delete(url, headers=headers, verify=self.verify)
         if response.status_code >= 400:
             raise BadRequestException(response)
-        return response
+        return response

{cloudos_cli-2.62.0 → cloudos_cli-2.64.1}/cloudos_cli/link/link.py

@@ -68,6 +68,7 @@ class Link(Cloudos):
             data = self.parse_file_explorer_path(folder)
             type_folder = "File Explorer"
         r = retry_requests_post(url, headers=headers, json=data, verify=self.verify)
+        
         if r.status_code == 403:
             raise ValueError(f"Provided {type_folder} folder already exists with 'mounted' status")
         elif r.status_code == 401:

cloudos_cli-2.62.0/README.md → cloudos_cli-2.64.1/cloudos_cli.egg-info/PKG-INFO

@@ -1,3 +1,38 @@
+Metadata-Version: 2.4
+Name: cloudos_cli
+Version: 2.64.1
+Summary: Python package for interacting with CloudOS
+Home-page: https://github.com/lifebit-ai/cloudos-cli
+Author: David Piñeyro
+Author-email: david.pineyro@lifebit.ai
+Classifier: Programming Language :: Python :: 3
+Classifier: Operating System :: POSIX :: Linux
+Classifier: License :: OSI Approved :: GNU General Public License v3 or later (GPLv3+)
+Requires-Python: >=3.9
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: click>=8.0.1
+Requires-Dist: rich-click>=1.8.2
+Requires-Dist: pandas>=1.3.4
+Requires-Dist: numpy>=1.26.4
+Requires-Dist: requests>=2.26.0
+Provides-Extra: test
+Requires-Dist: pytest; extra == "test"
+Requires-Dist: mock; extra == "test"
+Requires-Dist: responses; extra == "test"
+Requires-Dist: requests_mock; extra == "test"
+Dynamic: author
+Dynamic: author-email
+Dynamic: classifier
+Dynamic: description
+Dynamic: description-content-type
+Dynamic: home-page
+Dynamic: license-file
+Dynamic: provides-extra
+Dynamic: requires-dist
+Dynamic: requires-python
+Dynamic: summary
+
 # cloudos-cli
 
 [![CI_tests](https://github.com/lifebit-ai/cloudos-cli/actions/workflows/ci.yml/badge.svg)](https://github.com/lifebit-ai/cloudos-cli/actions/workflows/ci.yml)
@@ -608,6 +643,12 @@ Nextflow standard output: s3://path/to/location/of/logs/stdout.txt
 Trace file: s3://path/to/location/of/logs/trace.txt
 ```
 
+You can also link the logs directory to an interactive session using the `--link` flag. This will mount the entire logs directory, providing access to all log files in your interactive session:
+
+```bash
+cloudos job logs --profile my_profile --job-id "12345678910" --link --session-id your_session_id
+```
+
 #### Get Job Results
 
 The following command allows you to get the path where CloudOS stores the output files for a job. This can be used only on your user's jobs and for jobs with "completed" status.
@@ -621,6 +662,12 @@ Executing results...
 results: s3://path/to/location/of/results/results/
 ```
 
+You can also link all result directories to an interactive session using the `--link` flag. This will mount all result directories from the job, providing direct access to output files in your interactive session:
+
+```bash
+cloudos job results --profile my_profile --job-id "12345678910" --link --session-id your_session_id
+```
+
 
 #### Clone or Resume job
 
@@ -873,6 +920,15 @@ Finding working directory path...
 Working directory for job 68747bac9e7fe38ec6e022ad: az://123456789000.blob.core.windows.net/cloudos-987652349087/projects/455654676/jobs/54678856765/work
 ```
 
+You can also link the working directory to an interactive session using the `--link` flag. This requires specifying a session ID either through the `--session-id` option or from a configured profile:
+
+```shell
+cloudos job workdir \
+    --profile profile-name \
+    --job-id 62c83a1191fe06013b7ef355 \
+    --link --session-id your_session_id
+```
+
 #### List Jobs
 
 You can get a summary of workspace jobs in two different formats:
@@ -1352,7 +1408,7 @@ The output of this command is a list of files and folders present in the specifi
 > If the `<path>` is left empty, the command will return the list of folders present in the selected project.
 
 If you require more information on the files and folder listed, you can use the `--details` flag that will output a table containing the following columns:
-- Type (folder or file)
+- Type (s3 folder, azure folder , virtual folder, file (user uploaded) or file (virtual copy))
 - Owner
 - Size (in human readable format)
 - Last updated
@@ -1741,7 +1797,7 @@ j_id = j.send_job(job_config)
 To check the status:
 
 ```python
-j_status = j.get_job_status(j_id)
+j_status = j.get_job_status(j_id, workspace_id)
 j_status_h = json.loads(j_status.content)["status"]
 print(j_status_h)
 ```
@@ -1816,7 +1872,7 @@ print(c_status_h)
 j = jb.Job(cloudos_url, apikey, None, workspace_id, project_name, workflow_name, True, mainfile,
            importsfile)
 j_id = j.send_job(job_config, workflow_type='wdl', cromwell_id=json.loads(c_status.content)["_id"])
-j_status = j.get_job_status(j_id)
+j_status = j.get_job_status(j_id, workspace_id)
 j_status_h = json.loads(j_status.content)["status"]
 print(j_status_h)
 

cloudos_cli-2.62.0/cloudos_cli/_version.py

@@ -1 +0,0 @@
-__version__ = '2.62.0'