cloudos-cli 2.50.0__tar.gz → 2.53.0__tar.gz

{cloudos_cli-2.50.0 → cloudos_cli-2.53.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.50.0
+Version: 2.53.0
 Summary: Python package for interacting with CloudOS
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -615,6 +615,52 @@ Aborting jobs...
 ```
 
 
+#### Clone a job with optional parameter overrides
+
+The `clone` command allows you to create a new job based on an existing job's configuration, with the ability to override specific parameters. This is useful for re-running jobs with slight modifications without having to specify all parameters from scratch.
+
+Basic usage:
+```console
+cloudos job clone \
+    --profile MY_PROFILE
+    --job-id "60a7b8c9d0e1f2g3h4i5j6k7"
+```
+
+Clone with parameter overrides:
+```console
+cloudos job clone \
+    --profile MY_PROFILE
+    --job-id "60a7b8c9d0e1f2g3h4i5j6k7" \
+    --job-queue "high-priority-queue" \
+    --cost-limit 50.0 \
+    --instance-type "c5.2xlarge" \
+    --job-name "cloned_analysis_v2" \
+    --nextflow-version "24.04.4" \
+    --git-branch "dev" \
+    --nextflow-profile "production" \
+    --do-not-save-logs true \
+    --accelerate-file-staging true \
+    --workflow-name "updated-workflow" \
+    -p "input=s3://new-bucket/input.csv" \
+    -p "output_dir=s3://new-bucket/results"
+```
+
+Available override options:
+- `--job-queue`: Specify a different job queue
+- `--cost-limit`: Set a new cost limit (use -1 for no limit)
+- `--instance-type`: Change the master instance type
+- `--job-name`: Assign a custom name to the cloned job
+- `--nextflow-version`: Use a different Nextflow version
+- `--git-branch`: Switch to a different git branch
+- `--nextflow-profile`: Change the Nextflow profile
+- `--do-not-save-logs`: Enable/disable log saving
+- `--accelerate-file-staging`: Enable/disable fusion filesystem
+- `--workflow-name`: Use a different workflow
+- `-p, --parameter`: Override or add parameters (can be used multiple times)
+
+> [!NOTE]
+> Parameters can be overridden or new ones can be added using `-p` option
+
 #### Executor support
 
 CloudOS supports [AWS batch](https://www.nextflow.io/docs/latest/executor.html?highlight=executors#aws-batch) executor by default.

{cloudos_cli-2.50.0 → cloudos_cli-2.53.0}/README.md

@@ -580,6 +580,52 @@ Aborting jobs...
 ```
 
 
+#### Clone a job with optional parameter overrides
+
+The `clone` command allows you to create a new job based on an existing job's configuration, with the ability to override specific parameters. This is useful for re-running jobs with slight modifications without having to specify all parameters from scratch.
+
+Basic usage:
+```console
+cloudos job clone \
+    --profile MY_PROFILE
+    --job-id "60a7b8c9d0e1f2g3h4i5j6k7"
+```
+
+Clone with parameter overrides:
+```console
+cloudos job clone \
+    --profile MY_PROFILE
+    --job-id "60a7b8c9d0e1f2g3h4i5j6k7" \
+    --job-queue "high-priority-queue" \
+    --cost-limit 50.0 \
+    --instance-type "c5.2xlarge" \
+    --job-name "cloned_analysis_v2" \
+    --nextflow-version "24.04.4" \
+    --git-branch "dev" \
+    --nextflow-profile "production" \
+    --do-not-save-logs true \
+    --accelerate-file-staging true \
+    --workflow-name "updated-workflow" \
+    -p "input=s3://new-bucket/input.csv" \
+    -p "output_dir=s3://new-bucket/results"
+```
+
+Available override options:
+- `--job-queue`: Specify a different job queue
+- `--cost-limit`: Set a new cost limit (use -1 for no limit)
+- `--instance-type`: Change the master instance type
+- `--job-name`: Assign a custom name to the cloned job
+- `--nextflow-version`: Use a different Nextflow version
+- `--git-branch`: Switch to a different git branch
+- `--nextflow-profile`: Change the Nextflow profile
+- `--do-not-save-logs`: Enable/disable log saving
+- `--accelerate-file-staging`: Enable/disable fusion filesystem
+- `--workflow-name`: Use a different workflow
+- `-p, --parameter`: Override or add parameters (can be used multiple times)
+
+> [!NOTE]
+> Parameters can be overridden or new ones can be added using `-p` option
+
 #### Executor support
 
 CloudOS supports [AWS batch](https://www.nextflow.io/docs/latest/executor.html?highlight=executors#aws-batch) executor by default.

{cloudos_cli-2.50.0 → cloudos_cli-2.53.0}/cloudos_cli/__main__.py

@@ -101,7 +101,8 @@ def run_cloudos_cli(ctx, debug):
                 'list': shared_config,
                 'logs': shared_config,
                 'results': shared_config,
-                'details': shared_config
+                'details': shared_config,
+                'clone': shared_config
             },
             'workflow': {
                 'list': shared_config,
@@ -162,7 +163,8 @@ def run_cloudos_cli(ctx, debug):
                 'list': shared_config,
                 'logs': shared_config,
                 'results': shared_config,
-                'details': shared_config
+                'details': shared_config,
+                'clone': shared_config
             },
             'workflow': {
                 'list': shared_config,
@@ -1225,6 +1227,24 @@ def job_details(ctx,
 @click.option('--archived',
               help=('When this flag is used, only archived jobs list is collected.'),
               is_flag=True)
+@click.option('--filter-status',
+              help='Filter jobs by status (e.g., completed, running, failed, aborted).')
+@click.option('--filter-job-name',
+              help='Filter jobs by job name ( case insensitive ).')
+@click.option('--filter-project',
+              help='Filter jobs by project name.')
+@click.option('--filter-workflow',
+              help='Filter jobs by workflow/pipeline name.')
+@click.option('--last',
+              help=('When workflows are duplicated, use the latest imported workflow (by date).'),
+              is_flag=True)
+@click.option('--filter-job-id',
+              help='Filter jobs by specific job ID.')
+@click.option('--filter-only-mine',
+              help='Filter to show only jobs belonging to the current user.',
+              is_flag=True)
+@click.option('--filter-queue',
+              help='Filter jobs by queue name. Only applies to jobs running in batch environment. Non-batch jobs are preserved in results.')
 @click.option('--verbose',
               help='Whether to print information messages or not.',
               is_flag=True)
@@ -1246,6 +1266,15 @@ def list_jobs(ctx,
               last_n_jobs,
               page,
               archived,
+              filter_status,
+              filter_job_name,
+              filter_project,
+              filter_workflow,
+              last,
+              filter_job_id,
+              filter_only_mine,
+              #filter_owner,
+              filter_queue,
               verbose,
               disable_ssl_verification,
               ssl_cert,
@@ -1300,7 +1329,15 @@ def list_jobs(ctx,
             print("[ERROR] last-n-jobs value was not valid. Please use a positive int or 'all'")
             raise
 
-    my_jobs_r = cl.get_job_list(workspace_id, last_n_jobs, page, archived, verify_ssl)
+    my_jobs_r = cl.get_job_list(workspace_id, last_n_jobs, page, archived, verify_ssl,
+                                filter_status=filter_status,
+                                filter_job_name=filter_job_name,
+                                filter_project=filter_project,
+                                filter_workflow=filter_workflow,
+                                filter_job_id=filter_job_id,
+                                filter_only_mine=filter_only_mine,
+                                filter_queue=filter_queue,
+                                last=last)
     if len(my_jobs_r) == 0:
         if ctx.get_parameter_source('page') == click.core.ParameterSource.DEFAULT:
             print('\t[Message] A total of 0 jobs collected. This is likely because your workspace ' +
@@ -1419,6 +1456,177 @@ def abort_jobs(ctx,
             print(f"\tJob '{job}' aborted successfully.")
 
 
+@job.command('clone')
+@click.option('-k',
+              '--apikey',
+              help='Your CloudOS API key',
+              required=True)
+@click.option('-c',
+              '--cloudos-url',
+              help=(f'The CloudOS url you are trying to access to. Default={CLOUDOS_URL}.'),
+              default=CLOUDOS_URL,
+              required=True)
+@click.option('--workspace-id',
+              help='The specific CloudOS workspace id.',
+              required=True)
+@click.option('--project-name',
+              help='The name of a CloudOS project.')
+@click.option('-p',
+              '--parameter',
+              multiple=True,
+              help=('A single parameter to pass to the job call. It should be in the ' +
+                    'following form: parameter_name=parameter_value. E.g.: ' +
+                    '-p input=s3://path_to_my_file. You can use this option as many ' +
+                    'times as parameters you want to include.'))
+@click.option('--nextflow-profile',
+              help=('A comma separated string indicating the nextflow profile/s ' +
+                    'to use with your job.'))
+@click.option('--nextflow-version',
+              help=('Nextflow version to use when executing the workflow in CloudOS. ' +
+                    'Default=22.10.8.'),
+              type=click.Choice(['22.10.8', '24.04.4', '22.11.1-edge', 'latest']))
+@click.option('--git-branch',
+              help=('The branch to run for the selected pipeline. ' +
+                    'If not specified it defaults to the last commit ' +
+                    'of the default branch.'))
+@click.option('--job-name',
+              help='The name of the job. If not set, will take the name of the cloned job.')
+@click.option('--do-not-save-logs',
+              help=('Avoids process log saving. If you select this option, your job process ' +
+                    'logs will not be stored.'),
+              is_flag=True)
+@click.option('--job-queue',
+              help=('Name of the job queue to use with a batch job. ' +
+                   'In Azure workspaces, this option is ignored.'))
+@click.option('--instance-type',
+              help=('The type of compute instance to use as master node. ' +
+                    'Default=c5.xlarge(aws)|Standard_D4as_v4(azure).'))
+@click.option('--cost-limit',
+              help='Add a cost limit to your job. Default=30.0 (For no cost limit please use -1).',
+              type=float)
+@click.option('--job-id',
+              help='The CloudOS job id of the job to be cloned.',
+              required=True)
+@click.option('--accelerate-file-staging',
+              help='Enables AWS S3 mountpoint for quicker file staging.',
+              is_flag=True)
+@click.option('--resumable',
+              help='Whether to make the job able to be resumed or not.',
+              is_flag=True)
+@click.option('--verbose',
+              help='Whether to print information messages or not.',
+              is_flag=True)
+@click.option('--disable-ssl-verification',
+              help=('Disable SSL certificate verification. Please, remember that this option is ' +
+                    'not generally recommended for security reasons.'),
+              is_flag=True)
+@click.option('--ssl-cert',
+              help='Path to your SSL certificate file.')
+@click.option('--profile',
+              help='Profile to use from the config file',
+              default=None)
+@click.pass_context
+def clone_job(ctx,
+              apikey,
+              cloudos_url,
+              workspace_id,
+              project_name,
+              parameter,
+              nextflow_profile,
+              nextflow_version,
+              git_branch,
+              job_name,
+              do_not_save_logs,
+              job_queue,
+              instance_type,
+              cost_limit,
+              job_id,
+              accelerate_file_staging,
+              resumable,
+              verbose,
+              disable_ssl_verification,
+              ssl_cert,
+              profile):
+    """Clone an existing job with optional parameter overrides."""
+    profile = profile or ctx.default_map['job']['clone']['profile']
+
+    # Create a dictionary with required and non-required params
+    required_dict = {
+        'apikey': True,
+        'workspace_id': True,
+        'workflow_name': False,
+        'project_name': False,
+        'procurement_id': False
+    }
+
+    # Determine if the user provided all required parameters
+    config_manager = ConfigurationProfile()
+    user_options = (
+        config_manager.load_profile_and_validate_data(
+            ctx,
+            INIT_PROFILE,
+            CLOUDOS_URL,
+            profile=profile,
+            required_dict=required_dict,
+            apikey=apikey,
+            cloudos_url=cloudos_url,
+            workspace_id=workspace_id,
+            project_name=project_name
+        )
+    )
+    apikey = user_options['apikey']
+    cloudos_url = user_options['cloudos_url']
+    workspace_id = user_options['workspace_id']
+
+    verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
+
+    print('Cloning job...')
+    if verbose:
+        print('\t...Preparing objects')
+
+    # Create Job object (set dummy values for project_name and workflow_name, since they come from the cloned job)
+    job_obj = jb.Job(cloudos_url, apikey, None, workspace_id, None, None, workflow_id=1234, project_id="None",
+               mainfile=None, importsfile=None,verify=verify_ssl)
+
+    if verbose:
+        print('\tThe following Job object was created:')
+        print('\t' + str(job_obj) + '\n')
+        print(f'\tCloning job {job_id} in workspace: {workspace_id}')
+
+    try:
+        # Clone the job with provided overrides
+        cloned_job_id = job_obj.clone_job(
+            source_job_id=job_id,
+            queue_name=job_queue,
+            cost_limit=cost_limit,
+            master_instance=instance_type,
+            job_name=job_name,
+            nextflow_version=nextflow_version,
+            branch=git_branch,
+            profile=nextflow_profile,
+            do_not_save_logs=do_not_save_logs,
+            use_fusion=accelerate_file_staging,
+            resumable=resumable,
+            # only when explicitly setting --project-name will be overridden, else using the original project
+            project_name=project_name if ctx.get_parameter_source("project_name") == click.core.ParameterSource.COMMANDLINE else None,
+            parameters=list(parameter) if parameter else None,
+            verify=verify_ssl
+        )
+        if verbose:
+            print(f'\tCloned job ID: {cloned_job_id}')
+
+        print(f"Job successfully cloned. New job ID: {cloned_job_id}")
+
+    except BadRequestException as e:
+        if verbose:
+            print(f'\tError details: {e}')
+        raise ValueError(f"Failed to clone job: {e}")
+    except Exception as e:
+        if verbose:
+            print(f'\tError details: {e}')
+        raise ValueError(f"An error occurred while cloning the job: {e}")
+
+
 @workflow.command('list')
 @click.option('-k',
               '--apikey',

cloudos_cli-2.53.0/cloudos_cli/_version.py

@@ -0,0 +1 @@
+__version__ = '2.53.0'

{cloudos_cli-2.50.0 → cloudos_cli-2.53.0}/cloudos_cli/clos.py

@@ -357,8 +357,13 @@ class Cloudos:
         return r
 
     def get_job_list(self, workspace_id, last_n_jobs=30, page=1, archived=False,
-                     verify=True):
-        """Get jobs from a CloudOS workspace.
+                     verify=True, filter_status=None, filter_job_name=None,
+                     filter_project=None, filter_workflow=None, filter_job_id=None,
+                     filter_only_mine=False, filter_owner=None, filter_queue=None, last=False):
+        """Get jobs from a CloudOS workspace with optional filtering.
+        
+        Fetches jobs page by page, applies all filters after fetching.
+        Stops when enough jobs are collected or no more jobs are available.
 
         Parameters
         ----------
@@ -368,52 +373,170 @@ class Cloudos:
             How many of the last jobs from the user to retrieve. You can specify a
             very large int or 'all' to get all user's jobs.
         page : int
-            Response page to get.
+            Response page to get (ignored when using filters - starts from page 1).
         archived : bool
             When True, only the archived jobs are retrieved.
         verify: [bool|string]
             Whether to use SSL verification or not. Alternatively, if
             a string is passed, it will be interpreted as the path to
             the SSL certificate file.
+        filter_status : string, optional
+            Filter jobs by status (e.g., 'completed', 'running', 'failed').
+        filter_job_name : string, optional
+            Filter jobs by name.
+        filter_project : string, optional
+            Filter jobs by project name (will be resolved to project ID).
+        filter_workflow : string, optional
+            Filter jobs by workflow name (will be resolved to workflow ID).
+        filter_job_id : string, optional
+            Filter jobs by specific job ID.
+        filter_only_mine : bool, optional
+            Filter to show only jobs belonging to the current user.
+        filter_queue : string, optional
+            Filter jobs by queue name (will be resolved to queue ID).
+            Only applies to jobs running in batch environment.
+            Non-batch jobs are preserved in results as they don't use queues.
+        last : bool, optional
+            When workflows are duplicated, use the latest imported workflow (by date).
 
         Returns
         -------
         r : list
             A list of dicts, each corresponding to a jobs from the user and the workspace.
         """
+        if not workspace_id or not isinstance(workspace_id, str):
+            raise ValueError("Invalid workspace_id: must be a non-empty string")
+    
+        if last_n_jobs != 'all' and (not isinstance(last_n_jobs, int) or last_n_jobs <= 0):
+            raise ValueError("last_n_jobs must be a positive integer or 'all'")
+        
+        # Validate filter_status values
+        if filter_status:
+            valid_statuses = ['completed', 'running', 'failed', 'aborted', 'queued', 'pending', 'initializing']
+            if filter_status.lower() not in valid_statuses:
+                raise ValueError(f"Invalid filter_status '{filter_status}'. Valid values: {', '.join(valid_statuses)}")
+
         headers = {
             "Content-type": "application/json",
             "apikey": self.apikey
         }
-        if archived:
-            archived_status = "true"
-        else:
-            archived_status = "false"
-        r = retry_requests_get("{}/api/v2/jobs?teamId={}&page={}&archived.status={}".format(
-                               self.cloudos_url, workspace_id, page, archived_status),
-                               headers=headers, verify=verify)
-        if r.status_code >= 400:
-            raise BadRequestException(r)
-        content = json.loads(r.content)
-        n_jobs = len(content['jobs'])
-        if last_n_jobs == 'all':
-            jobs_to_get = n_jobs
-        elif last_n_jobs > 0:
-            jobs_to_get = last_n_jobs - n_jobs
-        else:
-            raise TypeError("[ERROR] Please select an int > 0 or 'all' for 'last_n_jobs'")
-        if jobs_to_get == 0 or n_jobs == 0:
-            return content['jobs']
-        if jobs_to_get > 0:
-            if last_n_jobs == 'all':
-                next_to_get = 'all'
-            else:
-                next_to_get = jobs_to_get
-            return content['jobs'] + self.get_job_list(workspace_id, last_n_jobs=next_to_get,
-                                                       page=page+1, archived=archived,
-                                                       verify=verify)
-        if jobs_to_get < 0:
-            return content['jobs'][:jobs_to_get]
+        
+        # Build query parameters for server-side filtering
+        params = {
+            "teamId": workspace_id,
+            "archived.status": str(archived).lower(),
+            "limit": 100,  # Use a reasonable page size
+            "page": 1     # Always start from page 1
+        }
+        
+        # --- Resolve IDs once (before pagination loop) ---
+        
+        # Add simple server-side filters
+        if filter_status:
+            params["status"] = filter_status.lower()
+        if filter_job_name:
+            params["name"] = filter_job_name
+        if filter_job_id:
+            params["id"] = filter_job_id
+            
+        # Resolve project name to ID
+        if filter_project:
+            try:
+                project_id = self.get_project_id_from_name(workspace_id, filter_project, verify=verify)
+                if project_id:
+                    params["project.id"] = project_id
+                else:
+                    raise ValueError(f"Project '{filter_project}' not found.")
+            except Exception as e:
+                raise ValueError(f"Error resolving project '{filter_project}': {str(e)}")
+        
+        # Resolve workflow name to ID
+        if filter_workflow:
+            try:
+                workflow_content = self.get_workflow_content(workspace_id, filter_workflow, verify=verify, last=last)
+                if workflow_content and workflow_content.get("workflows"):
+                    # Extract the first (and should be only) workflow from the list
+                    workflow = workflow_content["workflows"][0]
+                    workflow_id = workflow.get("_id")
+                    if workflow_id:
+                        params["workflow.id"] = workflow_id
+                    else:
+                        raise ValueError(f"Workflow '{filter_workflow}' not found.")
+                else:
+                    raise ValueError(f"Workflow '{filter_workflow}' not found.")
+            except Exception as e:
+                raise ValueError(f"Error resolving workflow '{filter_workflow}': {str(e)}")
+        
+        # Get current user ID for filter_only_mine
+        if filter_only_mine:
+            try:
+                user_info = self.get_user_info(verify=verify)
+                user_id = user_info.get("id") or user_info.get("_id")
+                if user_id:
+                    params["user.id"] = user_id
+                else:
+                    raise ValueError("Could not retrieve current user information.")
+            except Exception as e:
+                raise ValueError(f"Error getting current user info: {str(e)}")
+        
+
+        # --- Fetch jobs page by page ---
+        all_jobs = []
+        current_page = 1
+        
+        while True:
+            params["page"] = current_page
+            
+            r = retry_requests_get(f"{self.cloudos_url}/api/v2/jobs", params=params, headers=headers, verify=verify)
+            if r.status_code >= 400:
+                raise BadRequestException(r)
+            
+            content = r.json()
+            page_jobs = content.get('jobs', [])
+            
+            # No jobs returned, we've reached the end
+            if not page_jobs:
+                break
+                
+            all_jobs.extend(page_jobs)
+            
+            # Check if we have enough jobs or reached the last page
+            if last_n_jobs != 'all' and len(all_jobs) >= last_n_jobs:
+                break
+            if len(page_jobs) < params["limit"]:
+                break  # Last page (fewer jobs than requested page size)
+                
+            current_page += 1
+
+        # --- Local queue filtering (not supported by API) ---
+        if filter_queue:
+            try:
+                batch_jobs=[job for job in all_jobs if job.get("batch", {})]
+                if batch_jobs:
+                    from cloudos_cli.queue.queue import Queue
+                    queue_api = Queue(self.cloudos_url, self.apikey, self.cromwell_token, workspace_id, verify)
+                    queues = queue_api.get_job_queues()
+                    
+                    queue_id = None
+                    for queue in queues:
+                        if queue.get("label") == filter_queue or queue.get("name") == filter_queue:
+                            queue_id = queue.get("id") or queue.get("_id")
+                            break
+                    
+                    if not queue_id:
+                        raise ValueError(f"Queue with name '{filter_queue}' not found in workspace '{workspace_id}'")
+                    
+                    all_jobs = [job for job in all_jobs if job.get("batch", {}).get("jobQueue", {}).get("id") == queue_id]
+                else:
+                    raise ValueError(f"The environment is not a batch environment so queues do not exist. Please remove the --filter-queue option.")
+            except Exception as e:
+                raise ValueError(f"Error filtering by queue '{filter_queue}': {str(e)}")
+
+        # --- Apply limit after all filtering ---
+        if last_n_jobs != 'all' and isinstance(last_n_jobs, int) and last_n_jobs > 0:
+            all_jobs = all_jobs[:last_n_jobs]
+
+        return all_jobs
 
     @staticmethod
     def process_job_list(r, all_fields=False):
@@ -460,7 +583,13 @@ class Cloudos:
         if all_fields:
             df = df_full
         else:
-            df = df_full.loc[:, COLUMNS]
+            # Only select columns that actually exist in the DataFrame
+            existing_columns = [col for col in COLUMNS if col in df_full.columns]
+            if existing_columns:
+                df = df_full.loc[:, existing_columns]
+            else:
+                # If none of the predefined columns exist, raise missing error
+                raise ValueError(f"None of the predefined columns {COLUMNS} exist in retrieved columns:{list(df_full.columns)}")
         return df
 
     def reorder_job_list(self, my_jobs_df, filename='my_jobs.csv'):
@@ -1199,7 +1328,6 @@ class Cloudos:
             response = retry_requests_get(url, headers=headers, verify=verify)
             # return all content
             content = json.loads(response.content)
-
         if response.status_code >= 400:
             raise BadRequestException(response)
 

{cloudos_cli-2.50.0 → cloudos_cli-2.53.0}/cloudos_cli/jobs/job.py

@@ -921,3 +921,246 @@ class Job(Cloudos):
                 "parameterKind": "textValue",
                 "textValue": f"{rest}"
             }
+
+    def get_job_request_payload(self, job_id, verify=True):
+        """Get the original request payload for a job.
+        
+        Parameters
+        ----------
+        job_id : str
+            The CloudOS job ID to get the payload for.
+        verify : [bool|string]
+            Whether to use SSL verification or not. Alternatively, if
+            a string is passed, it will be interpreted as the path to
+            the SSL certificate file.
+            
+        Returns
+        -------
+        dict
+            The original job request payload.
+        """
+        headers = {
+            "Content-type": "application/json",
+            "apikey": self.apikey
+        }
+        url = f"{self.cloudos_url}/api/v1/jobs/{job_id}/request-payload?teamId={self.workspace_id}"
+        r = retry_requests_get(url, headers=headers, verify=verify)
+        if r.status_code >= 400:
+            raise BadRequestException(r)
+        return json.loads(r.content)
+
+    def update_parameter_value(self, parameters, param_name, new_value):
+        """Update a parameter value in the parameters list.
+        
+        Parameters
+        ----------
+        parameters : list
+            List of parameter dictionaries.
+        param_name : str
+            Name of the parameter to update.
+        new_value : str
+            New value for the parameter.
+            
+        Returns
+        -------
+        bool
+            True if parameter was found and updated, False otherwise.
+        """
+        for param in parameters:
+            if param.get('name') == param_name:
+                # Handle different parameter kinds
+                if param.get('parameterKind') == 'textValue':
+                    param['textValue'] = new_value
+                elif param.get('parameterKind') == 'dataItem':
+                    # For data items, we need to process the value to get file/folder ID
+                    # This is a simplified version - in practice you'd need more logic
+                    if new_value.startswith('s3://') or new_value.startswith('az://'):
+                        param['textValue'] = new_value
+                        param['parameterKind'] = 'textValue'
+                    else:
+                        # Try to process as file/data item
+                        processed_param = self.docker_workflow_param_processing(f"--{param_name}={new_value}", self.project_name)
+                        param.update(processed_param)
+                return True
+        return False
+
+    def clone_job(self, 
+                  source_job_id,
+                  queue_name=None,
+                  cost_limit=None,
+                  master_instance=None,
+                  job_name=None,
+                  nextflow_version=None,
+                  branch=None,
+                  profile=None,
+                  do_not_save_logs=None,
+                  use_fusion=None,
+                  resumable=None,
+                  project_name=None,
+                  parameters=None,
+                  verify=True):
+        """Clone an existing job with optional parameter overrides.
+        
+        Parameters
+        ----------
+        source_job_id : str
+            The CloudOS job ID to clone from.
+        queue_name : str, optional
+            Name of the job queue to use.
+        cost_limit : float, optional
+            Job cost limit override.
+        master_instance : str, optional
+            Master instance type override.
+        job_name : str, optional
+            New job name.
+        nextflow_version : str, optional
+            Nextflow version override.
+        branch : str, optional
+            Git branch override.
+        profile : str, optional
+            Nextflow profile override.
+        do_not_save_logs : bool, optional
+            Whether to save logs override.
+        use_fusion : bool, optional
+            Whether to use fusion filesystem override.
+        resumable : bool, optional
+            Whether to make the job resumable or not.
+        project_name : str, optional
+            Project name override (will look up new project ID).
+        parameters : list, optional
+            List of parameter overrides in format ['param1=value1', 'param2=value2'].
+        verify : [bool|string]
+            Whether to use SSL verification or not. Alternatively, if
+            a string is passed, it will be interpreted as the path to
+            the SSL certificate file.
+            
+        Returns
+        -------
+        str
+            The CloudOS job ID of the cloned job.
+        """
+        # Get the original job payload
+        original_payload = self.get_job_request_payload(source_job_id, verify=verify)
+
+        # Create a copy of the payload for modification
+        cloned_payload = json.loads(json.dumps(original_payload))
+
+        # remove unwanted fields
+        del cloned_payload['_id']
+        del cloned_payload['resourceId']
+
+        # Override job name if provided
+        if job_name:
+            cloned_payload['name'] = job_name
+
+        # Override cost limit if provided
+        if cost_limit is not None:
+            if 'execution' not in cloned_payload:
+                cloned_payload['execution'] = {}
+            cloned_payload['execution']['computeCostLimit'] = cost_limit
+
+        # Override master instance if provided
+        if master_instance:
+            if 'masterInstance' not in cloned_payload:
+                cloned_payload['masterInstance'] = {'requestedInstance': {}}
+            cloned_payload['masterInstance']['requestedInstance']['type'] = master_instance
+
+        # Override nextflow version if provided (only for non-docker workflows)
+        if nextflow_version and 'nextflowVersion' in cloned_payload:
+            cloned_payload['nextflowVersion'] = nextflow_version
+        elif nextflow_version and cloned_payload['executionPlatform'] == 'azure' and\
+            nextflow_version not in ['22.11.1-edge', 'latest']:
+            print("[Message]: Azure workspace only uses Nextflow version 22.11.1-edge, option '--nextflow-version' is ignored.\n")
+
+        # Override branch if provided
+        if branch:
+            if 'revision' not in cloned_payload:
+                cloned_payload['revision'] = {}
+            cloned_payload['revision']['revisionType'] = 'branch'
+            cloned_payload['revision']['branch'] = branch
+            # Clear other revision types
+            cloned_payload['revision'].pop('commit', None)
+            cloned_payload['revision'].pop('tag', None)
+
+        # Override profile if provided
+        if profile:
+            cloned_payload['profile'] = profile
+
+        # Override save logs if provided
+        if do_not_save_logs:
+            cloned_payload['saveProcessLogs'] = do_not_save_logs
+
+        # Override use fusion if provided
+        if use_fusion and cloned_payload['executionPlatform'] != 'azure':
+            cloned_payload['usesFusionFileSystem'] = use_fusion
+        elif use_fusion and cloned_payload['executionPlatform'] == 'azure':
+            print("[Message]: Azure workspace does not use fusion filesystem, option '--accelerate-file-staging' is ignored.\n")
+
+        # Override resumable if provided
+        if resumable:
+            cloned_payload['resumable'] = resumable
+
+        # Handle job queue override
+        if queue_name:
+            if cloned_payload['executionPlatform'] != 'azure':
+                try:
+                    from cloudos_cli.queue.queue import Queue
+                    queue_api = Queue(self.cloudos_url, self.apikey, self.cromwell_token, self.workspace_id, verify)
+                    queues = queue_api.get_job_queues()
+
+                    queue_id = None
+                    for queue in queues:
+                        if queue.get("label") == queue_name or queue.get("name") == queue_name:
+                            queue_id = queue.get("id") or queue.get("_id")
+                            break
+                    cloned_payload['batch']['jobQueue'] = queue_id
+                    if not queue_id:
+                        raise ValueError(f"Queue with name '{queue_name}' not found in workspace '{self.workspace_id}'")
+                except Exception as e:
+                    raise ValueError(f"Error filtering by queue '{queue_name}': {str(e)}")
+            else:
+                print("[Message]: Azure workspace does not use job queues, option '--job-queue' is ignored.\n")
+
+        # Handle parameter overrides
+        if parameters:
+            cloned_parameters = cloned_payload.get('parameters', [])
+            for param_override in parameters:
+                param_name, param_value = param_override.split('=', 1)
+                param_name = param_name.lstrip('-') # Remove leading dashes
+                if not self.update_parameter_value(cloned_parameters, param_name, param_value):
+                    # Parameter not found, add as new parameter
+                    # Determine workflow type to set proper prefix and format
+                    prefix = "--" if param_override.startswith('--') else ("-" if param_override.startswith('-') else "")
+                    new_param = {
+                        "prefix": prefix,
+                        "name": param_name,
+                        "parameterKind": "textValue",
+                        "textValue": param_value
+                    }
+                    cloned_parameters.append(new_param)
+            cloned_payload['parameters'] = cloned_parameters
+
+        # setup project name
+        if project_name:
+            # get project ID
+            project_id = self.get_project_id_from_name(self.workspace_id, project_name, verify=verify)
+            cloned_payload['project'] = project_id
+
+        # Send the cloned job
+        headers = {
+            "Content-type": "application/json",
+            "apikey": self.apikey
+        }
+
+        r = retry_requests_post(f"{self.cloudos_url}/api/v2/jobs?teamId={self.workspace_id}",
+                                data=json.dumps(cloned_payload), 
+                                headers=headers, 
+                                verify=verify)
+
+        if r.status_code >= 400:
+            raise BadRequestException(r)
+
+        j_id = json.loads(r.content)["jobId"]
+        print('\tJob successfully cloned and launched to CloudOS, please check the ' +
+              f"following link: {self.cloudos_url}/app/advanced-analytics/analyses/{j_id}\n")
+        return j_id

{cloudos_cli-2.50.0 → cloudos_cli-2.53.0}/cloudos_cli/procurement/images.py

@@ -113,6 +113,7 @@ class Images(Cloudos):
         payload = {
             "organisationId": organisation_id,
             "imageType": image_type,
+            "imageName": image_name,
             "provider": provider,
             "region": region,
             "imageId": image_id

{cloudos_cli-2.50.0 → cloudos_cli-2.53.0}/cloudos_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.50.0
+Version: 2.53.0
 Summary: Python package for interacting with CloudOS
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -615,6 +615,52 @@ Aborting jobs...
 ```
 
 
+#### Clone a job with optional parameter overrides
+
+The `clone` command allows you to create a new job based on an existing job's configuration, with the ability to override specific parameters. This is useful for re-running jobs with slight modifications without having to specify all parameters from scratch.
+
+Basic usage:
+```console
+cloudos job clone \
+    --profile MY_PROFILE
+    --job-id "60a7b8c9d0e1f2g3h4i5j6k7"
+```
+
+Clone with parameter overrides:
+```console
+cloudos job clone \
+    --profile MY_PROFILE
+    --job-id "60a7b8c9d0e1f2g3h4i5j6k7" \
+    --job-queue "high-priority-queue" \
+    --cost-limit 50.0 \
+    --instance-type "c5.2xlarge" \
+    --job-name "cloned_analysis_v2" \
+    --nextflow-version "24.04.4" \
+    --git-branch "dev" \
+    --nextflow-profile "production" \
+    --do-not-save-logs true \
+    --accelerate-file-staging true \
+    --workflow-name "updated-workflow" \
+    -p "input=s3://new-bucket/input.csv" \
+    -p "output_dir=s3://new-bucket/results"
+```
+
+Available override options:
+- `--job-queue`: Specify a different job queue
+- `--cost-limit`: Set a new cost limit (use -1 for no limit)
+- `--instance-type`: Change the master instance type
+- `--job-name`: Assign a custom name to the cloned job
+- `--nextflow-version`: Use a different Nextflow version
+- `--git-branch`: Switch to a different git branch
+- `--nextflow-profile`: Change the Nextflow profile
+- `--do-not-save-logs`: Enable/disable log saving
+- `--accelerate-file-staging`: Enable/disable fusion filesystem
+- `--workflow-name`: Use a different workflow
+- `-p, --parameter`: Override or add parameters (can be used multiple times)
+
+> [!NOTE]
+> Parameters can be overridden or new ones can be added using `-p` option
+
 #### Executor support
 
 CloudOS supports [AWS batch](https://www.nextflow.io/docs/latest/executor.html?highlight=executors#aws-batch) executor by default.

cloudos_cli-2.50.0/cloudos_cli/_version.py

@@ -1 +0,0 @@
-__version__ = '2.50.0'