cloudos-cli 2.81.2__tar.gz → 2.82.1__tar.gz

{cloudos_cli-2.81.2 → cloudos_cli-2.82.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.81.2
+Version: 2.82.1
 Summary: Python package for interacting with CloudOS
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -365,11 +365,21 @@ Projects in CloudOS provide logical separation of datasets, workflows, and resul
 
 #### List Projects
 
-You can get a summary of all available workspace projects in two different formats:
-- **CSV**: A table with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
-- **JSON**: All available information from projects in JSON format
+You can get a summary of all available workspace projects in three different output formats using the `--output-format` option:
 
-To get a CSV table with all available projects for a given workspace:
+- **stdout** (default): Displays a rich formatted table directly in the terminal with pagination and visual formatting
+- **csv**: Saves project data to a CSV file with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
+- **json**: Saves complete project information to a JSON file with all available fields
+
+To display projects as a formatted table in the terminal:
+
+```bash
+cloudos project list --profile my_profile
+# or explicitly:
+cloudos project list --profile my_profile --output-format stdout
+```
+
+To save projects to a CSV file with all available fields:
 
 ```bash
 cloudos project list --profile my_profile --output-format csv --all-fields
@@ -383,7 +393,7 @@ Executing list...
 	Project list saved to project_list.csv
 ```
 
-To get the same information in JSON format:
+To save the same information in JSON format:
 
 ```bash
 cloudos project list --profile my_profile --output-format json
@@ -406,13 +416,28 @@ The expected output is something similar to:
 
 ### Queue
 
-Job queues are required for running jobs using AWS batch executor. The available job queues in your CloudOS workspace are listed in the "Compute Resources" section in "Settings". You can get a summary of all available workspace job queues in two formats:
-- **CSV**: A table with a selection of the available job queue information. You can get all information using the `--all-fields` flag
-- **JSON**: All available information from job queues in JSON format
+Job queues are required for running jobs using AWS batch executor. The available job queues in your CloudOS workspace are listed in the "Compute Resources" section in "Settings".
+
+> [!NOTE]
+> **Azure Platform**: Queue listing is not available for CloudOS workspaces configured to use Azure execution platform, as Azure does not use AWS batch queues.
 
 #### List Queues
 
-This command allows you to view available computational queues and their configurations. Example command for getting all available job queues in JSON format:
+This command allows you to view available computational queues and their configurations. You can get a summary of all available workspace job queues in three different output formats using the `--output-format` option:
+
+- **stdout** (default): Displays a rich formatted table directly in the terminal with pagination and visual formatting
+- **csv**: Saves queue data to a CSV file with a selection of available queue information, or all information using the `--all-fields` flag
+- **json**: Saves complete queue information to a JSON file with all available fields
+
+To display queues as a formatted table in the terminal:
+
+```bash
+cloudos queue list --profile my_profile
+# or explicitly:
+cloudos queue list --profile my_profile --output-format stdout
+```
+
+To save all available job queues in JSON format:
 
 ```bash
 cloudos queue list --profile my_profile --output-format json --output-basename "available_queues"
@@ -424,7 +449,11 @@ Executing list...
 	Job queue list saved to available_queues.json
 ```
 
-This command will output the list of available job queues in JSON format and save it to a file named `available_queues.json`. You can use `--output-format csv` for a CSV file, or omit `--output-basename` to print to the console.
+To save queue data to a CSV file:
+
+```bash
+cloudos queue list --profile my_profile --output-format csv
+```
 
 > NOTE: The queue name that is visible in CloudOS and must be used with the `--job-queue` parameter is the one in the `label` field.
 
@@ -437,11 +466,21 @@ Platform workflows (those provided by CloudOS in your workspace as modules) run
 
 #### List All Available Workflows
 
-You can get a summary of all available workspace workflows in two different formats:
-- **CSV**: A table with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
-- **JSON**: All available information from workflows in JSON format
+You can get a list of all available workspace workflows in three different output formats using the `--output-format` option:
+
+- **stdout** (default): Displays a rich formatted table directly in the terminal with pagination and visual formatting
+- **csv**: Saves workflow data to a CSV file with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
+- **json**: Saves complete workflow information to a JSON file with all available fields
+
+To display the list of workflows as a formatted table in the terminal:
+
+```bash
+cloudos workflow list --profile my_profile
+# or explicitly:
+cloudos workflow list --profile my_profile --output-format stdout
+```
 
-To get a CSV table with all available workflows for a given workspace:
+To save the list of workflows to a CSV file with all available fields:
 
 ```bash
 cloudos workflow list --profile my_profile --output-format csv --all-fields
@@ -455,7 +494,7 @@ Executing list...
 	Workflow list saved to workflow_list.csv
 ```
 
-To get the same information in JSON format:
+To save the same information in JSON format:
 
 ```bash
 cloudos workflow list --profile my_profile --output-format json

{cloudos_cli-2.81.2 → cloudos_cli-2.82.1}/README.md

@@ -330,11 +330,21 @@ Projects in CloudOS provide logical separation of datasets, workflows, and resul
 
 #### List Projects
 
-You can get a summary of all available workspace projects in two different formats:
-- **CSV**: A table with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
-- **JSON**: All available information from projects in JSON format
+You can get a summary of all available workspace projects in three different output formats using the `--output-format` option:
 
-To get a CSV table with all available projects for a given workspace:
+- **stdout** (default): Displays a rich formatted table directly in the terminal with pagination and visual formatting
+- **csv**: Saves project data to a CSV file with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
+- **json**: Saves complete project information to a JSON file with all available fields
+
+To display projects as a formatted table in the terminal:
+
+```bash
+cloudos project list --profile my_profile
+# or explicitly:
+cloudos project list --profile my_profile --output-format stdout
+```
+
+To save projects to a CSV file with all available fields:
 
 ```bash
 cloudos project list --profile my_profile --output-format csv --all-fields
@@ -348,7 +358,7 @@ Executing list...
 	Project list saved to project_list.csv
 ```
 
-To get the same information in JSON format:
+To save the same information in JSON format:
 
 ```bash
 cloudos project list --profile my_profile --output-format json
@@ -371,13 +381,28 @@ The expected output is something similar to:
 
 ### Queue
 
-Job queues are required for running jobs using AWS batch executor. The available job queues in your CloudOS workspace are listed in the "Compute Resources" section in "Settings". You can get a summary of all available workspace job queues in two formats:
-- **CSV**: A table with a selection of the available job queue information. You can get all information using the `--all-fields` flag
-- **JSON**: All available information from job queues in JSON format
+Job queues are required for running jobs using AWS batch executor. The available job queues in your CloudOS workspace are listed in the "Compute Resources" section in "Settings".
+
+> [!NOTE]
+> **Azure Platform**: Queue listing is not available for CloudOS workspaces configured to use Azure execution platform, as Azure does not use AWS batch queues.
 
 #### List Queues
 
-This command allows you to view available computational queues and their configurations. Example command for getting all available job queues in JSON format:
+This command allows you to view available computational queues and their configurations. You can get a summary of all available workspace job queues in three different output formats using the `--output-format` option:
+
+- **stdout** (default): Displays a rich formatted table directly in the terminal with pagination and visual formatting
+- **csv**: Saves queue data to a CSV file with a selection of available queue information, or all information using the `--all-fields` flag
+- **json**: Saves complete queue information to a JSON file with all available fields
+
+To display queues as a formatted table in the terminal:
+
+```bash
+cloudos queue list --profile my_profile
+# or explicitly:
+cloudos queue list --profile my_profile --output-format stdout
+```
+
+To save all available job queues in JSON format:
 
 ```bash
 cloudos queue list --profile my_profile --output-format json --output-basename "available_queues"
@@ -389,7 +414,11 @@ Executing list...
 	Job queue list saved to available_queues.json
 ```
 
-This command will output the list of available job queues in JSON format and save it to a file named `available_queues.json`. You can use `--output-format csv` for a CSV file, or omit `--output-basename` to print to the console.
+To save queue data to a CSV file:
+
+```bash
+cloudos queue list --profile my_profile --output-format csv
+```
 
 > NOTE: The queue name that is visible in CloudOS and must be used with the `--job-queue` parameter is the one in the `label` field.
 
@@ -402,11 +431,21 @@ Platform workflows (those provided by CloudOS in your workspace as modules) run
 
 #### List All Available Workflows
 
-You can get a summary of all available workspace workflows in two different formats:
-- **CSV**: A table with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
-- **JSON**: All available information from workflows in JSON format
+You can get a list of all available workspace workflows in three different output formats using the `--output-format` option:
+
+- **stdout** (default): Displays a rich formatted table directly in the terminal with pagination and visual formatting
+- **csv**: Saves workflow data to a CSV file with a minimum predefined set of columns by default, or all available columns using the `--all-fields` parameter
+- **json**: Saves complete workflow information to a JSON file with all available fields
+
+To display the list of workflows as a formatted table in the terminal:
+
+```bash
+cloudos workflow list --profile my_profile
+# or explicitly:
+cloudos workflow list --profile my_profile --output-format stdout
+```
 
-To get a CSV table with all available workflows for a given workspace:
+To save the list of workflows to a CSV file with all available fields:
 
 ```bash
 cloudos workflow list --profile my_profile --output-format csv --all-fields
@@ -420,7 +459,7 @@ Executing list...
 	Workflow list saved to workflow_list.csv
 ```
 
-To get the same information in JSON format:
+To save the same information in JSON format:
 
 ```bash
 cloudos workflow list --profile my_profile --output-format json

cloudos_cli-2.82.1/cloudos_cli/_version.py

@@ -0,0 +1 @@
+__version__ = '2.82.1'

{cloudos_cli-2.81.2 → cloudos_cli-2.82.1}/cloudos_cli/clos.py

@@ -1179,6 +1179,24 @@ class Cloudos:
             user_id = self.resolve_user_id(filter_owner, workspace_id, verify)
             params["user.id"] = user_id
 
+        # --- Resolve queue ID before pagination (for local filtering during pagination) ---
+        queue_id = None
+        if filter_queue:
+            try:
+                from cloudos_cli.queue.queue import Queue
+                queue_api = Queue(self.cloudos_url, self.apikey, self.cromwell_token, workspace_id, verify)
+                queues = queue_api.get_job_queues()
+
+                for queue in queues:
+                    if queue.get("label") == filter_queue or queue.get("name") == filter_queue:
+                        queue_id = queue.get("id") or queue.get("_id")
+                        break
+
+                if not queue_id:
+                    raise ValueError(f"Queue with name '{filter_queue}' not found in workspace '{workspace_id}'")
+            except Exception as e:
+                raise ValueError(f"Error resolving queue '{filter_queue}'. {str(e)}")
+
         # --- Fetch jobs page by page ---
         all_jobs = []
         params["limit"] = current_page_size
@@ -1201,47 +1219,40 @@ class Cloudos:
             if not page_jobs:
                 break
 
+            # Apply queue filter during pagination (if specified)
+            # jobQueue is a dict with "id" and "name" keys, extract the id for comparison
+            if filter_queue and queue_id:
+                filtered_jobs = []
+                for job in page_jobs:
+                    job_queue = job.get("batch", {}).get("jobQueue", {})
+                    # jobQueue is a dict like {"id": "...", "name": "...", ...}
+                    job_queue_id = job_queue.get("id") if isinstance(job_queue, dict) else job_queue
+                    if job_queue_id == queue_id:
+                        filtered_jobs.append(job)
+                page_jobs = filtered_jobs
+
             all_jobs.extend(page_jobs)
 
             # Check stopping conditions based on mode
             if use_pagination_mode:
-                # In pagination mode (last_n_jobs), continue until we have enough jobs
+                # In pagination mode (last_n_jobs), continue until we have enough jobs (after filtering)
                 if target_job_count != 'all' and len(all_jobs) >= target_job_count:
                     break
             else:
-                # In direct mode (page/page_size), only get one page
-                break
+                # In direct mode (page/page_size), only get one page unless filter_queue is used
+                # When filter_queue is used, continue fetching pages until we have enough filtered results
+                if not filter_queue or len(all_jobs) >= current_page_size:
+                    break
 
             # Check if we reached the last page (fewer jobs than requested page size)
-            if len(page_jobs) < params["limit"]:
+            # Note: For queue filter, we check the unfiltered page_jobs count from the API
+            # This ensures we stop when the API has exhausted results
+            raw_page_jobs = content.get('jobs', [])
+            if len(raw_page_jobs) < params["limit"]:
                 break  # Last page
 
             current_page += 1
 
-        # --- Local queue filtering (not supported by API) ---
-        if filter_queue:
-            try:
-                batch_jobs=[job for job in all_jobs if job.get("batch", {})]
-                if batch_jobs:
-                    from cloudos_cli.queue.queue import Queue
-                    queue_api = Queue(self.cloudos_url, self.apikey, self.cromwell_token, workspace_id, verify)
-                    queues = queue_api.get_job_queues()
-
-                    queue_id = None
-                    for queue in queues:
-                        if queue.get("label") == filter_queue or queue.get("name") == filter_queue:
-                            queue_id = queue.get("id") or queue.get("_id")
-                            break
-
-                    if not queue_id:
-                        raise ValueError(f"Queue with name '{filter_queue}' not found in workspace '{workspace_id}'")
-
-                    all_jobs = [job for job in all_jobs if job.get("batch", {}).get("jobQueue", {}) == queue_id]
-                else:
-                    raise ValueError(f"The environment is not a batch environment so queues do not exist. Please remove the --filter-queue option.")
-            except Exception as e:
-                raise ValueError(f"Error filtering by queue '{filter_queue}'. {str(e)}")
-
         # --- Apply limit after all filtering ---
         if use_pagination_mode and target_job_count != 'all' and isinstance(target_job_count, int) and target_job_count > 0:
             all_jobs = all_jobs[:target_job_count]

{cloudos_cli-2.81.2 → cloudos_cli-2.82.1}/cloudos_cli/projects/cli.py

@@ -7,6 +7,7 @@ from cloudos_cli.clos import Cloudos
 from cloudos_cli.utils.resources import ssl_selector
 from cloudos_cli.configure.configure import with_profile_config, CLOUDOS_URL
 from cloudos_cli.utils.cli_helpers import pass_debug_to_subcommands
+from cloudos_cli.utils.details import create_project_list_table
 
 
 @click.group(cls=pass_debug_to_subcommands())
@@ -34,9 +35,13 @@ def project():
               default='project_list',
               required=False)
 @click.option('--output-format',
-              help='The desired file format (file extension) for the output. Default=csv.',
-              type=click.Choice(['csv', 'json'], case_sensitive=False),
-              default='csv')
+              help=('Output format for project list. Options: '
+                    'stdout (display as interactive table in terminal), '
+                    'csv (save as comma-separated values file), '
+                    'json (save as JSON file with full API response). '
+                    'Default=stdout.'),
+              type=click.Choice(['stdout', 'csv', 'json'], case_sensitive=False),
+              default='stdout')
 @click.option('--all-fields',
               help=('Whether to collect all available fields from projects or ' +
                     'just the preconfigured selected fields. Only applicable ' +
@@ -70,11 +75,12 @@ def list_projects(ctx,
                   disable_ssl_verification,
                   ssl_cert,
                   profile):
-    """Collect all projects from a CloudOS workspace in CSV format."""
+    """Collect and display all projects from a CloudOS workspace."""
     # apikey, cloudos_url, and workspace_id are now automatically resolved by the decorator
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
-    outfile = output_basename + '.' + output_format
+    if output_format != 'stdout':
+        outfile = output_basename + '.' + output_format
     print('Executing list...')
     if verbose:
         print('\t...Preparing objects')
@@ -93,25 +99,32 @@ def list_projects(ctx,
         if not isinstance(page, int) or page < 1:
             raise ValueError('Please, use a positive integer (>= 1) for the --page parameter')
     my_projects_r = cl.get_project_list(workspace_id, verify_ssl, page=page, get_all=get_all)
+    
+    # Print informative message if no projects found
     if len(my_projects_r) == 0:
         if ctx.get_parameter_source('page') == click.core.ParameterSource.DEFAULT:
-            print('A total of 0 projects collected. This is likely because your workspace ' +
+            print('\tA total of 0 projects collected. This is likely because your workspace ' +
                   'has no projects created yet.')
         else:
-            print('A total of 0 projects collected. This is likely because the --page you ' +
+            print('\tA total of 0 projects collected. This is likely because the --page you ' +
                   'requested does not exist. Please, try a smaller number for --page or collect all the ' +
                   'projects by not using --page parameter.')
+    
+    # Generate output files (even if empty, for consistency with automation)
+    if output_format == 'stdout':
+        create_project_list_table(my_projects_r, cloudos_url)
     elif output_format == 'csv':
         my_projects = cl.process_project_list(my_projects_r, all_fields)
         my_projects.to_csv(outfile, index=False)
         print(f'\tProject list collected with a total of {my_projects.shape[0]} projects.')
+        print(f'\tProject list saved to {outfile}')
     elif output_format == 'json':
         with open(outfile, 'w') as o:
             o.write(json.dumps(my_projects_r))
         print(f'\tProject list collected with a total of {len(my_projects_r)} projects.')
+        print(f'\tProject list saved to {outfile}')
     else:
-        raise ValueError('Unrecognised output format. Please use one of [csv|json]')
-    print(f'\tProject list saved to {outfile}')
+        raise ValueError('Unrecognised output format. Please use one of [stdout|csv|json]')
 
 
 @project.command('create')

{cloudos_cli-2.81.2 → cloudos_cli-2.82.1}/cloudos_cli/queue/cli.py

@@ -6,6 +6,7 @@ from cloudos_cli.queue.queue import Queue
 from cloudos_cli.utils.resources import ssl_selector
 from cloudos_cli.configure.configure import with_profile_config, CLOUDOS_URL
 from cloudos_cli.utils.cli_helpers import pass_debug_to_subcommands
+from cloudos_cli.utils.details import create_queue_list_table
 
 
 # Create the queue group
@@ -34,11 +35,15 @@ def queue():
               default='job_queue_list',
               required=False)
 @click.option('--output-format',
-              help='The desired file format (file extension) for the output. Default=csv.',
-              type=click.Choice(['csv', 'json'], case_sensitive=False),
-              default='csv')
+              help=('Output format for queue list. Options: '
+                    'stdout (display as interactive table in terminal), '
+                    'csv (save as comma-separated values file), '
+                    'json (save as JSON file with full API response). '
+                    'Default=stdout.'),
+              type=click.Choice(['stdout', 'csv', 'json'], case_sensitive=False),
+              default='stdout')
 @click.option('--all-fields',
-              help=('Whether to collect all available fields from workflows or ' +
+              help=('Whether to collect all available fields from queues or ' +
                     'just the preconfigured selected fields. Only applicable ' +
                     'when --output-format=csv'),
               is_flag=True)
@@ -61,24 +66,26 @@ def list_queues(ctx,
                 disable_ssl_verification,
                 ssl_cert,
                 profile):
-    """Collect all available job queues from a CloudOS workspace."""
+    """Collect and display all available job queues from a CloudOS workspace."""
     # apikey, cloudos_url, and workspace_id are now automatically resolved by the decorator
 
     verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
-    outfile = output_basename + '.' + output_format
     print('Executing list...')
     j_queue = Queue(cloudos_url, apikey, None, workspace_id, verify=verify_ssl)
     my_queues = j_queue.get_job_queues()
     if len(my_queues) == 0:
-        raise ValueError('No AWS batch queues found. Please, make sure that your CloudOS supports AWS bath queues')
-    if output_format == 'csv':
+        raise ValueError('No AWS batch queues found. Please, make sure that your CloudOS supports AWS batch queues')
+    if output_format == 'stdout':
+        create_queue_list_table(my_queues, cloudos_url)
+    elif output_format == 'csv':
+        outfile = output_basename + '.' + output_format
         queues_processed = j_queue.process_queue_list(my_queues, all_fields)
         queues_processed.to_csv(outfile, index=False)
         print(f'\tJob queue list collected with a total of {queues_processed.shape[0]} queues.')
+        print(f'\tJob queue list saved to {outfile}')
     elif output_format == 'json':
+        outfile = output_basename + '.' + output_format
         with open(outfile, 'w') as o:
             o.write(json.dumps(my_queues))
         print(f'\tJob queue list collected with a total of {len(my_queues)} queues.')
-    else:
-        raise ValueError('Unrecognised output format. Please use one of [csv|json]')
-    print(f'\tJob queue list saved to {outfile}')
+        print(f'\tJob queue list saved to {outfile}')