cloudos-cli 2.94.1__tar.gz → 2.95.1__tar.gz

{cloudos_cli-2.94.1 → cloudos_cli-2.95.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.94.1
+Version: 2.95.1
 Summary: Python package for interacting with the Lifebit Platform
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -609,7 +609,7 @@ First, configure your local environment to ease parameter input. We will try to
 cloudos job run --profile my_profile --workflow-name rnatoy --job-config cloudos_cli/examples/rnatoy.config --resumable
 ```
 
-As you can see, a file with the job parameters is used to configure the job. This file could be a regular `nextflow.config` file or any file with the following structure:
+As you can see, a file with the job parameters is used to configure the job. This file is a list of the parameters to pass to your job, and nothing else. It has the following structure:
 
 ```
 params {
@@ -618,6 +618,24 @@ params {
 }
 ```
 
+`--job-config` is shorthand for repeating `--parameter` on the command line: the CLI reads the file line by line, splits each line on the first `=`, and sends each result as one named parameter.
+
+Every value is passed through as text, exactly as it would be when launching Nextflow from the command line.
+
+> **IMPORTANT: `[ ]` is not expanded into a Nextflow list.** In a real `nextflow.config`, `my_param = ['a', 'b']` gives the pipeline a list of two items. Through `--job-config` it gives the pipeline the *string* `[a,b]`. The line is accepted and the job is submitted, so this fails silently at the point where your pipeline tries to iterate over the value. The CLI prints a warning when it sees a value that looks like a list, showing exactly what the pipeline will receive.
+
+To pass a genuine list, define it in the pipeline's own `nextflow.config` in the git repository, or use `--params-file` with a JSON or YAML file (see below), which preserves types.
+
+What a `--job-config` file cannot contain, because it is **not** a `nextflow.config` file:
+
+- values spanning several lines
+- nested blocks (e.g. `genomes { ... }`)
+- `process`, `profiles`, `docker` or other config sections
+
+Comments are supported, both on their own line and after a value, using `//` or `#`.
+
+If a parameter needs a genuinely structured value rather than a string, either define it in the pipeline's own `nextflow.config` in the git repository and expose the parts you want to change as individual parameters, or pass it with `--params-file` (see below), which supports JSON and YAML with full type fidelity.
+
 In addition, parameters can also be specified using the command-line `-p` or `--parameter`. For instance:
 
 ```bash
@@ -2648,7 +2666,7 @@ All configuration parameters are optional. If not specified, the session resumes
 
 ### Link
 
-The `cloudos interactive-session link` command provides a unified interface for linking files and folders to interactive analysis sessions. It consolidates functionality previously available through separate commands (`cloudos job results --link`, `cloudos job workdir --link`, `cloudos job logs --link`, and `cloudos datasets link`) into a single, intuitive interface.
+The `cloudos interactive-session link` command provides a unified interface for linking files and folders to interactive analysis sessions. It consolidates functionality previously available through separate commands (`cloudos job results --link`, `cloudos job workdir --link`, and `cloudos job logs --link`) into a single, intuitive interface.
 
 #### Link Files and Folders to Interactive Analysis
 
@@ -2880,41 +2898,8 @@ cloudos datasets cp AnalysesResults/my_analysis/results/my_plot.png Data/plots
 ```
 
 
-#### Link Files and Folders to Interactive Analysis
-
-Connect external S3 buckets, S3 files, or File Explorer files/folders to your interactive analysis sessions. This provides direct access to data without needing to copy files.
-
-This subcommand uses the `--session-id` option to access the correct interactive session. This option can be added to the CLI or defined in a profile, for convenience.
-
-```bash
-cloudos datasets link <PATH> --profile <profile> --session-id <SESSION_ID>
-```
-
-Link an S3 folder:
-```console
-cloudos datasets link s3://bucket/path/folder --profile test --session-id 1234
-```
-
-Link an S3 file:
-```console
-cloudos datasets link s3://bucket/path/data.csv --profile test --session-id 1234
-```
-
-Link a File Explorer folder (requires `--project-name`):
-```console
-cloudos datasets link "Data/HLA" --project-name my-project --session-id 1234
-```
-
-Link a File Explorer file (requires `--project-name`):
-```console
-cloudos datasets link "Data/observations.csv" --project-name my-project --session-id 1234
-```
-
-> [!NOTE]
-> If running the CLI inside a jupyter session, the pre-configured CLI installation will have the session ID already installed and only the `--apikey` needs to be added.
-
 > [!NOTE]
-> Virtual folders in File Explorer (folders created in File Explorer that are not actual storage locations) cannot be linked.
+> To link files or folders (S3 or File Explorer) to an interactive analysis session, use `cloudos interactive-session link` or the `--link` option of `cloudos interactive-session create`. Virtual folders in File Explorer cannot be linked.
 
 > [!NOTE]
 > A maximum of 100 items can be linked per session. If the new items combined with already-linked items exceed this limit, the entire request is rejected.

{cloudos_cli-2.94.1 → cloudos_cli-2.95.1}/README.md

@@ -574,7 +574,7 @@ First, configure your local environment to ease parameter input. We will try to
 cloudos job run --profile my_profile --workflow-name rnatoy --job-config cloudos_cli/examples/rnatoy.config --resumable
 ```
 
-As you can see, a file with the job parameters is used to configure the job. This file could be a regular `nextflow.config` file or any file with the following structure:
+As you can see, a file with the job parameters is used to configure the job. This file is a list of the parameters to pass to your job, and nothing else. It has the following structure:
 
 ```
 params {
@@ -583,6 +583,24 @@ params {
 }
 ```
 
+`--job-config` is shorthand for repeating `--parameter` on the command line: the CLI reads the file line by line, splits each line on the first `=`, and sends each result as one named parameter.
+
+Every value is passed through as text, exactly as it would be when launching Nextflow from the command line.
+
+> **IMPORTANT: `[ ]` is not expanded into a Nextflow list.** In a real `nextflow.config`, `my_param = ['a', 'b']` gives the pipeline a list of two items. Through `--job-config` it gives the pipeline the *string* `[a,b]`. The line is accepted and the job is submitted, so this fails silently at the point where your pipeline tries to iterate over the value. The CLI prints a warning when it sees a value that looks like a list, showing exactly what the pipeline will receive.
+
+To pass a genuine list, define it in the pipeline's own `nextflow.config` in the git repository, or use `--params-file` with a JSON or YAML file (see below), which preserves types.
+
+What a `--job-config` file cannot contain, because it is **not** a `nextflow.config` file:
+
+- values spanning several lines
+- nested blocks (e.g. `genomes { ... }`)
+- `process`, `profiles`, `docker` or other config sections
+
+Comments are supported, both on their own line and after a value, using `//` or `#`.
+
+If a parameter needs a genuinely structured value rather than a string, either define it in the pipeline's own `nextflow.config` in the git repository and expose the parts you want to change as individual parameters, or pass it with `--params-file` (see below), which supports JSON and YAML with full type fidelity.
+
 In addition, parameters can also be specified using the command-line `-p` or `--parameter`. For instance:
 
 ```bash
@@ -2613,7 +2631,7 @@ All configuration parameters are optional. If not specified, the session resumes
 
 ### Link
 
-The `cloudos interactive-session link` command provides a unified interface for linking files and folders to interactive analysis sessions. It consolidates functionality previously available through separate commands (`cloudos job results --link`, `cloudos job workdir --link`, `cloudos job logs --link`, and `cloudos datasets link`) into a single, intuitive interface.
+The `cloudos interactive-session link` command provides a unified interface for linking files and folders to interactive analysis sessions. It consolidates functionality previously available through separate commands (`cloudos job results --link`, `cloudos job workdir --link`, and `cloudos job logs --link`) into a single, intuitive interface.
 
 #### Link Files and Folders to Interactive Analysis
 
@@ -2845,41 +2863,8 @@ cloudos datasets cp AnalysesResults/my_analysis/results/my_plot.png Data/plots
 ```
 
 
-#### Link Files and Folders to Interactive Analysis
-
-Connect external S3 buckets, S3 files, or File Explorer files/folders to your interactive analysis sessions. This provides direct access to data without needing to copy files.
-
-This subcommand uses the `--session-id` option to access the correct interactive session. This option can be added to the CLI or defined in a profile, for convenience.
-
-```bash
-cloudos datasets link <PATH> --profile <profile> --session-id <SESSION_ID>
-```
-
-Link an S3 folder:
-```console
-cloudos datasets link s3://bucket/path/folder --profile test --session-id 1234
-```
-
-Link an S3 file:
-```console
-cloudos datasets link s3://bucket/path/data.csv --profile test --session-id 1234
-```
-
-Link a File Explorer folder (requires `--project-name`):
-```console
-cloudos datasets link "Data/HLA" --project-name my-project --session-id 1234
-```
-
-Link a File Explorer file (requires `--project-name`):
-```console
-cloudos datasets link "Data/observations.csv" --project-name my-project --session-id 1234
-```
-
-> [!NOTE]
-> If running the CLI inside a jupyter session, the pre-configured CLI installation will have the session ID already installed and only the `--apikey` needs to be added.
-
 > [!NOTE]
-> Virtual folders in File Explorer (folders created in File Explorer that are not actual storage locations) cannot be linked.
+> To link files or folders (S3 or File Explorer) to an interactive analysis session, use `cloudos interactive-session link` or the `--link` option of `cloudos interactive-session create`. Virtual folders in File Explorer cannot be linked.
 
 > [!NOTE]
 > A maximum of 100 items can be linked per session. If the new items combined with already-linked items exceed this limit, the entire request is rejected.

cloudos_cli-2.95.1/cloudos_cli/_version.py

@@ -0,0 +1 @@
+__version__ = '2.95.1'

{cloudos_cli-2.94.1 → cloudos_cli-2.95.1}/cloudos_cli/datasets/cli.py

@@ -4,7 +4,6 @@ import rich_click as click
 import csv
 import sys
 from cloudos_cli.datasets import Datasets
-from cloudos_cli.interactive_session.link import Link
 from cloudos_cli.utils.resources import ssl_selector, format_bytes
 from cloudos_cli.configure.configure import with_profile_config, CLOUDOS_URL
 from cloudos_cli.logging.logger import update_command_context_from_click
@@ -727,57 +726,18 @@ def rm_item(ctx,
         raise ValueError(f"Remove operation failed. {str(e)}")
 
 
-@datasets.command(name="link")
-@click.argument("path", required=True)
-@click.option('-k', '--apikey', help='Your Lifebit Platform API key', required=True)
-@click.option('-c', '--cloudos-url',
-              help=(f'The Lifebit Platform url you are trying to access to. Default={CLOUDOS_URL}.'),
-              default=CLOUDOS_URL)
-@click.option('--project-name',
-              help='The name of a Lifebit Platform project.',
-              required=False)
-@click.option('--workspace-id', help='The specific Lifebit Platform workspace id.', required=True)
-@click.option('--session-id', help='The specific Lifebit Platform interactive session id.', required=True)
-@click.option('--disable-ssl-verification', is_flag=True, help='Disable SSL certificate verification.')
-@click.option('--ssl-cert', help='Path to your SSL certificate file.')
-@click.option('--profile', help='Profile to use from the config file', default='default')
+@datasets.command(
+    name="link",
+    hidden=True,
+    add_help_option=False,
+    context_settings={"allow_extra_args": True, "ignore_unknown_options": True},
+)
 @click.pass_context
-@with_profile_config(required_params=['apikey', 'workspace_id', 'session_id'])
-def link(ctx,
-         path,
-         apikey,
-         cloudos_url,
-         project_name,
-         workspace_id,
-         session_id,
-         disable_ssl_verification,
-         ssl_cert,
-         profile):
-    """
-    Link a file or folder (S3 or File Explorer) to an active interactive analysis.
-
-    PATH [path]: the full path to the S3 file/folder or relative path in File Explorer
-    (relative to the project specified by --project-name).
-    E.g.: 's3://bucket-name/folder/subfolder', 's3://bucket/data/file.csv',
-    'Data/Downloads', 'Data', or 'Data/file.csv'.
-    """
-    if not path.startswith("s3://") and project_name is None:
-        raise click.UsageError("When using File Explorer paths '--project-name' needs to be defined")
-
-    verify_ssl = ssl_selector(disable_ssl_verification, ssl_cert)
-
-    link_p = Link(
-        cloudos_url=cloudos_url,
-        apikey=apikey,
-        workspace_id=workspace_id,
-        cromwell_token=None,
-        project_name=project_name,
-        verify=verify_ssl
+def link_deprecated(ctx):
+    raise click.ClickException(
+        "'cloudos datasets link' has been removed.\n\n"
+        "Use one of the supported alternatives:\n"
+        "  cloudos interactive-session link <PATH> --session-id <SESSION_ID>\n"
+        "  cloudos interactive-session create --link <PATH> [...]\n\n"
+        "Add --profile <my_profile> to either command to use a saved profile."
     )
-
-    try:
-        success = link_p.link_folder(path, session_id)
-    except Exception as e:
-        raise ValueError(f"Could not link item. {e}")
-    if not success:
-        raise click.ClickException("Linking failed: mount verification did not reach 'mounted' status.")

{cloudos_cli-2.94.1 → cloudos_cli-2.95.1}/cloudos_cli/interactive_session/link.py

@@ -78,8 +78,7 @@ class Link(Cloudos):
                            committed_count: int = 0) -> bool:
         """Link multiple folders/files (S3 or File Explorer) to an interactive session in one request.
 
-        Attempts to use API v2 (which supports multiple items per request) first,
-        with automatic fallback to v1 (individual requests) if v2 is not available.
+        Uses API v2 which supports multiple items per request.
 
         Parameters
         ----------
@@ -112,17 +111,13 @@ class Link(Cloudos):
         # Parse and validate all items
         data_items, folder_info = self._parse_items_to_data_items(folders, existing_mount_names)
 
-        # Try v2 API first (supports batch)
+        # Mount via v2 API (supports batch)
         status_code = self._try_mount_v2(data_items, session_id)
 
-        if status_code is None:
-            # v2 failed or not available, fall back to v1
-            status_code = self._fallback_mount_v1(folder_info, session_id)
-
         # Verify mount completion for all items (any 2xx response means success)
-        if status_code is not None and 200 <= status_code < 300:
+        if 200 <= status_code < 300:
             return self._verify_all_mounts(folder_info, session_id)
-        return True
+        return False
 
     def _parse_items_to_data_items(self, folders: list, existing_mount_names: set = None) -> tuple:
         """Parse and validate folders/files, extracting data items for API payload.
@@ -206,7 +201,7 @@ class Link(Cloudos):
         )
 
     def _try_mount_v2(self, data_items: list, session_id: str) -> int:
-        """Attempt to mount folders using API v2.
+        """Mount items using API v2.
 
         Parameters
         ----------
@@ -217,151 +212,35 @@ class Link(Cloudos):
 
         Returns
         -------
-        int or None
-            Status code if successful, None if v2 unavailable (triggering fallback).
+        int
+            HTTP status code on success.
 
         Raises
         ------
         ValueError
-            If v2 fails for reasons other than unavailability.
+            If the mount request fails.
         """
         v2_payload = {"dataItems": data_items}
 
         try:
-            status_code = self.mount_fuse_filesystem_v2(
+            return self.mount_fuse_filesystem_v2(
                 session_id=session_id,
                 team_id=self.workspace_id,
                 payload=v2_payload,
                 verify=self.verify
             )
-            return status_code
         except Exception as v2_error:
-            # Check if error indicates v2 endpoint not available (404 only, but not session-not-found)
-            error_str = str(v2_error)
-            # Only fall back to v1 if it's a genuine endpoint-not-available 404
-            # Session-not-found errors should propagate immediately
-            if "Session not found" in error_str:
-                raise  # Re-raise session-not-found errors immediately
-
-            should_fallback = (
-                "404" in error_str or "Not Found" in error_str or "not found" in error_str.lower()
-            )
-
-            if should_fallback:
-                return None  # Trigger v1 fallback
-            else:
-                # v2 failed for reasons other than not available
-                self._handle_mount_error(v2_error, "folder")
-
-    def _fallback_mount_v1(self, folder_info: list, session_id: str) -> int:
-        """Fall back to v1 API, mounting folders one at a time.
-
-        Parameters
-        ----------
-        folder_info : list
-            List of folder metadata dictionaries.
-        session_id : str
-            The interactive session ID.
-
-        Returns
-        -------
-        int
-            Status code from the last successful mount (typically 204).
-
-        Raises
-        ------
-        ValueError
-            If any item is a file (v1 only supports folders), or if any folder
-            fails to mount. Note: Earlier folders may have successfully mounted
-            before the failure.
-        """
-        for f in folder_info:
-            item_type = f['data'].get('type', '')
-            item_kind = f['data'].get('kind', '')
-            if item_type == 'S3File' or item_kind == 'File':
+            if isinstance(v2_error, ValueError):
+                raise
+            if isinstance(v2_error, BadRequestException) and v2_error.rv.status_code == 404:
                 raise ValueError(
-                    f"File linking requires API v2, which is not available for this session. "
-                    f"Only folder linking is supported via the v1 API fallback."
-                )
-
-        status_code = None
-        mounted_folders = []
-
-        for folder_data in folder_info:
-            try:
-                status_code = self._mount_single_folder_v1(folder_data, session_id)
-                mounted_folders.append(folder_data['path'])
-            except ValueError as e:
-                # If we've already mounted some folders, inform the user
-                if mounted_folders:
-                    error_msg = f"{str(e)}\n\nNote: The following folders were successfully mounted before this error: {', '.join(mounted_folders)}"
-                    raise ValueError(error_msg)
-                else:
-                    raise
-        return status_code
-
-    def _mount_single_folder_v1(self, folder_data: dict, session_id: str) -> int:
-        """Mount a single folder using API v1.
-
-        Parameters
-        ----------
-        folder_data : dict
-            Folder metadata including type, path, and data.
-        session_id : str
-            The interactive session ID.
-
-        Returns
-        -------
-        int
-            Status code (typically 204 on success).
-
-        Raises
-        ------
-        ValueError
-            If the mount request fails.
-        """
-        v1_payload = {"dataItem": folder_data["data"]}
-
-        url = (
-            f"{self.cloudos_url}/api/v1/"
-            f"interactive-sessions/{session_id}/fuse-filesystem/mount"
-            f"?teamId={self.workspace_id}"
-        )
-        headers = {
-            "Content-type": "application/json",
-            "apikey": self.apikey
-        }
-
-        try:
-            r = retry_requests_post(url, headers=headers, json=v1_payload, verify=self.verify)
-
-            if r.status_code >= 400:
-                # Handle v1 errors using consolidated error handling
-                if r.status_code == 403:
-                    raise ValueError(f"Provided {folder_data['type']} item already exists with 'mounted' status")
-                elif r.status_code == 401:
-                    raise ValueError("Unauthorized. Invalid API key or insufficient permissions.")
-                elif r.status_code == 400:
-                    try:
-                        r_content = json.loads(r.content)
-                        if r_content.get("message") == "Invalid Supported DataItem folderType. Supported values are S3Folder":
-                            raise ValueError(f"Invalid Supported DataItem '{folder_data['type']}' folderType. Virtual folders cannot be linked.")
-                        elif r_content.get("message") == "Request failed with status code 403":
-                            raise ValueError(f"Interactive Analysis session is not active")
-                        else:
-                            raise ValueError(f"Cannot link item")
-                    except json.JSONDecodeError:
-                        raise ValueError(f"Bad request (400): Unable to parse error response")
-                else:
-                    raise ValueError(f"Failed to mount item: HTTP {r.status_code}")
-
-            return r.status_code
-
-        except ValueError:
-            # Re-raise ValueError as-is
-            raise
-        except Exception as v1_error:
-            raise ValueError(f"Failed to mount {folder_data['type']} item: {str(v1_error)}")
+                    "The linking API (v2) is not available on this platform. "
+                    "Contact your platform administrator."
+                ) from v2_error
+            all_s3 = all(item.get("type", "").startswith("S3") for item in data_items)
+            all_fe = all(not item.get("type", "").startswith("S3") for item in data_items)
+            label = "S3" if all_s3 else "File Explorer" if all_fe else "item"
+            self._handle_mount_error(v2_error, label)
 
     def _verify_all_mounts(self, folder_info: list, session_id: str):
         """Verify mount completion status for all items (files and folders).

{cloudos_cli-2.94.1 → cloudos_cli-2.95.1}/cloudos_cli/jobs/cli.py

@@ -62,8 +62,13 @@ def job():
               help=('When the workflows are duplicated, use the latest imported workflow (by date).'),
               is_flag=True)
 @click.option('--job-config',
-              help=('A config file similar to a nextflow.config file, ' +
-                    'but only with the parameters to use with your job.'))
+              help=('A file listing the parameters to use with your job, as ' +
+                    '\'name = value\' pairs inside a \'params\' block. It is ' +
+                    'equivalent to repeating --parameter, so only flat ' +
+                    'parameters are supported: it is not a nextflow.config ' +
+                    'and cannot contain multi-line parameters, nested blocks or sections ' +
+                    'such as \'process\' and \'profiles\'. Use --params-file ' +
+                    'for parameters that need a structured value.'))
 @click.option('--params-file',
               help=('A file containing the parameters to pass to the job call. ' +
                     'It should be a .json or .yaml file with a dictionary structure ' +

{cloudos_cli-2.94.1 → cloudos_cli-2.95.1}/cloudos_cli/jobs/job.py

@@ -176,6 +176,167 @@ class Job(Cloudos):
         else:
             raise ValueError(f'No {name} element in {resource} was found')
 
+    @staticmethod
+    def strip_inline_comment(line):
+        """Remove a trailing '//' or '#' comment from a job config line.
+
+        Only comments preceded by whitespace are removed, so that values
+        containing these characters, such as 's3://bucket/key', are preserved.
+
+        Parameters
+        ----------
+        line : string
+            A single raw line from a job config file.
+
+        Returns
+        -------
+        string
+            The line with any trailing comment removed.
+        """
+        return re.sub(r'\s+(?://|#).*$', '', line)
+
+    @staticmethod
+    def list_literal_warning(p_name, p_value):
+        """Build a warning for a value that looks like a list but is sent as text.
+
+        A '[a, b]' literal in a job config file is not expanded into a Nextflow
+        list: every job config value is sent as text. Users coming from real
+        nextflow.config files expect the expansion, so warn and show the value
+        the pipeline actually receives.
+
+        Parameters
+        ----------
+        p_name : string
+            The parameter name.
+        p_value : string
+            The parameter value, as it will be sent.
+
+        Returns
+        -------
+        string or None
+            The warning message, or None if the value is not a list literal.
+        """
+        if not (p_value.startswith('[') and p_value.endswith(']')):
+            return None
+        elements = [e for e in p_value[1:-1].split(',') if len(e) > 0]
+        element_noun = 'element' if len(elements) == 1 else 'elements'
+        listed = ', '.join(f"'{e}'" for e in elements) if elements else 'nothing'
+        return (f"Warning: parameter '{p_name}' looks like a list of "
+                f"{len(elements)} {element_noun} ({listed}), but every "
+                '--job-config value is sent as text.\n'
+                f"\tThe pipeline will receive the string --{p_name} '{p_value}', "
+                'not a Nextflow list of '
+                f'{len(elements)} {element_noun}.\n'
+                "\tTo pass a real list, define it in the pipeline's own "
+                'nextflow.config, or use --params-file with a JSON or YAML file.')
+
+    @staticmethod
+    def parse_job_config_params(job_config, workflow_type='nextflow'):
+        """Parse a job config file into a list of (name, value) parameter pairs.
+
+        The job config file is a list of parameters, equivalent to repeating
+        '--parameter name=value' on the command line. Every value is sent as
+        text, exactly as when launching Nextflow from the command line, so a
+        literal such as "['a', 'b']" is passed through as a string. Values
+        spanning several lines and nested blocks cannot be represented.
+
+        Parameters
+        ----------
+        job_config : string
+            Path to the job config file.
+        workflow_type : string
+            The type of workflow to run. Quotes are kept for 'wdl' workflows.
+
+        Returns
+        -------
+        list
+            A list of (name, value) tuples, in file order.
+
+        Raises
+        ------
+        ValueError
+            If the file has no 'params' block, if the block is opened and closed
+            on the same line, or if a line inside the block is not a
+            'name = value' pair, has a value spanning several lines, or has no
+            value at all. Every message reports the offending line, and all but
+            the missing block report its line number.
+
+        Notes
+        -----
+        A value that looks like a list is parsed and returned, but prints a
+        warning first, since it is sent as text rather than as a Nextflow list.
+        The warning is skipped for 'wdl' workflows.
+        """
+        parsed = []
+        with open(job_config, 'r') as p:
+            reading = False
+            found_params_block = False
+            for line_no, p_l in enumerate(p, start=1):
+                if re.match(r'^\s*params\s*\{', p_l, re.IGNORECASE):
+                    found_params_block = True
+                    reading = True
+                    remainder = Job.strip_inline_comment(p_l).split('{', 1)[1].strip()
+                    if len(remainder) > 0 and remainder[0] not in ('/', '#'):
+                        raise ValueError(
+                            f'Could not parse line {line_no} of {job_config}: ' +
+                            f'"{p_l.strip()}". The \'params\' block must span ' +
+                            'several lines, with one \'name = value\' pair per ' +
+                            'line.')
+                    continue
+                if not reading:
+                    continue
+                p_l_no_comment = Job.strip_inline_comment(p_l)
+                if workflow_type == 'wdl':
+                    p_l_strip = p_l_no_comment.strip().replace(' ', '')
+                else:
+                    p_l_strip = p_l_no_comment.strip().replace(
+                        ' ', '').replace('\"', '').replace('\'', '')
+                if len(p_l_strip) == 0:
+                    continue
+                elif p_l_strip[0] in ('/', '#', '*'):
+                    continue
+                elif p_l_strip == '}':
+                    reading = False
+                    continue
+                p_list = p_l_strip.split('=')
+                if len(p_list) < 2:
+                    raise ValueError(
+                        f'Could not parse line {line_no} of {job_config}: ' +
+                        f'"{p_l.strip()}". Expected a \'name = value\' pair. ' +
+                        'The job config file is a list of parameters, not a ' +
+                        'Nextflow config: it cannot contain values spanning ' +
+                        'several lines, nested blocks, or sections such as ' +
+                        '\'process\' and \'profiles\'.')
+                p_name = p_list[0]
+                p_value = '='.join(p_list[1:])
+                if p_value.count('[') > p_value.count(']'):
+                    raise ValueError(
+                        f'Could not parse line {line_no} of {job_config}: ' +
+                        f'"{p_l.strip()}". Values spanning several lines are ' +
+                        'not supported. Keep the value on a single line, ' +
+                        'define it in the pipeline\'s own nextflow.config, ' +
+                        'or pass it with --params-file using a JSON or YAML ' +
+                        'file.')
+                if len(p_value) == 0:
+                    raise ValueError(
+                        f'Could not parse line {line_no} of {job_config}: ' +
+                        f'"{p_l.strip()}". Parameter \'{p_name}\' has no ' +
+                        'value. Give it a value, or remove the line. A value ' +
+                        'that starts with \'//\' or \'#\' must be quoted, ' +
+                        'otherwise it is read as a comment.')
+                if workflow_type != 'wdl':
+                    warning = Job.list_literal_warning(p_name, p_value)
+                    if warning is not None:
+                        click.secho(warning, fg='yellow', bold=True)
+                parsed.append((p_name, p_value))
+        if not found_params_block:
+            raise ValueError(
+                f'No \'params\' block was found in {job_config}. ' +
+                'The parameters must be listed inside a \'params { ... }\' ' +
+                'block, one \'name = value\' pair per line. The ' +
+                '\'params.name = value\' form is not supported.')
+        return parsed
+
     def build_parameters_file_payload(self, params_file):
         """Build the parametersFile payload for a params file path."""
         if params_file is None:
@@ -323,12 +484,12 @@ class Job(Cloudos):
                                  command,
                                  cpus,
                                  memory):
-        """Converts a nextflow.config file into a json formatted dict.
+        """Converts a job config file into a json formatted dict.
 
         Parameters
         ----------
         job_config : string
-            Path to a nextflow.config file with parameters scope.
+            Path to a file with parameters scope.
         params_file : string
             S3 or File Explorer path to a JSON/YAML file with Nextflow parameters.
         parameter : tuple
@@ -422,47 +583,12 @@ class Job(Cloudos):
             raise ValueError('No --job-config or --parameter were provided. At least one of ' +
                              'these are required for WDL workflows.')
         if job_config is not None:
-            with open(job_config, 'r') as p:
-                reading = False
-                for p_l in p:
-                    if 'params' in p_l.lower():
-                        reading = True
-                    else:
-                        if reading:
-                            if workflow_type == 'wdl':
-                                p_l_strip = p_l.strip().replace(
-                                    ' ', '')
-                            else:
-                                p_l_strip = p_l.strip().replace(
-                                    ' ', '').replace('\"', '').replace('\'', '')
-                            if len(p_l_strip) == 0:
-                                continue
-                            elif p_l_strip[0] == '/' or p_l_strip[0] == '#':
-                                continue
-                            elif p_l_strip == '}':
-                                reading = False
-                            else:
-                                p_list = p_l_strip.split('=')
-                                p_name = p_list[0]
-                                p_value = '='.join(p_list[1:])
-                                if len(p_list) < 2:
-                                    raise ValueError('Please, specify your ' +
-                                                     'parameters in ' +
-                                                     f'{job_config} using ' +
-                                                     'the \'=\' as spacer. ' +
-                                                     'E.g: name = my_name')
-                                elif workflow_type == 'wdl':
-                                    param = {"prefix": "",
-                                             "name": p_name,
-                                             "parameterKind": "textValue",
-                                             "textValue": p_value}
-                                    workflow_params.append(param)
-                                else:
-                                    param = {"prefix": "--",
-                                             "name": p_name,
-                                             "parameterKind": "textValue",
-                                             "textValue": p_value}
-                                    workflow_params.append(param)
+            p_prefix = "" if workflow_type == 'wdl' else "--"
+            for p_name, p_value in Job.parse_job_config_params(job_config, workflow_type):
+                workflow_params.append({"prefix": p_prefix,
+                                        "name": p_name,
+                                        "parameterKind": "textValue",
+                                        "textValue": p_value})
             if len(workflow_params) == 0:
                 raise ValueError(f'The {job_config} file did not contain any ' +
                                  'valid parameter')
@@ -611,7 +737,7 @@ class Job(Cloudos):
         Parameters
         ----------
         job_config : string
-            Path to a nextflow.config file with parameters scope.
+            Path to a file with parameters scope.
         params_file : string
             S3 or File Explorer path to a JSON/YAML file with Nextflow parameters.
         parameter : tuple

{cloudos_cli-2.94.1 → cloudos_cli-2.95.1}/cloudos_cli.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cloudos_cli
-Version: 2.94.1
+Version: 2.95.1
 Summary: Python package for interacting with the Lifebit Platform
 Home-page: https://github.com/lifebit-ai/cloudos-cli
 Author: David Piñeyro
@@ -609,7 +609,7 @@ First, configure your local environment to ease parameter input. We will try to
 cloudos job run --profile my_profile --workflow-name rnatoy --job-config cloudos_cli/examples/rnatoy.config --resumable
 ```
 
-As you can see, a file with the job parameters is used to configure the job. This file could be a regular `nextflow.config` file or any file with the following structure:
+As you can see, a file with the job parameters is used to configure the job. This file is a list of the parameters to pass to your job, and nothing else. It has the following structure:
 
 ```
 params {
@@ -618,6 +618,24 @@ params {
 }
 ```
 
+`--job-config` is shorthand for repeating `--parameter` on the command line: the CLI reads the file line by line, splits each line on the first `=`, and sends each result as one named parameter.
+
+Every value is passed through as text, exactly as it would be when launching Nextflow from the command line.
+
+> **IMPORTANT: `[ ]` is not expanded into a Nextflow list.** In a real `nextflow.config`, `my_param = ['a', 'b']` gives the pipeline a list of two items. Through `--job-config` it gives the pipeline the *string* `[a,b]`. The line is accepted and the job is submitted, so this fails silently at the point where your pipeline tries to iterate over the value. The CLI prints a warning when it sees a value that looks like a list, showing exactly what the pipeline will receive.
+
+To pass a genuine list, define it in the pipeline's own `nextflow.config` in the git repository, or use `--params-file` with a JSON or YAML file (see below), which preserves types.
+
+What a `--job-config` file cannot contain, because it is **not** a `nextflow.config` file:
+
+- values spanning several lines
+- nested blocks (e.g. `genomes { ... }`)
+- `process`, `profiles`, `docker` or other config sections
+
+Comments are supported, both on their own line and after a value, using `//` or `#`.
+
+If a parameter needs a genuinely structured value rather than a string, either define it in the pipeline's own `nextflow.config` in the git repository and expose the parts you want to change as individual parameters, or pass it with `--params-file` (see below), which supports JSON and YAML with full type fidelity.
+
 In addition, parameters can also be specified using the command-line `-p` or `--parameter`. For instance:
 
 ```bash
@@ -2648,7 +2666,7 @@ All configuration parameters are optional. If not specified, the session resumes
 
 ### Link
 
-The `cloudos interactive-session link` command provides a unified interface for linking files and folders to interactive analysis sessions. It consolidates functionality previously available through separate commands (`cloudos job results --link`, `cloudos job workdir --link`, `cloudos job logs --link`, and `cloudos datasets link`) into a single, intuitive interface.
+The `cloudos interactive-session link` command provides a unified interface for linking files and folders to interactive analysis sessions. It consolidates functionality previously available through separate commands (`cloudos job results --link`, `cloudos job workdir --link`, and `cloudos job logs --link`) into a single, intuitive interface.
 
 #### Link Files and Folders to Interactive Analysis
 
@@ -2880,41 +2898,8 @@ cloudos datasets cp AnalysesResults/my_analysis/results/my_plot.png Data/plots
 ```
 
 
-#### Link Files and Folders to Interactive Analysis
-
-Connect external S3 buckets, S3 files, or File Explorer files/folders to your interactive analysis sessions. This provides direct access to data without needing to copy files.
-
-This subcommand uses the `--session-id` option to access the correct interactive session. This option can be added to the CLI or defined in a profile, for convenience.
-
-```bash
-cloudos datasets link <PATH> --profile <profile> --session-id <SESSION_ID>
-```
-
-Link an S3 folder:
-```console
-cloudos datasets link s3://bucket/path/folder --profile test --session-id 1234
-```
-
-Link an S3 file:
-```console
-cloudos datasets link s3://bucket/path/data.csv --profile test --session-id 1234
-```
-
-Link a File Explorer folder (requires `--project-name`):
-```console
-cloudos datasets link "Data/HLA" --project-name my-project --session-id 1234
-```
-
-Link a File Explorer file (requires `--project-name`):
-```console
-cloudos datasets link "Data/observations.csv" --project-name my-project --session-id 1234
-```
-
-> [!NOTE]
-> If running the CLI inside a jupyter session, the pre-configured CLI installation will have the session ID already installed and only the `--apikey` needs to be added.
-
 > [!NOTE]
-> Virtual folders in File Explorer (folders created in File Explorer that are not actual storage locations) cannot be linked.
+> To link files or folders (S3 or File Explorer) to an interactive analysis session, use `cloudos interactive-session link` or the `--link` option of `cloudos interactive-session create`. Virtual folders in File Explorer cannot be linked.
 
 > [!NOTE]
 > A maximum of 100 items can be linked per session. If the new items combined with already-linked items exceed this limit, the entire request is rejected.

cloudos_cli-2.94.1/cloudos_cli/_version.py

@@ -1 +0,0 @@
-__version__ = '2.94.1'