Py4HEAppE 2.8.0__tar.gz → 2.9.0__tar.gz

{py4heappe-2.8.0 → py4heappe-2.9.0}/CHANGELOG.md

@@ -5,6 +5,14 @@ All notable changes to this project will be documented in this file.
 The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.1.0/),
 and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
 
+## 2.9.0
+
+### Compatible with HEAppE V6.4.X, V6.3.X, V6.2.X, V6.2.1, V5.0.X, V4.3.X and V4.2.X
+
+### Added
+- Added `--full/--minimal` to `py4heappe Job InitJobSpecification` so the initializer can generate either the original minimal template or a full swagger-shaped job specification.
+- Added `--save-result-job-specification` to `py4heappe Job Create` to persist the final parsed job specification JSON in the current working directory before submission.
+
 ## 2.8.0
 
 ### Compatible with HEAppE V6.4.X, V6.3.X, V6.2.X, V6.2.1, V5.0.X, V4.3.X and V4.2.X

{py4heappe-2.8.0 → py4heappe-2.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: Py4HEAppE
-Version: 2.8.0
+Version: 2.9.0
 Summary: Py4HEAppE is client library for easy access to HEAppE Middleware
 Author-email: IT4Innovations <support.heappe@it4i.cz>
 License-Expression: MIT
@@ -36,6 +36,7 @@ You can use it in two ways:
 ## Supported HEAppE Versions
 | Py4HEAppE | HEAppE Version | Notes |
 | :-------: | :------------- | :---- |
+| 2.9.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.8.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.7.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.6.X | 6.3.X, 6.2.X, 6.1.X, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
@@ -121,7 +122,7 @@ py4heappe Job List
 
 ### Create a Job from a JSON Template
 
-Py4HEAppE 2.8 adds a JSON-first workflow for job creation.
+Py4HEAppE supports a JSON-first workflow for job creation.
 
 Generate a minimal job specification template in the current directory:
 
@@ -129,6 +130,12 @@ Generate a minimal job specification template in the current directory:
 py4heappe Job InitJobSpecification
 ```
 
+Generate the full swagger-shaped job specification template instead:
+
+```shell
+py4heappe Job InitJobSpecification --full
+```
+
 Or generate it in a specific directory or target file:
 
 ```shell
@@ -139,21 +146,29 @@ py4heappe Job InitJobSpecification --file-destination ./job_specs/demo_job.json
 Create a job from the JSON file and override selected values from the command line:
 
 ```shell
-py4heappe Job Create \
-  --json-job-spec-file ./job_specification.json \
-  --name demo-job \
-  --cluster-id 2 \
-  --project-id 1 \
-  --task-name 0:demo-task \
-  --max-cores 0:128 \
-  --walltime-limit 0:1800 \
-  --cluster-node-type-id 0:18 \
-  --cmd-template-id 0:3 \
-  --cmd-template-parameters inputParam:testValue
+py4heappe Job Create --json-job-spec-file ./job_specification.json \
+    --name demo-job \
+    --cluster-id 2 \
+    --project-id 1 \
+    --task-name 0:demo-task \
+    --task-max-cores 0:128 \
+    --walltime-limit 0:1800 \
+    --cluster-node-type-id 0:18 \
+    --cmd-template-id 0:3 \
+    --cmd-template-parameters inputParam:testValue
 ```
 
 CLI overrides always take precedence over values loaded from `--json-job-spec-file`.
 
+Save the final parsed job specification that will be submitted to HEAppE:
+
+```shell
+py4heappe Job Create --json-job-spec-file ./job_specification.json \
+    --save-result-job-specification
+```
+
+This writes `resultJobSpecJson.json` into the current working directory.
+
 ### Override Format for Nested Task Data
 
 Some `Job Create` options work on indexed task items and nested collections.

{py4heappe-2.8.0 → py4heappe-2.9.0}/README.md

@@ -10,6 +10,7 @@ You can use it in two ways:
 ## Supported HEAppE Versions
 | Py4HEAppE | HEAppE Version | Notes |
 | :-------: | :------------- | :---- |
+| 2.9.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.8.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.7.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.6.X | 6.3.X, 6.2.X, 6.1.X, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
@@ -95,7 +96,7 @@ py4heappe Job List
 
 ### Create a Job from a JSON Template
 
-Py4HEAppE 2.8 adds a JSON-first workflow for job creation.
+Py4HEAppE supports a JSON-first workflow for job creation.
 
 Generate a minimal job specification template in the current directory:
 
@@ -103,6 +104,12 @@ Generate a minimal job specification template in the current directory:
 py4heappe Job InitJobSpecification
 ```
 
+Generate the full swagger-shaped job specification template instead:
+
+```shell
+py4heappe Job InitJobSpecification --full
+```
+
 Or generate it in a specific directory or target file:
 
 ```shell
@@ -113,21 +120,29 @@ py4heappe Job InitJobSpecification --file-destination ./job_specs/demo_job.json
 Create a job from the JSON file and override selected values from the command line:
 
 ```shell
-py4heappe Job Create \
-  --json-job-spec-file ./job_specification.json \
-  --name demo-job \
-  --cluster-id 2 \
-  --project-id 1 \
-  --task-name 0:demo-task \
-  --max-cores 0:128 \
-  --walltime-limit 0:1800 \
-  --cluster-node-type-id 0:18 \
-  --cmd-template-id 0:3 \
-  --cmd-template-parameters inputParam:testValue
+py4heappe Job Create --json-job-spec-file ./job_specification.json \
+    --name demo-job \
+    --cluster-id 2 \
+    --project-id 1 \
+    --task-name 0:demo-task \
+    --task-max-cores 0:128 \
+    --walltime-limit 0:1800 \
+    --cluster-node-type-id 0:18 \
+    --cmd-template-id 0:3 \
+    --cmd-template-parameters inputParam:testValue
 ```
 
 CLI overrides always take precedence over values loaded from `--json-job-spec-file`.
 
+Save the final parsed job specification that will be submitted to HEAppE:
+
+```shell
+py4heappe Job Create --json-job-spec-file ./job_specification.json \
+    --save-result-job-specification
+```
+
+This writes `resultJobSpecJson.json` into the current working directory.
+
 ### Override Format for Nested Task Data
 
 Some `Job Create` options work on indexed task items and nested collections.

{py4heappe-2.8.0 → py4heappe-2.9.0}/pyproject.toml

@@ -6,7 +6,7 @@ build-backend = "setuptools.build_meta"
 name = "Py4HEAppE"
 authors = [{name = "IT4Innovations", email = "support.heappe@it4i.cz"}]
 description = "Py4HEAppE is client library for easy access to HEAppE Middleware"
-version = "2.8.0"
+version = "2.9.0"
 license = "MIT"
 readme = "README.md"
 requires-python = ">=3.11"

{py4heappe-2.8.0 → py4heappe-2.9.0}/src/Py4HEAppE.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: Py4HEAppE
-Version: 2.8.0
+Version: 2.9.0
 Summary: Py4HEAppE is client library for easy access to HEAppE Middleware
 Author-email: IT4Innovations <support.heappe@it4i.cz>
 License-Expression: MIT
@@ -36,6 +36,7 @@ You can use it in two ways:
 ## Supported HEAppE Versions
 | Py4HEAppE | HEAppE Version | Notes |
 | :-------: | :------------- | :---- |
+| 2.9.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.8.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.7.X | 6.4.X, 6.3.X, 6.2.X, 6.2.1, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
 | 2.6.X | 6.3.X, 6.2.X, 6.1.X, 5.0.X, 4.3.X, 4.2.X | Without Admin CLI section |
@@ -121,7 +122,7 @@ py4heappe Job List
 
 ### Create a Job from a JSON Template
 
-Py4HEAppE 2.8 adds a JSON-first workflow for job creation.
+Py4HEAppE supports a JSON-first workflow for job creation.
 
 Generate a minimal job specification template in the current directory:
 
@@ -129,6 +130,12 @@ Generate a minimal job specification template in the current directory:
 py4heappe Job InitJobSpecification
 ```
 
+Generate the full swagger-shaped job specification template instead:
+
+```shell
+py4heappe Job InitJobSpecification --full
+```
+
 Or generate it in a specific directory or target file:
 
 ```shell
@@ -139,21 +146,29 @@ py4heappe Job InitJobSpecification --file-destination ./job_specs/demo_job.json
 Create a job from the JSON file and override selected values from the command line:
 
 ```shell
-py4heappe Job Create \
-  --json-job-spec-file ./job_specification.json \
-  --name demo-job \
-  --cluster-id 2 \
-  --project-id 1 \
-  --task-name 0:demo-task \
-  --max-cores 0:128 \
-  --walltime-limit 0:1800 \
-  --cluster-node-type-id 0:18 \
-  --cmd-template-id 0:3 \
-  --cmd-template-parameters inputParam:testValue
+py4heappe Job Create --json-job-spec-file ./job_specification.json \
+    --name demo-job \
+    --cluster-id 2 \
+    --project-id 1 \
+    --task-name 0:demo-task \
+    --task-max-cores 0:128 \
+    --walltime-limit 0:1800 \
+    --cluster-node-type-id 0:18 \
+    --cmd-template-id 0:3 \
+    --cmd-template-parameters inputParam:testValue
 ```
 
 CLI overrides always take precedence over values loaded from `--json-job-spec-file`.
 
+Save the final parsed job specification that will be submitted to HEAppE:
+
+```shell
+py4heappe Job Create --json-job-spec-file ./job_specification.json \
+    --save-result-job-specification
+```
+
+This writes `resultJobSpecJson.json` into the current working directory.
+
 ### Override Format for Nested Task Data
 
 Some `Job Create` options work on indexed task items and nested collections.

py4heappe-2.9.0/src/py4heappe/heappe_v6/cli/configuration.py

@@ -0,0 +1,97 @@
+import dotenv
+import os
+import typer
+import validators
+
+import py4heappe.heappe_v6.core as hp
+import py4heappe.heappe_v6.core.base.utils as utils
+from py4heappe.heappe_v6.core.base import exceptions
+from urllib.parse import urlparse
+
+app = typer.Typer(
+    name="HEAppEConfigurationCLI",
+    no_args_is_help=True,
+    pretty_exceptions_show_locals=False,
+)
+
+
+def get_api_instance():
+    dotenv.load_dotenv(os.path.join(os.path.dirname(__file__), ".env"))
+    url: str = os.environ.get("url")
+
+    if url is None:
+        raise exceptions.Py4HEAppEInternalException(
+            "Py4HEappE is not configured. Please check Conf option."
+        ) from None
+
+    configuration = hp.Configuration()
+    configuration.host = url
+    api_instance = hp.ApiClient(configuration)
+    return api_instance
+
+
+def get_project_from_configuration():
+    dotenv.load_dotenv(os.path.join(os.path.dirname(__file__), ".env"))
+
+    computationalProject: str = os.environ.get("project")
+    if computationalProject is None:
+        raise exceptions.Py4HEAppEInternalException(
+            "Py4HEappE is not configured. Please check Conf option."
+        ) from None
+
+    return computationalProject
+
+
+@app.command(name="Init")
+def environment_preparation():
+    """Initialization"""
+    utils.print_and_log("Preparation of Py4HEAppE Environment ...")
+    dotenv.load_dotenv()
+
+    url: str = os.environ.get("url")
+    computationalProject: str = os.environ.get("project")
+
+    if url is not None or computationalProject is not None:
+        reconfigure = typer.confirm(
+            f'Py4HEAppE is already configured (URL: "{url}", HEAppE accounting string (HPC Computational project): "{computationalProject}").\nDo you want to reconfigure it?'
+        )
+        if not reconfigure:
+            return
+
+    url = typer.prompt(
+        "Enter the HEAppE instance address (for example: https://heappe.domain.cz/production)"
+    )
+
+    normalized = url
+    parsed = urlparse(normalized)
+    if not parsed.scheme:
+        normalized = "http://" + normalized
+        parsed = urlparse(normalized)
+
+    host = parsed.hostname
+
+    if not validators.url(normalized) and host not in ("localhost", "127.0.0.1"):
+        raise exceptions.Py4HEAppEInternalException(
+            "The provided HEAppE URL is not valid."
+        ) from None
+
+    if "/swagger/index.html" in normalized:
+        normalized = normalized.replace("/swagger/index.html", "")
+
+    computationalProject = typer.prompt(
+        "Enter HEAppE accounting string", confirmation_prompt=True
+    )
+
+    dotenv.set_key(os.path.join(os.path.dirname(__file__), ".env"), "url", normalized)
+    dotenv.set_key(
+        os.path.join(os.path.dirname(__file__), ".env"),
+        "project",
+        computationalProject.upper(),
+    )
+
+    utils.print_and_log("Py4HEAppE is configured.")
+    return
+
+
+if __name__ == "__main__":
+    app()

{py4heappe-2.8.0 → py4heappe-2.9.0}/src/py4heappe/heappe_v6/cli/job_management.py

@@ -19,7 +19,7 @@ app = typer.Typer(
 )
 
 
-INIT_JOB_SPECIFICATION_TEMPLATE = {
+MINIMAL_JOB_SPECIFICATION_TEMPLATE = {
     "SessionCode": "",
     "JobSpecification": {
         "Name": "test",
@@ -50,6 +50,81 @@ INIT_JOB_SPECIFICATION_TEMPLATE = {
     },
 }
 
+FULL_JOB_SPECIFICATION_TEMPLATE = {
+    "SessionCode": "string",
+    "JobSpecification": {
+        "Name": "string",
+        "ProjectId": 0,
+        "SubProjectIdentifier": "string",
+        "WaitingLimit": 0,
+        "NotificationEmail": "string",
+        "PhoneNumber": "string",
+        "NotifyOnAbort": True,
+        "NotifyOnFinish": True,
+        "NotifyOnStart": True,
+        "ClusterId": 0,
+        "FileTransferMethodId": 0,
+        "Reservation": "string",
+        "IsExtraLong": True,
+        "EnvironmentVariables": [
+            {
+                "Name": "string",
+                "Value": "string",
+            }
+        ],
+        "Tasks": [
+            {
+                "Name": "string",
+                "MinCores": 0,
+                "MaxCores": 0,
+                "GpuCores": 0,
+                "GpuNodes": 0,
+                "WalltimeLimit": 0,
+                "PlacementPolicy": "string",
+                "Priority": 0,
+                "JobArrays": "string",
+                "IsExclusive": True,
+                "IsRerunnable": True,
+                "StandardInputFile": "string",
+                "StandardOutputFile": "string",
+                "StandardErrorFile": "string",
+                "ProgressFile": "string",
+                "LogFile": "string",
+                "ClusterTaskSubdirectory": "string",
+                "ClusterNodeTypeId": 0,
+                "CommandTemplateId": 0,
+                "CpuHyperThreading": True,
+                "RequiredNodes": ["string"],
+                "TaskParallelizationParameters": [
+                    {
+                        "MPIProcesses": 0,
+                        "OpenMPThreads": 0,
+                        "MaxCores": 0,
+                    }
+                ],
+                "EnvironmentVariables": [
+                    {
+                        "Name": "string",
+                        "Value": "string",
+                    }
+                ],
+                "Memory": 0,
+                "MemoryPerCPU": 0,
+                "MemoryPerGPU": 0,
+                "DependsOn": ["string"],
+                "TemplateParameterValues": [
+                    {
+                        "CommandParameterIdentifier": "string",
+                        "ParameterValue": "string",
+                    }
+                ],
+            }
+        ],
+    },
+}
+
+DEFAULT_RESULT_JOB_SPECIFICATION_FILENAME = "resultJobSpecJson.json"
+
 DEFAULT_CREATE_JOB_SPECIFICATION = {
     "SessionCode": "",
     "JobSpecification": {
@@ -580,10 +655,19 @@ def init_job_specification(
         None,
         help="Destination directory or JSON file path for the initialized job specification.",
     ),
+    full: bool = typer.Option(
+        False,
+        "--full/--minimal",
+        help="Initialize a full job specification instead of the default minimal template.",
+    ),
 ):
-    """Create a minimal HPC job specification JSON file."""
+    """Create a minimal or full HPC job specification JSON file."""
     try:
-        job_specification = deepcopy(INIT_JOB_SPECIFICATION_TEMPLATE)
+        job_specification = deepcopy(
+            FULL_JOB_SPECIFICATION_TEMPLATE
+            if full
+            else MINIMAL_JOB_SPECIFICATION_TEMPLATE
+        )
 
         destination = (
             Path(file_destination).expanduser() if file_destination else Path.cwd()
@@ -597,7 +681,7 @@ def init_job_specification(
 
         file_path.write_text(json.dumps(job_specification, indent=4), encoding="utf-8")
         utils.print_and_log(
-            f"Minimal job specification was created in: {file_path.resolve()}"
+            f"{'Full' if full else 'Minimal'} job specification was created in: {file_path.resolve()}"
         )
 
     except exceptions.Py4HEAppEInternalException as exception:
@@ -692,11 +776,10 @@ def create_job(
             help="Override task minimum cores as taskIndex:value.",
         ),
     ] = None,
-    max_cores: Annotated[
+    task_max_cores: Annotated[
         Optional[List[str]],
         typer.Option(
-            "--max-cores",
-            "--maxcores",
+            "--task-max-cores",
             help="Override task maximum cores as taskIndex:value.",
         ),
     ] = None,
@@ -885,6 +968,14 @@ def create_job(
             help="Override task template parameter values as taskIndex:itemIndex:field:value.",
         ),
     ] = None,
+    save_result_job_specification: bool = typer.Option(
+        False,
+        "--save-result-job-specification",
+        help=(
+            "Save the resulting job specification JSON to the current working "
+            f"directory as {DEFAULT_RESULT_JOB_SPECIFICATION_FILENAME}."
+        ),
+    ),
 ):
     """Create HPC job"""
     try:
@@ -911,7 +1002,7 @@ def create_job(
             task_scalar_overrides={
                 "Name": (task_name, str, "--task-name"),
                 "MinCores": (task_min_cores, int, "--task-min-cores"),
-                "MaxCores": (max_cores, int, "--max-cores"),
+                "MaxCores": (task_max_cores, int, "--task-max-cores"),
                 "GpuCores": (task_gpu_cores, int, "--task-gpu-cores"),
                 "GpuNodes": (task_gpu_nodes, int, "--task-gpu-nodes"),
                 "WalltimeLimit": (walltime_limit, int, "--walltime-limit"),
@@ -992,10 +1083,16 @@ def create_job(
             cmd_template_parameters=cmdTemplateParameters,
         )
 
-        # check content of 'parsed_job_specification'
-        resultFilePath = Path("resultJobSpecJson.json")
-        dict_data = parsed_job_specification.to_dict()
-        resultFilePath.write_text(json.dumps(dict_data, indent=4))
+        if save_result_job_specification:
+            result_file_path = Path.cwd() / DEFAULT_RESULT_JOB_SPECIFICATION_FILENAME
+            result_file_path.write_text(
+                json.dumps(parsed_job_specification.to_dict(), indent=4),
+                encoding="utf-8",
+            )
+            utils.print_and_log(
+                "Resulting job specification was saved in: "
+                f"{result_file_path.resolve()}"
+            )
 
         body = {
             "_preload_content": False,

py4heappe-2.8.0/src/py4heappe/heappe_v6/cli/configuration.py

@@ -1,74 +0,0 @@
-import dotenv
-import os
-import typer
-import validators
-
-import py4heappe.heappe_v6.core as hp
-import py4heappe.heappe_v6.core.base.utils as utils
-from py4heappe.heappe_v6.core.base import exceptions
-from urllib.parse import urlparse
-
-app = typer.Typer(name="HEAppEConfigurationCLI", no_args_is_help=True, pretty_exceptions_show_locals=False)
-
-def get_api_instance():
-    dotenv.load_dotenv(os.path.join(os.path.dirname(__file__), '.env'))
-    url : str = os.environ.get("url")
-
-    if url is None:
-        raise exceptions.Py4HEAppEInternalException("Py4HEappE is not configured. Please check Conf option.") from None
-    
-    configuration = hp.Configuration()
-    configuration.host = url
-    api_instance = hp.ApiClient(configuration)
-    return api_instance
-
-def get_project_from_configuration():
-    dotenv.load_dotenv(os.path.join(os.path.dirname(__file__), '.env'))
-
-    computationalProject : str = os.environ.get("project")
-    if computationalProject is None:
-        raise exceptions.Py4HEAppEInternalException("Py4HEappE is not configured. Please check Conf option.") from None
-
-    return computationalProject
-
-@app.command(name="Init")
-def environment_preparation():
-    """Initialization"""
-    utils.print_and_log("Preparation of Py4HEAppE Environment ...")
-    dotenv.load_dotenv()
-
-    url: str = os.environ.get("url")
-    computationalProject: str = os.environ.get("project")
-
-    if url is not None or computationalProject is not None:
-        reconfigure = typer.confirm(f"Py4HEAppE is already configured (URL: \"{url}\", HEAppE accounting string (HPC Computational project): \"{computationalProject}\").\nDo you want to reconfigure it?")
-        if not reconfigure:
-            return
-
-    url = typer.prompt("Enter the HEAppE instance address (for example: https://heappe.domain.cz/production)")
-
-    normalized = url
-    parsed = urlparse(normalized)
-    if not parsed.scheme:
-        normalized = "http://" + normalized
-        parsed = urlparse(normalized)
-
-    host = parsed.hostname
-
-    if not validators.url(normalized) and host not in ("localhost", "127.0.0.1"):
-        raise exceptions.Py4HEAppEInternalException("The provided HEAppE URL is not valid.") from None
-
-    if "/swagger/index.html" in normalized:
-        normalized = normalized.replace("/swagger/index.html", "")
-
-    computationalProject = typer.prompt("Enter HEAppE accounting string", confirmation_prompt=True)
-
-    dotenv.set_key(os.path.join(os.path.dirname(__file__), '.env'), "url", normalized)
-    dotenv.set_key(os.path.join(os.path.dirname(__file__), '.env'), "project", computationalProject)
-
-    utils.print_and_log("Py4HEAppE is configured.")
-    return
-
-
-if __name__ == '__main__':
-    app()