yellowdog-python-examples 7.8.8__tar.gz → 7.9.1__tar.gz

{yellowdog_python_examples-7.8.8/yellowdog_python_examples.egg-info → yellowdog_python_examples-7.9.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: yellowdog-python-examples
-Version: 7.8.8
+Version: 7.9.1
 Summary: Example Python commands using the YellowDog Python SDK
 Home-page: https://github.com/yellowdog/python-examples
 Author: YellowDog Limited
@@ -43,6 +43,7 @@ The command scripts support:
 
 - **Provisioning** Worker Pools with the **`yd-provision`** command
 - **Submitting** Work Requirements with the **`yd-submit`** command
+- **Starting** HELD Work Requirements and **Holding** (or pausing) RUNNING Work Requirements with the **`yd-start`** and **`yd-hold`** commands
 - **Uploading** files to the YellowDog Object Store with the **`yd-upload`** command
 - **Instantiating** Compute Requirements with the **`yd-instantiate`** command
 - **Downloading** Results from the YellowDog Object Store with the **`yd-download`** command
@@ -52,7 +53,9 @@ The command scripts support:
 - **Terminating** Compute Requirements with the **`yd-terminate`** command
 - **Deleting** objects in the YellowDog Object Store with the **`yd-delete`** command
 - **Listing** YellowDog items using the **`yd-list`** command
-- **Resizing** Worker Pools and Compute Requirements
-- **Creating, Updating and Removing** Source Templates, Compute Templates, Keyrings, Credentials, Namespace Storage Configurations, Image Families, and Configured Worker Pools
+- **Resizing** Worker Pools and Compute Requirements with the **`yd-resize`** command
+- **Boosting** Allowances with the **`yd-boost`** command
+- **Creating, Updating and Removing** Source Templates, Compute Templates, Keyrings, Credentials, Namespace Storage Configurations, Image Families, Allowances, and Configured Worker Pools with the **`yd-create`** and **`yd-remove`** commands
+- **Following Event Streams** for Work Requirements, Worker Pools and Compute Requirements with the **`yd-follow`** command
 
-Please see the [GitHub repository](https://github.com/yellowdog/python-examples) for more details.
+Please see the documenation in the [GitHub repository](https://github.com/yellowdog/python-examples) for more details.

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/PYPI_README.md

@@ -8,6 +8,7 @@ The command scripts support:
 
 - **Provisioning** Worker Pools with the **`yd-provision`** command
 - **Submitting** Work Requirements with the **`yd-submit`** command
+- **Starting** HELD Work Requirements and **Holding** (or pausing) RUNNING Work Requirements with the **`yd-start`** and **`yd-hold`** commands
 - **Uploading** files to the YellowDog Object Store with the **`yd-upload`** command
 - **Instantiating** Compute Requirements with the **`yd-instantiate`** command
 - **Downloading** Results from the YellowDog Object Store with the **`yd-download`** command
@@ -17,7 +18,9 @@ The command scripts support:
 - **Terminating** Compute Requirements with the **`yd-terminate`** command
 - **Deleting** objects in the YellowDog Object Store with the **`yd-delete`** command
 - **Listing** YellowDog items using the **`yd-list`** command
-- **Resizing** Worker Pools and Compute Requirements
-- **Creating, Updating and Removing** Source Templates, Compute Templates, Keyrings, Credentials, Namespace Storage Configurations, Image Families, and Configured Worker Pools
+- **Resizing** Worker Pools and Compute Requirements with the **`yd-resize`** command
+- **Boosting** Allowances with the **`yd-boost`** command
+- **Creating, Updating and Removing** Source Templates, Compute Templates, Keyrings, Credentials, Namespace Storage Configurations, Image Families, Allowances, and Configured Worker Pools with the **`yd-create`** and **`yd-remove`** commands
+- **Following Event Streams** for Work Requirements, Worker Pools and Compute Requirements with the **`yd-follow`** command
 
-Please see the [GitHub repository](https://github.com/yellowdog/python-examples) for more details.
+Please see the documenation in the [GitHub repository](https://github.com/yellowdog/python-examples) for more details.

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/README.md

@@ -87,6 +87,9 @@
    * [Namespace Storage Configurations](#namespace-storage-configurations)
    * [Configured Worker Pools](#configured-worker-pools)
    * [Allowances](#allowances)
+   * [Attribute Definitions](#attribute-definitions)
+      * [String Attribute Definitions](#string-attribute-definitions)
+      * [Numeric Attribute Definitions](#numeric-attribute-definitions)
 * [Jsonnet Support](#jsonnet-support)
    * [Jsonnet Installation](#jsonnet-installation)
    * [Variable Substitutions in Jsonnet Files](#variable-substitutions-in-jsonnet-files)
@@ -114,7 +117,7 @@
    * [yd-boost](#yd-boost)
 
 <!-- Created by https://github.com/ekalinin/github-markdown-toc -->
-<!-- Added by: pwt, at: Fri Apr 19 11:43:37 BST 2024 -->
+<!-- Added by: pwt, at: Thu May  9 16:47:20 BST 2024 -->
 
 <!--te-->
 
@@ -156,6 +159,8 @@ Run any command with the `--help`/`-h` option to discover the command's options.
 
 # YellowDog Prerequisites
 
+(See also the quick start guides for [Provisioned](https://github.com/yellowdog/python-examples-templates/blob/main/quickstart.md) and [Configured](https://github.com/yellowdog/python-examples-templates/blob/main/quickstart-configured.md) Worker Pools.)
+
 To submit **Work Requirements** to YellowDog for processing by Configured Worker Pools (on-premise) and/or Provisioned Worker Pools (cloud-provisioned resources), you'll need:
 
 1. A YellowDog Platform Account.
@@ -176,7 +181,7 @@ To set up **Configured Worker Pools**, you'll need:
 5. A Configured Worker Pool Token: from the **Workers** tab in the [YellowDog Portal](https://portal.yellowdog.co/#/workers), use the **+Add Configured Worker Pool** button to create a new Worker Pool and generate a token.
 
 
-6. Obtain the YellowDog Agent and install/configure it on your on-premise systems using the Token above.
+6. Obtain the YellowDog Agent and install/configure it on your on-premise systems using the Token obtained above. See guidance for [Linux](https://github.com/yellowdog/resources/blob/main/agent-install/linux/README.md) and [Windows](https://github.com/yellowdog/resources/blob/main/agent-install/windows/README-CONFIGURED.md).
 
 # Script Installation with Pip
 
@@ -391,21 +396,25 @@ Substitutions can also be performed for non-string (number, boolean, array, and
 
 The following substitutions are automatically created and can be used in any section of the configuration file, or in any JSON specification:
 
-| Directive       | Description                                                              | Example of Substitution |
-|:----------------|:-------------------------------------------------------------------------|:------------------------|
-| `{{username}}`  | The current user's login username, lower case, spaces replaced           | jane_smith              |
-| `{{date}}`      | The current date (UTC): YYYYMMDD                                         | 20221027                |
-| `{{time}}`      | The current time (UTC): HHMMSSss                                         | 16302699                |
-| `{{datetime}}`  | Concatenation of the date and time, with a '-' separator                 | 20221027-163026         |
-| `{{random}}`    | A random, three digit hexadecimal number (lower case)                    | a1c                     |
-| `{{namespace}}` | The `namespace` property.                                                | my_namespace            |
-| `{{tag}}`       | The `tag` property.                                                      | my_tag                  |
-| `{{key}}`       | The application `key` property.                                          |                         |
-| `{{secret}}`    | The application `secret` property.                                       |                         |
-| `{{url}}`       | The Platform `url` property.                                             |                         |
+| Directive             | Description                                                    | Example of Substitution |
+|:----------------------|:---------------------------------------------------------------|:------------------------|
+| `{{username}}`        | The current user's login username, lower case, spaces replaced | jane_smith              |
+| `{{date}}`            | The current date (UTC): YYYYMMDD                               | 20221027                |
+| `{{time}}`            | The current time (UTC): HHMMSSss                               | 16302699                |
+| `{{datetime}}`        | Concatenation of the date and time, with a '-' separator       | 20221027-163026         |
+| `{{random}}`          | A random, three digit hexadecimal number (lower case)          | a1c                     |
+| `{{namespace}}`       | The `namespace` property.                                      | my_namespace            |
+| `{{tag}}`             | The `tag` property.                                            | my_tag                  |
+| `{{key}}`             | The application `key` property.                                |                         |
+| `{{secret}}`          | The application `secret` property.                             |                         |
+| `{{url}}`             | The Platform `url` property.                                   |                         |
+| `{{config_dir_abs}}`  | The absolute directory path of the configuration file          | /yellowdog/workloads    |
+| `{{config_dir_name}}` | The immediate containing directory of the configuration file   | workloads               |
 
 For the `date`, `time`, `datetime` and `random` directives, the same values will be used for the duration of a command -- i.e., if `{{time}}` is used within multiple properties, the identical value will be used for each substitution.
 
+The `config_dir_` substitutions use the name of the directory containing the nominated TOML configuration file, or the invocation directory if no configuration file is supplied.
+
 ## User-Defined Variables
 
 User-defined variables can be supplied using an option on the command line, or by setting environment variables prefixed with `YD_VAR_`, or by including the directives in the `[common]` section of the TOML configuration file.
@@ -413,7 +422,7 @@ User-defined variables can be supplied using an option on the command line, or b
 1. The **command line** option is `--variable` (or `-v`). For example, `yd-submit -v project_code=pr-213-a -v run_id=1234` will establish two new variables that can be used as `{{project_code}}` and `{{run_id}}`, which will be substituted by `pr-213-a` and `1234` respectively.
 
 
-2. For **environment variables**, setting the variable `YD_VAR_project_code="pr-213-a"` will create a new variable that can be used as `{{project_code}}`, which will be substituted by `pr-213-a`.
+2. For **environment variables**, setting the variable `YD_VAR_project_code="pr-213-a"` will create a new variable that can be used as `{{project_code}}`, which will be substituted by `pr-213-a`. Note that if running on Windows, all environment variable names are case-insensitive and converted to upper case, so choose upper case variable names only.
 
 
 3. For **setting within the TOML file**, include a **`variables`** table in the `[common]` section of the file. E.g., `variables = {project_code = "pr-213a", run_id = "1234"}`. Note that this can also use the form:
@@ -662,7 +671,7 @@ inputs = ["my_bash_script.sh"]
 arguments = ["{{wr_name}}/my_bash_script.sh", "1", "2", "3"]
 ```
 
-In the case of Windows batch (`.bat`) files, a `/c` flag is prepended to the `cmd.exe` argument list to ensure correct execution bahaviour. For example:
+In the case of Windows batch (`.bat`) files, a `/c` flag is prepended to the `cmd.exe` argument list to ensure correct execution behaviour. For example:
 
 ```toml
 taskType = "cmd"  # or "bat"
@@ -1023,11 +1032,10 @@ A simple example of the JSON output is shown below, showing a Work Requirement w
 `% yd-submit --dry-run --quiet`
 ```json
 {
-  "fulfilOnSubmit": false,
-  "name": "pyex-bash_230114-095504-53a",
-  "namespace": "pyexamples",
+  "name": "pyex-bash-pwt2_240424-12051160",
+  "namespace": "pyexamples-pwt2",
   "priority": 0,
-  "tag": "pyex-bash",
+  "tag": "pyex-bash-pwt2",
   "taskGroups": [
     {
       "finishIfAllTasksFinished": true,
@@ -1035,22 +1043,33 @@ A simple example of the JSON output is shown below, showing a Work Requirement w
       "name": "task_group_1",
       "priority": 0,
       "runSpecification": {
-        "maximumTaskRetries": 0,
+        "maximumTaskRetries": 5,
         "taskTypes": ["bash"],
-        "workerTags": ["pyex-bash"]
+        "workerTags": ["pyex-bash-pwt2-worker", "onpremise-pwt2"]
       },
+      "starved": false,
+      "waitingOnDependency": false,
       "tasks": [
         {
-          "arguments": ["pyex-bash_230114-095504-53a/sleep_script.sh"],
-          "environment": {},
+          "arguments": ["pyex-bash-pwt2_240424-12051160/sleep_script.sh", 1, 2, 3],
+          "environment": {
+            "TEST_ENV_1": "100",
+            "TEST_ENV_2": "200",
+            "YD_TASK_NAME": "task_1",
+            "YD_TASK_NUMBER": "1",
+            "YD_TASK_GROUP_NAME": "task_group_1",
+            "YD_TASK_GROUP_NUMBER": "1",
+            "YD_WORK_REQUIREMENT_NAME": "pyex-bash-pwt2_240424-12051160",
+            "YD_NAMESPACE": "pyexamples-pwt2"
+          },
           "inputs": [
             {
-              "objectNamePattern": "pyex-bash_230114-095504-53a/sleep_script.sh",
+              "objectNamePattern": "pyex-bash-pwt2_240424-12051160/sleep_script.sh",
               "source": "TASK_NAMESPACE",
-              "verification": "VERIFY_WAIT"
+              "verification": "VERIFY_AT_START"
             }
           ],
-          "name": "task_01",
+          "name": "task_1",
           "outputs": [
             {"alwaysUpload": true, "required": false, "source": "PROCESS_OUTPUT"}
           ],
@@ -1587,13 +1606,13 @@ Here's an example of the `workerPool` section of a TOML configuration file, show
     templateId = "ydid:crt:D9C548:465a107c-7cea-46e3-9fdd-15116cb92c40"
     # Note: only one of 'userData'/'userDataFile'/'userDataFiles' should be set
     userData = ""
-    userDataFile = "myuserdata.txt"
-    userDataFiles = ["myuserdata1.txt", "myuserdata2.txt"]
+    # userDataFile = "myuserdata.txt"
+    # userDataFiles = ["myuserdata1.txt", "myuserdata2.txt"]
     workerTag = "tag-{{username}}"
     # Specify either workersPerNode or workersPerVCPU
     workersPerNode = 1
-    workersPerVCPU = 1
-#   workerPoolData = "worker_pool.json"
+    # workersPerVCPU = 1
+    # workerPoolData = "worker_pool.json"  # Optionally specify worker pool JSON specification
 ```
 
 ## Worker Pool Specification Using JSON Documents
@@ -1782,6 +1801,7 @@ The commands **yd-create** and **yd-remove** allow the creation, update and remo
 - Namespace Storage Configurations
 - Configured Worker Pools
 - Allowances
+- String Attribute Definitions
 
 ## Overview of Operation
 
@@ -2145,6 +2165,44 @@ When using `yd-remove`, Allowances are again matched using their `description` p
 
 Allowances can be **boosted** (have extra hours added to the Allowance) using the `yd-boost` command.
 
+## Attribute Definitions
+
+The Attribute Definition example and schema can be found at: https://docs.yellowdog.co/api/?spec=Compute%20API#tag/compute/post/compute/attributes/user.
+
+### String Attribute Definitions
+
+Example:
+
+```json
+{
+  "resource": "StringAttributeDefinition",
+  "name": "user.my-attribute",
+  "title": "My attribute title",
+  "description": "This is a description of my attribute",
+  "options": ["yes", "no", "maybe"]
+}
+```
+
+The `name` and `title` properties are required, while the rest are optional. The `user.` prefix is required when specifying the `name` property.
+
+### Numeric Attribute Definitions
+
+Example:
+
+```json
+{
+  "resource": "NumericAttributeDefinition",
+  "name": "my-numeric-attribute",
+  "title": "Attribute Title",
+  "defaultRankOrder": "PREFER_LOWER",
+  "description": "A description of the attribute",
+  "units": "$",
+  "range": {"min": 1, "max": 10}
+}
+```
+
+The `name`, `title` and `defaultRankOrder` properties are required, while the rest are optional. The `user.` prefix is required when specifying the `name` property.
+
 # Jsonnet Support
 
 In all circumstances where JSON files are used by the Python Examples commands,  **[Jsonnet](https://jsonnet.org)** files can be used instead. This allows the use of Jsonnet's powerful JSON extensions, including comments, variables, functions, etc.

yellowdog_python_examples-7.9.1/yd_commands/__init__.py

@@ -0,0 +1 @@
+__version__ = "7.9.1"

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/args.py

@@ -529,6 +529,13 @@ class CLIParser:
                 required=False,
                 help="list allowances",
             )
+            parser.add_argument(
+                "--attribute-definitions",
+                "-R",
+                action="store_true",
+                required=False,
+                help="list attribute definitions",
+            )
 
         if any(module in sys.argv[0] for module in ["upload"]):
             parser.add_argument(
@@ -1310,6 +1317,11 @@ class CLIParser:
     def allowances(self) -> Optional[bool]:
         return self.args.allowances
 
+    @property
+    @allow_missing_attribute
+    def attribute_definitions(self) -> Optional[bool]:
+        return self.args.attribute_definitions
+
     @property
     @allow_missing_attribute
     def show_keyring_passwords(self) -> Optional[bool]:

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/create.py

@@ -10,7 +10,7 @@ from typing import Dict, List, Optional
 
 import yellowdog_client.model as model
 from dateparser import parse as date_parse
-from requests import post
+from requests import post, put
 from requests.exceptions import HTTPError
 from yellowdog_client.model import (
     AccountAllowance,
@@ -51,9 +51,11 @@ from yd_commands.settings import (
     RN_CREDENTIAL,
     RN_IMAGE_FAMILY,
     RN_KEYRING,
+    RN_NUMERIC_ATTRIBUTE_DEFINITION,
     RN_REQUIREMENT_TEMPLATE,
     RN_SOURCE_TEMPLATE,
     RN_STORAGE_CONFIGURATION,
+    RN_STRING_ATTRIBUTE_DEFINITION,
 )
 from yd_commands.wrapper import ARGS_PARSER, CLIENT, CONFIG_COMMON, main_wrapper
 
@@ -118,6 +120,11 @@ def create_resources(
                 create_configured_worker_pool(resource)
             elif resource_type == RN_ALLOWANCE:
                 create_allowance(resource)
+            elif resource_type in [
+                RN_STRING_ATTRIBUTE_DEFINITION,
+                RN_NUMERIC_ATTRIBUTE_DEFINITION,
+            ]:
+                create_attribute_definition_via_api(resource, resource_type)
             else:
                 print_error(f"Unknown resource type '{resource_type}'")
         except Exception as e:
@@ -789,6 +796,59 @@ def _get_model_class(class_name: str):
     return getattr(model, class_name)
 
 
+def create_attribute_definition_via_api(resource: Dict, resource_type: str):
+    """
+    Use the API to create/update user attribute definitions.
+    """
+    try:
+        name = resource["name"]
+        title = resource["title"]
+        if resource_type == RN_NUMERIC_ATTRIBUTE_DEFINITION:
+            default_rank_order = resource["defaultRankOrder"]
+    except KeyError as e:
+        raise Exception(f"Expected property to be defined ({e})")
+
+    url = f"{CONFIG_COMMON.url}/compute/attributes/user"
+    headers = {"Authorization": f"yd-key {CONFIG_COMMON.key}:{CONFIG_COMMON.secret}"}
+    if resource_type == RN_STRING_ATTRIBUTE_DEFINITION:
+        payload = {
+            "type": "co.yellowdog.platform.model.StringAttributeDefinition",
+            "name": name,
+            "title": title,
+            "description": resource.get("description"),
+            "options": resource.get("options"),
+        }
+    else:  # RN_NUMERIC_ATTRIBUTE_DEFINITION
+        payload = {
+            "type": "co.yellowdog.platform.model.NumericAttributeDefinition",
+            "name": name,
+            "title": title,
+            "defaultRankOrder": default_rank_order,
+            "description": resource.get("description"),
+            "units": resource.get("units"),
+            "range": resource.get("range"),
+        }
+
+    # Attempt attribute creation
+    print_log(f"Attempting to create or update Attribute Definition '{name}'")
+    response = post(url=url, headers=headers, json=payload)
+
+    if response.status_code == 200:
+        print_log(f"Created new Attribute Definition '{name}'")
+        return
+
+    if "Attribute already exists" in response.text:
+        if not confirmed(f"Update existing Attribute Definition '{name}'?"):
+            return
+
+        response = put(url=url, headers=headers, json=payload)
+        if response.status_code == 200:
+            print_log(f"Updated existing Attribute Definition '{name}'")
+            return
+
+    raise Exception(f"HTTP {response.status_code} ({response.text})")
+
+
 # Entry point
 if __name__ == "__main__":
     main()

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/interactive.py

@@ -3,7 +3,7 @@ User interaction processing utilities.
 """
 
 from os import getenv
-from typing import List, Optional, Set, Union
+from typing import Dict, List, Optional, Set, Union
 
 from yellowdog_client import PlatformClient
 
@@ -30,13 +30,14 @@ YD_YES = "YD_YES"
 
 def select(
     client: PlatformClient,
-    objects: List[Union[Item, str]],
+    objects: List[Union[Item, str, Dict]],
     object_type_name: Optional[str] = None,
     override_quiet: bool = False,
     single_result: bool = False,
     showing_all: bool = False,
     force_interactive: bool = False,
     result_required: bool = False,
+    sort_objects: bool = True,
 ) -> List[Item]:
     """
     Print a numbered list of objects.
@@ -47,7 +48,8 @@ def select(
     if len(objects) == 0:
         return objects
 
-    objects = sorted_objects(objects)
+    if sort_objects:
+        objects = sorted_objects(objects)
 
     if not ARGS_PARSER.quiet or override_quiet or ARGS_PARSER.interactive:
         print_numbered_object_list(

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/list.py

@@ -5,6 +5,7 @@ Command to list YellowDog entities.
 """
 
 from dataclasses import asdict, fields
+from json import loads as json_loads
 from typing import Dict, List
 
 from requests import get
@@ -47,6 +48,7 @@ from yd_commands.object_utilities import (
 )
 from yd_commands.printing import (
     indent,
+    print_json,
     print_log,
     print_numbered_object_list,
     print_table_core,
@@ -85,6 +87,8 @@ def main():
         list_namespaces()
     elif ARGS_PARSER.allowances:
         list_allowances()
+    elif ARGS_PARSER.attribute_definitions:
+        list_attribute_definitions()
 
 
 def check_for_valid_option() -> bool:
@@ -105,6 +109,7 @@ def check_for_valid_option() -> bool:
         ARGS_PARSER.namespace_storage_configurations,
         ARGS_PARSER.instances,
         ARGS_PARSER.allowances,
+        ARGS_PARSER.attribute_definitions,
     ].count(True) == 1:
         return True
     else:
@@ -568,6 +573,39 @@ def list_allowances():
         print_yd_object(allowance)
 
 
+def list_attribute_definitions():
+    """
+    List user compute attribute definitions using the API.
+    """
+    response = get(
+        url=f"{CONFIG_COMMON.url}/compute/attributes/user",
+        headers={"Authorization": f"yd-key {CONFIG_COMMON.key}:{CONFIG_COMMON.secret}"},
+    )
+
+    if response.status_code != 200:
+        raise Exception(
+            "Unable to list user attribute definitions: HTTP "
+            f"{response.status_code} ({response.text})"
+        )
+
+    attribute_definition_list = json_loads(response.text)
+    attribute_definition_list.sort(key=lambda x: x["name"])
+
+    if not ARGS_PARSER.details:
+        print_numbered_object_list(
+            CLIENT, attribute_definition_list, object_type_name="Attribute Definition"
+        )
+        return
+
+    for selected_attribute_definition in select(
+        CLIENT,
+        attribute_definition_list,
+        object_type_name="Attribute Definition",
+        sort_objects=False,
+    ):
+        print_json(selected_attribute_definition)
+
+
 # Entry point
 if __name__ == "__main__":
     main()

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/load_config.py

@@ -4,7 +4,8 @@ Common utility functions, mostly related to loading configuration data.
 
 import os
 from os import getenv
-from os.path import dirname, join, relpath
+from os.path import abspath, dirname, join, relpath
+from pathlib import Path
 from sys import exit
 from typing import Dict, Optional
 
@@ -49,6 +50,11 @@ CONFIG_FILE = relpath(
 
 try:
     CONFIG_FILE_DIR = dirname(CONFIG_FILE)
+    config_dir_abs = abspath(CONFIG_FILE_DIR)
+    config_dir_short = Path(config_dir_abs).parts[-1]
+    VARIABLE_SUBSTITUTIONS.update(
+        {"config_dir_abs": config_dir_abs, "config_dir_name": config_dir_short}
+    )
     print_log(f"Loading configuration data from: '{CONFIG_FILE}'")
     CONFIG_TOML: Dict = load_toml_file_with_variable_substitutions(CONFIG_FILE)
     try:

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/load_resources.py

@@ -13,9 +13,11 @@ from yd_commands.settings import (
     RN_CREDENTIAL,
     RN_IMAGE_FAMILY,
     RN_KEYRING,
+    RN_NUMERIC_ATTRIBUTE_DEFINITION,
     RN_REQUIREMENT_TEMPLATE,
     RN_SOURCE_TEMPLATE,
     RN_STORAGE_CONFIGURATION,
+    RN_STRING_ATTRIBUTE_DEFINITION,
 )
 from yd_commands.variables import (
     load_json_file_with_variable_substitutions,
@@ -94,6 +96,8 @@ def _resequence_resources(
         RN_KEYRING,
         RN_CREDENTIAL,
         RN_IMAGE_FAMILY,
+        RN_STRING_ATTRIBUTE_DEFINITION,
+        RN_NUMERIC_ATTRIBUTE_DEFINITION,
         RN_SOURCE_TEMPLATE,
         RN_REQUIREMENT_TEMPLATE,
         RN_ALLOWANCE,

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/printing.py

@@ -552,6 +552,30 @@ def allowances_table(
     return headers, table
 
 
+def attribute_definitions_table(
+    attribute_definitions: List[Dict],
+) -> (List[str], List[str]):
+    headers = [
+        "#",
+        "Name",
+        "Type",
+        "Title",
+        "Description",
+    ]
+    table = []
+    for index, attribute_definition in enumerate(attribute_definitions):
+        table.append(
+            [
+                index + 1,
+                attribute_definition["name"],
+                attribute_definition["type"].split(".")[-1],
+                attribute_definition["title"],
+                attribute_definition.get("description", ""),
+            ]
+        )
+    return headers, table
+
+
 def aws_availability_zone_table(
     aws_azs: List[AWSAvailabilityZone],
 ) -> (List[str], List[str]):
@@ -576,7 +600,7 @@ def aws_availability_zone_table(
 
 def print_numbered_object_list(
     client: PlatformClient,
-    objects: List[Union[Item, str]],
+    objects: List[Union[Item, str, Dict]],
     object_type_name: Optional[str] = None,
     override_quiet: bool = False,
     showing_all: bool = False,
@@ -626,6 +650,8 @@ def print_numbered_object_list(
         headers, table = allowances_table(objects)
     elif isinstance(objects[0], AWSAvailabilityZone):
         headers, table = aws_availability_zone_table(objects)
+    elif object_type_name == "Attribute Definition":
+        headers, table = attribute_definitions_table(objects)
     else:
         table = []
         for index, obj in enumerate(objects):

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/remove.py

@@ -7,6 +7,7 @@ A script to remove YellowDog resources.
 from copy import deepcopy
 from typing import Dict, List, Optional
 
+from requests import delete
 from requests.exceptions import HTTPError
 from yellowdog_client.model import (
     MachineImage,
@@ -31,11 +32,13 @@ from yd_commands.settings import (
     RN_CREDENTIAL,
     RN_IMAGE_FAMILY,
     RN_KEYRING,
+    RN_NUMERIC_ATTRIBUTE_DEFINITION,
     RN_REQUIREMENT_TEMPLATE,
     RN_SOURCE_TEMPLATE,
     RN_STORAGE_CONFIGURATION,
+    RN_STRING_ATTRIBUTE_DEFINITION,
 )
-from yd_commands.wrapper import ARGS_PARSER, CLIENT, main_wrapper
+from yd_commands.wrapper import ARGS_PARSER, CLIENT, CONFIG_COMMON, main_wrapper
 
 
 @main_wrapper
@@ -92,6 +95,11 @@ def remove_resources(resources: Optional[List[Dict]] = None):
                         "alternatively, Allowances can be removed by their "
                         "YellowDog IDs (yd-remove --ids)"
                     )
+            elif resource_type in [
+                RN_STRING_ATTRIBUTE_DEFINITION,
+                RN_NUMERIC_ATTRIBUTE_DEFINITION,
+            ]:
+                remove_attribute_definition_via_api(resource)
             else:
                 print_error(f"Unknown resource type '{resource_type}'")
         except Exception as e:
@@ -407,6 +415,29 @@ def remove_resource_by_id(resource_id: str):
         print_error(f"Unable to remove resource with ID {resource_id}: {e}")
 
 
+def remove_attribute_definition_via_api(resource: Dict):
+    """
+    Use the API to remove user string attribute definitions.
+    """
+    try:
+        name = resource["name"]
+    except KeyError as e:
+        raise Exception(f"Expected property to be defined ({e})")
+
+    if not confirmed(f"Remove Attribute Definition '{name}'?"):
+        return
+
+    url = f"{CONFIG_COMMON.url}/compute/attributes/user/{name}"
+    headers = {"Authorization": f"yd-key {CONFIG_COMMON.key}:{CONFIG_COMMON.secret}"}
+    response = delete(url=url, headers=headers)
+
+    if response.status_code == 200:
+        print_log(f"Removed Attribute Definition '{name}' (if present)")
+        return
+
+    raise Exception(f"HTTP {response.status_code} ({response.text})")
+
+
 # Entry point
 if __name__ == "__main__":
     main()

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/settings.py

@@ -105,3 +105,5 @@ RN_KEYRING = "Keyring"
 RN_REQUIREMENT_TEMPLATE = "ComputeRequirementTemplate"
 RN_SOURCE_TEMPLATE = "ComputeSourceTemplate"
 RN_STORAGE_CONFIGURATION = "NamespaceStorageConfiguration"
+RN_STRING_ATTRIBUTE_DEFINITION = "StringAttributeDefinition"
+RN_NUMERIC_ATTRIBUTE_DEFINITION = "NumericAttributeDefinition"

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1}/yd_commands/shutdown.py

@@ -32,9 +32,11 @@ def main():
         return
 
     print_log(
-        "Shutting down Worker Pools with Compute Requirements in "
+        "Shutting down Provisioned Worker Pools with Compute Requirements in "
         f"namespace '{CONFIG_COMMON.namespace}' and "
-        f"tag starting with '{CONFIG_COMMON.name_tag}'"
+        f"tag starting with '{CONFIG_COMMON.name_tag}' "
+        "(or Configured Worker Pools with names starting with "
+        f"'{CONFIG_COMMON.name_tag}')"
     )
     worker_pool_summaries: List[WorkerPoolSummary] = (
         CLIENT.worker_pool_client.find_all_worker_pools()
@@ -43,11 +45,19 @@ def main():
 
     selected_worker_pool_summaries: List[WorkerPoolSummary] = []
     for worker_pool_summary in worker_pool_summaries:
-        if not (
-            "ProvisionedWorkerPool" not in worker_pool_summary.type
-            or worker_pool_summary.status
-            in [WorkerPoolStatus.TERMINATED, WorkerPoolStatus.SHUTDOWN]
-        ):
+        if worker_pool_summary.status not in [
+            WorkerPoolStatus.TERMINATED,
+            WorkerPoolStatus.SHUTDOWN,
+        ]:
+            if "ProvisionedWorkerPool" not in worker_pool_summary.type:
+                # Configured Worker Pool: check worker pool name only
+                if (
+                    worker_pool_summary.name is not None
+                    and worker_pool_summary.name.startswith(CONFIG_COMMON.name_tag)
+                ):
+                    selected_worker_pool_summaries.append(worker_pool_summary)
+                continue
+
             worker_pool: WorkerPool = get_worker_pool_by_id(
                 CLIENT, worker_pool_summary.id
             )

{yellowdog_python_examples-7.8.8 → yellowdog_python_examples-7.9.1/yellowdog_python_examples.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.1
 Name: yellowdog-python-examples
-Version: 7.8.8
+Version: 7.9.1
 Summary: Example Python commands using the YellowDog Python SDK
 Home-page: https://github.com/yellowdog/python-examples
 Author: YellowDog Limited
@@ -43,6 +43,7 @@ The command scripts support:
 
 - **Provisioning** Worker Pools with the **`yd-provision`** command
 - **Submitting** Work Requirements with the **`yd-submit`** command
+- **Starting** HELD Work Requirements and **Holding** (or pausing) RUNNING Work Requirements with the **`yd-start`** and **`yd-hold`** commands
 - **Uploading** files to the YellowDog Object Store with the **`yd-upload`** command
 - **Instantiating** Compute Requirements with the **`yd-instantiate`** command
 - **Downloading** Results from the YellowDog Object Store with the **`yd-download`** command
@@ -52,7 +53,9 @@ The command scripts support:
 - **Terminating** Compute Requirements with the **`yd-terminate`** command
 - **Deleting** objects in the YellowDog Object Store with the **`yd-delete`** command
 - **Listing** YellowDog items using the **`yd-list`** command
-- **Resizing** Worker Pools and Compute Requirements
-- **Creating, Updating and Removing** Source Templates, Compute Templates, Keyrings, Credentials, Namespace Storage Configurations, Image Families, and Configured Worker Pools
+- **Resizing** Worker Pools and Compute Requirements with the **`yd-resize`** command
+- **Boosting** Allowances with the **`yd-boost`** command
+- **Creating, Updating and Removing** Source Templates, Compute Templates, Keyrings, Credentials, Namespace Storage Configurations, Image Families, Allowances, and Configured Worker Pools with the **`yd-create`** and **`yd-remove`** commands
+- **Following Event Streams** for Work Requirements, Worker Pools and Compute Requirements with the **`yd-follow`** command
 
-Please see the [GitHub repository](https://github.com/yellowdog/python-examples) for more details.
+Please see the documenation in the [GitHub repository](https://github.com/yellowdog/python-examples) for more details.

yellowdog_python_examples-7.8.8/yd_commands/__init__.py

@@ -1 +0,0 @@
-__version__ = "7.8.8"