yellowdog-python-examples 8.1.3__tar.gz → 8.1.5__tar.gz

{yellowdog_python_examples-8.1.3/yellowdog_python_examples.egg-info → yellowdog_python_examples-8.1.5}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: yellowdog-python-examples
-Version: 8.1.3
+Version: 8.1.5
 Summary: Python CLI commands using the YellowDog Python SDK
 Author-email: YellowDog Limited <support@yellowdog.co>
 License-Expression: Apache-2.0
@@ -26,7 +26,7 @@ Requires-Dist: requests
 Requires-Dist: rich==13.9.4
 Requires-Dist: tabulate>=0.9.0
 Requires-Dist: toml
-Requires-Dist: yellowdog-sdk>=11.5.0
+Requires-Dist: yellowdog-sdk>=11.6.0
 Provides-Extra: jsonnet
 Requires-Dist: jsonnet; extra == "jsonnet"
 Provides-Extra: cloudwizard

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/README.md

@@ -1693,7 +1693,7 @@ The following properties are available:
 |:------------------------|:-------------------------------------------------------------------------------------------------------------------------------------------|:------------------------|
 | `idleNodeTimeout`       | The timeout in minutes after which an idle node will be shut down. Set this to `0` to disable the timeout.                                 | `5.0`                   |
 | `idlePoolTimeout`       | The timeout in minutes after which an idle Worker Pool will be shut down. Set this to `0` to disable the timeout.                          | `30.0`                  |
-| `imagesId`              | The image ID, Image Family ID, Image Family name, or Image Group name to use when booting instances.                                       |                         |
+| `imagesId`              | The Image ID, Image Family ID, Image Family name, or Image Group name to use when booting instances.                                       |                         |
 | `instanceTags`          | The dictionary of instance tags to apply to the instances. Tag names must be lower case.                                                   |                         |
 | `maintainInstanceCount` | Only used when instantiating Compute Requirements; attempt to maintain the requested number of instances.                                  | `False`                 |
 | `maxNodes`              | The maximum number of nodes to which the Worker Pool can be scaled up.                                                                     | `1`                     |
@@ -1717,7 +1717,7 @@ The following properties are available:
 
 The `templateId` property can be directly populated with the YellowDog ID (YDID), or it can be populated with the textual name of the template, in the form `namespace/template_name`.
 
-Similarly, the `imagesId` property can be populated with the YDID of an Image Family, Image Group, Image, or a string representing the native name of a cloud provider image (e.g., an AWS AMI). It can also be populated with an Image Family name in the form `namespace/image_family_name`, or an Image Group name in the form `namespace/image_family_name/image_group_name`.
+Similarly, the `imagesId` property can be populated with the YDID of an Image Family, Image Group, Image, or a string representing the native name of a cloud provider image (e.g., an AWS AMI). It can also be populated with an Image Family name in the form `namespace/image_family_name`, or an Image Group name in the form `namespace/image_family_name/image_group_name` or `image_family_name/image_group_name`. Optionally, a `yd/` prefix can be supplied. The CLI will aim to map the provided name into an Image Family or Group YDID.
 
 ## Automatic Properties
 
@@ -2171,7 +2171,7 @@ An example Compute Source resource specification is found below:
 }
 ```
 
-In the Compute Source Template `imageId` property, an Image Family name **namespace/family-name** or Image Group name **namespace/family-name/group-name** may be used instead of an ID. For example: `"imageId": "yellowdog/yd-agent-docker"`. The `yd-create` command will look up the Image Family name and substitute its ID. A **`yd/`** prefix may also optionally be used.
+In the Compute Source Template `imageId` property, an Image Family name **namespace/family-name** or Image Group name **namespace/family-name/group-name** may be used instead of an ID. For example: `"imageId": "yellowdog/yd-agent-docker"`. The `yd-create` command will look up the Image Family name and substitute with a well-formed name or ID. A **`yd/`** prefix may also optionally be used.
 
 ## Compute Requirement Templates
 
@@ -2197,7 +2197,7 @@ An example Compute Requirement resource specification is found below, for a **st
 
 Note that Compute Source Template **namespace/names** in the form `namespace/compute_source_template_name` can be used instead of their IDs: the **yd-create** command will look up the IDs and make the substitutions. The Compute Source Templates must already exist.
 
-Also, In the `imagesId` property, an Image Family name **namespace/family-name** or an Image Group name **namespace/family-name/group-name** may be used instead of an ID. For example: `"imagesId": "yellowdog/yd-agent-docker/latest"`. The `yd-create` command will look up the Image Family name and substitute its ID. A **`yd/`** prefix may also optionally be used.
+Also, In the `imagesId` property, an Image Family name **namespace/family-name** or an Image Group name **namespace/family-name/group-name** may be used instead of an ID. For example: `"imagesId": "yellowdog/yd-agent-docker/latest"`. The `yd-create` command will look up the Image Family name and substitute with a well-formed name or ID. A **`yd/`** prefix may also optionally be used.
 
 A **dynamic** template example is:
 

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/requirements.txt

@@ -5,4 +5,4 @@ requests
 rich == 13.9.4
 tabulate >= 0.9.0
 toml
-yellowdog-sdk >= 11.5.0
+yellowdog-sdk >= 11.6.0

yellowdog_python_examples-8.1.5/yellowdog_cli/__init__.py

@@ -0,0 +1 @@
+__version__ = "8.1.5"

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/yellowdog_cli/create.py

@@ -50,10 +50,10 @@ from yellowdog_cli.utils.entity_utils import (
     clear_application_caches,
     clear_compute_source_template_cache,
     clear_group_caches,
-    clear_image_family_search_cache,
+    clear_image_caches,
     find_compute_requirement_template_id_by_name,
     find_compute_source_template_id_by_name,
-    find_image_family_or_group_id_by_name,
+    find_image_name_or_id,
     get_application_groups,
     get_application_id_by_name,
     get_group_id_by_name,
@@ -130,7 +130,9 @@ from yellowdog_cli.utils.wrapper import ARGS_PARSER, CLIENT, CONFIG_COMMON, main
 from yellowdog_cli.utils.ydid_utils import YDIDType, get_ydid_type
 
 CLEAR_CST_CACHE: bool = False  # Track whether the CST cache needs to be cleared
-CLEAR_IMAGE_FAMILY_CACHE: bool = False  # Track whether the IF cache needs to be cleared
+CLEAR_IMAGE_FAMILY_CACHE: bool = (
+    False  # Track whether the image caches need to be cleared
+)
 
 
 @main_wrapper
@@ -233,7 +235,7 @@ def create_compute_source_template(resource: Dict):
     # Allow image families (etc.) to be referenced by name rather than ID
     global CLEAR_IMAGE_FAMILY_CACHE
     if CLEAR_IMAGE_FAMILY_CACHE:  # Update the IF cache if required
-        clear_image_family_search_cache()
+        clear_image_caches()
         CLEAR_IMAGE_FAMILY_CACHE = False
 
     # Google CSTs use property name 'image' instead of 'imageId'
@@ -244,17 +246,14 @@ def create_compute_source_template(resource: Dict):
         else PROP_IMAGE
     )
 
-    image_id = source.get(image_property_name)
-    if get_ydid_type(image_id) not in [
-        YDIDType.IMAGE_FAMILY,
-        YDIDType.IMAGE_GROUP,
-        YDIDType.IMAGE,
-    ]:
-        image_family_id = find_image_family_or_group_id_by_name(
-            client=CLIENT, image_family_name=image_id
-        )
-        if image_family_id is not None:
-            source[image_property_name] = image_family_id
+    image_id = find_image_name_or_id(
+        client=CLIENT,
+        image_name_or_id=source.get(image_property_name),
+        always_return_id=False,
+        report_substitutions=True,
+    )
+    if image_id is not None:
+        source[image_property_name] = image_id
 
     if ARGS_PARSER.dry_run:
         resource[PROP_SOURCE] = source
@@ -322,31 +321,26 @@ def create_compute_requirement_template(resource: Dict):
     # Allow image families to be referenced by name rather than ID
     global CLEAR_IMAGE_FAMILY_CACHE
     if CLEAR_IMAGE_FAMILY_CACHE:  # Update the IF cache if required
-        clear_compute_source_template_cache()
+        clear_image_caches()
         CLEAR_IMAGE_FAMILY_CACHE = False
 
-    def _get_images_id(image_str: str, context: Dict, key: str) -> int:
+    def _get_images_id(image_str: str, context: Dict, key: str):
         """
-        Helper function to match an image family name into an ID.
+        Helper function to resolve an image ID.
         """
-        if get_ydid_type(image_str) not in [
-            YDIDType.IMAGE_FAMILY,
-            YDIDType.IMAGE_GROUP,
-            YDIDType.IMAGE,
-        ]:
-            image_family_id = find_image_family_or_group_id_by_name(
-                client=CLIENT, image_family_name=image_str
-            )
-            if image_family_id is not None:
-                context[key] = image_family_id
-                return 1
-        return 0
+        images_id_ = find_image_name_or_id(
+            client=CLIENT,
+            image_name_or_id=image_str,
+            always_return_id=False,
+            report_substitutions=True,
+        )
+        if images_id_ is not None:
+            context[key] = images_id_
 
     # Prepend the namespace when searching for existing templates
     name = f"{namespace}{NAMESPACE_PREFIX_SEPARATOR}{name}"
 
     source_template_substitutions = 0
-    source_image_id_substitutions = 0
 
     # Dynamic templates don't have 'sources'; return '[]'
     for source in resource.get(PROP_SOURCES, []):
@@ -365,16 +359,12 @@ def create_compute_requirement_template(resource: Dict):
 
         source_image_id = source.get(PROP_IMAGE_ID)
         if source_image_id is not None:
-            source_image_id_substitutions += _get_images_id(
-                source_image_id, source, PROP_IMAGE_ID
-            )
+            _get_images_id(source_image_id, source, PROP_IMAGE_ID)
 
     if source_template_substitutions > 0:
         print_log(
             f"Replaced {source_template_substitutions} Compute Source Template name(s) with ID(s)"
         )
-    if source_image_id_substitutions > 0:
-        print_log(f"Replaced {source_image_id_substitutions} Image name(s) with ID(s)")
 
     images_id = resource.get(PROP_IMAGES_ID)
     if images_id is not None:
@@ -549,7 +539,7 @@ def create_image_family(resource):
             # This will create the Image Family and all of its constituent
             # Image Group/Image resources
             image_family = _create_image_family(image_family, fq_name)
-            print_log(f"Created Machine Image Family '{fq_name}' ('{image_family.id}')")
+            print_log(f"Created Machine Image Family '{fq_name}' ({image_family.id})")
             if ARGS_PARSER.quiet:
                 print(image_family.id)
         else:

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/yellowdog_cli/instantiate.py

@@ -30,7 +30,7 @@ from yellowdog_cli.utils.printing import (
     print_yd_object,
 )
 from yellowdog_cli.utils.provision_utils import (
-    get_image_family_id,
+    get_image_id,
     get_template_id,
     get_user_data_property,
 )
@@ -92,8 +92,8 @@ def main():
 
     # Allow use of IF name instead of ID
     if CONFIG_WP.images_id is not None:
-        CONFIG_WP.images_id = get_image_family_id(
-            client=CLIENT, image_family_id_or_name=CONFIG_WP.images_id
+        CONFIG_WP.images_id = get_image_id(
+            client=CLIENT, image_name_or_id=CONFIG_WP.images_id
         )
 
     if not ARGS_PARSER.report:
@@ -303,8 +303,8 @@ def _create_compute_requirement_from_json(
 
     # Allow use of IF name instead of ID
     if cr_data.get("imagesId") is not None:
-        cr_data["imagesId"] = get_image_family_id(
-            client=CLIENT, image_family_id_or_name=cr_data["imagesId"]
+        cr_data["imagesId"] = get_image_id(
+            client=CLIENT, image_name_or_id=cr_data["imagesId"]
         )
 
     if ARGS_PARSER.dry_run:

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/yellowdog_cli/provision.py

@@ -47,7 +47,7 @@ from yellowdog_cli.utils.property_names import (
     WORKER_TAG,
 )
 from yellowdog_cli.utils.provision_utils import (
-    get_image_family_id,
+    get_image_id,
     get_template_id,
     get_user_data_property,
 )
@@ -170,7 +170,7 @@ def create_worker_pool_from_json(wp_json_file: str) -> None:
 
         # Allow Image Family name to be used instead of ID
         if reqt_template_usage.get(IMAGES_ID) is not None:
-            reqt_template_usage[IMAGES_ID] = get_image_family_id(
+            reqt_template_usage[IMAGES_ID] = get_image_id(
                 CLIENT, reqt_template_usage[IMAGES_ID]
             )
 
@@ -289,8 +289,8 @@ def create_worker_pool_from_toml():
 
     # Allow the Image Family name to be used instead of ID
     if CONFIG_WP.images_id is not None:
-        CONFIG_WP.images_id = get_image_family_id(
-            client=CLIENT, image_family_id_or_name=CONFIG_WP.images_id
+        CONFIG_WP.images_id = get_image_id(
+            client=CLIENT, image_name_or_id=CONFIG_WP.images_id
         )
 
     node_boot_timeout = (

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/yellowdog_cli/remove.py

@@ -207,17 +207,17 @@ def remove_keyring(resource: Dict):
         print_error(f"Expected property to be defined ({e})")
         return
 
-    if not confirmed(f"Delete Keyring '{name}'?"):
+    if not confirmed(f"Remove Keyring '{name}'?"):
         return
 
     try:
         CLIENT.keyring_client.delete_keyring_by_name(name)
-        print_log(f"Deleted Keyring '{name}'")
+        print_log(f"Removed Keyring '{name}'")
     except HTTPError as e:
         if e.response.status_code == 404:
-            print_error(f"Keyring '{name}' not found")
+            print_error(f"Cannot find Keyring '{name}'")
         else:
-            print_error(f"Unable to delete Keyring '{name}': {e}")
+            print_error(f"Unable to remove Keyring '{name}': {e}")
 
 
 def remove_credential(resource: Dict):
@@ -246,7 +246,7 @@ def remove_credential(resource: Dict):
     except HTTPError as e:
         if e.response.status_code == 404:
             print_error(
-                f"Keyring '{keyring_name}' not found (possibly already deleted,"
+                f"Cannot find Keyring '{keyring_name}'(possibly already deleted,"
                 " including its credentials?)"
             )
         else:
@@ -275,7 +275,7 @@ def remove_image_family(resource: Dict):
         )
     except HTTPError as e:
         if e.response.status_code == 404:
-            print_error(f"Machine Image Family '{fq_name}' not found")
+            print_error(f"Cannot find Machine Image Family '{fq_name}'")
             return
         else:
             raise e
@@ -285,9 +285,9 @@ def remove_image_family(resource: Dict):
 
     try:
         CLIENT.images_client.delete_image_family(image_family)
-        print_log(f"Deleted Image Family '{fq_name}' ('{image_family.id}')")
+        print_log(f"Removed Image Family '{fq_name}' ({image_family.id})")
     except Exception as e:
-        print_error(f"Unable to delete Image Family '{fq_name}': {e}")
+        print_error(f"Unable to remove Image Family '{fq_name}': {e}")
 
 
 def remove_namespace_configuration(resource: Dict):
@@ -304,7 +304,7 @@ def remove_namespace_configuration(resource: Dict):
         CLIENT.object_store_client.get_namespace_storage_configurations()
     )
     if namespace not in [x.namespace for x in namespaces]:
-        print_error(f"Namespace Storage Configuration '{namespace}' not found")
+        print_error(f"Cannot find Namespace Storage Configuration '{namespace}'")
         return
 
     if not confirmed(f"Remove Namespace Storage Configuration '{namespace}'?"):
@@ -428,7 +428,7 @@ def remove_resource_by_id(resource_id: str):
                         CLIENT.keyring_client.delete_keyring_by_name(keyring.name)
                         print_log(f"Removed Keyring {resource_id}")
                         return
-                raise Exception(f"Keyring {resource_id} not found")
+                raise Exception(f"Cannot find Keyring {resource_id}")
 
         elif get_ydid_type(resource_id) == YDIDType.WORKER_POOL:
             if confirmed(f"Shut down Worker Pool {resource_id}?"):
@@ -497,7 +497,7 @@ def remove_namespace_policy(resource: Dict):
             return
     except Exception:
         # Assume it's not found ... 404 from API
-        print_error(f"Namespace Policy '{namespace}' not found")
+        print_error(f"Cannot find Namespace Policy '{namespace}'")
         return
 
     if not confirmed(f"Remove Namespace Policy '{namespace}'?"):
@@ -522,7 +522,7 @@ def remove_group(resource: Dict):
 
     group_id = get_group_id_by_name(CLIENT, group_name)
     if group_id is None:
-        print_warning(f"Group '{group_name}' not found")
+        print_warning(f"Cannot find Group '{group_name}'")
         return
 
     if not confirmed(f"Remove Group '{group_name}' ({group_id})?"):
@@ -533,7 +533,7 @@ def remove_group(resource: Dict):
         print_log(f"Removed Group '{group_name}' ({group_id})")
         clear_group_caches()
     except Exception as e:
-        print_error(f"Unable to delete Group '{group_name}' ({group_id}): {e}")
+        print_error(f"Unable to remove Group '{group_name}' ({group_id}): {e}")
 
 
 def remove_application(resource: Dict):
@@ -548,7 +548,7 @@ def remove_application(resource: Dict):
 
     app_id = get_application_id_by_name(CLIENT, app_name)
     if app_id is None:
-        print_warning(f"Application '{app_name}' not found")
+        print_warning(f"Cannot find Application '{app_name}'")
         return
 
     if not confirmed(f"Remove Application '{app_name}' ({app_id})?"):
@@ -574,7 +574,7 @@ def remove_namespace(resource: Dict):
 
     namespace_id = get_namespace_id_by_name(CLIENT, name)
     if namespace_id is None:
-        print_warning(f"Namespace '{name}' not found")
+        print_warning(f"Cannot find Namespace '{name}'")
         return
 
     if not confirmed(f"Remove Namespace '{name}'?"):

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/yellowdog_cli/utils/entity_utils.py

@@ -24,6 +24,7 @@ from yellowdog_client.model import (
     ExternalUser,
     GroupSearch,
     GroupSummary,
+    ImageAccess,
     InternalUser,
     MachineImageFamily,
     MachineImageFamilySearch,
@@ -53,7 +54,7 @@ from yellowdog_client.model import (
 
 from yellowdog_cli.utils.args import ARGS_PARSER
 from yellowdog_cli.utils.interactive import confirmed, select
-from yellowdog_cli.utils.printing import print_log, print_warning
+from yellowdog_cli.utils.printing import print_log
 from yellowdog_cli.utils.settings import NAMESPACE_PREFIX_SEPARATOR
 from yellowdog_cli.utils.ydid_utils import YDIDType, get_ydid_type
 
@@ -94,7 +95,6 @@ def get_filtered_work_requirements(
     Get a list of Work Requirements filtered by namespace, tag
     and status. Supply either include_filter OR exclude_filter.
     """
-
     if include_filter is None:
         wr_search = WorkRequirementSearch(namespaces=[namespace], tag=tag)
     else:
@@ -231,6 +231,7 @@ def _find_id_by_name(
     )
 
 
+@lru_cache
 def find_compute_source_template_id_by_name(
     client: PlatformClient, name: str
 ) -> Optional[str]:
@@ -238,7 +239,12 @@ def find_compute_source_template_id_by_name(
     Find a Compute Source Template id by name.
     Compute Source Template names are unique within a namespace.
     """
-    return _find_id_by_name(name, client, get_compute_source_templates)
+    template_id = _find_id_by_name(name, client, get_compute_source_templates)
+    if template_id is not None:
+        print_log(
+            f"Replaced Compute Source Template name '{name}' with ID {template_id}"
+        )
+    return template_id
 
 
 @lru_cache
@@ -277,8 +283,10 @@ def get_work_requirement_summaries(
 def clear_compute_source_template_cache():
     """
     Clear the cache of Compute Source Templates.
+    Clear name -> CST lookups.
     """
     get_compute_source_templates.cache_clear()
+    find_compute_source_template_id_by_name.cache_clear()
 
 
 def find_compute_requirement_template_id_by_name(
@@ -349,146 +357,172 @@ def get_worker_pools(
 
 
 @lru_cache
-def find_image_family_or_group_id_by_name(
-    client: PlatformClient, image_family_name
+def find_image_name_or_id(
+    client: PlatformClient,
+    image_name_or_id: Optional[str],
+    always_return_id: bool = True,
+    report_substitutions: bool = True,
 ) -> Optional[str]:
     """
-    Resolve image family references. Complicated logic.
-    Fully qualified name is used for non-ambiguous PRIVATE image families.
-    Return None if mapping can't be done, to be handled by the caller.
+    Attempts to resolve to a well-formed YD image name or ID, if it can.
+
+    Argument 'image_name_or_id' can take one of the following forms:
+     - Any image family or group YDID (returned unchanged)
+     - Strings prefixed or not prefixed with 'yd/' (will be added if required)
+     - Strings post-fixed or not post-fixed with '/latest' (will be removed)
+     - Any standalone image-family-name
+     - Any namespace/image-family-name combination
+     - Any image-family-name/image-group-name combination
+     - Any namespace/image-family-name/image-group-name combination
+
+     The call will attempt to resolve the image into its fully qualified
+     name if 'always_return_id' is false:
+      - yd/namespace/image-family-name or
+      - yd/namespace/image-family-name/image-group-name
+
+    If the resolved image is PUBLIC or 'always_return_id' is True,
+    the relevant YDID will always be returned; this is enforced for
+    PUBLIC images.
+
+    Finally, if nothing matches, the original ID is returned. This is
+    likely to be a provider specific string.
     """
+    if image_name_or_id is None:
+        return None
 
-    if image_family_name.startswith("ami-") or image_family_name.startswith(
-        "ocid1.image."
-    ):
-        # We can identify AWS AMIs and OCI OCIDs, and return immediately
-        return image_family_name
+    # Already a matching YDID?
+    if get_ydid_type(image_name_or_id) in [
+        YDIDType.IMAGE_FAMILY,
+        YDIDType.IMAGE_GROUP,
+        YDIDType.IMAGE,
+    ]:
+        return image_name_or_id
 
-    original_image_family_name = image_family_name
+    original_image_name_or_id = image_name_or_id
 
-    # Remove leading 'yd/' prefix if necessary
-    image_family_name = (
-        image_family_name[3:]
-        if image_family_name.startswith("yd/")
-        else image_family_name
+    # Remove a leading 'yd/' prefix; will be reinstated later if required
+    image_name_or_id = (
+        image_name_or_id[3:] if image_name_or_id.startswith("yd/") else image_name_or_id
     )
 
-    # Handle image group names of form namespace/image-family-name/image-group-name
-    split_name = image_family_name.split("/")
-    if len(split_name) == 3:
-        try:
-            namespace, name = split_namespace_and_name(
-                f"{split_name[0]}/{split_name[1]}"
-            )
-            return client.images_client.get_image_group_by_name(
-                namespace=namespace, family_name=name, group_name=split_name[2]
-            ).id
-        except:
-            return
-
-    namespace, name = split_namespace_and_name(image_family_name)
-    if_search = MachineImageFamilySearch(
-        familyName=name,
-        namespaces=None if namespace in [None, ""] else [namespace],
-        includePublic=True,
+    # Remove "/latest"; this is redundant/implied for YD image groups
+    image_name_or_id = (
+        image_name_or_id[:-7]
+        if image_name_or_id.endswith("/latest")
+        else image_name_or_id
     )
-    search_client: SearchClient = client.images_client.get_image_families(if_search)
 
-    try:
-        image_families: List[MachineImageFamilySummary] = search_client.list_all()
-    except Exception as e:
-        if "MissingPermissionException" in str(e) and "IMAGE_READ" in str(e):
-            # IMAGE_READ permission (globally, or for this namespace) is absent;
-            # abandon lookup and return the original image family name
-            print_log(
-                f"Note: failed to resolve image name '{original_image_family_name}' "
-                "due to lack of 'IMAGE_READ' permission; using original name"
+    def _replaced(return_val: str, is_ydid: bool = False):
+        """
+        Helper function to report the replacement.
+        """
+        if report_substitutions and return_val != original_image_name_or_id:
+            msg = f"{return_val}" if is_ydid else f"'{return_val}'"
+            print_log(f"Replaced Images ID '{original_image_name_or_id}' with {msg}")
+        return return_val
+
+    split_name = image_name_or_id.split("/")
+    image_family_summaries = get_image_family_summaries(client)  # All namespaces
+
+    # Search for image name (only) matches
+    if len(split_name) == 1:
+        matching_image_families = [
+            ifs for ifs in image_family_summaries if ifs.name == split_name[0]
+        ]
+        if len(matching_image_families) > 1:
+            namespaces = [ifs.namespace for ifs in matching_image_families]
+            raise Exception(
+                f"Ambiguous Images ID '{original_image_name_or_id}': please "
+                f"specify a namespace from: {', '.join(namespaces)}"
             )
-            return original_image_family_name
-        else:
-            raise
+        elif len(matching_image_families) == 1:
+            if (
+                matching_image_families[0].access == ImageAccess.PUBLIC
+                or always_return_id
+            ):
+                return _replaced(matching_image_families[0].id, True)
+            else:
+                return _replaced(
+                    f"yd/{matching_image_families[0].namespace}/"
+                    f"{matching_image_families[0].name}"
+                )
 
-    # Partial names will match, so filter for exact matches only
-    image_families = [
-        img_family for img_family in image_families if img_family.name == name
-    ]
+    # Search for namespace/family_name matches, *or* family_name/group_name matches
+    if len(split_name) == 2:
+        # namespace/family-name match
 
-    # No matches
-    if len(image_families) == 0:
-        return
+        # This will be tidied up when the Application can
+        # query its properties
+        if len(image_family_summaries) == 0:  # Global search didn't work
+            image_family_summaries = get_image_family_summaries(client, split_name[0])
 
-    # It's possible to have both a PRIVATE and a PUBLIC match for the same
-    # namespace/image_family_name. This is a corner case, but ...
-    if len(image_families) == 2:
-        image_families_public = [
-            img_family
-            for img_family in image_families
-            if img_family.access.name == "PUBLIC"
+        matching_image_families = [
+            ifs
+            for ifs in image_family_summaries
+            if ifs.namespace == split_name[0] and ifs.name == split_name[1]
         ]
-        image_families_private = [
-            img_family
-            for img_family in image_families
-            if img_family.access.name == "PRIVATE"
-        ]
-        if len(image_families_public) == 1 and len(image_families_private) == 1:
-            # Favour the PRIVATE image
-            print_warning(
-                f"Image Family '{name}' has both PUBLIC and PRIVATE "
-                "variants; using the PRIVATE image family: "
-                f"{image_families_private[0].namespace}/{image_families_private[0].name} "
-                f"({image_families_private[0].id})"
+        if len(matching_image_families) == 1:
+            if (
+                matching_image_families[0].access == ImageAccess.PUBLIC
+                or always_return_id
+            ):
+                return _replaced(matching_image_families[0].id, True)
+            return _replaced(
+                f"yd/{matching_image_families[0].namespace}/"
+                f"{matching_image_families[0].name}"
             )
-            image_families = image_families_private
-
-    # Single match
-    if len(image_families) == 1:
-        substituted_image_family_name = (
-            f"yd/{image_families[0].namespace}/{image_families[0].name}"
-        )
 
-        # If this is a PRIVATE image family, we can retain the fully-qualified
-        # image family name instead of substituting the ID
-        if image_families[0].access.name == "PRIVATE":
-            if original_image_family_name != substituted_image_family_name:
-                print_log(
-                    f"Substituting Image Family name '{original_image_family_name}' "
-                    f"with fully qualified name '{substituted_image_family_name}' "
-                    f"({image_families[0].id})"
+        # family-name/group-name match
+        matching_image_families = [
+            ifs for ifs in image_family_summaries if ifs.name == split_name[0]
+        ]
+        if_group_matches: List[Tuple[MachineImageFamilySummary, MachineImageGroup]] = []
+        for ifs in matching_image_families:
+            for if_group in get_image_family_groups(client, ifs.id):
+                if if_group.name == split_name[1]:
+                    if_group_matches.append((ifs, if_group))
+                    break
+        if len(if_group_matches) == 1:
+            if if_group_matches[0][0].access == ImageAccess.PUBLIC or always_return_id:
+                return _replaced(if_group_matches[0][1].id, True)
+            else:
+                return _replaced(
+                    f"yd/{if_group_matches[0][0].namespace}/"
+                    f"{if_group_matches[0][0].name}/"
+                    f"{if_group_matches[0][1].name}"
                 )
-            return substituted_image_family_name
-
-        # If PUBLIC, we need to replace with the YDID
-        else:
-            mid_msg = (
-                ""
-                if original_image_family_name == substituted_image_family_name
-                else f"('{substituted_image_family_name}') "
+        if len(if_group_matches) > 1:
+            namespaces = [match[0].namespace for match in if_group_matches]
+            raise Exception(
+                f"Ambiguous image-family/image-group '{original_image_name_or_id}': "
+                f"please specify a namespace from: {', '.join(namespaces)}"
             )
-            print_log(
-                f"Substituting Image Family name '{original_image_family_name}' {mid_msg}"
-                f"with ID {image_families[0].id}"
-            )
-            return image_families[0].id
-
-    # Multiple matches
-    matches = [
-        f"{img_fam.namespace}/{img_fam.name} [{img_fam.access.name}] ({img_fam.id})"
-        for img_fam in image_families
-    ]
 
-    raise Exception(
-        f"Ambiguous Image Family name '{name}': "
-        f"{matches}. "
-        "Please specify a namespace. Note: PRIVATE image family is selected "
-        "over PUBLIC if namespace/name are identical."
-    )
+    # Search for names of form 'namespace/image-family-name/image-group-name'
+    # (the platform prevents duplicates)
+    if len(split_name) == 3:
 
-
-def clear_image_family_search_cache():
-    """
-    Clear the cache of Image Family name searches.
-    """
-    find_image_family_or_group_id_by_name.cache_clear()
+        # This will be tidied up when the Application can
+        # query its properties
+        if len(image_family_summaries) == 0:  # Global search didn't work
+            image_family_summaries = get_image_family_summaries(client, split_name[0])
+
+        for ifs in image_family_summaries:
+            if ifs.namespace == split_name[0] and ifs.name == split_name[1]:
+                for ig in get_image_family_groups(client, ifs.id):
+                    if ig.name == split_name[2]:
+                        if ifs.access == ImageAccess.PUBLIC or always_return_id:
+                            return _replaced(ig.id, True)
+                        else:
+                            return _replaced(f"yd/{image_name_or_id}")
+                else:
+                    raise Exception(
+                        "Image family found, but no matching image "
+                        f"group for '{original_image_name_or_id}'"
+                    )
+
+    # Finally, fall through and return the unchanged, original ID string
+    return original_image_name_or_id
 
 
 def remove_allowances_matching_description(
@@ -631,7 +665,6 @@ def substitute_image_family_id_for_name_in_cst(
     Substitute Image Family IDs for namespace/name,
     if option is selected.
     """
-
     if not ARGS_PARSER.substitute_ids:
         return cst
 
@@ -664,7 +697,6 @@ def substitute_id_for_name_in_allowance(
     """
     Substitute IDs in Allowance objects.
     """
-
     if not ARGS_PARSER.substitute_ids:
         return allowance
 
@@ -679,7 +711,6 @@ def substitute_id_for_name_in_allowance(
         )
 
     # No processing for other allowance types
-
     return allowance
 
 
@@ -963,3 +994,53 @@ def get_compute_requirement_summaries(
         client.compute_client.get_compute_requirement_summaries(crs_search)
     )
     return search_client.list_all()
+
+
+@lru_cache
+def get_image_family_summaries(
+    client: PlatformClient,
+    namespace: Optional[str] = None,
+) -> List[MachineImageFamilySummary]:
+    """
+    Obtain and cache the list of image families.
+    """
+    # Temporarily suppress most permission errors: will be improved
+    # once the application can be queried for its admissible
+    # IMAGE_READ namespaces
+    try:
+        if_search = MachineImageFamilySearch(
+            familyName=None,
+            namespaces=None if namespace is None else [namespace],
+            includePublic=True,
+        )
+        search_client: SearchClient = client.images_client.get_image_families(if_search)
+        return search_client.list_all()
+    except Exception as e:
+        if namespace is not None and "MissingPermissionException" in str(e):
+            # Caching will prevent this warning appearing multiple times
+            print_log(
+                "Warning: Possible 'IMAGE_READ' permission missing if "
+                f"'{namespace}' is meant as an Image namespace?"
+            )
+        pass
+
+    return []
+
+
+@lru_cache
+def get_image_family_groups(
+    client: PlatformClient, image_family_id: str
+) -> List[MachineImageGroup]:
+    """
+    Obtain and cache the list of image groups for an image family.
+    """
+    return client.images_client.get_image_family_by_id(image_family_id).imageGroups
+
+
+def clear_image_caches():
+    """
+    Clear the image caches.
+    """
+    find_image_name_or_id.cache_clear()
+    get_image_family_summaries.cache_clear()
+    get_image_family_groups.cache_clear()

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/yellowdog_cli/utils/printing.py

@@ -16,6 +16,7 @@ from typing import Any, Dict, List, Optional, Tuple, Union
 
 from rich.console import Console, Theme
 from rich.highlighter import JSONHighlighter, RegexHighlighter
+from rich.markup import escape
 from tabulate import tabulate
 from yellowdog_client import PlatformClient
 from yellowdog_client.common.json import Json
@@ -192,7 +193,7 @@ def print_simple(
     if ARGS_PARSER.quiet and override_quiet is False:
         return
 
-    CONSOLE.print(log_message)
+    CONSOLE.print(escape(log_message))
 
 
 def print_log(
@@ -211,7 +212,7 @@ def print_log(
         print(print_string(log_message, no_fill=no_fill), flush=True)
         return
 
-    CONSOLE.print(print_string(log_message, no_fill=no_fill))
+    CONSOLE.print(escape(print_string(log_message, no_fill=no_fill)))
 
 
 def print_error(error_obj: Union[Exception, str]):
@@ -222,7 +223,7 @@ def print_error(error_obj: Union[Exception, str]):
         print(print_string(f"Error: {error_obj}"), flush=True)
         return
 
-    CONSOLE_ERR.print(print_string(f"Error: {error_obj}"), style=ERROR_STYLE)
+    CONSOLE_ERR.print(escape(print_string(f"Error: {error_obj}")), style=ERROR_STYLE)
 
 
 def print_warning(
@@ -241,7 +242,8 @@ def print_warning(
         return
 
     CONSOLE.print(
-        print_string(f"Warning: {warning}", no_fill=no_fill), style=WARNING_STYLE
+        escape(print_string(f"Warning: {warning}", no_fill=no_fill)),
+        style=WARNING_STYLE,
     )
 
 
@@ -280,7 +282,7 @@ def print_table_core(table: str):
     if ARGS_PARSER.no_format or table.count("\n") > MAX_LINES_COLOURED_FORMATTING:
         print(table, flush=True)
     else:
-        CONSOLE_TABLE.print(table)
+        CONSOLE_TABLE.print(escape(table))
 
 
 def get_type_name(obj: Item) -> str:
@@ -1098,9 +1100,9 @@ def print_json(
 
     else:
         if with_final_comma:
-            CONSOLE_JSON.print(json_string, end=",\n", soft_wrap=True)
+            CONSOLE_JSON.print(escape(json_string), end=",\n", soft_wrap=True)
         else:
-            CONSOLE_JSON.print(json_string, soft_wrap=True)
+            CONSOLE_JSON.print(escape(json_string), soft_wrap=True)
 
     if ARGS_PARSER.output_file is not None:  # Also output to a nominated file
         print_to_file(

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/yellowdog_cli/utils/provision_utils.py

@@ -10,7 +10,7 @@ from yellowdog_client import PlatformClient
 from yellowdog_cli.utils.config_types import ConfigWorkerPool
 from yellowdog_cli.utils.entity_utils import (
     find_compute_requirement_template_id_by_name,
-    find_image_family_or_group_id_by_name,
+    find_image_name_or_id,
     split_namespace_and_name,
 )
 from yellowdog_cli.utils.load_config import CONFIG_FILE_DIR
@@ -85,16 +85,13 @@ def get_template_id(client: PlatformClient, template_id_or_name: str) -> str:
     if get_ydid_type(template_id_or_name) == YDIDType.COMPUTE_REQUIREMENT_TEMPLATE:
         return template_id_or_name
 
-    # If this is a fully qualified CRT name, allow server-side lookup
-    namespace, name = split_namespace_and_name(template_id_or_name)
-    if namespace is not None:
-        return template_id_or_name
-
     template_id = find_compute_requirement_template_id_by_name(
         client=client, name=template_id_or_name
     )
     if template_id is None:
-        return template_id_or_name  # Return the original input
+        raise Exception(
+            f"Compute Requirement Template '{template_id_or_name}' not found"
+        )
 
     print_log(
         f"Substituting Compute Requirement Template name '{template_id_or_name}'"
@@ -103,28 +100,10 @@ def get_template_id(client: PlatformClient, template_id_or_name: str) -> str:
     return template_id
 
 
-def get_image_family_id(client: PlatformClient, image_family_id_or_name: str) -> str:
+def get_image_id(client: PlatformClient, image_name_or_id: str) -> str:
     """
-    Check if 'image_id_or_name' looks like a valid IF ID; if not,
-    assume it's an IF name and perform a lookup.
+    This function was simplified, hence the pass-through call for now.
     """
-    if get_ydid_type(image_family_id_or_name) in [
-        YDIDType.IMAGE_FAMILY,
-        YDIDType.IMAGE_GROUP,
-        YDIDType.IMAGE,
-    ]:
-        return image_family_id_or_name
-
-    image_family_id = find_image_family_or_group_id_by_name(
-        client=client, image_family_name=image_family_id_or_name
+    return find_image_name_or_id(
+        client=client, image_name_or_id=image_name_or_id, always_return_id=True
     )
-
-    # If a specific image (e.g., an AMI) has been supplied, we'll get here
-    # and will return the original value. This does incur the cost of
-    # listing image families, and there's a small chance of a collision
-    # between a supplied specific image and an Image Family name.
-
-    if image_family_id is None:
-        return image_family_id_or_name  # Return the original input
-
-    return image_family_id

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5/yellowdog_python_examples.egg-info}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: yellowdog-python-examples
-Version: 8.1.3
+Version: 8.1.5
 Summary: Python CLI commands using the YellowDog Python SDK
 Author-email: YellowDog Limited <support@yellowdog.co>
 License-Expression: Apache-2.0
@@ -26,7 +26,7 @@ Requires-Dist: requests
 Requires-Dist: rich==13.9.4
 Requires-Dist: tabulate>=0.9.0
 Requires-Dist: toml
-Requires-Dist: yellowdog-sdk>=11.5.0
+Requires-Dist: yellowdog-sdk>=11.6.0
 Provides-Extra: jsonnet
 Requires-Dist: jsonnet; extra == "jsonnet"
 Provides-Extra: cloudwizard

{yellowdog_python_examples-8.1.3 → yellowdog_python_examples-8.1.5}/yellowdog_python_examples.egg-info/requires.txt

@@ -5,7 +5,7 @@ requests
 rich==13.9.4
 tabulate>=0.9.0
 toml
-yellowdog-sdk>=11.5.0
+yellowdog-sdk>=11.6.0
 
 [cloudwizard]
 boto3

yellowdog_python_examples-8.1.3/yellowdog_cli/__init__.py

@@ -1 +0,0 @@
-__version__ = "8.1.3"