cs-image-system-aws-runtime-plugin 0.1.4__tar.gz → 0.1.5__tar.gz

cs_image_system_aws_runtime_plugin-0.1.4/README.md → cs_image_system_aws_runtime_plugin-0.1.5/PKG-INFO

@@ -1,3 +1,23 @@
+Metadata-Version: 2.5
+Name: cs-image-system-aws-runtime-plugin
+Version: 0.1.5
+Summary: The aws runtime type: Amazon EC2 as a place where images bake and instances run
+Project-URL: Repository, https://github.com/infrastructurebuilder/cs-image-system-3
+Project-URL: Documentation, https://github.com/infrastructurebuilder/cs-image-system-3/blob/develop/docs/OPERATIONS.md
+Project-URL: Package README, https://github.com/infrastructurebuilder/cs-image-system-3/blob/develop/packages/aws-runtime-plugin/README.md
+Author-email: Mykel Alvis <mykelalvis@infrastructurebuilder.org>
+License-Expression: Apache-2.0
+License-File: LICENSE
+Requires-Python: >=3.13
+Requires-Dist: boto3>=1.43
+Requires-Dist: botocore>=1.43
+Requires-Dist: cs-image-system-base==0.1.5
+Requires-Dist: cs-image-system-contract==0.1.5
+Requires-Dist: cs-image-system-hashicorp-utils==0.1.5
+Requires-Dist: pydantic>=2.13
+Requires-Dist: python-hcl2>=8.1
+Description-Content-Type: text/markdown
+
 # cs-image-system-aws-runtime-plugin
 
 The `aws` runtime type. A runtime is where images are baked and instances
@@ -56,9 +76,13 @@ Present in the package but not registered as a service:
 cs_image_system.aws_runtime.bootstrap:aws_section`. It owns the questions
 and the terraform for this cloud's one-time initialisation -- the GitHub
 OIDC identity provider, the READ-ONLY and WRITE roles of CI_SETUP.md 3.3
-with their trust and permission documents, and the state bucket and the
-Session Manager instance profile when the account lacks them
-(`tfmodules/bootstrap_aws`). The section is wanted by default where the
+with their trust and permission documents, and the state buckets (the
+default backend's, and since hygiene XX every other declared S3
+backend's that a root is bound to) and the Session Manager instance
+profile when the account lacks them (`tfmodules/bootstrap_aws`). The
+section renders its coverage audit for the generated README: what it
+made, read, adopted, or leaves by hand and why (a backend bound to no
+root is left out and said so). The section is wanted by default where the
 tree declares an AWS runtime or an S3 state backend. Its defaults come
 from the raw `cfg/runtime-builders.yml` and `cfg/state-backends.yml`
 (read as YAML, never loaded; a starter's `REPLACE-ME` values and its
@@ -287,27 +311,64 @@ included).
 
 `query_provider_image()` takes one OS-builder runtime entry
 (`OSBuilderBaseImageBuilderSubconfig` in
-[os_builder_runtime_config.py](../base/src/cs_image_system/base/models/os_builder_runtime_config.py)).
-`remap_for_image_query()` builds a `describe_images` request:
-
-- `Owners` is the entry's `get_owners()`: the OS builder's owners, then the runtime's `default_owners`, then the entry's own, deduplicated in order (the runtime's were never found before stage 63 item 22).
-- `query.filters` keys are mapped through `AWS_DI_MAP` (snake_case to EC2 filter names, `tag:<key>` kept), booleans lowercased, and `state: available` forced.
-- Keys the map does not know go to a post-query exact-match filter on the result dictionaries.
-- The rewrite works on a deep copy of the entry's `query` (stage 63). `get_query()` hands back the entry's own nested mappings, and until 2026-09-25 the rewrite injected `state: available` into the OS builder entry's own `filters` mapping on every resolution, so the loaded model changed each time an image was resolved. The entry is now left exactly as declared.
-
+[os_builder_runtime_config.py](../base/src/cs_image_system/base/models/os_builder_runtime_config.py))
+and asks it for its effective query, `entry.image_query()`: an
+`ImageQuery` in the SYSTEM's vocabulary
+([contract/image.py](../contract/src/cs_image_system/contract/image.py);
+stage 106). That query is the image-source catalog's `aws` block for the
+OS builder's family (the release's layer under the tree's
+`cfg/image-sources*.yml`), with the OS builder's own `query` landing on it
+key by key and the runtime entry's landing on that; the publishers declared
+anywhere (the OS builder's `owners`, this runtime's `default_owners`, the
+entry's `owners`, in that order, deduplicated) replace the catalog's. The
+entry's own `query` mapping is never rewritten (stage 63). The builder then
+translates it with `translate_image_query()` (the table below) and
 `query_image()` in [aws_utils.py](src/cs_image_system/aws_runtime/aws_utils.py)
-runs that `DescribeImages` query (paging through `describe_images`), keeps
-every match that agrees with the post-query filter, and returns the newest
-by `CreationDate`, or `None` when nothing matches. Its docstring says so
-since stage 63; the unreachable `raise` that used to follow the return is
-gone. The builder then reads the owner straight from the AMI record,
-`ImageOwnerAlias` (for example `amazon`) else `OwnerId` (the account id),
-and returns `(ImageId, owner, raw)`; the base registers an
-`AwsProviderSpecificImage` in the resolved state. A record with neither
-field raises `Could not get owner alias or owner id ...`. Before stage 63 a
-second, unreachable `Could not get owner` branch followed that one; it was
-removed on 2026-09-25. No match returns `None`, and the base's resolve step
-then fails the run.
+runs the `DescribeImages` request `describe_images_request()` makes from
+it, paging through the results, and returns the newest by `CreationDate`,
+or `None` when nothing matches; with `newest: false` more than one match
+raises `AMIQueryError` as ambiguous. The builder reads the owner straight
+from the AMI record, `ImageOwnerAlias` (for example `amazon`) else
+`OwnerId` (the account id), and returns `(ImageId, owner, raw)`; the base
+registers an `AwsProviderSpecificImage` in the resolved state. A record
+with neither field raises `Could not get owner alias or owner id ...`. No
+match returns `None`, and the base's resolve step then fails the run.
+`validate` has already refused an entry whose query uses a key this
+runtime cannot translate, or has no source at all, and has printed every
+entry's query in both vocabularies (`image source: OS builder '<name>' on
+runtime <rt> (aws): <system query> -> <EC2 query>`).
+
+Before stage 106 the entry's `query.filters` were EC2's field names in
+snake case (the system started on AWS and used its filter keys: the
+operator's incidental choice), passed through a key map, with unknown keys
+going to a post-query exact-match filter. The vocabulary is the system's
+now, EC2's spellings kept where they were good names; the old spellings
+still read (`root_device_type`, `virtualization_type`, `owners`, the
+nested `filters:` form), and an unknown key is refused by name instead of
+buried.
+
+### Image query translation
+
+What `translate_image_query()` does with each key of the system's
+vocabulary (`supported_query_keys()` is the set of rows not marked
+refused; [tests/test_docs_image_sources.py](../../tests/test_docs_image_sources.py)
+holds this table to it). The answer is `{"owners": [...], "filters":
+{<EC2 filter name>: value}, "most_recent": bool}`: what packer's
+`data "amazon-ami"` body and `describe_images` both take.
+
+| System key | EC2 filter | Note |
+| --- | --- | --- |
+| `name` | `name` | a glob over the AMI name |
+| `family` | refused | EC2 has no image families; name the image |
+| `version` | refused | the catalog fills `{version}` into `name` |
+| `architecture` | `architecture` | `x86_64` or `arm64`, EC2's own words |
+| `publisher` | `owners` | account ids and the aliases `self`, `amazon`, `aws-marketplace` |
+| `state` | `state` | `available` when unset: the query always forced it |
+| `virtualization` | `virtualization-type` | `hvm` |
+| `root_device` | `root-device-type` | `ebs` |
+| `tags` | `tag:<key>` | one filter per tag |
+| `newest` | `most_recent` | true when unset; false refuses more than one match |
+| `native` | verbatim | each key an EC2 filter name, passed as written |
 
 When the entry declares `image_id` (stage 63), the base's resolve step
 does not call `query_provider_image()` at all: it calls
@@ -315,29 +376,26 @@ does not call `query_provider_image()` at all: it calls
 with `ImageIds=[<id>]` through `_query_by_ami_id` in
 [aws_utils.py](src/cs_image_system/aws_runtime/aws_utils.py), with the
 runtime's own client configuration (profile and region). The entry's
-`owners` and `query` are not consulted. The owner is read from the record
-the same way, `ImageOwnerAlias` else `OwnerId` (else `self`), and the
-result is `(ImageId, owner, raw)`. An id AWS answers
-`InvalidImageID.NotFound` or `InvalidImageID.Malformed` for, or one that
-returns no image, gives `None`, and the base stops resolution with
-`OS builder <name>: image_id '<id>' on runtime <rt> is not known to the
-provider`; any other API refusal raises `AMIQueryError`. An AMI id is
-regional, so a pinned id must exist in this runtime's region. Pinning
-makes the base bake reproducible: the same vendor AMI every time, where
-the query takes the newest match.
-
-Stage 63 also removed helpers from
-[aws_utils.py](src/cs_image_system/aws_runtime/aws_utils.py) that nothing
-called: `get_ami_owner`, `get_ami_ssh_user`, `remap_for_aws`,
-`_query_by_name` and `_build_ami_filters`. `_query_by_ami_id` (an exact
-lookup by AMI id) was kept, and since stage 63 it is what
-`query_provider_image_by_id()` calls.
+query is not consulted. The owner is read from the record the same way,
+`ImageOwnerAlias` else `OwnerId` (else `self`), and the result is
+`(ImageId, owner, raw)`. An id AWS answers `InvalidImageID.NotFound` or
+`InvalidImageID.Malformed` for, or one that returns no image, gives
+`None`, and the base stops resolution with `OS builder <name>: image_id
+'<id>' on runtime <rt> is not known to the provider`; any other API
+refusal raises `AMIQueryError`. An AMI id is regional, so a pinned id must
+exist in this runtime's region. Pinning makes the base bake reproducible:
+the same vendor AMI every time, where the query takes the newest match.
 
 `AwsProviderSpecificImage.get_query_assets()` gives the packer
 `data "amazon-ami"` body:
 
 - resolved: `filters = { image-id = "<ami>" }` plus `owners = ["<owner>"]` when known;
-- deferred (an image a build produces later): `owners = ["self"]`, `filters = { name = "<pattern>*", root-device-type = "ebs", virtualization-type = "hvm", architecture = "<arch or x86_64>" }`, `most_recent = true`.
+- deferred (an image a build produces later): an `ImageQuery` (`name =
+  "<pattern>*"`, `root_device = ebs`, `virtualization = hvm`, the
+  architecture or `x86_64`, publisher `self`) translated like a vendor
+  query and quoted: `owners = ["self"]`, `filters = { name = "<pattern>*",
+  root-device-type = "ebs", virtualization-type = "hvm", architecture =
+  "<arch>" }`, `most_recent = true`.
 
 [tests/test_aws_provider_specific_image.py](tests/test_aws_provider_specific_image.py)
 pins both shapes.

cs_image_system_aws_runtime_plugin-0.1.4/PKG-INFO → cs_image_system_aws_runtime_plugin-0.1.5/README.md

@@ -1,23 +1,3 @@
-Metadata-Version: 2.5
-Name: cs-image-system-aws-runtime-plugin
-Version: 0.1.4
-Summary: The aws runtime type: Amazon EC2 as a place where images bake and instances run
-Project-URL: Repository, https://github.com/infrastructurebuilder/cs-image-system-3
-Project-URL: Documentation, https://github.com/infrastructurebuilder/cs-image-system-3/blob/develop/docs/OPERATIONS.md
-Project-URL: Package README, https://github.com/infrastructurebuilder/cs-image-system-3/blob/develop/packages/aws-runtime-plugin/README.md
-Author-email: Mykel Alvis <mykelalvis@infrastructurebuilder.org>
-License-Expression: Apache-2.0
-License-File: LICENSE
-Requires-Python: >=3.13
-Requires-Dist: boto3>=1.43
-Requires-Dist: botocore>=1.43
-Requires-Dist: cs-image-system-base==0.1.4
-Requires-Dist: cs-image-system-contract==0.1.4
-Requires-Dist: cs-image-system-hashicorp-utils==0.1.4
-Requires-Dist: pydantic>=2.13
-Requires-Dist: python-hcl2>=8.1
-Description-Content-Type: text/markdown
-
 # cs-image-system-aws-runtime-plugin
 
 The `aws` runtime type. A runtime is where images are baked and instances
@@ -76,9 +56,13 @@ Present in the package but not registered as a service:
 cs_image_system.aws_runtime.bootstrap:aws_section`. It owns the questions
 and the terraform for this cloud's one-time initialisation -- the GitHub
 OIDC identity provider, the READ-ONLY and WRITE roles of CI_SETUP.md 3.3
-with their trust and permission documents, and the state bucket and the
-Session Manager instance profile when the account lacks them
-(`tfmodules/bootstrap_aws`). The section is wanted by default where the
+with their trust and permission documents, and the state buckets (the
+default backend's, and since hygiene XX every other declared S3
+backend's that a root is bound to) and the Session Manager instance
+profile when the account lacks them (`tfmodules/bootstrap_aws`). The
+section renders its coverage audit for the generated README: what it
+made, read, adopted, or leaves by hand and why (a backend bound to no
+root is left out and said so). The section is wanted by default where the
 tree declares an AWS runtime or an S3 state backend. Its defaults come
 from the raw `cfg/runtime-builders.yml` and `cfg/state-backends.yml`
 (read as YAML, never loaded; a starter's `REPLACE-ME` values and its
@@ -307,27 +291,64 @@ included).
 
 `query_provider_image()` takes one OS-builder runtime entry
 (`OSBuilderBaseImageBuilderSubconfig` in
-[os_builder_runtime_config.py](../base/src/cs_image_system/base/models/os_builder_runtime_config.py)).
-`remap_for_image_query()` builds a `describe_images` request:
-
-- `Owners` is the entry's `get_owners()`: the OS builder's owners, then the runtime's `default_owners`, then the entry's own, deduplicated in order (the runtime's were never found before stage 63 item 22).
-- `query.filters` keys are mapped through `AWS_DI_MAP` (snake_case to EC2 filter names, `tag:<key>` kept), booleans lowercased, and `state: available` forced.
-- Keys the map does not know go to a post-query exact-match filter on the result dictionaries.
-- The rewrite works on a deep copy of the entry's `query` (stage 63). `get_query()` hands back the entry's own nested mappings, and until 2026-09-25 the rewrite injected `state: available` into the OS builder entry's own `filters` mapping on every resolution, so the loaded model changed each time an image was resolved. The entry is now left exactly as declared.
-
+[os_builder_runtime_config.py](../base/src/cs_image_system/base/models/os_builder_runtime_config.py))
+and asks it for its effective query, `entry.image_query()`: an
+`ImageQuery` in the SYSTEM's vocabulary
+([contract/image.py](../contract/src/cs_image_system/contract/image.py);
+stage 106). That query is the image-source catalog's `aws` block for the
+OS builder's family (the release's layer under the tree's
+`cfg/image-sources*.yml`), with the OS builder's own `query` landing on it
+key by key and the runtime entry's landing on that; the publishers declared
+anywhere (the OS builder's `owners`, this runtime's `default_owners`, the
+entry's `owners`, in that order, deduplicated) replace the catalog's. The
+entry's own `query` mapping is never rewritten (stage 63). The builder then
+translates it with `translate_image_query()` (the table below) and
 `query_image()` in [aws_utils.py](src/cs_image_system/aws_runtime/aws_utils.py)
-runs that `DescribeImages` query (paging through `describe_images`), keeps
-every match that agrees with the post-query filter, and returns the newest
-by `CreationDate`, or `None` when nothing matches. Its docstring says so
-since stage 63; the unreachable `raise` that used to follow the return is
-gone. The builder then reads the owner straight from the AMI record,
-`ImageOwnerAlias` (for example `amazon`) else `OwnerId` (the account id),
-and returns `(ImageId, owner, raw)`; the base registers an
-`AwsProviderSpecificImage` in the resolved state. A record with neither
-field raises `Could not get owner alias or owner id ...`. Before stage 63 a
-second, unreachable `Could not get owner` branch followed that one; it was
-removed on 2026-09-25. No match returns `None`, and the base's resolve step
-then fails the run.
+runs the `DescribeImages` request `describe_images_request()` makes from
+it, paging through the results, and returns the newest by `CreationDate`,
+or `None` when nothing matches; with `newest: false` more than one match
+raises `AMIQueryError` as ambiguous. The builder reads the owner straight
+from the AMI record, `ImageOwnerAlias` (for example `amazon`) else
+`OwnerId` (the account id), and returns `(ImageId, owner, raw)`; the base
+registers an `AwsProviderSpecificImage` in the resolved state. A record
+with neither field raises `Could not get owner alias or owner id ...`. No
+match returns `None`, and the base's resolve step then fails the run.
+`validate` has already refused an entry whose query uses a key this
+runtime cannot translate, or has no source at all, and has printed every
+entry's query in both vocabularies (`image source: OS builder '<name>' on
+runtime <rt> (aws): <system query> -> <EC2 query>`).
+
+Before stage 106 the entry's `query.filters` were EC2's field names in
+snake case (the system started on AWS and used its filter keys: the
+operator's incidental choice), passed through a key map, with unknown keys
+going to a post-query exact-match filter. The vocabulary is the system's
+now, EC2's spellings kept where they were good names; the old spellings
+still read (`root_device_type`, `virtualization_type`, `owners`, the
+nested `filters:` form), and an unknown key is refused by name instead of
+buried.
+
+### Image query translation
+
+What `translate_image_query()` does with each key of the system's
+vocabulary (`supported_query_keys()` is the set of rows not marked
+refused; [tests/test_docs_image_sources.py](../../tests/test_docs_image_sources.py)
+holds this table to it). The answer is `{"owners": [...], "filters":
+{<EC2 filter name>: value}, "most_recent": bool}`: what packer's
+`data "amazon-ami"` body and `describe_images` both take.
+
+| System key | EC2 filter | Note |
+| --- | --- | --- |
+| `name` | `name` | a glob over the AMI name |
+| `family` | refused | EC2 has no image families; name the image |
+| `version` | refused | the catalog fills `{version}` into `name` |
+| `architecture` | `architecture` | `x86_64` or `arm64`, EC2's own words |
+| `publisher` | `owners` | account ids and the aliases `self`, `amazon`, `aws-marketplace` |
+| `state` | `state` | `available` when unset: the query always forced it |
+| `virtualization` | `virtualization-type` | `hvm` |
+| `root_device` | `root-device-type` | `ebs` |
+| `tags` | `tag:<key>` | one filter per tag |
+| `newest` | `most_recent` | true when unset; false refuses more than one match |
+| `native` | verbatim | each key an EC2 filter name, passed as written |
 
 When the entry declares `image_id` (stage 63), the base's resolve step
 does not call `query_provider_image()` at all: it calls
@@ -335,29 +356,26 @@ does not call `query_provider_image()` at all: it calls
 with `ImageIds=[<id>]` through `_query_by_ami_id` in
 [aws_utils.py](src/cs_image_system/aws_runtime/aws_utils.py), with the
 runtime's own client configuration (profile and region). The entry's
-`owners` and `query` are not consulted. The owner is read from the record
-the same way, `ImageOwnerAlias` else `OwnerId` (else `self`), and the
-result is `(ImageId, owner, raw)`. An id AWS answers
-`InvalidImageID.NotFound` or `InvalidImageID.Malformed` for, or one that
-returns no image, gives `None`, and the base stops resolution with
-`OS builder <name>: image_id '<id>' on runtime <rt> is not known to the
-provider`; any other API refusal raises `AMIQueryError`. An AMI id is
-regional, so a pinned id must exist in this runtime's region. Pinning
-makes the base bake reproducible: the same vendor AMI every time, where
-the query takes the newest match.
-
-Stage 63 also removed helpers from
-[aws_utils.py](src/cs_image_system/aws_runtime/aws_utils.py) that nothing
-called: `get_ami_owner`, `get_ami_ssh_user`, `remap_for_aws`,
-`_query_by_name` and `_build_ami_filters`. `_query_by_ami_id` (an exact
-lookup by AMI id) was kept, and since stage 63 it is what
-`query_provider_image_by_id()` calls.
+query is not consulted. The owner is read from the record the same way,
+`ImageOwnerAlias` else `OwnerId` (else `self`), and the result is
+`(ImageId, owner, raw)`. An id AWS answers `InvalidImageID.NotFound` or
+`InvalidImageID.Malformed` for, or one that returns no image, gives
+`None`, and the base stops resolution with `OS builder <name>: image_id
+'<id>' on runtime <rt> is not known to the provider`; any other API
+refusal raises `AMIQueryError`. An AMI id is regional, so a pinned id must
+exist in this runtime's region. Pinning makes the base bake reproducible:
+the same vendor AMI every time, where the query takes the newest match.
 
 `AwsProviderSpecificImage.get_query_assets()` gives the packer
 `data "amazon-ami"` body:
 
 - resolved: `filters = { image-id = "<ami>" }` plus `owners = ["<owner>"]` when known;
-- deferred (an image a build produces later): `owners = ["self"]`, `filters = { name = "<pattern>*", root-device-type = "ebs", virtualization-type = "hvm", architecture = "<arch or x86_64>" }`, `most_recent = true`.
+- deferred (an image a build produces later): an `ImageQuery` (`name =
+  "<pattern>*"`, `root_device = ebs`, `virtualization = hvm`, the
+  architecture or `x86_64`, publisher `self`) translated like a vendor
+  query and quoted: `owners = ["self"]`, `filters = { name = "<pattern>*",
+  root-device-type = "ebs", virtualization-type = "hvm", architecture =
+  "<arch>" }`, `most_recent = true`.
 
 [tests/test_aws_provider_specific_image.py](tests/test_aws_provider_specific_image.py)
 pins both shapes.

{cs_image_system_aws_runtime_plugin-0.1.4 → cs_image_system_aws_runtime_plugin-0.1.5}/pyproject.toml

@@ -4,7 +4,7 @@
 
 [project]
 name = "cs-image-system-aws-runtime-plugin"
-version = "0.1.4"
+version = "0.1.5"
 description = "The aws runtime type: Amazon EC2 as a place where images bake and instances run"
 readme = "README.md"
 authors = [{ name = "Mykel Alvis", email = "mykelalvis@infrastructurebuilder.org" }]
@@ -12,9 +12,9 @@ license = "Apache-2.0"
 license-files = ["LICENSE"]
 requires-python = ">=3.13"
 dependencies = [
-    "cs-image-system-contract==0.1.4",
-    "cs-image-system-base==0.1.4",
-    "cs-image-system-hashicorp-utils==0.1.4",
+    "cs-image-system-contract==0.1.5",
+    "cs-image-system-base==0.1.5",
+    "cs-image-system-hashicorp-utils==0.1.5",
     "boto3>=1.43",
     "botocore>=1.43",
     "python-hcl2>=8.1",

{cs_image_system_aws_runtime_plugin-0.1.4 → cs_image_system_aws_runtime_plugin-0.1.5}/src/cs_image_system/aws_runtime/aws_provider_specific_image.py

@@ -28,16 +28,13 @@ class AwsProviderSpecificImage(ProviderSpecificImage):
                 ret["owners"] = [f'"{self.owner}"']
             return ret
         # Deferred: the image is produced by a build in this account; find the
-        # most recent image matching the output-name pattern (same shape as the
-        # chained-image lookup in the packer-ebs builder).
-        architecture = self.architecture if self.architecture else "x86_64"
-        return {
-            "owners": ['"self"'],
-            "filters": {
-                "name": f'"{self.deferred_name_pattern}*"',
-                "root-device-type": '"ebs"',
-                "virtualization-type": '"hvm"',
-                "architecture": f'"{architecture}"',
-            },
-            "most_recent": True,
-        }
+        # most recent image matching the output-name pattern. stage 106: the
+        # lookup is an ImageQuery in the system's vocabulary, translated like
+        # a vendor query, then quoted for the packer data source.
+        from cs_image_system.contract.image import ImageQuery
+        from .aws_utils import translate_image_query
+        native = translate_image_query(ImageQuery(
+            name=f"{self.deferred_name_pattern}*", root_device="ebs", virtualization="hvm",
+            architecture=self.architecture if self.architecture else "x86_64", publisher=("self",), newest=True))
+        filters = {k: f'"{v}"' for k, v in native["filters"].items() if k != "state"}
+        return {"owners": [f'"{o}"' for o in native["owners"]], "filters": filters, "most_recent": True}

{cs_image_system_aws_runtime_plugin-0.1.4 → cs_image_system_aws_runtime_plugin-0.1.5}/src/cs_image_system/aws_runtime/aws_runtime_builders.py

@@ -19,7 +19,8 @@ from cs_image_system.contract.core import PluginArtifactProtocol
 
 from .aws_provider_specific_image import AwsProviderSpecificImage
 from .aws_runtime_models import AWS_CLI, AwsCloudBuilderModel
-from .aws_utils import remap_for_image_query, query_image
+from .aws_utils import AWS_QUERY_KEYS, query_image, translate_image_query
+from cs_image_system.contract.image import ImageQuery
 from cs_image_system.aws_runtime import aws_utils
 
 log = logging.getLogger(__name__)
@@ -43,12 +44,18 @@ class AwsCloudBuilder(CloudBuilderBase[AwsCloudBuilderModel], PluginArtifactProt
     def provider_specific_image_class(self) -> type[ProviderSpecificImage]:
         return AwsProviderSpecificImage
 
+    # --- the image query (stage 106)
+    def supported_query_keys(self) -> frozenset[str]:
+        return AWS_QUERY_KEYS
+
+    def translate_image_query(self, query: ImageQuery) -> dict[str, Any]:
+        return translate_image_query(query)
+
     def query_provider_image(self, os_builder: OSBuilderBaseImageBuilderSubconfig) -> tuple[str, str, Mapping[str,Any]] | None:
         retval: str | None = None
-        q,missed = remap_for_image_query(os_builder) if os_builder else ({}, {})
+        q = self.translate_image_query(os_builder.image_query()) if os_builder else {}
         kv = query_image(
                 query=q,
-                post_query_filter=missed,
                 session_config=self.model.self_to_aws_client_config(),
             )
         if kv:

{cs_image_system_aws_runtime_plugin-0.1.4 → cs_image_system_aws_runtime_plugin-0.1.5}/src/cs_image_system/aws_runtime/aws_utils.py

@@ -6,14 +6,14 @@
 
 from __future__ import annotations
 
-import copy
 import logging
-from typing import Any, Sequence, cast
+from typing import Any, Mapping, cast
 import boto3  # type: ignore[import-untyped]
 from botocore.client import BaseClient as EC2  # type: ignore[import-untyped]
 from botocore.exceptions import BotoCoreError, ClientError
 
-from cs_image_system.base.models.os_builder_runtime_config import OSBuilderBaseImageBuilderSubconfig
+from cs_image_system.contract.image import ImageQuery
+
 
 log = logging.getLogger(__name__)
 
@@ -120,81 +120,75 @@ AWS_DI_MAP: dict[str, str] = {
 }
 
 
-def remap_for_image_query(rc: OSBuilderBaseImageBuilderSubconfig) -> tuple[dict[str, Any], dict[str, Any]]:
-    ret: dict[str, Any] = {
-        "DryRun": False,
-        "IncludeDisabled": False,
-        "IncludeDeprecated": False,
-    }
-    owners: Sequence[Any] = rc.get_owners() if rc else []
-    ret["Owners"] = owners # completely overridden if specified in config
-    missed: dict[str, Any] = {}
-    # stage 63: a deep copy -- the loop below rewrites the filters mapping in
-    # place, and get_query() hands back the entry's own nested dicts, so the
-    # model's query used to be mutated by every resolution
-    for topk, topv in copy.deepcopy(dict(rc.get_query())).items():
-        _key2 = AWS_DI_MAP.get(topk, None)
-        if not _key2:
-            missed[topk] = topv
-            continue
-        topk = _key2
-        if topk == "Owners":
-            if isinstance(topv, list):
-                topv = [str(i) for i in topv]
-            else:
-                topv = [str(topv)]
-        elif topk == "Filters":
-            if isinstance(topv, dict):
-                topv["state"] = (
-                    "available"  # Ensure we only get available images, but allow override
-                )
-                new_dict: dict[str, Any] = {}
-                for _k, _v in topv.items():
-                    _k = AWS_DI_MAP.get(_k, _k)
-                    assert _k is not None, "Bad Filter key is None"
-                    if _k == "Owners":
-                        if isinstance(_v, list):
-                            _v = [str(i) for i in _v]
-                        else:
-                            _v = [str(_v)]
-                    if _k.startswith("tag:"):
-                        _k = f"tag:{_k[4:]}"
-                    if isinstance(_v, bool):
-                        _v = str(_v).lower()
-                    new_dict[_k] = _v
-                v = new_dict
-                topv = [
-                    {
-                        "Name": AWS_DI_MAP.get(_k, _k),
-                        "Values": _v if isinstance(_v, list) else [str(_v)],
-                    }
-                    for _k, _v in v.items()
-                ]
-            else:
-                raise ValueError(f"Expected 'filter' value to be a dict, got {type(topv)}")
-        ret[topk] = topv
-    return (ret, missed)
+#: stage 106: the keys of the system's image-query vocabulary EC2's
+#: DescribeImages can answer, and the filter each becomes.
+AWS_QUERY_KEYS: frozenset[str] = frozenset({"name", "architecture", "publisher", "state", "virtualization",
+                                            "root_device", "tags", "newest", "native"})
+AWS_FILTER_NAMES: dict[str, str] = {              # in the order the filters are written out
+    "name": "name",
+    "root_device": "root-device-type",
+    "virtualization": "virtualization-type",
+    "architecture": "architecture",
+    "state": "state",
+}
+
+
+def translate_image_query(query: ImageQuery) -> dict[str, Any]:
+    """The system's query in EC2's words (stage 106): ``{"owners": [...],
+    "filters": {<DescribeImages filter name>: value}, "most_recent": bool}``,
+    the shape packer's ``amazon-ami`` data source and ``describe_images``
+    both take (``describe_images_request`` makes the boto call from it).
+    ``state`` defaults to ``available``, as the query always has. A key EC2
+    cannot answer raises ValueError naming it (``validate`` refuses it
+    first); ``native`` keys are EC2 filter names, passed verbatim."""
+    unsupported = sorted(k for k in query.keys() if k not in AWS_QUERY_KEYS)
+    if unsupported:
+        raise ValueError(f"AWS cannot translate image query key(s) {', '.join(unsupported)}")
+    filters: dict[str, Any] = {}
+    for key, name in AWS_FILTER_NAMES.items():
+        value = getattr(query, key)
+        if value is not None:
+            filters[name] = value
+    filters.setdefault("state", "available")
+    for k, v in sorted(query.tags.items()):
+        filters[f"tag:{k}"] = v
+    for k, v in query.native.items():
+        filters[str(k)] = v
+    return {"owners": list(query.publisher), "filters": filters,
+            "most_recent": True if query.newest is None else bool(query.newest)}
+
+
+def describe_images_request(native: Mapping[str, Any]) -> dict[str, Any]:
+    """A ``describe_images`` request from ``translate_image_query``'s answer."""
+    filters = []
+    for name, value in dict(native.get("filters") or {}).items():
+        if isinstance(value, bool):
+            value = str(value).lower()
+        filters.append({"Name": str(name), "Values": [str(v) for v in value] if isinstance(value, (list, tuple)) else [str(value)]})
+    return {"DryRun": False, "IncludeDisabled": False, "IncludeDeprecated": False,
+            "Owners": [str(o) for o in (native.get("owners") or [])], "Filters": filters}
 
 
 def query_image(
     query: dict[str, Any],
-    post_query_filter: dict[str, Any] | None = None,
     session_config: dict[str, Any] | None = None,
 ) -> dict[str, Any] | None:
-    """The newest AMI matching an EC2 ``DescribeImages`` query.
-
-    ``query`` is what ``remap_for_image_query`` builds from an OS builder
-    entry: ``Owners``, ``Filters`` (``[{"Name": ..., "Values": [...]}]``) and
-    the ``DryRun``/``IncludeDisabled``/``IncludeDeprecated`` flags. Every
-    match is kept that agrees with ``post_query_filter`` (a mapping of AMI
-    record keys to required values); the newest by ``CreationDate`` is
-    returned, or None when nothing matches. ``session_config`` is the
-    runtime's boto3 session configuration and is required. A pinned
-    image id does not come through here (stage 63 reads the entry's
-    ``image_id`` through ``query_image_by_id``).
+    """The AMI matching an image query in EC2's words (stage 106:
+    ``translate_image_query``'s answer, ``owners``/``filters``/``most_recent``;
+    the boto-shaped request of before is accepted too). The newest by
+    ``CreationDate`` is returned, or None when nothing matches; with
+    ``most_recent`` false, more than one match is refused as ambiguous.
+    ``session_config`` is the runtime's boto3 session configuration and is
+    required. A pinned image id does not come through here (stage 63 reads
+    the entry's ``image_id`` through ``query_image_by_id``).
     """
     if not query:
         raise ValueError("Query dictionary cannot be empty")
+    if "Filters" not in query and "Owners" not in query:
+        most_recent = query.get("most_recent", True)
+        query = describe_images_request(query)
+    else:
+        most_recent = True
 
     try:
         if not session_config:
@@ -207,22 +201,12 @@ def query_image(
     except (BotoCoreError, ClientError) as e:
         raise AMIQueryError(f"Failed to create EC2 client: {e}") from e
 
-    results = _query_by_filters(ec2_client, query)
-    # Return most recently created AMI if multiple matches
-    ilist = []
-    for r in results:
-        if post_query_filter:
-            match = True
-            for k, v in post_query_filter.items():
-                if r.get(k) != v:
-                    match = False
-                    break
-            if not match:
-                continue
-        ilist.append(r)
+    ilist = list(_query_by_filters(ec2_client, query))
     if not ilist:
         return None
     if len(ilist) > 1:
+        if not most_recent:
+            raise AMIQueryError(f"{len(ilist)} AMIs match and the query says newest: false; narrow it")
         ilist.sort(key=lambda x: x.get("CreationDate", ""), reverse=True)
 
     return cast(dict[str, Any], ilist[0])

{cs_image_system_aws_runtime_plugin-0.1.4 → cs_image_system_aws_runtime_plugin-0.1.5}/src/cs_image_system/aws_runtime/bootstrap.py

@@ -30,8 +30,10 @@ from __future__ import annotations
 import json
 from typing import Any
 
-from cs_image_system.base.bootstrap.facts import Facts, default_state_backend, real, runtime_of_type
-from cs_image_system.base.bootstrap.questions import Answers, Question, Raw, Rendered, Secret, Section
+from cs_image_system.base.bootstrap.facts import (Facts, bound_backend_names, default_state_backend, other_state_backends,
+                                                   real, runtime_of_type)
+from cs_image_system.base.bootstrap.questions import (ADOPTED, BY_HAND, MADE, READ, Answers, Question, Raw, Rendered,
+                                                       Secret, Section)
 
 MODULE = "bootstrap_aws"
 OIDC_HOST = "token.actions.githubusercontent.com"
@@ -137,6 +139,55 @@ def _bucket_exists(facts: Facts, answers: Answers) -> bool | None:
                    ("Not Found", "404", "NoSuchBucket"))
 
 
+def _bucket_named_exists(facts: Facts, answers: Answers, bucket: str) -> bool | None:
+    return _exists(_aws(facts, answers, "s3api", "head-bucket", "--bucket", bucket), ("Not Found", "404", "NoSuchBucket"))
+
+
+def other_s3_buckets(facts: Facts) -> list[tuple[str, str, str, bool]]:
+    """The tree's other S3 backends (hygiene XX item 1): ``(backend name,
+    bucket, prefix, bound)`` for each with a real bucket, in declaration
+    order; ``bound`` when some root's ``state_configuration`` names it."""
+    bound = bound_backend_names(facts)
+    out: list[tuple[str, str, str, bool]] = []
+    for backend in other_state_backends(facts, "s3"):
+        bucket = real(backend.get("bucket"))
+        if not bucket:
+            continue
+        prefix = (real(backend.get("key")) or "").strip("/")
+        out.append((str(backend["name"]), bucket, prefix, str(backend["name"]) in bound))
+    return out
+
+
+def _other_buckets_default(facts: Facts, answers: Answers) -> str | None:
+    """The other S3 backends' buckets this root should make: every one that
+    is bound to a root and that the account says is absent, as
+    ``bucket=prefix,...``; a bucket bound to no root is left out (the README
+    says so); a bucket the account cannot be asked about leaves no default."""
+    pairs: list[str] = []
+    for _name, bucket, prefix, bound in other_s3_buckets(facts):
+        if not bound:
+            continue
+        exists = _bucket_named_exists(facts, answers, bucket)
+        if exists is None:
+            return None
+        if not exists:
+            pairs.append(f"{bucket}={prefix}")
+    return ",".join(pairs)
+
+
+def parse_buckets(text: Any) -> dict[str, str]:
+    """``bucket=prefix,bucket=prefix`` as a mapping (an empty prefix allowed)."""
+    out: dict[str, str] = {}
+    for part in str(text or "").split(","):
+        part = part.strip()
+        if not part:
+            continue
+        bucket, _, prefix = part.partition("=")
+        if bucket.strip():
+            out[bucket.strip()] = prefix.strip().strip("/")
+    return out
+
+
 def _profile_exists(facts: Facts, answers: Answers) -> bool | None:
     name = answers.get("instance_profile")
     if not name:
@@ -236,6 +287,14 @@ QUESTIONS: tuple[Question, ...] = (
              "here, so a destroy of the root removes it? (a bucket that stood before this root is read, never "
              "managed; one this root makes is managed from then on)", kind="bool",
              default=lambda f, a: not a.get("state_bucket_exists")),
+    # hygiene XX item 1: the tree's OTHER S3 backends. A choice, not an
+    # observation: the second interview keeps the answer, so a bucket this
+    # root made never leaves the plan (the trap item 5 of hygiene XII closed
+    # for the default bucket).
+    Question("other_state_buckets", "The tree's other S3 backends' buckets this root MAKES when absent and manages "
+             "from then on, as bucket=prefix,bucket=prefix (by default every one bound to a root that the account "
+             "says is absent; a backend bound to no root is left out)", default=_other_buckets_default,
+             help="the account could not be asked about one of the other buckets: no AWS session", may_be_empty=True),
     Question("instance_profile", "The instance profile build and standing instances wear for Session Manager",
              default=_session_profile,
              help="the AWS runtime's `session_instance_profile` in cfg/runtime-builders.yml is unset or a placeholder"),
@@ -245,12 +304,29 @@ QUESTIONS: tuple[Question, ...] = (
 )
 
 
-def render(answers: Answers) -> Rendered:
+def render(answers: Answers, facts: Facts | None = None) -> Rendered:
     profile = str(answers.get("profile") or "")
     prefix = str(answers["state_prefix"]).strip("/")
     tags = {**parse_tags(str(answers.get("tags") or "")), "ManagedBy": MANAGED_BY}
     bucket_exists = bool(answers["state_bucket_exists"])
     bucket_managed = bool(answers.get("state_bucket_managed")) or not bucket_exists   # made here: managed, always
+    other_made = parse_buckets(answers.get("other_state_buckets"))
+    # the other backends this root does NOT make, with the reason (item 1: a line, never a silence)
+    others = other_s3_buckets(facts) if facts is not None else []
+    other_rows: list[tuple[str, str, str]] = []
+    for name, bucket, _bprefix, bound in others:
+        if bucket in other_made:
+            other_rows.append((f"the S3 backend `{name}`'s bucket `{bucket}`", MADE,
+                               "declared, bound to a root and absent at the interview: made here (versioned, encrypted, private) and managed from then on"))
+        elif not bound:
+            other_rows.append((f"the S3 backend `{name}`'s bucket `{bucket}`", BY_HAND,
+                               "declared but bound to no root, so nothing would use it: not made; bind a root to it and re-run the interview, or drop the backend"))
+        else:
+            other_rows.append((f"the S3 backend `{name}`'s bucket `{bucket}`", READ,
+                               "stood at the interview, or was left out of the answer: read, never managed here"))
+    for bucket in other_made:
+        if bucket not in {b for _n, b, _p, _b in others}:
+            other_rows.append((f"the bucket `{bucket}`", MADE, "named in the answer; no declared S3 backend names it"))
     backend = None
     if bucket_exists:
         settings: dict[str, Any] = {"bucket": answers["state_bucket"], "key": f"{prefix}/bootstrap.tfstate",
@@ -285,6 +361,7 @@ def render(answers: Answers) -> Rendered:
         "instance_profile_exists": "the instance profile already exists (read, not made)",
     }
     variables = tuple((f"aws_{n}", types.get(n, "string"), described[n]) for n in names) + (
+        ("aws_other_state_buckets", "map(string)", "the tree's other S3 backends' buckets this root makes and manages: bucket -> key prefix"),
         ("aws_profile", "string", "the AWS profile this root is applied with (empty: the environment's credentials)"),
         ("aws_tags", "map(string)", "tags on everything this root makes"),
         ("aws_read_extra_subjects", "list(string)", "other OIDC subjects the READ-ONLY role keeps trusting"),
@@ -294,6 +371,7 @@ def render(answers: Answers) -> Rendered:
                               for n in names}
     tfvars["aws_state_bucket_managed"] = bucket_managed
     tfvars["aws_state_prefix"] = prefix
+    tfvars["aws_other_state_buckets"] = other_made
     tfvars["aws_profile"] = profile
     tfvars["aws_tags"] = tags
     extras = {k: parse_subjects(answers.get(f"{k}_extra_subjects")) for k in ("read", "write")}
@@ -308,6 +386,7 @@ def render(answers: Answers) -> Rendered:
                                                         if extras[k] else "")
             for k in ("read", "write") if answers.get(f"{k}_role_exists")),
         module_args={**{n: Raw(f"var.aws_{n}") for n in names},
+                     "other_state_buckets": Raw("var.aws_other_state_buckets"),
                      "read_extra_subjects": Raw("var.aws_read_extra_subjects"),
                      "write_extra_subjects": Raw("var.aws_write_extra_subjects")},
         variables=variables,
@@ -317,6 +396,7 @@ def render(answers: Answers) -> Rendered:
             ("aws_write_role_arn", f"module.{MODULE}.write_role_arn", "the WRITE role's ARN: the AWS_APPLY_ROLE_ARN secret"),
             ("aws_oidc_provider_arn", f"module.{MODULE}.oidc_provider_arn", "the GitHub OIDC identity provider"),
             ("aws_state_bucket", f"module.{MODULE}.state_bucket", "the state bucket"),
+            ("aws_other_state_buckets", f"module.{MODULE}.other_state_buckets", "the other S3 backends' buckets made here"),
             ("aws_instance_profile_arn", f"module.{MODULE}.instance_profile_arn", "the Session Manager instance profile"),
         ),
         required_providers={"aws": {"source": "hashicorp/aws", "version": ">= 5.81"}},
@@ -349,11 +429,33 @@ def render(answers: Answers) -> Rendered:
                "exists: this root's state is bound to it (decision D8); the bucket stood before this root and is not managed here"
                if bucket_exists else
                "is CREATED here (versioned, encrypted, private) and managed from then on: local state for this first apply"),
+            *(f"the S3 backend's bucket `{b}` (prefix `{p or '(none)'}`) is CREATED here (versioned, encrypted, private) "
+              "and managed from then on" for b, p in sorted(other_made.items())),
             f"the instance profile `{answers['instance_profile']}` "
             + ("read (it exists)" if answers["instance_profile_exists"] else "created with AmazonSSMManagedInstanceCore"),
         ),
         backend=backend,
         root_hcl=imports,
+        coverage=(
+            ("the GitHub OIDC identity provider", READ if answers["oidc_provider_exists"] else MADE,
+             "read as a data source when the account has it" if answers["oidc_provider_exists"] else "the account lacked it"),
+            *((f"the {k.upper()} role `{answers[f'{k}_role_name']}`", ADOPTED if answers.get(f"{k}_role_exists") else MADE,
+               "stood at the interview: an import block, so its trust and policy are maintained from here"
+               if answers.get(f"{k}_role_exists") else "the account lacked it") for k in ("read", "write")),
+            (f"the default S3 backend's bucket `{answers['state_bucket']}`",
+             MADE if bucket_managed else READ,
+             "made by an apply of this root and managed from then on" if bucket_managed
+             else "stood before this root: read, never managed"),
+            *other_rows,
+            (f"the Session Manager instance profile `{answers['instance_profile']}`",
+             READ if answers["instance_profile_exists"] else MADE,
+             "read as a data source when the account has it" if answers["instance_profile_exists"]
+             else "the account lacked it: a role with AmazonSSMManagedInstanceCore"),
+            ("the network: VPC, subnets, security groups, the access gateway", BY_HAND,
+             "the team's, by decision (stage 1's network rule): the system never creates or modifies network configuration"),
+            ("the bake hosts' route to SSM and the package repositories", BY_HAND,
+             "part of the network: the team's"),
+        ),
     )
 
 
@@ -363,5 +465,5 @@ def wanted(facts: Facts) -> bool:
 
 
 def aws_section() -> Section:
-    return Section(name="aws", title="AWS", questions=QUESTIONS, render=render, wanted=wanted,
-                   description="the GitHub OIDC provider, the read-only and write roles, the optional state bucket and instance profile")
+    return Section(name="aws", title="AWS", questions=QUESTIONS, render=render, wanted=wanted, wants_facts=True,
+                   description="the GitHub OIDC provider, the read-only and write roles, the state buckets and the instance profile")