cfn-check 0.6.2__tar.gz → 0.7.1__tar.gz

{cfn_check-0.6.2 → cfn_check-0.7.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cfn-check
-Version: 0.6.2
+Version: 0.7.1
 Summary: Validate Cloud Formation
 Author-email: Ada Lundhe <adalundhe@lundhe.audio>
 License: MIT License
@@ -34,7 +34,7 @@ Requires-Python: >=3.12
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: pydantic
-Requires-Dist: pyyaml
+Requires-Dist: ruamel.yaml
 Requires-Dist: hyperlight-cocoa
 Requires-Dist: async-logging
 Dynamic: license-file
@@ -50,7 +50,7 @@ Dynamic: license-file
 
 | Package     | cfn-check                                                           |
 | ----------- | -----------                                                     |
-| Version     | 0.3.3                                                           |
+| Version     | 0.7.1                                                           |
 | Download    | https://pypi.org/project/cfn-check/                             | 
 | Source      | https://github.com/adalundhe/cfn-check                          |
 | Keywords    | cloud-formation, testing, aws, cli                              |
@@ -70,15 +70,20 @@ problems inherint to `cfn-lint` more than `cfn-guard`, primarily:
 - Inability to parse non-resource wildcards
 - Inability to validate non-resource template data
 - Inabillity to use structured models to validate input
+- Poor ability to parse and render CloudFormation Refs/Functions
 
 In comparison to `cfn-guard`, `cfn-check` is pure Python, thus
 avoiding YADSL (Yet Another DSL) headaches. It also proves
 significantly more configurable/modular/hackable as a result.
+`cfn-check` can resolve _some_ (not all) CloudFormation Intrinsic
+Functions and Refs.
 
 CFN-Check uses a combination of simple depth-first-search tree
 parsing, friendly `cfn-lint` like query syntax, `Pydantic` models,
 and `pytest`-like assert-driven checks to make validating your
 Cloud Formation easy while offering both CLI and Python API interfaces.
+CFN-Check also uses a lightning-fast AST-parser to render your templates,
+allowing you to validate policy, not just a YAML document.
 
 <br/>
 
@@ -447,7 +452,7 @@ Resources::*::Type
 Selects all `Resource` objects. If we convert the Wildcard Token in the query to a Wildcard Range Token:
 
 ```
-Resources::*::Type
+Resources::[*]::Type
 ```
 
 The Rule will fail as below:
@@ -585,3 +590,52 @@ class ValidateResourceType(Collection):
 ```
 
 By deferring type and existence assertions to `Pydantic` models, you can focus your actual assertion logic on business/security policy checks.
+
+<br/>
+
+# The Rendering Engine
+
+### Overview
+
+In Version 0.6.X, CFN-Check introduced a rendering engine, which allows it
+to parse and execute Refs and all CloudFormation intrinsic functions via 
+either the CloudFormation document or user-supplied values. This additional
+also resulted in the:
+
+```bash
+cfn-check render <TEMPLATE_PATH >
+```
+
+command being added, allowing you to effectively "dry run" render your
+CloudFormation templates akin to the `helm template` command for Helm.
+
+By default, `cfn-check render` outputs to stdout, however you can easily
+save rendered output to a file via the `-o/--output-file` flag. For example:
+
+```bash
+cfn-check render template.yml -o rendered.yml
+```
+
+The `cfn-check render` command also offers the following options:
+
+- `-a/--attributes`:  A list of <key>=<value> input `!GetAtt` attributes to use
+- `-m/--mappings`: A list of <key>=<value> input `Mappings` to use
+- `-p/--parameters`: A list of <key>=<value> input `Parameters` to use
+- `-l/--log-level`: The log level to use
+
+### The Rendering Engine during Checks
+
+By default rendering is enabled when running `cfn-check` validation. You can 
+disable it by supplying `no-render` to the `-F/--flags` option as below:
+
+```bash
+cfn-check validate -F no-render -r rules.py template.yaml
+```
+
+Disabling rendering means CFN-Check will validate your template as-is, with
+no additional pre-processing and no application of user input values.
+
+> [!WARNING]
+> CloudFormation documents are <b>not</b> "plain yaml" and disabling
+> rendering means any dynamically determined values will likely fail
+> to pass validation, resulting in false positives for failures!

{cfn_check-0.6.2 → cfn_check-0.7.1}/README.md

@@ -9,7 +9,7 @@
 
 | Package     | cfn-check                                                           |
 | ----------- | -----------                                                     |
-| Version     | 0.3.3                                                           |
+| Version     | 0.7.1                                                           |
 | Download    | https://pypi.org/project/cfn-check/                             | 
 | Source      | https://github.com/adalundhe/cfn-check                          |
 | Keywords    | cloud-formation, testing, aws, cli                              |
@@ -29,15 +29,20 @@ problems inherint to `cfn-lint` more than `cfn-guard`, primarily:
 - Inability to parse non-resource wildcards
 - Inability to validate non-resource template data
 - Inabillity to use structured models to validate input
+- Poor ability to parse and render CloudFormation Refs/Functions
 
 In comparison to `cfn-guard`, `cfn-check` is pure Python, thus
 avoiding YADSL (Yet Another DSL) headaches. It also proves
 significantly more configurable/modular/hackable as a result.
+`cfn-check` can resolve _some_ (not all) CloudFormation Intrinsic
+Functions and Refs.
 
 CFN-Check uses a combination of simple depth-first-search tree
 parsing, friendly `cfn-lint` like query syntax, `Pydantic` models,
 and `pytest`-like assert-driven checks to make validating your
 Cloud Formation easy while offering both CLI and Python API interfaces.
+CFN-Check also uses a lightning-fast AST-parser to render your templates,
+allowing you to validate policy, not just a YAML document.
 
 <br/>
 
@@ -406,7 +411,7 @@ Resources::*::Type
 Selects all `Resource` objects. If we convert the Wildcard Token in the query to a Wildcard Range Token:
 
 ```
-Resources::*::Type
+Resources::[*]::Type
 ```
 
 The Rule will fail as below:
@@ -543,4 +548,53 @@ class ValidateResourceType(Collection):
         assert value is not None
 ```
 
-By deferring type and existence assertions to `Pydantic` models, you can focus your actual assertion logic on business/security policy checks.
+By deferring type and existence assertions to `Pydantic` models, you can focus your actual assertion logic on business/security policy checks.
+
+<br/>
+
+# The Rendering Engine
+
+### Overview
+
+In Version 0.6.X, CFN-Check introduced a rendering engine, which allows it
+to parse and execute Refs and all CloudFormation intrinsic functions via 
+either the CloudFormation document or user-supplied values. This additional
+also resulted in the:
+
+```bash
+cfn-check render <TEMPLATE_PATH >
+```
+
+command being added, allowing you to effectively "dry run" render your
+CloudFormation templates akin to the `helm template` command for Helm.
+
+By default, `cfn-check render` outputs to stdout, however you can easily
+save rendered output to a file via the `-o/--output-file` flag. For example:
+
+```bash
+cfn-check render template.yml -o rendered.yml
+```
+
+The `cfn-check render` command also offers the following options:
+
+- `-a/--attributes`:  A list of <key>=<value> input `!GetAtt` attributes to use
+- `-m/--mappings`: A list of <key>=<value> input `Mappings` to use
+- `-p/--parameters`: A list of <key>=<value> input `Parameters` to use
+- `-l/--log-level`: The log level to use
+
+### The Rendering Engine during Checks
+
+By default rendering is enabled when running `cfn-check` validation. You can 
+disable it by supplying `no-render` to the `-F/--flags` option as below:
+
+```bash
+cfn-check validate -F no-render -r rules.py template.yaml
+```
+
+Disabling rendering means CFN-Check will validate your template as-is, with
+no additional pre-processing and no application of user input values.
+
+> [!WARNING]
+> CloudFormation documents are <b>not</b> "plain yaml" and disabling
+> rendering means any dynamically determined values will likely fail
+> to pass validation, resulting in false positives for failures!

{cfn_check-0.6.2 → cfn_check-0.7.1}/cfn_check/cli/render.py

@@ -3,45 +3,30 @@ from async_logging import LogLevelName, Logger, LoggingConfig
 from cocoa.cli import CLI
 
 from cfn_check.cli.utils.files import load_templates, write_to_file
+from cfn_check.cli.utils.stdout import write_to_stdout
 from cfn_check.rendering import Renderer
 from cfn_check.logging.models import InfoLog
 
 
-@CLI.command(
-        display_help_on_error=False
-)
+@CLI.command()
 async def render(
     path: str,
-    output_file: str  = 'rendered.yml',
+    output_file: str | None = None,
+    attributes: list[str] | None = None,
+    mappings: list[str] | None = None,
     parameters: list[str] | None = None,
     references: list[str] | None = None,
-    tags: list[str] = [
-        'Ref',
-        'Sub',
-        'Join',
-        'Select',
-        'Split',
-        'GetAtt',
-        'GetAZs',
-        'ImportValue',
-        'Equals',
-        'If',
-        'Not',
-        'And',
-        'Or',
-        'Condition',
-        'FindInMap',
-    ],
     log_level: LogLevelName = 'info',
 ):
     """
     Render a Cloud Formation template
 
     @param output_file Path to output the rendered CloudFormation template to
+    @param attributes A list of <key>=<value> input !GetAtt attributes to use
+    @param mappings A list of <key>=<value> input Mappings to use
     @param parameters A list of <key>=<value> input Parameters to use
     @param references A list of <key>=<value> input !Ref values to use
-    @param tags List of CloudFormation intrinsic function tags
-    @param log_level The log level to use
+    @param log-level The log level to use
     """
     logging_config = LoggingConfig()
     logging_config.update(
@@ -49,6 +34,18 @@ async def render(
         log_output='stderr',
     )
 
+    parsed_attributes: dict[str, str] | None = None
+    if attributes:
+        parsed_attributes = dict([
+            attribute.split('=', maxsplit=1) for attribute in attributes if len(attribute.split('=', maxsplit=1)) > 0
+        ])
+
+    parsed_mappings: dict[str, str] | None = None
+    if mappings:
+        parsed_mappings = dict([
+            mapping.split('=', maxsplit=1) for mapping in mappings if len(mapping.split('=', maxsplit=1)) > 0
+        ])
+
     parsed_parameters: dict[str, str] | None = None
     if parameters:
         parsed_parameters = dict([
@@ -65,7 +62,6 @@ async def render(
 
     templates = await load_templates(
         path,
-        tags,
     )
 
     assert len(templates) == 1 , '❌ Can only render one file'
@@ -74,10 +70,15 @@ async def render(
     renderer = Renderer()
     rendered = renderer.render(
         template,
+        attributes=parsed_attributes,
+        mappings=parsed_mappings,
         parameters=parsed_parameters,
         references=parsed_references,
     )
 
-    await write_to_file(output_file, rendered)
+    if output_file is False:
+        await write_to_file(output_file, rendered)
+        await logger.log(InfoLog(message=f'✅ {path} template rendered'))
 
-    await logger.log(InfoLog(message=f'✅ {path} template rendered'))
+    else:
+        await write_to_stdout(rendered)

{cfn_check-0.6.2 → cfn_check-0.7.1}/cfn_check/cli/utils/files.py

@@ -67,7 +67,6 @@ async def localize_path(path: str, loop: asyncio.AbstractEventLoop):
 
 async def load_templates(
     path: str,
-    tags: list[str],
     file_pattern: str | None = None,
 ):
 

cfn_check-0.7.1/cfn_check/cli/utils/stdout.py

@@ -0,0 +1,18 @@
+import asyncio
+import sys
+from ruamel.yaml import YAML
+from ruamel.yaml.comments import CommentedBase
+
+async def write_to_stdout(data: CommentedBase):
+    loop = asyncio.get_event_loop()
+
+    yaml = YAML(typ=['rt'])
+    yaml.preserve_quotes = True
+    yaml.width = 4096
+    yaml.indent(mapping=2, sequence=4, offset=2)
+    await loop.run_in_executor(
+        None,
+        yaml.dump,
+        data,
+        sys.stdout,
+    )

{cfn_check-0.6.2 → cfn_check-0.7.1}/cfn_check/cli/validate.py

@@ -11,36 +11,24 @@ from cfn_check.collection.collection import Collection
 from cfn_check.validation.validator import Validator
 
 
-@CLI.command()
+@CLI.command(
+    shortnames={
+        'flags': 'F'
+    }
+)
 async def validate(
     path: str,
     file_pattern: str | None = None,
     rules: ImportType[Collection] = None,
-    tags: list[str] = [
-        'Ref',
-        'Sub',
-        'Join',
-        'Select',
-        'Split',
-        'GetAtt',
-        'GetAZs',
-        'ImportValue',
-        'Equals',
-        'If',
-        'Not',
-        'And',
-        'Or',
-        'Condition',
-        'FindInMap',
-    ],
+    flags: list[str] | None = None,
     log_level: LogLevelName = 'info',
 ):
     '''
     Validate Cloud Foundation
     
-    @param rules Path to a file containing Collections
+    @param disabled A list of string features to disable during checks
     @param file_pattern A string pattern used to find template files
-    @param tags List of CloudFormation intrinsic function tags
+    @param rules Path to a file containing Collections
     @param log_level The log level to use
     '''
 
@@ -52,9 +40,11 @@ async def validate(
 
     logger = Logger()
 
+    if flags is None:
+        flags = []
+
     templates = await load_templates(
         path,
-        tags,
         file_pattern=file_pattern,
     )
 
@@ -71,7 +61,7 @@ async def validate(
         for rule in rules.data.values()
         for _, validation in inspect.getmembers(rule)
         if isinstance(validation, Validator)
-    ])
+    ], flags=flags)
     
     if validation_error := validation_set.validate([
         template_data for _, template_data in templates

{cfn_check-0.6.2 → cfn_check-0.7.1}/cfn_check/evaluation/evaluator.py

@@ -12,7 +12,14 @@ from .parsing import QueryParser
 
 class Evaluator:
 
-    def __init__(self):
+    def __init__(
+        self,
+        flags: list[str] | None = None
+    ):
+        if flags is None:
+            flags = []
+
+        self.flags = flags
         self._query_parser = QueryParser()
         self._renderer = Renderer()
 
@@ -22,9 +29,11 @@ class Evaluator:
         path: str,
     ):
         items: Items = deque()
+        
+        if 'no-render' not in self.flags:
+            resources = self._renderer.render(resources)
 
-        rendered = self._renderer.render(resources)
-        items.append(rendered)
+        items.append(resources)
 
         segments = path.split("::")[::-1]
         # Queries can be multi-segment,

{cfn_check-0.6.2 → cfn_check-0.7.1}/cfn_check/evaluation/validate.py

@@ -9,8 +9,13 @@ class ValidationSet:
     def __init__(
         self,
         validators: list[Validator],
+        flags: list[str] | None = None
     ):
-        self._evaluator = Evaluator()
+        
+        if flags is None:
+            flags = []
+
+        self._evaluator = Evaluator(flags=flags)
         self._validators = validators
 
     @property

cfn_check-0.7.1/cfn_check/rendering/cidr_solver.py

@@ -0,0 +1,66 @@
+class IPv4CIDRSolver:
+
+    def __init__(
+        self,
+        host: str,
+        desired: int,
+        bits: int,
+    ):
+        self.host = host
+        self.subnets_desired = desired
+        self.subnet_bits = bits
+
+        host_ip, mask = self.host.split('/', maxsplit=1)
+
+        self.host_ip = host_ip
+        self._host_mask_string = f'/{mask}'
+        self.host_mask = int(mask)
+
+        self.subnet_mask = 32 - bits
+
+        self._host_octets = [
+            int(octet) for octet in self.host.strip(self._host_mask_string).split('.')
+        ]
+    
+    def provision_subnets(self):
+        subnet_requested_ips = 2**self.subnet_bits
+        host_available_ips = 2**(32 - self.host_mask)
+
+        total_ips_requested = subnet_requested_ips * self.subnets_desired
+        if host_available_ips < total_ips_requested:
+            return []
+
+        return [
+            self._provision_subnet(
+                subnet_requested_ips,
+                idx
+            ) for idx in range(self.subnets_desired)
+        ]
+
+    
+    def _provision_subnet(
+        self,
+        requested_ips: int,
+        idx: int,
+    ):
+        increment = requested_ips
+        octet_idx = -1
+        if requested_ips > 255:
+            increment /= 256
+            octet_idx -= 1
+
+        increment *= idx
+
+        subnet = list(self._host_octets)
+
+        subnet[octet_idx] += increment
+
+        subnet_base_ip = '.'.join([
+            str(octet) for octet in subnet
+        ])
+
+        return f'{subnet_base_ip}/{self.subnet_mask}'
+
+
+
+