cfn-check 0.3.2__tar.gz → 0.9.0__tar.gz

{cfn_check-0.3.2 → cfn_check-0.9.0}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cfn-check
-Version: 0.3.2
+Version: 0.9.0
 Summary: Validate Cloud Formation
 Author-email: Ada Lundhe <adalundhe@lundhe.audio>
 License: MIT License
@@ -27,14 +27,15 @@ License: MIT License
         
 Project-URL: Homepage, https://github.com/adalundhe/cfn-check
 Keywords: cloud-formation,testing,aws,cli
+Classifier: Programming Language :: Python :: 3.11
 Classifier: Programming Language :: Python :: 3.12
 Classifier: Programming Language :: Python :: 3.13
 Classifier: Operating System :: OS Independent
-Requires-Python: >=3.12
+Requires-Python: >=3.11
 Description-Content-Type: text/markdown
 License-File: LICENSE
 Requires-Dist: pydantic
-Requires-Dist: pyyaml
+Requires-Dist: ruamel.yaml
 Requires-Dist: hyperlight-cocoa
 Requires-Dist: async-logging
 Dynamic: license-file
@@ -50,7 +51,7 @@ Dynamic: license-file
 
 | Package     | cfn-check                                                           |
 | ----------- | -----------                                                     |
-| Version     | 0.3.2                                                           |
+| Version     | 0.8.1                                                           |
 | Download    | https://pypi.org/project/cfn-check/                             | 
 | Source      | https://github.com/adalundhe/cfn-check                          |
 | Keywords    | cloud-formation, testing, aws, cli                              |
@@ -70,15 +71,20 @@ problems inherint to `cfn-lint` more than `cfn-guard`, primarily:
 - Inability to parse non-resource wildcards
 - Inability to validate non-resource template data
 - Inabillity to use structured models to validate input
+- Poor ability to parse and render CloudFormation Refs/Functions
 
 In comparison to `cfn-guard`, `cfn-check` is pure Python, thus
 avoiding YADSL (Yet Another DSL) headaches. It also proves
 significantly more configurable/modular/hackable as a result.
+`cfn-check` can resolve _some_ (not all) CloudFormation Intrinsic
+Functions and Refs.
 
 CFN-Check uses a combination of simple depth-first-search tree
 parsing, friendly `cfn-lint` like query syntax, `Pydantic` models,
 and `pytest`-like assert-driven checks to make validating your
 Cloud Formation easy while offering both CLI and Python API interfaces.
+CFN-Check also uses a lightning-fast AST-parser to render your templates,
+allowing you to validate policy, not just a YAML document.
 
 <br/>
 
@@ -250,19 +256,20 @@ Congrats! You've just made the cloud a bit better place!
 
 # Queries, Tokens, and Syntax
 
-A `cfn-check` Query is a string made up of double-colon (`::`) delimited "Tokens" centered around three primary types:
+A `cfn-check` Query is a string made up of period (`.`) delimited "Tokens" centered around four primary types:
 
 - <b>`Keys`</b>  - `<KEY>`: String name key Tokens that perform exact matching on keys of key/value pairs in a CloudFormation document.
-- <b>`Patterns`</b>  - `(\d+)`: Paren-enclosed regex pattern Tokens that perform pattern-based matching on keys of key/value pairs in a CloudFormation document.
+- <b>`Values`</b> - `(<KEY> = <VALUE>)`:  Parenthesis-enclosed `K==V` pairs that perform matching on values of key/value pairs in a CloudFormation document.
+- <b>`Patterns`</b>  - `<\d+>`: Arrow-enclosed regex pattern Tokens that perform pattern-based matching on keys of key/value pairs in a CloudFormation document.
 - <b>`Ranges`</b> - `[]`: Brackets enclosed Tokens that perform array selection and filtering in a CloudFormation document.
 
 
-In addition to `Key`, `Pattern`, and `Range` selection, you can also incorporate:
+In addition to `Key`, `Value`, `Pattern`, and `Range` selection, you can also incorporate:
 
 - <b>`Bounded Ranges`</b> - `[<A>-<B>]`: Exact matches from the starting position (if specified) to the end position (if specified) of an array
 - <b>`Indicies`</b> - `[<A>]`: Exact matches the specified indicies of an array
 - <b>`Key Ranges`</b> - `[<KEY>]`: Exact matches keys of objects within an array
-- <b>`Pattern Ranges`</b> (`[(\d+)]`): Matches they keys of objects within an array based on the specified pattern
+- <b>`Pattern Ranges`</b> (`[<\d+>]`): Matches they keys of objects within an array based on the specified pattern
 - <b>`Wildcards`</b> (`*`): Selects all values for a given object or array or returns the non-object/array value at the specified path
 - <b>`Wildcard Ranges`</b> (`[*]`): Selects all values for a given array and ensures that *only* the values of a valid array type are returned (any other type will be treated as a mismatch).
 
@@ -276,6 +283,27 @@ Resources
 
 as your query, you'll select all items within the CloudFormation document under the `Resources` key.
 
+
+### Working with Values
+
+In addition to searching by keys, filtering by the values associated with those keys is the most common way you'll traverse and validate your CloudFormation template. To filter `Resource` key matches by the value of their `Type` value to only return EC2 Instances, we would specify a query like:
+
+```
+Resources.*.(Type == AWS::EC2::Instance)
+```
+
+You can also match multiple values by utilizing the `in`/`IN` operator:
+
+```
+Resources.*.(Type in AWS::Lambda::Function,AWS::Serverless::Function)
+```
+
+and providing a comma-delimited list of values.
+
+> [!NOTE]
+> Value queries do not support Nested Ranges or nested queries, but *do* support
+> Wildcards and Patterns. See below for more info!
+
 ### Working with Patterns
 
 If an object within a CloudFormation document contains multiple similar keys you want to select, `Pattern` Tokens are your go-to solution. Consider this segment of CloudFormation:
@@ -302,7 +330,7 @@ Resources:
 We want to select <i>both</i> `SecurityGroupIngress` and `SecurityGroupEgress` to perform the same rule evaluations. Since the keys for both blocks start with `SecurityGroup`, we could write a Query using a Pattern Token like:
 
 ```
-Resources::SecurityGroup::Properties::(SecurityGroup)
+Resources.SecurityGroup.Properties.<SecurityGroup>
 ```
 
 which would allow us to use a single rule to evaluate both:
@@ -311,7 +339,7 @@ which would allow us to use a single rule to evaluate both:
 class ValidateSecurityGroups(Collection):
 
     @Rule(
-        "Resources::SecurityGroup::Properties::(SecurityGroup)",
+        "Resources.SecurityGroup.Properties.<SecurityGroup>",
         "It checks Security Groups are correctly definined",
     )
     def validate_security_groups(self, value: list[dict]):
@@ -342,7 +370,7 @@ Wildcard Tokens allow you to select all matching objects, array entries, or valu
 In fact, you've already used one! In the first example, we use a Wildcard Token in the below query:
 
 ```
-Resources::*::Type
+Resources.*.Type
 ```
 
 To select all `Resource` objects, then further extract the `Type` field from each object. This helps us avoid copy-paste rules at the potential cost of deferring more work to individual `Rule` methods if we aren't careful and select too much!
@@ -361,7 +389,7 @@ Ranges allow you to perform sophisticated selection of objects or data within a
 Unbounded ranges allow you to select and return an array in its entirety. For example:
 
 ```
-Resources::SecurityGroup::Properties::SecurityGroupIngress::[]
+Resources.SecurityGroup.Properties.SecurityGroupIngress.[]
 ```
 
 Would return all SecurityGroupIngress objects in the CloudFormation document as a list, allowing you to check that the array of ingresses has been both defined *and* populated.
@@ -372,7 +400,7 @@ Would return all SecurityGroupIngress objects in the CloudFormation document as
 Indexes allow you to select specific positions within an array. For example:
 
 ```
-Resources::SecurityGroup::Properties::SecurityGroupIngress::[0]
+Resources.SecurityGroup.Properties.SecurityGroupIngress.[0]
 ```
 
 Would return the first SecurityGroupIngress objects in the document.
@@ -386,7 +414,7 @@ Bounded Ranges allow you to select subsets of indicies within an array (much lik
 As an example:
 
 ```
-Resources::SecurityGroup::Properties::SecurityGroupIngress::[1-3]
+Resources.SecurityGroup.Properties.SecurityGroupIngress.[1-3]
 ```
 
 Would select the second and third SecurityGroupIngress objects in the document.
@@ -394,13 +422,13 @@ Would select the second and third SecurityGroupIngress objects in the document.
 Start or end positions are optional for Bounded Ranges. If a starting position is not defined, `cfn-check` will default to `0`. Likewise, if an end position is not defined, `cfn-check` will default to the end of given list. For example:
 
 ```
-Resources::SecurityGroup::Properties::SecurityGroupIngress::[-3]
+Resources.SecurityGroup.Properties.SecurityGroupIngress.[-3]
 ```
 
 selects the first through third SecurityGroupIngress objects in the document while:
 
 ```
-Resources::SecurityGroup::Properties::SecurityGroupIngress::[3-]
+Resources.SecurityGroup.Properties.SecurityGroupIngress.[3-]
 ```
 
 selects the remaining SecurityGroupIngress objects starting from the third.
@@ -417,7 +445,7 @@ Often times it's easier to match based upon an array's contents than by exact in
 For example:
 
 ```
-Resources::MyEC2Instance::Properties::ImageId::[AWSRegionArch2AMI]
+Resources.MyEC2Instance.Properties.ImageId.[AWSRegionArch2AMI]
 ```
 
 returns only the EC2 ImageIds where the ImageId exactly matches `AWSRegionArch2AMI`.
@@ -428,7 +456,7 @@ returns only the EC2 ImageIds where the ImageId exactly matches `AWSRegionArch2A
 Pattern Ranges function much like Key Ranges, but utilize regex-based pattern matching for comparison. Adapting the above example:
 
 ```
-Resources::MyEC2Instance::Properties::ImageId::[(^AWSRegion)]
+Resources.MyEC2Instance.Properties.ImageId.[<^AWSRegion>]
 ```
 
 returns only the EC2 ImageIds where the ImageId begins with `AWSRegion`. This can be helpful in checking for and enforcing naming standards, etc.
@@ -441,13 +469,13 @@ Wildcard Ranges extend the powerful functionality of Wildcard Tokens with the ad
 For example we know:
 
 ```
-Resources::*::Type
+Resources.*.Type
 ```
 
 Selects all `Resource` objects. If we convert the Wildcard Token in the query to a Wildcard Range Token:
 
 ```
-Resources::*::Type
+Resources.[*].Type
 ```
 
 The Rule will fail as below:
@@ -481,7 +509,7 @@ Note that the array we want is nested within another array, and we need to make
 We can accomplish this by using a Wildcard Range Token in our Query as below:
 
 ```
-Resources::AppendItemToListFunction::Properties::Code::ZipFile::[*]::[]
+Resources.AppendItemToListFunction.Properties.Code.ZipFile.[*].[]
 ```
 
 Which allows us to then evaluate the Unbounded Range token against each array item, returning only the array we want.
@@ -498,7 +526,7 @@ You can use multiple Tokens within a Range Token by seperating each token with a
 For example:
 
 ```
-Resources::SecurityGroup::Properties::SecurityGroupIngress::[0, -2]
+Resources.SecurityGroup.Properties.SecurityGroupIngress.[0, -2]
 ```
 
 Would select all except the last element of an array.
@@ -506,7 +534,7 @@ Would select all except the last element of an array.
 This also applies to Bounded Ranges, Key Ranges, Pattern Ranges, and Wildcard Ranges! For example:
 
 ```
-Resources::MyEC2Instance::Properties::ImageId::[(^AWSRegion),(^),(^Custom)]
+Resources.MyEC2Instance.Properties.ImageId.[(^AWSRegion),(^),(^Custom)]
 ```
 
 will select any EC2 ImageIds that start with either `AWSRegion` or `Custom`.
@@ -530,12 +558,160 @@ ZipFile: !Join
 from our previous examples, we used the below query to select the nested array:
 
 ```
-Resources::AppendItemToListFunction::Properties::Code::ZipFile::[*]::[]
+Resources.AppendItemToListFunction.Properties.Code.ZipFile.[*].[]
 ```
 
 With Nested Ranges, this can be shortened to:
 
 ```
-Resources::AppendItemToListFunction::Properties::Code::ZipFile::[[]]
+Resources.AppendItemToListFunction.Properties.Code.ZipFile.[[]]
+```
 
 Which is both more concise *and* more representitave of our intention to select only the array.
+
+# Grouping Queries
+
+CFN-Check grouping allows you significant freedom of expression in how you write queries while *also* allowing you to more easily restrict and filter results by multiple criterion. Queries support both logical "or" and "and" statements via the `|` and `&` operators respectively. For example, consider the previous values query where we used an `in` operator:
+
+```
+Resources.*.(Type in AWS::Lambda::Function,AWS::Serverless::Function)
+```
+
+This could be rewritten as:
+```
+Resources.*(Type == AWS::Lambda::Function | Type == AWS::Serverless::Function)
+```
+
+A more likely scenario might be finding specifically NodeJS Lambda functions. For example:
+
+```
+Resources.*.(Type == AWS::Lambda::Function,AWS::Serverless::Function & Properties.Runtime == <nodejs20>)
+```
+
+Queries are "split" by `|` operator and then "grouped" by `&` operator. That means if we want a query to match one set of criterion <i>or</i> another we could write:
+
+```
+Resources.*.(Type == AWS::Lambda::Function & Properties.Runtime == <nodejs20> | Type == AWS::Serverless::Function & Properties.Runtime == <nodejs20>)
+```
+
+<br/>
+
+# Using Pydantic Models
+
+In addition to traditional `pytest`-like assert statements, `cfn-lint` can validate results returned by queries via `Pydantic` models.
+
+For example, consider again the initial example where we validate the `Type` field of `Resource` objects.
+
+```python
+from cfn_check import Collection, Rule
+
+
+class ValidateResourceType(Collection):
+
+    @Rule(
+        "Resources.*.Type",
+        "It checks Resource.Type is correctly definined",
+    )
+    def validate_test(self, value: str): 
+        assert value is not None, '❌ Resource Type not defined'
+        assert isinstance(value, str), '❌ Resource Type not a string'
+```
+
+Rather than explicitly querying for the type field and writing assertions, we can instead define a `Pydantic` schema, then pass all `Resource` objects to that schema by specifying it as a Python type hint in our `Rule` method's signature.
+
+```python
+from cfn_check import Collection, Rule
+from pydantic import BaseModel, StrictStr
+
+class Resource(BaseModel):
+    Type: StrictStr
+
+
+class ValidateResourceType(Collection):
+
+    @Rule(
+        "Resources.*",
+        "It checks Resource.Type is correctly definined",
+    )
+    def validate_test(self, value: Resource):
+        assert value is not None
+```
+
+By deferring type and existence assertions to `Pydantic` models, you can focus your actual assertion logic on business/security policy checks.
+
+<br/>
+
+# Using .query()
+
+Some of the most challenging validations to write in CFN-Guard or CFN-Lint are those requring validation of other template information against an existing selection. For example, validating that a Lambda has a LoggingGroup attached and specified within the same template.
+
+CFN-Check makes performing these complex assertions intuitive and painless by allowing you to execute additional querieis within a rule via the `.query()` method. For example, to perform the LoggingGroup validation above you might write:
+
+```python
+@Rule("Resources.*.(Type in AWS::Lambda::Function,AWS::Serverless::Function)", "It validates a lambda is configured correctly")
+def validate_lambda(self, lambda_resource: Lambda):
+    assert isinstance(lambda_resource, Lambda), "Not a valid Lambda"
+    
+    log_groups = self.query(
+        f"Resources.{lambda_resource.Properties.LoggingConfig.LogGroup}",
+        transforms=[
+            lambda data: LoggingGroup(**data)
+        ]
+    )
+
+    assert log_groups is not None, f"No resources found for LoggingGroup {lambda_resource.Properties.LoggingConfig.LogGroup}"
+    assert len(log_groups) > 0, "No matching logging group found in Resources for Lambda"
+```       
+
+The  `query()` method accepts the following parameters:
+
+- `query - (str, required)`: A string CFN-Check query as used when defining Rules.
+- `document - (dict/list/any, optional)`: The filepath to a CloudFormation template document. This document must have been loaded by and specified to CFN-Check either via the default CLI path param, the `-i/--import-values` optional CLI arg, or under the `input_values` config key. This will cause the query specified in the call to execute against the template specified.
+- `transforms - (list[Function], optional)`: A list of functions that can be used to modify and filter returned results. In the example above, we use a single transform function to convert matches returned to a `LoggingGroup` Pydantic model instance.
+
+# The Rendering Engine
+
+### Overview
+
+In Version 0.6.X, CFN-Check introduced a rendering engine, which allows it
+to parse and execute Refs and all CloudFormation intrinsic functions via 
+either the CloudFormation document or user-supplied values. This additional
+also resulted in the:
+
+```bash
+cfn-check render <TEMPLATE_PATH >
+```
+
+command being added, allowing you to effectively "dry run" render your
+CloudFormation templates akin to the `helm template` command for Helm.
+
+By default, `cfn-check render` outputs to stdout, however you can easily
+save rendered output to a file via the `-o/--output-file` flag. For example:
+
+```bash
+cfn-check render template.yml -o rendered.yml
+```
+
+The `cfn-check render` command also offers the following options:
+
+- `-a/--attributes`:  A list of <key>=<value> input `!GetAtt` attributes to use
+- `-m/--mappings`: A list of <key>=<value> input `Mappings` to use
+- `-p/--parameters`: A list of <key>=<value> input `Parameters` to use
+- `-l/--log-level`: The log level to use
+
+### The Rendering Engine during Checks
+
+By default rendering is enabled when running `cfn-check` validation. You can 
+disable it by supplying `no-render` to the `-F/--flags` option as below:
+
+```bash
+cfn-check validate -F no-render -r rules.py template.yaml
+```
+
+Disabling rendering means CFN-Check will validate your template as-is, with
+no additional pre-processing and no application of user input values.
+
+> [!WARNING]
+> CloudFormation documents are <b>not</b> "plain yaml" and disabling
+> rendering means any dynamically determined values will likely fail
+> to pass validation, resulting in false positives for failures!