cdk-factory 0.19.13__py3-none-any.whl → 0.19.19__py3-none-any.whl

cdk_factory/stack_library/lambda_edge/EDGE_LOG_RETENTION_TODO.md

@@ -0,0 +1,226 @@
+# Lambda@Edge Log Retention - Implementation Plan
+
+## 🚨 Current Status: DISABLED
+
+Lambda@Edge log retention configuration has been **disabled** because edge log groups are created on-demand when the function is invoked at edge locations, not during CloudFormation deployment.
+
+## 🔍 Problem Analysis
+
+### Why Deployment-Time Configuration Fails
+1. **On-Demand Creation**: Lambda@Edge log groups are created only when the function is actually invoked at edge locations
+2. **Timing Issue**: CloudFormation deployment happens before any edge invocations occur
+3. **Error**: `The specified log group does not exist` when trying to set retention policies
+
+### Log Group Naming Pattern
+```
+Pattern: /aws/lambda/{edge-region}.{function-name}
+Example: /aws/lambda/eu-central-1.trav-talks-blue-green-edge-function
+Location: All edge log groups are created in us-east-1
+```
+
+## 💡 Proposed Solutions
+
+### Solution 1: EventBridge + Lambda (Recommended)
+```yaml
+# EventBridge rule to detect log group creation
+EventPattern:
+  source: ["aws.logs"]
+  detail-type: ["AWS API Call via CloudTrail"]
+  detail:
+    eventSource: ["logs.amazonaws.com"]
+    eventName: ["CreateLogGroup"]
+    requestParameters:
+      logGroupName: ["/aws/lambda/*.edge-function"]
+```
+
+**Implementation:**
+1. Create EventBridge rule that triggers on log group creation
+2. Lambda function receives event and sets retention policy
+3. Automatic handling of new edge log groups
+
+**Pros:**
+- Automatic and real-time
+- No manual intervention required
+- Handles all edge regions
+
+**Cons:**
+- Additional Lambda function to maintain
+- Requires CloudTrail enabled for CloudWatch Logs
+
+### Solution 2: Periodic Lambda Function
+```python
+def lambda_handler(event, context):
+    # Scan for edge log groups
+    log_groups = logs.describe_log_groups(
+        logGroupNamePrefix='/aws/lambda/eu-central-1.trav-talks-blue-green-edge-function'
+    )
+    
+    # Apply retention policy
+    for log_group in log_groups['logGroups']:
+        logs.put_retention_policy(
+            logGroupName=log_group['logGroupName'],
+            retentionInDays=7
+        )
+```
+
+**Implementation:**
+1. Create Lambda function on schedule (e.g., every hour)
+2. Scan for edge log groups with function name pattern
+3. Apply retention policy if not already set
+
+**Pros:**
+- Simple to implement
+- No CloudTrail dependency
+- Can handle existing log groups
+
+**Cons:**
+- Not real-time (delayed retention)
+- Runs periodically even when not needed
+
+### Solution 3: Post-Deployment Script
+```bash
+#!/bin/bash
+# Wait for edge log groups to appear
+function_name="trav-talks-blue-green-edge-function"
+edge_regions=("eu-central-1" "eu-west-1" "ap-southeast-1")
+
+for region in "${edge_regions[@]}"; do
+    log_group="/aws/lambda/${region}.${function_name}"
+    
+    # Wait for log group to exist
+    until aws logs describe-log-groups --log-group-name-prefix "$log_group" --region us-east-1; do
+        echo "Waiting for log group: $log_group"
+        sleep 30
+    done
+    
+    # Set retention policy
+    aws logs put-retention-policy --log-group-name "$log_group" --retention-in-days 7 --region us-east-1
+done
+```
+
+**Implementation:**
+1. Script runs after Lambda@Edge deployment
+2. Waits for edge log groups to be created
+3. Sets retention policies when they appear
+
+**Pros:**
+- Direct control over timing
+- No additional AWS resources needed
+
+**Cons:**
+- Manual process
+- Hard to determine when log groups will appear
+- Not automated
+
+### Solution 4: CloudWatch Logs Subscription
+```python
+# Lambda triggered by log group creation via subscription filter
+def lambda_handler(event, context):
+    for record in event['Records']:
+        log_group = record['logGroup']
+        if 'edge-function' in log_group:
+            # Set retention policy
+            logs.put_retention_policy(
+                logGroupName=log_group,
+                retentionInDays=7
+            )
+```
+
+**Implementation:**
+1. Create subscription filter on log group pattern
+2. Lambda function triggered by log events
+3. Set retention policy on first log event
+
+**Pros:**
+- Event-driven
+- No CloudTrail needed
+
+**Cons:**
+- Requires log group to exist first
+- Complex subscription filter setup
+
+## 🎯 Recommended Implementation
+
+### Phase 1: Quick Win (Solution 2)
+Implement periodic Lambda function as temporary solution:
+- Easy to implement quickly
+- Solves immediate problem
+- Can be replaced later with better solution
+
+### Phase 2: Production Solution (Solution 1)
+Implement EventBridge + Lambda for production:
+- Real-time response
+- Automatic handling
+- Best long-term solution
+
+## 📋 Implementation Steps for Solution 1
+
+### 1. Create EventBridge Rule
+```python
+event_rule = events.Rule(
+    self, "EdgeLogGroupRule",
+    event_pattern=events.EventPattern(
+        source=["aws.logs"],
+        detail_type=["AWS API Call via CloudTrail"],
+        detail={
+            "eventSource": ["logs.amazonaws.com"],
+            "eventName": ["CreateLogGroup"],
+            "requestParameters": {
+                "logGroupName": [{"prefix": "/aws/lambda/"}]
+            }
+        }
+    )
+)
+```
+
+### 2. Create Lambda Function
+```python
+retention_handler = _lambda.Function(
+    self, "EdgeLogRetentionHandler",
+    runtime=_lambda.Runtime.PYTHON_3_9,
+    handler="handler.lambda_handler",
+    code=_lambda.Code.from_asset("lambda/edge_log_retention"),
+    environment={
+        "RETENTION_DAYS": "7",
+        "FUNCTION_NAME_PATTERN": "*edge-function"
+    }
+)
+```
+
+### 3. Add Permissions
+```python
+retention_handler.add_to_role_policy(
+    iam.PolicyStatement(
+        actions=["logs:PutRetentionPolicy", "logs:DescribeLogGroups"],
+        resources=["*"]
+    )
+)
+```
+
+### 4. Connect EventBridge to Lambda
+```python
+event_rule.add_target(targets.LambdaFunction(retention_handler))
+```
+
+## 🔧 Current Configuration
+
+The edge log retention configuration is currently **disabled** in the Lambda Edge stack:
+
+```python
+def _configure_edge_log_retention(self, function_name: str) -> None:
+    # DISABLED: See implementation plan above
+    logger.warning("Edge log retention disabled - see TODO for implementation")
+    return
+```
+
+## 📊 Configuration Impact
+
+| Setting | Current Behavior | Target Behavior |
+|---------|------------------|-----------------|
+| `edge_log_retention_days` | Warning logged, no action applied | Retention policy set on all edge log groups |
+| Edge log groups | Created with default retention (never expire) | Created with specified retention (e.g., 7 days) |
+| Cost impact | Potential high log storage costs | Controlled log storage costs |
+
+---
+
+**Status**: Ready for implementation when edge log retention is required.

cdk_factory/stack_library/lambda_edge/lambda_edge_stack.py

@@ -248,19 +248,12 @@ class LambdaEdgeStack(IStack, StandardizedSsmMixin):
             self.edge_config.runtime,
             _lambda.Runtime.PYTHON_3_11
         )
-        
-        # Lambda@Edge does NOT support environment variables
-        # Configuration must be handled via:
-        # 1. Hardcoded in the function code
-        # 2. Fetched from SSM Parameter Store at runtime
-        # 3. Other configuration mechanisms
-        
+
         # Log warning if environment variables are configured
         if self.edge_config.environment:
             logger.warning(
                 f"Lambda@Edge function '{function_name}' has environment variables configured, "
-                "but Lambda@Edge does not support environment variables. "
-                "The function must fetch these values from SSM Parameter Store at runtime."
+                "but Lambda@Edge does not support environment variables. The function must fetch these values from SSM Parameter Store at runtime."
             )
             for key, value in self.edge_config.environment.items():
                 logger.warning(f"  - {key}: {value}")
@@ -298,7 +291,20 @@ class LambdaEdgeStack(IStack, StandardizedSsmMixin):
                 )
             )
         
-        # Create the Lambda function WITHOUT environment variables
+        # Add Secrets Manager permissions for origin secret access
+        execution_role.add_to_policy(
+            iam.PolicyStatement(
+                effect=iam.Effect.ALLOW,
+                actions=[
+                    "secretsmanager:GetSecretValue",
+                    "secretsmanager:DescribeSecret"
+                ],
+                resources=[
+                    f"arn:aws:secretsmanager:*:{cdk.Aws.ACCOUNT_ID}:secret:{self.deployment.environment}/{self.workload.name}/origin-secret*"
+                ]
+            )
+        )
+    
         self.function = _lambda.Function(
             self,
             function_name,
@@ -311,6 +317,7 @@ class LambdaEdgeStack(IStack, StandardizedSsmMixin):
             description=self.edge_config.description,
             role=execution_role,
             # Lambda@Edge does NOT support environment variables
+            # Configuration must be fetched from SSM at runtime
             log_retention=logs.RetentionDays.ONE_WEEK,
         )
         
@@ -365,60 +372,45 @@ class LambdaEdgeStack(IStack, StandardizedSsmMixin):
 
     def _configure_edge_log_retention(self, function_name: str) -> None:
         """
-        Configure log retention for Lambda@Edge regional logs.
+        Configure log retention for Lambda@Edge log groups in all edge regions
+        
+        TODO: IMPLEMENT POST-DEPLOYMENT SOLUTION
+        --------------------------------------
+        Lambda@Edge log groups are created on-demand when the function is invoked
+        at edge locations, not during deployment. This means we cannot set retention
+        policies during CloudFormation deployment.
+        
+        Possible solutions to implement:
+        1. EventBridge rule that triggers on log group creation
+        2. Custom Lambda function that runs periodically to set retention
+        3. Post-deployment script that waits for log groups to appear
+        4. CloudWatch Logs subscription filter that handles new log groups
         
-        Lambda@Edge creates log groups in multiple regions that need
-        separate retention configuration from the primary log group.
+        Current behavior: DISABLED to prevent deployment failures
         """
-        from aws_cdk import custom_resources as cr
         
-        # Get edge log retention from config (default to same as primary logs)
+        # DISABLED: Edge log groups don't exist during deployment
+        # Lambda@Edge creates log groups on-demand at edge locations
+        # Setting retention policies during deployment fails with "log group does not exist"
+        
         edge_retention_days = self.edge_config.dictionary.get("edge_log_retention_days", 7)
+        logger.warning(
+            f"Edge log retention configuration disabled - log groups are created on-demand. "
+            f"Desired retention: {edge_retention_days} days. "
+            f"See TODO in _configure_edge_log_retention() for implementation approach."
+        )
         
-        # List of common Lambda@Edge regions
-        edge_regions = [
-            'us-east-1', 'us-east-2', 'us-west-1', 'us-west-2',
-            'eu-west-1', 'eu-west-2', 'eu-central-1',
-            'ap-southeast-1', 'ap-southeast-2', 'ap-northeast-1',
-            'ca-central-1', 'sa-east-1'
-        ]
-        
-        # Create custom resource to set log retention for each region
-        for region in edge_regions:
-            log_group_name = f"/aws/lambda/{region}.{function_name}"
-            
-            # Use AwsCustomResource to set log retention
-            cr.AwsCustomResource(
-                self, f"EdgeLogRetention-{region}",
-                on_update={
-                    "service": "Logs",
-                    "action": "putRetentionPolicy",
-                    "parameters": {
-                        "logGroupName": log_group_name,
-                        "retentionInDays": edge_retention_days
-                    },
-                    "physical_resource_id": cr.PhysicalResourceId.from_response("logGroupName")
-                },
-                on_delete={
-                    "service": "Logs", 
-                    "action": "deleteRetentionPolicy",
-                    "parameters": {
-                        "logGroupName": log_group_name
-                    },
-                    "physical_resource_id": cr.PhysicalResourceId.from_response("logGroupName")
-                },
-                policy=cr.AwsCustomResourcePolicy.from_sdk_calls(
-                    resources=[f"arn:aws:logs:{region}:*:log-group:{log_group_name}*"]
-                )
-            )
+        # TODO: Implement one of these solutions:
+        # 1. EventBridge + Lambda: Trigger on log group creation and set retention
+        # 2. Periodic Lambda: Scan for edge log groups and apply retention policies  
+        # 3. Post-deployment script: Wait for log groups to appear after edge replication
+        # 4. CloudWatch Logs subscription: Process new log group events
         
-        logger.info(f"Configured edge log retention to {edge_retention_days} days for {len(edge_regions)} regions")
+        return
 
     def _add_outputs(self, function_name: str) -> None:
         """Add CloudFormation outputs and SSM exports"""
         
-        
-        
         # SSM Parameter Store exports (if configured)
         ssm_exports = self.edge_config.dictionary.get("ssm", {}).get("exports", {})
         if ssm_exports:
@@ -440,40 +432,31 @@ class LambdaEdgeStack(IStack, StandardizedSsmMixin):
                         description=f"{key} for Lambda@Edge function {function_name}"
                     )
         
-        # Export environment variables as SSM parameters
-        # Since Lambda@Edge doesn't support environment variables, we export them
-        # to SSM so the Lambda function can fetch them at runtime
-        if self.edge_config.environment:
-            logger.info("Exporting Lambda@Edge environment variables as SSM parameters")
-            env_ssm_exports = self.edge_config.dictionary.get("environment_ssm_exports", {})
-            
-            # If no explicit environment_ssm_exports, create default SSM paths
-            if not env_ssm_exports:
-                # Auto-generate SSM parameter names based on environment variable names
-                for env_key in self.edge_config.environment.keys():
-                    # Use snake_case version of the key for SSM path
-                    ssm_key = env_key.lower().replace('_', '-')
-                    env_ssm_exports[env_key] = f"/{self.deployment.environment}/{function_name}/{ssm_key}"
-            
-            # Resolve and export environment variables to SSM
-            resolved_env = self._resolve_environment_variables()
-            for env_key, ssm_path in env_ssm_exports.items():
-                if env_key in resolved_env:
-                    env_value = resolved_env[env_key]
-                    
-                    # Handle empty values - SSM doesn't allow empty strings
-                    # Use sentinel value "NONE" to indicate explicitly unset
-                    if not env_value or (isinstance(env_value, str) and env_value.strip() == ""):
-                        env_value = "NONE"
-                        logger.info(
-                            f"Environment variable {env_key} is empty - setting SSM parameter to 'NONE'. "
-                            f"Lambda function should treat 'NONE' as unset/disabled."
-                        )
-                    
-                    self.export_ssm_parameter(
-                        self,
-                        f"env-{env_key}-param",
-                        env_value,
-                        ssm_path,
-                        description=f"Configuration for Lambda@Edge: {env_key}"
-                    )
+        # Export the complete configuration as a single SSM parameter
+        config_ssm_path = f"/{self.deployment.environment}/{self.workload.name}/lambda-edge/config"
+        configuration = self.edge_config.dictionary.get("configuration", {})
+        environment_variables = configuration.get("environment_variables", {})
+        
+        full_config = {
+            "environment_variables": environment_variables
+        }
+        
+        self.export_ssm_parameter(
+            self,
+            "full-config-param",
+            json.dumps(full_config),
+            config_ssm_path,
+            description=f"Complete Lambda@Edge configuration for {function_name} - update this for dynamic changes"
+        )
+        
+        # Export cache TTL parameter for dynamic cache control
+        cache_ttl_ssm_path = f"/{self.deployment.environment}/{self.workload.name}/lambda-edge/cache-ttl"
+        default_cache_ttl = self.edge_config.dictionary.get("cache_ttl_seconds", 300)  # Default 5 minutes
+        
+        self.export_ssm_parameter(
+            self,
+            "cache-ttl-param",
+            str(default_cache_ttl),
+            cache_ttl_ssm_path,
+            description=f"Lambda@Edge configuration cache TTL in seconds for {function_name} - adjust for maintenance windows (30-3600)"
+        )

cdk_factory/version.py

@@ -1 +1 @@
-__version__ = "0.19.13"
+__version__ = "0.19.19"

{cdk_factory-0.19.13.dist-info → cdk_factory-0.19.19.dist-info}/METADATA

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: cdk_factory
-Version: 0.19.13
+Version: 0.19.19
 Summary: CDK Factory. A QuickStarter and best practices setup for CDK projects
 Author-email: Eric Wilson <eric.wilson@geekcafe.com>
 License: MIT License

{cdk_factory-0.19.13.dist-info → cdk_factory-0.19.19.dist-info}/RECORD

@@ -2,7 +2,7 @@ cdk_factory/__init__.py,sha256=47DEQpj8HBSa-_TImW-5JCeuQeRkm5NMpJWZG3hSuFU,0
 cdk_factory/app.py,sha256=RnX0-pwdTAPAdKJK_j13Zl8anf9zYKBwboR0KA8K8xM,10346
 cdk_factory/cdk.json,sha256=SKZKhJ2PBpFH78j-F8S3VDYW-lf76--Q2I3ON-ZIQfw,3106
 cdk_factory/cli.py,sha256=FGbCTS5dYCNsfp-etshzvFlGDCjC28r6rtzYbe7KoHI,6407
-cdk_factory/version.py,sha256=qfLAZUWz8ZIxPUsFWlqQCfCwTNmUwrL2jNL4w6ZRFi0,24
+cdk_factory/version.py,sha256=7SkXrcCPPywUj5iTSUoEUN_6FonxdNwtTRVl_D3253s,24
 cdk_factory/builds/README.md,sha256=9BBWd7bXpyKdMU_g2UljhQwrC9i5O_Tvkb6oPvndoZk,90
 cdk_factory/commands/command_loader.py,sha256=QbLquuP_AdxtlxlDy-2IWCQ6D-7qa58aphnDPtp_uTs,3744
 cdk_factory/configurations/base_config.py,sha256=eJ3Pl3GWk1jVr_bYQaaWlw4_-ZiFGaiXllI_fOOX1i0,9323
@@ -100,8 +100,9 @@ cdk_factory/stack_library/ecr/ecr_stack.py,sha256=KLbd5WN5-ZiojsS5wJ4PX-tIL0cCyl
 cdk_factory/stack_library/ecs/__init__.py,sha256=o5vGDtD_h-gVXb3-Ysr8xUNpEcMsnmMVgZv2Pupcdow,219
 cdk_factory/stack_library/ecs/ecs_cluster_stack.py,sha256=sAPTLU5CAwMoLTW_pNy_cd0OtVkfDR7IxxsSq5AE0yo,12091
 cdk_factory/stack_library/ecs/ecs_service_stack.py,sha256=KB4YCIsMm5JIGM9Bm-bKcr3eX5xXFgnoA7jST_ekK44,28209
+cdk_factory/stack_library/lambda_edge/EDGE_LOG_RETENTION_TODO.md,sha256=nD49nLm5OyrZUvcGNFBy9H1MfSUOuZ7sasHNI-IO0Zk,6635
 cdk_factory/stack_library/lambda_edge/__init__.py,sha256=ByBJ_CWdc4UtTmFBZH-6pzBMNkjkdtE65AmnB0Fs6lM,156
-cdk_factory/stack_library/lambda_edge/lambda_edge_stack.py,sha256=eHh_k4mbNp1prEnNvKqfK82lLNrMZS-7HAaUYzFEoOU,21040
+cdk_factory/stack_library/lambda_edge/lambda_edge_stack.py,sha256=b8PE4v5zGsqu_mVOaPejb8JKkV6b83N-VS0AxSlLTtk,20119
 cdk_factory/stack_library/load_balancer/__init__.py,sha256=wZpKw2OecLJGdF5mPayCYAEhu2H3c2gJFFIxwXftGDU,52
 cdk_factory/stack_library/load_balancer/load_balancer_stack.py,sha256=ApW5q3SAvSJtiK0RInNljmubqXqKZU5QBAaUoeIW-pM,28287
 cdk_factory/stack_library/monitoring/__init__.py,sha256=k1G_KDx47Aw0UugaL99PN_TKlyLK4nkJVApCaAK7GJg,153
@@ -136,8 +137,8 @@ cdk_factory/utilities/os_execute.py,sha256=5Op0LY_8Y-pUm04y1k8MTpNrmQvcLmQHPQITE
 cdk_factory/utils/api_gateway_utilities.py,sha256=If7Xu5s_UxmuV-kL3JkXxPLBdSVUKoLtohm0IUFoiV8,4378
 cdk_factory/validation/config_validator.py,sha256=Pb0TkLiPFzUplBOgMorhRCVm08vEzZhRU5xXCDTa5CA,17602
 cdk_factory/workload/workload_factory.py,sha256=yDI3cRhVI5ELNDcJPLpk9UY54Uind1xQoV3spzT4z7E,6068
-cdk_factory-0.19.13.dist-info/METADATA,sha256=Rw6GGM7Hl5-md3ntKrGRfX2u1ppfZRfIextBHMW-FsE,2452
-cdk_factory-0.19.13.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
-cdk_factory-0.19.13.dist-info/entry_points.txt,sha256=S1DPe0ORcdiwEALMN_WIo3UQrW_g4YdQCLEsc_b0Swg,53
-cdk_factory-0.19.13.dist-info/licenses/LICENSE,sha256=NOtdOeLwg2il_XBJdXUPFPX8JlV4dqTdDGAd2-khxT8,1066
-cdk_factory-0.19.13.dist-info/RECORD,,
+cdk_factory-0.19.19.dist-info/METADATA,sha256=syphI_KKN3_606LBfCKYQLU_3YX4ppprIwsKcAENNBk,2452
+cdk_factory-0.19.19.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
+cdk_factory-0.19.19.dist-info/entry_points.txt,sha256=S1DPe0ORcdiwEALMN_WIo3UQrW_g4YdQCLEsc_b0Swg,53
+cdk_factory-0.19.19.dist-info/licenses/LICENSE,sha256=NOtdOeLwg2il_XBJdXUPFPX8JlV4dqTdDGAd2-khxT8,1066
+cdk_factory-0.19.19.dist-info/RECORD,,