driftwatch-cli 3.0.0__tar.gz → 3.0.2__tar.gz

driftwatch_cli-3.0.2/PKG-INFO

@@ -0,0 +1,181 @@
+Metadata-Version: 2.4
+Name: driftwatch-cli
+Version: 3.0.2
+Summary: CLI tool that detects Terraform infrastructure drift against live AWS, explains it with AI, and guides remediation.
+Author: Nitin Gupta
+License: MIT
+Project-URL: Homepage, https://github.com/hastagnitin/driftwatch
+Project-URL: Repository, https://github.com/hastagnitin/driftwatch
+Keywords: terraform,aws,drift-detection,iac,devops,cloud-security,remediation
+Classifier: Development Status :: 4 - Beta
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.10
+Classifier: Programming Language :: Python :: 3.11
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Topic :: System :: Systems Administration
+Classifier: Topic :: Security
+Requires-Python: >=3.10
+Description-Content-Type: text/markdown
+License-File: LICENSE
+Requires-Dist: boto3>=1.34.0
+Requires-Dist: typer>=0.9.0
+Requires-Dist: requests>=2.31.0
+Requires-Dist: python-dotenv>=1.0.0
+Provides-Extra: postgres
+Requires-Dist: psycopg2-binary>=2.9.0; extra == "postgres"
+Provides-Extra: dev
+Requires-Dist: pytest>=7.0.0; extra == "dev"
+Requires-Dist: pytest-cov>=4.0.0; extra == "dev"
+Requires-Dist: moto[all]>=5.0.0; extra == "dev"
+Requires-Dist: build; extra == "dev"
+Requires-Dist: twine; extra == "dev"
+Dynamic: license-file
+
+# DriftWatch 🛡️
+
+**DriftWatch** is a production-ready CLI tool and automation engine that detects Terraform infrastructure drift against live AWS environments, explains the security and reliability impact using AI, and safely guides remediation.
+
+---
+
+## 🚀 Key Features
+
+- **Multi-Resource Drift Detection**: Continuously monitors and compares EC2 instances, S3 buckets, Security Groups, RDS databases, Lambda functions, and IAM roles against your Terraform state.
+- **Data-Driven Severity Scoring**: Evaluates changes dynamically at the attribute level (e.g. security group open ports vs description updates) to classify drifts as `CRITICAL`, `HIGH`, `MEDIUM`, or `LOW`.
+- **AI-Powered Risk Summaries**: Integrates with LLMs to provide plain-English security analysis and compliance impact assessments.
+- **Deterministic IaC Remediation**: Recommends safe, template-generated `terraform import` and `terraform apply` commands rather than hallucinated AI outputs.
+- **Guarded Auto-Remediation**: Pre-flight validation checks for EC2 (EBS verification, Spot skip, running state), RDS maintenance-window defaults, and explicit interactive confirmations.
+- **Multi-Channel Alerting**: Instant notifications via Telegram, Slack, and Email.
+- **CI/CD Quality Gate**: Built-in GitHub Actions integration to enforce zero-tolerance drift policies in pull requests.
+
+---
+
+## 🏛️ Architecture Overview
+
+```
+driftwatch/
+├── drift_engine/              # Core drift detection & reconciliation engine
+│   ├── aws_client.py         # Live AWS resource discovery (boto3)
+│   ├── core.py               # Diff evaluation & data-driven severity engine
+│   ├── database.py           # PostgreSQL scan history recorder
+│   ├── explain.py            # AI risk summaries & deterministic IaC templates
+│   ├── models.py             # Data models & attribute severity tables
+│   ├── notifications.py      # Alert dispatcher (Telegram, Slack, Email)
+│   ├── remediation.py        # Guarded auto-remediation handlers
+│   └── tf_parser.py          # Terraform state JSON parser
+├── driftwatch/               # CLI Entrypoint (Typer)
+│   └── cli.py                # Command definitions: scan, explain, remediate
+├── terraform/                # Example infrastructure and state configuration
+├── kubernetes/               # Kubernetes CronJob deployment
+└── tests/                    # Comprehensive unit tests with moto AWS mocks
+```
+
+---
+
+## 📋 Prerequisites
+
+- **Python**: `>= 3.10`
+- **AWS Credentials**: Configured via environment variables, IAM roles, or AWS CLI credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`).
+- **Terraform State File**: Local JSON state or remote state (`terraform.tfstate`).
+- **PostgreSQL** *(Optional)*: For persistent scan audit history.
+- **Groq API Key** *(Optional)*: `GROQ_API_KEY` for AI risk explanations.
+
+---
+
+## 📦 Installation
+
+### From PyPI (Recommended)
+```bash
+pip install driftwatch-cli
+```
+
+### From Source (Local Development)
+```bash
+git clone https://github.com/hastagnitin/driftwatch.git
+cd driftwatch
+pip install -e .[dev]
+```
+
+---
+
+## ⚙️ Configuration
+
+Create a `.env` file in the root directory:
+
+```env
+AWS_DEFAULT_REGION=ap-south-1
+TF_STATE_PATH=terraform/terraform.tfstate
+
+# Optional: AI Risk Summaries
+GROQ_API_KEY=your_groq_api_key
+
+# Optional: Notifications
+SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
+TELEGRAM_BOT_TOKEN=your_telegram_bot_token
+TELEGRAM_CHAT_ID=your_telegram_chat_id
+
+# Optional: PostgreSQL Database
+DB_HOST=localhost
+DB_PORT=5432
+DB_NAME=driftwatch
+DB_USER=postgres
+DB_PASSWORD=your_db_password
+```
+
+---
+
+## 💻 Usage & CLI Commands
+
+### 1. Scan for Drift
+Scan live AWS infrastructure against your Terraform state:
+```bash
+# Basic scan
+driftwatch scan --region ap-south-1 --state terraform/terraform.tfstate
+
+# Enforce CI Gate (fails build if CRITICAL drift is found)
+driftwatch scan --region ap-south-1 --fail-on CRITICAL
+```
+
+### 2. Explain Drift
+Generate AI risk analysis and deterministic IaC fix recommendations:
+```bash
+driftwatch explain sg-0123456789abcdef0 --region ap-south-1
+```
+
+### 3. Remediate Drift
+Safely remediate drifted resources back to IaC specifications:
+```bash
+# Dry run mode (default)
+driftwatch remediate sg-0123456789abcdef0 --region ap-south-1 --dry-run
+
+# Apply mode with interactive confirmation
+driftwatch remediate sg-0123456789abcdef0 --region ap-south-1 --apply
+```
+
+---
+
+## ⚠️ Security & Safety Guidelines
+
+> [!WARNING]
+> **Auto-Remediation Safety**:
+> - Automated drift remediation is intended for **Development** and **Staging** environments.
+> - In **Production**, DriftWatch enforces manual confirmation prompts (`confirm_action()`) and recommends template-generated `terraform apply` / `terraform import` workflows.
+> - RDS modifications default to maintenance windows (`ApplyImmediately=False`) to avoid unplanned reboots.
+
+---
+
+## 🧪 Testing
+
+Run the test suite with test coverage:
+```bash
+pytest tests/ -v --cov=drift_engine --cov=driftwatch --cov-report=term-missing
+```
+
+---
+
+## 📄 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

driftwatch_cli-3.0.2/README.md

@@ -0,0 +1,144 @@
+# DriftWatch 🛡️
+
+**DriftWatch** is a production-ready CLI tool and automation engine that detects Terraform infrastructure drift against live AWS environments, explains the security and reliability impact using AI, and safely guides remediation.
+
+---
+
+## 🚀 Key Features
+
+- **Multi-Resource Drift Detection**: Continuously monitors and compares EC2 instances, S3 buckets, Security Groups, RDS databases, Lambda functions, and IAM roles against your Terraform state.
+- **Data-Driven Severity Scoring**: Evaluates changes dynamically at the attribute level (e.g. security group open ports vs description updates) to classify drifts as `CRITICAL`, `HIGH`, `MEDIUM`, or `LOW`.
+- **AI-Powered Risk Summaries**: Integrates with LLMs to provide plain-English security analysis and compliance impact assessments.
+- **Deterministic IaC Remediation**: Recommends safe, template-generated `terraform import` and `terraform apply` commands rather than hallucinated AI outputs.
+- **Guarded Auto-Remediation**: Pre-flight validation checks for EC2 (EBS verification, Spot skip, running state), RDS maintenance-window defaults, and explicit interactive confirmations.
+- **Multi-Channel Alerting**: Instant notifications via Telegram, Slack, and Email.
+- **CI/CD Quality Gate**: Built-in GitHub Actions integration to enforce zero-tolerance drift policies in pull requests.
+
+---
+
+## 🏛️ Architecture Overview
+
+```
+driftwatch/
+├── drift_engine/              # Core drift detection & reconciliation engine
+│   ├── aws_client.py         # Live AWS resource discovery (boto3)
+│   ├── core.py               # Diff evaluation & data-driven severity engine
+│   ├── database.py           # PostgreSQL scan history recorder
+│   ├── explain.py            # AI risk summaries & deterministic IaC templates
+│   ├── models.py             # Data models & attribute severity tables
+│   ├── notifications.py      # Alert dispatcher (Telegram, Slack, Email)
+│   ├── remediation.py        # Guarded auto-remediation handlers
+│   └── tf_parser.py          # Terraform state JSON parser
+├── driftwatch/               # CLI Entrypoint (Typer)
+│   └── cli.py                # Command definitions: scan, explain, remediate
+├── terraform/                # Example infrastructure and state configuration
+├── kubernetes/               # Kubernetes CronJob deployment
+└── tests/                    # Comprehensive unit tests with moto AWS mocks
+```
+
+---
+
+## 📋 Prerequisites
+
+- **Python**: `>= 3.10`
+- **AWS Credentials**: Configured via environment variables, IAM roles, or AWS CLI credentials (`AWS_ACCESS_KEY_ID`, `AWS_SECRET_ACCESS_KEY`, `AWS_DEFAULT_REGION`).
+- **Terraform State File**: Local JSON state or remote state (`terraform.tfstate`).
+- **PostgreSQL** *(Optional)*: For persistent scan audit history.
+- **Groq API Key** *(Optional)*: `GROQ_API_KEY` for AI risk explanations.
+
+---
+
+## 📦 Installation
+
+### From PyPI (Recommended)
+```bash
+pip install driftwatch-cli
+```
+
+### From Source (Local Development)
+```bash
+git clone https://github.com/hastagnitin/driftwatch.git
+cd driftwatch
+pip install -e .[dev]
+```
+
+---
+
+## ⚙️ Configuration
+
+Create a `.env` file in the root directory:
+
+```env
+AWS_DEFAULT_REGION=ap-south-1
+TF_STATE_PATH=terraform/terraform.tfstate
+
+# Optional: AI Risk Summaries
+GROQ_API_KEY=your_groq_api_key
+
+# Optional: Notifications
+SLACK_WEBHOOK_URL=https://hooks.slack.com/services/...
+TELEGRAM_BOT_TOKEN=your_telegram_bot_token
+TELEGRAM_CHAT_ID=your_telegram_chat_id
+
+# Optional: PostgreSQL Database
+DB_HOST=localhost
+DB_PORT=5432
+DB_NAME=driftwatch
+DB_USER=postgres
+DB_PASSWORD=your_db_password
+```
+
+---
+
+## 💻 Usage & CLI Commands
+
+### 1. Scan for Drift
+Scan live AWS infrastructure against your Terraform state:
+```bash
+# Basic scan
+driftwatch scan --region ap-south-1 --state terraform/terraform.tfstate
+
+# Enforce CI Gate (fails build if CRITICAL drift is found)
+driftwatch scan --region ap-south-1 --fail-on CRITICAL
+```
+
+### 2. Explain Drift
+Generate AI risk analysis and deterministic IaC fix recommendations:
+```bash
+driftwatch explain sg-0123456789abcdef0 --region ap-south-1
+```
+
+### 3. Remediate Drift
+Safely remediate drifted resources back to IaC specifications:
+```bash
+# Dry run mode (default)
+driftwatch remediate sg-0123456789abcdef0 --region ap-south-1 --dry-run
+
+# Apply mode with interactive confirmation
+driftwatch remediate sg-0123456789abcdef0 --region ap-south-1 --apply
+```
+
+---
+
+## ⚠️ Security & Safety Guidelines
+
+> [!WARNING]
+> **Auto-Remediation Safety**:
+> - Automated drift remediation is intended for **Development** and **Staging** environments.
+> - In **Production**, DriftWatch enforces manual confirmation prompts (`confirm_action()`) and recommends template-generated `terraform apply` / `terraform import` workflows.
+> - RDS modifications default to maintenance windows (`ApplyImmediately=False`) to avoid unplanned reboots.
+
+---
+
+## 🧪 Testing
+
+Run the test suite with test coverage:
+```bash
+pytest tests/ -v --cov=drift_engine --cov=driftwatch --cov-report=term-missing
+```
+
+---
+
+## 📄 License
+
+This project is licensed under the MIT License - see the [LICENSE](LICENSE) file for details.

{driftwatch_cli-3.0.0 → driftwatch_cli-3.0.2}/drift_engine/aws_client.py

@@ -1,41 +1,59 @@
+import os
+import sys
 import boto3
 from datetime import datetime, timedelta
 
-def fetch_live_ec2_instances(region: str) -> dict:
-    ec2 = boto3.client("ec2", region_name=region)
+def get_boto3_session(profile: str = None, region: str = None) -> boto3.Session:
+    actual_profile = profile or os.environ.get("AWS_PROFILE")
+    actual_region = region or os.environ.get("AWS_DEFAULT_REGION")
+    if actual_profile:
+        return boto3.Session(profile_name=actual_profile, region_name=actual_region)
+    return boto3.Session(region_name=actual_region)
+
+def get_boto3_client(service_name: str, profile: str = None, region: str = None):
+    session = get_boto3_session(profile=profile, region=region)
+    return session.client(service_name, region_name=region or session.region_name)
+
+def fetch_live_ec2_instances(region: str, profile: str = None) -> dict:
+    ec2 = get_boto3_client("ec2", profile=profile, region=region)
     live = {}
-    paginator = ec2.get_paginator("describe_instances")
-    
-    for page in paginator.paginate():
-        for reservation in page["Reservations"]:
-            for instance in reservation["Instances"]:
-                if instance["State"]["Name"] == "terminated":
-                    continue
-                
-                tags_list = instance.get("Tags", [])
-                tags_dict = {t["Key"]: t["Value"] for t in tags_list}
-                name = tags_dict.get("Name", "Unknown")
-                
-                sg_ids = []
-                for sg in instance.get("SecurityGroups", []):
-                    sg_ids.append(sg.get("GroupId"))
-                sg_ids.sort()
-                
-                live[instance["InstanceId"]] = {
-                    "type": "aws_instance",
-                    "name": name,
-                    "attributes": {
-                        "id": instance["InstanceId"],
-                        "instance_type": instance["InstanceType"],
-                        "ami": instance["ImageId"],
-                        "tags": tags_dict,
-                        "vpc_security_group_ids": sg_ids
-                    },
-                }
+    try:
+        paginator = ec2.get_paginator("describe_instances")
+        for page in paginator.paginate():
+            for reservation in page.get("Reservations", []):
+                for instance in reservation.get("Instances", []):
+                    if instance.get("State", {}).get("Name") == "terminated":
+                        continue
+                    
+                    tags_list = instance.get("Tags", [])
+                    tags_dict = {t["Key"]: t["Value"] for t in tags_list if "Key" in t and "Value" in t}
+                    name = tags_dict.get("Name", "Unknown")
+                    
+                    sg_ids = []
+                    for sg in instance.get("SecurityGroups", []):
+                        if "GroupId" in sg:
+                            sg_ids.append(sg.get("GroupId"))
+                    sg_ids.sort()
+                    
+                    instance_id = instance["InstanceId"]
+                    live[instance_id] = {
+                        "type": "aws_instance",
+                        "name": name,
+                        "attributes": {
+                            "id": instance_id,
+                            "instance_type": instance.get("InstanceType"),
+                            "ami": instance.get("ImageId"),
+                            "tags": tags_dict,
+                            "vpc_security_group_ids": sg_ids
+                        },
+                    }
+    except Exception as e:
+        print(f"Failed to fetch aws_instance: {e}", file=sys.stderr)
+        return None
     return live
 
-def fetch_live_s3_buckets(region: str) -> dict:
-    s3 = boto3.client("s3", region_name=region)
+def fetch_live_s3_buckets(region: str, profile: str = None) -> dict:
+    s3 = get_boto3_client("s3", profile=profile, region=region)
     live = {}
     try:
         response = s3.list_buckets()
@@ -44,7 +62,7 @@ def fetch_live_s3_buckets(region: str) -> dict:
             try:
                 tags_response = s3.get_bucket_tagging(Bucket=bucket_name)
                 tags_list = tags_response.get("TagSet", [])
-                tags_dict = {t["Key"]: t["Value"] for t in tags_list}
+                tags_dict = {t["Key"]: t["Value"] for t in tags_list if "Key" in t and "Value" in t}
                 name = tags_dict.get("Name", bucket_name)
             except Exception:
                 tags_dict = {}
@@ -60,22 +78,22 @@ def fetch_live_s3_buckets(region: str) -> dict:
                 },
             }
     except Exception as e:
-        pass
+        print(f"Failed to fetch aws_s3_bucket: {e}", file=sys.stderr)
+        return None
     return live
 
-def fetch_live_security_groups(region: str) -> dict:
-    ec2 = boto3.client("ec2", region_name=region)
+def fetch_live_security_groups(region: str, profile: str = None) -> dict:
+    ec2 = get_boto3_client("ec2", profile=profile, region=region)
     live = {}
-    paginator = ec2.get_paginator("describe_security_groups")
-    
     try:
+        paginator = ec2.get_paginator("describe_security_groups")
         for page in paginator.paginate():
-            for sg in page["SecurityGroups"]:
+            for sg in page.get("SecurityGroups", []):
                 sg_id = sg["GroupId"]
-                sg_name = sg["GroupName"]
+                sg_name = sg.get("GroupName", "")
                 
                 tags_list = sg.get("Tags", [])
-                tags_dict = {t["Key"]: t["Value"] for t in tags_list}
+                tags_dict = {t["Key"]: t["Value"] for t in tags_list if "Key" in t and "Value" in t}
                 name_tag = tags_dict.get("Name", sg_name)
                 
                 ingress_rules = []
@@ -113,25 +131,26 @@ def fetch_live_security_groups(region: str) -> dict:
                     },
                 }
     except Exception as e:
-        pass
+        print(f"Failed to fetch aws_security_group: {e}", file=sys.stderr)
+        return None
     return live
 
-def fetch_live_rds_instances(region: str = "ap-south-1") -> dict:
-    rds = boto3.client("rds", region_name=region)
+def fetch_live_rds_instances(region: str, profile: str = None) -> dict:
+    rds = get_boto3_client("rds", profile=profile, region=region)
     live = {}
-    paginator = rds.get_paginator("describe_db_instances")
     try:
+        paginator = rds.get_paginator("describe_db_instances")
         for page in paginator.paginate():
-            for db in page["DBInstances"]:
-                db_id = db.get("DbiResourceId") 
-                db_name = db["DBInstanceIdentifier"]
+            for db in page.get("DBInstances", []):
+                db_id = db.get("DBInstanceIdentifier")
                 if not db_id:
                     continue
                 live[db_id] = {
                     "type": "aws_db_instance",
-                    "name": db_name,
+                    "name": db_id,
                     "attributes": {
                         "id": db_id,
+                        "identifier": db_id,
                         "allocated_storage": db.get("AllocatedStorage"),
                         "engine": db.get("Engine"),
                         "engine_version": db.get("EngineVersion"),
@@ -140,17 +159,20 @@ def fetch_live_rds_instances(region: str = "ap-south-1") -> dict:
                     },
                 }
     except Exception as e:
-        pass
+        print(f"Failed to fetch aws_db_instance: {e}", file=sys.stderr)
+        return None
     return live
 
-def fetch_live_lambda_functions(region: str = "ap-south-1") -> dict:
-    lambda_client = boto3.client("lambda", region_name=region)
+def fetch_live_lambda_functions(region: str, profile: str = None) -> dict:
+    lambda_client = get_boto3_client("lambda", profile=profile, region=region)
     live = {}
-    paginator = lambda_client.get_paginator("list_functions")
     try:
+        paginator = lambda_client.get_paginator("list_functions")
         for page in paginator.paginate():
-            for func in page["Functions"]:
-                func_name = func["FunctionName"]
+            for func in page.get("Functions", []):
+                func_name = func.get("FunctionName")
+                if not func_name:
+                    continue
                 live[func_name] = {
                     "type": "aws_lambda_function",
                     "name": func_name,
@@ -165,35 +187,50 @@ def fetch_live_lambda_functions(region: str = "ap-south-1") -> dict:
                     },
                 }
     except Exception as e:
-        pass
+        print(f"Failed to fetch aws_lambda_function: {e}", file=sys.stderr)
+        return None
     return live
 
-def fetch_live_iam_roles(region: str = "ap-south-1") -> dict:
-    iam = boto3.client("iam", region_name=region)
+def fetch_live_iam_roles(region: str, profile: str = None) -> dict:
+    iam = get_boto3_client("iam", profile=profile, region=region)
     live = {}
     try:
         paginator = iam.get_paginator("list_roles")
         for page in paginator.paginate():
-            for role in page["Roles"]:
-                role_name = role["RoleName"]
+            for role in page.get("Roles", []):
+                role_name = role.get("RoleName")
+                if not role_name:
+                    continue
                 if role_name.startswith("AWSServiceRoleFor") or role.get("Path", "").startswith("/aws-service-role/"):
                     continue
+                
+                try:
+                    policies = iam.list_attached_role_policies(RoleName=role_name)
+                    attached_policies = sorted(
+                        p["PolicyArn"] for p in policies.get("AttachedPolicies", []) if "PolicyArn" in p
+                    )
+                except Exception:
+                    attached_policies = []
+                
                 live[role_name] = {
                     "type": "aws_iam_role",
                     "name": role_name,
                     "attributes": {
                         "id": role_name,
                         "name": role_name,
-                        "arn": role["Arn"]
+                        "path": role.get("Path", "/"),
+                        "arn": role.get("Arn", ""),
+                        "attached_policies": attached_policies
                     }
                 }
     except Exception as e:
-        pass
+        print(f"Failed to fetch aws_iam_role: {e}", file=sys.stderr)
+        return None
     return live
 
-def get_resource_cost(resource_id: str) -> float:
+def get_resource_cost(resource_id: str, profile: str = None) -> float | None:
     try:
-        client = boto3.client("ce", region_name="us-east-1")
+        client = get_boto3_client("ce", profile=profile, region="us-east-1")
         
         end_date = datetime.today().strftime("%Y-%m-%d")
         start_date = (datetime.today() - timedelta(days=30)).strftime("%Y-%m-%d")
@@ -210,8 +247,10 @@ def get_resource_cost(resource_id: str) -> float:
             }
         )
         
-        usd_cost = float(response["ResultsByTime"][0]["Total"]["UnblendedCost"]["Amount"])
-        inr_cost = round(usd_cost * 83.5, 2)
-        return inr_cost
+        results_by_time = response.get("ResultsByTime", [])
+        if not results_by_time:
+            return None
+        usd_cost = float(results_by_time[0]["Total"]["UnblendedCost"]["Amount"])
+        return usd_cost
     except Exception:
-        return 0.0
+        return None