ddutil 0.1.0__tar.gz

ddutil-0.1.0/.gitignore

@@ -0,0 +1,26 @@
+# Byte-compiled / optimized / DLL files
+__pycache__/
+*.py[cod]
+*$py.class
+
+# Environment Variables
+.env.development.local
+.env.production.local
+.env.local
+.env-*
+.env
+
+# IDEs
+.idea
+.vscode
+
+# Mypy
+.mypy_cache
+
+# User Ignores
+.old
+.testing
+
+# Virtual Environments
+venv/
+.venv/

ddutil-0.1.0/LICENSE

@@ -0,0 +1,16 @@
+Copyright 2026 Tom Burge
+
+Permission is hereby granted, free of charge, to any person obtaining a copy of this software 
+and associated documentation files (the “Software”), to deal in the Software without restriction, 
+including without limitation the rights to use, copy, modify, merge, publish, distribute, 
+sublicense, and/or sell copies of the Software, and to permit persons to whom the Software is 
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all copies or 
+substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND, EXPRESS OR IMPLIED, INCLUDING 
+BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND 
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, 
+DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, 
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

ddutil-0.1.0/PKG-INFO

@@ -0,0 +1,568 @@
+Metadata-Version: 2.4
+Name: ddutil
+Version: 0.1.0
+Summary: A CLI tool for managing DataDog AWS integrations
+Project-URL: Homepage, https://github.com/tomburge/datadog-utility
+Project-URL: Repository, https://github.com/tomburge/datadog-utility
+Project-URL: Bug Tracker, https://github.com/tomburge/datadog-utility/issues
+Author-email: Tom Burge <tom@tomburge.org>
+License: MIT
+License-File: LICENSE
+Keywords: aws,cli,datadog,devops,integration,monitoring
+Classifier: Development Status :: 3 - Alpha
+Classifier: Intended Audience :: Developers
+Classifier: Intended Audience :: System Administrators
+Classifier: License :: OSI Approved :: MIT License
+Classifier: Operating System :: OS Independent
+Classifier: Programming Language :: Python :: 3
+Classifier: Programming Language :: Python :: 3.12
+Classifier: Programming Language :: Python :: 3.13
+Classifier: Programming Language :: Python :: 3.14
+Classifier: Topic :: System :: Systems Administration
+Classifier: Topic :: Utilities
+Requires-Python: >=3.12
+Requires-Dist: boto3>=1.28.0
+Requires-Dist: certifi>=2023.7.22
+Requires-Dist: click>=8.0.0
+Requires-Dist: datadog-api-client>=2.0.0
+Requires-Dist: loguru>=0.7.0
+Requires-Dist: pydantic>=2.0.0
+Requires-Dist: python-dotenv>=0.19.0
+Requires-Dist: rich>=10.0.0
+Description-Content-Type: text/markdown
+
+# DataDog AWS Integration CLI
+
+A command-line tool for managing DataDog AWS integrations with ease.
+
+## Features
+
+- 🚀 Easy setup of DataDog integrations for AWS accounts
+- 🔧 Automated IAM role and policy creation/management
+- ✅ Comprehensive status checking and validation
+  - IAM role and policy verification
+  - DataDog account configuration validation
+  - Settings comparison (regions, services, metrics, resources)
+- 🔄 Update existing integrations (regions, services, metrics, resources)
+- 🗑️ Clean deletion of integrations
+- 🎨 Beautiful terminal output with tables and colors
+- ⚙️ Flexible configuration via .env files or CLI arguments
+- 🔒 Dry-run mode to preview changes before applying
+- 📝 Verbose logging for debugging
+- 📊 JSON output support for automation
+- 🔐 Support for multiple AWS partitions (standard, GovCloud, China)
+- ⚡ No complex YAML configuration required
+
+## Build Status
+
+[![Test Build](https://github.com/tomburge/datadog-utility/actions/workflows/test-build.yml/badge.svg)](https://github.com/tomburge/datadog-utility/actions/workflows/test-build.yml)
+
+[![Publish to PyPI](https://github.com/tomburge/datadog-utility/actions/workflows/publish.yml/badge.svg)](https://github.com/tomburge/datadog-utility/actions/workflows/publish.yml)
+
+## Installation
+
+### Using UV (Recommended)
+
+[UV](https://github.com/astral-sh/uv) is a blazing-fast Python package manager.
+
+```bash
+# Install UV if you haven't already
+curl -LsSf https://astral.sh/uv/install.sh | sh
+# or on Windows:
+powershell -c "irm https://astral.sh/uv/install.ps1 | iex"
+
+# Clone the repository
+git clone <repository-url>
+cd datadog-utility
+
+# Create a virtual environment and install dependencies
+uv venv
+uv pip install -e .
+```
+
+### Using pip
+
+```bash
+# Clone the repository
+git clone <repository-url>
+cd datadog-utility
+
+# Create and activate virtual environment
+python -m venv .venv
+source .venv/bin/activate  # On Windows: .venv\Scripts\activate
+
+# Install the CLI tool
+pip install -e .
+```
+
+After installation, the `ddutil` command will be available globally.
+
+## Quick Start
+
+### 1. Set Up Environment Variables
+
+Copy the example `.env` file and fill in your credentials:
+
+```bash
+cp .env.example .env
+```
+
+Edit `.env` with your values:
+
+```bash
+# Required
+DD_API_KEY=your_datadog_api_key
+DD_APP_KEY=your_datadog_app_key
+AWS_ACCOUNT_ID=123456789012
+
+# Optional but commonly used
+DD_ACCOUNT_ID=your_dd_account_id
+AWS_PROFILE=default
+DD_IAM_ROLE_NAME=datadog-integration-role
+```
+
+### 2. Preview the Setup (Dry-Run)
+
+```bash
+ddutil setup --dry-run
+```
+
+### 3. Apply the Changes
+
+```bash
+ddutil setup
+```
+
+That's it! The tool will:
+
+- Create the necessary IAM role and policies
+- Configure the DataDog integration
+- Update the IAM role with the external ID from DataDog
+
+## Usage
+
+### Available Commands
+
+```bash
+ddutil --help              # Show all available commands
+ddutil --version           # Show version information
+ddutil --license           # Show license information
+ddutil -v, --verbose       # Enable verbose output for any command
+ddutil -q, --quiet         # Suppress non-error output
+```
+
+**Commands:**
+
+- `setup` - Set up DataDog integration for an AWS account
+- `status` - Check the status of DataDog integration and validate configuration
+- `update` - Update an existing DataDog integration
+- `delete` - Delete DataDog integration for an AWS account
+
+### Quick Reference
+
+```bash
+# Setup with defaults from .env
+ddutil setup
+
+# Preview setup without making changes
+ddutil setup --dry-run
+
+# Check configuration status and validation
+ddutil status
+
+# Update configuration
+ddutil update --regions us-east-1,us-west-2
+
+# Delete integration (with confirmation)
+ddutil delete --confirm
+
+# Any command with verbose output
+ddutil --verbose <command>
+```
+
+### Setup Command
+
+Set up DataDog integration for an AWS account:
+
+```bash
+# Using environment variables (.env file)
+ddutil setup
+
+# Override with CLI arguments
+ddutil setup --account-id 123456789012 --dd-account-id YOUR_DD_ACCOUNT_ID
+
+# Specify AWS profile
+ddutil setup --profile aws-prod
+
+# Dry-run mode (preview changes without applying)
+ddutil setup --dry-run
+
+# Verbose output (shows debug logs)
+ddutil --verbose setup
+
+# Specify regions and services
+ddutil setup --regions us-east-1,us-west-2 --services AWS/Lambda,AWS/EC2
+
+# Configure metric and resource collection
+ddutil setup --metric-automute true --metric-collect-cloudwatch true \
+  --resource-collect-cspm false --resource-collect-extended true
+
+# Specify AWS partition (for GovCloud or China regions)
+ddutil setup --partition aws-us-gov
+```
+
+### Status Command
+
+Check the status of an existing integration and validate configuration:
+
+```bash
+# Basic status check
+ddutil status
+
+# Check specific account
+ddutil status --account-id 123456789012
+
+# Use specific AWS profile
+ddutil status --profile aws-prod
+
+# JSON output for programmatic use
+ddutil status --output json
+
+# Verbose output with debug info
+ddutil --verbose status
+```
+
+The status command validates:
+
+- ✅ IAM role existence and policies
+- ✅ DataDog account registration
+- ✅ Configuration sync (regions, services, partition)
+- ✅ Metric settings (automute, CloudWatch, custom metrics)
+- ✅ Resource settings (CSPM, extended collection)
+- ✅ Role name and External ID matching
+
+### Update Command
+
+Update an existing integration:
+
+```bash
+# Update services monitored
+ddutil update --services AWS/Lambda,AWS/EC2,AWS/RDS
+
+# Update regions
+ddutil update --regions us-west-2,eu-west-1
+
+# Update metric settings
+ddutil update --metric-collect-custom true --metric-automute false
+
+# Update resource collection
+ddutil update --resource-collect-cspm true
+
+# Preview changes before applying
+ddutil update --regions us-west-2,eu-west-1 --dry-run
+
+# Update with verbose output
+ddutil --verbose update --services AWS/Lambda
+```
+
+### Delete Command
+
+Remove DataDog integration:
+
+```bash
+# With confirmation prompt (interactive)
+ddutil delete
+
+# Specify account ID
+ddutil delete --account-id 123456789012
+
+# Skip confirmation prompt (for automation)
+ddutil delete --confirm
+
+# Delete with specific role name
+ddutil delete --role-name custom-datadog-role --confirm
+
+# Use specific AWS profile
+ddutil delete --profile aws-prod --confirm
+```
+
+## Configuration
+
+### Configuration Priority
+
+Configuration values are resolved in the following order (highest to lowest priority):
+
+1. **CLI arguments** - Values passed directly via command-line flags
+2. **Environment variables** - Values from `.env` file or system environment
+
+### Environment Variables
+
+Create a `.env` file in your project directory:
+
+```bash
+# Copy the example file
+cp .env.example .env
+
+# Edit with your values
+nano .env  # or your favorite editor
+```
+
+### Required Environment Variables
+
+```bash
+# DataDog API Credentials
+DD_API_KEY=your_datadog_api_key_here    # DataDog API key (required)
+DD_APP_KEY=your_datadog_app_key_here    # DataDog application key (required)
+
+# AWS Configuration
+AWS_ACCOUNT_ID=123456789012             # Your AWS account ID (required)
+```
+
+### Optional Environment Variables
+
+```bash
+# Application Settings
+LOG_LEVEL=INFO                         # Logging level (DEBUG, INFO, WARNING, ERROR)
+
+# DataDog Configuration
+DD_ACCOUNT_ID=                         # DataDog account ID (obtained from DataDog)
+DD_SITE=datadoghq.com                  # DataDog site (datadoghq.com, datadoghq.eu, etc.)
+DD_PARTITION=aws                       # AWS partition (aws, aws-cn, aws-us-gov)
+DATADOG_VERIFY_SSL=false               # SSL verification for DataDog API
+
+# AWS Configuration
+AWS_PROFILE=default                    # AWS CLI profile name
+
+# IAM Configuration
+DD_IAM_ROLE_NAME=datadog-integration-role  # IAM role name
+DD_MANAGED_POLICIES=                   # Comma-separated managed policy ARNs
+                                       # Default: ReadOnlyAccess, SecurityAudit
+DD_POLICY_ACTIONS=                     # Comma-separated additional IAM actions
+                                       # Default: 32 standard actions (see .env.example)
+
+# Monitoring Configuration
+DD_REGIONS=                            # Comma-separated AWS regions (empty = all)
+DD_SERVICES=                           # Comma-separated AWS services (empty = default)
+DD_TRACES=                             # Comma-separated services for X-Ray tracing
+
+# Metric Settings
+DD_METRIC_AUTOMUTE=true                # Auto-mute monitors (true/false)
+DD_METRIC_COLLECT_CLOUDWATCH=true      # Collect CloudWatch alarms (true/false)
+DD_METRIC_COLLECT_CUSTOM=false         # Collect custom metrics (true/false)
+DD_METRIC_COLLECT_METRICS=true         # Enable metric collection (true/false)
+DD_METRIC_ENABLE=true                  # Enable metrics globally (true/false)
+
+# Resource Settings
+DD_RESOURCE_COLLECT_CSPM=false         # Cloud Security Posture Management (true/false)
+DD_RESOURCE_COLLECT_EXTENDED=true      # Extended resource collection (true/false)
+```
+
+## Examples
+
+### Basic Setup with .env File
+
+Create `.env`:
+
+```bash
+DD_API_KEY=abc123def456...
+DD_APP_KEY=xyz789ghi012...
+AWS_ACCOUNT_ID=123456789012
+AWS_PROFILE=production
+```
+
+Run setup:
+
+```bash
+# Preview changes
+ddutil setup --dry-run
+
+# Apply configuration
+ddutil setup
+
+# Check status after setup
+ddutil status
+```
+
+### Multi-Region Setup
+
+```bash
+# Monitor specific regions only
+ddutil setup --regions us-east-1,us-west-2,eu-west-1
+
+# Or set in .env
+DD_REGIONS=us-east-1,us-west-2,eu-west-1
+ddutil setup
+```
+
+### Specific Services Only
+
+```bash
+# Monitor specific AWS services
+ddutil setup --services AWS/Lambda,AWS/EC2,AWS/RDS
+
+# With X-Ray tracing for Lambda
+ddutil setup --services AWS/Lambda,AWS/EC2 --traces AWS/Lambda
+```
+
+### Metric and Resource Configuration
+
+```bash
+# Enable Cloud Security Posture Management
+ddutil setup --resource-collect-cspm true
+
+# Enable custom metrics collection
+ddutil setup --metric-collect-custom true
+
+# Configure multiple metric settings
+ddutil setup \
+  --metric-automute true \
+  --metric-collect-cloudwatch true \
+  --metric-collect-custom false \
+  --metric-enable true
+```
+
+### Override with CLI Arguments
+
+```bash
+# Override environment variables for one-off operations
+ddutil setup --account-id 999888777666 --profile dev-account
+
+# Use different AWS partition
+ddutil setup --partition aws-us-gov --profile govcloud
+```
+
+### Custom IAM Configuration
+
+```bash
+# Custom role name and policies
+ddutil setup \
+  --role-name custom-datadog-role \
+  --managed-policies arn:aws:iam::aws:policy/ReadOnlyAccess \
+  --policy-actions logs:PutSubscriptionFilter,s3:GetBucketNotification
+
+# Update existing role policies
+ddutil update \
+  --managed-policies arn:aws:iam::aws:policy/ReadOnlyAccess,arn:aws:iam::aws:policy/SecurityAudit
+```
+
+### Validation and Status Checking
+
+```bash
+# Check if configuration matches .env settings
+ddutil status
+
+# Get detailed JSON output for monitoring
+ddutil status --output json > status.json
+
+# Validate specific account configuration
+ddutil --verbose status --account-id 123456789012
+```
+
+## Development
+
+### Project Structure
+
+```text
+datadog-utility/
+├── src/
+│   └── ddutil/          # Main package
+│       ├── __init__.py  # Package initialization
+│       ├── cli.py       # CLI command definitions
+│       └── common/      # Shared utilities
+│           ├── aws/     # AWS-related modules
+│           │   ├── auth.py      # AWS authentication
+│           │   └── iam.py       # IAM role management
+│           ├── datadog/         # DataDog-related modules
+│           │   └── aws.py       # DataDog API interactions
+│           ├── logs.py          # Logging configuration
+│           └── utils.py         # Utility functions
+├── pyproject.toml       # Package configuration
+├── .env.example         # Example environment variables
+├── .python-version      # Python version for UV
+├── requirements.txt     # Python dependencies
+└── responses/           # Sample API responses
+```
+
+## Troubleshooting
+
+### Common Issues
+
+**Missing required environment variables:**
+
+```bash
+# Check if all required variables are set with dry-run
+ddutil setup --dry-run
+
+# Set missing variables in .env file or export them
+export AWS_ACCOUNT_ID=123456789012
+export DD_API_KEY=your_api_key
+export DD_APP_KEY=your_app_key
+```
+
+**AWS authentication errors:**
+
+```bash
+# Ensure your AWS profile is configured
+aws configure --profile your-profile-name
+
+# Or specify profile explicitly
+ddutil setup --profile your-profile-name
+
+# Or use environment variables
+export AWS_PROFILE=your-profile-name
+
+# Test AWS connectivity
+aws sts get-caller-identity --profile your-profile-name
+```
+
+**DataDog API errors:**
+
+```bash
+# Verify your API keys are set
+echo $DD_API_KEY
+echo $DD_APP_KEY
+
+# Make sure they're in your .env file
+cat .env | grep DD_
+
+# Test DataDog API connectivity
+ddutil --verbose status
+```
+
+**Configuration mismatch errors:**
+
+```bash
+# Use status command to see what doesn't match
+ddutil status
+
+# Status will show:
+# - IAM role and policy status
+# - DataDog account configuration
+# - Mismatches between actual and expected settings
+# - Specific issues with regions, services, metrics, resources
+
+# Fix mismatches by updating
+ddutil update --regions us-east-1,us-west-2
+ddutil update --metric-collect-cloudwatch true
+```
+
+**Debug with verbose and dry-run:**
+
+```bash
+# Always test with dry-run first
+ddutil setup --dry-run
+
+# Use verbose for detailed debugging
+ddutil --verbose setup --dry-run
+
+# Check logs for detailed error messages
+tail -f ~/.ddutil/logs/ddutil.log  # if logging to file
+```
+
+## Roadmap
+
+See [ROADMAP.md](ROADMAP.md) for planned features and improvements.
+
+## License
+
+MIT License - See LICENSE file for details.