runbooks 0.2.5__py3-none-any.whl → 0.7.0__py3-none-any.whl

runbooks/inventory/delete_s3_buckets_objects.py

@@ -0,0 +1,169 @@
+#!/usr/bin/env python3
+"""
+AWS S3 Bucket Object Deletion and Bucket Management Tool
+
+A specialized utility for safely emptying and optionally deleting S3 buckets,
+including all object versions and delete markers. Essential for bucket lifecycle
+management and compliance with data retention policies.
+
+**AWS API Mapping**:
+- `boto3.resource('s3').Bucket.object_versions.delete()`
+- `boto3.resource('s3').Bucket.delete()`
+
+**SECURITY WARNING**: This script performs DESTRUCTIVE operations:
+- Permanently deletes ALL objects and versions from specified bucket
+- Can delete the bucket itself with --force-delete flag
+- Cannot be undone - ensure proper backups before execution
+- May affect compliance with data retention requirements
+
+Features:
+    - Complete object version deletion (including delete markers)
+    - Interactive bucket deletion confirmation
+    - Force deletion mode for automation scenarios
+    - Comprehensive error handling and logging
+    - Single-bucket targeted operation for safety
+
+Security Controls:
+    - Requires explicit bucket name parameter
+    - Interactive confirmation for bucket deletion
+    - Force flag available for automated scenarios
+    - Detailed logging of all destructive operations
+
+Compliance Considerations:
+    - Verify data retention policy compliance before execution
+    - Ensure proper backup procedures are in place
+    - Document destruction for audit trails
+    - Consider legal hold and litigation requirements
+
+Example:
+    Empty bucket but keep it:
+    ```bash
+    python delete_s3_buckets_objects.py --profile my-profile --bucket my-bucket
+    ```
+
+    Empty and delete bucket with confirmation:
+    ```bash
+    python delete_s3_buckets_objects.py --profile my-profile --bucket my-bucket +delete
+    ```
+
+Requirements:
+    - IAM permissions: `s3:DeleteObject`, `s3:DeleteObjectVersion`, `s3:DeleteBucket`
+    - Bucket must be in accessible region
+    - Python 3.8+ with required dependencies
+
+Author:
+    AWS Cloud Foundations Team
+
+Version:
+    2023.05.04
+"""
+
+import logging
+
+from account_class import aws_acct_access
+from ArgumentsClass import CommonArguments
+
+__version__ = "2023.05.04"
+
+parser = CommonArguments()
+parser.singleprofile()
+parser.singleregion()
+parser.verbosity()
+parser.version(__version__)
+parser.my_parser.add_argument(
+    "-b",
+    "--bucket",
+    dest="pBucketName",
+    metavar="bucket to empty and delete",
+    required=True,
+    help="To specify a bucket, use this parameter.",
+)
+parser.my_parser.add_argument(
+    "+delete",
+    "+force-delete",
+    help="Whether or not to delete the bucket after it's been emptied",
+    action="store_const",
+    dest="pForceQuit",
+    const=True,
+    default=False,
+)
+args = parser.my_parser.parse_args()
+
+pProfile = args.Profile
+pRegion = args.Region
+pBucketDelete = args.pForceQuit
+pBucketName = args.pBucketName
+verbose = args.loglevel
+logging.basicConfig(level=args.loglevel, format="[%(filename)s:%(lineno)s - %(funcName)20s() ] %(message)s")
+
+# Establish AWS session and S3 resource connection
+# Uses the specified profile to access S3 services
+aws_acct = aws_acct_access(pProfile)
+s3 = aws_acct.session.resource(service_name="s3")
+
+# CRITICAL WARNING: Display destructive operation warning
+# This ensures users understand the irreversible nature of this operation
+print()
+print(f"This script is about to delete all versions of all objects from bucket {pBucketName}")
+print()
+
+# Create S3 bucket resource for the specified bucket
+# This provides access to bucket operations and object management
+bucket = s3.Bucket(pBucketName)
+
+try:
+    # DESTRUCTIVE OPERATION: Delete all object versions and delete markers
+    # This includes:
+    # - All current object versions
+    # - All historical object versions (if versioning enabled)
+    # - All delete markers
+    # - Cannot be undone once executed
+    logging.info(f"Starting deletion of all object versions in bucket {pBucketName}")
+    bucket.object_versions.delete()
+    logging.info(f"Successfully deleted all object versions from bucket {pBucketName}")
+
+except Exception as my_Error:
+    # Handle S3 API errors during object deletion
+    # Common errors: AccessDenied, NoSuchBucket, InvalidBucketState
+    logging.error(f"Failed to delete objects from bucket {pBucketName}: {my_Error}")
+    print(f"Error message: {my_Error}")
+
+# Handle bucket deletion with safety controls
+# Provides both automated and interactive deletion modes
+DeleteBucket = False
+
+if pBucketDelete:
+    # Force deletion mode: Delete bucket without additional confirmation
+    # Used for automated scenarios where confirmation is handled externally
+    print(f"As per your request, we're deleting the bucket {pBucketName}")
+    logging.warning(f"Force deleting bucket {pBucketName} as requested")
+
+    try:
+        bucket.delete()
+        print(f"Bucket: {pBucketName} has been deleted")
+        logging.info(f"Successfully deleted bucket {pBucketName}")
+    except Exception as delete_error:
+        logging.error(f"Failed to delete bucket {pBucketName}: {delete_error}")
+        print(f"Failed to delete bucket: {delete_error}")
+
+else:
+    # Interactive deletion mode: Prompt user for confirmation
+    # Provides additional safety control for manual operations
+    DeleteBucket = input("Now that the bucket is empty, do you want to delete the bucket? (y/n): ") in ["y", "Y"]
+
+    if DeleteBucket:
+        try:
+            bucket.delete()
+            print(f"Bucket: {pBucketName} has been deleted")
+            logging.info(f"User confirmed deletion of bucket {pBucketName}")
+        except Exception as delete_error:
+            logging.error(f"Failed to delete bucket {pBucketName}: {delete_error}")
+            print(f"Failed to delete bucket: {delete_error}")
+    else:
+        print(f"Bucket: {pBucketName} has NOT been deleted")
+        logging.info(f"User chose to preserve bucket {pBucketName}")
+# Operation completion notification
+print()
+print("Thanks for using this script...")
+logging.info("S3 bucket operation completed successfully")
+print()

runbooks/inventory/discovery.md

@@ -0,0 +1,81 @@
+# Using these Inventory Scripts as Discovery
+
+## AWS Cloud Foundations boto3-Aligned Scripts
+
+All scripts have been updated to follow boto3 API naming conventions for better AWS professional compatibility.
+
+The following script runs for all accounts within either your specified profile, or (if no profile is used) your default credentials (could be environment variables). This will assess whether ALL of your accounts are suitable to be migrated to Control Tower or not, and if not - what the issues preventing their adoption would be. The "-r global" specifies that ALL regions (even those you have not opted into) should be looked at. The script will (because of the "-v") inform you of the failure to connect to an account in the excluded region, but won't fail because of it. This script executes 10 commands for every account in every region, so it will take a **long** time to run.
+
+```sh
+controltower_check_account_readiness.py -v -r global --timing [-p <profile of Org Account>]
+```
+
+This script will go through your Org and find all members accounts and their statuses - thereby showing you which accounts should - perhaps - be moved to a "SUSPENDED" OU or otherwise treated specially. It's useful because the output is very purposeful and it's pretty fast.
+
+```sh
+org_list_accounts.py -v
+```
+
+This next script will find the status of all of your accounts and regions and whether you have CloudTrail enabled in each.
+
+```sh
+cloudtrail_describe_trails_compliance.py -v -r global --timing --filename cloudtrail_check.out [-p <profile of Org Account>]
+```
+
+The following script can draw out the Organization. The output will be a file in the current directory called "aws_organization.png" - please either get that file, or a screenshot of it. Assuming the user has the graphviz tool installed within their environment, running this tool should end with the diagram itself being shown. The parameter "--policy" can also be mitigated by "--aws" to include those policies which AWS owns (like the AWSFullAccess policy assigned by default to every OU and account). The default (below) excludes that AWS-managed policy for diagram clarity's sake.
+
+```sh
+org_describe_structure.py --policy --timing
+```
+
+The following script can do soooo much _(Yeah - I'm pretty proud of this one)_. As it's shown here, it doesn't yet support the "--filename" parameter, since I haven't decided how to write out the data. The goal of using this output in Discovery, is to find those accounts which have been closed (and may no longer be in the Org at all), but are still represented in the stacksets of the Org - and therefore may (eventually) cause stacksets to slow down or fail. Best to find these issues ahead of time, rather than after the fact. For instance - I found a customer with 450 accounts in their Org, but their largest stackset had over 100 closed (and already dropped out) accounts, so while the stackset was still considered "CURRENT", more than 20% of the time spent on that stackset was spent attempting to connect to previously closed accounts.
+```sh
+cfn_update_stack_sets.py -v -r <home region> --timing [-p <profile of Org Account>] -check
+```
+
+The following script shows whether the "Public S3 block" has been enabled on all accounts within the Org. While Control Tower has a control that can enable this on new accounts, it doesn't mean that it hasn't been removed somewhere. It's a good idea to run this, and you can use the same script to re-enable the block if it's been removed.
+
+```sh
+s3_put_public_access_block.py -v
+```
+
+The following script finds any and all config recorders and delivery channels in your environment - again, this is a tool that is used when trying to determine what blockers exist before moving to Control Tower. It's also a good tool (if you don't need the full complement of checks in the controltower_check_account_readiness.py above) to find any accounts where Config isn't running at all. This tool also can be used to **delete** the config recorders and delivery channels - if needed.
+
+```sh
+config_describe_configuration_recorders.py -v -r global --timing
+```
+
+These scripts will find those IAM/ IDC users, local directories, or SAML providers in your child accounts which can be exposures to unwanted access, without you realizing it. It's always a good idea to look for these - since these can represent a significant threat vector to protect from.
+
+```sh
+org_list_account_users.py -v
+iam_list_saml_providers.py -v
+ds_describe_directories.py -v
+```
+
+While it's normal for this script to find nothing, it's very illuminating if it *does* find something...
+
+```sh
+cfn_find_orphaned_stacks.py --filename Drift_Detection -v
+```
+
+The following scripts will just show very useful Inventory information that will help the Discovery process flesh out its understanding of the customer's environment.
+
+```sh
+ec2_describe_vpcs.py -v
+route53_list_hosted_zones.py -v
+```
+
+Whenever we do Discovery, we always want to find possible money-savings areas for the customer as well. The script below will find any Log Groups and their retention settings. This gives the customer the opportunity (perhaps) to update those retention settings (from their default of "NEVER") to something that will purge data after a specific time. The bottom of the script gives an *idea* of how much you're spending on Log Groups anyway, so you have an idea if taking action is worthwhile.
+
+```sh
+logs_put_retention_policy.py -v
+```
+
+ALZ used Service Catalog to create and manage accounts. It's important that these Service Catalog products are properly terminated when ALZ is decommissioned, so this tool will report on the accounts in the Org reconciled with the Service Catalog Products that were created and point out if there are products for already closed accounts, or whether there are more than one product for a given account (or no products for a given account).
+
+> **Note:** Control Tower use Service Catalog as well. Please ensure you do not terminate Control Tower provisioned products.
+
+```sh
+servicecatalog_list_provisioned_products.py -v --timing
+```