runbooks 0.7.9__py3-none-any.whl โ 0.9.1__py3-none-any.whl
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- runbooks/__init__.py +1 -1
- runbooks/cfat/README.md +12 -1
- runbooks/cfat/__init__.py +1 -1
- runbooks/cfat/assessment/compliance.py +4 -1
- runbooks/cfat/assessment/runner.py +42 -34
- runbooks/cfat/models.py +1 -1
- runbooks/cloudops/__init__.py +123 -0
- runbooks/cloudops/base.py +385 -0
- runbooks/cloudops/cost_optimizer.py +811 -0
- runbooks/cloudops/infrastructure_optimizer.py +29 -0
- runbooks/cloudops/interfaces.py +828 -0
- runbooks/cloudops/lifecycle_manager.py +29 -0
- runbooks/cloudops/mcp_cost_validation.py +678 -0
- runbooks/cloudops/models.py +251 -0
- runbooks/cloudops/monitoring_automation.py +29 -0
- runbooks/cloudops/notebook_framework.py +676 -0
- runbooks/cloudops/security_enforcer.py +449 -0
- runbooks/common/__init__.py +152 -0
- runbooks/common/accuracy_validator.py +1039 -0
- runbooks/common/context_logger.py +440 -0
- runbooks/common/cross_module_integration.py +594 -0
- runbooks/common/enhanced_exception_handler.py +1108 -0
- runbooks/common/enterprise_audit_integration.py +634 -0
- runbooks/common/mcp_cost_explorer_integration.py +900 -0
- runbooks/common/mcp_integration.py +548 -0
- runbooks/common/performance_monitor.py +387 -0
- runbooks/common/profile_utils.py +216 -0
- runbooks/common/rich_utils.py +172 -1
- runbooks/feedback/user_feedback_collector.py +440 -0
- runbooks/finops/README.md +377 -458
- runbooks/finops/__init__.py +4 -21
- runbooks/finops/account_resolver.py +279 -0
- runbooks/finops/accuracy_cross_validator.py +638 -0
- runbooks/finops/aws_client.py +721 -36
- runbooks/finops/budget_integration.py +313 -0
- runbooks/finops/cli.py +59 -5
- runbooks/finops/cost_optimizer.py +1340 -0
- runbooks/finops/cost_processor.py +211 -37
- runbooks/finops/dashboard_router.py +900 -0
- runbooks/finops/dashboard_runner.py +990 -232
- runbooks/finops/embedded_mcp_validator.py +288 -0
- runbooks/finops/enhanced_dashboard_runner.py +8 -7
- runbooks/finops/enhanced_progress.py +327 -0
- runbooks/finops/enhanced_trend_visualization.py +423 -0
- runbooks/finops/finops_dashboard.py +184 -1829
- runbooks/finops/helpers.py +509 -196
- runbooks/finops/iam_guidance.py +400 -0
- runbooks/finops/markdown_exporter.py +466 -0
- runbooks/finops/multi_dashboard.py +1502 -0
- runbooks/finops/optimizer.py +15 -15
- runbooks/finops/profile_processor.py +2 -2
- runbooks/finops/runbooks.inventory.organizations_discovery.log +0 -0
- runbooks/finops/runbooks.security.report_generator.log +0 -0
- runbooks/finops/runbooks.security.run_script.log +0 -0
- runbooks/finops/runbooks.security.security_export.log +0 -0
- runbooks/finops/schemas.py +589 -0
- runbooks/finops/service_mapping.py +195 -0
- runbooks/finops/single_dashboard.py +710 -0
- runbooks/finops/tests/test_reference_images_validation.py +1 -1
- runbooks/inventory/README.md +12 -1
- runbooks/inventory/core/collector.py +157 -29
- runbooks/inventory/list_ec2_instances.py +9 -6
- runbooks/inventory/list_ssm_parameters.py +10 -10
- runbooks/inventory/organizations_discovery.py +210 -164
- runbooks/inventory/rich_inventory_display.py +74 -107
- runbooks/inventory/run_on_multi_accounts.py +13 -13
- runbooks/inventory/runbooks.inventory.organizations_discovery.log +0 -0
- runbooks/inventory/runbooks.security.security_export.log +0 -0
- runbooks/main.py +1371 -240
- runbooks/metrics/dora_metrics_engine.py +711 -17
- runbooks/monitoring/performance_monitor.py +433 -0
- runbooks/operate/README.md +394 -0
- runbooks/operate/base.py +215 -47
- runbooks/operate/ec2_operations.py +435 -5
- runbooks/operate/iam_operations.py +598 -3
- runbooks/operate/privatelink_operations.py +1 -1
- runbooks/operate/rds_operations.py +508 -0
- runbooks/operate/s3_operations.py +508 -0
- runbooks/operate/vpc_endpoints.py +1 -1
- runbooks/remediation/README.md +489 -13
- runbooks/remediation/base.py +5 -3
- runbooks/remediation/commons.py +8 -4
- runbooks/security/ENTERPRISE_SECURITY_FRAMEWORK.md +506 -0
- runbooks/security/README.md +12 -1
- runbooks/security/__init__.py +265 -33
- runbooks/security/cloudops_automation_security_validator.py +1164 -0
- runbooks/security/compliance_automation.py +12 -10
- runbooks/security/compliance_automation_engine.py +1021 -0
- runbooks/security/enterprise_security_framework.py +930 -0
- runbooks/security/enterprise_security_policies.json +293 -0
- runbooks/security/executive_security_dashboard.py +1247 -0
- runbooks/security/integration_test_enterprise_security.py +879 -0
- runbooks/security/module_security_integrator.py +641 -0
- runbooks/security/multi_account_security_controls.py +2254 -0
- runbooks/security/real_time_security_monitor.py +1196 -0
- runbooks/security/report_generator.py +1 -1
- runbooks/security/run_script.py +4 -8
- runbooks/security/security_baseline_tester.py +39 -52
- runbooks/security/security_export.py +99 -120
- runbooks/sre/README.md +472 -0
- runbooks/sre/__init__.py +33 -0
- runbooks/sre/mcp_reliability_engine.py +1049 -0
- runbooks/sre/performance_optimization_engine.py +1032 -0
- runbooks/sre/production_monitoring_framework.py +584 -0
- runbooks/sre/reliability_monitoring_framework.py +1011 -0
- runbooks/validation/__init__.py +2 -2
- runbooks/validation/benchmark.py +154 -149
- runbooks/validation/cli.py +159 -147
- runbooks/validation/mcp_validator.py +291 -248
- runbooks/vpc/README.md +478 -0
- runbooks/vpc/__init__.py +2 -2
- runbooks/vpc/manager_interface.py +366 -351
- runbooks/vpc/networking_wrapper.py +68 -36
- runbooks/vpc/rich_formatters.py +22 -8
- runbooks-0.9.1.dist-info/METADATA +308 -0
- {runbooks-0.7.9.dist-info โ runbooks-0.9.1.dist-info}/RECORD +120 -59
- {runbooks-0.7.9.dist-info โ runbooks-0.9.1.dist-info}/entry_points.txt +1 -1
- runbooks/finops/cross_validation.py +0 -375
- runbooks-0.7.9.dist-info/METADATA +0 -636
- {runbooks-0.7.9.dist-info โ runbooks-0.9.1.dist-info}/WHEEL +0 -0
- {runbooks-0.7.9.dist-info โ runbooks-0.9.1.dist-info}/licenses/LICENSE +0 -0
- {runbooks-0.7.9.dist-info โ runbooks-0.9.1.dist-info}/top_level.txt +0 -0
@@ -1,636 +0,0 @@
|
|
1
|
-
Metadata-Version: 2.4
|
2
|
-
Name: runbooks
|
3
|
-
Version: 0.7.9
|
4
|
-
Summary: CloudOps Automation Toolkit with Enhanced Cloud Foundations Assessment for DevOps and SRE teams.
|
5
|
-
Author-email: Maintainers <nnthanh101@gmail.com>
|
6
|
-
License-Expression: Apache-2.0
|
7
|
-
Project-URL: Homepage, https://cloudops.oceansoft.io
|
8
|
-
Project-URL: Repository, https://github.com/1xOps/CloudOps-Runbooks
|
9
|
-
Project-URL: Documentation, https://cloudops.oceansoft.io/runbooks/
|
10
|
-
Project-URL: Issues, https://github.com/1xOps/CloudOps-Runbooks/issues
|
11
|
-
Project-URL: Changelog, https://github.com/1xOps/CloudOps-Runbooks/blob/main/CHANGELOG.md
|
12
|
-
Keywords: runbooks,automation,DevOps,SRE,CloudOps,AWS,cloud-foundations,FinOps,enterprise,cost-optimization,security-compliance,multi-account,business-intelligence
|
13
|
-
Classifier: Development Status :: 5 - Production/Stable
|
14
|
-
Classifier: Environment :: Console
|
15
|
-
Classifier: Operating System :: OS Independent
|
16
|
-
Classifier: Programming Language :: Python :: 3
|
17
|
-
Classifier: Programming Language :: Python :: 3.11
|
18
|
-
Classifier: Programming Language :: Python :: 3.12
|
19
|
-
Classifier: Programming Language :: Python :: 3.13
|
20
|
-
Classifier: Topic :: System :: Systems Administration
|
21
|
-
Classifier: Topic :: Software Development :: Libraries :: Python Modules
|
22
|
-
Classifier: Topic :: Utilities
|
23
|
-
Requires-Python: <3.14,>=3.11
|
24
|
-
Description-Content-Type: text/markdown
|
25
|
-
License-File: LICENSE
|
26
|
-
Requires-Dist: boto3>=1.35.40
|
27
|
-
Requires-Dist: botocore>=1.35.40
|
28
|
-
Requires-Dist: diagrams>=0.24.4
|
29
|
-
Requires-Dist: click>=8.2.1
|
30
|
-
Requires-Dist: pydantic>=2.10.0
|
31
|
-
Requires-Dist: jinja2>=3.1.4
|
32
|
-
Requires-Dist: werkzeug>=3.1.0
|
33
|
-
Requires-Dist: markdown>=3.7.0
|
34
|
-
Requires-Dist: prettytable>=3.16.0
|
35
|
-
Requires-Dist: simplejson>=3.20.1
|
36
|
-
Requires-Dist: python-dateutil>=2.9.0
|
37
|
-
Requires-Dist: loguru>=0.7.3
|
38
|
-
Requires-Dist: tqdm>=4.67.1
|
39
|
-
Requires-Dist: graphviz>=0.20.1
|
40
|
-
Requires-Dist: rich>=14.0.0
|
41
|
-
Requires-Dist: reportlab>=3.6.1
|
42
|
-
Requires-Dist: requests>=2.32.0
|
43
|
-
Requires-Dist: packaging>=21.0
|
44
|
-
Requires-Dist: pyyaml>=6.0.2
|
45
|
-
Requires-Dist: jmespath>=1.0.1
|
46
|
-
Requires-Dist: urllib3<1.27,>=1.26.18
|
47
|
-
Requires-Dist: mcp>=1.12.3
|
48
|
-
Requires-Dist: pandas>=2.3.1
|
49
|
-
Requires-Dist: ipython>=9.4.0
|
50
|
-
Requires-Dist: psutil>=7.0.0
|
51
|
-
Dynamic: license-file
|
52
|
-
|
53
|
-
# ๐ CloudOps Runbooks - Enterprise AWS Automation Toolkit
|
54
|
-
|
55
|
-
[](https://pypi.org/project/runbooks/)
|
56
|
-
[](https://pypi.org/project/runbooks/)
|
57
|
-
[](https://opensource.org/licenses/Apache-2.0)
|
58
|
-
[](https://cloudops.oceansoft.io/runbooks/)
|
59
|
-
[](https://github.com/1xOps/CloudOps-Runbooks/actions)
|
60
|
-
[](https://github.com/astral-sh/ruff)
|
61
|
-
[](https://mypy-lang.org/)
|
62
|
-
[](https://pytest.org/)
|
63
|
-
|
64
|
-
> **Enterprise-grade AWS automation toolkit for cloud operations (SRE and DevOps teams) at scale**
|
65
|
-
|
66
|
-
CloudOps Runbooks provides comprehensive AWS resource discovery, inventory management, and automation capabilities with enterprise-grade architecture, type safety, and validation.
|
67
|
-
|
68
|
-
|
69
|
-
## ๐ Overview
|
70
|
-
|
71
|
-
CloudOps Runbooks is a production-ready AWS automation framework that combines traditional scripting excellence with modern AI orchestration. Designed for enterprises managing complex multi-account AWS environments, it delivers comprehensive discovery, intelligent analysis, and automated remediation across 50+ AWS services.
|
72
|
-
|
73
|
-
> **Strategic Achievement: 3 Major Objectives Complete** โ
|
74
|
-
> 1. **runbooks package** - PyPI v0.7.8 production deployment
|
75
|
-
> 2. **Enterprise FAANG/Agile SDLC** - 6-agent coordination framework operational
|
76
|
-
> 3. **GitHub Single Source of Truth** - Complete documentation and workflow integration
|
77
|
-
|
78
|
-
### ๐ Validated Business Impact
|
79
|
-
- **$1.4M Annual Savings**: Identified across 60-account AWS organization
|
80
|
-
- **$548/month Transit Gateway**: Optimization (168% above target performance)
|
81
|
-
- **$114/month VPC Savings**: Demonstrated through manager interface
|
82
|
-
- **200+ Account Scale**: Production-validated enterprise deployment
|
83
|
-
|
84
|
-
> Why CloudOps Runbooks?
|
85
|
-
|
86
|
-
- **๐ฏ Proven in Production**: Deployed across enterprises managing 200+ AWS accounts
|
87
|
-
- **๐ค AI-Agent Orchestration**: 6-agent FAANG SDLC with tmux coordination
|
88
|
-
- **โก Blazing Fast**: 0.11s execution (99% performance improvement)
|
89
|
-
- **๐ Enterprise Security**: Zero-trust validation, SOC2/PCI-DSS compliance
|
90
|
-
- **๐ฐ Quantified ROI**: 25-50% optimization with validated business metrics
|
91
|
-
- **๐๏ธ AWS Landing Zone Ready**: Multi-Organizations deployment proven
|
92
|
-
|
93
|
-
## ๐ Key Features
|
94
|
-
|
95
|
-
### ๐ **Comprehensive AWS Discovery**
|
96
|
-
- **Multi-Account Inventory**: Seamless discover resources (EC2, RDS, Lambda, ECS, S3, IAM, and more) across entire AWS Organizations
|
97
|
-
- **Cross-Region Support**: Parallel scanning of all available AWS regions
|
98
|
-
- **Resource Coverage**: 50+ AWS resource types across all major services
|
99
|
-
- **Real-time Collection**: Concurrent collection with progress tracking
|
100
|
-
|
101
|
-
### ๐๏ธ **Enterprise Architecture**
|
102
|
-
- **Type Safety**: Full Pydantic V2 models with runtime validation
|
103
|
-
- **Modular Design**: Service-specific collectors with common interfaces
|
104
|
-
- **Extensibility**: Easy to add new collectors and resource types
|
105
|
-
- **Error Handling**: Comprehensive error tracking and retry logic
|
106
|
-
|
107
|
-
|
108
|
-
### Hybrid Intelligence Integration
|
109
|
-
|
110
|
-
- **MCP Server Integration**: Real-time AWS API access without custom code
|
111
|
-
- **AI Agent Orchestration**: AI-powered analysis and recommendations
|
112
|
-
- **Evidence Pipeline**: Unified data normalization and correlation
|
113
|
-
- **Intelligent Prioritization**: ML-based resource targeting
|
114
|
-
|
115
|
-
### ๐ฐ **Cost Integration**
|
116
|
-
- **Cost Estimation**: Automatic cost calculations for billable resources
|
117
|
-
- **Cost Analytics**: Cost breakdown by service, account, and region
|
118
|
-
- **Budget Tracking**: Resource cost monitoring and alerting
|
119
|
-
|
120
|
-
### ๐ **Multiple Output Formats**
|
121
|
-
- **Structured Data**: JSON, CSV, Excel, Parquet
|
122
|
-
- **Visual Reports**: HTML reports with charts and graphs
|
123
|
-
- **Console Output**: Rich table formatting with colors
|
124
|
-
- **API Integration**: REST API for programmatic access
|
125
|
-
|
126
|
-
### ๐ **Security & Compliance**
|
127
|
-
- **IAM Integration**: Role-based access control
|
128
|
-
- **Audit Logging**: Comprehensive operation logging
|
129
|
-
- **Encryption**: Secure credential management
|
130
|
-
- **Compliance Reports**: Security and compliance validation
|
131
|
-
|
132
|
-
## ๐ Quick Start Excellence: Progressive Examples
|
133
|
-
|
134
|
-
### ๐ฆ Installation
|
135
|
-
|
136
|
-
```bash
|
137
|
-
# Install using UV (recommended for speed and reliability)
|
138
|
-
uv add runbooks
|
139
|
-
|
140
|
-
# Or using pip
|
141
|
-
pip install runbooks
|
142
|
-
|
143
|
-
# Development installation
|
144
|
-
git clone https://github.com/1xOps/CloudOps-Runbooks.git
|
145
|
-
cd CloudOps-Runbooks
|
146
|
-
uv sync --all-extras --dev
|
147
|
-
```
|
148
|
-
|
149
|
-
### ๐ฏ **Modern CLI Commands Overview**
|
150
|
-
|
151
|
-
CloudOps Runbooks now provides enterprise-grade CLI commands for comprehensive AWS operations:
|
152
|
-
|
153
|
-
```bash
|
154
|
-
# ๐ฏ STANDARDIZED CLI (Human & AI-Agent Friendly) v0.7.3
|
155
|
-
runbooks --help # Main CLI help
|
156
|
-
runbooks inventory collect # Read-only discovery & analysis
|
157
|
-
runbooks operate ec2 start # Resource lifecycle operations
|
158
|
-
runbooks cfat assess # Cloud Foundations Assessment
|
159
|
-
runbooks security assess # Security Baseline Testing
|
160
|
-
runbooks org list-ous # Organizations Management
|
161
|
-
runbooks finops dashboard # Cost and Usage Analytics
|
162
|
-
|
163
|
-
# ๐ค AI-Agent Optimized: Consistent options across all commands
|
164
|
-
# --profile, --region, --dry-run, --output, --force (where applicable)
|
165
|
-
```
|
166
|
-
|
167
|
-
### ๐ฐ Level 1: Basic Single Account Discovery
|
168
|
-
|
169
|
-
**Goal**: Discover EC2 instances in your current AWS account
|
170
|
-
|
171
|
-
```bash
|
172
|
-
# Set up your AWS credentials
|
173
|
-
export AWS_PROFILE="your-aws-profile"
|
174
|
-
aws sts get-caller-identity # Verify access
|
175
|
-
|
176
|
-
# Basic EC2 instance discovery
|
177
|
-
cd CloudOps-Runbooks
|
178
|
-
python src/runbooks/inventory/list_ec2_instances.py --profile $AWS_PROFILE --regions us-east-1 --timing
|
179
|
-
|
180
|
-
# Example output:
|
181
|
-
# Finding instances from 1 locations: 100%|โโโโโโโโโโ| 1/1 [00:02<00:00, 2.43 locations/s]
|
182
|
-
# Found 12 instances across 1 account across 1 region
|
183
|
-
# This script completed in 3.45 seconds
|
184
|
-
```
|
185
|
-
|
186
|
-
### ๐ Level 2: Multi-Service Resource Discovery
|
187
|
-
|
188
|
-
**Goal**: Discover multiple AWS resource types efficiently
|
189
|
-
|
190
|
-
```bash
|
191
|
-
# EBS Volumes with orphan detection
|
192
|
-
python src/runbooks/inventory/list_ec2_ebs_volumes.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
193
|
-
|
194
|
-
# Lambda Functions with cost analysis
|
195
|
-
python src/runbooks/inventory/list_lambda_functions.py --profile $AWS_PROFILE --regions ap-southeast-2
|
196
|
-
|
197
|
-
# RDS Instances across multiple regions
|
198
|
-
python src/runbooks/inventory/list_rds_db_instances.py --profile $AWS_PROFILE --regions us-east-1,eu-west-1,ap-southeast-2
|
199
|
-
|
200
|
-
# Security Groups analysis
|
201
|
-
python src/runbooks/inventory/find_ec2_security_groups.py --profile $AWS_PROFILE --regions us-east-1 --defaults
|
202
|
-
```
|
203
|
-
|
204
|
-
### ๐ข Level 3: Enterprise Multi-Account Operations
|
205
|
-
|
206
|
-
**Goal**: Organization-wide resource discovery and compliance
|
207
|
-
|
208
|
-
```bash
|
209
|
-
# Comprehensive inventory across AWS Organizations
|
210
|
-
python src/runbooks/inventory/list_org_accounts.py --profile $AWS_PROFILE
|
211
|
-
|
212
|
-
# Multi-account CloudFormation stack discovery
|
213
|
-
python src/runbooks/inventory/list_cfn_stacks.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
214
|
-
|
215
|
-
# Organization-wide GuardDuty detector inventory
|
216
|
-
python src/runbooks/inventory/list_guardduty_detectors.py --profile $AWS_PROFILE --regions ap-southeast-2
|
217
|
-
|
218
|
-
# CloudTrail compliance validation
|
219
|
-
python src/runbooks/inventory/check_cloudtrail_compliance.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
220
|
-
```
|
221
|
-
|
222
|
-
### ๐ Level 4: Autonomous Testing Framework
|
223
|
-
|
224
|
-
**Goal**: Automated testing and validation of entire inventory suite
|
225
|
-
|
226
|
-
```bash
|
227
|
-
# Test individual script
|
228
|
-
./src/runbooks/inventory/inventory.sh list_ec2_instances.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
229
|
-
|
230
|
-
# Test specific script category with detailed analysis
|
231
|
-
./src/runbooks/inventory/inventory.sh list_ec2_ebs_volumes.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
232
|
-
|
233
|
-
# Full autonomous test suite (20+ core scripts)
|
234
|
-
./src/runbooks/inventory/inventory.sh all --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
235
|
-
|
236
|
-
# Review test results and analysis
|
237
|
-
ls test_logs_*/
|
238
|
-
cat test_logs_*/test_execution.log
|
239
|
-
```
|
240
|
-
|
241
|
-
### ๐ฌ Level 5: Advanced Integration & Analysis
|
242
|
-
|
243
|
-
**Goal**: Production-grade automation with comprehensive reporting
|
244
|
-
|
245
|
-
```bash
|
246
|
-
# 1. VPC Network Discovery with Subnet Analysis
|
247
|
-
python src/runbooks/inventory/list_vpc_subnets.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
248
|
-
python src/runbooks/inventory/list_vpcs.py --profile $AWS_PROFILE --regions ap-southeast-2
|
249
|
-
|
250
|
-
# 2. Load Balancer Infrastructure Mapping
|
251
|
-
python src/runbooks/inventory/list_elbs_load_balancers.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
252
|
-
|
253
|
-
# 3. IAM Security Posture Assessment
|
254
|
-
python src/runbooks/inventory/list_iam_roles.py --profile $AWS_PROFILE --timing
|
255
|
-
python src/runbooks/inventory/list_iam_policies.py --profile $AWS_PROFILE --timing
|
256
|
-
|
257
|
-
# 4. ECS Container Platform Discovery
|
258
|
-
python src/runbooks/inventory/list_ecs_clusters_and_tasks.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
259
|
-
|
260
|
-
# 5. Network Interface and ENI Analysis
|
261
|
-
python src/runbooks/inventory/list_enis_network_interfaces.py --profile $AWS_PROFILE --regions ap-southeast-2 --timing
|
262
|
-
```
|
263
|
-
|
264
|
-
### ๐ฏ Level 6: Specialized Operations
|
265
|
-
|
266
|
-
**Goal**: Advanced scenarios for specific use cases
|
267
|
-
|
268
|
-
```bash
|
269
|
-
# 1. Landing Zone Readiness Assessment
|
270
|
-
python src/runbooks/inventory/check_landingzone_readiness.py --profile $AWS_PROFILE
|
271
|
-
|
272
|
-
# 2. CloudFormation Drift Detection
|
273
|
-
python src/runbooks/inventory/find_cfn_drift_detection.py --profile $AWS_PROFILE --regions ap-southeast-2
|
274
|
-
|
275
|
-
# 3. Organizations Structure Analysis
|
276
|
-
python src/runbooks/inventory/list_org_accounts_users.py --profile $AWS_PROFILE --timing
|
277
|
-
|
278
|
-
# 4. Config Compliance Monitoring
|
279
|
-
python src/runbooks/inventory/list_config_recorders_delivery_channels.py --profile $AWS_PROFILE --regions ap-southeast-2
|
280
|
-
|
281
|
-
# 5. Route53 DNS Infrastructure
|
282
|
-
python src/runbooks/inventory/list_route53_hosted_zones.py --profile $AWS_PROFILE --timing
|
283
|
-
```
|
284
|
-
|
285
|
-
### ๐ Integration Examples
|
286
|
-
|
287
|
-
**Modern Architecture Integration:**
|
288
|
-
|
289
|
-
```python
|
290
|
-
# collectors/ and core/ directories provide modern modular architecture
|
291
|
-
from runbooks.inventory.collectors.aws_compute import ComputeCollector
|
292
|
-
from runbooks.inventory.core.collector import InventoryCollector
|
293
|
-
from runbooks.inventory.core.formatter import OutputFormatter
|
294
|
-
|
295
|
-
# Enterprise-grade type-safe collection
|
296
|
-
collector = InventoryCollector(profile='production')
|
297
|
-
results = collector.collect_compute_resources(include_costs=True)
|
298
|
-
formatter = OutputFormatter()
|
299
|
-
report = formatter.generate_html_report(results)
|
300
|
-
```
|
301
|
-
|
302
|
-
## ๐ข **Level 7: Enterprise CLI Operations**
|
303
|
-
|
304
|
-
### **AWS Resource Operations**
|
305
|
-
|
306
|
-
**Goal**: Comprehensive AWS resource lifecycle management with enterprise safety features
|
307
|
-
|
308
|
-
```bash
|
309
|
-
# EC2 Instance Operations
|
310
|
-
runbooks operate ec2 start --instance-ids i-1234567890abcdef0 --profile production
|
311
|
-
runbooks operate ec2 stop --instance-ids i-1234567890abcdef0 i-0987654321fedcba0 --dry-run
|
312
|
-
runbooks operate ec2 terminate --instance-ids i-1234567890abcdef0 --force
|
313
|
-
|
314
|
-
# S3 Bucket Operations with Security Best Practices
|
315
|
-
runbooks operate s3 create-bucket --bucket-name secure-prod-bucket \
|
316
|
-
--encryption --versioning --public-access-block --region us-west-2
|
317
|
-
runbooks operate s3 delete-bucket-and-objects --bucket-name old-test-bucket --dry-run
|
318
|
-
runbooks operate s3 set-public-access-block --account-id 123456789012 --profile management
|
319
|
-
|
320
|
-
# CloudFormation StackSet Operations
|
321
|
-
runbooks operate cloudformation move-stack-instances \
|
322
|
-
--source-stackset-name old-baseline --target-stackset-name new-baseline \
|
323
|
-
--account-ids 111111111111,222222222222 --regions us-east-1,us-west-2 --dry-run
|
324
|
-
runbooks operate cloudformation lockdown-stackset-role \
|
325
|
-
--target-role-name AWSCloudFormationStackSetExecutionRole \
|
326
|
-
--management-account-id 123456789012
|
327
|
-
|
328
|
-
# IAM Cross-Account Role Management
|
329
|
-
runbooks operate iam update-roles-cross-accounts \
|
330
|
-
--role-name CrossAccountAccessRole \
|
331
|
-
--trusted-account-ids 111111111111,222222222222 \
|
332
|
-
--external-id MySecureExternalId --require-mfa
|
333
|
-
|
334
|
-
# CloudWatch Log Management
|
335
|
-
runbooks operate cloudwatch update-log-retention-policy \
|
336
|
-
--retention-days 30 --update-all-log-groups --profile production
|
337
|
-
|
338
|
-
# DynamoDB Table Operations
|
339
|
-
runbooks operate dynamodb create-table \
|
340
|
-
--table-name user-sessions --hash-key user_id --range-key session_id \
|
341
|
-
--billing-mode PAY_PER_REQUEST --tags Environment=production Team=backend
|
342
|
-
runbooks operate dynamodb backup-table --table-name critical-data --backup-name weekly-backup
|
343
|
-
runbooks operate dynamodb delete-table --table-name temp-table --confirm --dry-run
|
344
|
-
|
345
|
-
# Cross-Service Resource Tagging
|
346
|
-
runbooks operate tag apply-template --template production \
|
347
|
-
--resource-arns arn:aws:ec2:us-east-1:123456789012:instance/i-1234567890abcdef0
|
348
|
-
```
|
349
|
-
|
350
|
-
### **Cloud Foundations Assessment Tool (CFAT)**
|
351
|
-
|
352
|
-
**Goal**: Comprehensive AWS account assessment against best practices
|
353
|
-
|
354
|
-
```bash
|
355
|
-
# Basic CFAT assessment with HTML report
|
356
|
-
runbooks cfat assess --profile production
|
357
|
-
|
358
|
-
# Multiple output formats with compliance framework
|
359
|
-
runbooks cfat assess --profile prod \
|
360
|
-
--compliance-framework "AWS Well-Architected" \
|
361
|
-
--output all \
|
362
|
-
--serve-web --web-port 8080
|
363
|
-
|
364
|
-
# Targeted assessment with specific categories
|
365
|
-
runbooks cfat assess --profile dev \
|
366
|
-
--categories iam,s3,vpc \
|
367
|
-
--severity CRITICAL \
|
368
|
-
--parallel --max-workers 10
|
369
|
-
|
370
|
-
# Export results to project management tools
|
371
|
-
runbooks cfat assess --profile staging \
|
372
|
-
--export-jira --export-asana \
|
373
|
-
--output json
|
374
|
-
```
|
375
|
-
|
376
|
-
### **Security Baseline Assessment**
|
377
|
-
|
378
|
-
**Goal**: Multi-language security compliance validation
|
379
|
-
|
380
|
-
```bash
|
381
|
-
# Comprehensive security assessment (English)
|
382
|
-
runbooks security assess --profile production --language EN
|
383
|
-
|
384
|
-
# Localized security reports for global teams
|
385
|
-
runbooks security assess --profile prod-asia \
|
386
|
-
--language JP --format html --output /reports/security
|
387
|
-
|
388
|
-
# Run specific security checks
|
389
|
-
runbooks security check root_mfa --profile management
|
390
|
-
runbooks security check iam_password_policy --profile prod
|
391
|
-
|
392
|
-
# List available security checks
|
393
|
-
runbooks security list-checks
|
394
|
-
|
395
|
-
# Multiple checks with JSON output
|
396
|
-
runbooks security assess --profile dev \
|
397
|
-
--checks root_mfa --checks bucket_public_access \
|
398
|
-
--format json
|
399
|
-
```
|
400
|
-
|
401
|
-
### **AWS Organizations Management**
|
402
|
-
|
403
|
-
**Goal**: Enterprise OU structure setup and management
|
404
|
-
|
405
|
-
```bash
|
406
|
-
# List current organizational structure
|
407
|
-
runbooks org list-ous --profile management --output table
|
408
|
-
|
409
|
-
# Setup standard OU structure (dry-run first)
|
410
|
-
runbooks org setup-ous --profile management \
|
411
|
-
--template standard --dry-run
|
412
|
-
|
413
|
-
# Create production OU structure
|
414
|
-
runbooks org setup-ous --profile management \
|
415
|
-
--template security
|
416
|
-
|
417
|
-
# Custom OU structure from file
|
418
|
-
runbooks org setup-ous --profile management \
|
419
|
-
--file custom-ou-structure.yaml
|
420
|
-
|
421
|
-
# Export OU structure to different formats
|
422
|
-
runbooks org list-ous --profile management --output json > ou-structure.json
|
423
|
-
runbooks org list-ous --profile management --output yaml > ou-structure.yaml
|
424
|
-
|
425
|
-
# Delete empty organizational unit (with confirmation)
|
426
|
-
runbooks org delete-ou ou-1234567890abcdef --confirm
|
427
|
-
```
|
428
|
-
|
429
|
-
### **Advanced Multi-Command Workflows**
|
430
|
-
|
431
|
-
**Goal**: Combine multiple tools for comprehensive AWS operations
|
432
|
-
|
433
|
-
```bash
|
434
|
-
# 1. Complete AWS account assessment and remediation workflow
|
435
|
-
echo "๐ Step 1: Security Baseline Assessment"
|
436
|
-
runbooks security assess --profile prod --language EN --format json > security-report.json
|
437
|
-
|
438
|
-
echo "๐๏ธ Step 2: Cloud Foundations Assessment"
|
439
|
-
runbooks cfat assess --profile prod --output all --compliance-framework "AWS Well-Architected"
|
440
|
-
|
441
|
-
echo "๐ข Step 3: Organizations Structure Review"
|
442
|
-
runbooks org list-ous --profile management --output yaml > current-ou-structure.yaml
|
443
|
-
|
444
|
-
echo "๐ Step 4: Resource Inventory"
|
445
|
-
runbooks inventory collect -r ec2 -r s3 --profile prod --output json > resource-inventory.json
|
446
|
-
|
447
|
-
echo "โ๏ธ Step 5: Automated Remediation"
|
448
|
-
runbooks operate s3 set-public-access-block --account-id 123456789012 --profile management
|
449
|
-
runbooks operate cloudwatch update-log-retention-policy --retention-days 90 --update-all
|
450
|
-
runbooks operate tag apply-template --template production --resource-arns $(cat resource-inventory.json | jq -r '.ec2[].arn')
|
451
|
-
|
452
|
-
# 2. Setup new AWS environment workflow with security hardening
|
453
|
-
echo "๐ Setting up new secure AWS environment"
|
454
|
-
runbooks org setup-ous --template security --profile management
|
455
|
-
runbooks operate s3 set-public-access-block --account-id NEW_ACCOUNT_ID --profile management
|
456
|
-
runbooks operate iam update-roles-cross-accounts --role-name CrossAccountAuditRole --trusted-account-ids AUDIT_ACCOUNT_ID
|
457
|
-
runbooks security assess --profile new-account --language EN
|
458
|
-
runbooks cfat assess --profile new-account --categories iam,s3,vpc,security --output html
|
459
|
-
|
460
|
-
# 3. Disaster recovery and cleanup workflow
|
461
|
-
echo "๐ง Emergency cleanup and recovery"
|
462
|
-
runbooks operate ec2 stop --instance-ids $(runbooks inventory collect -r ec2 --filter state=running --output json | jq -r '.[].InstanceId') --dry-run
|
463
|
-
runbooks operate s3 delete-bucket-and-objects --bucket-name old-backup-bucket --dry-run
|
464
|
-
runbooks operate cloudformation move-stack-instances --source-stackset old-infra --target-stackset new-infra --dry-run
|
465
|
-
|
466
|
-
# 4. Compliance and governance workflow
|
467
|
-
echo "๐ Running compliance checks and governance"
|
468
|
-
runbooks security assess --profile all-accounts --format json
|
469
|
-
runbooks cfat assess --profile all-accounts --compliance-framework "SOC2" --export-jira
|
470
|
-
runbooks org list-ous --profile management --output json
|
471
|
-
runbooks operate tag apply-template --template compliance --resource-arns $(runbooks inventory collect --profile all-accounts | jq -r '.[].arn')
|
472
|
-
```
|
473
|
-
|
474
|
-
### ๐ Performance & Success Metrics
|
475
|
-
|
476
|
-
**Enterprise CLI Results (Latest v0.7.3):**
|
477
|
-
- โ
**Production-Ready CLI**: 18+ complete AWS operations across all major services
|
478
|
-
- โ
**Complete EC2 Operations**: 7/7 commands (start, stop, terminate, run, copy, cleanup)
|
479
|
-
- โ
**Complete S3 Operations**: 3/3 commands (create, delete, public-access-block)
|
480
|
-
- โ
**Enterprise CloudFormation**: 3/3 commands (move-instances, lockdown-role, update-stacksets)
|
481
|
-
- โ
**IAM & CloudWatch**: Complete operational commands for enterprise management
|
482
|
-
- โ
**DynamoDB Operations**: 3/3 commands (create-table, delete-table, backup-table)
|
483
|
-
- โ
**KISS Architecture**: No legacy directories - clean, maintainable codebase
|
484
|
-
- โ
**Standardized CLI**: Consistent options across all commands for human & AI-Agent use
|
485
|
-
- โ
**Complete Migration**: All AWS operations properly categorized (inventory vs operate)
|
486
|
-
- โ
**CFAT Module**: Fully integrated with enterprise features and web reporting
|
487
|
-
- โ
**Security Module**: 15+ security checks with multi-language support (EN, JP, KR, VN)
|
488
|
-
- โ
**Organizations Module**: Complete OU management with template-based setup
|
489
|
-
- โ
**Legacy Inventory**: 20/43 core scripts passing (47% success rate) - maintained compatibility
|
490
|
-
- โก **Performance**: 8-12 seconds average execution, parallel processing support
|
491
|
-
- ๐๏ธ **Architecture**: Modern modular design with collectors/, core/, operate/, and assessment/ directories
|
492
|
-
- ๐ง **Integration**: Enhanced with EC2, S3, CloudFormation, IAM, and CloudWatch operations
|
493
|
-
- ๐ค **AI-Agent Ready**: Predictable CLI patterns, consistent error handling, standardized outputs
|
494
|
-
|
495
|
-
## ๐ Architecture Overview
|
496
|
-
|
497
|
-
### ๐๏ธ **Enterprise Module Structure**
|
498
|
-
|
499
|
-
```
|
500
|
-
src/runbooks/
|
501
|
-
โโโ ๐๏ธ cfat/ # Cloud Foundations Assessment Tool
|
502
|
-
โ โโโ assessment/ # Assessment engine and runners
|
503
|
-
โ โ โโโ runner.py # CloudFoundationsAssessment (enhanced)
|
504
|
-
โ โ โโโ collectors.py # AWS resource collection logic
|
505
|
-
โ โ โโโ validators.py # Compliance rule validation
|
506
|
-
โ โโโ reporting/ # Multi-format report generation
|
507
|
-
โ โ โโโ formatters.py # HTML, JSON, CSV, Markdown generators
|
508
|
-
โ โ โโโ templates.py # Executive, Technical, Compliance templates
|
509
|
-
โ โ โโโ exporters.py # Jira, Asana, ServiceNow integration
|
510
|
-
โ โโโ tests/ # Comprehensive test suite
|
511
|
-
โ โโโ models.py # Pydantic data models with validation
|
512
|
-
โ โโโ cli.py # Enterprise CLI with web server
|
513
|
-
โโโ ๐ security/ # Security Baseline Assessment
|
514
|
-
โ โโโ checklist/ # 15+ security validation modules
|
515
|
-
โ โโโ security_baseline_tester.py # Multi-language assessment engine
|
516
|
-
โ โโโ report_generator.py # HTML reports with remediation
|
517
|
-
โ โโโ utils/ # Security-specific utilities
|
518
|
-
โโโ ๐ inventory/ # Multi-account Resource Discovery
|
519
|
-
โ โโโ core/ # Business Logic & Orchestration
|
520
|
-
โ โ โโโ collector.py # Main inventory orchestration engine
|
521
|
-
โ โ โโโ formatter.py # Multi-format output handling
|
522
|
-
โ โ โโโ session_manager.py # AWS session management
|
523
|
-
โ โโโ collectors/ # Specialized Resource Collectors
|
524
|
-
โ โ โโโ base.py # Abstract base collector interface
|
525
|
-
โ โ โโโ aws_compute.py # EC2, Lambda, ECS, Batch
|
526
|
-
โ โ โโโ aws_networking.py # VPC, ELB, Route53, CloudFront
|
527
|
-
โ โ โโโ aws_management.py # Organizations, CloudFormation, SSM
|
528
|
-
โ โโโ models/ # Type-safe Data Structures
|
529
|
-
โ โ โโโ account.py # AWS account representation
|
530
|
-
โ โ โโโ resource.py # Resource models with metadata
|
531
|
-
โ โ โโโ inventory.py # Collection results and analytics
|
532
|
-
โ โโโ utils/ # Shared Utilities & Helpers
|
533
|
-
โ โ โโโ aws_helpers.py # AWS session and API utilities
|
534
|
-
โ โ โโโ threading_utils.py # Concurrent execution helpers
|
535
|
-
โ โ โโโ validation.py # Input validation and sanitization
|
536
|
-
โ โโโ ๐ legacy/ # Legacy Script Compatibility
|
537
|
-
โ โโโ migration_guide.md # Legacy script migration guide
|
538
|
-
โโโ โ๏ธ operate/ # AWS Resource Operations (v0.7.3 - KISS Principle)
|
539
|
-
โ โโโ base.py # Abstract operation framework
|
540
|
-
โ โโโ ec2_operations.py # Complete EC2 lifecycle operations
|
541
|
-
โ โโโ s3_operations.py # Complete S3 bucket and object operations
|
542
|
-
โ โโโ dynamodb_operations.py # DynamoDB table operations
|
543
|
-
โ โโโ cloudformation_operations.py # CloudFormation and StackSet operations
|
544
|
-
โ โโโ iam_operations.py # IAM role and policy operations
|
545
|
-
โ โโโ cloudwatch_operations.py # CloudWatch logs and metrics
|
546
|
-
โ โโโ tagging_operations.py # Cross-service resource tagging
|
547
|
-
โ โโโ tags.json # Shared tag templates (no legacy complexity)
|
548
|
-
โโโ ๐ฐ finops/ # Cost and Usage Analytics
|
549
|
-
โโโ ๐ ๏ธ utils/ # Core Framework Utilities
|
550
|
-
โโโ ๐งช tests/ # Enterprise Test Framework
|
551
|
-
โโโ ๐ docs/ # Comprehensive Documentation
|
552
|
-
```
|
553
|
-
|
554
|
-
## ๐งช Testing & Development
|
555
|
-
|
556
|
-
### Running Tests
|
557
|
-
|
558
|
-
```bash
|
559
|
-
# Run full test suite
|
560
|
-
task test
|
561
|
-
|
562
|
-
# Run specific test categories
|
563
|
-
pytest tests/unit/test_inventory.py -v
|
564
|
-
pytest tests/integration/test_collectors.py -v
|
565
|
-
|
566
|
-
# Test with coverage
|
567
|
-
task _test.coverage
|
568
|
-
|
569
|
-
# Test inventory module specifically
|
570
|
-
task inventory.test
|
571
|
-
```
|
572
|
-
|
573
|
-
### Development Workflow
|
574
|
-
|
575
|
-
```bash
|
576
|
-
# Install development dependencies
|
577
|
-
task install
|
578
|
-
|
579
|
-
# Code quality checks
|
580
|
-
task code_quality
|
581
|
-
|
582
|
-
# Validate module structure
|
583
|
-
task inventory.validate
|
584
|
-
|
585
|
-
# Full validation workflow
|
586
|
-
task validate
|
587
|
-
```
|
588
|
-
|
589
|
-
## ๐ Documentation
|
590
|
-
|
591
|
-
### **Enterprise Documentation Suite** ๐
|
592
|
-
- **[Executive Summary](docs/EXECUTIVE-SUMMARY.md)** - Strategic achievements and business impact
|
593
|
-
- **[Architecture Guide](docs/ARCHITECTURE.md)** - Complete system architecture and design patterns
|
594
|
-
- **[Deployment Guide](docs/DEPLOYMENT.md)** - Enterprise deployment patterns and procedures
|
595
|
-
- **[Agent Coordination](docs/AGENT-COORDINATION-GUIDE.md)** - 6-agent FAANG SDLC implementation
|
596
|
-
- **[Business Value Metrics](docs/BUSINESS-VALUE-METRICS.md)** - ROI analysis and financial impact
|
597
|
-
- **[Issue Summaries](docs/ISSUE-SUMMARIES.md)** - Completed strategic missions documentation
|
598
|
-
|
599
|
-
### **Technical Documentation** ๐ง
|
600
|
-
- [API Reference](docs/api-reference.md) - CLI and SDK documentation
|
601
|
-
- [Configuration Guide](docs/configuration.md) - Multi-profile setup and enterprise configuration
|
602
|
-
- [Migration Guide](src/runbooks/inventory/legacy/migration_guide.md) - Legacy system migration patterns
|
603
|
-
- [Contributing Guide](CONTRIBUTING.md) - Development workflow and standards
|
604
|
-
|
605
|
-
### **GitHub Workflow Integration** ๐
|
606
|
-
- **[Strategic Mission Template](.github/ISSUE_TEMPLATE/enterprise-strategic-mission.md)** - High-impact business initiatives
|
607
|
-
- **[Agent Coordination Template](.github/ISSUE_TEMPLATE/agent-coordination-task.md)** - Multi-agent FAANG SDLC workflows
|
608
|
-
- **[Manager Communication Template](.github/ISSUE_TEMPLATE/manager-communication.md)** - Executive stakeholder coordination
|
609
|
-
|
610
|
-
|
611
|
-
## ๐ฆ Roadmap
|
612
|
-
|
613
|
-
- **v1.0** (Q4 2025): Enhanced AI agent orchestration
|
614
|
-
- **v1.5** (Q1 2026): Self-healing infrastructure capabilities
|
615
|
-
|
616
|
-
## ๐ License
|
617
|
-
|
618
|
-
This project is licensed under the Apache License 2.0 - see the [LICENSE](LICENSE) file for details.
|
619
|
-
|
620
|
-
## ๐ Support
|
621
|
-
|
622
|
-
### Community
|
623
|
-
- [GitHub Issues](https://github.com/1xOps/CloudOps-Runbooks/issues)
|
624
|
-
- [Discussions](https://github.com/1xOps/CloudOps-Runbooks/discussions)
|
625
|
-
|
626
|
-
### Enterprise Support
|
627
|
-
- Professional services and training available
|
628
|
-
- Custom collector development
|
629
|
-
- Enterprise deployment assistance
|
630
|
-
- Contact: [info@oceansoft.io](mailto:info@oceansoft.io)
|
631
|
-
|
632
|
-
---
|
633
|
-
|
634
|
-
**Built with โค๏ธ by the xOps team at OceanSoft**
|
635
|
-
|
636
|
-
[Website](https://cloudops.oceansoft.io) โข [Documentation](https://cloudops.oceansoft.io/runbooks/) โข [GitHub](https://github.com/1xOps/CloudOps-Runbooks)
|
File without changes
|
File without changes
|
File without changes
|