iam-policy-validator 1.1.2__py3-none-any.whl → 1.3.0__py3-none-any.whl

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Potentially problematic release.


This version of iam-policy-validator might be problematic. Click here for more details.

@@ -1,6 +1,6 @@
1
1
  Metadata-Version: 2.4
2
2
  Name: iam-policy-validator
3
- Version: 1.1.2
3
+ Version: 1.3.0
4
4
  Summary: Validate AWS IAM policies for correctness and security using AWS Service Reference API
5
5
  Project-URL: Homepage, https://github.com/boogy/iam-policy-validator
6
6
  Project-URL: Documentation, https://github.com/boogy/iam-policy-validator/tree/main/docs
@@ -42,47 +42,129 @@ Description-Content-Type: text/markdown
42
42
 
43
43
  # IAM Policy Validator
44
44
 
45
- A high-performance GitHub Action and Python CLI tool that validates AWS IAM policies for correctness and security by checking against the official AWS Service Reference API.
46
-
47
- ## ✨ Features
48
-
49
- ### Core Validation
50
- - **Real-time Validation**: Validates IAM actions against AWS's official service reference API
51
- - **AWS IAM Access Analyzer Integration**: Validate policies using AWS's official policy validation service
52
- - **Custom Policy Checks**: Verify policies don't grant specific actions, check for new access, and detect public exposure (29+ resource types supported)
53
- - **Condition Key Checking**: Verifies that condition keys are valid for each action
54
- - **ARN Format Validation**: Ensures resource ARNs follow proper AWS format with compiled regex patterns
55
- - **Security Best Practices**: Identifies overly permissive policies and security risks
56
-
57
- ### Performance Enhancements
58
- - **Service Pre-fetching**: Common AWS services cached at startup for faster validation
59
- - **LRU Memory Cache**: Recently accessed services cached with TTL support
60
- - **Request Coalescing**: Duplicate API requests automatically deduplicated
61
- - **Parallel Check Execution**: Multiple validation checks run concurrently
62
- - **HTTP/2 Support**: Multiplexed connections for better API performance
63
- - **Optimized Connection Pool**: 20 keepalive connections, 50 max connections
64
-
65
- ### GitHub Integration
66
- - **PR Comments**: Post detailed validation reports as PR comments
67
- - **Line-Specific Reviews**: Add review comments on exact policy lines
68
- - **Label Management**: Automatically add/remove PR labels based on results
69
- - **Commit Status**: Set commit status to pass/fail based on validation
70
- - **Comment Updates**: Update existing comments instead of creating duplicates
71
-
72
- ### Output Formats
73
- - **Console** (default): Clean terminal output with colors and tables
74
- - **Enhanced**: Modern visual output with progress bars, tree structure, and rich visuals
75
- - **JSON**: Structured format for programmatic processing
76
- - **Markdown**: GitHub-flavored markdown for PR comments
77
- - **SARIF**: GitHub code scanning integration format
78
- - **CSV**: Spreadsheet-compatible format for analysis
79
- - **HTML**: Interactive reports with filtering and search
80
-
81
- ### Extensibility
82
- - **Plugin System**: Easy-to-add custom validation checks
83
- - **Middleware Support**: Cross-cutting concerns like caching, timing, error handling
84
- - **Formatter Registry**: Pluggable output format system
85
- - **Configuration-Driven**: YAML-based configuration for all aspects
45
+ > **Catch IAM policy errors before they reach production** - A comprehensive security and validation tool for AWS IAM policies that combines AWS's official Access Analyzer with powerful custom security checks.
46
+
47
+ [![GitHub Actions](https://img.shields.io/badge/GitHub%20Actions-Ready-blue)](https://github.com/marketplace/actions/iam-policy-validator)
48
+ [![Python 3.12+](https://img.shields.io/badge/python-3.12+-blue.svg)](https://www.python.org/downloads/)
49
+ [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](LICENSE)
50
+
51
+ ## 🚀 Why IAM Policy Validator?
52
+
53
+ **IAM policy errors are costly and dangerous.** A single misconfigured policy can:
54
+ - Grant unintended admin access (privilege escalation)
55
+ - Expose sensitive data to the public
56
+ - ❌ Break production deployments with invalid syntax
57
+ - Create security vulnerabilities that persist for months
58
+
59
+ **This tool prevents these issues** by:
60
+ - **Validating early** - Catch errors in PRs before merge
61
+ - **Comprehensive checks** - AWS Access Analyzer + 15+ security checks
62
+ - **Smart filtering** - Auto-detects IAM policies from mixed JSON/YAML files
63
+ - **Developer-friendly** - Clear error messages with fix suggestions
64
+ - ✅ **Zero setup** - Works as a GitHub Action out of the box
65
+
66
+ ## Key Features
67
+
68
+ ### 🔍 Multi-Layer Validation
69
+ - **AWS IAM Access Analyzer** - Official AWS validation (syntax, permissions, security)
70
+ - **Custom Security Checks** - 15+ specialized checks for best practices
71
+ - **Policy Comparison** - Detect new permissions vs baseline (prevent scope creep)
72
+ - **Public Access Detection** - Check 29+ AWS resource types for public exposure
73
+ - **Privilege Escalation Detection** - Identify dangerous action combinations
74
+
75
+ ### 🎯 Smart & Efficient
76
+ - **Automatic IAM Policy Detection** - Scans mixed repos, filters non-IAM files automatically
77
+ - **Wildcard Expansion** - Expands `s3:Get*` patterns to validate specific actions
78
+ - **Offline Validation** - Download AWS service definitions for air-gapped environments
79
+ - **JSON + YAML Support** - Native support for both formats
80
+ - **Streaming Mode** - Memory-efficient processing for large policy sets
81
+
82
+ ### Performance Optimized
83
+ - **Service Pre-fetching** - Common AWS services cached at startup (faster validation)
84
+ - **LRU Memory Cache** - Recently accessed services cached with TTL
85
+ - **Request Coalescing** - Duplicate API requests automatically deduplicated
86
+ - **Parallel Execution** - Multiple checks run concurrently
87
+ - **HTTP/2 Support** - Multiplexed connections for better API performance
88
+
89
+ ### 📊 Output Formats
90
+ - **Console** (default) - Clean terminal output with colors and tables
91
+ - **Enhanced** - Modern visual output with progress bars and tree structure
92
+ - **JSON** - Structured format for programmatic processing
93
+ - **Markdown** - GitHub-flavored markdown for PR comments
94
+ - **SARIF** - GitHub code scanning integration format
95
+ - **CSV** - Spreadsheet-compatible for analysis
96
+ - **HTML** - Interactive reports with filtering and search
97
+
98
+ ### 🔌 Extensibility
99
+ - **Plugin System** - Easy-to-add custom validation checks
100
+ - **Configuration-Driven** - YAML-based configuration for all aspects
101
+ - **CI/CD Ready** - GitHub Actions, GitLab CI, Jenkins, CircleCI
102
+
103
+ ## 📈 Real-World Impact
104
+
105
+ ### Common IAM Policy Issues This Tool Catches
106
+
107
+ **Before IAM Policy Validator:**
108
+ ```json
109
+ {
110
+ "Statement": [{
111
+ "Effect": "Allow",
112
+ "Action": "s3:*", // ❌ Too permissive
113
+ "Resource": "*" // ❌ All buckets!
114
+ }]
115
+ }
116
+ ```
117
+ **Issue:** Grants full S3 access to ALL buckets (data breach risk)
118
+
119
+ **After IAM Policy Validator:**
120
+ ```
121
+ ❌ MEDIUM: Statement applies to all resources (*)
122
+ ❌ HIGH: Wildcard action 's3:*' with resource '*' is overly permissive
123
+ 💡 Suggestion: Specify exact actions and bucket ARNs
124
+ ```
125
+
126
+ ### Privilege Escalation Detection
127
+
128
+ **Dangerous combination across multiple statements:**
129
+ ```json
130
+ {
131
+ "Statement": [
132
+ {"Action": "iam:CreateUser"}, // Seems innocent
133
+ {"Action": "iam:AttachUserPolicy"} // Also seems innocent
134
+ ]
135
+ }
136
+ ```
137
+
138
+ **What the validator catches:**
139
+ ```
140
+ 🚨 CRITICAL: Privilege escalation risk detected!
141
+ Actions ['iam:CreateUser', 'iam:AttachUserPolicy'] allow:
142
+ 1. Create new IAM user
143
+ 2. Attach AdministratorAccess policy to that user
144
+ 3. Gain full AWS account access
145
+
146
+ 💡 Add conditions or separate these permissions
147
+ ```
148
+
149
+ ### Public Access Prevention
150
+
151
+ **Before merge:**
152
+ ```json
153
+ {
154
+ "Principal": "*", // ❌ Anyone on the internet!
155
+ "Action": "s3:GetObject",
156
+ "Resource": "arn:aws:s3:::my-private-data/*"
157
+ }
158
+ ```
159
+
160
+ **Blocked by validator:**
161
+ ```
162
+ 🛑 CRITICAL: Resource policy allows public access
163
+ 29 resource types checked: AWS::S3::Bucket
164
+ Principal "*" grants internet-wide access to private data
165
+
166
+ 💡 Use specific AWS principals or add IP restrictions
167
+ ```
86
168
 
87
169
  ## Quick Start
88
170
 
@@ -387,27 +469,56 @@ See [default-config.yaml](default-config.yaml) for a complete configuration exam
387
469
 
388
470
  ### GitHub Action Inputs
389
471
 
390
- | Input | Description | Required | Default |
391
- | ----------------------------- | ---------------------------------------------------------------------- | -------- | ----------------- |
392
- | `path` | Path(s) to IAM policy file or directory (newline-separated) | Yes | - |
393
- | `config-file` | Path to custom configuration file (iam-validator.yaml) | No | "" |
394
- | `fail-on-warnings` | Fail validation if warnings are found | No | `false` |
395
- | `post-comment` | Post validation results as PR comment | No | `true` |
396
- | `create-review` | Create line-specific review comments on PR | No | `true` |
397
- | `format` | Output format (console, enhanced, json, markdown, sarif, csv, html) | No | `console` |
398
- | `output-file` | Path to save output file | No | "" |
399
- | `recursive` | Recursively search directories for policy files | No | `true` |
400
- | `use-access-analyzer` | Use AWS IAM Access Analyzer for validation | No | `false` |
401
- | `access-analyzer-region` | AWS region for Access Analyzer | No | `us-east-1` |
402
- | `policy-type` | Policy type (IDENTITY_POLICY, RESOURCE_POLICY, SERVICE_CONTROL_POLICY) | No | `IDENTITY_POLICY` |
403
- | `run-all-checks` | Run custom checks after Access Analyzer | No | `false` |
404
- | `check-access-not-granted` | Actions that should NOT be granted (space-separated) | No | "" |
405
- | `check-access-resources` | Resources to check with check-access-not-granted | No | "" |
406
- | `check-no-new-access` | Path to baseline policy to compare against | No | "" |
407
- | `check-no-public-access` | Check that resource policies do not allow public access | No | `false` |
408
- | `public-access-resource-type` | Resource type(s) for public access check | No | `AWS::S3::Bucket` |
409
-
410
- See [examples/github-actions/](examples/github-actions/) for more workflow examples.
472
+ #### Core Options
473
+ | Input | Description | Required | Default |
474
+ | ------------------ | ----------------------------------------------------------- | -------- | ------- |
475
+ | `path` | Path(s) to IAM policy file or directory (newline-separated) | Yes | - |
476
+ | `config-file` | Path to custom configuration file (.yaml) | No | `""` |
477
+ | `fail-on-warnings` | Fail validation if warnings are found | No | `false` |
478
+ | `recursive` | Recursively search directories for policy files | No | `true` |
479
+
480
+ #### GitHub Integration
481
+ | Input | Description | Required | Default |
482
+ | --------------- | ------------------------------------------ | -------- | ------- |
483
+ | `post-comment` | Post validation results as PR comment | No | `true` |
484
+ | `create-review` | Create line-specific review comments on PR | No | `true` |
485
+
486
+ #### Output Options
487
+ | Input | Description | Required | Default |
488
+ | ------------- | -------------------------------------------------------------------------------- | -------- | --------- |
489
+ | `format` | Output format: `console`, `enhanced`, `json`, `markdown`, `sarif`, `csv`, `html` | No | `console` |
490
+ | `output-file` | Path to save output file (for non-console formats) | No | `""` |
491
+
492
+ #### AWS Access Analyzer
493
+ | Input | Description | Required | Default |
494
+ | ------------------------ | --------------------------------------------------------------------------- | -------- | ----------------- |
495
+ | `use-access-analyzer` | Use AWS IAM Access Analyzer for validation | No | `false` |
496
+ | `access-analyzer-region` | AWS region for Access Analyzer | No | `us-east-1` |
497
+ | `policy-type` | Policy type: `IDENTITY_POLICY`, `RESOURCE_POLICY`, `SERVICE_CONTROL_POLICY` | No | `IDENTITY_POLICY` |
498
+ | `run-all-checks` | Run custom checks after Access Analyzer (sequential mode) | No | `false` |
499
+
500
+ #### Custom Policy Checks (Access Analyzer)
501
+ | Input | Description | Required | Default |
502
+ | ----------------------------- | --------------------------------------------------------------------------- | -------- | ----------------- |
503
+ | `check-access-not-granted` | Actions that should NOT be granted (space-separated, max 100) | No | `""` |
504
+ | `check-access-resources` | Resources to check with check-access-not-granted (space-separated, max 100) | No | `""` |
505
+ | `check-no-new-access` | Path to baseline policy to compare against (detect new permissions) | No | `""` |
506
+ | `check-no-public-access` | Check that resource policies do not allow public access | No | `false` |
507
+ | `public-access-resource-type` | Resource type(s) for public access check (29+ types supported, or `all`) | No | `AWS::S3::Bucket` |
508
+
509
+ #### Advanced Options
510
+ | Input | Description | Required | Default |
511
+ | ------------------- | -------------------------------------------------------------- | -------- | --------- |
512
+ | `custom-checks-dir` | Path to directory containing custom validation checks | No | `""` |
513
+ | `log-level` | Logging level: `debug`, `info`, `warning`, `error`, `critical` | No | `warning` |
514
+
515
+ **💡 Pro Tips:**
516
+ - Use `custom-checks-dir` to add organization-specific validation rules
517
+ - Set `log-level: debug` when troubleshooting workflow issues
518
+ - Configure `aws-services-dir` in your config file for offline validation
519
+ - The action automatically filters IAM policies from mixed JSON/YAML files
520
+
521
+ See [examples/github-actions/](examples/github-actions/) for 8 ready-to-use workflow examples.
411
522
 
412
523
  ### As a CLI Tool
413
524
 
@@ -712,7 +823,8 @@ The comprehensive [DOCS.md](DOCS.md) file contains everything you need:
712
823
  - [GitHub Actions Workflows](examples/github-actions/)
713
824
  - [Custom Checks](examples/custom_checks/)
714
825
  - [Configuration Files](examples/configs/)
715
- - [Sample Policies](examples/policies/)
826
+ - [Test IAM Policies](examples/iam-test-policies/)
827
+ - **[AWS Services Backup Guide](docs/aws-services-backup.md)** - Offline validation
716
828
  - **[Contributing Guide](CONTRIBUTING.md)** - Contribution guidelines
717
829
  - **[Publishing Guide](docs/development/PUBLISHING.md)** - Release process
718
830
 
@@ -1,6 +1,6 @@
1
1
  iam_validator/__init__.py,sha256=APnMR3Fu4fHhxfsHBvUM2dJIwazgvLKQbfOsSgFPidg,693
2
2
  iam_validator/__main__.py,sha256=to_nz3n_IerJpVVZZ6WSFlFR5s_06J0csfPOTfQZG8g,197
3
- iam_validator/__version__.py,sha256=Gnq5VxEcBl7XBv8aQLOpUL1P1P6Gnsze4sf794FzMWs,206
3
+ iam_validator/__version__.py,sha256=BOzo0kDxoue17MkZOqACxqP9TwbfCJhkzZuMsC5TMac,206
4
4
  iam_validator/checks/__init__.py,sha256=eKTPgiZ1i3zvyP6OdKgLx9s3u69onITMYifmJPJwZgM,968
5
5
  iam_validator/checks/action_condition_enforcement.py,sha256=3M1Wj89Af6H-ywBTruZbJPzhCBBQVanVb5hwv-fkiDE,29721
6
6
  iam_validator/checks/action_resource_constraint.py,sha256=p-gP7S9QYR6M7vffrnJY6LOlMUTn0kpEbrxQ8pTY5rs,6031
@@ -8,37 +8,38 @@ iam_validator/checks/action_validation.py,sha256=IpxtTsk58f2zEZ-xzAoyHw4QK8BCRV4
8
8
  iam_validator/checks/condition_key_validation.py,sha256=bc4LQ8IRKyt0RquaQvQvVjmeJnuOUAFRL8xdduLPa_U,2661
9
9
  iam_validator/checks/policy_size.py,sha256=4cvZiWRJXGuvYo8PRcdD1Py_ZL8Xw0lOJfXTs6EX-_I,5753
10
10
  iam_validator/checks/resource_validation.py,sha256=AEIoiR6AKYLuVaA8ne3QE5qy6NCMDe98_2JAiwE9-JU,4261
11
- iam_validator/checks/security_best_practices.py,sha256=-gqxtcx_cUV1ZnyZ8Flydwan1vxb-RmnanWoIlU6YyY,21711
12
- iam_validator/checks/sid_uniqueness.py,sha256=7S8RtVJgYPTKgr7gSEmxgT0oIGkSoXN6iu0ALHbcSfw,5015
11
+ iam_validator/checks/security_best_practices.py,sha256=uf3ZAhBkyN8ka9bZHWi2kkAGIibhqWMIF06DBXsgu9U,23093
12
+ iam_validator/checks/sid_uniqueness.py,sha256=U2Kk5lYi9mHhhTpCWAD0ZQfxcLnIJJa7KGC5nOzTEbY,5145
13
13
  iam_validator/checks/utils/__init__.py,sha256=j0X4ibUB6RGx2a-kNoJnlVZwHfoEvzZsIeTmJIAoFzA,45
14
14
  iam_validator/checks/utils/policy_level_checks.py,sha256=2V60C0zhKfsFPjQ-NMlD3EemtwA9S6-4no8nETgXdQE,5274
15
15
  iam_validator/checks/utils/sensitive_action_matcher.py,sha256=VlTpgjMnympYa28kOdm6xRIUL2P87rOvm1O2NdnjtVI,8900
16
- iam_validator/checks/utils/wildcard_expansion.py,sha256=L6AWrLRacqXqk9k-5ZmXv5HyoAyz98cg5AlTvzH2tTI,3158
17
- iam_validator/commands/__init__.py,sha256=lF0fSUukLSxTAvhjg-0P79YMseYwihIr_tmQYbfNgcY,425
16
+ iam_validator/checks/utils/wildcard_expansion.py,sha256=V3V_KRpapOzPBhpUObJjGHoMhvCH90QvDxppeEHIG_U,3152
17
+ iam_validator/commands/__init__.py,sha256=M-5bo8w0TCWydK0cXgJyPD2fmk8bpQs-3b26YbgLzlc,565
18
18
  iam_validator/commands/analyze.py,sha256=TWlDaZ8gVOdNv6__KQQfzeLVW36qLiL5IzlhGYfvq_g,16501
19
19
  iam_validator/commands/base.py,sha256=5baCCMwxz7pdQ6XMpWfXFNz7i1l5dB8Qv9dKKR04Gzs,1074
20
- iam_validator/commands/cache.py,sha256=1E-irKF3e2CFUEG9s1z64hIKLVSYFQ-L92ld6L3za5g,14368
20
+ iam_validator/commands/cache.py,sha256=NHfbIDWI8tj-3o-4fIZJQS-Vvd9bxIH3Lk6kBtNuiUU,14212
21
+ iam_validator/commands/download_services.py,sha256=anRcobOuhkiEmHpwW_AJb1e2ifgkgYAO2-b9-JBrBcg,9152
21
22
  iam_validator/commands/post_to_pr.py,sha256=hl_K-XlELYN-ArjMdgQqysvIE-26yf9XdrMl4ToDwG0,2148
22
23
  iam_validator/commands/validate.py,sha256=R295cOTly8n7zL1jfvbh9RuCgiM5edBqbf6YMn_4G9A,14013
23
24
  iam_validator/core/__init__.py,sha256=1FvJPMrbzJfS9YbRUJCshJLd5gzWwR9Fd_slS0Aq9c8,416
24
25
  iam_validator/core/access_analyzer.py,sha256=poeT1i74jXpKr1B3UmvqiTvCTbq82zffWgZHwiFUwoo,24337
25
26
  iam_validator/core/access_analyzer_report.py,sha256=IrQVszlhFfQ6WykYLpig7TU3hf8dnQTegPDsOvHjR5Q,24873
26
- iam_validator/core/aws_fetcher.py,sha256=P7fYX1Q1TICuTOlGaqH97U3m8B0bqGE9jP7cxfmny8k,27418
27
+ iam_validator/core/aws_fetcher.py,sha256=6W4ixYEMx4Y5bx9rCB65CDqZh7iUVANAvhFVHu0MOKQ,32654
27
28
  iam_validator/core/aws_global_conditions.py,sha256=ADVcMEWhgvDZWdBmRUQN3HB7a9OycbTLecXFAy3LPbo,5837
28
29
  iam_validator/core/check_registry.py,sha256=wxqaF2t_3lWgT6x7_PnnZ8XGjHKUxUk72UlmdYBLFyo,15679
29
30
  iam_validator/core/cli.py,sha256=PkXiZjlgrQ21QustBbspefYsdbxst4gxoClyG2_HQR8,3843
30
31
  iam_validator/core/config_loader.py,sha256=Pq2rd6LJtEZET0ZeW4hEZS2ZRLC5gNRsKbtLyIsT21I,16516
31
- iam_validator/core/defaults.py,sha256=sPQJUMyjv4yalGCuyQhlY42rDc_-BZLq6qS0GgoP4mc,11893
32
+ iam_validator/core/defaults.py,sha256=tp8MPrFicRvI0dp8yH95MzJ9tC33n0N92aUC3HMkmYc,13289
32
33
  iam_validator/core/models.py,sha256=rWIZnD-I81Sg4asgOhnB10FWJC5mxQ2JO9bdS0sHb4Q,10772
33
- iam_validator/core/policy_checks.py,sha256=vIzRkqf5k1BB0elry5a4E2SRBlp6Vz3jPqrav29k3PM,24842
34
+ iam_validator/core/policy_checks.py,sha256=pMlZ2XkuqppVOUZq__e8w_yGoy7lIHjAB5RiTXwJo4Q,25114
34
35
  iam_validator/core/policy_loader.py,sha256=TR7SpzlRG3TwH4HBGEFUuhNOmxIR8Cud2SQ-AmHWBpM,14040
35
36
  iam_validator/core/pr_commenter.py,sha256=TOhVXKTFcRHQ9EVuShXQcKXn9aNjB1mU6FnR2jvltmw,10581
36
- iam_validator/core/report.py,sha256=wPkLvsIej-AaW5FlqntvUuHuEMvyi2iBI6NQF496gCM,33064
37
+ iam_validator/core/report.py,sha256=Yeh_u9jQvTyDV3ignyPcWEQVfFcxNZNrxf4T0fjeWb4,33283
37
38
  iam_validator/core/formatters/__init__.py,sha256=fnCKAEBXItnOf2m4rhVs7zwMaTxbG6ESh3CF8V5j5ec,868
38
39
  iam_validator/core/formatters/base.py,sha256=SShDeDiy5mYQnS6BpA8xYg91N-KX1EObkOtlrVHqx1Q,4451
39
40
  iam_validator/core/formatters/console.py,sha256=lX4Yp4bTW61fxe0fCiHuO6bCZtC_6cjCwqDNQ55nT_8,1937
40
41
  iam_validator/core/formatters/csv.py,sha256=2FaN6Y_0TPMFOb3A3tNtj0-9bkEc5P-6eZ7eLROIqFE,5899
41
- iam_validator/core/formatters/enhanced.py,sha256=k_DwzhGTARUKMv4bjkaCrpI6ypT10z9LcSk8gKlyDIM,16547
42
+ iam_validator/core/formatters/enhanced.py,sha256=-W9JACV4FNVWoWtfVxXLla4d__Gg96SASbNAijpJnT0,16638
42
43
  iam_validator/core/formatters/html.py,sha256=j4sQi-wXiD9kCHldW5JCzbJe0frhiP5uQI9KlH3Sj_g,22994
43
44
  iam_validator/core/formatters/json.py,sha256=A7gZ8P32GEdbDvrSn6v56yQ4fOP_kyMaoFVXG2bgnew,939
44
45
  iam_validator/core/formatters/markdown.py,sha256=aPAY6FpZBHsVBDag3FAsB_X9CZzznFjX9dQr0ysDrTE,2251
@@ -46,8 +47,8 @@ iam_validator/core/formatters/sarif.py,sha256=tqp8g7RmUh0HRk-kKDaucx4sa-5I9ikgkS
46
47
  iam_validator/integrations/__init__.py,sha256=7Hlor_X9j0NZaEjFuSvoXAAuSKQ-zgY19Rk-Dz3JpKo,616
47
48
  iam_validator/integrations/github_integration.py,sha256=bKs94vNT4PmcmUPUeuY2WJFhCYpUY2SWiBP1vj-andA,25673
48
49
  iam_validator/integrations/ms_teams.py,sha256=t2PlWuTDb6GGH-eDU1jnOKd8D1w4FCB68bahGA7MJcE,14475
49
- iam_policy_validator-1.1.2.dist-info/METADATA,sha256=mjrd4ODBydvBtCh_0Ztq0-FqQTNunBpEdIrDQ-5RXos,25144
50
- iam_policy_validator-1.1.2.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
51
- iam_policy_validator-1.1.2.dist-info/entry_points.txt,sha256=8HtWd8O7mvPiPdZR5YbzY8or_qcqLM4-pKaFdhtFT8M,62
52
- iam_policy_validator-1.1.2.dist-info/licenses/LICENSE,sha256=AMnbFTBDcK4_MITe2wiQBkj0vg-jjBBhsc43ydC7tt4,1098
53
- iam_policy_validator-1.1.2.dist-info/RECORD,,
50
+ iam_policy_validator-1.3.0.dist-info/METADATA,sha256=FOWdp3xcENWJmZJtZ8lQlg53Bd6-8v9RpdBLKVvEY3Q,29136
51
+ iam_policy_validator-1.3.0.dist-info/WHEEL,sha256=qtCwoSJWgHk21S1Kb4ihdzI2rlJ1ZKaIurTj_ngOhyQ,87
52
+ iam_policy_validator-1.3.0.dist-info/entry_points.txt,sha256=8HtWd8O7mvPiPdZR5YbzY8or_qcqLM4-pKaFdhtFT8M,62
53
+ iam_policy_validator-1.3.0.dist-info/licenses/LICENSE,sha256=AMnbFTBDcK4_MITe2wiQBkj0vg-jjBBhsc43ydC7tt4,1098
54
+ iam_policy_validator-1.3.0.dist-info/RECORD,,
@@ -3,5 +3,5 @@
3
3
  This file is the single source of truth for the package version.
4
4
  """
5
5
 
6
- __version__ = "1.1.2"
6
+ __version__ = "1.3.0"
7
7
  __version_info__ = tuple(int(part) for part in __version__.split("."))
@@ -7,10 +7,7 @@ from iam_validator.checks.utils.sensitive_action_matcher import (
7
7
  DEFAULT_SENSITIVE_ACTIONS,
8
8
  check_sensitive_actions,
9
9
  )
10
- from iam_validator.checks.utils.wildcard_expansion import (
11
- compile_wildcard_pattern,
12
- expand_wildcard_actions,
13
- )
10
+ from iam_validator.checks.utils.wildcard_expansion import expand_wildcard_actions
14
11
  from iam_validator.core.aws_fetcher import AWSServiceFetcher
15
12
  from iam_validator.core.check_registry import CheckConfig, PolicyCheck
16
13
  from iam_validator.core.models import Statement, ValidationIssue
@@ -86,22 +83,26 @@ class SecurityBestPracticesCheck(PolicyCheck):
86
83
  if self._is_sub_check_enabled(config, "wildcard_resource_check"):
87
84
  if "*" in resources:
88
85
  # Check if all actions are in the allowed_wildcards list
89
- # This allows Resource: "*" when only safe read-only wildcard actions are used
90
- allowed_wildcards = self._get_allowed_wildcards_for_resources(config)
86
+ # allowed_wildcards works by expanding wildcard patterns (like "ec2:Describe*")
87
+ # to all matching AWS actions using the AWS API, then checking if the policy's
88
+ # actions are in that expanded list. This ensures only validated AWS actions
89
+ # are allowed with Resource: "*".
90
+ allowed_wildcards_expanded = await self._get_expanded_allowed_wildcards(
91
+ config, fetcher
92
+ )
91
93
 
92
- # Check if ALL actions (excluding full wildcard "*") match allowed patterns
94
+ # Check if ALL actions (excluding full wildcard "*") are in the expanded list
93
95
  non_wildcard_actions = [a for a in actions if a != "*"]
94
96
 
95
- if allowed_wildcards and non_wildcard_actions:
96
- # Check if all actions are allowed wildcards
97
+ if allowed_wildcards_expanded and non_wildcard_actions:
98
+ # Check if all actions are in the expanded allowed list (exact match)
97
99
  all_actions_allowed = all(
98
- self._is_action_allowed_wildcard(action, allowed_wildcards)
99
- for action in non_wildcard_actions
100
+ action in allowed_wildcards_expanded for action in non_wildcard_actions
100
101
  )
101
102
 
102
- # If all actions are in the allowed list, skip the wildcard resource warning
103
+ # If all actions are in the expanded list, skip the wildcard resource warning
103
104
  if all_actions_allowed:
104
- # All actions are safe wildcards, Resource: "*" is acceptable
105
+ # All actions are safe, Resource: "*" is acceptable
105
106
  pass
106
107
  else:
107
108
  # Some actions are not in allowed list, flag the issue
@@ -443,45 +444,6 @@ class SecurityBestPracticesCheck(PolicyCheck):
443
444
 
444
445
  return set()
445
446
 
446
- def _is_action_allowed_wildcard(
447
- self, action: str, allowed_wildcards: frozenset[str] | list[str] | set[str]
448
- ) -> bool:
449
- """Check if an action matches the allowed_wildcards list.
450
-
451
- This method checks if a given action is in the allowed_wildcards configuration
452
- from action_validation_check. This is used to determine if wildcard resources
453
- are acceptable when only safe wildcard actions are used.
454
-
455
- Args:
456
- action: The action to check (e.g., "s3:List*", "ec2:DescribeInstances")
457
- allowed_wildcards: Set or list of allowed wildcard patterns
458
-
459
- Returns:
460
- True if the action matches any pattern in the allowlist
461
-
462
- Note:
463
- Exact matches use O(1) set lookup for performance.
464
- Pattern matches (wildcards in allowlist) require O(n) iteration.
465
- """
466
- # Fast O(1) exact match using set membership
467
- if action in allowed_wildcards:
468
- return True
469
-
470
- # Pattern match - check if action matches any pattern in allowlist
471
- # This is needed when allowlist contains wildcards like "s3:*"
472
- # Uses cached compiled patterns for 20-30x speedup
473
- for pattern in allowed_wildcards:
474
- # Skip exact matches (already checked above)
475
- if "*" not in pattern:
476
- continue
477
-
478
- # Use cached compiled pattern
479
- compiled = compile_wildcard_pattern(pattern)
480
- if compiled.match(action):
481
- return True
482
-
483
- return False
484
-
485
447
  def _get_allowed_wildcards_for_resources(self, config: CheckConfig) -> frozenset[str]:
486
448
  """Get allowed_wildcards for resource check configuration.
487
449
 
@@ -513,3 +475,61 @@ class SecurityBestPracticesCheck(PolicyCheck):
513
475
 
514
476
  # No configuration found, return empty set (flag all Resource: "*")
515
477
  return frozenset()
478
+
479
+ async def _get_expanded_allowed_wildcards(
480
+ self, config: CheckConfig, fetcher: AWSServiceFetcher
481
+ ) -> frozenset[str]:
482
+ """Get and expand allowed_wildcards configuration.
483
+
484
+ This method retrieves wildcard patterns from the allowed_wildcards config
485
+ and expands them using the AWS API to get all matching actual AWS actions.
486
+
487
+ How it works:
488
+ 1. Retrieves patterns from config (e.g., ["ec2:Describe*", "s3:List*"])
489
+ 2. Expands each pattern using AWS API:
490
+ - "ec2:Describe*" → ["ec2:DescribeInstances", "ec2:DescribeImages", ...]
491
+ - "s3:List*" → ["s3:ListBucket", "s3:ListObjects", ...]
492
+ 3. Returns a set of all expanded actions
493
+
494
+ This allows you to:
495
+ - Specify patterns like "ec2:Describe*" in config
496
+ - Have the validator allow specific actions like "ec2:DescribeInstances" with Resource: "*"
497
+ - Ensure only real AWS actions (validated via API) are allowed
498
+
499
+ Example:
500
+ Config: allowed_wildcards: ["ec2:Describe*"]
501
+ Expands to: ["ec2:DescribeInstances", "ec2:DescribeImages", ...]
502
+ Policy: "Action": ["ec2:DescribeInstances"], "Resource": "*"
503
+ Result: ✅ Allowed (ec2:DescribeInstances is in expanded list)
504
+
505
+ Args:
506
+ config: The check configuration
507
+ fetcher: AWS service fetcher for expanding wildcards via AWS API
508
+
509
+ Returns:
510
+ A frozenset of all expanded action names from the configured patterns
511
+ """
512
+ # Check wildcard_resource_check first for override
513
+ sub_check_config = config.config.get("wildcard_resource_check", {})
514
+ patterns_to_expand: list[str] = []
515
+
516
+ if isinstance(sub_check_config, dict) and "allowed_wildcards" in sub_check_config:
517
+ # Explicitly configured in wildcard_resource_check (override)
518
+ patterns = sub_check_config.get("allowed_wildcards", [])
519
+ if isinstance(patterns, list):
520
+ patterns_to_expand = patterns
521
+ else:
522
+ # Fall back to parent security_best_practices_check's allowed_wildcards
523
+ parent_patterns = config.config.get("allowed_wildcards", [])
524
+ if isinstance(parent_patterns, list):
525
+ patterns_to_expand = parent_patterns
526
+
527
+ # If no patterns configured, return empty set
528
+ if not patterns_to_expand:
529
+ return frozenset()
530
+
531
+ # Expand the wildcard patterns using the AWS API
532
+ # This converts patterns like "ec2:Describe*" to actual AWS actions
533
+ expanded_actions = await expand_wildcard_actions(patterns_to_expand, fetcher)
534
+
535
+ return frozenset(expanded_actions)
@@ -46,13 +46,15 @@ def _check_sid_uniqueness_impl(policy: IAMPolicy, severity: str) -> list[Validat
46
46
  # (the first occurrence is "original", subsequent ones are "duplicates")
47
47
  for idx in indices[1:]:
48
48
  statement = policy.statement[idx]
49
+ # Convert to 1-indexed statement numbers for user-facing message
50
+ statement_numbers = ", ".join(f"#{i + 1}" for i in indices)
49
51
  issues.append(
50
52
  ValidationIssue(
51
53
  severity=severity,
52
54
  statement_sid=duplicate_sid,
53
55
  statement_index=idx,
54
56
  issue_type="duplicate_sid",
55
- message=f"Statement ID '{duplicate_sid}' is used {count} times in this policy (found in statements {', '.join(f'[{i}]' for i in indices)})",
57
+ message=f"Statement ID '{duplicate_sid}' is used {count} times in this policy (found in statements {statement_numbers})",
56
58
  suggestion="Change this SID to a unique value. Statement IDs help identify and reference specific statements, so duplicates can cause confusion.",
57
59
  line_number=statement.line_number,
58
60
  )
@@ -30,9 +30,7 @@ def compile_wildcard_pattern(pattern: str) -> Pattern[str]:
30
30
  return re.compile(regex_pattern, re.IGNORECASE)
31
31
 
32
32
 
33
- async def expand_wildcard_actions(
34
- actions: list[str], fetcher: AWSServiceFetcher
35
- ) -> list[str]:
33
+ async def expand_wildcard_actions(actions: list[str], fetcher: AWSServiceFetcher) -> list[str]:
36
34
  """
37
35
  Expand wildcard actions to their actual action names using AWS API.
38
36
 
@@ -2,6 +2,7 @@
2
2
 
3
3
  from .analyze import AnalyzeCommand
4
4
  from .cache import CacheCommand
5
+ from .download_services import DownloadServicesCommand
5
6
  from .post_to_pr import PostToPRCommand
6
7
  from .validate import ValidateCommand
7
8
 
@@ -11,6 +12,14 @@ ALL_COMMANDS = [
11
12
  PostToPRCommand(),
12
13
  AnalyzeCommand(),
13
14
  CacheCommand(),
15
+ DownloadServicesCommand(),
14
16
  ]
15
17
 
16
- __all__ = ["ValidateCommand", "PostToPRCommand", "AnalyzeCommand", "CacheCommand", "ALL_COMMANDS"]
18
+ __all__ = [
19
+ "ValidateCommand",
20
+ "PostToPRCommand",
21
+ "AnalyzeCommand",
22
+ "CacheCommand",
23
+ "DownloadServicesCommand",
24
+ "ALL_COMMANDS",
25
+ ]
@@ -223,12 +223,7 @@ Examples:
223
223
  size = f.stat().st_size
224
224
  mtime = f.stat().st_mtime
225
225
 
226
- services.append({
227
- "name": name,
228
- "size": size,
229
- "file": f.name,
230
- "mtime": mtime
231
- })
226
+ services.append({"name": name, "size": size, "file": f.name, "mtime": mtime})
232
227
 
233
228
  # Sort by service name
234
229
  services.sort(key=lambda x: x["name"])
@@ -256,12 +251,7 @@ Examples:
256
251
  size_kb = svc["size"] / 1024
257
252
  cached_time = datetime.fromtimestamp(svc["mtime"]).strftime("%Y-%m-%d %H:%M")
258
253
 
259
- table.add_row(
260
- svc["name"],
261
- svc["file"],
262
- f"{size_kb:.1f} KB",
263
- cached_time
264
- )
254
+ table.add_row(svc["name"], svc["file"], f"{size_kb:.1f} KB", cached_time)
265
255
 
266
256
  console.print(table)
267
257
 
@@ -0,0 +1,260 @@
1
+ """Download AWS service definitions command."""
2
+
3
+ import argparse
4
+ import asyncio
5
+ import json
6
+ import logging
7
+ from datetime import datetime, timezone
8
+ from pathlib import Path
9
+
10
+ import httpx
11
+ from rich.console import Console
12
+ from rich.progress import (
13
+ BarColumn,
14
+ Progress,
15
+ TaskID,
16
+ TextColumn,
17
+ TimeRemainingColumn,
18
+ )
19
+
20
+ from iam_validator.commands.base import Command
21
+
22
+ logger = logging.getLogger(__name__)
23
+ console = Console()
24
+
25
+ BASE_URL = "https://servicereference.us-east-1.amazonaws.com/"
26
+ DEFAULT_OUTPUT_DIR = Path("aws_services")
27
+
28
+
29
+ class DownloadServicesCommand(Command):
30
+ """Download all AWS service definition JSON files."""
31
+
32
+ @property
33
+ def name(self) -> str:
34
+ return "sync-services"
35
+
36
+ @property
37
+ def help(self) -> str:
38
+ return "Sync/download all AWS service definitions for offline use"
39
+
40
+ @property
41
+ def epilog(self) -> str:
42
+ return """
43
+ Examples:
44
+ # Sync all AWS service definitions to default directory (aws_services/)
45
+ iam-validator sync-services
46
+
47
+ # Sync to a custom directory
48
+ iam-validator sync-services --output-dir /path/to/backup
49
+
50
+ # Limit concurrent downloads
51
+ iam-validator sync-services --max-concurrent 5
52
+
53
+ # Enable verbose output
54
+ iam-validator sync-services --log-level debug
55
+
56
+ Directory structure:
57
+ aws_services/
58
+ _manifest.json # Metadata about the download
59
+ _services.json # List of all services
60
+ s3.json # Individual service definitions
61
+ ec2.json
62
+ iam.json
63
+ ...
64
+
65
+ This command is useful for:
66
+ - Creating offline backups of AWS service definitions
67
+ - Avoiding API rate limiting during development
68
+ - Ensuring consistent service definitions across environments
69
+ """
70
+
71
+ def add_arguments(self, parser: argparse.ArgumentParser) -> None:
72
+ """Add sync-services command arguments."""
73
+ parser.add_argument(
74
+ "--output-dir",
75
+ type=Path,
76
+ default=DEFAULT_OUTPUT_DIR,
77
+ help=f"Output directory for downloaded files (default: {DEFAULT_OUTPUT_DIR})",
78
+ )
79
+
80
+ parser.add_argument(
81
+ "--max-concurrent",
82
+ type=int,
83
+ default=10,
84
+ help="Maximum number of concurrent downloads (default: 10)",
85
+ )
86
+
87
+ async def execute(self, args: argparse.Namespace) -> int:
88
+ """Execute the sync-services command."""
89
+ output_dir = args.output_dir
90
+ max_concurrent = args.max_concurrent
91
+
92
+ try:
93
+ await self._download_all_services(output_dir, max_concurrent)
94
+ return 0
95
+ except Exception as e:
96
+ console.print(f"[red]Error:[/red] {e}")
97
+ logger.error(f"Download failed: {e}", exc_info=True)
98
+ return 1
99
+
100
+ async def _download_services_list(self, client: httpx.AsyncClient) -> list[dict]:
101
+ """Download the list of all AWS services.
102
+
103
+ Args:
104
+ client: HTTP client for making requests
105
+
106
+ Returns:
107
+ List of service info dictionaries
108
+ """
109
+ console.print(f"[cyan]Fetching services list from {BASE_URL}...[/cyan]")
110
+
111
+ try:
112
+ response = await client.get(BASE_URL, timeout=30.0)
113
+ response.raise_for_status()
114
+ services = response.json()
115
+
116
+ console.print(f"[green]✓[/green] Found {len(services)} AWS services")
117
+ return services
118
+ except Exception as e:
119
+ logger.error(f"Failed to fetch services list: {e}")
120
+ raise
121
+
122
+ async def _download_service_detail(
123
+ self,
124
+ client: httpx.AsyncClient,
125
+ service_name: str,
126
+ service_url: str,
127
+ semaphore: asyncio.Semaphore,
128
+ progress: Progress,
129
+ task_id: TaskID,
130
+ ) -> tuple[str, dict | None]:
131
+ """Download detailed JSON for a single service.
132
+
133
+ Args:
134
+ client: HTTP client for making requests
135
+ service_name: Name of the service
136
+ service_url: URL to fetch service details
137
+ semaphore: Semaphore to limit concurrent requests
138
+ progress: Progress bar instance
139
+ task_id: Progress task ID
140
+
141
+ Returns:
142
+ Tuple of (service_name, service_data) or (service_name, None) if failed
143
+ """
144
+ async with semaphore:
145
+ try:
146
+ logger.debug(f"Downloading {service_name}...")
147
+ response = await client.get(service_url, timeout=30.0)
148
+ response.raise_for_status()
149
+ data = response.json()
150
+ logger.debug(f"✓ Downloaded {service_name}")
151
+ progress.update(task_id, advance=1)
152
+ return service_name, data
153
+ except Exception as e:
154
+ logger.error(f"✗ Failed to download {service_name}: {e}")
155
+ progress.update(task_id, advance=1)
156
+ return service_name, None
157
+
158
+ async def _download_all_services(self, output_dir: Path, max_concurrent: int = 10) -> None:
159
+ """Download all AWS service definitions.
160
+
161
+ Args:
162
+ output_dir: Directory to save the downloaded files
163
+ max_concurrent: Maximum number of concurrent downloads
164
+ """
165
+ # Create output directory
166
+ output_dir.mkdir(parents=True, exist_ok=True)
167
+ console.print(f"[cyan]Output directory:[/cyan] {output_dir.absolute()}\n")
168
+
169
+ # Create HTTP client with connection pooling
170
+ async with httpx.AsyncClient(
171
+ limits=httpx.Limits(max_connections=max_concurrent, max_keepalive_connections=5),
172
+ timeout=httpx.Timeout(30.0),
173
+ ) as client:
174
+ # Download services list
175
+ services = await self._download_services_list(client)
176
+
177
+ # Save services list (underscore prefix for easy discovery at top of directory)
178
+ services_file = output_dir / "_services.json"
179
+ with open(services_file, "w") as f:
180
+ json.dump(services, f, indent=2)
181
+ console.print(f"[green]✓[/green] Saved services list to {services_file}\n")
182
+
183
+ # Download all service details with rate limiting and progress bar
184
+ semaphore = asyncio.Semaphore(max_concurrent)
185
+ tasks = []
186
+
187
+ # Set up progress bar
188
+ with Progress(
189
+ TextColumn("[progress.description]{task.description}"),
190
+ BarColumn(),
191
+ TextColumn("[progress.percentage]{task.percentage:>3.0f}%"),
192
+ TextColumn("({task.completed}/{task.total})"),
193
+ TimeRemainingColumn(),
194
+ console=console,
195
+ ) as progress:
196
+ task_id = progress.add_task(
197
+ "[cyan]Downloading service definitions...", total=len(services)
198
+ )
199
+
200
+ for item in services:
201
+ service_name = item.get("service")
202
+ service_url = item.get("url")
203
+
204
+ if service_name and service_url:
205
+ task = self._download_service_detail(
206
+ client, service_name, service_url, semaphore, progress, task_id
207
+ )
208
+ tasks.append(task)
209
+
210
+ # Download all services concurrently
211
+ results = await asyncio.gather(*tasks)
212
+
213
+ # Save individual service files
214
+ successful = 0
215
+ failed = 0
216
+
217
+ console.print("\n[cyan]Saving service definitions...[/cyan]")
218
+
219
+ for service_name, data in results:
220
+ if data is not None:
221
+ # Normalize filename (lowercase, safe characters)
222
+ filename = f"{service_name.lower().replace(' ', '_')}.json"
223
+ service_file = output_dir / filename
224
+
225
+ with open(service_file, "w") as f:
226
+ json.dump(data, f, indent=2)
227
+
228
+ successful += 1
229
+ else:
230
+ failed += 1
231
+
232
+ # Create manifest with metadata
233
+ manifest = {
234
+ "download_date": datetime.now(timezone.utc).isoformat(),
235
+ "total_services": len(services),
236
+ "successful_downloads": successful,
237
+ "failed_downloads": failed,
238
+ "base_url": BASE_URL,
239
+ }
240
+
241
+ manifest_file = output_dir / "_manifest.json"
242
+ with open(manifest_file, "w") as f:
243
+ json.dump(manifest, f, indent=2)
244
+
245
+ # Print summary
246
+ console.print(f"\n{'=' * 60}")
247
+ console.print("[bold cyan]Download Summary:[/bold cyan]")
248
+ console.print(f" Total services: {len(services)}")
249
+ console.print(f" [green]Successful:[/green] {successful}")
250
+ if failed > 0:
251
+ console.print(f" [red]Failed:[/red] {failed}")
252
+ console.print(f" Output directory: {output_dir.absolute()}")
253
+ console.print(f" Manifest: {manifest_file}")
254
+ console.print(f"{'=' * 60}")
255
+
256
+ if failed > 0:
257
+ console.print(
258
+ "\n[yellow]Warning:[/yellow] Some services failed to download. "
259
+ "Check the logs for details."
260
+ )
@@ -182,19 +182,25 @@ class AWSServiceFetcher:
182
182
  keepalive_connections: int = 20,
183
183
  prefetch_common: bool = True,
184
184
  cache_dir: Path | str | None = None,
185
+ aws_services_dir: Path | str | None = None,
185
186
  ):
186
187
  """Initialize aws service fetcher.
187
188
 
188
189
  Args:
189
190
  timeout: Request timeout in seconds
190
- retries: Number of retry attempts
191
- enable_cache: Enable disk caching
192
- cache_ttl: Cache time to live in seconds (default: 7 days)
193
- memory_cache_size: Max items in memory cache
194
- connection_pool_size: Max connections in pool
191
+ retries: Number of retries for failed requests
192
+ enable_cache: Enable persistent disk caching
193
+ cache_ttl: Cache time-to-live in seconds
194
+ memory_cache_size: Size of in-memory LRU cache
195
+ connection_pool_size: HTTP connection pool size
195
196
  keepalive_connections: Number of keepalive connections
196
- prefetch_common: Pre-fetch common services on init
197
- cache_dir: Custom cache directory (defaults to platform-specific user cache dir)
197
+ prefetch_common: Prefetch common AWS services
198
+ cache_dir: Custom cache directory path
199
+ aws_services_dir: Directory containing pre-downloaded AWS service JSON files.
200
+ When set, the fetcher will load services from local files
201
+ instead of making API calls. Directory should contain:
202
+ - _services.json: List of all services
203
+ - {service}.json: Individual service files (e.g., s3.json)
198
204
  """
199
205
  self.timeout = timeout
200
206
  self.retries = retries
@@ -202,6 +208,14 @@ class AWSServiceFetcher:
202
208
  self.cache_ttl = cache_ttl
203
209
  self.prefetch_common = prefetch_common
204
210
 
211
+ # AWS services directory for offline mode
212
+ self.aws_services_dir: Path | None = None
213
+ if aws_services_dir:
214
+ self.aws_services_dir = Path(aws_services_dir)
215
+ if not self.aws_services_dir.exists():
216
+ raise ValueError(f"AWS services directory does not exist: {aws_services_dir}")
217
+ logger.info(f"Using local AWS services from: {self.aws_services_dir}")
218
+
205
219
  self._client: httpx.AsyncClient | None = None
206
220
  self._memory_cache = LRUCache(maxsize=memory_cache_size, ttl=cache_ttl)
207
221
  self._cache_dir = self._get_cache_directory(cache_dir)
@@ -470,8 +484,84 @@ class AWSServiceFetcher:
470
484
 
471
485
  raise last_exception or Exception(f"Failed to fetch {url} after {self.retries} attempts")
472
486
 
487
+ def _load_services_from_file(self) -> list[ServiceInfo]:
488
+ """Load services list from local _services.json file.
489
+
490
+ Returns:
491
+ List of ServiceInfo objects loaded from _services.json
492
+
493
+ Raises:
494
+ FileNotFoundError: If _services.json doesn't exist
495
+ ValueError: If _services.json is invalid
496
+ """
497
+ if not self.aws_services_dir:
498
+ raise ValueError("aws_services_dir is not set")
499
+
500
+ services_file = self.aws_services_dir / "_services.json"
501
+ if not services_file.exists():
502
+ raise FileNotFoundError(f"_services.json not found in {self.aws_services_dir}")
503
+
504
+ try:
505
+ with open(services_file) as f:
506
+ data = json.load(f)
507
+
508
+ if not isinstance(data, list):
509
+ raise ValueError("Expected list of services from _services.json")
510
+
511
+ services: list[ServiceInfo] = []
512
+ for item in data:
513
+ if isinstance(item, dict):
514
+ service = item.get("service")
515
+ url = item.get("url")
516
+ if service and url:
517
+ services.append(ServiceInfo(service=str(service), url=str(url)))
518
+
519
+ logger.info(f"Loaded {len(services)} services from local file: {services_file}")
520
+ return services
521
+
522
+ except json.JSONDecodeError as e:
523
+ raise ValueError(f"Invalid JSON in services.json: {e}")
524
+
525
+ def _load_service_from_file(self, service_name: str) -> ServiceDetail:
526
+ """Load service detail from local JSON file.
527
+
528
+ Args:
529
+ service_name: Name of the service (case-insensitive)
530
+
531
+ Returns:
532
+ ServiceDetail object loaded from {service}.json
533
+
534
+ Raises:
535
+ FileNotFoundError: If service JSON file doesn't exist
536
+ ValueError: If service JSON is invalid
537
+ """
538
+ if not self.aws_services_dir:
539
+ raise ValueError("aws_services_dir is not set")
540
+
541
+ # Normalize filename (lowercase, replace spaces with underscores)
542
+ filename = f"{service_name.lower().replace(' ', '_')}.json"
543
+ service_file = self.aws_services_dir / filename
544
+
545
+ if not service_file.exists():
546
+ raise FileNotFoundError(f"Service file not found: {service_file}")
547
+
548
+ try:
549
+ with open(service_file) as f:
550
+ data = json.load(f)
551
+
552
+ service_detail = ServiceDetail.model_validate(data)
553
+ logger.debug(f"Loaded service {service_name} from local file: {service_file}")
554
+ return service_detail
555
+
556
+ except json.JSONDecodeError as e:
557
+ raise ValueError(f"Invalid JSON in {service_file}: {e}")
558
+
473
559
  async def fetch_services(self) -> list[ServiceInfo]:
474
- """Fetch list of AWS services with caching."""
560
+ """Fetch list of AWS services with caching.
561
+
562
+ When aws_services_dir is set, loads from local services.json file.
563
+ Otherwise, fetches from AWS API.
564
+ """
475
565
  # Check if we have the parsed services list in cache
476
566
  services_cache_key = "parsed_services_list"
477
567
  cached_services = await self._memory_cache.get(services_cache_key)
@@ -479,7 +569,14 @@ class AWSServiceFetcher:
479
569
  logger.debug(f"Retrieved {len(cached_services)} services from parsed cache")
480
570
  return cached_services
481
571
 
482
- # Not in parsed cache, fetch the raw data
572
+ # Load from local file if aws_services_dir is set
573
+ if self.aws_services_dir:
574
+ services = self._load_services_from_file()
575
+ # Cache the loaded services
576
+ await self._memory_cache.set(services_cache_key, services)
577
+ return services
578
+
579
+ # Not in parsed cache, fetch the raw data from API
483
580
  data = await self._make_request_with_batching(self.BASE_URL)
484
581
 
485
582
  if not isinstance(data, list):
@@ -501,7 +598,11 @@ class AWSServiceFetcher:
501
598
  return services
502
599
 
503
600
  async def fetch_service_by_name(self, service_name: str) -> ServiceDetail:
504
- """Fetch service detail with optimized caching."""
601
+ """Fetch service detail with optimized caching.
602
+
603
+ When aws_services_dir is set, loads from local {service}.json file.
604
+ Otherwise, fetches from AWS API.
605
+ """
505
606
  # Normalize service name
506
607
  service_name_lower = service_name.lower()
507
608
 
@@ -512,12 +613,33 @@ class AWSServiceFetcher:
512
613
  logger.debug(f"Memory cache hit for service {service_name}")
513
614
  return cached_detail
514
615
 
515
- # Fetch service list and find URL
616
+ # Load from local file if aws_services_dir is set
617
+ if self.aws_services_dir:
618
+ try:
619
+ service_detail = self._load_service_from_file(service_name_lower)
620
+ # Cache the loaded service
621
+ await self._memory_cache.set(cache_key, service_detail)
622
+ return service_detail
623
+ except FileNotFoundError:
624
+ # Try to find the service in services.json to get proper name
625
+ services = await self.fetch_services()
626
+ for service in services:
627
+ if service.service.lower() == service_name_lower:
628
+ # Try with the exact service name from services.json
629
+ try:
630
+ service_detail = self._load_service_from_file(service.service)
631
+ await self._memory_cache.set(cache_key, service_detail)
632
+ return service_detail
633
+ except FileNotFoundError:
634
+ pass
635
+ raise ValueError(f"Service '{service_name}' not found in {self.aws_services_dir}")
636
+
637
+ # Fetch service list and find URL from API
516
638
  services = await self.fetch_services()
517
639
 
518
640
  for service in services:
519
641
  if service.service.lower() == service_name_lower:
520
- # Fetch service detail
642
+ # Fetch service detail from API
521
643
  data = await self._make_request_with_batching(service.url)
522
644
 
523
645
  # Validate and parse
@@ -36,6 +36,7 @@ DEFAULT_CONFIG = {
36
36
  "max_concurrent": 10,
37
37
  "enable_builtin_checks": True,
38
38
  "parallel_execution": True,
39
+ "aws_services_dir": None,
39
40
  "cache_enabled": True,
40
41
  "cache_ttl_hours": 168,
41
42
  "fail_on_severity": ["error", "critical"],
@@ -161,6 +162,15 @@ With specific values:
161
162
  "sensitive_action_check": {
162
163
  "enabled": True,
163
164
  "severity": "medium",
165
+ "message_single": "Sensitive action '{action}' should have conditions to limit when it can be used",
166
+ "message_multiple": "Sensitive actions '{actions}' should have conditions to limit when they can be used",
167
+ "suggestion": "Add IAM conditions to limit when this action can be used. Consider: ABAC (ResourceTag OR RequestTag must match PrincipalTag), IP restrictions (aws:SourceIp), MFA requirements (aws:MultiFactorAuthPresent), or time-based restrictions (aws:CurrentTime)",
168
+ "example": """"Condition": {
169
+ "StringEquals": {
170
+ "aws:ResourceTag/owner": "${aws:PrincipalTag/owner}"
171
+ }
172
+ }
173
+ """,
164
174
  "sensitive_actions": [
165
175
  "iam:AddClientIDToOpenIDConnectProvider",
166
176
  "iam:AttachRolePolicy",
@@ -234,7 +244,6 @@ With specific values:
234
244
  "organizations:LeaveOrganization",
235
245
  "organizations:RemoveAccountFromOrganization",
236
246
  ],
237
- "sensitive_action_patterns": ["^iam:Delete.*"],
238
247
  },
239
248
  },
240
249
  "action_condition_enforcement_check": {
@@ -244,7 +253,6 @@ With specific values:
244
253
  "action_condition_requirements": [
245
254
  {
246
255
  "actions": ["iam:PassRole"],
247
- "action_patterns": ["^iam:Pas?.*$"],
248
256
  "severity": "high",
249
257
  "required_conditions": [
250
258
  {
@@ -267,7 +275,6 @@ With specific values:
267
275
  },
268
276
  {
269
277
  "actions": [
270
- "iam:Create*",
271
278
  "iam:CreateRole",
272
279
  "iam:Put*Policy*",
273
280
  "iam:PutUserPolicy",
@@ -276,7 +283,6 @@ With specific values:
276
283
  "iam:AttachUserPolicy",
277
284
  "iam:AttachRolePolicy",
278
285
  ],
279
- "action_patterns": ["^iam:Create", "^iam:Put.*Policy", "^iam:Attach.*Policy"],
280
286
  "severity": "high",
281
287
  "required_conditions": [
282
288
  {
@@ -293,6 +299,32 @@ With specific values:
293
299
  },
294
300
  ],
295
301
  },
302
+ {
303
+ "actions": ["s3:PutObject", "s3:DeleteObject", "s3:CreateBucket"],
304
+ "severity": "medium",
305
+ "required_conditions": [
306
+ {
307
+ "condition_key": "aws:ResourceOrgId",
308
+ "description": "Require aws:ResourceOrgId condition for S3 write actions to enforce organization-level access control",
309
+ "example": """"Condition": {
310
+ "StringEquals": {
311
+ "aws:ResourceOrgId": "${aws:PrincipalOrgID}"
312
+ }
313
+ }
314
+ """,
315
+ },
316
+ ],
317
+ },
318
+ {
319
+ "action_patterns": ["^s3:.*"],
320
+ "required_conditions": [
321
+ {
322
+ "condition_key": "aws:SecureTransport",
323
+ "description": "Require HTTPS for all S3 operations",
324
+ "expected_value": True,
325
+ },
326
+ ],
327
+ },
296
328
  {
297
329
  "action_patterns": [
298
330
  "^ssm:StartSession$",
@@ -345,10 +345,11 @@ class EnhancedFormatter(OutputFormatter):
345
345
 
346
346
  def _add_issue_to_tree(self, branch: Tree, issue, color: str) -> None:
347
347
  """Add an issue to a tree branch."""
348
- # Build location string
349
- location = f"Statement {issue.statement_index}"
348
+ # Build location string (use 1-indexed statement numbers for user-facing output)
349
+ statement_num = issue.statement_index + 1
350
+ location = f"Statement {statement_num}"
350
351
  if issue.statement_sid:
351
- location = f"{issue.statement_sid} (#{issue.statement_index})"
352
+ location = f"{issue.statement_sid} (#{statement_num})"
352
353
  if issue.line_number is not None:
353
354
  location += f" @L{issue.line_number}"
354
355
 
@@ -481,10 +481,14 @@ async def validate_policies(
481
481
  cache_enabled = config.get_setting("cache_enabled", True)
482
482
  cache_ttl_hours = config.get_setting("cache_ttl_hours", 168)
483
483
  cache_directory = config.get_setting("cache_directory", None)
484
+ aws_services_dir = config.get_setting("aws_services_dir", None)
484
485
  cache_ttl_seconds = cache_ttl_hours * 3600
485
486
 
486
487
  async with AWSServiceFetcher(
487
- enable_cache=cache_enabled, cache_ttl=cache_ttl_seconds, cache_dir=cache_directory
488
+ enable_cache=cache_enabled,
489
+ cache_ttl=cache_ttl_seconds,
490
+ cache_dir=cache_directory,
491
+ aws_services_dir=aws_services_dir,
488
492
  ) as fetcher:
489
493
  validator = PolicyValidator(fetcher)
490
494
 
@@ -545,11 +549,15 @@ async def validate_policies(
545
549
  cache_enabled = config.get_setting("cache_enabled", True)
546
550
  cache_ttl_hours = config.get_setting("cache_ttl_hours", 168) # 7 days default
547
551
  cache_directory = config.get_setting("cache_directory", None)
552
+ aws_services_dir = config.get_setting("aws_services_dir", None)
548
553
  cache_ttl_seconds = cache_ttl_hours * 3600
549
554
 
550
555
  # Validate policies using registry
551
556
  async with AWSServiceFetcher(
552
- enable_cache=cache_enabled, cache_ttl=cache_ttl_seconds, cache_dir=cache_directory
557
+ enable_cache=cache_enabled,
558
+ cache_ttl=cache_ttl_seconds,
559
+ cache_dir=cache_directory,
560
+ aws_services_dir=aws_services_dir,
553
561
  ) as fetcher:
554
562
  tasks = [
555
563
  _validate_policy_with_registry(policy, file_path, registry, fetcher, fail_on_severities)
@@ -195,7 +195,9 @@ class ReportGenerator:
195
195
  "low": "[cyan]LOW[/cyan]",
196
196
  }.get(issue.severity, issue.severity.upper())
197
197
 
198
- location = f"Statement {issue.statement_index}"
198
+ # Use 1-indexed statement numbers for user-facing output
199
+ statement_num = issue.statement_index + 1
200
+ location = f"Statement {statement_num}"
199
201
  if issue.statement_sid:
200
202
  location += f" ({issue.statement_sid})"
201
203
  if issue.line_number is not None:
@@ -776,9 +778,11 @@ class ReportGenerator:
776
778
 
777
779
  def _format_issue_markdown(self, issue: ValidationIssue) -> str:
778
780
  """Format a single issue as markdown."""
779
- location = f"Statement {issue.statement_index}"
781
+ # Use 1-indexed statement numbers for user-facing output
782
+ statement_num = issue.statement_index + 1
783
+ location = f"Statement {statement_num}"
780
784
  if issue.statement_sid:
781
- location = f"`{issue.statement_sid}` (index {issue.statement_index})"
785
+ location = f"`{issue.statement_sid}` (statement #{statement_num})"
782
786
 
783
787
  parts = []
784
788