hedra 1.0.0 → 2.0.0

data/README.md

@@ -1,205 +1,248 @@
-# Hedra 🛡️
+# Hedra
 
-A comprehensive security header analyzer for modern web applications. Scan, audit, and monitor HTTP security headers with ease.
+[![Ruby](https://img.shields.io/badge/Ruby-3.0%2B-CC342D?style=flat&logo=ruby)](https://www.ruby-lang.org/)
+[![Gem Version](https://img.shields.io/gem/v/hedra?style=flat&logo=rubygems&color=E9573F)](https://rubygems.org/gems/hedra)
+[![License](https://img.shields.io/badge/License-MIT-00A98F?style=flat)](LICENSE)
+[![Downloads](https://img.shields.io/gem/dt/hedra?style=flat&color=blue)](https://rubygems.org/gems/hedra)
 
-```
- _   _          _           
-| | | | ___  __| |_ __ __ _ 
-| |_| |/ _ \/ _` | '__/ _` |
-|  _  |  __/ (_| | | | (_| |
-|_| |_|\___|\__,_|_|  \__,_|
-                             
-Security Header Analyzer
-```
-
-## Features
-
-- 🔍 **Comprehensive Scanning** - Analyze security headers for single or multiple URLs
-- 🎯 **Deep Auditing** - Detailed security header analysis with recommendations
-- 👁️ **Continuous Monitoring** - Watch URLs for header changes over time
-- 📊 **Multiple Output Formats** - Table, JSON, and CSV export options
-- 🔌 **Plugin Architecture** - Extend with custom header checks
-- ⚡ **Concurrent Scanning** - Fast parallel URL scanning with configurable concurrency
-- 🌐 **Proxy Support** - HTTP and SOCKS proxy compatibility
-- 🎨 **Beautiful CLI** - Color-coded output with severity badges
-- 📈 **Security Scoring** - 0-100 score based on header coverage
+> Security header analyzer with SSL/TLS validation, baseline tracking, and CI/CD integration.
 
 ## Installation
-
-### From Source
-
 ```bash
-# Clone the repository
-git clone https://github.com/hedra/hedra.git
-cd hedra
-
-# Install dependencies
-bundle install
-
-# Build the gem
-rake build
-
-# Install the gem
-gem install pkg/hedra-1.0.0.gem
+gem install hedra
 ```
 
-### Quick Start
-
+## Quick Start
 ```bash
-bundle install
-chmod +x bin/hedra
-bin/hedra --help
+hedra scan https://github.com
+hedra audit https://stripe.com --json
+hedra scan -f urls.txt --format html --output report.html
 ```
 
-## Usage
+## Commands
 
-### Basic Scanning
+### scan
 
-Scan a single URL:
+Scan URLs for security headers with flexible output options.
 ```bash
-hedra scan https://example.com
+hedra scan https://github.com
+hedra scan -f urls.txt --concurrency 20
+hedra scan https://stripe.com --cache --rate 10/s
 ```
 
-Scan multiple URLs from a file:
+**Key Options:**
+- `-f, --file FILE` • Read URLs from file
+- `-c, --concurrency N` • Concurrent requests (default: 10)
+- `-t, --timeout N` • Request timeout in seconds (default: 10)
+- `--rate RATE` • Rate limit: 10/s, 100/m, 1000/h
+- `--cache` • Enable response caching
+- `--cache-ttl N` • Cache TTL in seconds (default: 3600)
+- `-o, --output FILE` • Output file
+- `--format FORMAT` • table, json, csv, html (default: table)
+- `--proxy URL` • HTTP/SOCKS proxy
+- `--user-agent STRING` • Custom User-Agent
+- `--save-baseline NAME` • Save results as baseline
+- `--[no-]progress` • Show/hide progress bar
+- `--[no-]check-certificates` • SSL checks (default: enabled)
+- `--[no-]check-security-txt` • RFC 9116 checks
+
+### audit
+
+Deep security audit with detailed recommendations.
 ```bash
-hedra scan -f urls.txt
+hedra audit https://github.com
+hedra audit https://api.stripe.com --json --output report.json
 ```
 
-### Deep Audit
+**Options:**
+- `--json` • JSON output format
+- `-o, --output FILE` • Output file
+- `--proxy URL` • HTTP/SOCKS proxy
+- `--user-agent STRING` • Custom User-Agent
+- `-t, --timeout N` • Request timeout
+- `--[no-]check-certificates` • SSL/TLS validation
+- `--[no-]check-security-txt` • security.txt checks
 
-Perform detailed security analysis:
-```bash
-hedra audit https://example.com
-```
+### watch
 
-Export audit results as JSON:
+Monitor security headers periodically.
 ```bash
-hedra audit https://example.com --json --output result.json
+hedra watch https://myapp.com --interval 3600
 ```
 
-### Advanced Scanning
+**Options:**
+- `--interval N` • Check interval in seconds (default: 3600)
 
-Concurrent scanning with custom settings:
-```bash
-hedra scan -f urls.txt --concurrency 20 --timeout 15
-```
+### compare
 
-Scan through a proxy:
+Compare security headers between environments.
 ```bash
-hedra scan https://example.com --proxy http://127.0.0.1:8080
+hedra compare https://staging.myapp.com https://myapp.com
 ```
 
-Custom User-Agent and follow redirects:
+### ci_check
+
+CI/CD-friendly check with exit codes and thresholds.
 ```bash
-hedra scan https://example.com --user-agent "MyBot/1.0" --follow-redirects
+hedra ci_check https://myapp.com --threshold 85
+hedra ci_check -f urls.txt --fail-on-critical
 ```
 
-### Continuous Monitoring
+**Options:**
+- `-f, --file FILE` • Read URLs from file
+- `--threshold N` • Minimum score threshold (default: 80)
+- `--fail-on-critical` • Fail on critical issues (default: true)
 
-Watch a URL and check every hour:
-```bash
-hedra watch https://example.com --interval 3600
-```
+**Exit Codes:**
+- `0` • All checks passed
+- `1` • Score below threshold or critical issues found
 
-### Compare Headers
+### baseline
 
-Compare security headers between two URLs:
+Track security posture changes over time.
 ```bash
-hedra compare https://staging.example.com https://prod.example.com
+hedra baseline list
+hedra baseline compare production-v1 -f urls.txt
+hedra baseline delete production-v1
 ```
 
-### Export Results
+### cache
 
-Export scan results:
+Manage response cache for faster repeated scans.
 ```bash
-hedra scan -f urls.txt --output results.csv --format csv
+hedra cache clear
+hedra cache clear-expired
 ```
 
-### Plugin Management
+### plugin
 
-List installed plugins:
+Extend functionality with custom security checks.
 ```bash
 hedra plugin list
-```
-
-Install a custom plugin:
-```bash
 hedra plugin install path/to/plugin.rb
+hedra plugin remove plugin_name
 ```
 
-Remove a plugin:
-```bash
-hedra plugin remove my_plugin
-```
+## Security Checks
 
-## Security Headers Checked
+### HTTP Headers Analyzed
 
-Hedra analyzes the following security headers:
+| Header | Weight | Purpose |
+|--------|--------|---------|
+| Content-Security-Policy | 25 pts | Prevent XSS and injection attacks |
+| Strict-Transport-Security | 25 pts | Enforce HTTPS connections |
+| X-Frame-Options | 15 pts | Prevent clickjacking |
+| X-Content-Type-Options | 10 pts | Stop MIME-type sniffing |
+| Referrer-Policy | 10 pts | Control referrer information |
+| Permissions-Policy | 5 pts | Manage browser features |
+| Cross-Origin-Opener-Policy | 5 pts | Isolate browsing context |
+| Cross-Origin-Embedder-Policy | 3 pts | Enable cross-origin isolation |
+| Cross-Origin-Resource-Policy | 2 pts | Control resource loading |
 
-### Critical Headers
-- **Content-Security-Policy (CSP)** - Prevents XSS and injection attacks
-- **Strict-Transport-Security (HSTS)** - Enforces HTTPS connections
+### Additional Validations
 
-### Important Headers
-- **X-Frame-Options** - Prevents clickjacking attacks
-- **X-Content-Type-Options** - Prevents MIME-sniffing attacks
+**SSL/TLS Checks:**
+- Certificate expiry dates
+- Signature algorithm strength
+- Key size validation
+- Chain verification
 
-### Recommended Headers
-- **Referrer-Policy** - Controls referrer information
-- **Permissions-Policy** - Controls browser features
-- **Cross-Origin-Opener-Policy (COOP)** - Isolates browsing context
-- **Cross-Origin-Embedder-Policy (COEP)** - Controls resource embedding
-- **Cross-Origin-Resource-Policy (CORP)** - Controls resource sharing
+**RFC 9116:**
+- security.txt file presence and format
 
-## Configuration
+### Scoring System
+
+**Base:** 100 points from header weights
 
-Create a config file at `~/.hedra/config.yml`:
+**Penalties:**
+- Critical issue: -20 points
+- Warning: -10 points
+- Info: -5 points
 
+## Configuration
+
+Create `~/.hedra/config.yml`:
 ```yaml
+# HTTP settings
 timeout: 10
 concurrency: 10
-follow_redirects: false
-user_agent: "Hedra/1.0.0"
-output_format: table
+user_agent: "Hedra/2.0.0"
+follow_redirects: true
+max_retries: 3
+
+# Performance
+cache_enabled: false
+cache_ttl: 3600
+rate_limit: "10/s"
+
+# Security checks
+check_certificates: true
+check_security_txt: false
+
+# Output
+output_format: "table"
+progress_bar: true
+
+# Circuit breaker
+circuit_breaker_threshold: 5
+circuit_breaker_timeout: 60
 ```
 
-### Custom Rules
-
-Add custom header checks in `~/.hedra/rules.yml`:
+## Custom Rules
 
+Define organization-specific policies in `~/.hedra/rules.yml`:
 ```yaml
 rules:
   - header: "X-Custom-Security"
     type: missing
     severity: warning
     message: "Custom security header is missing"
-    fix: "Add X-Custom-Security header"
-  
+    fix: "Add X-Custom-Security: enabled"
+    
   - header: "Server"
     type: pattern
-    pattern: "(Apache|nginx|IIS)"
+    pattern: "^(Apache|nginx)"
     severity: info
-    message: "Server header exposes server software"
+    message: "Server header exposes software version"
     fix: "Remove or obfuscate Server header"
 ```
 
-## Plugin Development
+**Rule Types:**
+- `missing` • Header should be present
+- `pattern` • Header value must match regex
+
+**Severity Levels:**
+- `critical` • -20 points, immediate action required
+- `warning` • -10 points, should be addressed
+- `info` • -5 points, best practice
 
-Create custom plugins to extend Hedra's functionality:
+## Plugin System
 
+Create custom checks in `~/.hedra/plugins/`:
 ```ruby
-# ~/.hedra/plugins/my_plugin.rb
+# ~/.hedra/plugins/corporate_policy.rb
 module Hedra
-  class MyPlugin < Plugin
+  class CorporatePolicyPlugin < Plugin
     def self.check(headers)
       findings = []
       
-      unless headers.key?('x-my-header')
+      # Enforce corporate header
+      unless headers.key?('x-corp-security')
+        findings << {
+          header: 'x-corp-security',
+          issue: 'Corporate security header missing',
+          severity: :critical,
+          recommended_fix: 'Add X-Corp-Security: v2'
+        }
+      end
+      
+      # Check version disclosure
+      if headers['server']&.match?(/\d+\.\d+/)
         findings << {
-          header: 'x-my-header',
-          issue: 'My custom header is missing',
+          header: 'server',
+          issue: 'Server version exposed',
           severity: :warning,
-          recommended_fix: 'Add X-My-Header'
+          recommended_fix: 'Remove version from Server header'
         }
       end
       
@@ -209,134 +252,252 @@ module Hedra
 end
 ```
 
-## Output Examples
-
-### Table Output
-```
-https://example.com
-Score: 75/100
-Timestamp: 2025-11-12T10:30:00Z
-
-┌─────────────────────────────┬──────────────────────────────┬──────────┐
-│ Header                      │ Issue                        │ Severity │
-├─────────────────────────────┼──────────────────────────────┼──────────┤
-│ permissions-policy          │ Header is missing            │ ● INFO   │
-│ cross-origin-opener-policy  │ Header is missing            │ ● INFO   │
-└─────────────────────────────┴──────────────────────────────┴──────────┘
-```
-
-### JSON Output
-```json
-{
-  "url": "https://example.com",
-  "timestamp": "2025-11-12T10:30:00Z",
-  "headers": {
-    "content-security-policy": "default-src 'self'",
-    "strict-transport-security": "max-age=31536000"
-  },
-  "findings": [
-    {
-      "header": "x-frame-options",
-      "issue": "X-Frame-Options header is missing",
-      "severity": "warning",
-      "recommended_fix": "Add X-Frame-Options: DENY or SAMEORIGIN"
+**Management:**
+```bash
+hedra plugin install ~/.hedra/plugins/corporate_policy.rb
+hedra plugin list
+hedra plugin remove corporate_policy
+```
+
+## CI/CD Integration
+
+### GitHub Actions
+```yaml
+name: Security Headers Check
+
+on: [push, pull_request]
+
+jobs:
+  security-scan:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v3
+      
+      - name: Setup Ruby
+        uses: ruby/setup-ruby@v1
+        with:
+          ruby-version: '3.2'
+      
+      - name: Install Hedra
+        run: gem install hedra
+      
+      - name: Run Security Check
+        run: hedra ci_check ${{ secrets.APP_URL }} --threshold 85
+      
+      - name: Generate HTML Report
+        if: always()
+        run: hedra scan ${{ secrets.APP_URL }} --output report.html --format html
+      
+      - name: Upload Report
+        if: always()
+        uses: actions/upload-artifact@v3
+        with:
+          name: security-report
+          path: report.html
+```
+
+### GitLab CI
+```yaml
+security_headers:
+  image: ruby:3.2
+  script:
+    - gem install hedra
+    - hedra ci_check $APP_URL --threshold 85
+    - hedra scan $APP_URL --output report.json --format json
+  artifacts:
+    reports:
+      junit: report.json
+    paths:
+      - report.json
+  only:
+    - merge_requests
+    - main
+```
+
+### Jenkins Pipeline
+```groovy
+pipeline {
+    agent any
+    
+    stages {
+        stage('Security Headers') {
+            steps {
+                sh 'gem install hedra'
+                sh 'hedra ci_check ${APP_URL} --threshold 85'
+            }
+        }
+    }
+    
+    post {
+        always {
+            sh 'hedra scan ${APP_URL} --output report.html --format html'
+            publishHTML([
+                reportDir: '.',
+                reportFiles: 'report.html',
+                reportName: 'Security Report'
+            ])
+        }
     }
-  ],
-  "score": 75
 }
 ```
 
-## Development
+## Export Formats
+
+### Table (Default)
+```bash
+hedra scan https://github.com
+```
 
-### Running Tests
+Clean, colored terminal output with scores and recommendations.
 
+### JSON
 ```bash
-# Run all tests
-bundle exec rspec
+hedra scan https://stripe.com --output report.json --format json
+```
 
-# Run with coverage
-bundle exec rspec --format documentation
+Structured data for automation and parsing.
 
-# Run specific test file
-bundle exec rspec spec/hedra/analyzer_spec.rb
+### CSV
+```bash
+hedra scan -f urls.txt --output report.csv --format csv
 ```
 
-### Linting
+Import into spreadsheets for analysis and tracking.
 
+### HTML
 ```bash
-# Run RuboCop
-bundle exec rubocop
+hedra scan -f urls.txt --output report.html --format html
+```
+
+Interactive report with sorting, filtering, and charts.
+
+## Real-World Examples
 
-# Auto-fix issues
-bundle exec rubocop -a
+### Basic Security Audit
+```bash
+hedra scan https://myapp.com
 ```
 
-### Building
+### Production Deployment Check
+```bash
+# Save baseline after deployment
+hedra scan -f production-urls.txt --save-baseline prod-v2.1.0
 
+# Compare before next deployment
+hedra baseline compare prod-v2.1.0 -f production-urls.txt
+```
+
+### High-Volume Scanning
 ```bash
-# Build gem
-rake build
+# Scan 1000 URLs with rate limiting and caching
+hedra scan -f large-list.txt \
+  --concurrency 50 \
+  --rate 20/s \
+  --cache \
+  --output results.json \
+  --format json
+```
 
-# Install locally
-gem install pkg/hedra-1.0.0.gem
+### Continuous Monitoring
+```bash
+# Check every hour
+hedra watch https://api.myapp.com --interval 3600
 ```
 
-## CI/CD
+### Environment Comparison
+```bash
+hedra compare https://staging.myapp.com https://myapp.com
+```
 
-Hedra includes GitHub Actions CI configuration that:
-- Runs tests on Ruby 3.0, 3.1, and 3.2
-- Executes RuboCop linting
-- Builds the gem package
+### Proxy-Based Testing
+```bash
+# Route through Burp Suite
+hedra scan https://target.com --proxy http://127.0.0.1:8080
+```
 
-## Architecture
+### Custom User-Agent
+```bash
+hedra scan https://myapp.com --user-agent "Mozilla/5.0 (iPhone; CPU iPhone OS 14_0)"
+```
 
-### Core Components
+## Performance Tuning
 
-- **CLI** - Thor-based command-line interface with subcommands
-- **Analyzer** - Core logic for header analysis and validation
-- **HttpClient** - HTTP wrapper with retry logic, proxy support, and TLS verification
-- **Scorer** - Calculates security scores based on header coverage
-- **PluginManager** - Discovers and executes custom plugins
-- **Exporter** - Handles JSON and CSV output formats
+### Caching Strategy
+```bash
+# Enable caching for repeated scans
+hedra scan -f urls.txt --cache --cache-ttl 7200
 
-### Design Decisions
+# Clear cache when needed
+hedra cache clear
+```
 
-1. **Modular Architecture** - Each header check is isolated, making it easy to add new checks
-2. **Secure Defaults** - TLS verification on, no redirect following, conservative timeouts
-3. **Thread-Safe Concurrency** - Uses Ruby's concurrent-ruby gem for safe parallel scanning
-4. **Extensible Plugin System** - Simple base class for custom header checks
-5. **Comprehensive Testing** - WebMock stubs prevent live network calls in tests
+### Rate Limiting
+```bash
+# Conservative approach
+hedra scan -f urls.txt --rate 10/s --concurrency 5
 
-## Contributing
+# Aggressive scanning
+hedra scan -f urls.txt --rate 100/s --concurrency 50
+```
 
-1. Fork the repository
-2. Create your feature branch (`git checkout -b feature/amazing-feature`)
-3. Write tests for your changes
-4. Ensure tests pass (`bundle exec rspec`)
-5. Ensure linting passes (`bundle exec rubocop`)
-6. Commit your changes (`git commit -am 'Add amazing feature'`)
-7. Push to the branch (`git push origin feature/amazing-feature`)
-8. Open a Pull Request
+### Timeout Configuration
+```bash
+# Fast scan for responsive servers
+hedra scan -f urls.txt --timeout 5
 
-## License
+# Patient scan for slow servers
+hedra scan -f urls.txt --timeout 30
+```
+
+## Development
+```bash
+# Clone and setup
+git clone https://github.com/blackstack/hedra.git
+cd hedra
+bundle install
+
+# Run tests
+bundle exec rspec
+
+# Check code style
+bundle exec rubocop
 
-MIT License - see [LICENSE](LICENSE) file for details.
+# Build gem
+rake build
+gem install pkg/hedra-*.gem
+```
+
+## Troubleshooting
+
+### SSL Certificate Errors
+```bash
+# Skip certificate validation
+hedra scan https://self-signed.badssl.com --no-check-certificates
+```
 
-## Support
+### Rate Limiting Issues
+```bash
+# Reduce load on target server
+hedra scan -f urls.txt --concurrency 1 --rate 1/s
+```
 
-- 📖 Documentation: [GitHub Wiki](https://github.com/hedra/hedra/wiki)
-- 🐛 Issues: [GitHub Issues](https://github.com/hedra/hedra/issues)
-- 💬 Discussions: [GitHub Discussions](https://github.com/hedra/hedra/discussions)
+### Timeout Problems
+```bash
+# Increase timeout for slow servers
+hedra scan https://slow-server.com --timeout 60
+```
 
-## Acknowledgments
+## Resources
+
+**GitHub:** https://github.com/blackstack/hedra  
+**RubyGems:** https://rubygems.org/gems/hedra  
+**Issues:** https://github.com/blackstack/hedra/issues  
+**OWASP Headers:** https://owasp.org/www-project-secure-headers/
+
+## License
 
-Built with:
-- [Thor](https://github.com/rails/thor) - CLI framework
-- [HTTP.rb](https://github.com/httprb/http) - HTTP client
-- [TTY::Table](https://github.com/piotrmurach/tty-table) - Terminal tables
-- [Pastel](https://github.com/piotrmurach/pastel) - Terminal colors
-- [RSpec](https://rspec.info/) - Testing framework
+MIT License - see [LICENSE](LICENSE) for details.
 
 ---
 
-Made with ❤️ by the Hedra Team
+**Built by [BlackStack](https://github.com/blackstack)** • Securing the web, one header at a time.