heimdall_tools 1.3.48 → 1.3.49

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: aa900ce8ff5cabccb6e138bb636a0c56972f1894431c9ce67f95bcb0811b1c91
-  data.tar.gz: 830006eea9df8dfe413e8d0e472d9b38794f41f92a3ec5c468b882b01d54b39b
+  metadata.gz: f6386ac453df3c036d6fb57fd6993c338d3432c6c65dda92d06df94c808d4394
+  data.tar.gz: 1f215414fb5063abee81d84a760c1208a211dac9478dc8a96b95eb9f00d75b8f
 SHA512:
-  metadata.gz: b304f53e1ffd55c28734ca094c2d4ccedb23123ca144bad0d06c80f5a29e5b6f143b48cce2dedec0467ae344c8bbc55bd93cf6ca53dae6d0d7486019e73dcbe1
-  data.tar.gz: 7096c8751c387912720ba51fe18f69c1a8568d98a4e9a1d7962a5693d7469049eb71d24878dc7a75d8af96de70211324a173b9f3eb87c6104b0e5f6df647cdcd
+  metadata.gz: 6a1644af8db70b6de1853899547037c651886a1e8fed8d611826b8a367e5e26b60620bd2edc0d93735c4eb5da379a456f38d65be11bcc9b13e9977545f8566f5
+  data.tar.gz: 89b993467f7bf734dc5624218937090d3d093bbcdb979e80ecfe239b131a520abab7785a03204e4b707d613659ba82cee7e7894fdacef826599adebd440ce922

data/README.md

@@ -5,20 +5,22 @@
 
 HeimdallTools supplies several methods to convert output from various tools to "Heimdall Data Format"(HDF) format to be viewable in Heimdall. The current converters are:
 
-1.  [**aws_config_mapper**](#aws_config_mapper) - assess, audit, and evaluate AWS resources
-1.  [**burpsuite_mapper**](#burpsuite_mapper) - commercial dynamic analysis tool
-1.  [**dbprotect_mapper**](#dbprotect_mapper) - database vulnerability scanner
-1.  [**fortify_mapper**](#fortify_mapper) - commercial static code analysis tool
-1.  [**jfrog_xray_mapper**](#jfrog_xray_mapper) - package vulnerability scanner
-1.  [**nessus_mapper**](#nessus_mapper) - commercial security scanner (supports compliance and vulnerability scans from Tenable.sc and Tenable.io)
-1.  [**netsparker_mapper**](#netsparker_mapper) - web application security scanner
-1.  [**nikto_mapper**](#nikto_mapper) - open-source web server scanner
-1.  [**sarif_mapper**](#sarif_mapper) - static analysis results interchange format
+1. [**asff_mapper**](#asff_mapper) - custom findings format for AWS Security Hub
+1. [**aws_config_mapper**](#aws_config_mapper) - assess, audit, and evaluate AWS resources
+1. [**burpsuite_mapper**](#burpsuite_mapper) - commercial dynamic analysis tool
+1. [**dbprotect_mapper**](#dbprotect_mapper) - database vulnerability scanner
+1. [**fortify_mapper**](#fortify_mapper) - commercial static code analysis tool
+1. [**jfrog_xray_mapper**](#jfrog_xray_mapper) - package vulnerability scanner
+1. [**nessus_mapper**](#nessus_mapper) - commercial security scanner (supports compliance and vulnerability scans from Tenable.sc and Tenable.io)
+1. [**netsparker_mapper**](#netsparker_mapper) - web application security scanner
+1. [**nikto_mapper**](#nikto_mapper) - open-source web server scanner
+1. [**prowler_mapper**](#prowler_mapper) - assess, audit, harden, and facilitate incidence response for AWS resources
+1. [**sarif_mapper**](#sarif_mapper) - static analysis results interchange format
 1. [**scoutsuite_mapper**](#scoutsuite_mapper) - multi-cloud security auditing tool
 1. [**snyk_mapper**](#snyk_mapper) - commercial package vulnerability scanner
 1. [**sonarqube_mapper**](#sonarqube_mapper) - open-source static code analysis tool
 1. [**xccdf_results_mapper**](#xccdf_results_mapper) - extensible configuration checklist description results format
-1. [*scc_mapper](#xccdf_results_mapper) - scap compliance checker format
+1. [**scc_mapper**](#xccdf_results_mapper) - scap compliance checker format
 1. [**zap_mapper**](#zap_mapper) - OWASP ZAP - open-source dynamic code analysis tool
 
 ## Want to recommend a mapper for another tool? Please use these steps:
@@ -84,6 +86,27 @@ For Docker usage, replace the `heimdall_tools` command with the correct Docker c
 
 Note that all of the above Docker commands will mount your current directory on the Docker container. Ensure that you have navigated to the directory you intend to convert files in before executing the command.
 
+## asff_mapper
+
+asff_mapper translates AWS Security Finding Format results from JSON to HDF-formatted JSON so as to be viewable on Heimdall
+
+Note: The following commands are examples to extract data via the AWS CLI that need to be fed to the mapper:
+
+Output|Use|Command
+---|---|---
+ASFF json|All the findings that will be fed into the mapper|aws securityhub get-findings > asff.json
+AWS SecurityHub enabled standards json|Get all the enabled standards so you can get their identifiers|aws securityhub get-enabled-standards > asff_standards.json
+AWS SecurityHub standard controls json|Get all the controls for a standard that will be fed into the mapper|aws securityhub describe-standards-controls --standards-subscription-arn "arn:aws:securityhub:us-east-1:123456789123:subscription/cis-aws-foundations-benchmark/v/1.2.0" > asff_cis_standard.json
+
+    USAGE: heimdall_tools asff_mapper -i <asff-finding-json> [--sh <standard-1-json> ... <standard-n-json>] -o <hdf-scan-results-json>
+
+    FLAGS:
+        -i --input -j --json <asff-finding-json>                   : path to ASFF findings file.
+        --sh --securityhub-standards --input-securityhub-standards : array of paths to AWS SecurityHub standard files.
+        -o --output <hdf-scan-results-json>                        : path to output scan-results json.
+
+    example: heimdall_tools asff_mapper -i asff_findings.json --sh aws_standard.json cis_standard.json -o asff_hdf.json
+
 ## aws_config_mapper
 
 aws_config_mapper pulls Ruby AWS SDK data to translate AWS Config Rule results into HDF format json to be viewable in Heimdall
@@ -99,8 +122,8 @@ aws_config_mapper pulls Ruby AWS SDK data to translate AWS Config Rule results i
     USAGE: heimdall_tools aws_config_mapper [OPTIONS] -o
 
     FLAGS:
-        -o --output        : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -o --output  : path to output scan-results json.
+        -V --verbose : verbose run [optional].
 
     example: heimdall_tools aws_config_mapper -o aws_config_results_hdf.json
 
@@ -111,9 +134,9 @@ burpsuite_mapper translates an BurpSuite Pro exported XML results file into HDF
     USAGE: heimdall_tools burpsuite_mapper [OPTIONS] -x  -o
 
     FLAGS:
-        -x                : path to BurpSuitePro exported XML results file.
-        -o --output        : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -x           : path to BurpSuitePro exported XML results file.
+        -o --output  : path to output scan-results json.
+        -V --verbose : verbose run [optional].
 
     example: heimdall_tools burpsuite_mapper -x burpsuite_results.xml -o scan_results.json
 
@@ -124,9 +147,9 @@ dbprotect_mapper translates DBProtect report in `Check Results Details` format X
     USAGE: heimdall_tools dbprotect_mapper [OPTIONS] -x  -o
 
     FLAGS:
-        -x            : path to DBProtect report XML file.
-        -o --output        : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -x           : path to DBProtect report XML file.
+        -o --output  : path to output scan-results json.
+        -V --verbose : verbose run [optional].
 
     example: heimdall_tools dbprotect_mapper -x check_results_details_report.xml -o db_protect_hdf.json
 
@@ -137,9 +160,9 @@ fortify_mapper translates an Fortify results FVDL file into HDF format json to b
     USAGE: heimdall_tools fortify_mapper [OPTIONS] -f  -o
 
     FLAGS:
-    	-f --fvdl          : path to Fortify Scan FVDL file.
-    	-o --output        : path to output scan-results json.
-    	-V --verbose                     : verbose run [optional].
+    	-f --fvdl    : path to Fortify Scan FVDL file.
+    	-o --output  : path to output scan-results json.
+    	-V --verbose : verbose run [optional].
 
     example: heimdall_tools fortify_mapper -f audit.fvdl -o scan_results.json
 
@@ -150,9 +173,9 @@ jfrog_xray_mapper translates an JFrog Xray results JSON file into HDF format JSO
     USAGE: heimdall_tools jfrog_xray_mapper [OPTIONS] -j  -o
 
     FLAGS:
-        -j            : path to xray results JSON file.
-        -o --output        : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -j           : path to xray results JSON file.
+        -o --output  : path to output scan-results json.
+        -V --verbose : verbose run [optional].
 
     example: heimdall_tools jfrog_xray_mapper -j xray_results.json -o xray_results_hdf.json
 
@@ -166,9 +189,9 @@ Note: A separate HDF JSON file is generated for each host reported in the Nessus
     USAGE: heimdall_tools nessus_mapper [OPTIONS] -x  -o
 
     FLAGS:
-        -x           : path to Nessus-exported XML results file.
-        -o --output_prefix       : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -x                 : path to Nessus-exported XML results file.
+        -o --output_prefix : path to output scan-results json.
+        -V --verbose       : verbose run [optional].
 
     example: heimdall_tools nessus_mapper -x nessus-results.xml -o test-env
 
@@ -181,9 +204,9 @@ The current iteration only works with Netsparker Enterprise Vulnerabilities Scan
     USAGE: heimdall_tools netsparker_mapper [OPTIONS] -x  -o
 
     FLAGS:
-        -x       : path to netsparker results XML file.
-        -o --output        : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -x           : path to netsparker results XML file.
+        -o --output  : path to output scan-results json.
+        -V --verbose : verbose run [optional].
 
     example: heimdall_tools netsparker_mapper -x netsparker_results.xml -o netsparker_hdf.json
 
@@ -196,12 +219,26 @@ Note: Current this mapper only support single target Nikto Scans.
     USAGE: heimdall_tools nikto_mapper [OPTIONS] -x  -o
 
     FLAGS:
-        -j           : path to Nikto results JSON file.
-        -o --output_prefix       : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -j                 : path to Nikto results JSON file.
+        -o --output_prefix : path to output scan-results json.
+        -V --verbose       : verbose run [optional].
 
     example: heimdall_tools nikto_mapper -j nikto_results.json -o nikto_results.json
 
+## prowler_mapper
+
+prowler_mapper translates Prowler-derived AWS Security Finding Format results from concatenated JSON blobs to HDF-formatted JSON so as to be viewable on Heimdall
+
+Note: Currently this mapper only supports Prowler's ASFF output format.
+
+    USAGE: heimdall_tools prowler_mapper -i <prowler-asff-json> -o <hdf-scan-results-json>
+
+    FLAGS:
+        -i --input -j --json <prowler-asff-json> : path to Prowler ASFF findings file.
+        -o --output <hdf-scan-results-json>      : path to output scan-results json.
+
+    example: heimdall_tools prowler_mapper -i prowler_results.js -o prowler_hdf.json
+
 ## sarif_mapper
 
 sarif_mapper translates a SARIF JSON file into HDF format JSON to be viewable in Heimdall
@@ -209,9 +246,9 @@ sarif_mapper translates a SARIF JSON file into HDF format JSON to be viewable in
     USAGE: heimdall_tools sarif_mapper [OPTIONS] -j  -o
 
     FLAGS:
-        -j           : path to SARIF results JSON file.
-        -o --output_prefix       : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -j                 : path to SARIF results JSON file.
+        -o --output_prefix : path to output scan-results json.
+        -V --verbose       : verbose run [optional].
 
     example: heimdall_tools sarif_mapper -j sarif_results.json -o sarif_results_hdf.json
 
@@ -224,8 +261,8 @@ Note: Currently this mapper only supports AWS.
     USAGE: heimdall_tools scoutsuite_mapper -i  -o
 
     FLAGS:
-        -i --input -j --javascript  : path to Scout Suite results Javascript file.
-        -o --output                 : path to output scan-results json.
+        -i --input -j --javascript : path to Scout Suite results Javascript file.
+        -o --output                : path to output scan-results json.
 
     example: heimdall_tools scoutsuite_mapper -i scoutsuite_results.js -o scoutsuite_hdf.json
 
@@ -238,9 +275,9 @@ Note: A separate HDF JSON is generated for each project reported in the Snyk Rep
     USAGE: heimdall_tools snyk_mapper [OPTIONS] -x  -o
 
     FLAGS:
-        -j           : path to Snyk results JSON file.
-        -o --output_prefix       : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -j                 : path to Snyk results JSON file.
+        -o --output_prefix : path to output scan-results json.
+        -V --verbose       : verbose run [optional].
 
     example: heimdall_tools snyk_mapper -j snyk_results.json -o output-file-prefix
 
@@ -251,11 +288,11 @@ sonarqube_mapper pulls SonarQube results, for the specified project, from the AP
     USAGE: heimdall_tools sonarqube_mapper [OPTIONS] -n  -u  -o
 
     FLAGS:
-        -n --name          : Project Key of the project in SonarQube
-        -u --api_url            : url of the SonarQube Server API. Typically ends with /api.
-        --auth               : username:password or token [optional].
-        -o --output        : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -n --name    : Project Key of the project in SonarQube
+        -u --api_url : url of the SonarQube Server API. Typically ends with /api.
+        --auth       : username:password or token [optional].
+        -o --output  : path to output scan-results json.
+        -V --verbose : verbose run [optional].
 
     example:
 
@@ -272,8 +309,8 @@ xccdf_results_mapper translates an XCCDF_Results XML scan into HDF format json t
     USAGE: heimdall_tools xccdf_results_mapper [OPTIONS] -x  -o
 
     FLAGS:
-        -x            : path to XCCDF-Results XML file.
-        -o --output        : path to output scan-results json.
+        -x          : path to XCCDF-Results XML file.
+        -o --output : path to output scan-results json.
 
     example: heimdall_tools xccdf_results_mapper -x xccdf_results.xml -o scan_results.json
 
@@ -284,10 +321,10 @@ zap_mapper translates OWASP ZAP results Json to HDF format Json be viewed on Hei
     USAGE: heimdall_tools zap_mapper [OPTIONS] -j  -n  -o
 
     FLAGS:
-        -j --json              : path to OWASP ZAP results JSON file.
-        -n --name             : URL of the site being evaluated.
-        -o --output        : path to output scan-results json.
-        -V --verbose                     : verbose run [optional].
+        -j --json    : path to OWASP ZAP results JSON file.
+        -n --name    : URL of the site being evaluated.
+        -o --output  : path to output scan-results json.
+        -V --verbose : verbose run [optional].
 
     example: heimdall_tools zap_mapper -j zap_results.json -n site_name -o scan_results.json
 
@@ -355,6 +392,7 @@ To release a new version, update the version number in `version.rb` according to
 
 ### Authors
 
+-   Author:: Amndeep Singh Mann [Amndeep7](https://github.com/Amndeep7)
 -   Author:: Rony Xavier [rx294](https://github.com/rx294)
 -   Author:: Dan Mirsky [mirskiy](https://github.com/mirskiy)
 

data/lib/heimdall_tools/asff_compatible_products/firewall_manager.rb

@@ -0,0 +1,11 @@
+module HeimdallTools
+  class FirewallManager
+    def self.finding_id(finding, *, encode:, **)
+      encode.call(finding['Title'])
+    end
+
+    def self.product_name(findings, *, encode:, **)
+      encode.call("#{findings[0]['ProductFields']['aws/securityhub/CompanyName']} #{findings[0]['ProductFields']['aws/securityhub/ProductName']}")
+    end
+  end
+end

data/lib/heimdall_tools/asff_compatible_products/prowler.rb

@@ -0,0 +1,19 @@
+module HeimdallTools
+  class Prowler
+    def self.subfindings_code_desc(finding, *, encode:, **)
+      encode.call(finding['Description'])
+    end
+
+    def self.finding_id(finding, *, encode:, **)
+      encode.call(finding['GeneratorId'].partition('-')[-1])
+    end
+
+    def self.product_name(findings, *, encode:, **)
+      encode.call(findings[0]['ProductFields']['ProviderName'])
+    end
+
+    def self.desc(*, **)
+      ' '
+    end
+  end
+end

data/lib/heimdall_tools/asff_compatible_products/securityhub.rb

@@ -0,0 +1,89 @@
+require 'csv'
+require 'json'
+
+module HeimdallTools
+  class SecurityHub
+    private_class_method def self.corresponding_control(controls, finding)
+      controls.find { |c| c['StandardsControlArn'] == finding['ProductFields']['StandardsControlArn'] }
+    end
+
+    def self.supporting_docs(standards:)
+      begin
+        controls = standards.nil? ? nil : standards.map { |s| JSON.parse(s)['Controls'] }.flatten
+      rescue StandardError => e
+        raise "Invalid supporting docs for Security Hub:\nException: #{e}"
+      end
+
+      begin
+        resource_dir = Pathname.new(__FILE__).join('../../../data')
+        aws_config_mapping_file = File.join(resource_dir, 'aws-config-mapping.csv')
+        aws_config_mapping = CSV.read(aws_config_mapping_file, { encoding: 'UTF-8', headers: true, header_converters: :symbol }).map(&:to_hash)
+      rescue StandardError => e
+        raise "Invalid AWS Config mapping file:\nException: #{e}"
+      end
+
+      { controls: controls, aws_config_mapping: aws_config_mapping }
+    end
+
+    def self.finding_id(finding, *, encode:, controls: nil, **)
+      ret = if !controls.nil? && !(control = corresponding_control(controls, finding)).nil?
+              control['ControlId']
+            elsif finding['ProductFields'].member?('ControlId') # check if aws
+              finding['ProductFields']['ControlId']
+            elsif finding['ProductFields'].member?('RuleId') # check if cis
+              finding['ProductFields']['RuleId']
+            else
+              finding['GeneratorId'].split('/')[-1]
+            end
+      encode.call(ret)
+    end
+
+    def self.finding_impact(finding, *, controls: nil, **)
+      if !controls.nil? && !(control = corresponding_control(controls, finding)).nil?
+        imp = control['SeverityRating'].to_sym
+      else
+        # severity is required, but can be either 'label' or 'normalized' internally with 'label' being preferred.  other values can be in here too such as the original severity rating.
+        imp = finding['Severity'].key?('Label') ? finding['Severity']['Label'].to_sym : finding['Severity']['Normalized']/100.0
+        # securityhub asff file does not contain accurate severity information by setting things that shouldn't be informational to informational: when additional context, i.e. standards, is not provided, set informational to medium.
+        imp = :MEDIUM if imp.is_a?(Symbol) && imp == :INFORMATIONAL
+      end
+      imp
+    end
+
+    def self.finding_nist_tag(finding, *, aws_config_mapping:, **)
+      return {} unless finding['ProductFields']['RelatedAWSResources:0/type'] == 'AWS::Config::ConfigRule'
+
+      entries = aws_config_mapping.select { |rule| finding['ProductFields']['RelatedAWSResources:0/name'].include? rule[:awsconfigrulename] }
+      entries.map do |rule|
+        tags_joined = rule[:nistid].split('|') # subheadings are joined together in the csv file
+        tags_joined.map do |tag|
+          if (i = tag.index('(')).nil?
+            tag
+          else
+            tag[i..-1].scan(/\(.+?\)/).map { |subheading| "#{tag[0..i-1]}#{subheading}" }
+          end
+        end
+      end.flatten.uniq
+    end
+
+    def self.finding_title(finding, *, encode:, controls: nil, **)
+      ret = if !controls.nil? && !(control = corresponding_control(controls, finding)).nil?
+              control['Title']
+            else
+              finding['Title']
+            end
+      encode.call(ret)
+    end
+
+    def self.product_name(findings, *, encode:, **)
+      # "#{findings[0]['ProductFields']['aws/securityhub/CompanyName']} #{findings[0]['ProductFields']['aws/securityhub/ProductName']}"
+      # not using above due to wanting to provide the standard's name instead
+      if findings[0]['Types'][0].split('/')[-1].gsub(/-/, ' ').downcase == findings[0]['ProductFields']['StandardsControlArn'].split('/')[-4].gsub(/-/, ' ').downcase
+        standardname = findings[0]['Types'][0].split('/')[-1].gsub(/-/, ' ')
+      else
+        standardname = findings[0]['ProductFields']['StandardsControlArn'].split('/')[-4].gsub(/-/, ' ').split.map(&:capitalize).join(' ')
+      end
+      encode.call("#{standardname} v#{findings[0]['ProductFields']['StandardsControlArn'].split('/')[-2]}")
+    end
+  end
+end

data/lib/heimdall_tools/asff_mapper.rb

@@ -0,0 +1,232 @@
+require 'json'
+require 'set'
+
+require 'htmlentities'
+
+require 'heimdall_tools/hdf'
+require 'heimdall_tools/asff_compatible_products/firewall_manager'
+require 'heimdall_tools/asff_compatible_products/prowler'
+require 'heimdall_tools/asff_compatible_products/securityhub'
+
+module HeimdallTools
+  DEFAULT_NIST_TAG = %w{SA-11 RA-5}.freeze
+
+  INSPEC_INPUTS_MAPPING = {
+    string: 'String',
+    numeric: 'Numeric',
+    regexp: 'Regexp',
+    array: 'Array',
+    hash: 'Hash',
+    boolean: 'Boolean',
+    any: 'Any'
+  }.freeze
+
+  # Loading spinner sign
+  $spinner = Enumerator.new do |e|
+    loop do
+      e.yield '|'
+      e.yield '/'
+      e.yield '-'
+      e.yield '\\'
+    end
+  end
+
+  # TODO: use hash.dig and safe navigation operator throughout
+  class ASFFMapper
+    IMPACT_MAPPING = {
+      CRITICAL: 0.9,
+      HIGH: 0.7,
+      MEDIUM: 0.5,
+      LOW: 0.3,
+      INFORMATIONAL: 0.0
+    }.freeze
+
+    PRODUCT_ARN_MAPPING = {
+      %r{arn:.+:securityhub:.+:.*:product/aws/firewall-manager} => FirewallManager,
+      %r{arn:.+:securityhub:.+:.*:product/aws/securityhub} => SecurityHub,
+      %r{arn:.+:securityhub:.+:.*:product/prowler/prowler} => Prowler
+    }.freeze
+
+    def initialize(asff_json, securityhub_standards_json_array: nil, meta: nil)
+      @meta = meta
+
+      @supporting_docs = {}
+      @supporting_docs[SecurityHub] = SecurityHub.supporting_docs({ standards: securityhub_standards_json_array })
+
+      begin
+        asff_required_keys = %w{AwsAccountId CreatedAt Description GeneratorId Id ProductArn Resources SchemaVersion Severity Title Types UpdatedAt}
+        @report = JSON.parse(asff_json)
+        if @report.length == 1 && @report.member?('Findings') && @report['Findings'].each { |finding| asff_required_keys.to_set.difference(finding.keys.to_set).none? }.all?
+          # ideal case that is spec compliant
+          # might need to ensure that the file is utf-8 encoded and remove a BOM if one exists
+        elsif asff_required_keys.to_set.difference(@report.keys.to_set).none?
+          # individual finding so have to add wrapping array
+          @report = { 'Findings' => [@report] }
+        else
+          raise 'Not a findings file nor an individual finding'
+        end
+      rescue StandardError => e
+        raise "Invalid ASFF file provided:\nException: #{e}"
+      end
+
+      @coder = HTMLEntities.new
+    end
+
+    def encode(string)
+      @coder.encode(string, :basic, :named, :decimal)
+    end
+
+    def external_product_handler(product, data, func, default)
+      if (product.is_a?(Regexp) || (arn = PRODUCT_ARN_MAPPING.keys.find { |a| product.match(a) })) && PRODUCT_ARN_MAPPING.key?(arn || product) && PRODUCT_ARN_MAPPING[arn || product].respond_to?(func)
+        keywords = { encode: method(:encode) }
+        keywords = keywords.merge(@supporting_docs[PRODUCT_ARN_MAPPING[arn || product]]) if @supporting_docs.member?(PRODUCT_ARN_MAPPING[arn || product])
+        PRODUCT_ARN_MAPPING[arn || product].send(func, data, **keywords)
+      elsif default.is_a? Proc
+        default.call
+      else
+        default
+      end
+    end
+
+    def nist_tag(finding)
+      tags = external_product_handler(finding['ProductArn'], finding, :finding_nist_tag, {})
+      tags.empty? ? DEFAULT_NIST_TAG : tags
+    end
+
+    def impact(finding)
+      # there can be findings listed that are intentionally ignored due to the underlying control being superceded by a control from a different standard
+      if finding.member?('Workflow') && finding['Workflow'].member?('Status') && finding['Workflow']['Status'] == 'SUPPRESSED'
+        imp = :INFORMATIONAL
+      else
+        # severity is required, but can be either 'label' or 'normalized' internally with 'label' being preferred.  other values can be in here too such as the original severity rating.
+        default = proc { finding['Severity'].key?('Label') ? finding['Severity']['Label'].to_sym : finding['Severity']['Normalized']/100.0 }
+        imp = external_product_handler(finding['ProductArn'], finding, :finding_impact, default)
+      end
+      imp.is_a?(Symbol) ? IMPACT_MAPPING[imp] : imp
+    end
+
+    def desc_tags(data, label)
+      { data: data || NA_STRING, label: label || NA_STRING }
+    end
+
+    def subfindings(finding)
+      subfinding = {}
+
+      statusreason = finding['Compliance']['StatusReasons'].map { |reason| reason.flatten.map { |string| encode(string) } }.flatten.join("\n") if finding.key?('Compliance') && finding['Compliance'].key?('StatusReasons')
+      if finding.key?('Compliance') && finding['Compliance'].key?('Status')
+        case finding['Compliance']['Status']
+        when 'PASSED'
+          subfinding['status'] = 'passed'
+          subfinding['message'] = statusreason if statusreason
+        when 'WARNING'
+          subfinding['status'] = 'skipped'
+          subfinding['skip_message'] = statusreason if statusreason
+        when 'FAILED'
+          subfinding['status'] = 'failed'
+          subfinding['message'] = statusreason if statusreason
+        when 'NOT_AVAILABLE'
+          # primary meaning is that the check could not be performed due to a service outage or API error, but it's also overloaded to mean NOT_APPLICABLE so technically 'skipped' or 'error' could be applicable, but AWS seems to do the equivalent of skipped
+          subfinding['status'] = 'skipped'
+          subfinding['skip_message'] = statusreason if statusreason
+        else
+          subfinding['status'] = 'error' # not a valid value for the status enum
+          subfinding['message'] = statusreason if statusreason
+        end
+      else
+        subfinding['status'] = 'skipped' # if no compliance status is provided which is a weird but possible case, then skip
+        subfinding['skip_message'] = statusreason if statusreason
+      end
+
+      subfinding['code_desc'] = external_product_handler(finding['ProductArn'], finding, :subfindings_code_desc, '')
+      subfinding['code_desc'] += '; ' unless subfinding['code_desc'].empty?
+      subfinding['code_desc'] += "Resources: [#{finding['Resources'].map { |r| "Type: #{encode(r['Type'])}, Id: #{encode(r['Id'])}#{", Partition: #{encode(r['Partition'])}" if r.key?('Partition')}#{", Region: #{encode(r['Region'])}" if r.key?('Region')}" }.join(', ')}]"
+
+      subfinding['start_time'] = finding.key?('LastObservedAt') ? finding['LastObservedAt'] : finding['UpdatedAt']
+
+      [subfinding]
+    end
+
+    def to_hdf
+      product_groups = {}
+      @report['Findings'].each do |finding|
+        printf("\rProcessing: %s", $spinner.next)
+
+        external = method(:external_product_handler).curry(4)[finding['ProductArn']][finding]
+
+        # group subfindings by asff productarn and then hdf id
+        item = {}
+        item['id'] = external[:finding_id][encode(finding['GeneratorId'])]
+
+        item['title'] = external[:finding_title][encode(finding['Title'])]
+
+        item['tags'] = { nist: nist_tag(finding) }
+
+        item['impact'] = impact(finding)
+
+        item['desc'] = encode(finding['Description'])
+
+        item['descriptions'] = []
+        item['descriptions'] << desc_tags(finding['Remediation']['Recommendation'].map { |_k, v| encode(v) }.join("\n"), 'fix') if finding.key?('Remediation') && finding['Remediation'].key?('Recommendation')
+
+        item['refs'] = []
+        item['refs'] << { url: finding['SourceUrl'] } if finding.key?('SourceUrl')
+
+        item['source_location'] = NA_HASH
+
+        item['results'] = subfindings(finding)
+
+        arn = PRODUCT_ARN_MAPPING.keys.find { |a| finding['ProductArn'].match(a) }
+        if arn.nil?
+          product_info = finding['ProductArn'].split(':')[-1]
+          arn = Regexp.new "arn:.+:securityhub:.+:.*:product/#{product_info.split('/')[1]}/#{product_info.split('/')[2]}"
+        end
+        product_groups[arn] = {} if product_groups[arn].nil?
+        product_groups[arn][item['id']] = [] if product_groups[arn][item['id']].nil?
+        product_groups[arn][item['id']] << [item, finding]
+      end
+
+      controls = []
+      product_groups.each do |product, id_groups|
+        id_groups.each do |id, data|
+          printf("\rProcessing: %s", $spinner.next)
+
+          external = method(:external_product_handler).curry(4)[product]
+
+          group = data.map { |d| d[0] }
+          findings = data.map { |d| d[1] }
+
+          product_info = findings[0]['ProductArn'].split(':')[-1].split('/')
+          product_name = external[findings][:product_name][encode("#{product_info[1]}/#{product_info[2]}")]
+
+          item = {}
+          # add product name to id if any ids are the same across products
+          item['id'] = product_groups.reject { |pg| pg == product }.values.any? { |ig| ig.keys.include?(id) } ? "[#{product_name}] #{id}" : id
+
+          item['title'] = "#{product_name}: #{group.map { |d| d['title'] }.uniq.join(';')}"
+
+          item['tags'] = { nist: group.map { |d| d['tags'][:nist] }.flatten.uniq }
+
+          item['impact'] = group.map { |d| d['impact'] }.max
+
+          item['desc'] = external[group][:desc][group.map { |d| d['desc'] }.uniq.join("\n")]
+
+          item['descriptions'] = group.map { |d| d['descriptions'] }.flatten.compact.reject(&:empty?).uniq
+
+          item['refs'] = group.map { |d| d['refs'] }.flatten.compact.reject(&:empty?).uniq
+
+          item['source_location'] = NA_HASH
+          item['code'] = JSON.pretty_generate({ Findings: findings })
+
+          item['results'] = group.map { |d| d['results'] }.flatten.uniq
+
+          controls << item
+        end
+      end
+
+      results = HeimdallDataFormat.new(profile_name: @meta&.key?('name') ? @meta['name'] : 'AWS Security Finding Format',
+                                       title: @meta&.key?('title') ? @meta['title'] : 'ASFF findings',
+                                       controls: controls)
+      results.to_hdf
+    end
+  end
+end

data/lib/heimdall_tools/aws_config_mapper.rb

@@ -8,7 +8,7 @@ RESOURCE_DIR = Pathname.new(__FILE__).join('../../data')
 AWS_CONFIG_MAPPING_FILE = File.join(RESOURCE_DIR, 'aws-config-mapping.csv')
 
 NOT_APPLICABLE_MSG = 'No AWS resources found to evaluate complaince for this rule'.freeze
-INSUFFICIENT_DATA_MSG = 'Not enough data has been collectd to determine compliance yet.'.freeze
+INSUFFICIENT_DATA_MSG = 'Not enough data has been collected to determine compliance yet.'.freeze
 
 ##
 # HDF mapper for use with AWS Config rules.

data/lib/heimdall_tools/cli.rb

@@ -70,7 +70,7 @@ module HeimdallTools
     option :output_prefix, required: true, aliases: '-o'
     def snyk_mapper
       hdfs = HeimdallTools::SnykMapper.new(File.read(options[:json]), options[:name]).to_hdf
-      puts "\r\HDF Generated:\n"
+      puts "\rHDF Generated:\n"
       hdfs.each_key do |host|
         File.write("#{options[:output_prefix]}-#{host}.json", hdfs[host])
         puts "#{options[:output_prefix]}-#{host}.json"
@@ -84,7 +84,7 @@ module HeimdallTools
     def nikto_mapper
       hdf = HeimdallTools::NiktoMapper.new(File.read(options[:json])).to_hdf
       File.write(options[:output], hdf)
-      puts "\r\HDF Generated:\n"
+      puts "\rHDF Generated:\n"
       puts options[:output].to_s
     end
 
@@ -95,7 +95,7 @@ module HeimdallTools
     def jfrog_xray_mapper
       hdf = HeimdallTools::JfrogXrayMapper.new(File.read(options[:json])).to_hdf
       File.write(options[:output], hdf)
-      puts "\r\HDF Generated:\n"
+      puts "\rHDF Generated:\n"
       puts options[:output].to_s
     end
 
@@ -106,7 +106,7 @@ module HeimdallTools
     def dbprotect_mapper
       hdf = HeimdallTools::DBProtectMapper.new(File.read(options[:xml])).to_hdf
       File.write(options[:output], hdf)
-      puts "\r\HDF Generated:\n"
+      puts "\rHDF Generated:\n"
       puts options[:output].to_s
     end
 
@@ -117,7 +117,7 @@ module HeimdallTools
     def aws_config_mapper
       hdf = HeimdallTools::AwsConfigMapper.new(options[:custom_mapping]).to_hdf
       File.write(options[:output], hdf)
-      puts "\r\HDF Generated:\n"
+      puts "\rHDF Generated:\n"
       puts options[:output].to_s
     end
 
@@ -128,7 +128,7 @@ module HeimdallTools
     def netsparker_mapper
       hdf = HeimdallTools::NetsparkerMapper.new(File.read(options[:xml])).to_hdf
       File.write(options[:output], hdf)
-      puts "\r\HDF Generated:\n"
+      puts "\rHDF Generated:\n"
       puts options[:output].to_s
     end
 
@@ -140,7 +140,7 @@ module HeimdallTools
     def sarif_mapper
       hdf = HeimdallTools::SarifMapper.new(File.read(options[:json])).to_hdf
       File.write(options[:output], hdf)
-      puts "\r\HDF Generated:\n"
+      puts "\rHDF Generated:\n"
       puts options[:output].to_s
     end
 
@@ -155,6 +155,29 @@ module HeimdallTools
       puts options[:output].to_s
     end
 
+    desc 'asff_mapper', 'asff_mapper translates AWS Security Finding Format results from JSON to HDF-formatted JSON so as to be viewable on Heimdall'
+    long_desc Help.text(:asff_mapper)
+    option :json, required: true, banner: 'ASFF-FINDING-JSON', aliases: ['-i', '--input', '-j']
+    option :securityhub_standards, required: false, type: :array, banner: 'ASFF-SECURITYHUB-STANDARDS-JSON', aliases: ['--sh', '--input-securityhub-standards']
+    option :output, required: true, banner: 'HDF-SCAN-RESULTS-JSON', aliases: '-o'
+    def asff_mapper
+      hdf = HeimdallTools::ASFFMapper.new(File.read(options[:json]), securityhub_standards_json_array: options[:securityhub_standards].nil? ? nil : options[:securityhub_standards].map { |filename| File.read(filename) }).to_hdf
+      File.write(options[:output], hdf)
+      puts "\rHDF Generated:\n"
+      puts options[:output].to_s
+    end
+
+    desc 'prowler_mapper', 'prowler_mapper translates Prowler-derived AWS Security Finding Format results from concatenated JSON blobs to HDF-formatted JSON so as to be viewable on Heimdall'
+    long_desc Help.text(:prowler_mapper)
+    option :json, required: true, banner: 'PROWLER-ASFF-JSON', aliases: ['-i', '--input', '-j']
+    option :output, required: true, banner: 'HDF-SCAN-RESULTS-JSON', aliases: '-o'
+    def prowler_mapper
+      hdf = HeimdallTools::ProwlerMapper.new(File.read(options[:json])).to_hdf
+      File.write(options[:output], hdf)
+      puts "\rHDF Generated:\n"
+      puts options[:output].to_s
+    end
+
     desc 'version', 'prints version'
     def version
       puts VERSION

data/lib/heimdall_tools/fortify_mapper.rb

@@ -55,15 +55,13 @@ module HeimdallTools
       findings.uniq
     end
 
-    # rubocop:disable Layout/LineEndStringConcatenationIndentation
     def snippet(snippetid)
       snippet = @snippets.select { |x| x['id'].eql?(snippetid) }.first
       "\nPath: #{snippet['File']}\n" \
-      "StartLine: #{snippet['StartLine']}, " \
-      "EndLine: #{snippet['EndLine']}\n" \
-      "Code:\n#{snippet['Text']['#cdata-section'].strip}" \
+        "StartLine: #{snippet['StartLine']}, " \
+        "EndLine: #{snippet['EndLine']}\n" \
+        "Code:\n#{snippet['Text']['#cdata-section'].strip}" \
     end
-    # rubocop:enable Layout/LineEndStringConcatenationIndentation
 
     def nist_tag(rule)
       references = rule['References']['Reference']

data/lib/heimdall_tools/help/asff_mapper.md

@@ -0,0 +1,6 @@
+  asff_mapper translates AWS Security Finding Format results from JSON to HDF-formatted JSON so as to be viewable on Heimdall
+
+Examples:
+
+  heimdall_tools asff_mapper -i <asff-finding-json> -o <hdf-scan-results-json>
+  heimdall_tools asff_mapper -i <asff-finding-json> --sh <standard-1-json> ... <standard-n-json> -o <hdf-scan-results-json>

data/lib/heimdall_tools/help/prowler_mapper.md

@@ -0,0 +1,5 @@
+  prowler_mapper translates Prowler-derived AWS Security Finding Format results from concatenated JSON blobs to HDF-formatted JSON so as to be viewable on Heimdall
+
+Examples:
+
+  heimdall_tools prowler_mapper -i <prowler-asff-json> -o <hdf-scan-results-json>

data/lib/heimdall_tools/nessus_mapper.rb

@@ -92,11 +92,17 @@ module HeimdallTools
 
     def finding(issue, timestamp)
       finding = {}
-      # if compliance-result field, this is a policy compliance result entry
-      # nessus policy compliance result provides a pass/fail data
-      # For non policy compliance  results are defaulted to failed
       if issue['compliance-result']
-        finding['status'] = issue['compliance-result'].eql?('PASSED') ? 'passed' : 'failed'
+        case issue['compliance-result']
+        when 'PASSED'
+          finding['status'] = 'passed'
+        when 'ERROR'
+          finding['status'] = 'error'
+        when 'WARNING'
+          finding['status'] = 'skipped'
+        else
+          finding['status'] = 'failed'
+        end
       else
         finding['status'] = 'failed'
       end

data/lib/heimdall_tools/prowler_mapper.rb

@@ -0,0 +1,8 @@
+module HeimdallTools
+  class ProwlerMapper < ASFFMapper
+    def initialize(prowler_asff_json)
+      # comes as an asff-json file which is basically all the findings concatenated into one file instead of putting it in the proper wrapper data structure
+      super("{ \"Findings\": [#{prowler_asff_json.split("\n").join(',')}]}", meta: { 'name' => 'Prowler', 'title' => 'Prowler findings' })
+    end
+  end
+end

data/lib/heimdall_tools.rb

@@ -19,4 +19,6 @@ module HeimdallTools
   autoload :SarifMapper, 'heimdall_tools/sarif_mapper'
   autoload :ScoutSuiteMapper, 'heimdall_tools/scoutsuite_mapper'
   autoload :XCCDFResultsMapper, 'heimdall_tools/xccdf_results_mapper'
+  autoload :ASFFMapper, 'heimdall_tools/asff_mapper'
+  autoload :ProwlerMapper, 'heimdall_tools/prowler_mapper'
 end

metadata

@@ -1,16 +1,17 @@
 --- !ruby/object:Gem::Specification
 name: heimdall_tools
 version: !ruby/object:Gem::Version
-  version: 1.3.48
+  version: 1.3.49
 platform: ruby
 authors:
 - Robert Thew
 - Rony Xavier
+- Amndeep Singh Mann
 - Aaron Lippold
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2021-06-29 00:00:00.000000000 Z
+date: 2021-11-18 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: aws-sdk-configservice
@@ -26,6 +27,20 @@ dependencies:
     - - "~>"
       - !ruby/object:Gem::Version
         version: '1'
+- !ruby/object:Gem::Dependency
+  name: aws-sdk-securityhub
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: '1'
 - !ruby/object:Gem::Dependency
   name: csv
   requirement: !ruby/object:Gem::Requirement
@@ -54,6 +69,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: 0.17.2
+- !ruby/object:Gem::Dependency
+  name: htmlentities
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 4.3.4
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - "~>"
+      - !ruby/object:Gem::Version
+        version: 4.3.4
 - !ruby/object:Gem::Dependency
   name: httparty
   requirement: !ruby/object:Gem::Requirement
@@ -216,6 +245,10 @@ files:
 - lib/data/owasp-nist-mapping.csv
 - lib/data/scoutsuite-nist-mapping.csv
 - lib/heimdall_tools.rb
+- lib/heimdall_tools/asff_compatible_products/firewall_manager.rb
+- lib/heimdall_tools/asff_compatible_products/prowler.rb
+- lib/heimdall_tools/asff_compatible_products/securityhub.rb
+- lib/heimdall_tools/asff_mapper.rb
 - lib/heimdall_tools/aws_config_mapper.rb
 - lib/heimdall_tools/burpsuite_mapper.rb
 - lib/heimdall_tools/cli.rb
@@ -224,6 +257,7 @@ files:
 - lib/heimdall_tools/fortify_mapper.rb
 - lib/heimdall_tools/hdf.rb
 - lib/heimdall_tools/help.rb
+- lib/heimdall_tools/help/asff_mapper.md
 - lib/heimdall_tools/help/aws_config_mapper.md
 - lib/heimdall_tools/help/burpsuite_mapper.md
 - lib/heimdall_tools/help/dbprotect_mapper.md
@@ -232,6 +266,7 @@ files:
 - lib/heimdall_tools/help/nessus_mapper.md
 - lib/heimdall_tools/help/netsparker_mapper.md
 - lib/heimdall_tools/help/nikto_mapper.md
+- lib/heimdall_tools/help/prowler_mapper.md
 - lib/heimdall_tools/help/sarif_mapper.md
 - lib/heimdall_tools/help/scoutsuite_mapper.md
 - lib/heimdall_tools/help/snyk_mapper.md
@@ -241,6 +276,7 @@ files:
 - lib/heimdall_tools/nessus_mapper.rb
 - lib/heimdall_tools/netsparker_mapper.rb
 - lib/heimdall_tools/nikto_mapper.rb
+- lib/heimdall_tools/prowler_mapper.rb
 - lib/heimdall_tools/sarif_mapper.rb
 - lib/heimdall_tools/scoutsuite_mapper.rb
 - lib/heimdall_tools/snyk_mapper.rb
@@ -268,8 +304,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.15
+rubygems_version: 3.2.22
 signing_key: 
 specification_version: 4
-summary: Convert Forify, Openzap and Sonarqube results to HDF
+summary: Convert a variety of security product results to HDF
 test_files: []