heimdall_tools 1.3.23 → 1.3.28

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 87c18b112ab38b1f06c4e7e85041c5c4d388d974c8ce256fdf928dbc4e2ecdf5
-  data.tar.gz: f7fcefb5bc73c34fa17039cc2b40a068fbf754a8a7b9a07e35c4d6ad1dace8eb
+  metadata.gz: 793e9f28cb98cddc239829d4164d2f316100b2157ae9e196d2c32e5271fca664
+  data.tar.gz: 267e61797626c6ec797fdffda2d03bee9de9ce5ed45f918d2cfcda9af86a9957
 SHA512:
-  metadata.gz: 983087a23a15d02b927b43e07ab5fa84451aa6d09bf6feba0acdbd3c24d2710374d2fde2e8b140650bdb1dcf90a5490a651c1672d0f8b4b2a05abc688136e041
-  data.tar.gz: 717f833a3901385a7d15869f019c1ccacb2a9cdf9c9f4107e08b574620164596fcfcb00a050c7c4e3b9aed2aa6ff749e93fd095d3767e9d25dfbed5a8927ce91
+  metadata.gz: 671713a138539ad4f68615aa1c74d727ddd221b27fb0d7ea22531b8c5caa19aed0d00eabab2b685edc00566b2098ea7514e3281e30fbcbc09ea4db31c0171762
+  data.tar.gz: 2f589a0a0db9cfd64cf23f25420f6609229bbc0dfc8e435e7bde4eb7c19c431a8446ccae650e2bb6820cee8a04e996eb205999a2ad2429063e491818127a8c86

data/CHANGELOG.md

@@ -1,5 +1,72 @@
 # Changelog
 
+## [Unreleased](https://github.com/mitre/heimdall_tools/tree/HEAD)
+
+[Full Changelog](https://github.com/mitre/heimdall_tools/compare/v1.3.27...HEAD)
+
+**Closed issues:**
+
+- Map 'Policy Compliance' entries for nessus\_mapper [\#49](https://github.com/mitre/heimdall_tools/issues/49)
+
+**Merged pull requests:**
+
+- Add code to translate Policy compliance results [\#51](https://github.com/mitre/heimdall_tools/pull/51) ([rx294](https://github.com/rx294))
+
+## [v1.3.27](https://github.com/mitre/heimdall_tools/tree/v1.3.27) (2020-05-22)
+
+[Full Changelog](https://github.com/mitre/heimdall_tools/compare/v1.3.26...v1.3.27)
+
+**Merged pull requests:**
+
+- Updated the Dockerfile to run in an alpine ruby container [\#47](https://github.com/mitre/heimdall_tools/pull/47) ([jsa5593](https://github.com/jsa5593))
+- Require a newer version of git-lite-version-bump for Windows support [\#46](https://github.com/mitre/heimdall_tools/pull/46) ([rbclark](https://github.com/rbclark))
+
+## [v1.3.26](https://github.com/mitre/heimdall_tools/tree/v1.3.26) (2020-05-06)
+
+[Full Changelog](https://github.com/mitre/heimdall_tools/compare/v1.3.25...v1.3.26)
+
+**Implemented enhancements:**
+
+- Converter: Nessus Transform for Audit results and vulnerability scan results  [\#29](https://github.com/mitre/heimdall_tools/issues/29)
+
+**Merged pull requests:**
+
+- Nessus Mapper  [\#45](https://github.com/mitre/heimdall_tools/pull/45) ([rx294](https://github.com/rx294))
+
+## [v1.3.25](https://github.com/mitre/heimdall_tools/tree/v1.3.25) (2020-04-16)
+
+[Full Changelog](https://github.com/mitre/heimdall_tools/compare/v1.3.24...v1.3.25)
+
+**Closed issues:**
+
+- Add minimum required json fields to work heimdall server    [\#5](https://github.com/mitre/heimdall_tools/issues/5)
+
+**Merged pull requests:**
+
+- Make sure the fields we are looking for in Fortify exist before we parse the element [\#44](https://github.com/mitre/heimdall_tools/pull/44) ([rbclark](https://github.com/rbclark))
+- Update actions to use ruby/setup-ruby [\#43](https://github.com/mitre/heimdall_tools/pull/43) ([Bialogs](https://github.com/Bialogs))
+
+## [v1.3.24](https://github.com/mitre/heimdall_tools/tree/v1.3.24) (2020-04-07)
+
+[Full Changelog](https://github.com/mitre/heimdall_tools/compare/v1.3.23...v1.3.24)
+
+**Implemented enhancements:**
+
+- Converter: Burp Suite Pro [\#28](https://github.com/mitre/heimdall_tools/issues/28)
+
+**Fixed bugs:**
+
+- \[Bug\] Import mapping csvs by relative path  [\#41](https://github.com/mitre/heimdall_tools/issues/41)
+
+**Merged pull requests:**
+
+- Update to pull data csvs by relative path [\#42](https://github.com/mitre/heimdall_tools/pull/42) ([rx294](https://github.com/rx294))
+- Burpsuite mapper [\#40](https://github.com/mitre/heimdall_tools/pull/40) ([rx294](https://github.com/rx294))
+
+## [v1.3.23](https://github.com/mitre/heimdall_tools/tree/v1.3.23) (2020-03-31)
+
+[Full Changelog](https://github.com/mitre/heimdall_tools/compare/v1.3.23.pre5...v1.3.23)
+
 ## [v1.3.23.pre5](https://github.com/mitre/heimdall_tools/tree/v1.3.23.pre5) (2020-03-31)
 
 [Full Changelog](https://github.com/mitre/heimdall_tools/compare/v1.3.23.pre4...v1.3.23.pre5)

data/README.md

@@ -1,7 +1,6 @@
 # Heimdall Tools
 
 ![Overall Status](https://github.com/mitre/heimdall_tools/workflows/heimdall_tools/badge.svg)
-
 ![Heimdall Tools Build](https://github.com/mitre/heimdall_tools/workflows/Build%20and%20release%20gem/badge.svg)
 
 HeimdallTools supplies several methods to convert output from various tools to "Heimdall Data Format"(HDF) format to be viewable in Heimdall. The current converters are:
@@ -9,8 +8,8 @@ HeimdallTools supplies several methods to convert output from various tools to "
 - **sonarqube_mapper** - open-source static code analysis tool
 - **fortify_mapper** - commercial static code analysis tool
 - **zap_mapper** - OWASP ZAP - open-source dynamic code analysis tool
-
-# Prerequisites
+- **burpsuite_mapper** - commercial dynamic analysis tool
+- **nessus_mapper** - commercial vulnerability scanner
 
 Ruby 2.4 or higher (check using "ruby -v")
 
@@ -55,6 +54,13 @@ Verify the installed version number:
 On the Command Line, `heimdall_tools help` will print a listing of all the command with a short description.
 For detailed help on any command, run `heimdall_tools help [COMMAND]`. Help can also be called with the `-h, --help` flags after any command, like `heimdall_tools fortify_mapper -h`.
 
+For Docker usage, replace the `heimdall_tools` command with the correct Docker command below for your operating system:
+
+- **On Linux and Mac:** `docker run -it -v$(pwd):/share mitre/heimdall_tools`
+- **On Windows CMD:** `docker run -it -v%cd%:/share mitre/heimdall_tools`
+
+Note that all of the above Docker commands will mount your current directory on the Docker container. Ensure that you have navigated to the directory you intend to convert files in before executing the command.
+
 ## sonarqube_mapper
 
 sonarqube_mapper pulls SonarQube results, for the specified project, from the API and outputs in HDF format Json to be viewed on Heimdall
@@ -107,7 +113,39 @@ FLAGS:
 example: heimdall_tools zap_mapper -j zap_results.json -n site_name -o scan_results.json
 ```
 
-## version
+## burpsuite_mapper
+
+burpsuite_mapper translates an BurpSuite Pro exported XML results file into HDF format json to be viewable in Heimdall
+
+```
+USAGE: heimdall_tools burpsuite_mapper [OPTIONS] -x <burpsuite-xml> -o <scan-results.json>
+
+FLAGS:
+    -x <burpsuite_xml>               : path to BurpSuitePro exported XML results file.
+    -o --output <scan-results>       : path to output scan-results json.
+    -V --verbose                     : verbose run [optional].
+
+example: heimdall_tools burpsuite_mapper -x burpsuite_results.xml -o scan_results.json
+```
+
+## nessus_mapper
+
+nessus_mapper translates a Nessus-exported XML results file into HDF format json to be viewable in Heimdall
+
+Note: A separate HDF JSON file is generated for each host reported in the Nessus Report.
+
+```
+USAGE: heimdall_tools nessus_mapper [OPTIONS] -x <nessus-results-xml> -o <hdf-file-prefix>
+
+FLAGS:
+    -x <nessus-results-xml>          : path to Nessus-exported XML results file.
+    -o --output_prefix <prefix>      : path to output scan-results json.
+    -V --verbose                     : verbose run [optional].
+
+example: heimdall_tools nessus_mapper -x nessus-results.xml -o test-env
+```
+
+## version  
 
 Prints out the gem version
 

data/lib/data/nessus-plugins-nist-mapping.csv

@@ -0,0 +1,108 @@
+pluginFamily,pluginID,NIST-ID,Rev
+AIX Local Security Checks,*,SI-2|RA-5,4
+Amazon Linux Local Security Checks,*,SI-2|RA-5,4
+CentOS Local Security Checks,*,SI-2|RA-5,4
+Debian Local Security Checks,*,SI-2|RA-5,4
+F5 Networks Local Security Checks,*,SI-2|RA-5,4
+Fedora Local Security Checks,*,SI-2|RA-5,4
+FreeBSD Local Security Checks,*,SI-2|RA-5,4
+Gentoo Local Security Checks,*,SI-2|RA-5,4
+HP-UX Local Security Checks,*,SI-2|RA-5,4
+Huawei Local Security Checks,*,SI-2|RA-5,4
+Junos Local Security Checks,*,SI-2|RA-5,4
+MacOS X Local Security Checks,*,SI-2|RA-5,4
+Mandriva Local Security Checks,*,SI-2|RA-5,4
+NewStart CGSL Local Security Checks,*,SI-2|RA-5,4
+Oracle Linux Local Security Checks,*,SI-2|RA-5,4
+OracleVM Local Security Checks,*,SI-2|RA-5,4
+Palo Alto Local Security Checks,*,SI-2|RA-5,4
+PhotonOS Local Security Checks,*,SI-2|RA-5,4
+Red Hat Local Security Checks,*,SI-2|RA-5,4
+Scientific Linux Local Security Checks,*,SI-2|RA-5,4
+Slackware Local Security Checks,*,SI-2|RA-5,4
+Solaris Local Security Checks,*,SI-2|RA-5,4
+SuSE Local Security Checks,*,SI-2|RA-5,4
+Ubuntu Local Security Checks,*,SI-2|RA-5,4
+VMware ESX Local Security Checks,*,SI-2|RA-5,4
+Virtuozzo Local Security Checks,*,SI-2|RA-5,4
+Backdoors,,,
+Brute force attacks,,,
+CGI abuses,,,
+CGI abuses : XSS,,,
+CISCO,,,
+DNS,,,
+Databases,,,
+Default Unix Accounts,,,
+Denial of Service,,,
+FTP,,,
+Firewalls,56310,SC-7,4
+Gain a shell remotely,,,
+General,133964,AC-3(4),4
+General,117530,UM-1,4
+General,110483,CM-7,4
+General,95928,AC-2,4
+General,90191,CM-8,4
+General,86420,CM-8,4
+General,70544,AC-17(2)|SC-13,4
+General,66334,SI-2|RA-5,4
+General,64582,CM-8,4
+General,57582,SC-12,4
+General,57041,AC-17(2)|SC-13,4
+General,56984,AC-17(2)|SC-13,4
+General,56468,CM-8,4
+General,55472,CM-8,4
+General,54615,CM-8,4
+General,51192,SC-12,4
+General,45590,CM-8,4
+General,45432,CM-8,4
+General,45410,SC-12,4
+General,39520,SI-2|RA-5,4
+General,35351,CM-8,4
+General,34098,CM-8,4
+General,33276,CM-8,4
+General,25220,SC-8,4
+General,25203,CM-8,4
+General,25202,CM-8,4
+General,22869,CM-8,4
+General,21643,AC-17(2)|SC-13,4
+General,12053,CM-8,4
+General,11936,CM-8,4
+General,10881,AC-17(2)|SC-13,4
+General,10863,SC-12,4
+General,10287,CM-8,4
+General,10114,CM-6,4
+Misc.,118237,CM-8,4
+Misc.,97993,CM-8,4
+Misc.,90707,CM-8,4
+Misc.,84821,AC-17(2)|SC-13,4
+Misc.,83875,AC-17(2)|SC-13,4
+Misc.,70657,AC-17(2)|SC-13,4
+Misc.,58651,AC-17,4
+Mobile Devices,,,
+Netware,,,
+Peer-To-Peer File Sharing,,,
+Policy Compliance,,,
+Port scanners,14272,CM-8,4
+RPC,53335,CM-8,4
+RPC,10223,CM-8,4
+SCADA,,,
+SMTP problems,,,
+SNMP,,,
+Service detection,121010,AC-17(2)|SC-13,4
+Service detection,104743,AC-17(2)|SC-13,4
+Service detection,25221,CM-8,4
+Service detection,22964,CM-8,4
+Service detection,11111,CM-8,4
+Service detection,10884,AU-8(1),4
+Service detection,10267,AC-17(2),4
+Settings,117887,UM-1,4
+Settings,110095,UM-1,4
+Settings,19506,UM-1,4
+Web Servers,85805,SC-8|SC-13,4
+Web Servers,84502,AC-17(2)|SC-13,4
+Web Servers,43111,CM-8,4
+Web Servers,24260,CM-8,4
+Web Servers,10107,CM-8,4
+Windows,,,
+Windows : Microsoft Bulletins,,,
+Windows : User management,,,

data/lib/heimdall_tools.rb

@@ -8,4 +8,6 @@ module HeimdallTools
   autoload :FortifyMapper, 'heimdall_tools/fortify_mapper'
   autoload :ZapMapper, 'heimdall_tools/zap_mapper'
   autoload :SonarQubeMapper, 'heimdall_tools/sonarqube_mapper'
+  autoload :BurpSuiteMapper, 'heimdall_tools/burpsuite_mapper'
+  autoload :NessusMapper, 'heimdall_tools/nessus_mapper'
 end

data/lib/heimdall_tools/burpsuite_mapper.rb

@@ -0,0 +1,138 @@
+require 'json'
+require 'csv'
+require 'heimdall_tools/hdf'
+require 'utilities/xml_to_hash'
+
+RESOURCE_DIR = Pathname.new(__FILE__).join('../../data')
+
+CWE_NIST_MAPPING_FILE = File.join(RESOURCE_DIR, 'cwe-nist-mapping.csv')
+
+IMPACT_MAPPING = {
+  High: 0.7,
+  Medium: 0.5,
+  Low: 0.3,
+  Information: 0.3
+}.freeze
+
+CWE_REGEX = 'CWE-(\d*):'.freeze
+
+DEFAULT_NIST_TAG = ["SA-11", "RA-5", "Rev_4"].freeze
+
+# rubocop:disable Metrics/AbcSize
+
+module HeimdallTools
+  class BurpSuiteMapper
+    def initialize(burps_xml, name=nil, verbose = false)
+      @burps_xml = burps_xml
+      @verbose = verbose
+
+      begin
+        @cwe_nist_mapping = parse_mapper
+        data = xml_to_hash(burps_xml)
+
+        @issues = data['issues']['issue']
+        @burpVersion = data['issues']['burpVersion']
+        @timestamp = data['issues']['exportTime']
+
+      rescue StandardError => e
+        raise "Invalid Burpsuite XML file provided Exception: #{e}"
+      end
+
+    end
+
+    def parse_html(block)
+      Nokogiri::HTML(block['#cdata-section']).text.to_s.strip unless block.nil?
+    end
+
+    def finding(issue)
+      finding = {}
+      finding['status'] = 'failed'
+      finding['code_desc'] = format_code_desc(issue)
+      finding['run_time'] = NA_FLOAT
+      finding['start_time'] = @timestamp
+      [finding]
+    end
+
+    def format_code_desc(issue)
+      desc = ''
+      desc += "Host: ip: #{issue['host']['ip']}, url: #{issue['host']['text']}\n"
+      desc += "Location: #{parse_html(issue['location'])}\n"
+      desc += "issueDetail: #{parse_html(issue['issueDetail'])}\n" unless issue['issueDetail'].nil?
+      desc += "confidence: #{issue['confidence']}\n" unless issue['confidence'].nil?
+      desc
+    end
+
+    def nist_tag(cweid)
+      entries = @cwe_nist_mapping.select { |x| cweid.include? x[:cweid].to_s }
+      tags = entries.map { |x| [x[:nistid], "Rev_#{x[:rev]}"] }
+      tags.empty? ? DEFAULT_NIST_TAG : tags.flatten.uniq
+    end
+
+    def parse_cwe(text)
+      reg = Regexp.new(CWE_REGEX, Regexp::IGNORECASE)
+      text.scan(reg).map(&:first)
+    end
+
+    def impact(severity)
+      IMPACT_MAPPING[severity.to_sym]
+    end
+
+    def parse_mapper
+      csv_data = CSV.read(CWE_NIST_MAPPING_FILE, { encoding: 'UTF-8',
+                                                   headers: true,
+                                                   header_converters: :symbol,
+                                                   converters: :all })
+      csv_data.map(&:to_hash)
+    end
+
+    def desc_tags(data, label)
+      { "data": data || NA_STRING, "label": label || NA_STRING }
+    end
+
+    # Burpsuite report could have multiple issue entries for multiple findings of same issue type.
+    # The meta data is identical across entries 
+    # method collapse_duplicates return unique controls with applicable findings collapsed into it.
+    def collapse_duplicates(controls)
+      unique_controls = []
+
+      controls.map { |x| x['id'] }.uniq.each do |id|
+        collapsed_results = controls.select { |x| x['id'].eql?(id) }.map {|x| x['results']}
+        unique_control = controls.find { |x| x['id'].eql?(id) }
+        unique_control['results'] = collapsed_results.flatten
+        unique_controls << unique_control
+      end
+      unique_controls
+    end
+
+    def to_hdf
+      controls = []
+      @issues.each do |issue|
+        @item = {}
+        @item['id']                 = issue['type'].to_s
+        @item['title']              = parse_html(issue['name'])
+        @item['desc']               = parse_html(issue['issueBackground'])
+        @item['impact']             = impact(issue['severity'])
+        @item['tags']               = {}
+        @item['descriptions']       = []
+        @item['descriptions']       <<  desc_tags(parse_html(issue['issueBackground']), 'check')
+        @item['descriptions']       <<  desc_tags(parse_html(issue['remediationBackground']), 'fix')
+        @item['refs']               = NA_ARRAY
+        @item['source_location']    = NA_HASH
+        @item['tags']['nist']       = nist_tag(parse_cwe(parse_html(issue['vulnerabilityClassifications'])))
+        @item['tags']['cweid']      = parse_html(issue['vulnerabilityClassifications'])
+        @item['tags']['confidence'] = issue['confidence'].to_s
+        @item['code']               = ''
+        @item['results']            = finding(issue)
+
+        controls << @item
+      end
+      controls = collapse_duplicates(controls)
+      results = HeimdallDataFormat.new(profile_name: 'BurpSuite Pro Scan',
+                                       version: @burpVersion,
+                                       title: "BurpSuite Pro Scan",
+                                       summary: "BurpSuite Pro Scan",
+                                       controls: controls)
+      results.to_hdf
+    end
+  end
+end

data/lib/heimdall_tools/cli.rb

@@ -35,6 +35,31 @@ module HeimdallTools
       File.write(options[:output], hdf)
     end
 
+    desc 'burpsuite_mapper', 'burpsuite_mapper translates Burpsuite xml report to HDF format Json be viewed on Heimdall'
+    long_desc Help.text(:burpsuite_mapper)
+    option :xml, required: true, aliases: '-x'
+    option :output, required: true, aliases: '-o'
+    option :verbose, type: :boolean, aliases: '-V'
+    def burpsuite_mapper
+      hdf = HeimdallTools::BurpSuiteMapper.new(File.read(options[:xml])).to_hdf
+      File.write(options[:output], hdf)
+    end
+
+    desc 'nessus_mapper', 'nessus_mapper translates nessus xml report to HDF format Json be viewed on Heimdall'
+    long_desc Help.text(:nessus_mapper)
+    option :xml, required: true, aliases: '-x'
+    option :output_prefix, required: true, aliases: '-o'
+    option :verbose, type: :boolean, aliases: '-V'
+    def nessus_mapper
+      hdfs = HeimdallTools::NessusMapper.new(File.read(options[:xml])).to_hdf
+
+      hdfs.keys.each do | host |
+        File.write("#{options[:output_prefix]}-#{host}.json", hdfs[host])
+        puts "HDF Generated: #{options[:output_prefix]}-#{host}.json"
+      end
+      
+    end
+
     desc 'version', 'prints version'
     def version
       puts VERSION

data/lib/heimdall_tools/fortify_mapper.rb

@@ -43,7 +43,11 @@ module HeimdallTools
         traces.each do |trace|
           entries = trace['Primary']['Entry']
           entries = [entries] unless entries.is_a?(Array)
-          entries = entries.reject { |x| x['Node'].nil? }
+          # This is just regular array access, it is just written in a manner that allows us
+          # to use Ruby's safe navigation operator. We rely on
+          # entry['Node']['SourceLocation']['snippet'] to exist on all of our entries, so if any
+          # of those are empty we reject that element.
+          entries = entries.reject { |x| x&.[]('Node')&.[]('SourceLocation')&.[]('snippet').nil? }
           entries.each do |entry|
             findings << process_entry(entry)
           end

data/lib/heimdall_tools/hdf.rb

@@ -2,6 +2,7 @@ require 'json'
 require 'heimdall_tools/version'
 require 'openssl'
 
+NA_STRING = "".freeze
 NA_TAG = nil.freeze
 NA_ARRAY = [].freeze
 NA_HASH = {}.freeze
@@ -27,12 +28,14 @@ module HeimdallTools
                    depends: NA_ARRAY,
                    groups: NA_ARRAY,
                    status: 'loaded',
-                   controls: NA_TAG)
+                   controls: NA_TAG,
+                   target_id: NA_TAG)
 
       @results_json = {}
       @results_json['platform'] = {}
       @results_json['platform']['name'] = 'Heimdall Tools'
       @results_json['platform']['release'] = HeimdallTools::VERSION
+      @results_json['platform']['target_id'] = target_id.to_s
       @results_json['version'] = HeimdallTools::VERSION
 
       @results_json['statistics'] = {}

data/lib/heimdall_tools/help/burpsuite_mapper.md

@@ -0,0 +1,5 @@
+  burpsuite_mapper translates an BurpSuite Pro exported XML results file into HDF format json to be viewable in Heimdall
+  
+Examples:
+
+  heimdall_tools burpsuite_mapper -x burpsuite_results.xml -o scan_results.json

data/lib/heimdall_tools/help/nessus_mapper.md

@@ -0,0 +1,9 @@
+  nessus_mapper translates an Nessus exported XML results file into HDF format json to be viewable in Heimdall
+
+  The current iteration maps all plugin families except 'Policy Compliance'
+  
+  A separate HDF JSON is generated for each host reported in the Nessus Report.
+
+Examples:
+
+  heimdall_tools nessus_mapper -x nessus_results.xml -o file-prefix

data/lib/heimdall_tools/nessus_mapper.rb

@@ -0,0 +1,231 @@
+require 'json'
+require 'csv'
+require 'heimdall_tools/hdf'
+require 'utilities/xml_to_hash'
+
+RESOURCE_DIR = Pathname.new(__FILE__).join('../../data')
+
+NESSUS_PLUGINS_NIST_MAPPING_FILE =   File.join(RESOURCE_DIR, 'nessus-plugins-nist-mapping.csv')
+
+IMPACT_MAPPING = {
+  Info: 0.0,
+  Low: 0.3,
+  Medium: 0.5,
+  High: 0.7,
+  Critical: 0.9,
+}.freeze
+
+DEFAULT_NIST_TAG = ["unmapped"].freeze
+
+# Nessus results file 800-53 refs does not contain Nist rev version. Using this default
+# version in that case
+DEFAULT_NIST_REV = 'Rev_4'.freeze
+
+NA_PLUGIN_OUTPUT = "This Nessus Plugin does not provide output message.".freeze
+
+# rubocop:disable Metrics/AbcSize
+
+module HeimdallTools
+  class NessusMapper
+    def initialize(nessus_xml, verbose = false)
+      @nessus_xml = nessus_xml
+      @verbose = verbose
+
+      begin
+        @cwe_nist_mapping = parse_mapper
+        @data = xml_to_hash(nessus_xml)
+
+        File.write("273970.json", @data.to_json)
+
+        @reports = extract_report
+        @scaninfo = extract_scaninfo
+      rescue StandardError => e
+        raise "Invalid Nessus XML file provided Exception: #{e}"
+      end
+
+    end
+
+    def extract_report
+      begin
+        # When there are multiple hosts in the nessus report ReportHost field is an array
+        # When there is only one host in the nessus report ReportHost field is a hash
+        # Array() converts ReportHost to array in case there is only one host
+        reports = @data['NessusClientData_v2']['Report']['ReportHost']
+        reports.kind_of?(Array) ? reports : [reports]
+      rescue StandardError => e
+        raise "Invalid Nessus XML file provided Exception: #{e}"
+      end
+    end
+    def parse_refs(refs, key)
+      refs.split(',').map { |x| x.split('|')[1] if x.include?(key) }.compact
+    end
+
+    def extract_scaninfo
+      begin
+        policy = @data['NessusClientData_v2']['Policy']
+        info = {}
+
+        info['policyName'] = policy['policyName']
+        info['version'] = policy['Preferences']['ServerPreferences']['preference'].select {|x| x['name'].eql? 'sc_version'}.first['value']
+        info
+      rescue StandardError => e
+        raise "Invalid Nessus XML file provided Exception: #{e}"
+      end
+    end
+
+    def extract_timestamp(report)
+      begin
+        timestamp = report['HostProperties']['tag'].select {|x| x['name'].eql? 'HOST_START'}.first['text']
+      rescue StandardError => e
+        raise "Invalid Nessus XML file provided Exception: #{e}"
+      end
+    end
+
+    def format_desc(issue)
+      desc = ''
+      desc += "Plugin Family: #{issue['pluginFamily']}; "
+      desc += "Port: #{issue['port']}; "
+      desc += "Protocol: #{issue['protocol']};"
+      desc
+    end
+
+    def finding(issue, timestamp)
+      finding = {}
+      # if compliance-result field, this is a policy compliance result entry
+      # nessus policy compliance result provides a pass/fail data
+      # For non policy compliance  results are defaulted to failed
+      if issue['compliance-result']
+        finding['status'] = issue['compliance-result'].eql?('PASSED') ? 'passed' : 'failed'
+      else
+        finding['status'] = 'failed'
+      end
+
+      if issue['description']
+        finding['code_desc'] = issue['description'].to_s || NA_PLUGIN_OUTPUT
+      else
+        finding['code_desc'] = issue['plugin_output'] || NA_PLUGIN_OUTPUT
+      end
+      finding['run_time'] = NA_FLOAT
+      finding['start_time'] = timestamp
+      [finding]
+    end
+
+    def nist_tag(pluginfamily, pluginid)
+      entries = @cwe_nist_mapping.select { |x| (x[:pluginfamily].eql?(pluginfamily) && (x[:pluginid].eql?('*') || x[:pluginid].eql?(pluginid.to_i)) ) }
+      tags = entries.map { |x| [x[:nistid].split('|'), "Rev_#{x[:rev]}"] }
+      tags.empty? ? DEFAULT_NIST_TAG : tags.flatten.uniq
+    end
+
+    def impact(severity)
+      # Map CAT levels and Plugin severity to HDF impact levels
+      case severity
+      when "0"
+        IMPACT_MAPPING[:Info]
+      when "1","III"
+        IMPACT_MAPPING[:Low]
+      when "2","II"
+        IMPACT_MAPPING[:Medium]
+      when "3","I"
+        IMPACT_MAPPING[:High]
+      when "4"
+        IMPACT_MAPPING[:Critical]
+      else
+        -1
+      end
+    end
+
+    def parse_mapper
+      csv_data = CSV.read(NESSUS_PLUGINS_NIST_MAPPING_FILE, { encoding: 'UTF-8',
+                                                   headers: true,
+                                                   header_converters: :symbol,
+                                                   converters: :all })
+      csv_data.map(&:to_hash)
+    end
+
+    def desc_tags(data, label)
+      { "data": data || NA_STRING, "label": label || NA_STRING }
+    end
+
+    # Nessus report could have multiple issue entries for multiple findings of same issue type.
+    # The meta data is identical across entries 
+    # method collapse_duplicates return unique controls with applicable findings collapsed into it.
+    def collapse_duplicates(controls)
+      unique_controls = []
+
+      controls.map { |x| x['id'] }.uniq.each do |id|
+        collapsed_results = controls.select { |x| x['id'].eql?(id) }.map {|x| x['results']}
+        unique_control = controls.find { |x| x['id'].eql?(id) }
+        unique_control['results'] = collapsed_results.flatten
+        unique_controls << unique_control
+      end
+      unique_controls
+    end
+
+    def to_hdf
+      host_results = {}
+      @reports.each do | report|
+        controls = []
+        report['ReportItem'].each do | item |
+          @item = {}
+          @item['tags']               = {}
+          @item['descriptions']       = []
+          @item['refs']               = NA_ARRAY
+          @item['source_location']    = NA_HASH
+
+          # Nessus results field set are different for 'Policy Compliance' plug-in family vs other plug-in families
+          # Following if conditions capture compliance* if it exists else it will default to plugin* fields
+          # Current version covers STIG based 'Policy Compliance' results
+          # TODO Cover cases for 'Policy Compliance' results based on CIS
+          if item['compliance-reference']
+            @item['id']                 = parse_refs(item['compliance-reference'],'Vuln-ID').join.to_s
+          else
+            @item['id']                 = item['pluginID'].to_s
+          end
+          if item['compliance-check-name']
+            @item['title']              = item['compliance-check-name'].to_s
+          else
+            @item['title']              = item['pluginName'].to_s
+          end
+          if item['compliance-info']
+            @item['desc']              = item['compliance-info'].to_s
+          else
+            @item['desc']              = format_desc(item).to_s
+          end
+          if item['compliance-reference']
+            @item['impact']            = impact(parse_refs(item['compliance-reference'],'CAT').join.to_s)
+          else
+            @item['impact']            = impact(item['severity']) 
+          end
+          if item['compliance-reference']
+            # TODO: Cover cases where 800-53 refs are not provided in nessus `compliance-reference` field
+            @item['tags']['nist']     = parse_refs(item['compliance-reference'],'800-53') << DEFAULT_NIST_REV
+          else
+            @item['tags']['nist']     = nist_tag(item['pluginFamily'],item['pluginID'])
+          end
+          if item['compliance-solution']
+            # TODO: Cover cases where 800-53 refs are not provided in nessus `compliance-reference` field
+            @item['tags']['nist']     = parse_refs(item['compliance-reference'],'800-53') << DEFAULT_NIST_REV
+          else
+            @item['tags']['nist']     = nist_tag(item['pluginFamily'],item['pluginID'])
+          end
+          if item['compliance-solution']
+            @item['descriptions']       <<  desc_tags(item['compliance-solution'], 'check')
+          end
+
+          @item['code']               = ''
+          @item['results']            = finding(item, extract_timestamp(report))
+          controls << @item
+        end
+        controls = collapse_duplicates(controls)
+        results = HeimdallDataFormat.new(profile_name: "Nessus #{@scaninfo['policyName']}",
+                                         version: @scaninfo['version'],
+                                         title: "Nessus #{@scaninfo['policyName']}",
+                                         summary: "Nessus #{@scaninfo['policyName']}",
+                                         controls: controls,
+                                         target_id: report['name'])
+        host_results[report['name']] = results.to_hdf
+      end
+      host_results
+    end
+  end
+end

data/lib/heimdall_tools/sonarqube_mapper.rb

@@ -3,9 +3,11 @@ require 'json'
 require 'csv'
 require 'heimdall_tools/hdf'
 
+RESOURCE_DIR = Pathname.new(__FILE__).join('../../data')
+
 MAPPING_FILES = {
-  cwe: './lib/data/cwe-nist-mapping.csv'.freeze,
-  owasp: './lib/data/owasp-nist-mapping.csv'.freeze
+  cwe: File.join(RESOURCE_DIR, 'cwe-nist-mapping.csv'),
+  owasp: File.join(RESOURCE_DIR, 'owasp-nist-mapping.csv')
 }.freeze
 
 IMPACT_MAPPING = {

data/lib/heimdall_tools/zap_mapper.rb

@@ -4,7 +4,9 @@ require 'csv'
 require 'heimdall_tools/hdf'
 
 
-CWE_NIST_MAPPING_FILE = './lib/data/cwe-nist-mapping.csv'.freeze
+RESOURCE_DIR = Pathname.new(__FILE__).join('../../data')
+
+CWE_NIST_MAPPING_FILE = File.join(RESOURCE_DIR, 'cwe-nist-mapping.csv')
 
 # rubocop:disable Metrics/AbcSize
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: heimdall_tools
 version: !ruby/object:Gem::Version
-  version: 1.3.23
+  version: 1.3.28
 platform: ruby
 authors:
 - Robert Thew
@@ -10,7 +10,7 @@ authors:
 autorequire: 
 bindir: exe
 cert_chain: []
-date: 2020-03-31 00:00:00.000000000 Z
+date: 2020-05-28 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: nokogiri
@@ -116,14 +116,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '0.17'
+        version: 0.17.2
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '0.17'
+        version: 0.17.2
 - !ruby/object:Gem::Dependency
   name: bundler
   requirement: !ruby/object:Gem::Requirement
@@ -211,16 +211,21 @@ files:
 - exe/heimdall_tools
 - lib/data/cwe-nist-mapping.csv
 - lib/data/gitkeep
+- lib/data/nessus-plugins-nist-mapping.csv
 - lib/data/owasp-nist-mapping.csv
 - lib/heimdall_tools.rb
+- lib/heimdall_tools/burpsuite_mapper.rb
 - lib/heimdall_tools/cli.rb
 - lib/heimdall_tools/command.rb
 - lib/heimdall_tools/fortify_mapper.rb
 - lib/heimdall_tools/hdf.rb
 - lib/heimdall_tools/help.rb
+- lib/heimdall_tools/help/burpsuite_mapper.md
 - lib/heimdall_tools/help/fortify_mapper.md
+- lib/heimdall_tools/help/nessus_mapper.md
 - lib/heimdall_tools/help/sonarqube_mapper.md
 - lib/heimdall_tools/help/zap_mapper.md
+- lib/heimdall_tools/nessus_mapper.rb
 - lib/heimdall_tools/sonarqube_mapper.rb
 - lib/heimdall_tools/version.rb
 - lib/heimdall_tools/zap_mapper.rb