dawnscanner 2.0.0.rc3 → 2.0.0.rc4

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 72552d751163f1b0a97daa602bf4251d879e66155cb213aaecb0e7b4e9654af1
-  data.tar.gz: 2f50331bf792286959cd79efb3fc36f106548e11f7c338318b84779b5cb3aa4a
+  metadata.gz: b9ae4a53a59b132a6ce6c85407f0d4fddd88eda75958474a1aee0ce2369b4cf4
+  data.tar.gz: '01479eaa5129162d83ddcce897f74f250b5fe4b0cbe5e7c5e3bb2444b9cffce8'
 SHA512:
-  metadata.gz: 4e1d78d9d5e1b8c407b560a5199d0a3536382e13ce09bda3de2e98a9bffde2e8a0b472acb90a18b16e252355bbfbdde35180072c034500388389613dabd8497a
-  data.tar.gz: 600b924b3801499ef17e75fbfd5b9e01f9973394f7a76d34828f2702ae76ad4b0364b75403fc0c12820089f9b56a55ec1d2c42a8b7c8c57d5e434f96dac621f1
+  metadata.gz: b2ddc397e425922f848612c1244b8e0ace964a766673b9a6f70317e441c00dd35c2687de1040fcfa0ef5203c5f7304d274e4a9091637670a8b5c3ccefec33804
+  data.tar.gz: '0345397057005fcd021910298c1befb282eddfcad7e13f876e5d36a8715579d71990dd18bcb9cd3ca3f8387fe99e962ad8e97224d5060dad1f711f49a82e62d2'

data/.gitignore

@@ -19,3 +19,4 @@ test/tmp
 test/version_tmp
 tmp
 db/*
+tags

data/.ruby-version

@@ -1 +1 @@
-2.5.1
+3

data/Changelog.md

@@ -16,9 +16,12 @@ _latest update: mer 28 nov 2018, 11.03.53, CET_
 * Removed rake osvdb[name] and rake cve[name] tasks
 * Adding telemetry
 * Dawn::Utils include refactory. Now it's available application wide
-* debug information refactory. 
+* debug information refactory.
 * engine class, apply_all method now accepts an optional parameter containing a
   list of security checks to be excluded (issue #230).
+* Fix issue #244. Now the KB path is no more hardcoded but it is relative to
+  $HOME and 'dawnscanner' folder where results are stored.
+* Fix issue #245. Pattern matching check is skipped on empty files.
 
 ## Version 1.6.9 - codename: Tow Mater (2018-11-28)
 

data/LICENSE.txt

@@ -1,4 +1,4 @@
-Copyright (c) 2013-2019 Paolo Perego
+Copyright (c) 2013-2021 Paolo Perego
 
 MIT License
 

data/README.md

@@ -1,12 +1,10 @@
 # Dawnscanner - The raising security scanner for ruby web applications
 
-dawnscanner is a source code scanner designed to review your ruby code for
+dawnscanner is a source code scanner designed to review your web applications for
 security issues.
 
-dawnscanner is able to scan plain ruby scripts (e.g. command line applications) but
-all its features are unleashed when dealing with web applications source code.
-dawnscanner is able to scan major MVC (Model View Controller) frameworks, out of the
-box:
+dawnscanner is able to scan web applications written in Ruby and it supports all 
+major MVC (Model View Controller) frameworks, out of the box:
 
 * [Ruby on Rails](http://rubyonrails.org)
 * [Sinatra](http://www.sinatrarb.com)

data/Rakefile

@@ -44,7 +44,7 @@ namespace :version do
       f.puts("module Dawn")
 
       puts "#{branch_name}|"
-      if branch_name != "master"
+      if branch_name != "main"
         av = version.split('.')
         f.puts "    VERSION = \"#{av[0]}.#{av[1]}.#{commit_hash.chop}\""
         f.puts "    CODENAME = \"#{codename.lstrip!.chop}\""
@@ -74,7 +74,8 @@ namespace :kb do
   desc 'Pack the library for shipping'
 
   task :pack do
-    YAML_KB = File.join(Dir.pwd, 'db')
+    YAML_KB = File.join(Dir.home, "dawnscanner", 'db')
+    FileUtils.mkdir_p(YAML_KB)
     __kb_pack
   end
 
@@ -199,7 +200,7 @@ def __kb_pack
 
 
   open(File.join(YAML_KB, "kb.yaml"), 'w') do |f|
-    f.puts(Dawn::KnowledgeBaseExperimental.kb_descriptor)
+    f.puts(Dawn::KnowledgeBase.kb_descriptor)
   end
   puts "kb.yaml created"
   system "shasum -a 256 #{YAML_KB}/kb.yaml > #{YAML_KB}/kb.yaml.sig"

data/Roadmap.md

@@ -11,12 +11,24 @@ The document is _dynamic_ and feature schedule may vary. If you do need a
 feature to be included sooner, please open an [issue on
 github](https://github.com/thesp0nge/dawnscanner/issues/new)
 
-_latest update: Thu Dec  3 18:29:11 CET 2015_
+_latest update: mar 7 mag 2019, 17:48:53, CEST_
 
 
-## Version 1.5.5 (est. Jan 2016)
+* Add Hanami support
+* Add node.js support
+
+* Add Maven support (this will lead of creating the skeleton of a
+  dawnscanner-java gem. I will decide later if it will stay with the core or if
+  it will be a separted gem plugging into dawnscanner as plugin).
+* Add support for pure Rack applications
+* Add basic support for Javascript. At the beginning, it will be a signature
+  based support. dawnscanner will try to detect the js library version by using
+  SHA hashing functions, comparing it with fingerprint of vulnerable libraies.
+  Of course, this will lead to false negatives if a user tamper the original
+  JS. We must consider also minified versions and we're not able to deal with
+  obfuscated code.
+
 
-* close all issues on github markedsfor milestone 1.5.5
 * Issue #131 - Adding a check for OSVDB 119927 : http Gem for Ruby SSL Certificate Validation MitM Spoofing
 * Issue #119 - Adding a check for OSVDB 114641 : Ruby lib/rexml/entity.rb NULL String Handling Recursive XML External Entity (XXE) Expansion Resource Consumption Remote DoS
 * Issue #118 - Adding a check for OSVDB 113965 : Sprockets Gem for Ruby Unspecified Request Handling File Enumeration
@@ -39,24 +51,6 @@ _latest update: Thu Dec  3 18:29:11 CET 2015_
 * adding test for CVE-2011-4969  XSS in jquery < 1.6.2
 
 
-## Version 2.0.0 (est. June 2016)
-
-### New supported frameworks
-
-* Add Lotus support
-* Add Maven support (this will lead of creating the skeleton of a
-  dawnscanner-java gem. I will decide later if it will stay with the core or if
-  it will be a separted gem plugging into dawnscanner as plugin).
-* Add support for pure Rack applications
-* Add basic support for Javascript. At the beginning, it will be a signature
-  based support. dawnscanner will try to detect the js library version by using
-  SHA hashing functions, comparing it with fingerprint of vulnerable libraies.
-  Of course, this will lead to false negatives if a user tamper the original
-  JS. We must consider also minified versions and we're not able to deal with
-  obfuscated code.
-
-### New checks
-
 * Add a language check. It will handle a ruby script as input and a
   ruby\_parser line as unsafe pattern. It will compile the ruby and look for
   the unsafe pattern
@@ -67,7 +61,6 @@ _latest update: Thu Dec  3 18:29:11 CET 2015_
   dawnscanner the proper way. This is a dynamic tests that it must be run in a
   static way, looking for the public directory for old and backup files
   pattern.
-* Security checks for vulnerabilities out until 31 May 2016.
 
 ### New features
 
@@ -115,7 +108,6 @@ _latest update: Thu Dec  3 18:29:11 CET 2015_
 ## Version 2.5.0 (est. December 2016)
 
 * Add automatic mitigation patch generation for Ruby
-* Add node.js support
 * Add Opal support
 
 ## Long term Roadmap

data/VERSION

@@ -12,4 +12,4 @@
 # |   "Guido"       |  x.x.0  |
 # |   "Luigi"       |  x.x.0  |
 # | "Doc Hudson"    |  x.x.0  |
-2.0.0.rc3 - Finn McMissile
+2.0.0.rc4 - Finn McMissile

data/bin/dawn

@@ -8,6 +8,7 @@ require 'justify'
 require 'dawnscanner'
 
 APPNAME = File.basename($0)
+
 LIST_KNOWN_FRAMEWORK  = %w(rails sinatra padrino)
 VALID_OUTPUT_FORMAT   = %w(console json csv html)
 
@@ -23,7 +24,7 @@ $logger.formatter = proc do |severity, datetime, progname, msg|
     date_format = datetime.strftime("%Y-%m-%d %H:%M:%S")
     if severity == "INFO" or severity == "WARN"
         "[#{date_format}] #{severity}  (dawn): #{msg}\n"
-    else        
+    else
         "[#{date_format}] #{severity} (dawn): #{msg}\n"
     end
 end

data/checksum/dawnscanner-2.0.0.rc3.gem.sha1

@@ -0,0 +1 @@
+55641656f0a1979b283c10ac526f00f5fc449d89

data/docs/CNAME

@@ -0,0 +1 @@
+www.dawnscanner.org

data/docs/_config.yml

@@ -0,0 +1 @@
+theme: jekyll-theme-cayman

data/lib/dawn/cli/dawn_cli.rb

@@ -3,31 +3,43 @@ require 'dawn/utils'
 
 module Dawn
   module Cli
-    
     # This class is responsible for the "dawn kb" command and related
     # subcommands.
     class Kb < Thor
       package_name "dawnscanner"
-      desc "search", "Searches the knowledge base for a given security test"
-      def search(string)
+      desc "find", "Searches the knowledge base for a given security test"
+      def find(string)
         kb = Dawn::KnowledgeBase.instance
         kb.find(string)
       end
 
+      desc "lint", "Checks knowledge base content for correcteness"
+      def lint
+        kb = Dawn::KnowledgeBase.instance
+        kb.load(true)
+      end
+
+      desc "unpack", "Unpacks security checks in KB library path"
+      def unpack
+        $logger.helo APPNAME, Dawn::VERSION
+        kb = Dawn::KnowledgeBase.instance
+        kb.unpack
+        $logger.bye
+        Kernel.exit(0)
+      end
+
       desc "status", "Checks the status of the knowledge base"
       def status
         $logger.helo APPNAME, Dawn::VERSION
-        Dawn::KnowledgeBase.path="/home/thesp0nge/src/hacking/dawnscanner/db"
         Dawn::KnowledgeBase.enabled_checks=[:bulletin, :generic_check]
         kb = Dawn::KnowledgeBase.instance
         kb.load
         if kb.security_checks.empty?
           $logger.error(kb.error)
         end
-        
         $logger.info("" + kb.security_checks.count.to_s + " security checks loaded")
         if kb.is_packed?
-          $logger.error "The knowledge base is packed. It must be unpacked with the 'unpack' command before it can be used" 
+          $logger.error "The knowledge base is packed. It must be unpacked with the 'unpack' command before it can be used"
         end
         $logger.bye
         Kernel.exit(0)
@@ -50,14 +62,15 @@ module Dawn
       desc "kb SUBCOMMAND ... ARGS", "Interacts with the knowledge base"
       subcommand "kb", Dawn::Cli::Kb
 
-      desc "scan", "scans a ruby written application for security issues"
-      option :config_file
-      method_option :gemfile, :type=>:boolean, :default=>true, :aliases => "-G", :desc => "uses Gemfile.lock to detect MVC"
-      method_option :skip, :type=>:array, :aliases => "-S", :desc => "specify a list of security checks to be skipped"
-      option :exit_on_warn, :type=>:boolean
-      option :count, :type=>:boolean
-      option :s
-      option :output
+      desc "scan", "scans a ruby written web application for security issues"
+      method_option :config_file,   :type=>:string,   :default=>"",     :aliases => "-c", :desc=>"tells dawn to load configuration from filename"
+      method_option :gemfile,       :type=>:boolean,  :default=>true,   :aliases => "-G", :desc => "uses Gemfile.lock to detect MVC"
+      method_option :skip,          :type=>:array,                      :aliases => "-S", :desc => "specify a list of security checks to be skipped"
+      method_option :report_format, :type=>:string,                     :aliases => "-F", :desc=>"specify the report format (text, html, json). Default is plain text files."
+      method_option :exit_on_warn,  :type=>:boolean,  :default=>false,  :aliases => "-z", :desc =>"return number of found vulnerabilities as exit code"
+      method_option :count,         :type=>:boolean,  :default=>false,  :aliases => "-C", :desc=>"count vulnerabilities (useful for scripts)"
+      method_option :output,        :type=>:string,                     :aliases => "-O", :desc=>"write output to a file with the name specified by the parameter"
+      method_option :dependencies,  :type=>:boolean,  :default=>false,  :aliases => "-d", :desc=>"scan only for vulnerabilities affecting dependencies in Gemfile.lock"
 
       def scan(target)
         $logger.helo APPNAME, Dawn::VERSION
@@ -70,21 +83,21 @@ module Dawn
         checks_to_be_skipped = []
         checks_to_be_skipped = options[:skip] unless options[:skip].nil?
 
-        $logger.error("#{options[:skip]}")
-
         debug_me("scanning #{target}")
 
         $config_file= Dawn::Core.find_conf(true) if options[:config_file].nil?
         $config = Dawn::Core.read_conf($config_file)
 
+        debug_me($config)
+
         $telemetry_url = $config[:telemetry][:endpoint] if $config[:telemetry][:enabled]
         debug_me("telemetry url is " + $telemetry_url) unless @telemetry_url.nil?
-        
+
         $telemetry_id = $config[:telemetry][:id] if $config[:telemetry][:enabled]
         debug_me("telemetry id is " + $telemetry_id) unless @telemetry_id.nil?
 
-        $logger.info("telemetry is disabled in config file") unless $config[:telemetry][:enabled]
- 
+        debug_me("telemetry is disabled in config file") unless $config[:telemetry][:enabled]
+
         engine = Dawn::Core.detect_mvc(target) unless options[:gemfile]
         engine = Dawn::GemfileLock.new(target) if options[:gemfile]
 
@@ -101,17 +114,20 @@ module Dawn
           end
         end
 
-      
+
         engine.load_knowledge_base
-        
+
         ret = engine.apply_all(checks_to_be_skipped)
-        if options[:output]
-          STDERR.puts (ret)? engine.vulnerabilities.count : "-1" unless options[:output] == "json"
-          STDERR.puts (ret)? {:status=>"OK", :vulnerabilities_count=>engine.count_vulnerabilities}.to_json : {:status=>"KO", :vulnerabilities_count=>-1}.to_json if options[:output] == "json"
+
+
+        if options[:report_format] and options[:report_format].eql? "json"
+          STDERR.puts (ret)? {:status=>"OK", :vulnerabilities_count=>engine.count_vulnerabilities}.to_json : {:status=>"KO", :vulnerabilities_count=>-1}.to_json
           $logger.bye
           Kernel.exit(0)
         end
 
+        $logger.info("#{engine.count_vulnerabilities} issues found")
+
         Dawn::Reporter.new({:engine=>engine, :apply_all_code=>ret}).report
         $logger.bye
 

data/lib/dawn/core.rb

@@ -123,7 +123,7 @@ module Dawn
 
       # If create_if_none flag is set to true, than I'll create a config file
       # on the current directory with the default configuration.
-      conf = {:verbose=>false, :output=>"tabular", :mvc=>"", :gemfile_scan=>false, :gemfile_name=>"", :filename=>nil, :debug=>false, :exit_on_warn => false, :enabled_checks=> Dawn::Kb::BasicCheck::ALLOWED_FAMILIES}
+      conf = {:verbose=>false, :output=>"tabular", :mvc=>"", :gemfile_scan=>false, :gemfile_name=>"", :filename=>nil, :debug=>false, :exit_on_warn => false, :enabled_checks=> Dawn::Kb::BasicCheck::ALLOWED_FAMILIES, :telemetry=>{:enabled=>false, :endpoint=>"", :id=>""}}
 
       # Calculate the conf file path
       conf_path = File.expand_path('~') +'/.'+conf_name
@@ -138,7 +138,7 @@ module Dawn
     end
 
     def self.read_conf(file=nil)
-      conf = {:verbose=>false, :output=>"tabular", :mvc=>"", :gemfile_scan=>false, :gemfile_name=>"", :filename=>nil, :debug=>false, :exit_on_warn => false, :enabled_checks=> Dawn::Kb::BasicCheck::ALLOWED_FAMILIES}
+      conf = {:verbose=>false, :output=>"tabular", :mvc=>"", :gemfile_scan=>false, :gemfile_name=>"", :filename=>nil, :debug=>false, :exit_on_warn => false, :enabled_checks=> Dawn::Kb::BasicCheck::ALLOWED_FAMILIES, :telemetry=>{:enabled=>false, :endpoint=>"", :id=>""}}
       begin
         debug_me("returning a default config") if file.nil? or ! File.exist?(file)
         return conf if file.nil?

data/lib/dawn/engine.rb

@@ -39,7 +39,7 @@ module Dawn
     attr_reader :controllers
 
     # Models I don't know right now. Let them initialized as Array... we
-    # will see later 
+    # will see later
     attr_reader :models
 
     attr_accessor :debug
@@ -69,10 +69,10 @@ module Dawn
 
       set_target(dir) unless dir.nil?
 
-      
+
 
       @ruby_version = get_ruby_version if dir.nil?
-      @gemfile_lock = options[:gemfile_name] unless options[:gemfile_name].nil? 
+      @gemfile_lock = options[:gemfile_name] unless options[:gemfile_name].nil?
 
       # @stats        = gather_statistics
 
@@ -89,12 +89,12 @@ module Dawn
       end
       $logger.warn "pattern matching security checks are disabled for Gemfile.lock scan" if @name == "Gemfile.lock"
       $logger.warn "combo security checks are disabled for Gemfile.lock scan" if @name == "Gemfile.lock"
-      debug_me "engine is in debug mode" 
+      debug_me "engine is in debug mode"
 
       if @name == "Gemfile.lock" && ! options[:guessed_mvc].nil?
         # since all checks relies on @name a Gemfile.lock engine must
         # impersonificate the engine for the mvc it was detected
-        debug_me "now I'm switching my name from #{@name} to #{options[:guessed_mvc][:name]}" 
+        debug_me "now I'm switching my name from #{@name} to #{options[:guessed_mvc][:name]}"
         $logger.err "there are no connected gems... it seems Gemfile.lock parsing failed" if options[:guessed_mvc][:connected_gems].empty?
         @name = options[:guessed_mvc][:name]
         @mvc_version = options[:guessed_mvc][:version]
@@ -111,7 +111,7 @@ module Dawn
       # load_knowledge_base
     end
 
-    
+
 
     def detect_views
       []
@@ -125,10 +125,10 @@ module Dawn
 
     def build_view_array(dir)
 
-      return [] unless File.exist?(dir) and File.directory?(dir) 
+      return [] unless File.exist?(dir) and File.directory?(dir)
 
       ret = []
-      Dir.glob(File.join("#{dir}", "*")).each do |filename| 
+      Dir.glob(File.join("#{dir}", "*")).each do |filename|
         ret << {:filename=>filename, :language=>:haml} if File.extname(filename) == ".haml"
       end
 
@@ -151,9 +151,9 @@ module Dawn
         # does the target use rvm?
         ver = get_rvm_ruby_ver if  ver[:version].empty? && ver[:patchlevel].empty?
         # take the running ruby otherwise
-        ver = {:engine=>RUBY_ENGINE, :version=>RUBY_VERSION, :patchlevel=>"p#{RUBY_PATCHLEVEL}"} if ver[:version].empty? && ver[:patchlevel].empty? 
+        ver = {:engine=>RUBY_ENGINE, :version=>RUBY_VERSION, :patchlevel=>"p#{RUBY_PATCHLEVEL}"} if ver[:version].empty? && ver[:patchlevel].empty?
       else
-        ver = {:engine=>RUBY_ENGINE, :version=>RUBY_VERSION, :patchlevel=>"p#{RUBY_PATCHLEVEL}"} 
+        ver = {:engine=>RUBY_ENGINE, :version=>RUBY_VERSION, :patchlevel=>"p#{RUBY_PATCHLEVEL}"}
 
       end
 
@@ -174,10 +174,8 @@ module Dawn
     def load_knowledge_base(enabled_checks=[])
       debug_me("load_knowledge_base called. Enabled checks are: #{enabled_checks}")
 
-      Dawn::KnowledgeBase.path="/home/thesp0nge/src/hacking/dawnscanner/db"
       Dawn::KnowledgeBase.enabled_checks=[:bulletin, :generic_check]
       kb = Dawn::KnowledgeBase.instance
-      $logger.warn "KB path is forced @ /home/thesp0nge/src/hacking/dawnscanner/db"
 
       @checks=kb.load
       debug_me("#{@checks.count} checks loaded")
@@ -192,13 +190,13 @@ module Dawn
       return ver unless has_gemfile_lock?
 
       my_dir = Dir.pwd
-      Dir.chdir(@target) 
+      Dir.chdir(@target)
       lockfile = Bundler::LockfileParser.new(Bundler.read_file("Gemfile.lock"))
       lockfile.specs.each do |s|
         # detecting MVC version using @name in case of sinatra, padrino or rails engine
-        ver= s.version.to_s if s.name == @name && @name != "Gemfile.lock" 
+        ver= s.version.to_s if s.name == @name && @name != "Gemfile.lock"
         # detecting MVC version using @force in case of Gemfile.lock engine
-        ver= s.version.to_s if s.name == @force.to_s && @name == "Gemfile.lock" 
+        ver= s.version.to_s if s.name == @force.to_s && @name == "Gemfile.lock"
         @connected_gems << {:name=>s.name, :version=>s.version.to_s}
       end
       Dir.chdir(my_dir)
@@ -297,7 +295,7 @@ module Dawn
     def have_a_telemetry_id?
       debug_me ($telemetry_id != ""  and ! $telemetry_id.nil?)
       return ($telemetry_id != ""  and ! $telemetry_id.nil?)
-      
+
     end
 
     def get_a_telemetry_id
@@ -312,6 +310,11 @@ module Dawn
     end
 
     def telemetry
+      unless $config[:telemetry][:enabled]
+        debug_me("telemetry is disabled")
+        return false
+      end
+
       unless have_a_telemetry_id?
         $telemetry_id = get_a_telemetry_id
         $config[:telemetry][:id] = $telemetry_id
@@ -321,11 +324,11 @@ module Dawn
       end
 
       debug_me("Telemetry ID is: " + $telemetry_id)
-      
+
       uri=URI.parse($telemetry_url+"/"+$telemetry_id)
       header = {'Content-Type': 'text/json'}
-      tele = { "kb_version" => Dawn::KnowledgeBase::VERSION , 
-               "ip" => Socket.ip_address_list.detect{|intf| intf.ipv4_private?}.ip_address, 
+      tele = { "kb_version" => Dawn::KnowledgeBase::VERSION ,
+               "ip" => Socket.ip_address_list.detect{|intf| intf.ipv4_private?}.ip_address,
                "message"=> Dawn::KnowledgeBase
             }
       http = Net::HTTP.new(uri.host, uri.port)
@@ -363,7 +366,7 @@ module Dawn
       end
 
       @checks.each do |check|
-        if checks_to_be_skipped.include?(check.name) 
+        if checks_to_be_skipped.include?(check.name)
           $logger.info("skipping security check #{check.name}")
         else
           _do_apply(check)
@@ -439,7 +442,8 @@ module Dawn
 
         check.ruby_version  = @ruby_version[:version]
         check.detected_ruby = @ruby_version                           if check.kind == Dawn::KnowledgeBase::RUBY_VERSION_CHECK
-        check.dependencies  = self.connected_gems                     if check.kind == Dawn::KnowledgeBase::DEPENDENCY_CHECK
+        check.dependencies  = self.connected_gems                     if check.kind == Dawn::KnowledgeBase::DEPENDENCY_CHECK or
+                                                                          check.kind == Dawn::KnowledgeBase::UNSAFE_DEPENDENCY_CHECK
         check.root_dir      = self.target                             if check.kind == Dawn::KnowledgeBase::PATTERN_MATCH_CHECK
         check.options       = {:detected_ruby => self.ruby_version,
                                :dependencies => self.connected_gems,

data/lib/dawn/kb/operating_system_check.rb

@@ -1,6 +1,6 @@
   module Dawn
     module Kb
-      module OperatingSystemCheck
+      class OperatingSystemCheck
         include BasicCheck
 
         # safe_os is an Hash with this form {:family=>"", :vendor=>"", :version=>""}

data/lib/dawn/kb/pattern_match_check.rb

@@ -25,7 +25,7 @@ module Dawn
 
       EXCLUSION_LIST = [
         "tags",
-        "vendor/bundle", 
+        "vendor/bundle",
         "features",
         "specs",
         "test"
@@ -38,10 +38,10 @@ module Dawn
         @attack_pattern_is_regex  = false
         @glob                     = "**"
         @attack_pattern           = options[:attack_pattern] unless options[:attack_pattern].nil?
-        @negative_search          = options[:negative_search] unless options[:negative_search].nil? 
-        @avoid_comments           = options[:avoid_comments] unless options[:avoid_comments].nil? 
-        @evidences                = options[:evidences] unless options[:evidences].nil? 
-        @attack_pattern_is_regex  = options[:attack_pattern_is_regex] unless options[:attack_pattern_is_regex].nil? 
+        @negative_search          = options[:negative_search] unless options[:negative_search].nil?
+        @avoid_comments           = options[:avoid_comments] unless options[:avoid_comments].nil?
+        @evidences                = options[:evidences] unless options[:evidences].nil?
+        @attack_pattern_is_regex  = options[:attack_pattern_is_regex] unless options[:attack_pattern_is_regex].nil?
         @glob                     = File.join(@glob, options[:glob]) unless options[:glob].nil?
         debug_me("EVIDENCES ARE #{@evidences.inspect}")
       end
@@ -60,6 +60,7 @@ module Dawn
         Dir.glob(File.join("#{root_dir}", @glob)).each do |filename|
           debug_me("#{File.basename(__FILE__)}@#{__LINE__}: analyzing #{filename}: search is #{@negative_search}")
           matches = []
+          raise ArgumentError.new("skipping empty file") if File.zero?(filename)
           begin
             matches = run(load_file(filename)) if File.exists?(filename) && File.file?(filename) && ! File.binary?(filename) && ! must_exclude?(filename)
             found = ! matches.empty?
@@ -84,17 +85,17 @@ module Dawn
         return ret_value
       end
 
-      private 
+      private
       def string_to_array(par)
         return par if par.class == Array
-        %w(par)  
+        %w(par)
       end
 
       def load_file(filename)
 
         f = File.open(filename)
         lines = f.readlines
-        f.close 
+        f.close
 
         lines
       end

data/lib/dawn/kb/unsafe_depedency_check.rb

@@ -0,0 +1,44 @@
+module Dawn
+  module Kb
+    # While working on the KB rebase, fetching data from NVD API, I suddenly
+    # realize I must change the way a vulnerable dependency must be handled.
+    # Instead of changing what is working right now, I'll add a new dependency
+    # check ruby class
+    # NVD bulletins lists versions that are vulnerable and it would break
+    # automatism adding a post data fetching step to realize which is the first
+    # safe version.
+    #
+    # This class will handle a dependency name, the version found in
+    # Gemfile.lock and an array of vulnerable versions. If the version found is
+    # in the array, than the vuln? method returns true.
+    # This is an approach far more easy rathern than the one chosen in the past.
+    class UnsafeDependencyCheck
+      include BasicCheck
+
+      attr_accessor :dependencies
+      attr_accessor :vulnerable_version_array
+
+      def initialize(options)
+        super(options)
+      end
+
+      def vuln?
+        ret = false
+
+        # 20210325: I know... a single check handles a single dependency so,
+        # this should not be an array. This involves too many underlying
+        # changes one day I'll make.
+        @dependencies.each do |dep|
+          unless @vulnerable_version_array.nil? or @vulnerable_version_array.empty?
+            if dep[:name] == @vulnerable_version_array[0][:name]
+              return false if @vulnerable_version_array[0][:version].nil? or @vulnerable_version_array[0][:version].empty?
+              return true if @vulnerable_version_array[0][:version].include? dep[:version]
+            end
+          end
+        end
+
+        return false
+      end
+    end
+  end
+end

data/lib/dawn/knowledge_base.rb

@@ -1,5 +1,7 @@
 require 'singleton'
 
+require 'rubygems/package'
+
 # For HTTPS communication to check for KB updates and to fetch them
 require 'net/http'
 require 'uri'
@@ -19,12 +21,10 @@ require "dawn/kb/combo_check"
 require "dawn/kb/version_check"
 require "dawn/kb/deprecation_check"
 require "dawn/kb/rubygem_check"
+require "dawn/kb/unsafe_depedency_check"
 
 module Dawn
-  # This is the YAML powered experimental knowledge base
-  #
-  # When the old KB format, using Ruby classes will be marked as deprecated,
-  # than this one will be the official.
+  # This is the YAML powered knowledge base
   #
   # Dawnscanner KB will be a bunch of YAML file, stored in a hierachy of
   # directories resembling security checks family. A digital signature will be
@@ -46,7 +46,7 @@ module Dawn
   #
   # Example
   #
-  # require "dawn/knowledge_base_experimental"
+  # require "dawn/knowledge_base"
   #
   # ...
   #
@@ -54,22 +54,22 @@ module Dawn
   # d.update if d.update?
   # d.load
   #
-  # Last update: gio 29 nov 2018, 17.34.57, CET
+  # Last update: Mon Mar 22 05:08:55 PM CET 2021
   class KnowledgeBase
     include Singleton
 
-    @@path = ""
     @error = ""
     @@enabled_checks = [:generic_check, :code_quality, :bulletin, :code_style, :owasp_top_10]
 
 
-    GEM_CHECK           = :rubygem_check
-    DEPENDENCY_CHECK    = :dependency_check
-    PATTERN_MATCH_CHECK = :pattern_match_check
-    RUBY_VERSION_CHECK  = :ruby_version_check
-    OS_CHECK            = :os_check
-    COMBO_CHECK         = :combo_check
-    CUSTOM_CHECK        = :custom_check
+    GEM_CHECK               = :rubygem_check
+    DEPENDENCY_CHECK        = :dependency_check
+    UNSAFE_DEPENDENCY_CHECK = :unsafe_dependency_check
+    PATTERN_MATCH_CHECK     = :pattern_match_check
+    RUBY_VERSION_CHECK      = :ruby_version_check
+    OS_CHECK                = :os_check
+    COMBO_CHECK             = :combo_check
+    CUSTOM_CHECK            = :custom_check
 
     REMOTE_KB_URL_PREFIX  = "https://dawnscanner.org/data/"
     FILES = %w(kb.yaml bulletin.tar.gz generic_check.tar.gz owasp_ror_cheatsheet.tar.gz code_style.tar.gz code_quality.tar.gz owasp_top_10.tar.gz signatures.tar.gz)
@@ -87,7 +87,10 @@ module Dawn
         $logger = Logger.new(STDOUT)
         $logger.helo "knowledge-base-experimental", Dawn::VERSION
       end
-      @path=@@path
+      @path = default_path
+      @path = options[:path] if options[:path]
+      FileUtils.mkdir_p(@path)
+
       @enabled_checks = @@enabled_checks
 
       debug_me "KB root path is #{@path}"
@@ -97,9 +100,13 @@ module Dawn
       @@enabled_checks=checks
     end
 
+    def default_path
+      @path = File.join(Dir.home, 'dawnscanner', 'kb')
+      return @path
+    end
 
     def self.path= path_name
-      @@path=path_name
+      @path=path_name
     end
 
     def is_packed?
@@ -116,8 +123,29 @@ module Dawn
     end
 
     def unpack
-      $logger.warn "unpack is not yet implemented"
-
+      # https://weblog.jamisbuck.org/2015/7/23/tar-gz-in-ruby.html
+      FILES.each do |f|
+        full_name = File.join(path,f)
+        if File.file?(full_name) and File.extname(full_name).eql?('.gz')
+          File.open(full_name, "rb") do |file|
+            Zlib::GzipReader.wrap(file) do |gz|
+              Gem::Package::TarReader.new(gz) do |tar|
+                tar.each do |entry|
+                  if entry.file?
+                    FileUtils.mkdir_p(File.dirname(File.join(path, entry.full_name)))
+                    File.open(File.join(path, entry.full_name), "wb") do |f|
+                      f.write(entry.read)
+                    end
+                    File.chmod(entry.header.mode, File.join(path,entry.full_name))
+                  end
+                end
+              end
+            end
+          end
+        else
+          $logger.warn("can't open " + f)
+        end
+      end
     end
 
     def self.kb_descriptor
@@ -153,7 +181,10 @@ module Dawn
     #
     # Returns an array of security checks, matching the mvc to be reviewed and
     # the enabled check list or an empty array if an error occured.
-    def load
+    def load(lint=false)
+      good    =0
+      invalid =0
+
       @security_checks = []
       # $path = File.join(Dir.pwd, "db")
 
@@ -178,10 +209,20 @@ module Dawn
           $logger.warn "Missing check directory #{dir}"
         else
           Dir.glob(dir+"/**/*.yml").each do |f|
-            data = YAML.load_file(f)
-            @security_checks << data
+            begin
+              data = YAML.load_file(f)
+              @security_checks << data
+              good+=1
+              $logger.info("#{File.basename(f)} loaded") if lint
+            rescue Exception => e
+              $logger.error(e.message)
+              invalid+=1
+            end
           end
+        end
 
+        if lint
+          $logger.info("#{invalid} invalid checks out of #{good+invalid}")
         end
 
 

data/lib/dawn/reporter.rb

@@ -199,7 +199,8 @@ module Dawn
         # 1_a) Third party gem vulnerabilities
         rows = []
         @engine.vulnerabilities.each do |vuln|
-          rows << [vuln[:name].justify(10), vuln[:severity], vuln[:message].justify(30), vuln[:remediation].justify(15), vuln[:evidences].join.justify(15)]
+          $logger.error(vuln)
+          rows << [vuln[:name]&.justify(10), vuln[:severity], vuln[:message]&.justify(30), vuln[:remediation]&.justify(15), vuln[:evidences].join&.justify(15)]
           rows << :separator
         end
         table = Terminal::Table.new :title=>"Vulnerabilities", :headings=>['Issue', 'Severity', 'Description', 'Solution', 'Evidences'], :rows=>rows

data/lib/dawn/version.rb

@@ -1,7 +1,7 @@
 module Dawn
-  VERSION = "2.0.0.rc3"
-  CODENAME = "Finn McMissile"
-  RELEASE = "(development)"
-  BUILD = "2"
-  COMMIT = "g8c963e9"
+    VERSION = "2.0.0.rc4"
+    CODENAME = "Finn McMissile"
+    RELEASE = "20210406"
+    BUILD = "26"
+    COMMIT = "g9f7c8c3"
 end

data/spec/lib/kb/codesake_unsafe_dependency_check_spec.rb

@@ -0,0 +1,29 @@
+require 'spec_helper'
+
+
+describe "The security check for gem unsafe dependency should" do
+  before(:all) do
+    @check =  YAML.load_file("./spec/lib/kb/dependency_check.yml")
+    @check.debug=true
+    puts @check.vulnerable_version_array
+  end
+
+  it "fires if vulnerable 0.5.0 version is detected" do
+    @check.dependencies = [{:name=>"acme-gem", :version=>'0.5.0'}]
+    expect(@check.vuln?).to    eq(true)
+  end
+  it "fires if vulnerable 1.3.2 version is found" do
+    @check.dependencies = [{:name=>"acme-gem", :version=>'1.3.2'}]
+    expect(@check.vuln?).to    eq(true)
+  end
+
+  it "fires if vulnerable 3.4.0 version is found" do
+    @check.dependencies = [{:name=>"acme-gem", :version=>'3.4.0'}]
+    expect(@check.vuln?).to    eq(true)
+  end
+
+  it "doesn't fire if not vulnerable 3.0.0 version is found" do
+    @check.dependencies = [{:name=>"acme-gem", :version=>'3.0.0'}]
+    expect(@check.vuln?).to    eq(false)
+  end
+end

data/spec/lib/kb/dependency_check.yml

@@ -0,0 +1,29 @@
+--- !ruby/object:Dawn::Kb::UnsafeDependencyCheck
+applies:
+- rails
+- sinatra
+- padrino
+title: A test here
+cvss: '(AV:L/AC:L/Au:S/C:N/I:C/A:C)'
+cve: 'CVE-2021-99999'
+owasp: A9
+release_date: '25/03/2021'
+kind: :unsafe_dependency_check
+message: |-
+  Lorem ipsum dolor sit amet, consectetur adipiscing elit. Curabitur nisi turpis, tincidunt rhoncus leo sed, euismod sollicitudin nisl. In a arcu accumsan, fermentum quam vel, auctor risus. Nulla non sollicitudin libero. Cras hendrerit consectetur pulvinar. Vivamus ligula quam, vulputate eget justo in, varius rhoncus lorem. Nulla vel volutpat enim. Nulla hendrerit posuere tempor. Nulla in metus eget lacus tempor sollicitudin sed et dolor. Ut interdum volutpat felis, ac bibendum mauris volutpat ut. Etiam posuere justo ex, nec faucibus orci suscipit sit amet. Vivamus rutrum massa fermentum mi pellentesque vehicula. Nullam elementum urna mauris, nec cursus risus convallis vel. Nulla consectetur enim ut magna rutrum, et mollis ante auctor. Etiam accumsan in lacus et ultricies. Morbi ullamcorper velit a ipsum egestas, quis laoreet lectus placerat. Maecenas nunc augue, pulvinar non ligula ac, maximus venenatis mi.
+
+remediation: |-
+  Lorem ipsum dolor sit amet, consectetur adipiscing elit. Suspendisse et metus blandit, viverra ante a, auctor urna. Integer eget est ac nisl bibendum pharetra. Vivamus rhoncus neque vitae felis congue luctus. Praesent vitae lobortis mi. Nulla malesuada elit dictum tincidunt volutpat. Quisque tincidunt lorem nec eros ullamcorper lobortis. Nunc in felis sit amet purus sollicitudin tincidunt. Sed semper sapien nisi, non rutrum orci ultricies eget. Integer neque mauris, gravida ac varius nec, tincidunt consequat turpis. Fusce nisi metus, iaculis a eros eget, interdum sodales lectus. Pellentesque purus nisi, venenatis ut quam vitae, lacinia tristique turpis. Morbi sed maximus odio, et interdum risus. Duis nec congue lacus. Nunc sed elit a leo fermentum feugiat a aliquam arcu.
+
+severity: :critical
+priority: :high
+check_family: :bulletin
+vulnerable_version_array:
+- :name: 'acme-gem'
+  :version:
+  - '0.5.0'
+  - '0.9.0'
+  - '0.9.2'
+  - '1.3.2'
+  - '2.9.0'
+  - '3.4.0'

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: dawnscanner
 version: !ruby/object:Gem::Version
-  version: 2.0.0.rc3
+  version: 2.0.0.rc4
 platform: ruby
 authors:
 - Paolo Perego
-autorequire: 
+autorequire:
 bindir: bin
 cert_chain: []
-date: 2019-05-07 00:00:00.000000000 Z
+date: 2021-04-06 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: cvss
@@ -289,6 +289,7 @@ files:
 - checksum/dawnscanner-1.6.8.gem.sha1
 - checksum/dawnscanner-2.0.0.rc1.gem.sha1
 - checksum/dawnscanner-2.0.0.rc2.gem.sha1
+- checksum/dawnscanner-2.0.0.rc3.gem.sha1
 - code_of_conduct.md
 - dawnscanner.gemspec
 - doc/change.sh
@@ -297,8 +298,12 @@ files:
 - doc/dawn_1_2_announcement.md
 - doc/dawn_1_5_announcement.md
 - doc/dawnscanner.yml.sample
+- doc/kickstart_kb.tar.gz
 - doc/knowledge_base.rb
 - doc/new_knowledge_base_v1.0.md
+- docs/.placeholder
+- docs/CNAME
+- docs/_config.yml
 - features/dawn_complains_about_an_incorrect_command_line.feature.disabled
 - features/dawn_scan_a_secure_sinatra_app.feature.disabled
 - features/dawn_scan_a_vulnerable_sinatra_app.feature.disabled
@@ -316,6 +321,7 @@ files:
 - lib/dawn/kb/pattern_match_check.rb
 - lib/dawn/kb/ruby_version_check.rb
 - lib/dawn/kb/rubygem_check.rb
+- lib/dawn/kb/unsafe_depedency_check.rb
 - lib/dawn/kb/version_check.rb
 - lib/dawn/knowledge_base.rb
 - lib/dawn/logger.rb
@@ -340,7 +346,9 @@ files:
 - spec/lib/kb/codesake_dependency_version_check_spec.rb
 - spec/lib/kb/codesake_deprecation_check_spec.rb
 - spec/lib/kb/codesake_ruby_version_check_spec.rb
+- spec/lib/kb/codesake_unsafe_dependency_check_spec.rb
 - spec/lib/kb/codesake_version_check_spec.rb
+- spec/lib/kb/dependency_check.yml
 - spec/lib/kb/owasp_ror_cheatsheet_disabled.rb
 - spec/lib/kb/yamilize_kb_spec.rb
 - spec/spec_helper.rb
@@ -351,7 +359,7 @@ homepage: https://dawnscanner.org
 licenses:
 - MIT
 metadata: {}
-post_install_message: 
+post_install_message:
 rdoc_options: []
 require_paths:
 - lib
@@ -366,9 +374,8 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: 1.3.1
 requirements: []
-rubyforge_project: 
-rubygems_version: 2.7.7
-signing_key: 
+rubygems_version: 3.2.3
+signing_key:
 specification_version: 4
 summary: Dawnscanner is a security source code scanner for ruby powered code. It is
   crafted with love to make your sinatra, padrino and ruby on rails web applications
@@ -389,7 +396,9 @@ test_files:
 - spec/lib/kb/codesake_dependency_version_check_spec.rb
 - spec/lib/kb/codesake_deprecation_check_spec.rb
 - spec/lib/kb/codesake_ruby_version_check_spec.rb
+- spec/lib/kb/codesake_unsafe_dependency_check_spec.rb
 - spec/lib/kb/codesake_version_check_spec.rb
+- spec/lib/kb/dependency_check.yml
 - spec/lib/kb/owasp_ror_cheatsheet_disabled.rb
 - spec/lib/kb/yamilize_kb_spec.rb
 - spec/spec_helper.rb