dawnscanner 1.6.8 → 2.0.0.rc4

data/Roadmap.md

@@ -11,12 +11,24 @@ The document is _dynamic_ and feature schedule may vary. If you do need a
 feature to be included sooner, please open an [issue on
 github](https://github.com/thesp0nge/dawnscanner/issues/new)
 
-_latest update: Thu Dec  3 18:29:11 CET 2015_
+_latest update: mar 7 mag 2019, 17:48:53, CEST_
 
 
-## Version 1.5.5 (est. Jan 2016)
+* Add Hanami support
+* Add node.js support
+
+* Add Maven support (this will lead of creating the skeleton of a
+  dawnscanner-java gem. I will decide later if it will stay with the core or if
+  it will be a separted gem plugging into dawnscanner as plugin).
+* Add support for pure Rack applications
+* Add basic support for Javascript. At the beginning, it will be a signature
+  based support. dawnscanner will try to detect the js library version by using
+  SHA hashing functions, comparing it with fingerprint of vulnerable libraies.
+  Of course, this will lead to false negatives if a user tamper the original
+  JS. We must consider also minified versions and we're not able to deal with
+  obfuscated code.
+
 
-* close all issues on github markedsfor milestone 1.5.5
 * Issue #131 - Adding a check for OSVDB 119927 : http Gem for Ruby SSL Certificate Validation MitM Spoofing
 * Issue #119 - Adding a check for OSVDB 114641 : Ruby lib/rexml/entity.rb NULL String Handling Recursive XML External Entity (XXE) Expansion Resource Consumption Remote DoS
 * Issue #118 - Adding a check for OSVDB 113965 : Sprockets Gem for Ruby Unspecified Request Handling File Enumeration
@@ -39,24 +51,6 @@ _latest update: Thu Dec  3 18:29:11 CET 2015_
 * adding test for CVE-2011-4969  XSS in jquery < 1.6.2
 
 
-## Version 2.0.0 (est. June 2016)
-
-### New supported frameworks
-
-* Add Lotus support
-* Add Maven support (this will lead of creating the skeleton of a
-  dawnscanner-java gem. I will decide later if it will stay with the core or if
-  it will be a separted gem plugging into dawnscanner as plugin).
-* Add support for pure Rack applications
-* Add basic support for Javascript. At the beginning, it will be a signature
-  based support. dawnscanner will try to detect the js library version by using
-  SHA hashing functions, comparing it with fingerprint of vulnerable libraies.
-  Of course, this will lead to false negatives if a user tamper the original
-  JS. We must consider also minified versions and we're not able to deal with
-  obfuscated code.
-
-### New checks
-
 * Add a language check. It will handle a ruby script as input and a
   ruby\_parser line as unsafe pattern. It will compile the ruby and look for
   the unsafe pattern
@@ -67,7 +61,6 @@ _latest update: Thu Dec  3 18:29:11 CET 2015_
   dawnscanner the proper way. This is a dynamic tests that it must be run in a
   static way, looking for the public directory for old and backup files
   pattern.
-* Security checks for vulnerabilities out until 31 May 2016.
 
 ### New features
 
@@ -115,7 +108,6 @@ _latest update: Thu Dec  3 18:29:11 CET 2015_
 ## Version 2.5.0 (est. December 2016)
 
 * Add automatic mitigation patch generation for Ruby
-* Add node.js support
 * Add Opal support
 
 ## Long term Roadmap

data/VERSION

@@ -12,4 +12,4 @@
 # |   "Guido"       |  x.x.0  |
 # |   "Luigi"       |  x.x.0  |
 # | "Doc Hudson"    |  x.x.0  |
-1.6.8 - Tow Mater
+2.0.0.rc4 - Finn McMissile

data/bin/dawn

@@ -1,7 +1,6 @@
 #!/usr/bin/env ruby
 
 require 'bundler'
-require 'getoptlong'
 require 'json'
 require 'terminal-table'
 require 'justify'
@@ -9,289 +8,34 @@ require 'justify'
 require 'dawnscanner'
 
 APPNAME = File.basename($0)
+
 LIST_KNOWN_FRAMEWORK  = %w(rails sinatra padrino)
 VALID_OUTPUT_FORMAT   = %w(console json csv html)
 
 # Datamapper stuff
-DataMapper.setup(:default, "sqlite3://#{Dawn::Core.registry_db_name}")
-DataMapper::Logger.new(Dawn::Core.sql_log_name, :debug)
-DataMapper.finalize
-DataMapper.auto_upgrade!
+#DataMapper.setup(:default, "sqlite3://#{Dawn::Core.registry_db_name}")
+#DataMapper::Logger.new(Dawn::Core.sql_log_name, :debug)
+#DataMapper.finalize
+#DataMapper.auto_upgrade!
 
 require 'logger'
 $logger = Logger.new(STDOUT)
-$logger.datetime_format = '%Y-%m-%d %H:%M:%S'
-
-opts    = GetoptLong.new(
-  # report formatting options
-
-  [ '--ascii-tabular-report',   '-a',   GetoptLong::NO_ARGUMENT], # Deprecated in 1.5.x - To be removed in 2.0.0
-  [ '--tabular',                '-T',   GetoptLong::NO_ARGUMENT],
-  [ '--json',                   '-j',   GetoptLong::NO_ARGUMENT],
-  [ '--html',                   '-H',   GetoptLong::NO_ARGUMENT],
-  [ '--console',                '-K',   GetoptLong::NO_ARGUMENT],
-
-  # MVC forcing
-  # Deprecated in 1.5.x
-  # To be removed in 2.0.0
-  [ '--rails',                  '-r',   GetoptLong::NO_ARGUMENT],
-  [ '--sinatra',                '-s',   GetoptLong::NO_ARGUMENT],
-  [ '--padrino',                '-p',   GetoptLong::NO_ARGUMENT],
-
-  [ '--gem-lock',               '-G',   GetoptLong::REQUIRED_ARGUMENT], # Deprecated in 1.5.x - To be removed in 2.0.0
-  [ '--dependencies',           '-d',   GetoptLong::REQUIRED_ARGUMENT],
-
-  [ '--count-only',             '-C',   GetoptLong::NO_ARGUMENT],
-  [ '--exit-on-warn',           '-z',   GetoptLong::NO_ARGUMENT],
-
-  # Disable checks by family type
-  [ '--disable-cve-bulletins',          GetoptLong::NO_ARGUMENT],
-  [ '--disable-code-quality',           GetoptLong::NO_ARGUMENT],
-  [ '--disable-code-style',             GetoptLong::NO_ARGUMENT],
-  [ '--disable-owasp-ror-cheatsheet',   GetoptLong::NO_ARGUMENT],
-  [ '--disable-owasp-top-10',           GetoptLong::NO_ARGUMENT],
-
-  # Search knowledge base
-  [ '--search-knowledge-base',  '-S',   GetoptLong::REQUIRED_ARGUMENT],
-  # List stuff
-  [ '--list-knowledge-base',            GetoptLong::NO_ARGUMENT],
-  [ '--list-known-framework',           GetoptLong::NO_ARGUMENT],
-  [ '--list-known-families',            GetoptLong::NO_ARGUMENT],
-  [ '--list-scan-registry',             GetoptLong::NO_ARGUMENT],
-  # please save output to file
-  [ '--file',                   '-F',   GetoptLong::REQUIRED_ARGUMENT],
-  # specify an alternate config file
-  [ '--config-file',            '-c',   GetoptLong::REQUIRED_ARGUMENT],
-
-  # service options
-  [ '--verbose',                '-V',   GetoptLong::NO_ARGUMENT],
-  [ '--debug',                  '-D',   GetoptLong::NO_ARGUMENT],
-  [ '--version',                '-v',   GetoptLong::NO_ARGUMENT],
-  [ '--help',                   '-h',   GetoptLong::NO_ARGUMENT]
-)
-opts.quiet=true
-
-engine  = nil
-
-
-options = Dawn::Core.read_conf(Dawn::Core.find_conf(true))
-check = ""
-guess = {:name=>"", :version=>"", :connected_gems=>[]}
-
-###############################################################################
-# CLI argument start.
-#
-# Refactoring is necessary here
-###############################################################################
-begin
-opts.each do |opt, val|
-  case opt
-  when '--version'
-    puts "#{Dawn::VERSION} [#{Dawn::CODENAME}]"
-    Kernel.exit(0)
-  when '--config-file'
-    options = Dawn::Core.read_conf(val)
-  when '--disable-cve-bulletins'
-    options[:enabled_checks].delete(:bulletin)
-  when '--disable-code-quality'
-    options[:enabled_checks].delete(:code_quality)
-  when '--disable-code-style'
-    options[:enabled_checks].delete(:code_style)
-  when '--disable-owasp-ror-cheatsheet'
-    options[:enabled_checks].delete(:owasp_ror_cheatsheet)
-  when '--disable-owasp-top-10'
-    options[:enabled_checks].delete(:owasp_top_10_1)
-    options[:enabled_checks].delete(:owasp_top_10_2)
-    options[:enabled_checks].delete(:owasp_top_10_3)
-    options[:enabled_checks].delete(:owasp_top_10_4)
-    options[:enabled_checks].delete(:owasp_top_10_5)
-    options[:enabled_checks].delete(:owasp_top_10_6)
-    options[:enabled_checks].delete(:owasp_top_10_7)
-    options[:enabled_checks].delete(:owasp_top_10_8)
-    options[:enabled_checks].delete(:owasp_top_10_9)
-    options[:enabled_checks].delete(:owasp_top_10_10)
-  when '--list-known-families'
-    printf "Dawn supports following check families:\n\n"
-    puts Dawn::Kb::BasicCheck.families
-    Kernel.exit(0)
-  when '--json'
-    options[:output] = "json"
-  when '--console'
-    options[:output] = "console"
-  when '--tabular'
-    options[:output] = "tabular"
-  when '--ascii-tabular-report'
-    $logger.warn "--ascii-tabular-report' it has been deprecated. It will be removed in version 2.0.0. Please use '--tabular' instead"
-    options[:output] = "tabular"
-  when '--html'
-    options[:output] = "html"
-  when '--rails'
-    options[:mvc]=:rails
-  when '--sinatra'
-    options[:mvc]=:sinatra
-  when '--padrino'
-    options[:mvc]=:padrino
-  when '--file'
-    options[:filename] = val
-  when '--gem-lock'
-    options[:gemfile_scan] = true
-    $logger.warn "--gem-lock flag it has been deprecated. It will be removed in version 2.0.0. Please use '--dependencies' instead"
-    unless val.empty?
-      options[:gemfile_name] = val
-      guess = Dawn::Core.guess_mvc(val)
-    end
-  when '--dependencies'
-    options[:gemfile_scan] = true
-    unless val.empty?
-      options[:gemfile_name] = val
-      guess = Dawn::Core.guess_mvc(val)
-    end
-
-  when '--verbose'
-    options[:verbose]=true
-  when '--count-only'
-    options[:output] = "count"
-  when '--debug'
-    options[:debug] = true
-  when '--exit-on-warn'
-    options[:exit_on_warn] = true
-
-  when '--search-knowledge-base'
-    found = Dawn::KnowledgeBase.find(nil, val)
-    puts "#{val} found in knowledgebase." if found
-    puts "#{val} not found in knowledgebase" if ! found
-    Kernel.exit(0)
-  when '--list-scan-registry'
-    puts "#{APPNAME} scan registry\n\n"
-    Dawn::Registry.dump
-    Kernel.exit(0)
-
-  when '--list-knowledge-base'
-    Dawn::KnowledgeBase.dump(options[:verbose])
-    Kernel.exit(0)
-  when '--list-known-framework'
-    puts "Ruby MVC framework supported by #{APPNAME}:"
-    LIST_KNOWN_FRAMEWORK.each do |mvc|
-      puts "* #{mvc}"
-    end
-    Kernel.exit(0)
-  when '--help'
-    Kernel.exit(Dawn::Core.help)
-  end
-end
-rescue GetoptLong::InvalidOption => e
-  $logger.helo APPNAME, Dawn::VERSION
-  $logger.error e.message
-  Kernel.exit(Dawn::Core.help)
-end
-###############################################################################
-# CLI argument stop
-###############################################################################
-
-target=ARGV.shift
-
-target = File.expand_path(".") if target == "."
-
-$logger.helo APPNAME, Dawn::VERSION
-r = Dawn::Registry.new
-
-unless Dir.exist?(Dawn::Core.registry_db_folder)
-  FileUtils.mkdir_p(Dawn::Core.registry_db_folder)
-  $logger.info "#{Dawn::Core.registry_db_folder} created" if Dir.exist?(Dawn::Core.registry_db_folder)
-end
-
-trap("INT") { $logger.die('[INTERRUPTED]') }
-$logger.die("missing target") if target.nil? && options[:gemfile_name].empty?
-$logger.die("invalid directory (#{target})") if options[:gemfile_name].empty?  &&! Dawn::Core.is_good_target?(target)
-$logger.die("if scanning Gemfile.lock file you must not force target MVC using one from -r, -s or -p flag") if ! options[:mvc].empty? && options[:gemfile_scan]
-$logger.debug("security check enabled: #{options[:enabled_checks]}") if options[:debug]
-
-# MVC flag deprecation warnings
-$logger.warn("the --rails is deprecated and it will be removed in version 2.0.0")   if options[:mvc] == :rails
-$logger.warn("the --sinatra is deprecated and it will be removed in version 2.0.0") if options[:mvc] == :sinatra
-$logger.warn("the --padrino is deprecated and it will be removed in version 2.0.0") if options[:mvc] == :padrino
-
-
-## MVC auto detect.
-
-# Skipping MVC autodetect if it's already been done by guess_mvc when choosing
-# Gemfile.lock scan
-
-unless options[:gemfile_scan]
-  begin
-    if options[:mvc].empty?
-      engine = Dawn::Core.detect_mvc(target)
-      $logger.debug("using #{engine.class.name} engine via autodect") if options[:debug]
+$logger.formatter = proc do |severity, datetime, progname, msg|
+    date_format = datetime.strftime("%Y-%m-%d %H:%M:%S")
+    if severity == "INFO" or severity == "WARN"
+        "[#{date_format}] #{severity}  (dawn): #{msg}\n"
     else
-      engine = Dawn::Rails.new(target)      if options[:mvc] == :rails
-      engine = Dawn::Sinatra.new(target)    if options[:mvc] == :sinatra
-      engine = Dawn::Padrino.new(target)    if options[:mvc] == :padrino
-    end
-  rescue ArgumentError => e
-    r.do_save({:target=>File.basename(target), :scan_started=>DateTime.now, :scan_duration => 0, :issues_found=> -1, :output_dir=> "", :message=>e.message, :scan_status=>:failed})
-    $logger.die(e.message)
-  end
-else
-  engine = Dawn::GemfileLock.new(target, options[:gemfile_name], guess) # if options[:gemfile_scan]
-end
-
-
-if engine.nil?
-  $logger.error("MVC detection failure. Please open an issue at https://github.com/thesp0nge/dawnscanner/issues")
-  r.do_save({:target=>File.basename(target), :message=>"MVC detection failure. Please open an issue at https://github.com/thesp0nge/dawnscanner/issues", :scan_status=>:failed})
-  $logger.die("ruby framework auto detect failed. Please force if rails, sinatra or padrino with -r, -s or -p flags")
-end
-## end MVC auto detect.
-
-if options[:exit_on_warn]
-  Kernel.at_exit do
-    if engine.count_vulnerabilities != 0
-      r.do_save({:target=>engine.target, :scan_started=>engine.scan_start, :scan_duration => engine.scan_time.round(3), :issues_found=>engine.vulnerabilities.count, :output_dir=>engine.output_dir_name, :scan_status=>:completed})
-      Kernel.exit(engine.count_vulnerabilities)
+        "[#{date_format}] #{severity} (dawn): #{msg}\n"
     end
-  end
-end
-
-if options[:debug]
-  $logger.warn "putting engine in debug mode"
-  engine.debug = true
 end
 
-$logger.warn "this is a development Dawn version" if Dawn::RELEASE == "(development)"
-
-if engine.nil?
-  r.do_save({:target=>File.basename(target), :message=>"missing target framework option", :scan_status=>:failed})
-  $logger.die "missing target framework option"
-end
-
-if ! options[:gemfile_scan] && ! engine.can_apply?
-  r.do_save({:target=>File.basename(target), :message=>"nothing to do on #{target}", :scan_status=>:failed})
-  $logger.die "nothing to do on #{target}"
-end
-
-engine.load_knowledge_base(options[:enabled_checks])
-ret = engine.apply_all
+engine  = nil
+$debug=false
+$verbose=false
 
-if options[:output] == "count"
-  STDERR.puts (ret)? engine.vulnerabilities.count : "-1" unless options[:output] == "json"
-  STDERR.puts (ret)? {:status=>"OK", :vulnerabilities_count=>engine.count_vulnerabilities}.to_json : {:status=>"KO", :vulnerabilities_count=>-1}.to_json if options[:output] == "json"
 
-  r.do_save({:target=>engine.target, :scan_started=>engine.scan_start, :scan_duration => engine.scan_time.round(3), :issues_found=>engine.vulnerabilities.count, :output_dir=>engine.output_dir_name, :scan_status=>:completed})
-  $logger.bye
-  Kernel.exit(0)
-end
+check = ""
+guess = {:name=>"", :version=>"", :connected_gems=>[]}
 
-Dawn::Reporter.new({:engine=>engine, :apply_all_code=>ret, :format=>options[:output].to_sym, :filename=>options[:filename]}).report
-if (r.do_save({:target=>File.basename(engine.target),
-           :scan_started=>engine.scan_start,
-           :scan_duration => engine.scan_time.round(3),
-           :issues_found=>engine.vulnerabilities.count,
-           :output_dir=>engine.output_dir_name,
-           :scan_status=>:completed}))
-  $logger.info "#{Dawn::Core.registry_db_name} updated with scan infos"
-else
-  r.errors.each do |error|
-    $logger.error error
-  end
-end
-$logger.bye
+Dawn::Cli::DawnCli.start
+Kernel.exit(0)

data/checksum/dawnscanner-1.6.8.gem.sha1

@@ -0,0 +1 @@
+7f56617eeab5f897c910d9bfbfd54425c4856fc1

data/checksum/dawnscanner-2.0.0.rc1.gem.sha1

@@ -0,0 +1 @@
+04dc5b15006b4ee5912b789160756c57b4c9036a

data/checksum/dawnscanner-2.0.0.rc2.gem.sha1

@@ -0,0 +1 @@
+1c96f786d3683b79311855a14b8ef7d7ebe7b13d

data/checksum/dawnscanner-2.0.0.rc3.gem.sha1

@@ -0,0 +1 @@
+55641656f0a1979b283c10ac526f00f5fc449d89

data/dawnscanner.gemspec

@@ -10,17 +10,14 @@ Gem::Specification.new do |gem|
   gem.email         = ["paolo@dawnscanner.org"]
   gem.description   = %q{Dawnscanner is a security source code scanner for ruby powered code. It is especially designed for web applications, but it works also with general purpose ruby scripts. Dawn supports all major MVC frameworks like ruby on rails, padrino and sinatra; it provides more than 150 security checks with their own mitigation suggestion.}
   gem.summary       = %q{Dawnscanner is a security source code scanner for ruby powered code. It is crafted with love to make your sinatra, padrino and ruby on rails web applications secure.}
-  gem.homepage      = "http://dawnscanner.org"
+  gem.homepage      = "https://dawnscanner.org"
   gem.files         = `git ls-files`.split($/)
   gem.license       = "MIT"
   gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
   gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
   gem.require_paths = ["lib"]
 
-  gem.cert_chain  = ['certs/paolo_at_dawnscanner_dot_org.pem']
-  gem.signing_key = File.expand_path("~/.ssh/paolo_at_dawnscanner_dot_org_private_key.pem") if $0 =~ /gem\z/
-
-  gem.required_ruby_version = '>= 1.9.3'
+  gem.required_ruby_version = '>= 2.3.0'
 
   gem.add_dependency 'cvss'
   gem.add_dependency 'haml'
@@ -30,11 +27,15 @@ Gem::Specification.new do |gem|
   gem.add_dependency 'justify'
   gem.add_dependency 'logger-colors'
   gem.add_dependency 'ptools'
-  gem.add_dependency 'sqlite3'
-  gem.add_dependency 'dm-sqlite-adapter'
-  gem.add_dependency 'data_mapper'
+  gem.add_dependency 'psych'
+
+  # For CLI we will use thor
+  gem.add_dependency 'thor'
+
+  # gem.add_dependency 'sqlite3'
+  # gem.add_dependency 'datamapper'
+  # gem.add_dependency 'dm-sqlite-adapter'
 
-  # Dependencies for code stats
   # To be added back in 1.5.5
   # gem.add_dependency 'code_metrics'
   # gem.add_dependency 'metric_fu-Saikuro'

data/doc/change.sh

@@ -0,0 +1,13 @@
+LIST=`ls *.yml | sort`
+TOCHANGE=$1
+
+if [ -z $TOCHANGE ]; then
+  echo "an argument is required"
+  exit 1
+fi
+
+for i in $LIST 
+do
+  sed -i 's/object:Dawn::Kb::'`basename $i .yml`'/object:Dawn::Kb::'$TOCHANGE'/g' $i
+done
+