watobo 0.9.9.pre3 → 0.9.9

data/.yardopts

@@ -0,0 +1,24 @@
+# .
+# .yardopts
+# 
+# Copyright 2012 by siberas, http://www.siberas.de
+# 
+# This file is part of WATOBO (Web Application Tool Box)
+#        http://watobo.sourceforge.com
+# 
+# WATOBO is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation version 2 of the License.
+# 
+# WATOBO is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with WATOBO; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+# .
+--no-private
+*.xxx
+- CHANGELOG

data/CHANGELOG

@@ -1,19 +1,29 @@
 = Version 0.9.9
 == NEW
-* Time-based SQL injection module
-* new XSS module which gives a more accurate exploitability result
-* ConversationTable: values in coloumn Parameters are url-decoded
-* Added a WebCrawler Plugin based on Mechanize
-* Manual Request Editor: Url is displayed in the window title
-* Menubar items are disabled if no project is defined
+* [Module] Time-based SQL injection module
+* [Module] Rated XSS which gives a more accurate exploitability result
+* [GUI] ConversationTable: values in coloumn Parameters are url-decoded
+* [Plugin] WebCrawler - based on Mechanize
+* [GUI] Manual Request Editor: Url is displayed in the window title
+* [GUI] Menubar items are disabled if no project is defined
+* [CORE] Create SSL certificates for each target on-the-fly, now you only have to trust the internal CA once
+* [Interceptor] Rewrite/Inject Feature to Interceptor
+* [CORE] added .yml file extension for chats, findings, logs, ...
+* [Plugin] SQLmap - easy to use sqlmap interface
+* [Interceptor] Transparent Proxy Feature - only available on Linux (depends on netfilter_queue)
+* [CatalogScanner] added predefined database paths
+* [CORE] general unzipping and unchunking of server responses
 
 == Fixes
 * CA Directory is now created in WATOBO working directory '.watobo'
 * Fixed Crash on opening client-certificate dialog
+* Improved Socket communication
 * ConversationTable: GET and POST parameters are shown in the parameters coloumn 
 * TreeView-Pane: Show full conversation list when Findings tab is selected
 * Fixed a bug in parsing post parameters
-* Also some minor bugs
+* QuickScan: double scanning each module
+* the disclaimer.chk file now is written to .watobo
+* some minor bugs
 
 
 = Version 0.9.8

data/README

@@ -3,7 +3,9 @@
 WATOBO is a security tool for web applications. WATOBO is intended to enable security professionals to perform efficient (semi-automated) web application security audits.
 
 Most important features:
-* WATOBO has Session Management capabilities! You can define login scripts as well as logout signatures. So you don’t have to login manually each time you get logged out. 
+* WATOBO has Session Management capabilities! You can define login scripts as well as logout signatures. So you don’t have to login manually each time you get logged out.
+* WATOB can act as an transparent proxy
+* WATOBO has anti-CSRF features 
 * WATOBO can perform vulnerability checks out of the box. 
 * WATOBO supports Inline De-/Encoding, so you don’t have to copy strings to a transcoder and back again. Just do it inside the request/response window with a simple mouse click. 
 * WATOBO has smart filter functions, so you can find and navigate to the most interesting parts of the application easily. 
@@ -11,66 +13,8 @@ Most important features:
 * WATOBO is free software ( licensed under the GNU General Public License Version 2) 
 * It’s by siberas ;) 
 
-== Installation
-Please install Ruby 1.9.2+ first before you continue.
-
-Note: Ruby 1.8 is no longer supported!
-
-Note: WATOBO will not run under Ruby 1.8 anymore!
-
-Note: Please upgrade Ruby to 1.9.2+, because WATOBO will not run under Ruby 1.8!
-
-Note: Ruby 1.8 is crap, so get rid of it!
-
-... just want to be sure ;)
-
-=== Windows 7/Vista/XP
-  gem install watobo
-  
-=== BackTrack 5
-  gem install --user-install selenium-webdriver
-  gem install --user-install watobo
-  
-Add the following line to your ~/.bashrc file:
-  export PATH=$PATH:/root/.gem/ruby/1.9.2/bin/
-
-=== Generic Linux (with APT)
-* Install Ruby via RVM
-* Setting up a build environment for linux
-Based on Lyle Johnsons tutorial https://github.com/lylejohnson/fxruby/wiki/Setting-Up-a-Linux-Build-Environment
-  apt-get -y install ruby-full
-  apt-get -y install install zlib1g-dev libbz2-dev libpng12-dev libjpeg62-dev libtiff4-dev
-  apt-get -y install zlib1g-dev libbz2-dev libpng12-dev libjpeg62-dev libtiff4-dev
-  apt-get -y install libx11-dev libglu1-xorg-dev libxcursor-dev libxext-dev libxrandr-dev libxft2-dev
-  apt-get -y install g++
-
-* Install the Fox-Toolkit libs
-Use version 1.6.44 only. The 1.7 branch is incompatible with fxruby!
-You can download it from the fox-tookit homepage http://www.fox-toolkit.org/
-  wget http://ftp.fox-toolkit.org/pub/fox-1.6.44.tar.gz
-  tar xzvf fox-1.6.44.tar.gz
-  cd fox-1.6.44
-  ./configure
-  make
-  make install
-  cd .. 
-
-* Install the Gems
-First install the selenium-webdriver gem which is necessary on xnix platforms for the browser preview feature of watobo.
-  gem install selenium-webdriver
-Finally install the watobo gem.
-  gem install watobo
-
-== Usage
-In your command prompt start WATOBO with the command:
-  watobo_gui.rb
-
-After starting WATOBO the interception proxy is listening on localhost:8081.
-     
-Configure your browser to use WATOBO as its proxy and visit the site you want to audit.
-       
 == Documentation
-Check the online (video) tutorials at http://watobo.sourceforge.net
+Check out the online documentation and video tutorials at http://watobo.sourceforge.net
 
 == Tips & Tricks
 * On Linux you should use RVM to install Ruby (http://beginrescueend.com/rvm/install/)

data/bin/nfq_server.rb

@@ -0,0 +1,191 @@
+#!/usr/bin/ruby
+# .
+# nfq_server.rb
+# 
+# Copyright 2012 by siberas, http://www.siberas.de
+# 
+# This file is part of WATOBO (Web Application Tool Box)
+#        http://watobo.sourceforge.com
+# 
+# WATOBO is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation version 2 of the License.
+# 
+# WATOBO is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with WATOBO; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+# .
+require 'drb'
+require 'yaml'
+require 'openssl'
+
+begin
+  require "nfqueue"
+  @nfq_present = true
+rescue LoadError
+  puts "NFQUEUE not available on this system"
+  exit
+end
+
+module Watobo
+  module NFQ
+    class Connections
+      attr :nfqueue
+      def add_ssl_request(c_host, c_port, s_host, s_port)
+        ck = "#{c_host}:#{c_port}"
+        sk = "#{s_host}:#{s_port}"
+
+        begin
+
+          unless @cert_list.has_key? sk
+            if cert = acquire_cert(s_host,s_port)
+            @connections[ck] = sk
+            @cert_list[sk] = cert
+            else
+            return false
+            end
+          else
+          @connections[ck] = sk
+          end
+
+          return true
+        rescue => bang
+          puts bang
+          puts bang.backtrace
+        end
+        return false
+
+      end
+
+      def to_yaml
+        @connections.to_yaml
+      end
+
+      def info(data)
+        begin
+          ck = "#{data['host']}:#{data['port']}"
+          target_site = ''
+          cert_cn = ''
+          @netqueue_lock.synchronize do
+            if @connections.has_key? ck
+              target_site = @connections[ck]
+              if @cert_list.has_key? target_site
+                cert = @cert_list[target_site]
+                cert_cn = cert.subject.to_s.gsub(/.*=/,"")
+              end
+            end
+          end
+          r = { 'target' => target_site, 'cn' => cert_cn}
+          return r
+        rescue => bang
+          puts bang
+          puts bang.backtrace
+        end
+        return {}
+      end
+
+      def initialize
+        @connections = Hash.new
+        @cert_list = Hash.new
+        @netqueue_lock = Mutex.new
+        @dh_key = OpenSSL::PKey::DH.new(512)
+        @nfqueue = start
+      end
+
+      def acquire_cert(host, port)
+
+        begin
+          tcp_socket = TCPSocket.new( host, port )
+          tcp_socket.setsockopt( Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, 1)
+          tcp_socket.sync = true
+          ctx = OpenSSL::SSL::SSLContext.new()
+
+          ctx.tmp_dh_callback = proc { |*args|
+            @dh_key
+          }
+
+          socket = OpenSSL::SSL::SSLSocket.new(tcp_socket, ctx)
+
+          socket.connect
+          #socket.setsockopt( Socket::SOL_SOCKET, Socket::SO_KEEPALIVE, 1)
+          sk = "#{host}:#{port}"
+          cert = socket.peer_cert
+          @netqueue_lock.synchronize do
+            @cert_list[sk] = cert
+          end
+          # puts cert.subject.methods.sort
+          return cert
+
+        rescue => bang
+          puts bang
+          puts bang.backtrace if $DEBUG
+        end
+        return nil
+      end
+
+      def start
+
+        puts "starting netfilter_queue ..."
+        t = Thread.new{
+          begin
+            Netfilter::Queue.create(0) do |p|
+            #   puts ">> Netfilter Packet #" + p.id.to_s
+            #  $stdout.flush
+            #   puts p.data.class
+              raw_src = p.data[12..15]
+              raw_dst = p.data[16..19]
+              src_port = p.data[20..21].unpack("H4")[0].hex
+              dst_port = p.data[22..24].unpack("H4")[0].hex
+              # if p.data.length > 47
+              # flags = p.data[47].unpack("H*")[0].hex
+              # puts flags.to_s
+              # if flags == 2
+              #    puts  "ADD SSL REQUEST"
+              puts "NFQ >> #{get_ip_string(raw_src)}:#{src_port} -> #{get_ip_string(raw_dst)}:#{dst_port}"
+              add_ssl_request(get_ip_string(raw_src), src_port, get_ip_string(raw_dst), dst_port)
+
+              Netfilter::Packet::ACCEPT
+            end
+          rescue => bang
+            puts bang
+            puts bang.backtrace
+            # retry
+          rescue Netfilter::QueueError
+            puts "NetfilterERROR"
+            exit
+          end
+        }
+
+        t
+      end
+
+      private
+
+      def get_ip_string(raw_addr)
+        begin
+          ip = ""
+          raw_addr.length.times do |i|
+            ip << "." unless ip.empty?
+            ip << raw_addr[i].ord.to_s
+          end
+        rescue => bang
+          puts bang
+          puts bang.backtrace
+        end
+        ip
+      end
+
+    end
+
+  end
+end
+
+DRb.start_service "druby://127.0.0.1:666", Watobo::NFQ::Connections.new
+#puts DRb.uri
+DRb.thread.join
+

data/config/interceptor.yml

@@ -1,6 +1,3 @@
-:cert_file: cert.pem
-:key_file: privkey.pem
-:dh_key_file: watobo_dh.key
 :pass_through: 
   :content_types: 
   - application\/audio
@@ -10,7 +7,6 @@
   - application\/.*flash
   - image\/
   :content_length: 100000
-:certificate_path: certificates
 :port: 8081
-:server: "127.0.0.1"
-  
+:proxy_mode: 1
+:bind_addr: "127.0.0.1"  

data/lib/watobo/adapters/data_store.rb

@@ -22,7 +22,7 @@
 module Watobo
   class DataStore  
       
-    def self.aquire(project_name, session_name)
+    def self.acquire(project_name, session_name)
       a = Watobo::Conf::Datastore.adapter
       store = case
       when 'file'

data/lib/watobo/adapters/file/file_store.rb

@@ -22,15 +22,18 @@
 module Watobo
   class FileSessionStore < SessionStore
     def num_chats
-      get_file_list(@conversation_path, "*-chat").length
+      get_file_list(@conversation_path, "*-chat*").length
     end
 
     def num_findings
-      get_file_list(@findings_path, "*-finding").length
+      get_file_list(@findings_path, "*-finding*").length
     end
 
     def add_finding(finding)
-      finding_file = File.join("#{@findings_path}", "#{finding.id}-finding")
+      return false unless finding.respond_to? :request
+      return false unless finding.respond_to? :response
+
+      finding_file = File.join("#{@findings_path}", "#{finding.id}-finding.yml")
       if not File.exists?(finding_file) then
 
         finding_data = {
@@ -40,23 +43,24 @@ module Watobo
         }
         finding_data[:details].update(finding.details)
 
-        if not File.exists?(finding_file) then
-          fh = File.new(finding_file, "w+b")
-          fh.print YAML.dump(finding_data)
-        fh.close
-        end
+        fh = File.new(finding_file, "w+b")
+        fh.print YAML.dump(finding_data)
+      fh.close
+      return true
       end
-
+      return false
     end
 
     def delete_finding(finding)
       finding_file = File.join("#{@findings_path}", "#{finding.id}-finding")
       File.delete finding_file if File.exist? finding_file
+      finding_file << ".yml"
+      File.delete finding_file if File.exist? finding_file
 
     end
 
     def update_finding(finding)
-      finding_file = File.join("#{@findings_path}", "#{finding.id}-finding")
+      finding_file = File.join("#{@findings_path}", "#{finding.id}-finding.yml")
       finding_data = {
         :request => finding.request.map{|x| x.inspect},
         :response => finding.response.map{|x| x.inspect},
@@ -77,20 +81,22 @@ module Watobo
     # needs a scan_name (STRING) as its destination which will be created
     # if the scan name does not exist.
     def add_scan_log(chat, scan_name = nil)
+      return false unless chat.respond_to? :request
+      return false unless chat.respond_to? :response
       begin
         return false if scan_name.nil?
-       # puts ">> scan_name"
+        # puts ">> scan_name"
         path = File.join(@scanlog_path, scan_name)
 
         Dir.mkdir path unless File.exist? path
 
-        log_file = File.join( path, "log_" + Time.now.to_f.to_s)
+        log_file = File.join( path, "log_" + Time.now.to_f.to_s + ".yml")
 
         chat_data = {
           :request => chat.request.map{|x| x.inspect},
           :response => chat.response.map{|x| x.inspect},
         }
-       # puts log_file
+        # puts log_file
         chat_data.update(chat.settings)
         File.open(log_file, "w") { |fh|
           YAML.dump(chat_data, fh)
@@ -104,7 +110,8 @@ module Watobo
     end
 
     def add_chat(chat)
-      chat_file = File.join("#{@conversation_path}", "#{chat.id}-chat")
+      return false unless chat_valid? chat
+      chat_file = File.join("#{@conversation_path}", "#{chat.id}-chat.yml")
       chat_data = {
         :request => chat.request.map{|x| x.inspect},
         :response => chat.response.map{|x| x.inspect},
@@ -116,11 +123,13 @@ module Watobo
           YAML.dump(chat_data, fh)
         }
       chat.file = chat_file
+      return true
       end
+      return false
     end
 
     def each_chat(&block)
-      get_file_list(@conversation_path, "*-chat").each do |fname|
+      get_file_list(@conversation_path, "*-chat*").each do |fname|
         chat = Watobo::Utils.loadChatYAML(fname)
         next unless chat
         yield chat if block_given?
@@ -128,7 +137,7 @@ module Watobo
     end
 
     def each_finding(&block)
-      get_file_list(@findings_path, "*-finding").each do |fname|
+      get_file_list(@findings_path, "*-finding*").each do |fname|
         f = Watobo::Utils.loadFindingYAML(fname)
         next unless f
         yield f if block_given?
@@ -146,7 +155,7 @@ module Watobo
         Dir.mkdir(@project_path)
       end
 
-      @project_config_path = File.join(@project_path, "config")
+      @project_config_path = File.join(@project_path, ".config")
       Dir.mkdir @project_config_path unless File.exist? @project_config_path
 
       @session_path = File.join(@project_path, session_name)
@@ -156,7 +165,7 @@ module Watobo
         Dir.mkdir(@session_path)
       end
 
-      @session_config_path = File.join(@session_path, "config")
+      @session_config_path = File.join(@session_path, ".config")
       Dir.mkdir @session_config_path unless File.exist? @session_config_path
 
       sext = Watobo::Conf::General.session_settings_file_ext
@@ -190,51 +199,51 @@ module Watobo
     end
 
     def save_session_settings(group, session_settings)
-     # puts ">> save_session_settings <<"
+      # puts ">> save_session_settings <<"
       file = Watobo::Utils.snakecase group.gsub(/\.yml/,'')
       file << ".yml"
 
       session_file = File.join(@session_config_path, file)
-     # puts "Dest.File: #{session_file}"
-    #  puts session_settings.to_yaml
-     # puts "---"
+      # puts "Dest.File: #{session_file}"
+      #  puts session_settings.to_yaml
+      # puts "---"
       Watobo::Utils.save_settings(session_file, session_settings)
     end
 
     def load_session_settings(group)
-     # puts ">> load_session_settings : #{group}"
+      # puts ">> load_session_settings : #{group}"
       file = Watobo::Utils.snakecase group.gsub(/\.yml/,'')
       file << ".yml"
 
       session_file = File.join(@session_config_path, file)
-     # puts "File: #{session_file}"
-    #  puts "---"
+      # puts "File: #{session_file}"
+      #  puts "---"
 
       s = Watobo::Utils.load_settings(session_file)
       s
     end
 
     def save_project_settings(group, project_settings)
-     # puts ">> save_project_settings : #{group}"
+      # puts ">> save_project_settings : #{group}"
       file = Watobo::Utils.snakecase group.gsub(/\.yml/,'')
       file << ".yml"
 
       project_file = File.join(@project_config_path, file)
-     # puts "Dest.File: #{project_file}"
-     # puts project_settings.to_yaml
-     # puts "---"
+      # puts "Dest.File: #{project_file}"
+      # puts project_settings.to_yaml
+      # puts "---"
       Watobo::Utils.save_settings(project_file, project_settings)
 
     end
 
     def load_project_settings(group)
-     # puts ">> load_project_settings : #{group}"
+      # puts ">> load_project_settings : #{group}"
       file = Watobo::Utils.snakecase group.gsub(/\.yml/,'')
       file << ".yml"
 
       project_file = File.join(@project_config_path, file)
-     # puts "File: #{project_file}"
-     # puts "---"
+      # puts "File: #{project_file}"
+      # puts "---"
 
       s = Watobo::Utils.load_settings(project_file)
       s
@@ -243,8 +252,16 @@ module Watobo
 
     private
 
+    def chat_valid?(chat)
+      return false unless chat.respond_to? :request
+      return false unless chat.respond_to? :response
+      true
+    end
+
     def get_file_list(path, pattern)
-      Dir["#{path}/#{pattern}"].sort_by{ |x| File.basename(x).sub(/[^0-9]*/,'').to_i }
+      fl = Dir["#{path}/#{pattern}"].sort_by{ |x| File.basename(x).sub(/[^0-9]*/,'').to_i }
+      #puts fl.length
+      fl
     end
 
   end

data/lib/watobo/ca.rb

@@ -0,0 +1,22 @@
+# .
+# ca.rb
+# 
+# Copyright 2012 by siberas, http://www.siberas.de
+# 
+# This file is part of WATOBO (Web Application Tool Box)
+#        http://watobo.sourceforge.com
+# 
+# WATOBO is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License as published by
+# the Free Software Foundation version 2 of the License.
+# 
+# WATOBO is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+# 
+# You should have received a copy of the GNU General Public License
+# along with WATOBO; if not, write to the Free Software
+# Foundation, Inc., 51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+# .
+require 'watobo/core/ca.rb'

data/lib/watobo/config.rb

@@ -91,6 +91,8 @@ module Watobo
         def self.save_project(data_store, *filter, &b)
           raise ArgumentError, "Need a valid Watobo::DataStore" unless data_store.respond_to? :save_project_settings
           s = filter_settings filter
+         # puts @settings.to_yaml
+         # puts s.to_yaml
           data_store.save_project_settings(group_name, s)
         end
         
@@ -175,6 +177,10 @@ module Watobo
         def self.dump
           @settings
         end
+        
+        def self.to_h
+          @settings
+        end
 
         #@@settings = settings
         def self.method_missing(name, *args, &block)