dorothy2 1.2.0 → 2.0.0

checksums.yaml

@@ -1,15 +1,15 @@
 ---
 !binary "U0hBMQ==":
   metadata.gz: !binary |-
-    YzVhOTZjZDZjMjNiMThjNGJhZWM5ZGFhZmMyNTViZjk3NTVhNzAyMA==
+    OGViZGY5NjEyNDVlZmY1MWZlYjhlZGRjZmE5YTQ5ZDI1NmU3MGRlNg==
   data.tar.gz: !binary |-
-    NWI4ODIzZGU1NTJhZDA5ZDIxYTJjNjE2Mzk4ZmI5MmFlOGNiMGRmOQ==
+    ZjgyMDY3MDBmNDU5N2M0MjI2MzdlN2NmMzBiMzg1MzAwODVkYzIzMA==
 SHA512:
   metadata.gz: !binary |-
-    NWM0OTBlMDFjNmUwMTg4NjYzYWY3Zjc5NTdiOWRkYTUyOWM0YzY2Yjg0YzY3
-    MDg2YTFiOWZhZTU5YzEwODIzNzIxNzRmODcwMjBlODhhMTg3ZTk5YTJkYzYx
-    NGRmMGY5NjRhNzExYjJkMDg4ZWIyMWQyOWU2MzE2MDcyY2YxOWY=
+    NmI0ODkxZDM4OTAyNzY4YWEyNmFlOTg2MDYwNmI4OGM3YjJlYjA0MThhMzc1
+    MDNkNWQwNDUwNmM1MzRjYjk5Nzc4MjYwZWQwZjcyNmFkYWVkMjVmNDkzN2E0
+    YzM1NGNlNWY1OTA1NGFmMjY4ZjMxNzk3NTc2OTQ4MTY3ZGFkYmI=
   data.tar.gz: !binary |-
-    ZGFmNWJiMDg2NmEwYTFjNzFjMWU0MGMxM2E5NGVlOTdjMWY1ZDJlZjBlYWVm
-    NWQxZmI4M2MzOWM0MzgyYzczNDNmNTgwY2I0ZTM2MGE5MTlhODg5NjA2ODcy
-    ZDgwZjA1NWNiOTc5NmY5NGVlZWIyYjQyNjMwZjEwM2JiMTVlM2U=
+    MTkxZWY4OTQzOTExOTk1YWI5NjNiMDcxOGM3MjQyNjg1ZTAzMTcxN2FhNTg4
+    ZGJiYTQwMDkwMTI5ODc4ZmVhZjZhNGIzMDI5MTU0YjI5MDI5M2UzM2I3ODlk
+    YWMwZWI1MTg4MDc3ZjY4ZmY2ZmJiYzI0OTYzYjllMzZlYTdjMGQ=

data/CHANGELOG

@@ -1,14 +1,39 @@
-Dorothy 1.2.0
-
-dorothy.yml
-     added sandbox’s network (needed by DEM)
-     added GeoIP.ISP 
-
-fix dparser
-     iconv deprecated
-     added GeoIP.ISP
-     removed lot of unused classes in DEM
- 
-dorothive
-    samples.hash -> sample.sha256
-    traffic_dumps.hash -> traffic_dumps.sha256
+#Dorothy 2.0
+############
+    Dorothive
+        Schema expanded and fixed for sinatra/rails application
+
+    Binaries
+        Remove unnecessary binaries (_start, _stop) now everything can be done with dorothy2.
+        Added more arguments for controlling the modules, see the help -h
+
+    Core
+        Introduction of an analysis queue
+        Samples can be submitted through the WebGUI now
+        Samples can be retrieved from email accounts
+        Modules (analysed, BFM, DEM and WGUI) can now be executed alone
+        Introduced the use of analysis profiles
+
+
+    WGUI
+        Added a dummy sinatra webgui
+        Interactive reporting
+        PCAP data analysis
+
+
+
+
+#Dorothy 1.2.0
+###############
+    dorothy.yml
+         added sandbox’s network (needed by DEM)
+         added GeoIP.ISP
+
+    fix dparser
+         iconv deprecated
+         added GeoIP.ISP
+         removed lot of unused classes in DEM
+
+    dorothive
+        samples.hash -> sample.sha256
+        traffic_dumps.hash -> traffic_dumps.sha256

data/README.md

@@ -5,47 +5,71 @@ A malware/botnet analysis framework written in Ruby.
 For a perfect view of this document (images and links), open it through the project's [code repository](https://github.com/m4rco-/dorothy2/blob/master/README.md).
 
 For any issue, use our [Redmine](https://redmine.honeynet.it/projects/dorothy2)
+
+A wiki page for dorothy2 is under construction. Please take a look at [it](https://redmine.honeynet.it/projects/dorothy2/wiki).
+
 ##Introduction
 
-Dorothy2 is a framework created for mass malware analysis. Currently, it is mainly based on analyzing the network behavior of a virtual machine where a suspicious executable was executed.
-Additionally, it is able to recognize new spawned processes by comparing them with a previously created baseline.
-Static binary analysis and an improved system behavior analysis will be shortly introduced in the next versions.
+Dorothy2 is a framework created for suspicious binary analysis. 
+It’s main strengths are a very flexible modular environment, and an interactive investigation framework with a particular care of the network analysis.
+Additionally, it is able to recognise new spawned processes by comparing them with a previously created baseline.
+Static binary analysis and an improved system behaviour analysis will be shortly introduced in the next versions.
+Dorothy2 analyses binaries by the use of pre-configured _analysis profiles_. An analysis profile is composed by the following elements:
+
+* A certain sandbox OS type  
+* A certain sandbox OS version
+* A certain sandbox OS language
+* A fixed analysis timeout (how long to wait before reverting the VM)
+* The number of screenshots requested (and the delay between them)
+* A list of the supported extensions, and how the guest OS should execute them
+
+The use of profiles gives the researcher the possibility to run analysis on a set of binaries by using different environments. As it is known, some malwares are configured to run only in specific environment. A CSIRT, might use them to test suspicious malwares only against an environment that reflects the one of its customers. 
+Sources can also be configured to be automatically analysed by certain profiles (e.g. use Profile_Windows_30sc for all the binaries retrieved by Kippo_source).
+ 
+
+Dorothy2 is a continuation of my Bachelor degree's final project ([Dorothy: inside the Storm](https://www.honeynet.it/wp-content/uploads/Dorothy/The_Dorothy_Project.pdf) ) that I presented on Feb 2009. More information about the whole project can be found on the Italian Honeyproject [website](http://www.honeynet.it).
 
-Dorothy2 is a continuation of my Bachelor degree's final project ([Dorothy: inside the Storm](https://www.honeynet.it/wp-content/uploads/Dorothy/The_Dorothy_Project.pdf) ) that I presented on Feb 2009.
-The main framework's structure remained almost the same, and it has been fully detailed in my degree's final project or in this short [paper](http://www.honeynet.it/wp-content/uploads/Dorothy/EC2ND-Dorothy.pdf). More information about the whole project can be found on the Italian Honeyproject [website](http://www.honeynet.it).
 
+The framework is mainly composed by five modules that can be even executed separately. 
+The following picture gives an overview of the current modules and how they are connected each others.
+>![dorothy.modules](http://www.honeynet.it/wp-content/uploads/dorothy2-structure.jpg)
 
-The framework is mainly composed by four big elements that can be even executed separately:
+* The Binary Fetcher Module (BFM)
 
-* The Dorothy analysis engine (included in this gem)
+ In charge of retrieving the binaries from the configured sources. Currently a “binary source” can be system folder, an email-box, or a host reachable by ssh. Once the binaries have been retrieved, the BFM will populate the analysis queue.
 
- In charge of executing a binary file into a sandbox, and then storing the generated network traffic and its screenshots into the analysis folder (moreover populating Dorothive with the basic information of the file, and CouchDB with the network pcaps).
+* The Dorothy analysis engine 
 
-* The (network) Data Extraction Module aka dparser (included in this gem)
+ In charge of analysing the queue by executing the scheduled binaries into a sandbox, and then storing the generated network traffic and its screenshots into the analysis folder (moreover populating Dorothive with the basic information of the file, and CouchDB with the network pcaps).
+
+* The (network) Data Extraction Module (old dparser) 
 
  In charge of dissecting the pcaps file, and storing the most relevant information (flows data, GeoIP info, etc) into Dorothive. In addition, it extracts all the files downloaded by the sandbox through HTTP/HTTPS and store them into the binary file's analysis folder.
 
-* The Webgui (Coded in Rails by Andrea Valerio, and not yet included in this gem)
+* The (dummy) Webgui 
+
+ A dummy Sinatra application which gives an interactive overview on all the acquired data. WARNING: this module is intended to be executed in an controlled environment. The author strongly discourage to expose it on the Internet.
 
- (Working in progress) A rails application which allows to have an interactive overview on all the acquired data. For a first glance on Andrea's first PoC take a look [this](http://youtu.be/W4DdMYPp4Ws) video (commented in Italian).
 
 * The Java Dorothy Drone (Mainly coded by Patrizia Martemucci and Domenico Chiarito, but not part of this gem and not publicly available.)
 
  Our botnet infiltration module, refers to this [ppt](https://www.honeynet.it/wp-content/uploads/Presentations/JDrone.pptx) presentation for an overview.
 
-The first three modules are (or will be soon) publicly released under GPL 3 license as tribute to the the [Honeynet Project Alliance](http://www.honeynet.org).
+The first four modules are publicly released under GPL 3 license as tribute to the the [Honeynet Project Alliance](http://www.honeynet.org).
 All the information generated by the framework - i.e. binary info, timestamps, dissected network analysis - are stored into a postgres DB (Dorothive) in order to be used for further analysis.
 A no-SQL database (CouchDB) is also used to mass store all the traffic dumps thanks to the [pcapr/xtractr](https://code.google.com/p/pcapr/wiki/Xtractr) technology.
 
 I started to code this project in late 2009 while learning Ruby at the same time. Since then, I´ve been changing/improving it as long as my Ruby coding skills were improving. Because of that, you may find some parts of code not-really-tidy :)
 
+
+
 ##Requirements
 
 >WARNING:
-The current version of Dorothy only utilizes VMWare ESX5 as its Virtual Sandbox Module (VSM). Thus, the free version of ESXi is not supported due to its limitations in using the
+The current version of Dorothy only utilises VMWare ESX5 as its Virtual Sandbox Module (VSM). Thus, the free version of ESXi is not supported due to its limitations in using the
 vSphere 5 API.
-However, the overall framework could be easily customized in order to use another virtualization engine. Dorothy2 is
-very [modular](http://www.honeynet.it/wp-content/uploads/The_big_picture.pdf),and any customization or modification is very welcome.
+However, the overall framework could be easily customised in order to use another virtualization engine. Dorothy2 is
+very [modular](http://www.honeynet.it/wp-content/uploads/The_big_picture.pdf),and any customisation or modification is very welcome.
 
 Dorothy needs the following software (not expressly in the same host) in order to be executed:
 
@@ -230,65 +254,58 @@ Finally, check out the file extensions.yml within the /etc folder: it instructs
 
 ### Dorothy usage:
 
-	$dorothy_start [options]
+	Usage:
+  	dorothy2 [options]
 	where [options] are:
-       --Verbose, -V:   Print the current version
-       --verbose, -v:   Enable verbose mode
-      --infoflow, -i:   Print the analysis flow
-      --baseline, -b:   Create a new process baseline
-    --source, -s <s>:   Choose a source (from the ones defined in etc/sources.yml)
-        --daemon, -d:   Stay in the background, by constantly pooling datasources
-        --manual, -m:   Start everything, copy the file, and wait for me.
-  --SandboxUpdate, -S:  Update Dorothive with the new Sandbox file
-  --DorothiveInit, -D:  (RE)Install the Dorothy Database (Dorothive)
-          --help, -h:   Show this message
-
-
- >Example
->
-     $dorothy_start -v -s malwarefolder
+            --Version, -V:   Print the current version.
+            --verbose, -v:   Enable verbose mode
+           --infoflow, -i:   Print the analysis flow
+       --baseline, -b <s>:   Create a new process baseline
+         --source, -s <s>:   Choose a source (from the ones defined in etc/sources.yml)
+       --CreateSource, -C:   Create new source file
+         --daemon, -d <s>:   (start|stop) Execute/kill the selected module (-W, -B, -A) in backround. If no modules are specified, it will exec/kill all of them.
+              --debug, -e:   Add extensive log trails
+             --manual, -m:   Start everything, copy the file, and wait for me.
+      --SandboxUpdate, -S:   Update Dorothive with the new Sandbox file
+		--DorothiveInit, -D <s>:   (RE)Install the Dorothy Database (Dorothive)
+		--queue, -q:   Show the analysis queue
+		--Analyser, -A:   Execute only the Analyser Module (will analalyse only the current queue)
+		--BFM, -B:   Execute only the Binary Fetcher Module (BFM)
+		--DEM, -E:   Execute only the network Data Extation Module (DEM) aka doroParser
+		--WebGUI, -W:   Execute the WebGUI Module (WGUI)
+		--help, -h:   Show this message
 
-After the execution, if everything went fine, you will find the analysis output (screens/pcap/bin) into the analysis folder that you have configured e.g. dorothy/opt/analyzed/[:digit:]/
-Other information will be stored into Dorothive.
-If executed in daemon mode, Dorothy2 will poll the datasources every X seconds (where X is defined by the "dtimeout:" field in the configuration file) looking for new binaries.
 
-### DoroParser usage:
+>Example
 
-    $dparser_start [options]
-	where [options] are:
-    --verbose, -v:   Enable verbose mode
-    --nonetbios, -n:   Hide Netbios communication
-     --daemon, -d:   Stay in the backroud, by constantly pooling datasources
-       --help, -h:   Show this message
+>     $dorothy2 -v -d start
+> This will execute all the modules in background
+
+ The first time dorothy2 is executed it will drive the user into configuring the analysis environment, more specifically the user will get through the following configuration steps:
+
+* Configuring the general env variables ($home/.dorothy.yml)
+* Configuring the BFM sources ($dorothyhome/etc/sources.yml)
+* Configuring the sandboxes ($dorothyhome/etc/sandboxes.yml) 
+* Configuring the analysis profiles (auto-filled) ($dorothyhome/etc/profiles.yml)
+
+Once the configuration step will be performed, the user will be always able to edit the configuration files at anytime. 
 
- >Example
->
-     $dparser_start -d start
-     $dparser_stop
 
 
-After the execution, if everything went fine, doroParser will store all the donwloaded files into the binary's analysis folder e.g. dorothy/opt/analyzed/[:digit:]/downloads
-Other information -i.e. Network data- will be stored into Dorothive.
-If executed in daemon mode, DoroParser will poll the database every X seconds (where X is defined by the "dtimeout:" field in the configuration file) looking for new pcaps that has been inserted.
 
 ###6. Debugging problems
 
-I recognize that setting up Dorothy is not the easiest task of the world.
+I do recognise that setting up Dorothy is not the easiest task of the world.
 By considering that the whole framework consists in the union of several 3rd pats, it is very likely that one of them will fail during the process.
 Below there are some tips about how understand the root-cause of your crash.
 
-1. Execute the Dorothy UnitTest (tc_dorothy_full.rb) that resides in its gem home directory
-
- >Example
-
-    $cd /opt/local/lib/ruby/gems/1.9.3/gems/dorothy2-0.0.1/test/
-    $ruby tc_dorothy_full.rb
+1. Set the verbose flag (-v) while executing dorothy, or the —debug flag for additional debugging trails.
 
-2. Set the verbose flag (-v) while executing dorothy
+> $dorothy_start -v -d -s malwarefolder
 
-> $dorothy_start -v -s malwarefolder
+2. If any error occours, go to our Redmine and raise a [bug-ticket](https://redmine.honeynet.it/projects/dorothy2/)!
 
-3. If any error occours, go to our Redmine and raise a [bug-ticket](https://redmine.honeynet.it/projects/dorothy2/)!    
+3. Write at dorothy2 at googlegroups.com
 
 ------------------------------------------
 
@@ -296,13 +313,12 @@ Below there are some tips about how understand the root-cause of your crash.
 
 Thanks to all the people who have contributed in making the Dorothy2 project up&running:
 
-* Marco C. (research)
+* Marco C. (Research)
 * Davide C. (Dorothive)
-* Andrea V. (WGUI)
+* Federico S. - Calogero L. (Infrastructure)
 * Domenico C. - Patrizia P. (Dorothive/JDrone)
 * [All](https://www.honeynet.it/research) the graduating students from [UniMI](http://cdlonline.di.unimi.it/) who have contributed.
 * Sabrina P. (our students "headhunter" :)
-* Jorge C. and Nelson M. (betatesting/first release feedbacks)
 
 ## Contributing
 
@@ -315,6 +331,8 @@ Thanks to all the people who have contributed in making the Dorothy2 project up&
 Every contribution is more than welcome!
 For any help, please don't hesitate in contacting us at :
 info at honeynet.it
+or through our ML:
+dorothy2 at googlegroups.com
 
 ## License
 
@@ -326,4 +344,4 @@ following GNU General Public License version 3.
 
  Copyright (C) 2007 Free Software Foundation, Inc. <http://fsf.org/>
  Everyone is permitted to copy and distribute verbatim copies
- of this license document, but changing it is not allowed.
+ of this license document, but changing it is not allowed.

data/UPDATE

@@ -1,19 +1,11 @@
 #######################################
-#Updating from Dorothy 1.0.x to >= 1.2.0##
+#Updating from Dorothy 1.x to >= 2.0.0##
 #######################################
 
-Dorothy 1.2.0 introduces several features that improve the overall framework.
-Below, the recommended steps needed to update your Dorothy environment.
-
-a) Remove the Dorothy configuration file
-    rm ~/.dorothy.yml
-   And recreate it by restarting Dorothy. You will see that the init script will ask you more question than before.
-
-b) The last version of Dorothy modified the dorothive schema in order to let dorothive compatible with Sinatra and Rails.
-The columns modified are the following:
-    samples.hash -> sample.sha256
-    traffic_dumps.hash -> traffic_dumps.sha256
-You can modify them manually if you have already a previous Dorothy version up and running, or drop the database and recreate it (-D) using the updated .ddl .
-
+Dorothy 2.0.0 introduces several features that improve the overall framework.
+As it is a major update (since 1.2.0), it is not compatible this previous versions, thus it is highly recommended to redo
+ all the installation procedure. If you have been using Dorothy for a while and you have its DB full of precious data that you
+ like to keep on the newer version, please drop me an email and we'll try to find a solution. Keep in mind that also the
+ Dorothive schema has changed, thus a you should create a new DB for this dorothy version.
 
 

data/bin/dorothy2

@@ -0,0 +1,472 @@
+#!/usr/bin/env ruby
+
+# Copyright (C) 2010-2013 marco riccardi.
+# This file is part of Dorothy - http://www.honeynet.it/
+# See the file 'LICENSE' for copying permission.
+
+require 'rubygems'
+require 'trollop'
+require 'dorothy2'          #comment for testing/developmnet
+require 'doroParser'
+
+#load '../lib/dorothy2.rb'  #uncomment for testing/developmnet
+#load '../lib/doroParser.rb'
+
+
+include Dorothy
+include DoroParser
+
+def start_wgui
+  DoroGUI.run!(
+      :environment => DoroSettings.wgui[:environment],
+      :bind => DoroSettings.wgui[:host],
+      :port => DoroSettings.wgui[:port])
+end
+
+
+def doro_forker(dmodule)
+  fork do
+
+    unless Util.check_pid_file(DoroSettings.env[:pidfiles] + "/#{dmodule}.pid")
+
+      puts "[" + "+".red + "] " + "[#{dmodule.upcase}]".yellow + " Going in backround with pid #{Process.pid}"
+      Util.create_pid_file(DoroSettings.env[:pidfiles] + "/#{dmodule}.pid", Process.pid)
+
+      case dmodule
+        when 'webgui'
+          puts "[" + "+".red + "] " + "[#{dmodule.upcase}]".yellow + " Launching Web interface at http://#{DoroSettings.wgui[:host]}:#{DoroSettings.wgui[:port]}"
+          require File.dirname(__FILE__) + '/../lib/doroGUI'
+          puts "[" + "+".red + "] " + "[#{dmodule.upcase}]".yellow + " Logging at #{DoroSettings.wgui[:logfile]}"   if DEBUG
+
+          STDOUT.reopen DoroSettings.wgui[:logfile]
+          STDERR.reopen DoroSettings.wgui[:logfile]
+          start_wgui
+        when 'bfm'
+          STDOUT.reopen DoroSettings.env[:logfile]
+          STDERR.reopen DoroSettings.env[:logfile]
+          DorothyFetcher.loader(@selected_sources, true)
+
+        when 'analyser'
+          puts "[" + "+".red + "] " + "[#{dmodule.upcase}]".yellow + " Logging at #{DoroSettings.env[:logfile]}" if DEBUG
+
+          STDOUT.reopen DoroSettings.env[:logfile]
+          STDERR.reopen DoroSettings.env[:logfile]
+          Dorothy.start(true)
+
+        when 'dem'
+          puts "[" + "+".red + "] " + "[#{dmodule.upcase}]".yellow + " Logging at #{DoroSettings.env[:logfile_parser]}"  if DEBUG
+
+          STDOUT.reopen DoroSettings.env[:logfile_parser]
+          STDERR.reopen DoroSettings.env[:logfile_parser]
+          DoroParser.start(true)
+      end
+    end
+
+
+  end
+end
+
+
+
+
+opts = Trollop.options do
+  banner <<-EOS
+
+   ####################################################
+   ##                                                ##
+   ##  The Dorothy Malware Analysis Framework 2.0    ##
+   ##                                                ##
+   ####################################################
+
+        marco.riccardi@honeynet.it
+	www.honeynet.it/dorothy
+
+
+	Usage:
+  dorothy2 [options]
+	where [options] are:
+  EOS
+
+  opt :Version, "Print the current version."
+  opt :verbose, "Enable verbose mode"
+  opt :infoflow, "Print the analysis flow"
+  opt :baseline, "Create a new process baseline", :type => :string
+  opt :source, "Choose a source (from the ones defined in etc/sources.yml)", :type => :string
+  opt :CreateSource, "Create new source file"
+  opt :daemon, "(start|stop) Execute/kill the selected module (-W, -B, -A) in backround. If no modules are specified, it will exec/kill all of them.", :type => :string
+  opt :debug, "Add extensive log trails"
+  opt :manual, "Start everything, copy the file, and wait for me."
+  opt :SandboxUpdate, "Update Dorothive with the new Sandbox file"
+  opt :DorothiveInit, "(RE)Install the Dorothy Database (Dorothive)", :type => :string
+  opt :queue, "Show the analysis queue"
+  opt :Analyser, "Execute only the Analyser Module (will analalyse only the current queue)"
+  opt :BFM, "Execute only the Binary Fetcher Module (BFM)"
+  opt :DEM, "Execute only the network Data Extation Module (DEM) aka doroParser"
+  opt :WebGUI, "Execute the WebGUI Module (WGUI)"
+
+
+end
+
+print_proc = "
+	The Dorothy Malware Analysis Framework 2.0
+	---------------Execution Flow-------------
+	#0) Fetch new malwares
+	#1) Start VM
+	#2) Copy File to VM
+	#3) Start Sniffer
+	#4) Execute file into VM
+	#5) Make screenshot
+	#6) Wait X minutes (configure X in the conf file)
+  #7) Save the running processes
+	#8) Stop Sniffer
+	#9) Download Screenshot and trafficdump
+	#10) Compare the aquired process list with the one taken during the baseline run. Find the new spawned processes.
+  #11) Try to retreive malware info from VirusTotal
+	#12) Insert data into Dorothy-DB
+	------------------------------------------
+"
+
+if opts[:infoflow]
+  puts print_proc
+  exit(0)
+end
+
+if opts[:Version]
+  puts "Dorothy ".yellow + Dorothy::VERSION
+  exit(0)
+end
+
+puts "
+
+   ####################################################
+   ##                                                ##
+   ##  The Dorothy Malware Analysis Framework 2.0    ##
+   ##          marco.riccardi@honeynet.it            ##
+   ####################################################
+
+"
+
+#VARS
+HOME = File.expand_path("..",File.dirname(__FILE__))
+VERBOSE = (opts[:verbose] ? true : false)
+MANUAL =  (opts[:manual] ? true : false)
+DEBUG = (opts[:debug] ? true : false)
+
+
+
+modules2daemonise = []
+
+
+#DEFAULT CONF FILES
+#conf = HOME + '/etc/dorothy.yml'
+
+conf = "#{File.expand_path("~")}/.dorothy.yml"
+
+
+#LOAD ENV
+if Util.exists?(conf)
+  DoroSettings.load!(conf)
+else
+  DoroConfig.create
+  exit(0)
+end
+
+
+
+home = DoroSettings.env[:home]
+sfile = home + '/etc/sources.yml'
+sboxfile = home + '/etc/sandboxes.yml'
+profiles = home + '/etc/profiles.yml'
+
+
+case opts[:daemon]
+  when "start" then
+    if MANUAL
+      "[Dorothy]".yellow + " Manual and Deamon modes can't be executed together"
+      exit(1)
+    end
+
+    daemon = true
+  when "stop" then
+
+    #TODO the following two line sure can be avoided somehow...
+    LOGGER = DoroLogger.new(DoroSettings.env[:logfile], DoroSettings.env[:logage])
+    LOGGER.sev_threshold = DoroSettings.env[:loglevel]
+
+    modules2stop = []
+
+    if opts[:BFM]
+      modules2stop.push 'bfm'
+
+    elsif  opts[:Analyser]
+      modules2stop.push 'analyser'
+
+    elsif opts[:WebGUI]
+      modules2stop.push 'webgui'
+
+    elsif opts[:DEM]
+      modules2stop.push 'dem'
+
+    end
+
+    #if not specified, close all of them
+    modules2stop = %w(bfm analyser webgui dem) if modules2stop.empty?
+
+    modules2stop.each do |dmodule|
+      Util.stop_process(dmodule)
+    end
+
+    exit(0)
+
+  else
+    daemon = false
+end
+
+
+#Logging
+if daemon
+  logout =  DoroSettings.env[:logfile]
+  logout2=   DoroSettings.env[:logfile_parser]
+else
+  logout = logout2 = STDOUT
+end
+
+LOGGER = DoroLogger.new(logout, DoroSettings.env[:logage])
+LOGGER_PARSER = DoroLogger.new(logout2, DoroSettings.env[:logage])
+LOGGER.sev_threshold = LOGGER_PARSER.sev_threshold = DoroSettings.env[:loglevel]
+
+
+
+#check homefolder
+unless Util.exists?(home)
+  DoroConfig.init_home(home)
+end
+
+
+if opts[:baseline]
+  profile =  Util.load_profile(opts[:baseline])
+  exit(1) unless profile
+  puts "[" + "+".red + "] " +  "[Analyser]".yellow + " Creating a new process baseline."
+  Dorothy.run_baseline(profile)
+  puts "[" + "+".red + "] " +  "[Analyser]".yellow + " Baseline run finished."
+  exit(0)
+end
+
+
+
+if opts[:DorothiveInit]
+  Util.init_db(opts[:DorothiveInit])
+  puts "[" + "+".red + "] " +  "[Dorothy]".yellow + " Database loaded, now you can restart Dorothy!"
+  exit(0)
+end
+
+#INIT DB Connector
+begin
+  db = Insertdb.new
+rescue => e
+  if e.inspect =~ /exist/
+    puts "[" + "+".red + "] " +  "WARNING".yellow + " The database doesn't exist yet. Press Enter to load the ddl into the DB"
+    gets
+    Util.init_db(DoroSettings.dorothive[:ddl])
+    exit(0)
+  else
+    puts "[" + "+".red + "] " +  "ERROR".red + " Can't connect to the database"
+    puts e
+    exit(0)
+  end
+end
+
+
+if opts[:SandboxUpdate]
+  puts "[" + "+".red + "] " +  "[Dorothy]".yellow + " Loading #{sboxfile} into Dorothive"
+  DoroConfig.init_sandbox(sboxfile)
+  puts "[" + "+".red + "] " +  "[Dorothy]".yellow + " Done."
+  exit(0)
+end
+
+if opts[:CreateSource]
+  puts "[" + "+".red + "] " +  "[Dorothy]".yellow + " Creating a new binary source file"
+  DoroConfig.create_sources(sfile)
+  puts "[" + "+".red + "] " +  "[Dorothy]".yellow + " Done."
+  exit(0)
+end
+
+if opts[:queue]
+  db.analysis_queue_view
+  exit(0)
+end
+
+
+#check if Source file exists
+unless Util.exists?(sfile)
+  puts "[" + "+".red + "] " +  "[WARNING]".red + " A source file doesn't exist. Please fill the following info in order to create one now."
+  DoroConfig.create_sources(sfile)
+  puts "[" + "+".red + "] " +  "[WARNING]".red + " A source file has been created, please restart dorothy!"
+end
+
+#check if Sandbox file exists
+unless Util.exists?(sboxfile)
+  puts "[" + "+".red + "] " +  "[WARNING]".red + " There is no sandbox configured yet. Please do it now."
+  DoroConfig.create_sandbox(sboxfile)
+  DoroConfig.init_sandbox(sboxfile)
+end
+
+#check if Profile file exists
+unless Util.exists?(profiles)
+  puts "[" + "+".red + "] " +  "[WARNING]".red + " There is no profiles configured yet. A default one is being created for you."
+
+  #We use the first sandbox as default
+  firts_sandbox =  YAML.load_file(sboxfile).first[1]
+  DoroConfig.create_profiles(profiles, firts_sandbox, DoroSettings.virustotal[:vtapikey].empty?)
+end
+
+#Check DB sandbox data
+if db.table_empty?("sandboxes")
+  puts "[" + "+".red + "] " +  "[WARNING]".red + " No sandbox found in Dorothive, the DB will be filled with " + sboxfile
+  DoroConfig.init_sandbox(sboxfile)
+end
+
+#Check if a baseline is present for every profile specified
+YAML.load_file(profiles).each_key do |profile|
+  profile_vars =  Util.load_profile(profile)
+
+  baseline_file = "#{DoroSettings.env[:home]}/etc/#{profile_vars[0]}_baseline_procs.yml"
+
+  unless Util.exists?(baseline_file)
+    puts "[" + "+".red + "] " +  "[WARNING]".red + " There is no process-baseline file for the #{profile} profile. Dorothy is going to create one now."
+    Dorothy.run_baseline(profile_vars)
+    puts "[" + "+".red + "] " +  "[WARNING]".red + " Baseline run finished."
+    exit(0)
+  end
+
+end
+
+
+sources = YAML.load_file(sfile)
+
+#check if Sources dir exist
+sources.keys.each do |s|
+  unless Util.exists?("#{sources[s]["localdir"]}")
+    LOGGER.warn "INIT", "Warning, the source's localdir #{s} (#{sources[s]["localdir"]}) doesn't exist yet, I'm going to create it right now.."
+    Dir.mkdir("#{sources[s]["localdir"]}")
+    LOGGER.debug "INIT", "Source's localdir #{s} created."
+  end
+end
+
+#CHECK CONF CHANGES
+%w(sources.yml sandboxes.yml).each do |f|
+  full_p = DoroSettings.env[:home] + '/etc/' + f
+  c_md5 =  Digest::MD5.hexdigest(File.read(full_p))
+  o_md5 = db.find_last_conf_chksum(f)
+  if o_md5.nil?
+    puts "#{f} not presend in the database, adding"
+    db.insert("cfg_chk", ['default', f, c_md5, Util.get_time])
+    db.check_sources_modifications(sources) if f == "sources.yml"
+  elsif c_md5 != o_md5
+    puts "The version of #{f} present into the database has changed, fixing this accordingly"
+    db.insert("cfg_chk", ['default', f, c_md5, Util.get_time])
+    db.check_sources_modifications(sources) if f == "sources.yml"
+    DoroConfig.init_sandbox(sboxfile) if f == "sandboxes.yml"
+  end
+end
+
+db.close
+
+
+
+
+if opts[:source]
+  unless sources.key?(opts[:source])
+    puts "[" + "+".red + "] " +  "[WARNING]".red + " The selected source is not yet configured. The available sources are: "
+    sources.keys.each do |k|
+      puts "[" + "-".red + "] " +  k
+    end
+    exit(0)
+  end
+  @selected_sources = sources.select {|k,v| k == opts[:source]}
+end
+
+if opts[:BFM]
+  if daemon
+    modules2daemonise.push('bfm')
+  else
+    @selected_sources ||= sources
+    DorothyFetcher.loader(@selected_sources)
+    exit(0)
+  end
+end
+
+if opts[:WebGUI]
+  if daemon
+    modules2daemonise.push('webgui')
+  else
+    require File.dirname(__FILE__) + '/../lib/doroGUI'
+    puts "[" + "+".red + "] " +  "[WGUI]".yellow + "Starting WebGUI at http://#{DoroSettings.wgui[:host]}:#{DoroSettings.wgui[:port]}"
+    start_wgui
+    exit(0)
+  end
+end
+
+
+if opts[:Analyser]
+  if daemon
+    modules2daemonise.push('analyser')
+  else
+    puts "[" + "+".red + "] " +  "[Analyser]".yellow + "Starting the analyser."
+    Dorothy.start(daemon)
+    exit(0)
+  end
+end
+
+if opts[:DEM]
+  #check pcapr
+  if DoroSettings.pcapr[:local]=="true"
+    if system "sh -c 'type startpcapr > /dev/null 2>&1'"
+      pcapr_conf = "#{File.expand_path("~")}/.pcapr_local/config"
+      unless Util.exists?(pcapr_conf)
+        puts "[WARNING]".red + " Pcapr conf not found at #{File.expand_path("~")}/.pcapr_local/config "
+        puts "[WARNING]".red + " Although you have configured Dorothy in order to look for a *local* Pcapr instance,it seems that it is not configured yet,so please run \"startpcapr\" and configure it."
+        exit(1)
+      end
+    else
+      puts "[WARNING]".red + " Although you have configured Dorothy in order to look for a *local* Pcapr instance, it seems *NOT INSTALLED* in your system.\n\t Please install it by typing \"sudo gem install pcapr-local\. Then set Pcapr to scan #{DoroSettings.env[:analysis_dir]}"
+      exit(1)
+    end
+  end
+
+  if daemon
+    modules2daemonise.push('dem')
+  else
+    puts "[" + "+".red + "] " +  "[DEM]".yellow + "Starting the network Data Extration Module (DEM)."
+    DoroParser.start(daemon)
+    exit(0)
+  end
+end
+
+
+
+begin
+
+  @selected_sources ||= sources
+
+  if daemon
+    modules2daemonise = %w(webgui bfm analyser dem) if modules2daemonise.empty?
+
+    modules2daemonise.each do |dmodule|
+      doro_forker(dmodule)
+    end
+
+  else
+    DorothyFetcher.loader(@selected_sources, daemon)
+    Dorothy.start(daemon)
+  end
+
+rescue SignalException
+  LOGGER.warn "Analyser", "SIGINT".red + " Catched, exiting gracefully."
+rescue => e
+  puts "[" + "+".red + "] " +  "[Dorothy]".yellow + " An error occurred: \n".red + e.inspect
+  puts "[" + "+".red + "] " +  "[Dorothy]".yellow + " For more information check the logfile \n" + e.inspect if daemon
+  LOGGER.error "Dorothy", "An error occurred: \n" + e.inspect
+  LOGGER.debug "Dorothy", "#{e.inspect} --BACKTRACE:  #{e.backtrace}"
+  LOGGER.info "Dorothy", "Dorothy has been stopped"
+end