dorothy2 1.0.1 → 1.0.9
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- data/README.md +13 -4
- data/TODO +12 -6
- data/UPDATE +21 -0
- data/bin/dorothy_start +51 -7
- data/bin/dparser_start +11 -7
- data/dorothy2.gemspec +3 -1
- data/etc/ddl/dorothive.ddl +26 -0
- data/etc/extensions.yml +34 -0
- data/lib/doroParser.rb +1 -1
- data/lib/dorothy2.rb +211 -103
- data/lib/dorothy2/NAM.rb +2 -2
- data/lib/dorothy2/VSM.rb +60 -19
- data/lib/dorothy2/do-init.rb +48 -29
- data/lib/dorothy2/do-utils.rb +10 -8
- data/lib/dorothy2/version.rb +2 -2
- data/share/update-dorothive.sql +19 -0
- metadata +10 -7
data/README.md
CHANGED
@@ -7,7 +7,8 @@ For a perfect view of this document (images and links), open it through the proj
|
|
7
7
|
##Introduction
|
8
8
|
|
9
9
|
Dorothy2 is a framework created for mass malware analysis. Currently, it is mainly based on analyzing the network behavior of a virtual machine where a suspicious executable was executed.
|
10
|
-
|
10
|
+
Additionally, it is able to recognize new spawned processes by comparing them with a previously created baseline.
|
11
|
+
Static binary analysis and an improved system behavior analysis will be shortly introduced in the next versions.
|
11
12
|
|
12
13
|
Dorothy2 is a continuation of my Bachelor degree's final project ([Dorothy: inside the Storm](https://www.honeynet.it/wp-content/uploads/Dorothy/The_Dorothy_Project.pdf) ) that I presented on Feb 2009.
|
13
14
|
The main framework's structure remained almost the same, and it has been fully detailed in my degree's final project or in this short [paper](http://www.honeynet.it/wp-content/uploads/Dorothy/EC2ND-Dorothy.pdf). More information about the whole project can be found on the Italian Honeyproject [website](http://www.honeynet.it).
|
@@ -109,7 +110,7 @@ It is recommended to follow this step2step process:
|
|
109
110
|
add the following line:
|
110
111
|
dorothy ALL = NOPASSWD: /usr/sbin/tcpdump, /bin/kill
|
111
112
|
|
112
|
-
* If you want to install pcapr on this machine (
|
113
|
+
* If you want to install pcapr on this machine (if you want to use dorohy from a MacOSX machine, you have to do it) install also these packages (refer to this blog [post](https://github.com/pcapr-local/pcapr-local) for a detailed howto). However, if you are installing Dorothy into a Linux machine, I recommended you to install pcapr on the same machine where the Dorothy gem was installed.
|
113
114
|
|
114
115
|
#apt-get install ruby1.8 rubygems tshark zip couchdb
|
115
116
|
|
@@ -180,6 +181,10 @@ or
|
|
180
181
|
$ brew install libmagic
|
181
182
|
$ brew link libmagic
|
182
183
|
|
184
|
+
In case you want to install pcapr here do this as well:
|
185
|
+
|
186
|
+
$sudo apt-get install tshark zip couchdb
|
187
|
+
|
183
188
|
### 3. Install Dorothy gem
|
184
189
|
|
185
190
|
*Install Dorothy gem
|
@@ -207,6 +212,7 @@ The following message should appear
|
|
207
212
|
* The ESX Virtual machines used for the analysis
|
208
213
|
|
209
214
|
The first time you execute Dorothy, it will ask you to fill those information in order to create the required configuration files into the etc/ folder. However, you are free to modify/create such files directly - configuration example files can be found there too.
|
215
|
+
Finally, check out the file extensions.yml within the /etc folder: it instructs Dorothy's sandboxes about how to process the binaries to analize.
|
210
216
|
|
211
217
|
###5. Use Dorothy
|
212
218
|
1. Copy a .exe or .bat file into $yourdorothyhome/opt/bins/manual/
|
@@ -221,12 +227,15 @@ The first time you execute Dorothy, it will ask you to fill those information in
|
|
221
227
|
|
222
228
|
$dorothy_start [options]
|
223
229
|
where [options] are:
|
230
|
+
--Verbose, -V: Print the current version
|
224
231
|
--verbose, -v: Enable verbose mode
|
225
232
|
--infoflow, -i: Print the analysis flow
|
233
|
+
--baseline, -b: Create a new process baseline
|
226
234
|
--source, -s <s>: Choose a source (from the ones defined in etc/sources.yml)
|
227
235
|
--daemon, -d: Stay in the background, by constantly pooling datasources
|
228
|
-
|
229
|
-
--
|
236
|
+
--manual, -m: Start everything, copy the file, and wait for me.
|
237
|
+
--SandboxUpdate, -S: Update Dorothive with the new Sandbox file
|
238
|
+
--DorothiveInit, -D: (RE)Install the Dorothy Database (Dorothive)
|
230
239
|
--help, -h: Show this message
|
231
240
|
|
232
241
|
|
data/TODO
CHANGED
@@ -5,17 +5,23 @@
|
|
5
5
|
-PORT TO Ruby 2.0
|
6
6
|
-WGUI
|
7
7
|
|
8
|
+
#IMPROVE INSTALLATION PROCESS
|
9
|
+
-Include pcapr-local installation 80%
|
10
|
+
|
11
|
+
-ADD args from command line for recreating the baseline
|
12
|
+
|
13
|
+
|
8
14
|
-BINARY STATIC ANALYSIS
|
9
|
-
-ANALYZE SYSTEM CHANGES
|
10
|
-
-
|
11
|
-
-LIST PROCESSES-> pm.ListProcessesInGuest(:vm => vm, :auth => auth).inspect
|
15
|
+
-ANALYZE SYSTEM CHANGES 50%
|
16
|
+
-ListFileInGuest -> Create Files/Folder Baseline.
|
12
17
|
|
13
|
-
-
|
14
|
-
-INTERACTIVE CONSOLE
|
18
|
+
-MANAGE SIG-INT WHILE MULTITHREAD
|
19
|
+
-INTERACTIVE CONSOLE 10%
|
20
|
+
-ADD VNC CLIENT SPAWN IN MANUAL MODE
|
15
21
|
|
16
22
|
-REVIEW DOROTHIVE (binary fullpath?)
|
17
23
|
|
18
|
-
-ADD EMAIL AS SOURCETYPE (use ruby mail gem for
|
24
|
+
-ADD EMAIL AS SOURCETYPE (use ruby mail gem for retrieving the emails, and parse them)
|
19
25
|
|
20
26
|
-REPORT PLUGIN
|
21
27
|
-REPORT - MAEC
|
data/UPDATE
ADDED
@@ -0,0 +1,21 @@
|
|
1
|
+
#######################################
|
2
|
+
#Updating from Dorothy 1.0.x to 1.1.0##
|
3
|
+
#######################################
|
4
|
+
|
5
|
+
Dorothy 1.1.0 introduces several features that improve the overall framework.
|
6
|
+
Below, the recommended steps needed to update your Dorothy environment.
|
7
|
+
|
8
|
+
a) Remove the Dorothy configuration file
|
9
|
+
rm ~/.dorothy.yml
|
10
|
+
And recreate it by restarting Dorothy. You will see that the init script will ask you more question than before.
|
11
|
+
|
12
|
+
b) Since a new configuration file has been added in your Dorothy's etc/ folder (extension.yml), go and edit it
|
13
|
+
accordingly to your environment.
|
14
|
+
|
15
|
+
c) From Dorothy home, execute the following SQL script in order to update the database schema. It will add the new table sys_procs.
|
16
|
+
|
17
|
+
sudo -u postgres psql dorothive -f share/update_dorothive.sql
|
18
|
+
|
19
|
+
That's all! You are ready to go!
|
20
|
+
|
21
|
+
|
data/bin/dorothy_start
CHANGED
@@ -31,11 +31,13 @@ opts = Trollop.options do
|
|
31
31
|
where [options] are:
|
32
32
|
EOS
|
33
33
|
|
34
|
-
|
34
|
+
opt :Version, "Print the current version."
|
35
35
|
opt :verbose, "Enable verbose mode"
|
36
36
|
opt :infoflow, "Print the analysis flow"
|
37
|
+
opt :baseline, "Create a new process baseline"
|
37
38
|
opt :source, "Choose a source (from the ones defined in etc/sources.yml)", :type => :string
|
38
39
|
opt :daemon, "Stay in the backround, by constantly pooling datasources"
|
40
|
+
opt :manual, "Start everything, copy the file, and wait for me."
|
39
41
|
opt :SandboxUpdate, "Update Dorothive with the new Sandbox file"
|
40
42
|
opt :DorothiveInit, "(RE)Install the Dorothy Database (Dorothive)", :type => :string
|
41
43
|
|
@@ -52,12 +54,20 @@ if opts[:infoflow]
|
|
52
54
|
#4) Execute file into VM
|
53
55
|
#5) Make screenshot
|
54
56
|
#6) Wait X minutes (configure X in the conf file)
|
55
|
-
|
56
|
-
#8)
|
57
|
-
#9)
|
58
|
-
#10)
|
57
|
+
#7) Save the running processes
|
58
|
+
#8) Stop Sniffer
|
59
|
+
#9) Download Screenshot and trafficdump
|
60
|
+
#10) Compare the aquired process list with the one taken during the baseline run. Find the new spawned processes.
|
61
|
+
#11) Try to retreive malware info from VirusTotal
|
62
|
+
#12) Insert data into Dorothy-DB
|
59
63
|
------------------------------------------
|
60
|
-
|
64
|
+
"
|
65
|
+
|
66
|
+
exit(0)
|
67
|
+
end
|
68
|
+
|
69
|
+
if opts[:Version]
|
70
|
+
puts "Dorothy ".yellow + Dorothy::VERSION
|
61
71
|
exit(0)
|
62
72
|
end
|
63
73
|
|
@@ -75,25 +85,49 @@ puts "
|
|
75
85
|
HOME = File.expand_path("..",File.dirname(__FILE__))
|
76
86
|
VERBOSE = (opts[:verbose] ? true : false)
|
77
87
|
daemon = (opts[:daemon] ? true : false)
|
88
|
+
MANUAL = (opts[:manual] ? true : false)
|
89
|
+
|
90
|
+
if MANUAL && daemon
|
91
|
+
"[Dorothy]".yellow + " Manual and Deamon modes can't be executed together"
|
92
|
+
exit(1)
|
93
|
+
end
|
94
|
+
|
78
95
|
|
79
96
|
#DEFAULT CONF FILES
|
80
97
|
#conf = HOME + '/etc/dorothy.yml'
|
81
98
|
|
82
99
|
conf = "#{File.expand_path("~")}/.dorothy.yml"
|
83
100
|
|
101
|
+
|
84
102
|
#LOAD ENV
|
85
103
|
if Util.exists?(conf)
|
86
104
|
DoroSettings.load!(conf)
|
87
105
|
else
|
88
106
|
DoroConfig.create
|
89
107
|
exit(0)
|
90
|
-
end
|
108
|
+
end
|
109
|
+
|
110
|
+
|
111
|
+
#LOAD EXTENSION MGT FILE
|
112
|
+
EXTENSIONS=YAML.load_file("#{DoroSettings.env[:home]}/etc/extensions.yml")
|
113
|
+
|
114
|
+
|
91
115
|
|
92
116
|
#Logging
|
93
117
|
logout = (daemon ? DoroSettings.env[:logfile] : STDOUT)
|
94
118
|
LOGGER = DoroLogger.new(logout, DoroSettings.env[:logage])
|
95
119
|
LOGGER.sev_threshold = DoroSettings.env[:loglevel]
|
96
120
|
|
121
|
+
|
122
|
+
if opts[:baseline]
|
123
|
+
puts "[DOROTHY]".yellow + "Creating a new process baseline."
|
124
|
+
Dorothy.run_baseline
|
125
|
+
puts "[WARNING]".red + "Baseline run finished."
|
126
|
+
exit(0)
|
127
|
+
end
|
128
|
+
|
129
|
+
|
130
|
+
|
97
131
|
home = DoroSettings.env[:home]
|
98
132
|
#check homefolder
|
99
133
|
unless Util.exists?(home)
|
@@ -102,6 +136,7 @@ end
|
|
102
136
|
|
103
137
|
sfile = home + '/etc/sources.yml'
|
104
138
|
sboxfile = home + '/etc/sandboxes.yml'
|
139
|
+
baseline_procs = home + '/etc/baseline_processes.yml'
|
105
140
|
|
106
141
|
if opts[:DorothiveInit]
|
107
142
|
Util.init_db(opts[:DorothiveInit])
|
@@ -153,6 +188,15 @@ unless Util.exists?(sboxfile)
|
|
153
188
|
DoroConfig.init_sandbox(sboxfile)
|
154
189
|
end
|
155
190
|
|
191
|
+
unless Util.exists?(baseline_procs)
|
192
|
+
puts "[WARNING]".red + " There is no process-baseline file yet, Dorothy is going to create one."
|
193
|
+
Dorothy.run_baseline
|
194
|
+
puts "[WARNING]".red + "Baseline run finished."
|
195
|
+
exit(0)
|
196
|
+
end
|
197
|
+
|
198
|
+
BASELINE_PROCS = YAML.load_file(baseline_procs)
|
199
|
+
|
156
200
|
#Check DB sandbox data
|
157
201
|
if db.table_empty?("sandboxes")
|
158
202
|
puts "[WARNING]".red + " No sandbox found in Dorothive, the DB will be filled with " + sboxfile
|
data/bin/dparser_start
CHANGED
@@ -63,22 +63,26 @@ LOGGER_PARSER.sev_threshold = DoroSettings.env[:loglevel]
|
|
63
63
|
LOGGER = DoroLogger.new(logout, DoroSettings.env[:logage])
|
64
64
|
LOGGER.sev_threshold = DoroSettings.env[:loglevel]
|
65
65
|
|
66
|
-
|
67
|
-
|
68
|
-
|
66
|
+
if system "sh -c 'type startpcapr > /dev/null 2>&1'"
|
67
|
+
pcapr_conf = "#{File.expand_path("~")}/.pcapr_local/config"
|
68
|
+
unless Util.exists?(pcapr_conf)
|
69
|
+
puts "[WARNING]".red + " Pcapr conf not found at #{File.expand_path("~")}/.pcapr_local/config "
|
70
|
+
puts "[WARNING]".red + " Although you have configured Dorothy in order to look for a *local* Pcapr instance,it seems that it is not configured yet,so please run \"startpcapr\" and configure it."
|
71
|
+
exit(1)
|
72
|
+
end
|
73
|
+
else
|
74
|
+
puts "[WARNING]".red + "Although you have configured Dorothy in order to look for a *local* Pcapr instance, it seems *NOT INSTALLED* in your system.\n\t Please install it by typing \"sudo gem install pcapr-local\. Then set Pcapr to scan #{DoroSettings.env[:analysis_dir]}"
|
69
75
|
exit(1)
|
70
|
-
|
71
76
|
end
|
72
77
|
|
73
78
|
|
74
|
-
|
75
79
|
begin
|
76
80
|
DoroParser.start(daemon)
|
77
81
|
rescue => e
|
78
82
|
puts "[PARSER]".yellow + " An error occurred: ".red + $!
|
79
83
|
if daemon
|
80
|
-
|
81
|
-
|
84
|
+
puts "[PARSER]".yellow + " For more information check the logfile" + $!
|
85
|
+
puts "[PARSER]".yellow + "Dorothy-Parser has been stopped"
|
82
86
|
end
|
83
87
|
LOGGER_PARSER.error "Parser", "An error occurred: " + $!
|
84
88
|
LOGGER_PARSER.debug "Parser", "#{e.inspect} --BACKTRACE: #{e.backtrace}"
|
data/dorothy2.gemspec
CHANGED
@@ -5,7 +5,7 @@ require 'dorothy2/version'
|
|
5
5
|
|
6
6
|
Gem::Specification.new do |gem|
|
7
7
|
gem.name = "dorothy2"
|
8
|
-
gem.version =
|
8
|
+
gem.version = Dorothy::VERSION
|
9
9
|
gem.authors = ["marco riccardi"]
|
10
10
|
gem.email = ["marco.riccardi@honeynet.it"]
|
11
11
|
gem.description = %q{A malware/botnet analysis framework written in Ruby.}
|
@@ -14,6 +14,7 @@ Gem::Specification.new do |gem|
|
|
14
14
|
gem.files = `git ls-files`.split($/)
|
15
15
|
gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
|
16
16
|
gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
|
17
|
+
gem.extra_rdoc_files = ["README.md"]
|
17
18
|
gem.require_paths = ["lib"]
|
18
19
|
gem.add_dependency(%q<net-scp>, [">= 1.0.4"])
|
19
20
|
gem.add_dependency(%q<net-ssh>, [">= 2.2.1"])
|
@@ -30,5 +31,6 @@ Gem::Specification.new do |gem|
|
|
30
31
|
gem.add_dependency(%q<net-dns>, [">= 0.8.0"])
|
31
32
|
gem.add_dependency(%q<geoip>, [">= 1.2.1"])
|
32
33
|
gem.add_dependency(%q<tmail>, [">= 1.2.7.1"])
|
34
|
+
gem.post_install_message = 'If you are upgrating from a previous version, read the UPDATE file!'
|
33
35
|
end
|
34
36
|
|
data/etc/ddl/dorothive.ddl
CHANGED
@@ -1072,6 +1072,32 @@ ALTER SEQUENCE whois_id_seq OWNED BY whois.id;
|
|
1072
1072
|
SELECT pg_catalog.setval('whois_id_seq', 1, false);
|
1073
1073
|
|
1074
1074
|
|
1075
|
+
|
1076
|
+
|
1077
|
+
|
1078
|
+
--
|
1079
|
+
-- Name: sys_procs; Type: TABLE; Schema: dorothy; Owner: postgres; Tablespace:
|
1080
|
+
--
|
1081
|
+
|
1082
|
+
CREATE TABLE dorothy.sys_procs
|
1083
|
+
(
|
1084
|
+
analysis_id integer NOT NULL,
|
1085
|
+
pid integer NOT NULL,
|
1086
|
+
name character varying,
|
1087
|
+
owner character varying,
|
1088
|
+
"cmdLine" character varying,
|
1089
|
+
"startTime" timestamp without time zone,
|
1090
|
+
"endTime" timestamp without time zone,
|
1091
|
+
"exitCode" integer,
|
1092
|
+
CONSTRAINT "procs-pk" PRIMARY KEY (analysis_id , pid ),
|
1093
|
+
CONSTRAINT "anal_id-fk" FOREIGN KEY (analysis_id)
|
1094
|
+
REFERENCES dorothy.analyses (id) MATCH SIMPLE
|
1095
|
+
ON UPDATE NO ACTION ON DELETE NO ACTION
|
1096
|
+
)
|
1097
|
+
|
1098
|
+
|
1099
|
+
ALTER TABLE dorothy.sys_procs OWNER TO postgres;
|
1100
|
+
|
1075
1101
|
--
|
1076
1102
|
-- Name: id; Type: DEFAULT; Schema: dorothy; Owner: postgres
|
1077
1103
|
--
|
data/etc/extensions.yml
ADDED
@@ -0,0 +1,34 @@
|
|
1
|
+
#############################################
|
2
|
+
### DOROTHY EXTENSION MANAGER #
|
3
|
+
#############################################
|
4
|
+
### Choose how do you want to open the the
|
5
|
+
### binaries into the Sandbox VM.
|
6
|
+
### You can add as much extensions as you
|
7
|
+
### want.
|
8
|
+
#############################################
|
9
|
+
---
|
10
|
+
exe:
|
11
|
+
prog_name: Windows CMD.exe
|
12
|
+
prog_path: C:\windows\system32\cmd.exe
|
13
|
+
prog_args: /C
|
14
|
+
|
15
|
+
dll:
|
16
|
+
prog_name: Windows Rundll32.exe
|
17
|
+
prog_path: C:\windows\system32\rundll32.exe
|
18
|
+
prog_args:
|
19
|
+
|
20
|
+
html:
|
21
|
+
prog_name: Microsoft Explorer IEXPLORE.EXE
|
22
|
+
prog_path: C:\windows\system32\cmd.exe
|
23
|
+
prog_args: /C start "C:\\Programmi\\Internet Explorer\\IEXPLORE.EXE"
|
24
|
+
|
25
|
+
#doc:
|
26
|
+
# prog_name: Microsoft Word 2003
|
27
|
+
# prog_path:
|
28
|
+
# prog_args:
|
29
|
+
|
30
|
+
#pdf:
|
31
|
+
# prog_name: Acrobat Reader Version 1.0
|
32
|
+
# prog_path:
|
33
|
+
# prog_args:
|
34
|
+
|
data/lib/doroParser.rb
CHANGED
@@ -424,7 +424,6 @@ module DoroParser
|
|
424
424
|
#gets
|
425
425
|
@insertdb.set_analyzed(dump['hash'])
|
426
426
|
@insertdb.commit
|
427
|
-
@insertdb.close
|
428
427
|
end
|
429
428
|
end
|
430
429
|
|
@@ -454,6 +453,7 @@ module DoroParser
|
|
454
453
|
sleep DoroSettings.env[:dtimeout].to_i if daemon # Sleeping a while if -d wasn't set, then quit.
|
455
454
|
end
|
456
455
|
LOGGER_PARSER.info "PARSER" , "There are no more pcaps to analyze.".yellow
|
456
|
+
@insertdb.close
|
457
457
|
exit(0)
|
458
458
|
end
|
459
459
|
|
data/lib/dorothy2.rb
CHANGED
@@ -2,9 +2,9 @@
|
|
2
2
|
# This file is part of Dorothy - http://www.honeynet.it/
|
3
3
|
# See the file 'LICENSE' for copying permission.
|
4
4
|
|
5
|
-
|
5
|
+
##.for irb debug:
|
6
6
|
##from $home, irb and :
|
7
|
-
##load 'lib/dorothy2.rb'; include Dorothy; LOGGER = DoroLogger.new(STDOUT, "weekly"); DoroSettings.load!('etc/dorothy.yml')
|
7
|
+
##load 'lib/dorothy2.rb'; include Dorothy; LOGGER = DoroLogger.new(STDOUT, "weekly"); DoroSettings.load!('etc/dorothy.yml'); VERBOSE = true
|
8
8
|
|
9
9
|
require 'net/ssh'
|
10
10
|
require 'net/scp'
|
@@ -19,7 +19,7 @@ require 'filemagic'
|
|
19
19
|
require 'rbvmomi'
|
20
20
|
require 'timeout'
|
21
21
|
require 'virustotal'
|
22
|
-
require 'ftools' #deprecated
|
22
|
+
require 'ftools' #deprecated in ruby 1.9 !!!
|
23
23
|
require 'filemagic'
|
24
24
|
require 'md5'
|
25
25
|
|
@@ -33,35 +33,47 @@ require File.dirname(__FILE__) + '/dorothy2/NAM'
|
|
33
33
|
require File.dirname(__FILE__) + '/dorothy2/BFM'
|
34
34
|
require File.dirname(__FILE__) + '/dorothy2/do-utils'
|
35
35
|
require File.dirname(__FILE__) + '/dorothy2/do-logger'
|
36
|
+
require File.dirname(__FILE__) + '/dorothy2/version'
|
36
37
|
|
37
38
|
module Dorothy
|
38
39
|
|
39
|
-
def get_time
|
40
|
-
time =
|
40
|
+
def get_time(local=Time.new)
|
41
|
+
time = local
|
41
42
|
time.utc.strftime("%Y-%m-%d %H:%M:%S")
|
42
43
|
end
|
43
44
|
|
44
45
|
|
45
46
|
def start_analysis(bins)
|
47
|
+
@binum = bins.size
|
46
48
|
bins.each do |bin|
|
47
49
|
next unless check_support(bin)
|
48
50
|
scan(bin) unless DoroSettings.env[:testmode] #avoid to stress VT if we are just testing
|
49
|
-
|
51
|
+
if MANUAL #no multithread
|
50
52
|
db = Insertdb.new
|
51
|
-
|
53
|
+
guestvm = db.find_vm
|
52
54
|
analyze(bin, guestvm)
|
53
55
|
db.free_vm(guestvm[0])
|
54
56
|
db.close
|
55
|
-
|
57
|
+
else #Use multithreading
|
58
|
+
@analysis_threads << Thread.new(bin.filename){
|
59
|
+
db = Insertdb.new
|
60
|
+
sleep rand(@binum * 2) #OPTIMIZE #REVIEW
|
61
|
+
sleep rand(30) while !(guestvm = db.find_vm) #guestvm struct: array ["sandbox id", "sandbox name", "ipaddress", "user", "password"]
|
62
|
+
analyze(bin, guestvm)
|
63
|
+
db.free_vm(guestvm[0])
|
64
|
+
db.close
|
65
|
+
}
|
66
|
+
end
|
56
67
|
end
|
57
68
|
end
|
58
69
|
|
59
70
|
|
60
71
|
def check_support(bin)
|
61
|
-
if
|
72
|
+
if EXTENSIONS.key?(bin.extension)
|
62
73
|
true
|
63
74
|
else
|
64
|
-
LOGGER.warn("
|
75
|
+
LOGGER.warn("VSM", "File extension #{bin.extension} currently not configured in etc/extensions.yml, skipping")
|
76
|
+
LOGGER.debug("VSM", "Filtype: #{bin.type}") if VERBOSE
|
65
77
|
dir_not_supported = File.dirname(bin.binpath) + "/not_supported"
|
66
78
|
Dir.mkdir(dir_not_supported) unless File.exists?(dir_not_supported)
|
67
79
|
FileUtils.cp(bin.binpath,dir_not_supported) #mv?
|
@@ -77,12 +89,6 @@ module Dorothy
|
|
77
89
|
db = Insertdb.new
|
78
90
|
anal_id = db.get_anal_id
|
79
91
|
|
80
|
-
|
81
|
-
|
82
|
-
#source.each do |sname, sinfo|
|
83
|
-
|
84
|
-
#Dir.chdir(sinfo[:dir])
|
85
|
-
|
86
92
|
#set home vars
|
87
93
|
sample_home = DoroSettings.env[:analysis_dir] + "/#{anal_id}"
|
88
94
|
bin.dir_bin = "#{sample_home}/bin/"
|
@@ -90,13 +96,14 @@ module Dorothy
|
|
90
96
|
bin.dir_screens = "#{sample_home}/screens/"
|
91
97
|
bin.dir_downloads = "#{sample_home}/downloads/"
|
92
98
|
|
99
|
+
vm_log_header = "VM#{guestvm[0]} ".yellow + "[" + "#{anal_id}".red + "] "
|
93
100
|
|
94
|
-
LOGGER.info "
|
101
|
+
LOGGER.info "VSM", vm_log_header + "Analyzing binary #{bin.filename}"
|
95
102
|
|
96
103
|
begin
|
97
104
|
#crate dir structure in analisys home
|
98
105
|
unless File.directory?(sample_home)
|
99
|
-
LOGGER.info "VSM",
|
106
|
+
LOGGER.info "VSM",vm_log_header + "Creating DIRS"
|
100
107
|
Dir.mkdir sample_home
|
101
108
|
Dir.mkdir bin.dir_bin
|
102
109
|
Dir.mkdir bin.dir_pcap
|
@@ -111,7 +118,7 @@ module Dorothy
|
|
111
118
|
end
|
112
119
|
|
113
120
|
else
|
114
|
-
LOGGER.warn "
|
121
|
+
LOGGER.warn "VSM",vm_log_header + "Malware #{bin.md5} sample_home already present, WTF!? Skipping.."
|
115
122
|
#print "\n"
|
116
123
|
return false
|
117
124
|
end
|
@@ -122,9 +129,11 @@ module Dorothy
|
|
122
129
|
|
123
130
|
|
124
131
|
#Creating a new VSM object for managing the SandBox VM
|
125
|
-
LOGGER.info "VSM",
|
132
|
+
LOGGER.info "VSM",vm_log_header + "Connecting to ESX Server #{DoroSettings.esx[:host]}"
|
126
133
|
|
127
134
|
vsm = Doro_VSM::ESX.new(DoroSettings.esx[:host],DoroSettings.esx[:user],DoroSettings.esx[:pass],guestvm[1], guestvm[3], guestvm[4])
|
135
|
+
reverted = false
|
136
|
+
|
128
137
|
|
129
138
|
#Copy File to VM
|
130
139
|
r = 0
|
@@ -134,112 +143,147 @@ module Dorothy
|
|
134
143
|
rescue
|
135
144
|
if r <= 2
|
136
145
|
r = r+1
|
137
|
-
LOGGER.warn "
|
146
|
+
LOGGER.warn "VSM",vm_log_header + " GUESTOS Connection problem to Internet, retry n. #{r}/3"
|
138
147
|
sleep 20
|
139
148
|
retry
|
140
149
|
end
|
141
|
-
LOGGER.error "
|
150
|
+
LOGGER.error "VSM", vm_log_header + " Guest system is not able to connect to internet"
|
142
151
|
r = 0
|
143
152
|
retry
|
144
153
|
end
|
145
154
|
|
146
155
|
|
147
156
|
|
148
|
-
LOGGER.info "VSM",
|
157
|
+
LOGGER.info "VSM",vm_log_header + "Copying #{bin.md5} to VM"
|
149
158
|
|
150
159
|
filecontent = File.open(bin.binpath, "rb") { |byte| byte.read } #load filebinary
|
151
|
-
vsm.copy_file(
|
160
|
+
vsm.copy_file(bin.filename,filecontent)
|
152
161
|
|
153
162
|
#Start Sniffer
|
154
163
|
dumpname = anal_id.to_s + "-" + bin.md5
|
155
164
|
pid = @nam.start_sniffer(guestvm[2],DoroSettings.nam[:interface], dumpname, DoroSettings.nam[:pcaphome])
|
156
|
-
LOGGER.info "NAM",
|
157
|
-
LOGGER.debug "NAM",
|
158
|
-
|
159
|
-
sleep 5
|
160
|
-
|
161
|
-
|
162
|
-
|
163
|
-
|
164
|
-
|
165
|
-
|
166
|
-
|
167
|
-
LOGGER.debug "VSM","
|
168
|
-
|
169
|
-
|
170
|
-
|
171
|
-
|
172
|
-
|
173
|
-
|
174
|
-
|
175
|
-
|
176
|
-
|
177
|
-
|
178
|
-
|
179
|
-
|
180
|
-
|
181
|
-
|
165
|
+
LOGGER.info "NAM",vm_log_header + "Start sniffing module"
|
166
|
+
LOGGER.debug "NAM",vm_log_header + "Tcpdump instance #{pid} started" if VERBOSE
|
167
|
+
|
168
|
+
#sleep 5
|
169
|
+
|
170
|
+
@screenshots = Array.new
|
171
|
+
|
172
|
+
#Execute File into VM
|
173
|
+
LOGGER.info "VSM",vm_log_header + "Executing #{bin.md5} with #{EXTENSIONS["prog_args"]}"
|
174
|
+
|
175
|
+
if MANUAL
|
176
|
+
LOGGER.debug "VSM",vm_log_header + " MANUAL mode detected. You can now logon to rdp:// "
|
177
|
+
|
178
|
+
LOGGER.info "MANUAL-MODE",vm_log_header + "
|
179
|
+
|
180
|
+
Choose your next action:
|
181
|
+
1) Take Screenshot
|
182
|
+
2) Take ProcessList
|
183
|
+
3) Execute #{bin.filename}
|
184
|
+
0) Continue and revert the machine.
|
185
|
+
|
186
|
+
Select a nuber:"
|
187
|
+
answer = gets.chop
|
188
|
+
|
189
|
+
until answer == "0"
|
190
|
+
case answer
|
191
|
+
when "1" then
|
192
|
+
@screenshots.push vsm.screenshot
|
193
|
+
LOGGER.info "MANUAL-MODE",vm_log_header + "Screenshot taken"
|
194
|
+
when "2"
|
195
|
+
@current_procs = vsm.get_running_procs
|
196
|
+
LOGGER.info "MANUAL-MODE",vm_log_header + "Current ProcessList taken"
|
197
|
+
when "3"
|
198
|
+
guestpid = vsm.exec_file("C:\\#{bin.filename}",EXTENSIONS[bin.extension])
|
199
|
+
LOGGER.debug "MANUAL-MODE",vm_log_header + "Program executed with PID #{guestpid}"
|
200
|
+
#when "x" then -- More interactive actions to add
|
201
|
+
else
|
202
|
+
LOGGER.info "MANUAL-MODE",vm_log_header + "Please select a number"
|
203
|
+
end
|
204
|
+
answer = gets.chop
|
182
205
|
end
|
183
206
|
|
207
|
+
LOGGER.info "MANUAL-MODE",vm_log_header + "Moving forward.."
|
184
208
|
|
185
209
|
|
186
|
-
|
187
|
-
|
188
|
-
|
189
|
-
|
190
|
-
|
191
|
-
|
192
|
-
|
210
|
+
else
|
211
|
+
guestpid = vsm.exec_file("C:\\#{bin.filename}",EXTENSIONS[bin.extension])
|
212
|
+
LOGGER.debug "VSM",vm_log_header + "Program executed with PID #{guestpid}" if VERBOSE
|
213
|
+
sleep 1
|
214
|
+
returncode = vsm.get_status(guestpid)
|
215
|
+
raise "The program was not correctly executed into the Sandbox. Status code: #{returncode}" unless returncode == 0 || returncode.nil?
|
216
|
+
|
217
|
+
LOGGER.info "VSM",vm_log_header + " Sleeping #{DoroSettings.sandbox[:sleeptime]} seconds".yellow
|
218
|
+
sleep DoroSettings.sandbox[:screen1time] % DoroSettings.sandbox[:sleeptime]
|
219
|
+
|
220
|
+
DoroSettings.sandbox[:num_screenshots].times do
|
221
|
+
@screenshots.push vsm.screenshot
|
222
|
+
sleep DoroSettings.sandbox[:screen2time] % DoroSettings.sandbox[:sleeptime]
|
223
|
+
end
|
193
224
|
|
194
|
-
sleep
|
225
|
+
sleep DoroSettings.sandbox[:sleeptime]
|
195
226
|
|
196
|
-
|
227
|
+
#Get Procs
|
228
|
+
@current_procs = vsm.get_running_procs
|
197
229
|
|
198
|
-
|
230
|
+
end
|
199
231
|
|
200
|
-
LOGGER.debug "SANDBOX" , "#{$!}\n #{e.inspect} \n #{e.backtrace}" if VERBOSE
|
201
232
|
|
202
|
-
|
203
|
-
|
233
|
+
#Stop Sniffer, revert the VM
|
234
|
+
stop_nam_revertvm(@nam, pid, vsm, guestvm[0], reverted, vm_log_header)
|
204
235
|
|
205
|
-
|
206
|
-
|
207
|
-
sleep 5
|
236
|
+
vsm.revert_vm
|
237
|
+
reverted = true
|
208
238
|
|
209
|
-
|
210
|
-
|
211
|
-
return false
|
239
|
+
#Analyze new procs
|
240
|
+
LOGGER.info "VSM", vm_log_header + "Checking for spowned processes"
|
212
241
|
|
242
|
+
unless @current_procs.nil?
|
243
|
+
@procs = vsm.get_new_procs(@current_procs)
|
244
|
+
if @procs.size > 0
|
245
|
+
LOGGER.info "VSM", vm_log_header + "#{@procs.size} new process(es) found"
|
246
|
+
@procs.each_key do |pid|
|
247
|
+
LOGGER.info "VSM", vm_log_header + "[" + "+".red + "]" + " PID: #{pid}, NAME: #{@procs[pid]["pname"]}, COMMAND: #{@procs[pid]["cmdLine"]}"
|
248
|
+
end
|
249
|
+
end
|
213
250
|
end
|
214
251
|
|
215
|
-
|
216
252
|
#Downloading PCAP
|
217
|
-
LOGGER.info "NAM",
|
253
|
+
LOGGER.info "NAM", vm_log_header + "Downloading #{dumpname}.pcap to #{bin.dir_pcap}"
|
218
254
|
Ssh.download(DoroSettings.nam[:host], DoroSettings.nam[:user],DoroSettings.nam[:pass], DoroSettings.nam[:pcaphome] + "/#{dumpname}.pcap", bin.dir_pcap)
|
219
255
|
|
220
256
|
#Downloading Screenshots from esx
|
221
|
-
LOGGER.info "NAM",
|
222
|
-
Ssh.download(DoroSettings.esx[:host],DoroSettings.esx[:user], DoroSettings.esx[:pass], @screenshot1, bin.dir_screens)
|
223
|
-
Ssh.download(DoroSettings.esx[:host],DoroSettings.esx[:user], DoroSettings.esx[:pass], @screenshot2, bin.dir_screens)
|
257
|
+
LOGGER.info "NAM", vm_log_header + "Downloading Screenshots"
|
224
258
|
|
225
|
-
|
226
|
-
|
259
|
+
@screenshots.each do |screen|
|
260
|
+
Ssh.download(DoroSettings.esx[:host],DoroSettings.esx[:user], DoroSettings.esx[:pass], screen, bin.dir_screens)
|
261
|
+
#Put them to 644
|
262
|
+
File.chmod(0644, bin.dir_screens + File.basename(screen), bin.dir_screens + File.basename(screen) )
|
263
|
+
end
|
227
264
|
|
228
|
-
|
229
|
-
#UPDATE DOROTHIBE DB#
|
230
|
-
#####################
|
265
|
+
#UPDATE DOROTHIBE DB###################################
|
231
266
|
|
232
267
|
dump = Loadmalw.new(bin.dir_pcap + dumpname + ".pcap")
|
233
268
|
|
234
|
-
#
|
235
|
-
|
269
|
+
#TODO: dump.filname depends on the PCAPR pcaps path.
|
270
|
+
#9/pcap/9-7bbf2e721d9b03988dc448344fd45a0c.pcap
|
271
|
+
#pcapr_filename = anal_id + "/pcap/" + dump.filename
|
272
|
+
|
273
|
+
|
274
|
+
if DoroSettings.pcapr[:local]
|
275
|
+
pcapr_filename = "#{anal_id}/pcap/#{dump.filename}"
|
276
|
+
pcaprid = Loadmalw.calc_pcaprid(pcapr_filename, dump.size).rstrip
|
277
|
+
else
|
278
|
+
pcaprid = Loadmalw.calc_pcaprid(dump.filename, dump.size).rstrip
|
279
|
+
end
|
236
280
|
|
237
|
-
LOGGER.debug "NAM",
|
281
|
+
LOGGER.debug "NAM", vm_log_header + "Pcaprid: " + pcaprid if VERBOSE
|
238
282
|
|
239
283
|
empty_pcap = false
|
240
284
|
|
241
285
|
if dump.size <= 30
|
242
|
-
LOGGER.warn "NAM", "
|
286
|
+
LOGGER.warn "NAM", vm_log_header + "WARNING - EMPTY PCAP FILE!!!! ::.."
|
243
287
|
#FileUtils.rm_r(sample_home)
|
244
288
|
empty_pcap = true
|
245
289
|
end
|
@@ -249,56 +293,120 @@ module Dorothy
|
|
249
293
|
analysis_values = [anal_id, bin.sha, guestvm[0], dump.sha, get_time]
|
250
294
|
|
251
295
|
if pcaprid.nil? || bin.dir_pcap.nil? || bin.sha.nil? || bin.md5.nil?
|
252
|
-
LOGGER.error "
|
253
|
-
|
254
|
-
return false
|
296
|
+
LOGGER.error "VSM", "VM#{guestvm[0]} Can't retrieve the required information"
|
297
|
+
raise "Some info missing.."
|
255
298
|
end
|
256
299
|
|
257
300
|
|
258
301
|
LOGGER.debug "DB", "VM#{guestvm[0]} Database insert phase" if VERBOSE
|
259
302
|
|
260
|
-
db = Insertdb.new
|
261
303
|
db.begin_t #needed for rollbacks
|
304
|
+
in_transaction = true
|
262
305
|
|
263
306
|
unless empty_pcap
|
264
307
|
unless db.insert("traffic_dumps", dumpvalues)
|
265
308
|
LOGGER.fatal "DB", "VM#{guestvm[0]} Error while inserting data into table traffic_dumps. Skipping binary #{bin.md5}"
|
266
|
-
|
267
|
-
return false
|
309
|
+
raise "DB-ERROR"
|
268
310
|
end
|
269
311
|
end
|
270
312
|
|
271
313
|
|
272
314
|
|
273
315
|
unless db.insert("analyses", analysis_values)
|
274
|
-
LOGGER.fatal "DB", "
|
275
|
-
|
276
|
-
return false
|
316
|
+
LOGGER.fatal "DB", vm_log_header + "Error while inserting data into table analyses. Skipping binary #{bin.md5}"
|
317
|
+
raise "DB-ERROR"
|
277
318
|
end
|
278
319
|
|
320
|
+
@procs.each_key do |pid|
|
321
|
+
@procs[pid]["endTime"] ? end_time = get_time(@procs[pid]["endTime"]) : end_time = "null"
|
322
|
+
@procs[pid]["exitCode"] ? exit_code = @procs[pid]["exitCode"] : exit_code = "null"
|
323
|
+
sys_procs_values = [anal_id, pid, @procs[pid]["pname"], @procs[pid]["owner"], @procs[pid]["cmdLine"], get_time(@procs[pid]["startTime"]), end_time, exit_code ]
|
324
|
+
unless db.insert("sys_procs", sys_procs_values)
|
325
|
+
LOGGER.fatal "DB", vm_log_header + "Error while inserting data into table sys_procs. Skipping binary #{bin.md5}"
|
326
|
+
raise "DB-ERROR"
|
327
|
+
end
|
328
|
+
end
|
329
|
+
|
330
|
+
|
279
331
|
#TODO ADD RT CODE
|
280
332
|
|
281
333
|
db.commit
|
334
|
+
in_transaction = false
|
282
335
|
db.close
|
283
336
|
|
284
|
-
LOGGER.info "VSM",
|
337
|
+
LOGGER.info "VSM", vm_log_header + "Removing file from /bins directory"
|
285
338
|
FileUtils.rm(bin.binpath)
|
286
|
-
LOGGER.info "VSM",
|
339
|
+
LOGGER.info "VSM", vm_log_header + "Process compleated successfully"
|
287
340
|
|
288
|
-
rescue
|
341
|
+
rescue SignalException
|
342
|
+
LOGGER.warn "DOROTHY", "SIGINT".red + " Catched, exiting gracefully."
|
343
|
+
stop_nam_revertvm(@nam, pid, vsm, guestvm[0], reverted, vm_log_header)
|
344
|
+
LOGGER.debug "VSM", vm_log_header + "Removing working dir"
|
345
|
+
FileUtils.rm_r(sample_home)
|
346
|
+
if in_transaction
|
347
|
+
db.rollback #rollback in case there is a transaction on going
|
348
|
+
db.close
|
349
|
+
end
|
289
350
|
|
290
|
-
|
351
|
+
rescue Exception => e
|
352
|
+
LOGGER.error "VSM", vm_log_header + "An error occurred while analyzing #{bin.filename}, skipping\n"
|
291
353
|
LOGGER.debug "Dorothy" , "#{$!}\n #{e.inspect} \n #{e.backtrace}" if VERBOSE
|
292
354
|
|
355
|
+
LOGGER.warn "Dorothy", vm_log_header + "Stopping NAM instances if presents, reverting the Sandbox, and removing working directory"
|
356
|
+
|
357
|
+
stop_nam_revertvm(@nam, pid, vsm, guestvm[0], reverted, vm_log_header)
|
358
|
+
LOGGER.debug "VSM", vm_log_header + "Removing working dir"
|
359
|
+
|
293
360
|
FileUtils.rm_r(sample_home)
|
294
|
-
|
295
|
-
|
361
|
+
|
362
|
+
if in_transaction
|
363
|
+
db.rollback #rollback in case there is a transaction on going
|
364
|
+
db.close
|
365
|
+
end
|
366
|
+
|
367
|
+
LOGGER.warn "VSM", vm_log_header + "Recover finished."
|
368
|
+
|
369
|
+
|
296
370
|
end
|
297
371
|
|
372
|
+
end
|
298
373
|
|
374
|
+
#Stop NAM instance and Revert VM
|
375
|
+
def stop_nam_revertvm(nam, pid, vsm, guestvm, reverted, vm_log_header)
|
299
376
|
|
377
|
+
if pid
|
378
|
+
LOGGER.info "VSM", vm_log_header + " Stopping sniffing module " + pid.to_s
|
379
|
+
nam.stop_sniffer(pid)
|
380
|
+
end
|
300
381
|
|
382
|
+
unless reverted || vsm.nil?
|
383
|
+
LOGGER.info "VSM", vm_log_header + " Reverting VM"
|
384
|
+
vsm.revert_vm
|
385
|
+
sleep 3 #wait some seconds for letting the vm revert..
|
386
|
+
end
|
387
|
+
end
|
301
388
|
|
389
|
+
###Create Baseline
|
390
|
+
def run_baseline
|
391
|
+
db = Insertdb.new
|
392
|
+
guestvm = db.find_vm
|
393
|
+
LOGGER.info "VSM","VM#{guestvm[0]}".red + " Executng the baseline run"
|
394
|
+
vsm = Doro_VSM::ESX.new(DoroSettings.esx[:host],DoroSettings.esx[:user],DoroSettings.esx[:pass],guestvm[1], guestvm[3], guestvm[4])
|
395
|
+
vsm.check_internet
|
396
|
+
LOGGER.info "VSM","VM#{guestvm[0]}".red + " Sleeping #{DoroSettings.sandbox[:sleeptime]} seconds".yellow
|
397
|
+
sleep DoroSettings.sandbox[:sleeptime]
|
398
|
+
vsm.get_running_procs(nil, true) #save on file
|
399
|
+
LOGGER.info "VSM", "VM#{guestvm[0]} ".red + "Reverting VM".yellow
|
400
|
+
vsm.revert_vm
|
401
|
+
db.free_vm(guestvm[0])
|
402
|
+
db.close
|
403
|
+
rescue => e
|
404
|
+
LOGGER.error "VSM", "VM#{guestvm[0]} An error occurred while performing the BASELINE run, please retry"
|
405
|
+
LOGGER.debug "Dorothy" , "#{$!}\n #{e.inspect} \n #{e.backtrace}" if VERBOSE
|
406
|
+
LOGGER.warn "VSM", "VM#{guestvm[0]} ".red + "[RECOVER] Reverting VM".yellow
|
407
|
+
vsm.revert_vm
|
408
|
+
db.free_vm(guestvm[0])
|
409
|
+
db.close
|
302
410
|
end
|
303
411
|
|
304
412
|
########################
|
@@ -322,8 +430,8 @@ module Dorothy
|
|
322
430
|
sleep 30
|
323
431
|
end
|
324
432
|
|
325
|
-
LOGGER.info
|
326
|
-
LOGGER.info
|
433
|
+
LOGGER.info "VTOTAL", "#{bin.md5} Detection Rate: #{vt.rate}"
|
434
|
+
LOGGER.info "VTOTAL", "#{bin.md5} Family by McAfee: #{vt.family}"
|
327
435
|
|
328
436
|
LOGGER.info "VTOTAL", "Updating DB"
|
329
437
|
vtvalues = [bin.sha, vt.family, vt.vendor, vt.version, vt.rate, vt.updated, vt.detected]
|
@@ -418,7 +526,7 @@ module Dorothy
|
|
418
526
|
|
419
527
|
end
|
420
528
|
|
421
|
-
def check_pid_file
|
529
|
+
def check_pid_file(file)
|
422
530
|
if File.exist? file
|
423
531
|
# If we get Errno::ESRCH then process does not exist and
|
424
532
|
# we can safely cleanup the pid file.
|
@@ -436,7 +544,7 @@ module Dorothy
|
|
436
544
|
end
|
437
545
|
end
|
438
546
|
|
439
|
-
def create_pid_file
|
547
|
+
def create_pid_file(file)
|
440
548
|
File.open(file, "w") { |f| f.puts Process.pid }
|
441
549
|
|
442
550
|
# Remove pid file during shutdown
|
@@ -462,4 +570,4 @@ module Dorothy
|
|
462
570
|
end
|
463
571
|
end
|
464
572
|
|
465
|
-
end
|
573
|
+
end
|