arachni 0.2.2.1 → 0.2.2.2
Sign up to get free protection for your applications and to get access to all the features.
- data/CHANGELOG.md +30 -0
- data/CONTRIBUTORS.md +1 -0
- data/README.md +28 -8
- data/Rakefile +1 -0
- data/bin/arachni_web_autostart +46 -0
- data/lib/anemone/page.rb +1 -0
- data/lib/arachni.rb +1 -1
- data/lib/framework.rb +8 -3
- data/lib/http.rb +9 -39
- data/lib/mixins/observable.rb +87 -0
- data/lib/module/auditor.rb +14 -0
- data/lib/module/base.rb +0 -14
- data/lib/nokogiri/xml/node.rb +42 -0
- data/lib/ui/cli/cli.rb +1 -1
- data/lib/ui/web/log.rb +21 -14
- data/lib/ui/web/report_manager.rb +100 -15
- data/lib/ui/web/server.rb +24 -33
- data/lib/ui/web/server/public/reports/demo.testfire.net:Sun Mar 20 02:48:10 2011.afr +104829 -0
- data/lib/ui/web/server/views/layout.erb +1 -1
- data/lib/ui/web/server/views/options.erb +10 -2
- data/lib/ui/web/server/views/plugins.erb +1 -1
- data/lib/ui/web/server/views/reports.erb +8 -4
- data/lib/ui/xmlrpc/xmlrpc.rb +1 -1
- data/metamodules/autothrottle.rb +2 -2
- data/metamodules/timeout_notice.rb +1 -1
- data/modules/audit/sqli_blind_rdiff.rb +1 -1
- data/modules/recon/common_files/filenames.txt +2 -0
- data/modules/recon/directory_listing.rb +1 -0
- data/modules/recon/interesting_responses.rb +3 -3
- data/path_extractors/generic.rb +5 -1
- data/plugins/autologin.rb +15 -4
- data/plugins/content_types.rb +2 -2
- data/plugins/cookie_collector.rb +9 -16
- data/plugins/profiler.rb +237 -0
- data/reports/html.rb +21 -6
- data/reports/html/default.erb +4 -2
- data/reports/plugin_formatters/html/autologin.rb +63 -0
- data/reports/plugin_formatters/html/profiler.rb +71 -0
- data/reports/plugin_formatters/html/profiler/template.erb +177 -0
- data/reports/plugin_formatters/stdout/autologin.rb +55 -0
- data/reports/plugin_formatters/stdout/profiler.rb +90 -0
- data/reports/plugin_formatters/xml/autologin.rb +68 -0
- data/reports/plugin_formatters/xml/profiler.rb +120 -0
- metadata +23 -68
data/CHANGELOG.md
CHANGED
@@ -1,6 +1,36 @@
|
|
1
1
|
|
2
2
|
# ChangeLog
|
3
3
|
|
4
|
+
## Version 0.2.2.2 _(Under development)_
|
5
|
+
- Added "arachni_web_autostart" under bin -- Automatically starts all systems required by the WebUI and makes shutting down everything easier too (Original by: Brandon Potter <bpotter8705@gmail.com>)
|
6
|
+
- Overrided Nokogiri to revert to UTF-8 when it comes across an unknown charset instead of throwing exceptions
|
7
|
+
- Dependency versions are now defined explicitly [Issue #23]
|
8
|
+
- Updated to Sinatra v1.2.1
|
9
|
+
- HTTP
|
10
|
+
- Disabled peer verification on SSL [Issue #19]
|
11
|
+
- Replaced callbacks with the new _Observable_ mixin (also updated components to use the new conventions)
|
12
|
+
- WebUI
|
13
|
+
- Plug-in options are preserved [Issue #19]
|
14
|
+
- Check-all now skips disabled checkboxes
|
15
|
+
- Report info is stored in a database [Issue #19]
|
16
|
+
- Reports are now displayed in descending order based on scan completion datetime [Issue #19]
|
17
|
+
- Any existing reports will be migrated into the new database [Issue #19]
|
18
|
+
- XMLRPC service
|
19
|
+
- Fixed segfault on forced shutdown when spider-first was enabled
|
20
|
+
- Plug-ins
|
21
|
+
- AutoLogin now registers its results
|
22
|
+
- Reports -- Added formatters for the AutoLogin [Issue #19] and Profiler plug-ins
|
23
|
+
- HMTL
|
24
|
+
- Fixed exception on empty issue list
|
25
|
+
- Fixed encoding exceptions (cheers to Chris Weber <chris@casaba.com>)
|
26
|
+
- Path extractors
|
27
|
+
- Generic -- fixed error on invalid encoding sequences
|
28
|
+
- Modules
|
29
|
+
- Recon
|
30
|
+
- Directory listing -- Now skips non-200 pages because it used to log false positives on redirections
|
31
|
+
- Plug-ins
|
32
|
+
- Added Profiler -- Performs taint analysis (with benign inputs) and response time analysis
|
33
|
+
|
4
34
|
## Version 0.2.2.1 _(February 13, 2011)_
|
5
35
|
- Web UI v0.1-pre (Utilizing the Client - Dispatch-server XMLRPC architecture) (**New**)
|
6
36
|
- Basically a front-end to the XMLRPC client
|
data/CONTRIBUTORS.md
CHANGED
@@ -4,6 +4,7 @@ These are the people that helped improve Arachni either by submitting code, sugg
|
|
4
4
|
|
5
5
|
- [Matías Aereal Aeón](http://mfsec.com.ar/), **Arachni's official tester**.
|
6
6
|
- [Christos Chiotis](mailto:chris@survivetheinternet.com) for designing the new HTML report template.
|
7
|
+
- [Brandon Potter](mailto:bpotter8705@gmail.com) for the original "arachni_web_autostart" script
|
7
8
|
- [Steve Pinkham](http://github.com/spinkham) for beta testing and patches.
|
8
9
|
- [Aung Khant](mailto:aungkhant@yehg.net) for general suggestions.
|
9
10
|
|
data/README.md
CHANGED
@@ -1,7 +1,8 @@
|
|
1
1
|
# Arachni - Web Application Security Scanner Framework
|
2
|
-
**Version**: 0.2.2.
|
3
|
-
**Homepage**: [http://
|
4
|
-
**
|
2
|
+
**Version**: 0.2.2.2<br/>
|
3
|
+
**Homepage**: [http://arachni.segfault.gr](http://arachni.segfault.gr)<br/>
|
4
|
+
**Blog**: [http://trainofthought.segfault.gr/category/projects/arachni/](http://trainofthought.segfault.gr/category/projects/arachni/)<br/>
|
5
|
+
**Github page**: [http://github.com/zapotek/arachni](http://github.com/zapotek/arachni)<br/>
|
5
6
|
**Documentation**: [http://github.com/Zapotek/arachni/wiki](http://github.com/Zapotek/arachni/wiki)<br/>
|
6
7
|
**Code Documentation**: [http://zapotek.github.com/arachni/](http://zapotek.github.com/arachni/)<br/>
|
7
8
|
**Google Group**: [http://groups.google.com/group/arachni](http://groups.google.com/group/arachni)<br/>
|
@@ -159,14 +160,15 @@ The analyzer can graciously handle badly written HTML code due to a combination
|
|
159
160
|
- Plug-ins are framework demi-gods, they have direct access to the framework instance.
|
160
161
|
- Can be used to add any functionality to Arachni.
|
161
162
|
- Currently available plugins:
|
162
|
-
- Passive Proxy
|
163
|
+
- Passive Proxy -- Analyzes requests and responses between the web app and the browser assisting in AJAX audits, logging-in and/or restricting the scope of the audit
|
163
164
|
- Form based AutoLogin
|
164
165
|
- Dictionary attacker for HTTP Auth
|
165
166
|
- Dictionary attacker for form based authentication
|
166
|
-
-
|
167
|
+
- Profiler -- Performs taint analysis (with benign inputs) and response time analysis
|
168
|
+
- Cookie collector -- Keeps track of cookies while establishing a timeline of changes
|
167
169
|
- Healthmap -- Generates sitemap showing the health of each crawled/audited URL
|
168
170
|
- Content-types -- Logs content-types of server responses aiding in the identification of interesting (possibly leaked) files
|
169
|
-
- WAF (Web Application Firewall) Detector
|
171
|
+
- WAF (Web Application Firewall) Detector -- Establishes a baseline of normal behavior and uses rDiff analysis to determine if malicious inputs cause any behavioral changes
|
170
172
|
- MetaModules -- Loads and runs high-level meta-analysis modules pre/mid/post-scan
|
171
173
|
- AutoThrottle -- Dynamically adjusts HTTP throughput during the scan for maximum bandwidth utilization
|
172
174
|
- TimeoutNotice -- Provides a notice for issues uncovered by timing attacks when the affected audited pages returned unusually high response times to begin with.</br>
|
@@ -188,7 +190,16 @@ Still, this can be an invaluable asset to Fuzzer modules.
|
|
188
190
|
|
189
191
|
The Web User Interface is basically a Sinatra app which acts as an Arachni XMLRPC client and connects to a running XMLRPC Dispatch server.
|
190
192
|
|
191
|
-
|
193
|
+
#### Autostart
|
194
|
+
|
195
|
+
There's an autostart script to start all systems that are required by the WebUI:
|
196
|
+
$ arachni_web_autostart
|
197
|
+
|
198
|
+
**Note:**: _The "arachni_xmlrpcd" and "arachni_web" executables will need to be in your PATH._
|
199
|
+
|
200
|
+
#### Manually
|
201
|
+
|
202
|
+
You first need to start a Dispatcher like so:
|
192
203
|
$ arachni_xmlrpcd &
|
193
204
|
|
194
205
|
Then start the WebUI by running:
|
@@ -308,13 +319,22 @@ _If you installed the Gem then you'll have to look for the "profiles" directory
|
|
308
319
|
|
309
320
|
## Installation
|
310
321
|
|
322
|
+
### CDE packages for Linux
|
323
|
+
|
324
|
+
Arachni is released as [CDE packages](http://stanford.edu/~pgbovine/cde.html) for your convinience.<br/>
|
325
|
+
CDE packages are self contained and thus alleviate the need for Ruby and other dependencies to be installed or root access.<br/>
|
326
|
+
You can download the latest CDE package from the [download](https://github.com/Zapotek/arachni/downloads) page and escape the dependency hell.<br/>
|
327
|
+
If you decide to go the CDE route you can skip the rest, you're done.
|
328
|
+
|
329
|
+
|
330
|
+
### Gem
|
331
|
+
|
311
332
|
To install the Gem or work with the source code you'll also need the following system libraries:
|
312
333
|
$ sudo apt-get install libxml2-dev libxslt1-dev libcurl4-openssl-dev libsqlite3-dev
|
313
334
|
|
314
335
|
You will also need to have Ruby 1.9.2 installed *including* the dev package/headers.<br/>
|
315
336
|
The prefered ways to accomplish this is by either using [RVM](http://rvm.beginrescueend.com/) or by downloading and compiling the source code for [Ruby 1.9.2](http://www.ruby-lang.org/en/downloads/) manually.
|
316
337
|
|
317
|
-
### Gem
|
318
338
|
|
319
339
|
To install Arachni:
|
320
340
|
$ gem install arachni
|
data/Rakefile
CHANGED
@@ -0,0 +1,46 @@
|
|
1
|
+
#!/usr/bin/env ruby
|
2
|
+
|
3
|
+
=begin
|
4
|
+
Arachni
|
5
|
+
Copyright (c) 2010-2011 Tasos "Zapotek" Laskos <tasos.laskos@gmail.com>
|
6
|
+
|
7
|
+
This is free software; you can copy and distribute and modify
|
8
|
+
this program under the term of the GPL v2.0 License
|
9
|
+
(See LICENSE file for details)
|
10
|
+
|
11
|
+
|
12
|
+
Simple shell script to start all systems required for the WebUI.
|
13
|
+
Requires all arachni executables to be in PATH.
|
14
|
+
|
15
|
+
Original by: Brandon Potter <bpotter8705@gmail.com>
|
16
|
+
Modified by: Tasos Laskos <tasos.laskos@gmail.com>
|
17
|
+
|
18
|
+
=end
|
19
|
+
|
20
|
+
# the gemspec doesn't seem to able to handle shell scripts
|
21
|
+
# so we hack around it
|
22
|
+
exec <<END
|
23
|
+
|
24
|
+
export xterm="xterm -geometry 80X10 -hold"
|
25
|
+
|
26
|
+
echo "[>] Starting the Arachni Dispatch server..."
|
27
|
+
xterm -T "Arachni Dispatch server" -e "arachni_xmlrpcd" &
|
28
|
+
sleep 5
|
29
|
+
|
30
|
+
echo "[>] Starting the Arachni WebUI server..."
|
31
|
+
xterm -T "Arachni WebUI server" -e "arachni_web" &
|
32
|
+
sleep 3
|
33
|
+
|
34
|
+
echo "[>] Opening browser..."
|
35
|
+
xdg-open http://127.0.0.1:4567
|
36
|
+
|
37
|
+
echo "[>] Hit Ctrl+C to shut everything down."
|
38
|
+
|
39
|
+
while :
|
40
|
+
do
|
41
|
+
sleep 1
|
42
|
+
done
|
43
|
+
|
44
|
+
exit
|
45
|
+
|
46
|
+
END
|
data/lib/anemone/page.rb
CHANGED
data/lib/arachni.rb
CHANGED
data/lib/framework.rb
CHANGED
@@ -57,7 +57,7 @@ module Arachni
|
|
57
57
|
# @author: Tasos "Zapotek" Laskos
|
58
58
|
# <tasos.laskos@gmail.com>
|
59
59
|
# <zapotek@segfault.gr>
|
60
|
-
# @version: 0.2.
|
60
|
+
# @version: 0.2.2
|
61
61
|
#
|
62
62
|
class Framework
|
63
63
|
|
@@ -68,6 +68,7 @@ class Framework
|
|
68
68
|
#
|
69
69
|
include Arachni::UI::Output
|
70
70
|
include Arachni::Module::Utilities
|
71
|
+
include Arachni::Mixins::Observable
|
71
72
|
|
72
73
|
# the version of *this* class
|
73
74
|
REVISION = '0.2.1'
|
@@ -321,6 +322,8 @@ class Framework
|
|
321
322
|
end
|
322
323
|
}
|
323
324
|
|
325
|
+
return if @plugin_store[name]
|
326
|
+
|
324
327
|
@plugin_store[name] = {
|
325
328
|
:results => obj
|
326
329
|
}.merge( plugin.class.info )
|
@@ -453,7 +456,7 @@ class Framework
|
|
453
456
|
|
454
457
|
private
|
455
458
|
|
456
|
-
def clean_up!
|
459
|
+
def clean_up!( skip_audit_queue = false )
|
457
460
|
@opts.finish_datetime = Time.now
|
458
461
|
@opts.delta_time = @opts.finish_datetime - @opts.start_datetime
|
459
462
|
|
@@ -466,7 +469,7 @@ class Framework
|
|
466
469
|
@plugins.block!
|
467
470
|
|
468
471
|
# a plug-in may have updated the page queue, rock it!
|
469
|
-
audit_queue
|
472
|
+
audit_queue if !skip_audit_queue
|
470
473
|
|
471
474
|
# refresh the audit store
|
472
475
|
audit_store( true )
|
@@ -531,6 +534,8 @@ class Framework
|
|
531
534
|
def run_mods( page )
|
532
535
|
return if !page
|
533
536
|
|
537
|
+
call_on_run_mods( page.deep_clone )
|
538
|
+
|
534
539
|
@current_url = page.url.to_s
|
535
540
|
|
536
541
|
@modules.each_pair {
|
data/lib/http.rb
CHANGED
@@ -16,6 +16,7 @@ require Options.instance.dir['lib'] + 'typhoeus/request'
|
|
16
16
|
require Options.instance.dir['lib'] + 'typhoeus/response'
|
17
17
|
require Options.instance.dir['lib'] + 'module/utilities'
|
18
18
|
require Options.instance.dir['lib'] + 'module/trainer'
|
19
|
+
require Options.instance.dir['lib'] + 'mixins/observable'
|
19
20
|
|
20
21
|
#
|
21
22
|
# Arachni::Module::HTTP class
|
@@ -33,13 +34,14 @@ require Options.instance.dir['lib'] + 'module/trainer'
|
|
33
34
|
# @author: Tasos "Zapotek" Laskos
|
34
35
|
# <tasos.laskos@gmail.com>
|
35
36
|
# <zapotek@segfault.gr>
|
36
|
-
# @version: 0.2.
|
37
|
+
# @version: 0.2.5
|
37
38
|
#
|
38
39
|
class HTTP
|
39
40
|
|
40
41
|
include Arachni::UI::Output
|
41
42
|
include Singleton
|
42
43
|
include Arachni::Module::Utilities
|
44
|
+
include Arachni::Mixins::Observable
|
43
45
|
|
44
46
|
#
|
45
47
|
# @return [URI]
|
@@ -86,7 +88,6 @@ class HTTP
|
|
86
88
|
|
87
89
|
hydra_opts = {
|
88
90
|
:max_concurrency => req_limit,
|
89
|
-
:disable_ssl_peer_verification => true,
|
90
91
|
:username => opts.url.user,
|
91
92
|
:password => opts.url.password,
|
92
93
|
:method => :auto,
|
@@ -125,6 +126,7 @@ class HTTP
|
|
125
126
|
@opts = {
|
126
127
|
:user_agent => opts.user_agent,
|
127
128
|
:follow_location => false,
|
129
|
+
:disable_ssl_peer_verification => true,
|
128
130
|
# :timeout => 8000
|
129
131
|
}.merge( proxy_opts )
|
130
132
|
|
@@ -137,11 +139,7 @@ class HTTP
|
|
137
139
|
@curr_res_time = 0
|
138
140
|
@curr_res_cnt = 0
|
139
141
|
|
140
|
-
@on_complete = []
|
141
|
-
@on_queue = []
|
142
|
-
|
143
142
|
@after_run = []
|
144
|
-
@after_run_persistent = []
|
145
143
|
end
|
146
144
|
|
147
145
|
#
|
@@ -158,13 +156,9 @@ class HTTP
|
|
158
156
|
|block|
|
159
157
|
block.call
|
160
158
|
}
|
161
|
-
|
162
159
|
@after_run.clear
|
163
160
|
|
164
|
-
|
165
|
-
|block|
|
166
|
-
block.call
|
167
|
-
}
|
161
|
+
call_after_run_persistent( )
|
168
162
|
|
169
163
|
@curr_res_time = 0
|
170
164
|
@curr_res_cnt = 0
|
@@ -207,10 +201,7 @@ class HTTP
|
|
207
201
|
|
208
202
|
req.id = @request_count
|
209
203
|
|
210
|
-
|
211
|
-
|block|
|
212
|
-
exception_jail{ block.call( req, async ) }
|
213
|
-
}
|
204
|
+
call_on_queue( req, async )
|
214
205
|
|
215
206
|
if( !async )
|
216
207
|
@hydra_sync.queue( req )
|
@@ -237,10 +228,7 @@ class HTTP
|
|
237
228
|
@curr_res_cnt += 1
|
238
229
|
@curr_res_time += res.start_transfer_time
|
239
230
|
|
240
|
-
|
241
|
-
|block|
|
242
|
-
exception_jail{ block.call( res ) }
|
243
|
-
}
|
231
|
+
call_on_complete( res )
|
244
232
|
|
245
233
|
parse_and_set_cookies( res )
|
246
234
|
|
@@ -280,26 +268,6 @@ class HTTP
|
|
280
268
|
@after_run << block
|
281
269
|
end
|
282
270
|
|
283
|
-
def after_run_persistent( &block )
|
284
|
-
@after_run_persistent << block
|
285
|
-
end
|
286
|
-
|
287
|
-
#
|
288
|
-
# Gets called each time a request completes and passes the response
|
289
|
-
# to the block
|
290
|
-
#
|
291
|
-
def on_complete( &block )
|
292
|
-
@on_complete << block
|
293
|
-
end
|
294
|
-
|
295
|
-
#
|
296
|
-
# Gets called each time a request is queued and passes the request
|
297
|
-
# to the block
|
298
|
-
#
|
299
|
-
def on_queue( &block )
|
300
|
-
@on_queue << block
|
301
|
-
end
|
302
|
-
|
303
271
|
#
|
304
272
|
# Makes a generic request
|
305
273
|
#
|
@@ -656,6 +624,8 @@ class HTTP
|
|
656
624
|
# update framework cookies
|
657
625
|
Arachni::Options.instance.cookies = cookie_hash
|
658
626
|
|
627
|
+
call_on_new_cookies( cookie_hash, res )
|
628
|
+
|
659
629
|
current = parse_cookie_str( @init_headers['cookie'] )
|
660
630
|
set_cookies( current.merge( cookie_hash ) )
|
661
631
|
end
|
@@ -0,0 +1,87 @@
|
|
1
|
+
=begin
|
2
|
+
Arachni
|
3
|
+
Copyright (c) 2010-2011 Tasos "Zapotek" Laskos <tasos.laskos@gmail.com>
|
4
|
+
|
5
|
+
This is free software; you can copy and distribute and modify
|
6
|
+
this program under the term of the GPL v2.0 License
|
7
|
+
(See LICENSE file for details)
|
8
|
+
|
9
|
+
=end
|
10
|
+
|
11
|
+
module Arachni
|
12
|
+
|
13
|
+
require Options.instance.dir['lib'] + 'module/utilities'
|
14
|
+
|
15
|
+
module Mixins
|
16
|
+
|
17
|
+
#
|
18
|
+
# Provides a flexible way to make any Class observable via callbacks/hooks
|
19
|
+
# using simple dynamic programming with the help of "method_missing()".
|
20
|
+
#
|
21
|
+
# The observable classes (those which include this module) use:
|
22
|
+
# * call_<hookname>( *args )
|
23
|
+
# to call specific hooks.
|
24
|
+
#
|
25
|
+
# The observers set hooks using:
|
26
|
+
# * observer_instance.add_<hookname>( &block )
|
27
|
+
#
|
28
|
+
#
|
29
|
+
# @author: Tasos "Zapotek" Laskos
|
30
|
+
# <tasos.laskos@gmail.com>
|
31
|
+
# <zapotek@segfault.gr>
|
32
|
+
# @version: 0.1
|
33
|
+
#
|
34
|
+
module Observable
|
35
|
+
|
36
|
+
include Arachni::Module::Utilities
|
37
|
+
|
38
|
+
def method_missing( sym, *args, &block )
|
39
|
+
|
40
|
+
# grab the action (add/call) and the hook name
|
41
|
+
action, hook = sym.to_s.split( '_', 2 )
|
42
|
+
|
43
|
+
@__hooks ||= {}
|
44
|
+
@__hooks[hook] ||= []
|
45
|
+
|
46
|
+
if( action && hook )
|
47
|
+
case action
|
48
|
+
|
49
|
+
when 'add'
|
50
|
+
add_block( hook, &block )
|
51
|
+
return
|
52
|
+
|
53
|
+
when 'call'
|
54
|
+
call_blocks( hook, args )
|
55
|
+
return
|
56
|
+
end
|
57
|
+
end
|
58
|
+
|
59
|
+
raise NoMethodError.new( "Undefined method '#{sym.to_s}'.", sym, args )
|
60
|
+
end
|
61
|
+
|
62
|
+
private
|
63
|
+
|
64
|
+
def add_block( hook, &block )
|
65
|
+
@__hooks[hook] << block
|
66
|
+
end
|
67
|
+
|
68
|
+
def call_blocks( hook, *args )
|
69
|
+
@__hooks[hook].each {
|
70
|
+
|block|
|
71
|
+
|
72
|
+
exception_jail {
|
73
|
+
|
74
|
+
if args.flatten.size == 1
|
75
|
+
block.call( args.flatten[0] )
|
76
|
+
else
|
77
|
+
block.call( *args )
|
78
|
+
end
|
79
|
+
}
|
80
|
+
|
81
|
+
}
|
82
|
+
end
|
83
|
+
|
84
|
+
end
|
85
|
+
|
86
|
+
end
|
87
|
+
end
|