pastenum 0.2.2 → 0.3.0
Sign up to get free protection for your applications and to get access to all the features.
- data/Gemfile.lock +49 -0
- data/README.md +71 -33
- data/RELEASE_NOTES.md +39 -0
- data/lib/pastenum/cli.rb +48 -21
- data/lib/pastenum/main.rb +6 -1
- data/lib/pastenum/target.rb +24 -7
- data/lib/pastenum/targets/gist.rb +18 -10
- data/lib/pastenum/targets/github.rb +35 -12
- data/lib/pastenum/targets/pastebin.rb +10 -16
- data/lib/pastenum/targets/pastie.rb +30 -40
- data/lib/pastenum/version.rb +1 -1
- metadata +4 -3
- data/RELEASE_NOTES +0 -16
data/Gemfile.lock
ADDED
@@ -0,0 +1,49 @@
|
|
1
|
+
PATH
|
2
|
+
remote: .
|
3
|
+
specs:
|
4
|
+
pastenum (0.3.0)
|
5
|
+
|
6
|
+
GEM
|
7
|
+
remote: http://rubygems.org/
|
8
|
+
specs:
|
9
|
+
bump (0.3.9)
|
10
|
+
colored (1.2)
|
11
|
+
domain_name (0.5.7)
|
12
|
+
unf (~> 0.0.3)
|
13
|
+
gscraper (0.4.0)
|
14
|
+
json (~> 1.6)
|
15
|
+
mechanize (~> 2.0)
|
16
|
+
uri-query_params (~> 0.5)
|
17
|
+
highline (1.6.15)
|
18
|
+
json (1.7.7)
|
19
|
+
mechanize (2.5.1)
|
20
|
+
domain_name (~> 0.5, >= 0.5.1)
|
21
|
+
mime-types (~> 1.17, >= 1.17.2)
|
22
|
+
net-http-digest_auth (~> 1.1, >= 1.1.1)
|
23
|
+
net-http-persistent (~> 2.5, >= 2.5.2)
|
24
|
+
nokogiri (~> 1.4)
|
25
|
+
ntlm-http (~> 0.1, >= 0.1.1)
|
26
|
+
webrobots (~> 0.0, >= 0.0.9)
|
27
|
+
mime-types (1.21)
|
28
|
+
net-http-digest_auth (1.2.1)
|
29
|
+
net-http-persistent (2.8)
|
30
|
+
nokogiri (1.5.6)
|
31
|
+
ntlm-http (0.1.1)
|
32
|
+
unf (0.0.5)
|
33
|
+
unf_ext
|
34
|
+
unf_ext (0.0.5)
|
35
|
+
uri-query_params (0.7.0)
|
36
|
+
webrobots (0.0.13)
|
37
|
+
|
38
|
+
PLATFORMS
|
39
|
+
ruby
|
40
|
+
|
41
|
+
DEPENDENCIES
|
42
|
+
bump (~> 0.3)
|
43
|
+
colored (~> 1.2)
|
44
|
+
gscraper (~> 0.4)
|
45
|
+
highline
|
46
|
+
json (~> 1.7)
|
47
|
+
mechanize (~> 2.0)
|
48
|
+
pastenum!
|
49
|
+
uri-query_params (~> 0.7)
|
data/README.md
CHANGED
@@ -1,54 +1,92 @@
|
|
1
1
|
#Pastenum
|
2
2
|
|
3
|
-
Pastenum is a
|
3
|
+
Pastenum is a text dump enumeration tool. It is designed to help find and enumerate datadumps, and doxs posted on public sites.
|
4
|
+
|
5
|
+
It currently searches sites github.com, gist.github.com, pastebin.com, and pastie.org.
|
6
|
+
|
7
|
+
Pastenum is a gem rewrite of nullthreat's original [pastenum2](http://redmine.corelan.be/projects/corelan-pastenum) released in 2011.
|
8
|
+
|
9
|
+
## Install:
|
4
10
|
|
5
|
-
Install:
|
6
|
-
------
|
7
11
|
* Install Ruby 1.9.*
|
8
12
|
|
9
13
|
* Install RubyGems
|
10
14
|
|
11
15
|
* gem install pastenum
|
12
16
|
|
13
|
-
Troubleshooting:
|
14
|
-
======
|
17
|
+
### Troubleshooting:
|
15
18
|
|
16
19
|
** Note: nokogiri in the gemfile requires native libxslt and libxml2 to be installed
|
17
20
|
|
18
|
-
Usage:
|
19
|
-
|
20
|
-
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
|
25
|
-
|
26
|
-
|
27
|
-
|
28
|
-
|
29
|
-
|
21
|
+
## Usage:
|
22
|
+
|
23
|
+
```shell
|
24
|
+
(~)$ pastenum -h
|
25
|
+
Usage: pastenum [OPTIONS] "search string"
|
26
|
+
|
27
|
+
Sources::
|
28
|
+
-p, --[no-]pastebin Search Pastebin.com (Gscraper)
|
29
|
+
Default: true
|
30
|
+
-g, --[no-]gist Search Gist.github.com
|
31
|
+
Default: true
|
32
|
+
-G, --[no-]github Search github.com
|
33
|
+
Default: true
|
34
|
+
-i, --[no-]pasties Search pastie.org (Gscraper)
|
35
|
+
Default: true
|
36
|
+
Report Output:: (Default output to STDOUT)
|
37
|
+
-H, --html-report Create an html iframed (report.html)
|
38
|
+
-J, --json-report Create a JSON file (report.json)
|
39
|
+
Options::
|
40
|
+
-r, --[no-]raw use 'raw' urls instead if available
|
41
|
+
Default: false
|
42
|
+
-m, --maxpages= maximum number of search results pages to iterate through
|
43
|
+
Default: 2
|
44
|
+
-x, --tos Acknowledge Google INC. 'Terms of Service'
|
45
|
+
-V, --[no-]ssl-verify Verify SSL certificates
|
46
|
+
Default: true
|
47
|
+
-v, --verbose Run verbosely
|
48
|
+
-h, --help Display this screen
|
49
|
+
```
|
50
|
+
|
51
|
+
** Note: If generating a report iframe, it can take some time to load the "report.html" file.**
|
52
|
+
|
53
|
+
### Example Search
|
54
|
+
|
55
|
+
Gist and Github code search for 'shadowbq' skipping ssl cert verfication
|
56
|
+
|
57
|
+
```shell
|
58
|
+
(~)$ pastenum -x --no-pastebin --no-pastie --no-ssl-verify "shadowbq"
|
59
|
+
https://gist.github.com/shadowbq/325430
|
60
|
+
https://gist.github.com/shadowbq/4556950
|
61
|
+
https://gist.github.com/shadowbq/2718948
|
62
|
+
https://github.com/shadowbq/matrix.dot.files/blob/8241fc219a3b4601c6c4b1a487441cf31e90916b/docs/README.vim-colors
|
63
|
+
https://github.com/shadowbq/easy-bin/blob/6237442a25777e17a869b5131f0f4f38ca62def1/easy-bin.gemspec
|
64
|
+
https://github.com/shadowbq/pastenum/blob/f1d955652a9fd2f7928d7ae84093aea4fc105f85/pastenum.gemspec
|
65
|
+
https://github.com/shadowbq/matrix.dot.files/blob/8241fc219a3b4601c6c4b1a487441cf31e90916b/home/.matrix/vim/.gitmodules
|
66
|
+
https://github.com/shadowbq/matrix.dot.files/blob/4d1519387fb7bfc1e1134c179fe372f232e30cba/home/.gitconfig
|
67
|
+
https://github.com/shadowbq/pastenum/blob/79f620a98f5d6fbb49c805e25b4206fa70429929/README.md
|
68
|
+
https://github.com/fabrizioc1/WiFu/blob/d37853fd538319317bb4e4f5d16b3255501b76d1/Gemfile
|
69
|
+
https://github.com/shadowbq/matrix.dot.files/blob/8241fc219a3b4601c6c4b1a487441cf31e90916b/home/.ssh/config
|
70
|
+
https://github.com/shadowbq/snorby_suite/blob/a5bfc872c6bc8c2691adf3041fa821dadbec233b/snorby_suite.gemspec
|
71
|
+
https://github.com/shadowbq/rothparty/blob/22a5e8847592cdc158439faef925ca2a07bb0e4d/rothparty.gemspec
|
72
|
+
|
73
|
+
```
|
74
|
+
|
75
|
+
|
76
|
+
## Original Pastenum Credits:
|
30
77
|
|
31
|
-
|
32
|
-
|
33
|
-
|
34
|
-
Use the appropriate API for github
|
35
|
-
|
36
|
-
http://developer.github.com/v3/repos/
|
78
|
+
Pastenum - A Corelan Team Production
|
79
|
+
Written by: Nullthreat
|
37
80
|
|
38
|
-
|
39
|
-
------
|
81
|
+
### Website(s)
|
40
82
|
|
41
|
-
|
83
|
+
http://redmine.corelan.be/projects/corelan-pastenum
|
42
84
|
|
43
|
-
|
85
|
+
http://www.corelan.be
|
44
86
|
|
45
|
-
|
46
|
-
------
|
87
|
+
http://www.nullthreat.net
|
47
88
|
|
48
|
-
|
49
|
-
Written by: Nullthreat
|
89
|
+
### IRC
|
50
90
|
|
51
|
-
www.corelan.be
|
52
|
-
www.nullthreat.net
|
53
91
|
irc.freenode.net #corelan
|
54
92
|
|
data/RELEASE_NOTES.md
ADDED
@@ -0,0 +1,39 @@
|
|
1
|
+
|
2
|
+
## RELEASE NOTES:
|
3
|
+
|
4
|
+
2/13/2013
|
5
|
+
v2.2
|
6
|
+
Shadowbq - rewrite/refactor code framework. The durability, functionality, and reporting has been enhanced, but the fetch mechanizism (mechanize, gscraper) are still conceptionally original.
|
7
|
+
|
8
|
+
v2.1
|
9
|
+
Shadowbq - code cleanup.
|
10
|
+
|
11
|
+
6/05/2011
|
12
|
+
v .2
|
13
|
+
Added BT5 install instructions to README
|
14
|
+
Fixed some formatting
|
15
|
+
Generally Cleaned Code
|
16
|
+
|
17
|
+
3/25/2011
|
18
|
+
v .1 RC2
|
19
|
+
Added github support
|
20
|
+
Added ability to enable and disable searches individually
|
21
|
+
|
22
|
+
3/22/2011
|
23
|
+
v .1 RC1
|
24
|
+
Initial public release
|
25
|
+
|
26
|
+
## Notes on Future Versions:
|
27
|
+
|
28
|
+
### Gist / Github
|
29
|
+
|
30
|
+
Use the appropriate API for github
|
31
|
+
|
32
|
+
http://developer.github.com/v3/repos/
|
33
|
+
|
34
|
+
### Gscaper -> Google Custom Search API:
|
35
|
+
|
36
|
+
|
37
|
+
Important: The Google Custom Search API requires the use of an API key, which you can get from the Google APIs console. The API provides 100 search queries per day for free. If you need more, you may sign up for billing in the console.
|
38
|
+
|
39
|
+
https://developers.google.com/custom-search/v1/overview
|
data/lib/pastenum/cli.rb
CHANGED
@@ -17,50 +17,56 @@ module Pastenum
|
|
17
17
|
options[:pastebin] = true
|
18
18
|
options[:gist] = true
|
19
19
|
options[:github] = true
|
20
|
-
options[:pastie] =
|
20
|
+
options[:pastie] = true
|
21
21
|
|
22
|
+
options[:raw] = Pastenum::Defaults::Raw
|
22
23
|
options[:onion] = false
|
23
24
|
options[:test] = false
|
24
|
-
options[:verbose] =
|
25
|
-
options[:maxpages] =
|
25
|
+
options[:verbose] = Pastenum::Defaults::Verbose
|
26
|
+
options[:maxpages] = Pastenum::Defaults::MaxPages
|
26
27
|
options[:tos] = false
|
28
|
+
options[:ssl_verify] = true
|
27
29
|
|
28
30
|
opt_parser = OptionParser.new do |opt|
|
29
31
|
opt.banner = "Usage: pastenum [OPTIONS] \"search string\""
|
30
32
|
opt.separator ""
|
31
33
|
opt.separator "Sources::"
|
32
34
|
|
33
|
-
opt.on("-p","--[no-]pastebin","Search Pastebin.com (Gscraper)"," Default: #{options[:pastebin]}") do
|
34
|
-
options[:pastebin] =
|
35
|
+
opt.on("-p","--[no-]pastebin","Search Pastebin.com (Gscraper)"," Default: #{options[:pastebin]}") do |value|
|
36
|
+
options[:pastebin] = value
|
35
37
|
end
|
36
38
|
|
37
|
-
opt.on("-g","--[no-]gist","Search Gist.github.com"," Default: #{options[:gist]}") do
|
38
|
-
options[:gist] =
|
39
|
+
opt.on("-g","--[no-]gist","Search Gist.github.com"," Default: #{options[:gist]}") do |value|
|
40
|
+
options[:gist] = value
|
39
41
|
end
|
40
42
|
|
41
|
-
opt.on("-G","--[no-]github","Search github.com"," Default: #{options[:github]}") do
|
42
|
-
options[:github] =
|
43
|
+
opt.on("-G","--[no-]github","Search github.com"," Default: #{options[:github]}") do |value|
|
44
|
+
options[:github] = value
|
43
45
|
end
|
44
46
|
|
45
|
-
opt.on("-i","--[no-]
|
46
|
-
options[:pastie] =
|
47
|
+
opt.on("-i","--[no-]pastie","Search pastie.org (Gscraper)"," Default: #{options[:pastie]}") do |value|
|
48
|
+
options[:pastie] = value
|
47
49
|
end
|
48
50
|
|
49
|
-
opt.separator "Output:: (Default output to STDOUT)"
|
51
|
+
opt.separator "Report Output:: (Default output to STDOUT)"
|
50
52
|
|
51
|
-
opt.on("-
|
53
|
+
opt.on("-H","--html-report","Create an html iframed (report.html) ") do
|
52
54
|
options[:report] = true
|
53
55
|
end
|
54
56
|
|
55
|
-
opt.on("-J", "--json", "Create a JSON file
|
57
|
+
opt.on("-J", "--json-report", "Create a JSON file (report.json) ") do
|
56
58
|
options[:json] = true
|
57
59
|
end
|
58
60
|
|
59
61
|
|
60
62
|
opt.separator "Options::"
|
61
63
|
|
62
|
-
opt.on("-
|
63
|
-
options[:
|
64
|
+
opt.on("-r","--[no-]raw","use 'raw' urls instead if available" ," Default: #{options[:raw]}") do |value|
|
65
|
+
options[:raw] = value
|
66
|
+
end
|
67
|
+
|
68
|
+
opt.on("-m","--maxpages=","maximum number of search results pages to iterate through" ," Default: #{options[:maxpages]}") do |value|
|
69
|
+
options[:maxpages] = value
|
64
70
|
end
|
65
71
|
|
66
72
|
opt.on("-x", "--tos", "Acknowledge Google INC. 'Terms of Service'") do
|
@@ -75,8 +81,12 @@ module Pastenum
|
|
75
81
|
options[:test] = v
|
76
82
|
end
|
77
83
|
=end
|
84
|
+
opt.on("-V", "--[no-]ssl-verify", "Verify SSL certificates"," Default: #{options[:ssl_verify]}") do |value|
|
85
|
+
options[:ssl_verify] = value
|
86
|
+
end
|
87
|
+
|
78
88
|
opt.on("-v", "--verbose", "Run verbosely") do
|
79
|
-
options[:verbose] =
|
89
|
+
options[:verbose] = true
|
80
90
|
end
|
81
91
|
|
82
92
|
opt.on_tail("-h","--help","Display this screen") do
|
@@ -93,7 +103,10 @@ module Pastenum
|
|
93
103
|
opt_parser.parse!
|
94
104
|
|
95
105
|
#If options fail display help
|
96
|
-
rescue
|
106
|
+
#rescue Exception => e
|
107
|
+
# puts e.message
|
108
|
+
# puts e.backtrace.inspect
|
109
|
+
rescue
|
97
110
|
puts opt_parser
|
98
111
|
exit
|
99
112
|
end
|
@@ -137,16 +150,24 @@ module Pastenum
|
|
137
150
|
|
138
151
|
@gist.verbose = options[:verbose]
|
139
152
|
@gist.max_pages = options[:maxpages]
|
153
|
+
@gist.verify_ssl_mode = OpenSSL::SSL::VERIFY_NONE unless options[:ssl_verify]
|
140
154
|
@gist.search
|
141
155
|
@gist.summary
|
142
|
-
|
156
|
+
if options[:raw]
|
157
|
+
@gist.results.each { |hit| puts "#{@gist.raw_url}#{hit}" } if !options[:report] && !options[:json]
|
158
|
+
else
|
159
|
+
@gist.results.each { |hit| puts "#{@gist.vendor}#{hit}" } if !options[:report] && !options[:json]
|
160
|
+
end
|
143
161
|
end
|
144
162
|
|
145
163
|
if options[:github]
|
146
164
|
@github.verbose = options[:verbose]
|
147
165
|
@github.max_pages = options[:maxpages]
|
166
|
+
@github.verify_ssl_mode = OpenSSL::SSL::VERIFY_NONE unless options[:ssl_verify]
|
167
|
+
@github.raw = true if options[:raw]
|
148
168
|
@github.search
|
149
|
-
@github.summary
|
169
|
+
@github.summary
|
170
|
+
|
150
171
|
puts @github.results if !options[:report] && !options[:json]
|
151
172
|
end
|
152
173
|
|
@@ -155,12 +176,18 @@ module Pastenum
|
|
155
176
|
@pastebin.max_pages = options[:maxpages]
|
156
177
|
@pastebin.search
|
157
178
|
@pastebin.summary
|
158
|
-
|
179
|
+
|
180
|
+
if options[:raw]
|
181
|
+
@pastebin.results.each { |hit| puts "#{@pastebin.raw_url}#{hit}" } if !options[:report] && !options[:json]
|
182
|
+
else
|
183
|
+
@pastebin.results.each { |hit| puts "http://pastebin.com/#{hit}" } if !options[:report] && !options[:json]
|
184
|
+
end
|
159
185
|
end
|
160
186
|
|
161
187
|
if options[:pastie]
|
162
188
|
@pastie.verbose = options[:verbose]
|
163
189
|
@pastie.max_pages = 2
|
190
|
+
@pastie.raw = true if options[:raw]
|
164
191
|
@pastie.search
|
165
192
|
@pastie.summary
|
166
193
|
puts @pastie.results if !options[:report] && !options[:json]
|
data/lib/pastenum/main.rb
CHANGED
@@ -2,8 +2,13 @@ module Pastenum
|
|
2
2
|
|
3
3
|
class TargetTemplateErrorDirectCall < StandardError; end
|
4
4
|
class TargetUnreachable < StandardError; end
|
5
|
+
class IniuriamOrdinis < StandardError; end
|
5
6
|
|
6
|
-
|
7
|
+
module Defaults
|
8
|
+
Raw = false # Pastenum::Defaults::Raw
|
9
|
+
Verbose = false # Pastenum::Defaults::Verbose
|
10
|
+
MaxPages = 2
|
11
|
+
end
|
7
12
|
|
8
13
|
|
9
14
|
end
|
data/lib/pastenum/target.rb
CHANGED
@@ -1,15 +1,32 @@
|
|
1
1
|
module Pastenum
|
2
2
|
class Target
|
3
3
|
|
4
|
-
attr_accessor :max_pages, :verbose
|
5
|
-
attr_reader :dork, :results, :vendor
|
4
|
+
attr_accessor :max_pages, :verbose, :raw
|
5
|
+
attr_reader :dork, :results, :vendor, :raw_url
|
6
6
|
|
7
|
-
def initialize(
|
8
|
-
@dork
|
9
|
-
|
10
|
-
@
|
7
|
+
def initialize(search_string)
|
8
|
+
@dork ||= search_string
|
9
|
+
|
10
|
+
@vendor ||= "TargetTemplate"
|
11
|
+
@raw_url ||= "TargetRawTemplate"
|
12
|
+
|
13
|
+
@results ||= Array.new
|
14
|
+
@agent ||= Mechanize.new
|
15
|
+
|
16
|
+
@raw ||= Pastenum::Defaults::Raw
|
17
|
+
@verbose ||= Pastenum::Defaults::Verbose
|
18
|
+
|
19
|
+
@max_pages ||= Pastenum::Defaults::MaxPages
|
11
20
|
end
|
12
|
-
|
21
|
+
|
22
|
+
def verify_ssl_mode
|
23
|
+
@agent.verify_mode
|
24
|
+
end
|
25
|
+
|
26
|
+
def verify_ssl_mode=(mode)
|
27
|
+
@agent.verify_mode = mode
|
28
|
+
end
|
29
|
+
|
13
30
|
def search
|
14
31
|
raise TargetTemplateErrorDirectCall
|
15
32
|
end
|
@@ -4,10 +4,10 @@ module Pastenum
|
|
4
4
|
|
5
5
|
def initialize(dork)
|
6
6
|
@dork = URI.escape(dork)
|
7
|
-
@
|
8
|
-
@
|
9
|
-
|
10
|
-
|
7
|
+
@vendor = "https://gist.github.com/"
|
8
|
+
@raw_url = "https://raw.github.com/gist/"
|
9
|
+
|
10
|
+
super
|
11
11
|
end
|
12
12
|
|
13
13
|
def search
|
@@ -17,19 +17,27 @@ module Pastenum
|
|
17
17
|
print ".".green if @verbose
|
18
18
|
page = @agent.get("https://gist.github.com/search?page=#{current_page}&q=#{@dork}")
|
19
19
|
page.links.each do |link|
|
20
|
-
|
21
|
-
#
|
20
|
+
if @raw
|
21
|
+
# Raw links do not use username only the code
|
22
|
+
# "4556950"
|
23
|
+
if link.href.match(/\w+\/([0-9]+)/)
|
24
|
+
@results << link.href.match(/\w+\/([0-9]+)/)[1]
|
25
|
+
end
|
26
|
+
else
|
27
|
+
# Example Hits to find stad links need username
|
22
28
|
# "/shadowbq/4556950"
|
23
29
|
# "/shadowbq/2718948"
|
24
|
-
|
25
|
-
|
26
|
-
|
30
|
+
if link.href.match(/(\w+\/[0-9]+)/)
|
31
|
+
@results << link.href.match(/(\w+\/[0-9]+)/)[1]
|
32
|
+
end
|
27
33
|
end
|
34
|
+
|
28
35
|
end
|
29
36
|
current_page += 1
|
30
37
|
end
|
38
|
+
|
31
39
|
puts "\n" if @verbose
|
32
|
-
return @results
|
40
|
+
return @results.uniq! #light years faster than array.include X times along with an extra regex match
|
33
41
|
end
|
34
42
|
|
35
43
|
private
|
@@ -1,13 +1,34 @@
|
|
1
|
-
#Internal www.github.com search scraper
|
1
|
+
#Internal www.github.com search scraper (Code Type)
|
2
|
+
|
3
|
+
# Original pastenum enumerated the repo, we actually want the file.
|
4
|
+
|
5
|
+
# Standard result part 1
|
6
|
+
# https://github.com/ryuzee/PHPMyScrum
|
7
|
+
|
8
|
+
# Standard result part 2 (blob)
|
9
|
+
# https://github.com/ryuzee/PHPMyScrum/blob/71dc45c31220bfa04057f9c65a7dfbc046871fa6/.box
|
10
|
+
|
11
|
+
# Raw url
|
12
|
+
# https://raw.github.com/ryuzee/PHPMyScrum/71dc45c31220bfa04057f9c65a7dfbc046871fa6/.box
|
13
|
+
|
14
|
+
# [8] pry(main)> b = "https://github.com/ryuzee/foo1/blob/71dc45c31220bfa04057f9c65a7dfbc046871fa6/.box"
|
15
|
+
# [9] pry(main)> b.match(/(\w+\/\w+)\/(blob)\/([a-z0-9]+)/)
|
16
|
+
# => #<MatchData
|
17
|
+
# "ryuzee/foo1/blob/71dc45c31220bfa04057f9c65a7dfbc046871fa6"
|
18
|
+
# 1:"ryuzee/foo1"
|
19
|
+
# 2:"blob"
|
20
|
+
# 3:"71dc45c31220bfa04057f9c65a7dfbc046871fa6">
|
21
|
+
|
22
|
+
|
2
23
|
module Pastenum
|
3
24
|
class Github < Target
|
4
25
|
|
5
26
|
def initialize(dork)
|
6
27
|
@dork = URI.escape(dork)
|
7
|
-
@
|
8
|
-
@
|
9
|
-
|
10
|
-
|
28
|
+
@vendor = "https://github.com/"
|
29
|
+
@raw_url = "https://raw.github.com/gist/"
|
30
|
+
|
31
|
+
super
|
11
32
|
end
|
12
33
|
|
13
34
|
def search
|
@@ -17,15 +38,20 @@ module Pastenum
|
|
17
38
|
print ".".green if @verbose
|
18
39
|
page = @agent.get("https://github.com/search?langOverride=&language=&q=#{@dork}&repo=&start_value=#{current_page}&type=Code&x=21&y=22")
|
19
40
|
page.links.each do |link|
|
20
|
-
if link.
|
21
|
-
|
22
|
-
|
41
|
+
if link.href.match(/\/blob/)
|
42
|
+
if @raw
|
43
|
+
matchdata = link.href.match(/([a-zA-Z0-9\-_]+\/[a-zA-Z0-9\-_]+)\/(blob)\/([a-z0-9]+)/)
|
44
|
+
address = "https://raw.github.com/#{matchdata[1]}/#{matchdata[3]}/"
|
45
|
+
@results << address
|
46
|
+
else
|
47
|
+
@results << "https://github.com#{link.href.split("#").first}"
|
48
|
+
end
|
23
49
|
end
|
24
50
|
current_page += 1
|
25
51
|
end
|
26
52
|
end
|
27
53
|
puts "\n" if @verbose
|
28
|
-
return @results
|
54
|
+
return @results.uniq!
|
29
55
|
end
|
30
56
|
|
31
57
|
private
|
@@ -54,11 +80,8 @@ module Pastenum
|
|
54
80
|
end
|
55
81
|
|
56
82
|
if page_count.max > @max_pages
|
57
|
-
#puts "[*] #{page_count.max} pages of results found.".green
|
58
|
-
#puts "[*] Getting the first #{@max_pages} pages".green
|
59
83
|
return @max_pages
|
60
84
|
else
|
61
|
-
#puts "[*] #{page_count.max} pages of results found.".green
|
62
85
|
return page_count.max
|
63
86
|
end
|
64
87
|
end
|
@@ -4,9 +4,10 @@ module Pastenum
|
|
4
4
|
|
5
5
|
def initialize(dork)
|
6
6
|
@dork = dork
|
7
|
-
@max_pages = 25
|
8
|
-
@results = Array.new
|
9
7
|
@vendor = "pastebin.com"
|
8
|
+
@raw_url = "http://pastebin.com/raw.php?i="
|
9
|
+
|
10
|
+
super
|
10
11
|
end
|
11
12
|
|
12
13
|
def search
|
@@ -18,19 +19,12 @@ module Pastenum
|
|
18
19
|
print ".".green if @verbose
|
19
20
|
page = q.page(i)
|
20
21
|
page.each do |result|
|
21
|
-
|
22
|
-
|
23
|
-
|
24
|
-
#
|
25
|
-
|
26
|
-
|
27
|
-
if url4.length == 8
|
28
|
-
if
|
29
|
-
@results.include?(url4) == true
|
30
|
-
else
|
31
|
-
# result.cached_url.to_s
|
32
|
-
@results << url4
|
33
|
-
end
|
22
|
+
url = result.url.to_s.split("&").first.split("=").last
|
23
|
+
# url -> "http://pastebin.com/E7SmXKMs"
|
24
|
+
code = url.split("/").last
|
25
|
+
# code -> "E7SmXKMs"
|
26
|
+
if code.length == 8
|
27
|
+
@results << code
|
34
28
|
end
|
35
29
|
end
|
36
30
|
end
|
@@ -38,7 +32,7 @@ module Pastenum
|
|
38
32
|
raise TargetUnreachable, "[!] ERROR: Google search unreachable , Maybe the googles banned you?"
|
39
33
|
end
|
40
34
|
puts "\n" if @verbose
|
41
|
-
return @results
|
35
|
+
return @results.uniq!
|
42
36
|
end
|
43
37
|
|
44
38
|
end
|
@@ -1,57 +1,47 @@
|
|
1
1
|
#Internal search implementation for pastie.org (deprecated, migrate to Gscraper)
|
2
|
+
# Standard link
|
3
|
+
# http://pastie.org/pastes/6208073
|
4
|
+
# Raw link
|
5
|
+
# http://pastie.org/pastes/6208073/text
|
6
|
+
|
7
|
+
# url -> "http://pastie.org/pastes/6700343"
|
8
|
+
|
2
9
|
module Pastenum
|
3
10
|
class Pastie < Target
|
4
11
|
|
5
12
|
def initialize(dork)
|
6
13
|
@dork = URI.escape(dork)
|
7
|
-
@
|
8
|
-
|
9
|
-
|
14
|
+
@vendor = "http://pastie.org"
|
15
|
+
|
16
|
+
super
|
10
17
|
end
|
11
18
|
|
12
19
|
def search
|
20
|
+
puts "[*] Searching Pastebin.com (Limit: First #{@max_pages} Pages)".green if @verbose
|
21
|
+
q = GScraper::Search.query(:query => @dork + " inurl:pastes -inurl:page", :site => 'pastie.org')
|
13
22
|
print "[*] Parsing pages:".green if @verbose
|
14
|
-
|
15
|
-
|
16
|
-
|
17
|
-
|
18
|
-
|
19
|
-
|
20
|
-
|
23
|
+
begin
|
24
|
+
for i in 1..@max_pages do
|
25
|
+
print ".".green if @verbose
|
26
|
+
page = q.page(i)
|
27
|
+
page.each do |result|
|
28
|
+
if result.url.to_s.match(/pastes\/[0-9]+$/)
|
29
|
+
code = result.url.to_s.split("/").last
|
30
|
+
# code -> "6700343"
|
31
|
+
if @raw
|
32
|
+
@results << "http://pastie.org/pastes/#{code}/text"
|
33
|
+
else
|
34
|
+
@results << "http://pastie.org/pastes/#{code}"
|
35
|
+
end
|
36
|
+
end
|
21
37
|
end
|
22
38
|
end
|
23
|
-
current_page += 1
|
24
|
-
end
|
25
|
-
puts "\n" if @verbose
|
26
|
-
return @results
|
27
|
-
end
|
28
|
-
|
29
|
-
private
|
30
|
-
|
31
|
-
def page_numbers
|
32
|
-
puts "[*] Getting Results".green
|
33
|
-
begin
|
34
|
-
results = @agent.get("http://pastie.org/search?&commit=Start+Search&page=1&q=#{@dork}")
|
35
39
|
rescue
|
36
|
-
raise TargetUnreachable, "[!] ERROR:
|
40
|
+
raise TargetUnreachable, "[!] ERROR: Google search unreachable , Maybe the googles banned you?"
|
37
41
|
end
|
38
|
-
puts "
|
39
|
-
|
40
|
-
results.links.each do |link|
|
41
|
-
if link.href.match(/pastie.org\/pastes\//)
|
42
|
-
page_count << 1
|
43
|
-
else
|
44
|
-
page_count << 0
|
45
|
-
end
|
46
|
-
if link.href.match(/Start%2BSearch&page=/)
|
47
|
-
uri = link.href
|
48
|
-
url = URI("http://pastie.org/search/" + uri)
|
49
|
-
page_num = url.query_params['page']
|
50
|
-
page_count << page_num.to_i
|
51
|
-
end
|
52
|
-
end
|
53
|
-
page_count.max
|
42
|
+
puts "\n" if @verbose
|
43
|
+
return @results.uniq!
|
54
44
|
end
|
55
45
|
|
56
46
|
end
|
57
|
-
end
|
47
|
+
end
|
data/lib/pastenum/version.rb
CHANGED
metadata
CHANGED
@@ -1,7 +1,7 @@
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
2
2
|
name: pastenum
|
3
3
|
version: !ruby/object:Gem::Version
|
4
|
-
version: 0.
|
4
|
+
version: 0.3.0
|
5
5
|
prerelease:
|
6
6
|
platform: ruby
|
7
7
|
authors:
|
@@ -10,7 +10,7 @@ authors:
|
|
10
10
|
autorequire:
|
11
11
|
bindir: bin
|
12
12
|
cert_chain: []
|
13
|
-
date: 2013-02-
|
13
|
+
date: 2013-02-20 00:00:00.000000000 Z
|
14
14
|
dependencies: []
|
15
15
|
description:
|
16
16
|
email:
|
@@ -21,9 +21,10 @@ extensions: []
|
|
21
21
|
extra_rdoc_files: []
|
22
22
|
files:
|
23
23
|
- Gemfile
|
24
|
+
- Gemfile.lock
|
24
25
|
- LICENSE
|
25
26
|
- README.md
|
26
|
-
- RELEASE_NOTES
|
27
|
+
- RELEASE_NOTES.md
|
27
28
|
- bin/pastenum
|
28
29
|
- lib/pastenum.rb
|
29
30
|
- lib/pastenum/cli.rb
|
data/RELEASE_NOTES
DELETED
@@ -1,16 +0,0 @@
|
|
1
|
-
|
2
|
-
RELEASE NOTES:
|
3
|
-
6/05/2011
|
4
|
-
v .2
|
5
|
-
Added BT5 install instructions to README
|
6
|
-
Fixed some formatting
|
7
|
-
Generally Cleaned Code
|
8
|
-
|
9
|
-
3/25/2011
|
10
|
-
v .1 RC2
|
11
|
-
Added github support
|
12
|
-
Added ability to enable and disable searches individually
|
13
|
-
|
14
|
-
3/22/2011
|
15
|
-
v .1 RC1
|
16
|
-
Initial public release
|