pastenum 0.2.2 → 0.3.0

Sign up to get free protection for your applications and to get access to all the features.
data/Gemfile.lock ADDED
@@ -0,0 +1,49 @@
1
+ PATH
2
+ remote: .
3
+ specs:
4
+ pastenum (0.3.0)
5
+
6
+ GEM
7
+ remote: http://rubygems.org/
8
+ specs:
9
+ bump (0.3.9)
10
+ colored (1.2)
11
+ domain_name (0.5.7)
12
+ unf (~> 0.0.3)
13
+ gscraper (0.4.0)
14
+ json (~> 1.6)
15
+ mechanize (~> 2.0)
16
+ uri-query_params (~> 0.5)
17
+ highline (1.6.15)
18
+ json (1.7.7)
19
+ mechanize (2.5.1)
20
+ domain_name (~> 0.5, >= 0.5.1)
21
+ mime-types (~> 1.17, >= 1.17.2)
22
+ net-http-digest_auth (~> 1.1, >= 1.1.1)
23
+ net-http-persistent (~> 2.5, >= 2.5.2)
24
+ nokogiri (~> 1.4)
25
+ ntlm-http (~> 0.1, >= 0.1.1)
26
+ webrobots (~> 0.0, >= 0.0.9)
27
+ mime-types (1.21)
28
+ net-http-digest_auth (1.2.1)
29
+ net-http-persistent (2.8)
30
+ nokogiri (1.5.6)
31
+ ntlm-http (0.1.1)
32
+ unf (0.0.5)
33
+ unf_ext
34
+ unf_ext (0.0.5)
35
+ uri-query_params (0.7.0)
36
+ webrobots (0.0.13)
37
+
38
+ PLATFORMS
39
+ ruby
40
+
41
+ DEPENDENCIES
42
+ bump (~> 0.3)
43
+ colored (~> 1.2)
44
+ gscraper (~> 0.4)
45
+ highline
46
+ json (~> 1.7)
47
+ mechanize (~> 2.0)
48
+ pastenum!
49
+ uri-query_params (~> 0.7)
data/README.md CHANGED
@@ -1,54 +1,92 @@
1
1
  #Pastenum
2
2
 
3
- Pastenum is a gem rewrite of nullthreat's original pastenum2 google scrapper.
3
+ Pastenum is a text dump enumeration tool. It is designed to help find and enumerate datadumps, and doxs posted on public sites.
4
+
5
+ It currently searches sites github.com, gist.github.com, pastebin.com, and pastie.org.
6
+
7
+ Pastenum is a gem rewrite of nullthreat's original [pastenum2](http://redmine.corelan.be/projects/corelan-pastenum) released in 2011.
8
+
9
+ ## Install:
4
10
 
5
- Install:
6
- ------
7
11
  * Install Ruby 1.9.*
8
12
 
9
13
  * Install RubyGems
10
14
 
11
15
  * gem install pastenum
12
16
 
13
- Troubleshooting:
14
- ======
17
+ ### Troubleshooting:
15
18
 
16
19
  ** Note: nokogiri in the gemfile requires native libxslt and libxml2 to be installed
17
20
 
18
- Usage:
19
- ------
20
- `$> pastenum.rb`
21
-
22
- When prompted enter a search term.
23
- Once the application is complete it will generate "report.html"
24
-
25
- ** Note: If more then 100 results are found, it can take some time to load the "report.html" file.
26
-
27
-
28
- Notes on Future Versions:
29
- ------
21
+ ## Usage:
22
+
23
+ ```shell
24
+ (~)$ pastenum -h
25
+ Usage: pastenum [OPTIONS] "search string"
26
+
27
+ Sources::
28
+ -p, --[no-]pastebin Search Pastebin.com (Gscraper)
29
+ Default: true
30
+ -g, --[no-]gist Search Gist.github.com
31
+ Default: true
32
+ -G, --[no-]github Search github.com
33
+ Default: true
34
+ -i, --[no-]pasties Search pastie.org (Gscraper)
35
+ Default: true
36
+ Report Output:: (Default output to STDOUT)
37
+ -H, --html-report Create an html iframed (report.html)
38
+ -J, --json-report Create a JSON file (report.json)
39
+ Options::
40
+ -r, --[no-]raw use 'raw' urls instead if available
41
+ Default: false
42
+ -m, --maxpages= maximum number of search results pages to iterate through
43
+ Default: 2
44
+ -x, --tos Acknowledge Google INC. 'Terms of Service'
45
+ -V, --[no-]ssl-verify Verify SSL certificates
46
+ Default: true
47
+ -v, --verbose Run verbosely
48
+ -h, --help Display this screen
49
+ ```
50
+
51
+ ** Note: If generating a report iframe, it can take some time to load the "report.html" file.**
52
+
53
+ ### Example Search
54
+
55
+ Gist and Github code search for 'shadowbq' skipping ssl cert verfication
56
+
57
+ ```shell
58
+ (~)$ pastenum -x --no-pastebin --no-pastie --no-ssl-verify "shadowbq"
59
+ https://gist.github.com/shadowbq/325430
60
+ https://gist.github.com/shadowbq/4556950
61
+ https://gist.github.com/shadowbq/2718948
62
+ https://github.com/shadowbq/matrix.dot.files/blob/8241fc219a3b4601c6c4b1a487441cf31e90916b/docs/README.vim-colors
63
+ https://github.com/shadowbq/easy-bin/blob/6237442a25777e17a869b5131f0f4f38ca62def1/easy-bin.gemspec
64
+ https://github.com/shadowbq/pastenum/blob/f1d955652a9fd2f7928d7ae84093aea4fc105f85/pastenum.gemspec
65
+ https://github.com/shadowbq/matrix.dot.files/blob/8241fc219a3b4601c6c4b1a487441cf31e90916b/home/.matrix/vim/.gitmodules
66
+ https://github.com/shadowbq/matrix.dot.files/blob/4d1519387fb7bfc1e1134c179fe372f232e30cba/home/.gitconfig
67
+ https://github.com/shadowbq/pastenum/blob/79f620a98f5d6fbb49c805e25b4206fa70429929/README.md
68
+ https://github.com/fabrizioc1/WiFu/blob/d37853fd538319317bb4e4f5d16b3255501b76d1/Gemfile
69
+ https://github.com/shadowbq/matrix.dot.files/blob/8241fc219a3b4601c6c4b1a487441cf31e90916b/home/.ssh/config
70
+ https://github.com/shadowbq/snorby_suite/blob/a5bfc872c6bc8c2691adf3041fa821dadbec233b/snorby_suite.gemspec
71
+ https://github.com/shadowbq/rothparty/blob/22a5e8847592cdc158439faef925ca2a07bb0e4d/rothparty.gemspec
72
+
73
+ ```
74
+
75
+
76
+ ## Original Pastenum Credits:
30
77
 
31
- Gist / Github
32
- ------
33
-
34
- Use the appropriate API for github
35
-
36
- http://developer.github.com/v3/repos/
78
+ Pastenum - A Corelan Team Production
79
+ Written by: Nullthreat
37
80
 
38
- Gscaper -> Google Custom Search API:
39
- ------
81
+ ### Website(s)
40
82
 
41
- Important: The Google Custom Search API requires the use of an API key, which you can get from the Google APIs console. The API provides 100 search queries per day for free. If you need more, you may sign up for billing in the console.
83
+ http://redmine.corelan.be/projects/corelan-pastenum
42
84
 
43
- https://developers.google.com/custom-search/v1/overview
85
+ http://www.corelan.be
44
86
 
45
- Original Pastenum Credits:
46
- ------
87
+ http://www.nullthreat.net
47
88
 
48
- Pastenum - A Corelan Team Production
49
- Written by: Nullthreat
89
+ ### IRC
50
90
 
51
- www.corelan.be
52
- www.nullthreat.net
53
91
  irc.freenode.net #corelan
54
92
 
data/RELEASE_NOTES.md ADDED
@@ -0,0 +1,39 @@
1
+
2
+ ## RELEASE NOTES:
3
+
4
+ 2/13/2013
5
+ v2.2
6
+ Shadowbq - rewrite/refactor code framework. The durability, functionality, and reporting has been enhanced, but the fetch mechanizism (mechanize, gscraper) are still conceptionally original.
7
+
8
+ v2.1
9
+ Shadowbq - code cleanup.
10
+
11
+ 6/05/2011
12
+ v .2
13
+ Added BT5 install instructions to README
14
+ Fixed some formatting
15
+ Generally Cleaned Code
16
+
17
+ 3/25/2011
18
+ v .1 RC2
19
+ Added github support
20
+ Added ability to enable and disable searches individually
21
+
22
+ 3/22/2011
23
+ v .1 RC1
24
+ Initial public release
25
+
26
+ ## Notes on Future Versions:
27
+
28
+ ### Gist / Github
29
+
30
+ Use the appropriate API for github
31
+
32
+ http://developer.github.com/v3/repos/
33
+
34
+ ### Gscaper -> Google Custom Search API:
35
+
36
+
37
+ Important: The Google Custom Search API requires the use of an API key, which you can get from the Google APIs console. The API provides 100 search queries per day for free. If you need more, you may sign up for billing in the console.
38
+
39
+ https://developers.google.com/custom-search/v1/overview
data/lib/pastenum/cli.rb CHANGED
@@ -17,50 +17,56 @@ module Pastenum
17
17
  options[:pastebin] = true
18
18
  options[:gist] = true
19
19
  options[:github] = true
20
- options[:pastie] = false
20
+ options[:pastie] = true
21
21
 
22
+ options[:raw] = Pastenum::Defaults::Raw
22
23
  options[:onion] = false
23
24
  options[:test] = false
24
- options[:verbose] = false
25
- options[:maxpages] = 2
25
+ options[:verbose] = Pastenum::Defaults::Verbose
26
+ options[:maxpages] = Pastenum::Defaults::MaxPages
26
27
  options[:tos] = false
28
+ options[:ssl_verify] = true
27
29
 
28
30
  opt_parser = OptionParser.new do |opt|
29
31
  opt.banner = "Usage: pastenum [OPTIONS] \"search string\""
30
32
  opt.separator ""
31
33
  opt.separator "Sources::"
32
34
 
33
- opt.on("-p","--[no-]pastebin","Search Pastebin.com (Gscraper)"," Default: #{options[:pastebin]}") do
34
- options[:pastebin] = true
35
+ opt.on("-p","--[no-]pastebin","Search Pastebin.com (Gscraper)"," Default: #{options[:pastebin]}") do |value|
36
+ options[:pastebin] = value
35
37
  end
36
38
 
37
- opt.on("-g","--[no-]gist","Search Gist.github.com"," Default: #{options[:gist]}") do
38
- options[:gist] = true
39
+ opt.on("-g","--[no-]gist","Search Gist.github.com"," Default: #{options[:gist]}") do |value|
40
+ options[:gist] = value
39
41
  end
40
42
 
41
- opt.on("-G","--[no-]github","Search github.com"," Default: #{options[:github]}") do
42
- options[:github] = true
43
+ opt.on("-G","--[no-]github","Search github.com"," Default: #{options[:github]}") do |value|
44
+ options[:github] = value
43
45
  end
44
46
 
45
- opt.on("-i","--[no-]pasties","Search pastie.org (Gscraper)"," Default: #{options[:pastie]}") do
46
- options[:pastie] = true
47
+ opt.on("-i","--[no-]pastie","Search pastie.org (Gscraper)"," Default: #{options[:pastie]}") do |value|
48
+ options[:pastie] = value
47
49
  end
48
50
 
49
- opt.separator "Output:: (Default output to STDOUT)"
51
+ opt.separator "Report Output:: (Default output to STDOUT)"
50
52
 
51
- opt.on("-R","--report","Create an iframed report.html ") do
53
+ opt.on("-H","--html-report","Create an html iframed (report.html) ") do
52
54
  options[:report] = true
53
55
  end
54
56
 
55
- opt.on("-J", "--json", "Create a JSON file with results ") do
57
+ opt.on("-J", "--json-report", "Create a JSON file (report.json) ") do
56
58
  options[:json] = true
57
59
  end
58
60
 
59
61
 
60
62
  opt.separator "Options::"
61
63
 
62
- opt.on("-m","--maxpages=","maximum number of search results pages to iterate through" ," Default: #{options[:maxpages]}") do |v|
63
- options[:maxpages] = v
64
+ opt.on("-r","--[no-]raw","use 'raw' urls instead if available" ," Default: #{options[:raw]}") do |value|
65
+ options[:raw] = value
66
+ end
67
+
68
+ opt.on("-m","--maxpages=","maximum number of search results pages to iterate through" ," Default: #{options[:maxpages]}") do |value|
69
+ options[:maxpages] = value
64
70
  end
65
71
 
66
72
  opt.on("-x", "--tos", "Acknowledge Google INC. 'Terms of Service'") do
@@ -75,8 +81,12 @@ module Pastenum
75
81
  options[:test] = v
76
82
  end
77
83
  =end
84
+ opt.on("-V", "--[no-]ssl-verify", "Verify SSL certificates"," Default: #{options[:ssl_verify]}") do |value|
85
+ options[:ssl_verify] = value
86
+ end
87
+
78
88
  opt.on("-v", "--verbose", "Run verbosely") do
79
- options[:verbose] = v
89
+ options[:verbose] = true
80
90
  end
81
91
 
82
92
  opt.on_tail("-h","--help","Display this screen") do
@@ -93,7 +103,10 @@ module Pastenum
93
103
  opt_parser.parse!
94
104
 
95
105
  #If options fail display help
96
- rescue
106
+ #rescue Exception => e
107
+ # puts e.message
108
+ # puts e.backtrace.inspect
109
+ rescue
97
110
  puts opt_parser
98
111
  exit
99
112
  end
@@ -137,16 +150,24 @@ module Pastenum
137
150
 
138
151
  @gist.verbose = options[:verbose]
139
152
  @gist.max_pages = options[:maxpages]
153
+ @gist.verify_ssl_mode = OpenSSL::SSL::VERIFY_NONE unless options[:ssl_verify]
140
154
  @gist.search
141
155
  @gist.summary
142
- @gist.results.each { |hit| puts "https://gist.github.com#{hit}" } if !options[:report] && !options[:json]
156
+ if options[:raw]
157
+ @gist.results.each { |hit| puts "#{@gist.raw_url}#{hit}" } if !options[:report] && !options[:json]
158
+ else
159
+ @gist.results.each { |hit| puts "#{@gist.vendor}#{hit}" } if !options[:report] && !options[:json]
160
+ end
143
161
  end
144
162
 
145
163
  if options[:github]
146
164
  @github.verbose = options[:verbose]
147
165
  @github.max_pages = options[:maxpages]
166
+ @github.verify_ssl_mode = OpenSSL::SSL::VERIFY_NONE unless options[:ssl_verify]
167
+ @github.raw = true if options[:raw]
148
168
  @github.search
149
- @github.summary
169
+ @github.summary
170
+
150
171
  puts @github.results if !options[:report] && !options[:json]
151
172
  end
152
173
 
@@ -155,12 +176,18 @@ module Pastenum
155
176
  @pastebin.max_pages = options[:maxpages]
156
177
  @pastebin.search
157
178
  @pastebin.summary
158
- @pastebin.results.each { |hit| puts "http://pastebin.com#{hit}" } if !options[:report] && !options[:json]
179
+
180
+ if options[:raw]
181
+ @pastebin.results.each { |hit| puts "#{@pastebin.raw_url}#{hit}" } if !options[:report] && !options[:json]
182
+ else
183
+ @pastebin.results.each { |hit| puts "http://pastebin.com/#{hit}" } if !options[:report] && !options[:json]
184
+ end
159
185
  end
160
186
 
161
187
  if options[:pastie]
162
188
  @pastie.verbose = options[:verbose]
163
189
  @pastie.max_pages = 2
190
+ @pastie.raw = true if options[:raw]
164
191
  @pastie.search
165
192
  @pastie.summary
166
193
  puts @pastie.results if !options[:report] && !options[:json]
data/lib/pastenum/main.rb CHANGED
@@ -2,8 +2,13 @@ module Pastenum
2
2
 
3
3
  class TargetTemplateErrorDirectCall < StandardError; end
4
4
  class TargetUnreachable < StandardError; end
5
+ class IniuriamOrdinis < StandardError; end
5
6
 
6
-
7
+ module Defaults
8
+ Raw = false # Pastenum::Defaults::Raw
9
+ Verbose = false # Pastenum::Defaults::Verbose
10
+ MaxPages = 2
11
+ end
7
12
 
8
13
 
9
14
  end
@@ -1,15 +1,32 @@
1
1
  module Pastenum
2
2
  class Target
3
3
 
4
- attr_accessor :max_pages, :verbose
5
- attr_reader :dork, :results, :vendor
4
+ attr_accessor :max_pages, :verbose, :raw
5
+ attr_reader :dork, :results, :vendor, :raw_url
6
6
 
7
- def initialize(dork_url)
8
- @dork = dork_url
9
- @vendor = "TargetTemplate"
10
- @results = Array.new
7
+ def initialize(search_string)
8
+ @dork ||= search_string
9
+
10
+ @vendor ||= "TargetTemplate"
11
+ @raw_url ||= "TargetRawTemplate"
12
+
13
+ @results ||= Array.new
14
+ @agent ||= Mechanize.new
15
+
16
+ @raw ||= Pastenum::Defaults::Raw
17
+ @verbose ||= Pastenum::Defaults::Verbose
18
+
19
+ @max_pages ||= Pastenum::Defaults::MaxPages
11
20
  end
12
-
21
+
22
+ def verify_ssl_mode
23
+ @agent.verify_mode
24
+ end
25
+
26
+ def verify_ssl_mode=(mode)
27
+ @agent.verify_mode = mode
28
+ end
29
+
13
30
  def search
14
31
  raise TargetTemplateErrorDirectCall
15
32
  end
@@ -4,10 +4,10 @@ module Pastenum
4
4
 
5
5
  def initialize(dork)
6
6
  @dork = URI.escape(dork)
7
- @agent = Mechanize.new
8
- @max_pages = 25
9
- @results = Array.new
10
- @vendor = "gist.github.com"
7
+ @vendor = "https://gist.github.com/"
8
+ @raw_url = "https://raw.github.com/gist/"
9
+
10
+ super
11
11
  end
12
12
 
13
13
  def search
@@ -17,19 +17,27 @@ module Pastenum
17
17
  print ".".green if @verbose
18
18
  page = @agent.get("https://gist.github.com/search?page=#{current_page}&q=#{@dork}")
19
19
  page.links.each do |link|
20
-
21
- # Example Hits to find
20
+ if @raw
21
+ # Raw links do not use username only the code
22
+ # "4556950"
23
+ if link.href.match(/\w+\/([0-9]+)/)
24
+ @results << link.href.match(/\w+\/([0-9]+)/)[1]
25
+ end
26
+ else
27
+ # Example Hits to find stad links need username
22
28
  # "/shadowbq/4556950"
23
29
  # "/shadowbq/2718948"
24
-
25
- if link.href.match(/\w+\/[0-9]+/)
26
- @results << link.href unless @results.include?(link.href)
30
+ if link.href.match(/(\w+\/[0-9]+)/)
31
+ @results << link.href.match(/(\w+\/[0-9]+)/)[1]
32
+ end
27
33
  end
34
+
28
35
  end
29
36
  current_page += 1
30
37
  end
38
+
31
39
  puts "\n" if @verbose
32
- return @results
40
+ return @results.uniq! #light years faster than array.include X times along with an extra regex match
33
41
  end
34
42
 
35
43
  private
@@ -1,13 +1,34 @@
1
- #Internal www.github.com search scraper
1
+ #Internal www.github.com search scraper (Code Type)
2
+
3
+ # Original pastenum enumerated the repo, we actually want the file.
4
+
5
+ # Standard result part 1
6
+ # https://github.com/ryuzee/PHPMyScrum
7
+
8
+ # Standard result part 2 (blob)
9
+ # https://github.com/ryuzee/PHPMyScrum/blob/71dc45c31220bfa04057f9c65a7dfbc046871fa6/.box
10
+
11
+ # Raw url
12
+ # https://raw.github.com/ryuzee/PHPMyScrum/71dc45c31220bfa04057f9c65a7dfbc046871fa6/.box
13
+
14
+ # [8] pry(main)> b = "https://github.com/ryuzee/foo1/blob/71dc45c31220bfa04057f9c65a7dfbc046871fa6/.box"
15
+ # [9] pry(main)> b.match(/(\w+\/\w+)\/(blob)\/([a-z0-9]+)/)
16
+ # => #<MatchData
17
+ # "ryuzee/foo1/blob/71dc45c31220bfa04057f9c65a7dfbc046871fa6"
18
+ # 1:"ryuzee/foo1"
19
+ # 2:"blob"
20
+ # 3:"71dc45c31220bfa04057f9c65a7dfbc046871fa6">
21
+
22
+
2
23
  module Pastenum
3
24
  class Github < Target
4
25
 
5
26
  def initialize(dork)
6
27
  @dork = URI.escape(dork)
7
- @agent = Mechanize.new
8
- @max_pages = 25
9
- @results = Array.new
10
- @vendor = "github.com"
28
+ @vendor = "https://github.com/"
29
+ @raw_url = "https://raw.github.com/gist/"
30
+
31
+ super
11
32
  end
12
33
 
13
34
  def search
@@ -17,15 +38,20 @@ module Pastenum
17
38
  print ".".green if @verbose
18
39
  page = @agent.get("https://github.com/search?langOverride=&language=&q=#{@dork}&repo=&start_value=#{current_page}&type=Code&x=21&y=22")
19
40
  page.links.each do |link|
20
- if link.text.match(/\//)
21
- address = "https://github.com#{link.href}"
22
- @results << address unless @results.include?(address)
41
+ if link.href.match(/\/blob/)
42
+ if @raw
43
+ matchdata = link.href.match(/([a-zA-Z0-9\-_]+\/[a-zA-Z0-9\-_]+)\/(blob)\/([a-z0-9]+)/)
44
+ address = "https://raw.github.com/#{matchdata[1]}/#{matchdata[3]}/"
45
+ @results << address
46
+ else
47
+ @results << "https://github.com#{link.href.split("#").first}"
48
+ end
23
49
  end
24
50
  current_page += 1
25
51
  end
26
52
  end
27
53
  puts "\n" if @verbose
28
- return @results
54
+ return @results.uniq!
29
55
  end
30
56
 
31
57
  private
@@ -54,11 +80,8 @@ module Pastenum
54
80
  end
55
81
 
56
82
  if page_count.max > @max_pages
57
- #puts "[*] #{page_count.max} pages of results found.".green
58
- #puts "[*] Getting the first #{@max_pages} pages".green
59
83
  return @max_pages
60
84
  else
61
- #puts "[*] #{page_count.max} pages of results found.".green
62
85
  return page_count.max
63
86
  end
64
87
  end
@@ -4,9 +4,10 @@ module Pastenum
4
4
 
5
5
  def initialize(dork)
6
6
  @dork = dork
7
- @max_pages = 25
8
- @results = Array.new
9
7
  @vendor = "pastebin.com"
8
+ @raw_url = "http://pastebin.com/raw.php?i="
9
+
10
+ super
10
11
  end
11
12
 
12
13
  def search
@@ -18,19 +19,12 @@ module Pastenum
18
19
  print ".".green if @verbose
19
20
  page = q.page(i)
20
21
  page.each do |result|
21
- result_url = result.url
22
- url2 = result_url.to_s.split("&").first
23
- url3 = url2.split("=").last
24
- # url3 -> "http://pastebin.com/E7SmXKMs"
25
- url4 = url3.split("/").last
26
- # url4 -> "E7SmXKMs"
27
- if url4.length == 8
28
- if
29
- @results.include?(url4) == true
30
- else
31
- # result.cached_url.to_s
32
- @results << url4
33
- end
22
+ url = result.url.to_s.split("&").first.split("=").last
23
+ # url -> "http://pastebin.com/E7SmXKMs"
24
+ code = url.split("/").last
25
+ # code -> "E7SmXKMs"
26
+ if code.length == 8
27
+ @results << code
34
28
  end
35
29
  end
36
30
  end
@@ -38,7 +32,7 @@ module Pastenum
38
32
  raise TargetUnreachable, "[!] ERROR: Google search unreachable , Maybe the googles banned you?"
39
33
  end
40
34
  puts "\n" if @verbose
41
- return @results
35
+ return @results.uniq!
42
36
  end
43
37
 
44
38
  end
@@ -1,57 +1,47 @@
1
1
  #Internal search implementation for pastie.org (deprecated, migrate to Gscraper)
2
+ # Standard link
3
+ # http://pastie.org/pastes/6208073
4
+ # Raw link
5
+ # http://pastie.org/pastes/6208073/text
6
+
7
+ # url -> "http://pastie.org/pastes/6700343"
8
+
2
9
  module Pastenum
3
10
  class Pastie < Target
4
11
 
5
12
  def initialize(dork)
6
13
  @dork = URI.escape(dork)
7
- @agent = Mechanize.new
8
- @results = Array.new
9
- @vendor = "pastie.org"
14
+ @vendor = "http://pastie.org"
15
+
16
+ super
10
17
  end
11
18
 
12
19
  def search
20
+ puts "[*] Searching Pastebin.com (Limit: First #{@max_pages} Pages)".green if @verbose
21
+ q = GScraper::Search.query(:query => @dork + " inurl:pastes -inurl:page", :site => 'pastie.org')
13
22
  print "[*] Parsing pages:".green if @verbose
14
- current_page = 1
15
- page_numbers.times do
16
- print ".".green if @verbose
17
- page = @agent.get("http://pastie.org/search?&commit=Start+Search&page=#{current_page}&q=#{@dork}")
18
- page.links.each do |link|
19
- if link.href.match(/pastie\.org\/pastes/)
20
- @results << link.href
23
+ begin
24
+ for i in 1..@max_pages do
25
+ print ".".green if @verbose
26
+ page = q.page(i)
27
+ page.each do |result|
28
+ if result.url.to_s.match(/pastes\/[0-9]+$/)
29
+ code = result.url.to_s.split("/").last
30
+ # code -> "6700343"
31
+ if @raw
32
+ @results << "http://pastie.org/pastes/#{code}/text"
33
+ else
34
+ @results << "http://pastie.org/pastes/#{code}"
35
+ end
36
+ end
21
37
  end
22
38
  end
23
- current_page += 1
24
- end
25
- puts "\n" if @verbose
26
- return @results
27
- end
28
-
29
- private
30
-
31
- def page_numbers
32
- puts "[*] Getting Results".green
33
- begin
34
- results = @agent.get("http://pastie.org/search?&commit=Start+Search&page=1&q=#{@dork}")
35
39
  rescue
36
- raise TargetUnreachable, "[!] ERROR: Can not load pastie.org - Check Connectivity"
40
+ raise TargetUnreachable, "[!] ERROR: Google search unreachable , Maybe the googles banned you?"
37
41
  end
38
- puts "[*] Searching Pastie.org (Limit: 1000 Results)".green if @verbose
39
- page_count = []
40
- results.links.each do |link|
41
- if link.href.match(/pastie.org\/pastes\//)
42
- page_count << 1
43
- else
44
- page_count << 0
45
- end
46
- if link.href.match(/Start%2BSearch&page=/)
47
- uri = link.href
48
- url = URI("http://pastie.org/search/" + uri)
49
- page_num = url.query_params['page']
50
- page_count << page_num.to_i
51
- end
52
- end
53
- page_count.max
42
+ puts "\n" if @verbose
43
+ return @results.uniq!
54
44
  end
55
45
 
56
46
  end
57
- end
47
+ end
@@ -1,3 +1,3 @@
1
1
  module Pastenum
2
- VERSION = '0.2.2'
2
+ VERSION = '0.3.0'
3
3
  end
metadata CHANGED
@@ -1,7 +1,7 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: pastenum
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.2
4
+ version: 0.3.0
5
5
  prerelease:
6
6
  platform: ruby
7
7
  authors:
@@ -10,7 +10,7 @@ authors:
10
10
  autorequire:
11
11
  bindir: bin
12
12
  cert_chain: []
13
- date: 2013-02-14 00:00:00.000000000 Z
13
+ date: 2013-02-20 00:00:00.000000000 Z
14
14
  dependencies: []
15
15
  description:
16
16
  email:
@@ -21,9 +21,10 @@ extensions: []
21
21
  extra_rdoc_files: []
22
22
  files:
23
23
  - Gemfile
24
+ - Gemfile.lock
24
25
  - LICENSE
25
26
  - README.md
26
- - RELEASE_NOTES
27
+ - RELEASE_NOTES.md
27
28
  - bin/pastenum
28
29
  - lib/pastenum.rb
29
30
  - lib/pastenum/cli.rb
data/RELEASE_NOTES DELETED
@@ -1,16 +0,0 @@
1
-
2
- RELEASE NOTES:
3
- 6/05/2011
4
- v .2
5
- Added BT5 install instructions to README
6
- Fixed some formatting
7
- Generally Cleaned Code
8
-
9
- 3/25/2011
10
- v .1 RC2
11
- Added github support
12
- Added ability to enable and disable searches individually
13
-
14
- 3/22/2011
15
- v .1 RC1
16
- Initial public release