seo_params 0.0.1

Sign up to get free protection for your applications and to get access to all the features.
data/.gitignore ADDED
@@ -0,0 +1,18 @@
1
+ *~
2
+ *.gem
3
+ *.rbc
4
+ .bundle
5
+ .config
6
+ .yardoc
7
+ Gemfile.lock
8
+ InstalledFiles
9
+ _yardoc
10
+ coverage
11
+ doc/
12
+ lib/bundler/man
13
+ pkg
14
+ rdoc
15
+ spec/reports
16
+ test/tmp
17
+ test/version_tmp
18
+ tmp
data/Gemfile ADDED
@@ -0,0 +1,4 @@
1
+ source 'https://rubygems.org'
2
+
3
+ # Specify your gem's dependencies in seo_params.gemspec
4
+ gemspec
data/LICENSE ADDED
@@ -0,0 +1,22 @@
1
+ Copyright (c) 2012 none
2
+
3
+ MIT License
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining
6
+ a copy of this software and associated documentation files (the
7
+ "Software"), to deal in the Software without restriction, including
8
+ without limitation the rights to use, copy, modify, merge, publish,
9
+ distribute, sublicense, and/or sell copies of the Software, and to
10
+ permit persons to whom the Software is furnished to do so, subject to
11
+ the following conditions:
12
+
13
+ The above copyright notice and this permission notice shall be
14
+ included in all copies or substantial portions of the Software.
15
+
16
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
17
+ EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
18
+ MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
19
+ NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
20
+ LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
21
+ OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
22
+ WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,158 @@
1
+ # SeoParams
2
+
3
+ This is a small application for seo purposes: you can get Google PageRank for your site, number of pages in Google index, Yandex tIC, number of pages in Yandex index.
4
+
5
+ In addition, you can check the position of your site in the search for keywords in search engines.
6
+
7
+ ## Installation
8
+
9
+ Add this line to your application's Gemfile:
10
+
11
+ ``` ruby
12
+ gem 'seo_params'
13
+ ```
14
+
15
+ And then execute:
16
+
17
+ ``` bash
18
+ bundle
19
+ ```
20
+
21
+ Or install it yourself as:
22
+
23
+ ``` bash
24
+ gem install PageRankr
25
+ ```
26
+
27
+ ## Usage
28
+
29
+ ``` ruby
30
+ require 'seo_params'
31
+ ```
32
+ ### Fetch main SEO params
33
+
34
+ ``` ruby
35
+ SeoParams.all("gihub.com")
36
+ # => {"pr"=>7, "gp"=>33400000, "tic"=>3700, "yap"=>627119}
37
+ ```
38
+ Short description:
39
+ `pr` - Google PageRank,
40
+ `gp` - pages in Google index,
41
+ `tic` - Yandex tIC,
42
+ `yap` - pages in Yandex index.
43
+
44
+ ### Fetch specific SEO parameter
45
+
46
+ To fetch only Google PageRank:
47
+
48
+ ``` ruby
49
+ SeoParams.pr("gihub.com")
50
+ # => 7
51
+ ```
52
+
53
+ To fetch only pages in Google index:
54
+
55
+ ``` ruby
56
+ SeoParams.gp("gihub.com")
57
+ # => 44000000
58
+ ```
59
+ This request gives sometimes very strange results
60
+
61
+ To fetch only Yandex tIC:
62
+
63
+ ``` ruby
64
+ SeoParams.tic("gihub.com")
65
+ # => 3700
66
+ ```
67
+
68
+ To fetch only pages in Google index:
69
+
70
+ ``` ruby
71
+ SeoParams.yap("gihub.com")
72
+ # => 627119
73
+ ```
74
+
75
+ ### Check site position in the search for keywords in Google
76
+
77
+ The most visited sites - are sites on the first page of a search result the user. That is why it is so important control whether the position of your website for keywords.
78
+
79
+ To check the position of your site in Google for specific keywords:
80
+
81
+ ``` ruby
82
+ SeoParams.gposition("www.none.com.ua", "rails")
83
+ # => {"rails"=>19}
84
+ ```
85
+ or you can specify an array of keywords:
86
+
87
+ ``` ruby
88
+ keywords = ["rails", "ruby", "rvm", "spine.js"]
89
+ SeoParams.gposition("www.none.com.ua", keywords)
90
+ # => {"spine.js"=>1, "rails"=>19, "ruby"=>72, "rvm"=>9}
91
+ ```
92
+
93
+ Because the application was written for my purposes, the default search is performed in Russian for the country of Ukraine with 100 of results (-: But you can always change these settings. You have to specify the following:
94
+
95
+ `:hl` - specifies the interface language (host language) of your user interface. To improve the performance and the quality of your search results, you are strongly encouraged to set this parameter explicitly. List of languages you can find [here][1].
96
+
97
+ `:cr` - restricts search results to documents originating in a particular country. List of countries you can find [here][2].
98
+
99
+ `:num` - identifies the number of search results to return. I usually use 10 or 100.
100
+
101
+ The previous example can be rewritten as follows:
102
+
103
+ ``` ruby
104
+ keywords = ["rails", "ruby", "rvm", "spine.js"]
105
+ SeoParams.gposition("www.none.com.ua", keywords, :hl => "ru", :cr => "countryUA", :num => 100)
106
+ # => {"spine.js"=>1, "rvm"=>9, "rails"=>19, "ruby"=>72}
107
+ ```
108
+
109
+
110
+ [1]: https://developers.google.com/custom-search/docs/xml_results?hl=en#interfaceLanguages
111
+ [2]: https://developers.google.com/custom-search/docs/xml_results?hl=en#countryCollections
112
+
113
+
114
+ ### Check site position in the search for keywords in Yandex
115
+
116
+ Checking the position of keywords in Yandex need only for RuNET, so further description will be in Russian.
117
+
118
+ С проверкой позиций в Яндексе всё несколько сложнее.
119
+
120
+ Во-первых, Вам необходимо быть зарегестрированным пользователем Яндекса. Сделать это можно по вот этой [ссылке][1].
121
+ Во-вторых, добавить IP-адрес, с которого будут посылаться запросы к сервису Яндекса, на этой [страничке][2]
122
+ В-третьих, скопировать/сохранить `user` и `key` из поля "Ваш адрес для совершения запроса" на страничке в предыдущем пункте. Важно знать, что просто зарегистрированные пользователи могут делать не более 10 запросов в день. Если Вы подтвердите Ваш номер телефона, то сможете делать до 1000 запросов в день. Думаю, для многих этого будет достаточно, а кому и этого мало, тогда следует ознакомиться с информацией на самом сайте, как увеличить данное число.
123
+
124
+ В принципе, уже можно пробовать:
125
+
126
+ ``` ruby
127
+ keywords = ["rails", "ruby", "rvm", "spine.js"]
128
+ SeoParams.yaposition("www.none.com.ua", user, key, keywords)
129
+ # => {"ruby"=>0, "spine.js"=>2, "rvm"=>3, "rails"=>22}
130
+ ```
131
+
132
+ Опять же, поиск осуществляется по Украине и при 100 результатах в выдаче. Эти параметры можно изменить:
133
+
134
+ `:lr` - идентификатор страны или региона поиска. Список идентификаторов часто используемых стран и регионов приведен в [приложении][3].
135
+
136
+ `:num` - количество результатов в выдаче поиска. По умолчанию - 100. Ещё одно рекомендуемое значение 10.
137
+
138
+ Приведённый выше пример можно переписать следующим образом:
139
+
140
+ ``` ruby
141
+ keywords = ["rails", "ruby", "rvm", "spine.js"]
142
+ SeoParams.yaposition("www.none.com.ua", user, key, keywords, :lr => 187, :num => 100)
143
+ # => {"ruby"=>0, "spine.js"=>2, "rvm"=>3, "rails"=>22}
144
+ ```
145
+ Значение "0" в результатах поиска свидетельствует об отсутствии сайта в результате поиска по данному клучевому слову.
146
+
147
+ [1]: http://passport.yandex.ru/passport?mode=register
148
+ [2]: http://xml.yandex.ru/settings.xml
149
+ [3]: http://api.yandex.ru/xml/doc/dg/reference/regions.xml
150
+
151
+
152
+ ## Contributing
153
+
154
+ 1. Fork it
155
+ 2. Create your feature branch (`git checkout -b my-new-feature`)
156
+ 3. Commit your changes (`git commit -am 'Added some feature'`)
157
+ 4. Push to the branch (`git push origin my-new-feature`)
158
+ 5. Create new Pull Request
data/Rakefile ADDED
@@ -0,0 +1,2 @@
1
+ #!/usr/bin/env rake
2
+ require "bundler/gem_tasks"
@@ -0,0 +1,86 @@
1
+ # -*- encoding: utf-8 -*-
2
+
3
+ require "page_rankr"
4
+
5
+ require "em-synchrony"
6
+ require "em-synchrony/em-http"
7
+ require "em-synchrony/fiber_iterator"
8
+ require 'nokogiri'
9
+ require 'open-uri'
10
+
11
+ module SeoParams
12
+
13
+ class Google
14
+
15
+ def initialize(url)
16
+ @url = url
17
+ @host = get_host
18
+ end
19
+
20
+
21
+ def pagerank
22
+ query = PageRankr.ranks(@url, :google)
23
+ query[:google]
24
+ end
25
+
26
+ def google_pages
27
+ doc = Nokogiri::HTML(open("https://www.google.com/search?hl=en&tab=ww&safe=active&tbo=d&sclient=psy-ab&q=site:#{@url}&oq=site:#{@url}"))
28
+ pages = doc.css('div[@id="resultStats"]').to_s[/[\d,]+/] if doc.css('div[@id="subform_ctrl"]')
29
+ pages ? pages.tr(',', '').to_i : 0
30
+ end
31
+
32
+ def google_position(hl, cr, keywords, num)
33
+
34
+ h = Hash.new
35
+
36
+ EventMachine.synchrony do
37
+
38
+ EM::Synchrony::FiberIterator.new(keywords, keywords.size).each do |keyword|
39
+
40
+ new_keyword = check_keyword(keyword)
41
+
42
+ uri="http://www.google.com/search?hl=#{hl}&q=#{new_keyword}&btnG=Поиск+в+Google&&cr=#{cr}&meta=&num=#{num}"
43
+ resp = EventMachine::HttpRequest.new(uri).get
44
+ h[keyword] = parse_results resp.response
45
+ end
46
+ EventMachine.stop
47
+ end
48
+
49
+ h
50
+
51
+ end
52
+
53
+ private
54
+
55
+ def check_keyword(keyword)
56
+ (keyword.include? ' ') ? (keyword.tr(' ', '+')) : keyword
57
+ end
58
+
59
+ def parse_results response
60
+ pos = 0
61
+ i = 1
62
+ doc = Nokogiri::HTML(response)
63
+
64
+ doc.xpath('//ol/li/h3/a').each do |link|
65
+ url = link['href'][/(https?:\/\/[\S]+)(&sa)/,1]
66
+
67
+
68
+ if url.to_s[/#{Regexp.escape(@host)}/]
69
+ pos = i
70
+ break
71
+ else
72
+ i = i + 1
73
+ end
74
+ end
75
+ pos
76
+ end
77
+
78
+ def get_host
79
+ @url.match(/^(http:\/\/)/) ? full_url = @url : full_url = 'http://' + @url
80
+ uri = URI(full_url)
81
+ @host = uri.host
82
+ end
83
+
84
+ end
85
+
86
+ end
@@ -0,0 +1,3 @@
1
+ module SeoParams
2
+ VERSION = "0.0.1"
3
+ end
@@ -0,0 +1,114 @@
1
+ # -*- encoding: utf-8 -*-
2
+ require 'nokogiri'
3
+ require 'open-uri'
4
+
5
+ module SeoParams
6
+
7
+ class Yandex
8
+
9
+ def initialize(url)
10
+ url.match(/^(https?:\/\/)/) ? @url = url : @url = 'http://' + url
11
+ @host = URI(@url).host
12
+ end
13
+
14
+
15
+ def tic
16
+ query = Nokogiri::XML(open("http://bar-navig.yandex.ru/u?ver=2&show=32&url=#{@url}"))
17
+ tic = query.xpath('//@value')
18
+ tic.to_s.to_i
19
+ end
20
+
21
+ def yandex_pages
22
+ pages = ask_yandex(@url)
23
+ (pages.is_a? String) ? (url = pages; pages = ask_yandex(url); ) : pages
24
+ pages
25
+ end
26
+
27
+ def yandex_position(user, key, lr, keywords, num)
28
+
29
+ uri = URI.parse "http://xmlsearch.yandex.ru/xmlsearch?user=#{user}&key=#{key}&lr=#{lr}"
30
+
31
+ h = Hash.new
32
+
33
+ EventMachine.synchrony do
34
+ EM::Synchrony::FiberIterator.new(keywords, keywords.size).each do |keyword|
35
+ request = EventMachine::HttpRequest.new(uri)
36
+ response = request.post(:body => xml_request(keyword, num))
37
+
38
+ result = parse_results(response)
39
+
40
+ (result.is_a? Hash) ? (h.merge! result) : (h[keyword] = result)
41
+
42
+ end
43
+
44
+ EventMachine.stop
45
+ end
46
+
47
+ h
48
+
49
+ end
50
+
51
+
52
+ private
53
+ def ask_yandex(url)
54
+
55
+ doc = Nokogiri::HTML(open("http://webmaster.yandex.ua/check.xml?hostname=#{url}"))
56
+
57
+ if doc.css('div.error-message').length > 0
58
+ if doc.css('div.error-message').to_s.scan('Сайт является зеркалом')
59
+ url_regexp = /\?hostname\=([a-zA-Z\.:\/0-9]{1,})/
60
+ new_url = doc.css('div.error-message').to_s.scan(url_regexp)[0][0]
61
+ index = new_url
62
+ end
63
+
64
+ else
65
+ doc.css('div.header div').each do |link|
66
+ pages = /\d{1,}/
67
+ @pages_in_index = link.content.scan(pages)
68
+ index = @pages_in_index[0].to_i
69
+ end
70
+ end
71
+
72
+ index
73
+ end
74
+
75
+ def xml_request keyword, num
76
+ "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
77
+ <request>
78
+ <query>#{keyword}</query>
79
+ <groupings>
80
+ <groupby attr=\"d\" mode=\"deep\" groups-on-page=\"#{num}\" docs-in-group=\"1\" />
81
+ </groupings>
82
+ </request>"
83
+ end
84
+
85
+ def parse_results response
86
+ h_err = Hash.new
87
+ pos = 0
88
+ i = 1
89
+ doc = Nokogiri::XML(response.response)
90
+
91
+ if doc.xpath('//error')
92
+ doc.xpath("//error").map do |err|
93
+ h_err["error_code"] = err['code']
94
+ h_err["error_message"] = err.text()
95
+ end
96
+ end
97
+
98
+
99
+ doc.xpath('//url').each do |link|
100
+ if link.to_s[/#{Regexp.escape(@host)}/]
101
+ pos = i
102
+ break
103
+ else
104
+ i = i + 1
105
+ end
106
+ end
107
+
108
+
109
+ (h_err.length != 0) ? h_err : pos
110
+
111
+ end
112
+ end
113
+
114
+ end
data/lib/seo_params.rb ADDED
@@ -0,0 +1,56 @@
1
+ # -*- encoding: utf-8 -*-
2
+
3
+ require File.expand_path('../seo_params/google', __FILE__)
4
+ require File.expand_path('../seo_params/yandex', __FILE__)
5
+
6
+ module SeoParams
7
+
8
+ class << self
9
+
10
+ def all(url)
11
+ h = Hash.new
12
+ h["pr"] = pr(url)
13
+ h["gp"] = gp(url)
14
+ h["tic"] = tic(url)
15
+ h["yap"] = yap(url)
16
+ h
17
+ end
18
+
19
+ def pr(url)
20
+ Google.new(url).pagerank
21
+ end
22
+
23
+ def tic(url)
24
+ Yandex.new(url).tic
25
+ end
26
+
27
+ def yap(url)
28
+ Yandex.new(url).yandex_pages
29
+ end
30
+
31
+ def gp(url)
32
+ Google.new(url).google_pages
33
+ end
34
+
35
+ def yaposition(url, user, key, keywords, options = {})
36
+ options[:lr] ||= 187
37
+ options[:num] ||= 100
38
+
39
+ (keywords.is_a? Array) ? keywords : keywords = Array.new.push(keywords)
40
+
41
+ Yandex.new(url).yandex_position(user, key, options[:lr], keywords, options[:num])
42
+ end
43
+
44
+ def gposition(url, keywords, options = {})
45
+ options[:hl] ||= "ru"
46
+ options[:cr] ||= "countryUA"
47
+ options[:num] ||= 100
48
+
49
+ (keywords.is_a? Array) ? keywords : keywords = Array.new.push(keywords)
50
+
51
+ Google.new(url).google_position(options[:hl], options[:cr], keywords, options[:num])
52
+ end
53
+
54
+ end
55
+
56
+ end
@@ -0,0 +1,22 @@
1
+ # -*- encoding: utf-8 -*-
2
+ require File.expand_path('../lib/seo_params/version', __FILE__)
3
+
4
+ Gem::Specification.new do |gem|
5
+ gem.authors = ["Alexander Podkidyshev aka none"]
6
+ gem.email = ["alexander.podkidyshev@gmail.com"]
7
+ gem.description = %q{"Easy way to retrieve main SEO parameters."}
8
+ gem.summary = %q{"Easy way to retrieve main SEO parameters: Google PageRank, Yandex tIC, backlinks, SE site positions and etc."}
9
+ gem.homepage = "https://github.com/n0ne/seo_params"
10
+
11
+ gem.add_runtime_dependency "nokogiri", ">= 1.5.0"
12
+ gem.add_runtime_dependency "page_rankr", ">= 3.2.1"
13
+ gem.add_runtime_dependency "em-http-request", ">= 1.0.3"
14
+ gem.add_runtime_dependency "em-synchrony", ">= 1.0.2"
15
+
16
+ gem.files = `git ls-files`.split($\)
17
+ gem.executables = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
18
+ gem.test_files = gem.files.grep(%r{^(test|spec|features)/})
19
+ gem.name = "seo_params"
20
+ gem.require_paths = ["lib"]
21
+ gem.version = SeoParams::VERSION
22
+ end
metadata ADDED
@@ -0,0 +1,120 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: seo_params
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.0.1
5
+ prerelease:
6
+ platform: ruby
7
+ authors:
8
+ - Alexander Podkidyshev aka none
9
+ autorequire:
10
+ bindir: bin
11
+ cert_chain: []
12
+ date: 2012-11-16 00:00:00.000000000 Z
13
+ dependencies:
14
+ - !ruby/object:Gem::Dependency
15
+ name: nokogiri
16
+ requirement: !ruby/object:Gem::Requirement
17
+ none: false
18
+ requirements:
19
+ - - ! '>='
20
+ - !ruby/object:Gem::Version
21
+ version: 1.5.0
22
+ type: :runtime
23
+ prerelease: false
24
+ version_requirements: !ruby/object:Gem::Requirement
25
+ none: false
26
+ requirements:
27
+ - - ! '>='
28
+ - !ruby/object:Gem::Version
29
+ version: 1.5.0
30
+ - !ruby/object:Gem::Dependency
31
+ name: page_rankr
32
+ requirement: !ruby/object:Gem::Requirement
33
+ none: false
34
+ requirements:
35
+ - - ! '>='
36
+ - !ruby/object:Gem::Version
37
+ version: 3.2.1
38
+ type: :runtime
39
+ prerelease: false
40
+ version_requirements: !ruby/object:Gem::Requirement
41
+ none: false
42
+ requirements:
43
+ - - ! '>='
44
+ - !ruby/object:Gem::Version
45
+ version: 3.2.1
46
+ - !ruby/object:Gem::Dependency
47
+ name: em-http-request
48
+ requirement: !ruby/object:Gem::Requirement
49
+ none: false
50
+ requirements:
51
+ - - ! '>='
52
+ - !ruby/object:Gem::Version
53
+ version: 1.0.3
54
+ type: :runtime
55
+ prerelease: false
56
+ version_requirements: !ruby/object:Gem::Requirement
57
+ none: false
58
+ requirements:
59
+ - - ! '>='
60
+ - !ruby/object:Gem::Version
61
+ version: 1.0.3
62
+ - !ruby/object:Gem::Dependency
63
+ name: em-synchrony
64
+ requirement: !ruby/object:Gem::Requirement
65
+ none: false
66
+ requirements:
67
+ - - ! '>='
68
+ - !ruby/object:Gem::Version
69
+ version: 1.0.2
70
+ type: :runtime
71
+ prerelease: false
72
+ version_requirements: !ruby/object:Gem::Requirement
73
+ none: false
74
+ requirements:
75
+ - - ! '>='
76
+ - !ruby/object:Gem::Version
77
+ version: 1.0.2
78
+ description: ! '"Easy way to retrieve main SEO parameters."'
79
+ email:
80
+ - alexander.podkidyshev@gmail.com
81
+ executables: []
82
+ extensions: []
83
+ extra_rdoc_files: []
84
+ files:
85
+ - .gitignore
86
+ - Gemfile
87
+ - LICENSE
88
+ - README.md
89
+ - Rakefile
90
+ - lib/seo_params.rb
91
+ - lib/seo_params/google.rb
92
+ - lib/seo_params/version.rb
93
+ - lib/seo_params/yandex.rb
94
+ - seo_params.gemspec
95
+ homepage: https://github.com/n0ne/seo_params
96
+ licenses: []
97
+ post_install_message:
98
+ rdoc_options: []
99
+ require_paths:
100
+ - lib
101
+ required_ruby_version: !ruby/object:Gem::Requirement
102
+ none: false
103
+ requirements:
104
+ - - ! '>='
105
+ - !ruby/object:Gem::Version
106
+ version: '0'
107
+ required_rubygems_version: !ruby/object:Gem::Requirement
108
+ none: false
109
+ requirements:
110
+ - - ! '>='
111
+ - !ruby/object:Gem::Version
112
+ version: '0'
113
+ requirements: []
114
+ rubyforge_project:
115
+ rubygems_version: 1.8.24
116
+ signing_key:
117
+ specification_version: 3
118
+ summary: ! '"Easy way to retrieve main SEO parameters: Google PageRank, Yandex tIC,
119
+ backlinks, SE site positions and etc."'
120
+ test_files: []