seo_params 0.0.1

data/.gitignore

@@ -0,0 +1,18 @@
+*~
+*.gem
+*.rbc
+.bundle
+.config
+.yardoc
+Gemfile.lock
+InstalledFiles
+_yardoc
+coverage
+doc/
+lib/bundler/man
+pkg
+rdoc
+spec/reports
+test/tmp
+test/version_tmp
+tmp

data/Gemfile

@@ -0,0 +1,4 @@
+source 'https://rubygems.org'
+
+# Specify your gem's dependencies in seo_params.gemspec
+gemspec

data/LICENSE

@@ -0,0 +1,22 @@
+Copyright (c) 2012 none
+
+MIT License
+
+Permission is hereby granted, free of charge, to any person obtaining
+a copy of this software and associated documentation files (the
+"Software"), to deal in the Software without restriction, including
+without limitation the rights to use, copy, modify, merge, publish,
+distribute, sublicense, and/or sell copies of the Software, and to
+permit persons to whom the Software is furnished to do so, subject to
+the following conditions:
+
+The above copyright notice and this permission notice shall be
+included in all copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
+EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
+MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
+NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
+LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
+OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
+WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.

data/README.md

@@ -0,0 +1,158 @@
+# SeoParams
+
+This is a small application for seo purposes: you can get Google PageRank for your site, number of pages in Google index, Yandex tIC, number of pages in Yandex index.
+
+In addition, you can check the position of your site in the search for keywords in search engines.
+
+## Installation
+
+Add this line to your application's Gemfile:
+
+``` ruby
+    gem 'seo_params'
+```
+
+And then execute:
+
+``` bash
+    bundle
+```
+
+Or install it yourself as:
+
+``` bash
+    gem install PageRankr
+```
+
+## Usage
+
+``` ruby
+    require 'seo_params'
+```
+### Fetch main SEO params
+
+``` ruby
+    SeoParams.all("gihub.com")
+      # => {"pr"=>7, "gp"=>33400000, "tic"=>3700, "yap"=>627119}
+```
+Short description:
+`pr`  - Google PageRank,
+`gp`  - pages in Google index,
+`tic` - Yandex tIC,
+`yap` - pages in Yandex index.
+
+### Fetch specific SEO parameter
+
+To fetch only Google PageRank:
+
+``` ruby
+    SeoParams.pr("gihub.com")
+      # => 7
+```
+
+To fetch only pages in Google index:
+
+``` ruby
+    SeoParams.gp("gihub.com")
+      # => 44000000
+```
+This request gives sometimes very strange results
+
+To fetch only Yandex tIC:
+
+``` ruby
+    SeoParams.tic("gihub.com")
+      # => 3700
+```
+
+To fetch only pages in Google index:
+
+``` ruby
+    SeoParams.yap("gihub.com")
+      # => 627119
+```
+
+### Check site position in the search for keywords in Google
+
+The most visited sites - are sites on the first page of a search result the user. That is why it is so important control whether the position of your website for keywords.
+
+To check the position of your site in Google for specific keywords:
+
+``` ruby
+    SeoParams.gposition("www.none.com.ua", "rails")
+      # => {"rails"=>19}
+```
+or you can specify an array of keywords:
+
+``` ruby
+    keywords = ["rails", "ruby", "rvm", "spine.js"]
+    SeoParams.gposition("www.none.com.ua", keywords)
+      # => {"spine.js"=>1, "rails"=>19, "ruby"=>72, "rvm"=>9}
+```
+
+Because the application was written for my purposes, the default search is performed in Russian for the country of Ukraine with 100 of results (-: But you can always change these settings. You have to specify the following:
+
+`:hl` - specifies the interface language (host language) of your user interface. To improve the performance and the quality of your search results, you are strongly encouraged to set this parameter explicitly. List of languages you can find [here][1].
+
+`:cr` - restricts search results to documents originating in a particular country. List of countries you can find [here][2].
+
+`:num` - identifies the number of search results to return. I usually use 10 or 100.
+
+The previous example can be rewritten as follows:
+
+``` ruby
+    keywords = ["rails", "ruby", "rvm", "spine.js"]
+    SeoParams.gposition("www.none.com.ua", keywords, :hl => "ru", :cr => "countryUA", :num => 100)
+      # => {"spine.js"=>1, "rvm"=>9, "rails"=>19, "ruby"=>72}
+```
+
+
+[1]: https://developers.google.com/custom-search/docs/xml_results?hl=en#interfaceLanguages
+[2]: https://developers.google.com/custom-search/docs/xml_results?hl=en#countryCollections
+
+
+### Check site position in the search for keywords in Yandex
+
+Checking the position of keywords in Yandex need only for RuNET, so further description will be in Russian.
+
+С проверкой позиций в Яндексе всё несколько сложнее.
+
+Во-первых, Вам необходимо быть зарегестрированным пользователем Яндекса. Сделать это можно по вот этой [ссылке][1].
+Во-вторых, добавить IP-адрес, с которого будут посылаться запросы к сервису Яндекса, на этой [страничке][2]
+В-третьих, скопировать/сохранить `user` и `key` из поля "Ваш адрес для совершения запроса" на страничке в предыдущем пункте. Важно знать, что просто зарегистрированные пользователи могут делать не более 10 запросов в день. Если Вы подтвердите Ваш номер телефона, то сможете делать до 1000 запросов в день. Думаю, для многих этого будет достаточно, а кому и этого мало, тогда следует ознакомиться с информацией на самом сайте, как увеличить данное число.
+
+В принципе, уже можно пробовать:
+
+``` ruby
+    keywords = ["rails", "ruby", "rvm", "spine.js"]
+    SeoParams.yaposition("www.none.com.ua", user, key, keywords)
+      # => {"ruby"=>0, "spine.js"=>2, "rvm"=>3, "rails"=>22}
+```
+
+Опять же, поиск осуществляется по Украине и при 100 результатах в выдаче. Эти параметры можно изменить:
+
+`:lr`  - идентификатор страны или региона поиска. Список идентификаторов часто используемых стран и регионов приведен в [приложении][3].
+
+`:num` - количество результатов в выдаче поиска. По умолчанию - 100. Ещё одно рекомендуемое значение 10.
+
+Приведённый выше пример можно переписать следующим образом:
+
+``` ruby
+    keywords = ["rails", "ruby", "rvm", "spine.js"]
+    SeoParams.yaposition("www.none.com.ua", user, key, keywords, :lr => 187, :num => 100)
+      # => {"ruby"=>0, "spine.js"=>2, "rvm"=>3, "rails"=>22}
+```
+Значение "0" в результатах поиска свидетельствует об отсутствии сайта в результате поиска по данному клучевому слову.
+
+[1]: http://passport.yandex.ru/passport?mode=register
+[2]: http://xml.yandex.ru/settings.xml
+[3]: http://api.yandex.ru/xml/doc/dg/reference/regions.xml
+
+
+## Contributing
+
+1. Fork it
+2. Create your feature branch (`git checkout -b my-new-feature`)
+3. Commit your changes (`git commit -am 'Added some feature'`)
+4. Push to the branch (`git push origin my-new-feature`)
+5. Create new Pull Request

data/Rakefile

@@ -0,0 +1,2 @@
+#!/usr/bin/env rake
+require "bundler/gem_tasks"

data/lib/seo_params/google.rb

@@ -0,0 +1,86 @@
+# -*- encoding: utf-8 -*-
+
+require "page_rankr"
+
+require "em-synchrony"
+require "em-synchrony/em-http"
+require "em-synchrony/fiber_iterator"
+require 'nokogiri'
+require 'open-uri'
+
+module SeoParams
+
+  class Google
+
+    def initialize(url)
+      @url = url
+      @host = get_host
+    end
+
+
+    def pagerank
+      query = PageRankr.ranks(@url, :google)
+      query[:google]
+    end
+
+    def google_pages
+      doc = Nokogiri::HTML(open("https://www.google.com/search?hl=en&tab=ww&safe=active&tbo=d&sclient=psy-ab&q=site:#{@url}&oq=site:#{@url}"))
+      pages = doc.css('div[@id="resultStats"]').to_s[/[\d,]+/] if doc.css('div[@id="subform_ctrl"]')
+      pages ? pages.tr(',', '').to_i : 0
+    end
+
+    def google_position(hl, cr, keywords, num)
+
+      h = Hash.new
+
+      EventMachine.synchrony do
+
+        EM::Synchrony::FiberIterator.new(keywords, keywords.size).each do |keyword|
+
+          new_keyword = check_keyword(keyword)
+
+          uri="http://www.google.com/search?hl=#{hl}&q=#{new_keyword}&btnG=Поиск+в+Google&&cr=#{cr}&meta=&num=#{num}"
+          resp = EventMachine::HttpRequest.new(uri).get
+          h[keyword] = parse_results resp.response
+        end
+        EventMachine.stop
+      end
+
+      h
+
+    end
+
+    private
+
+      def check_keyword(keyword)
+        (keyword.include? ' ') ? (keyword.tr(' ', '+')) : keyword
+      end
+
+      def parse_results response
+        pos = 0
+        i = 1
+        doc = Nokogiri::HTML(response)
+
+        doc.xpath('//ol/li/h3/a').each do |link|
+          url = link['href'][/(https?:\/\/[\S]+)(&sa)/,1]
+
+
+          if url.to_s[/#{Regexp.escape(@host)}/]
+            pos = i
+            break
+          else
+            i = i + 1
+          end
+        end
+        pos
+      end
+
+      def get_host
+        @url.match(/^(http:\/\/)/) ? full_url = @url : full_url = 'http://' + @url
+        uri = URI(full_url)
+        @host = uri.host
+      end
+
+  end
+
+end

data/lib/seo_params/version.rb

@@ -0,0 +1,3 @@
+module SeoParams
+  VERSION = "0.0.1"
+end

data/lib/seo_params/yandex.rb

@@ -0,0 +1,114 @@
+# -*- encoding: utf-8 -*-
+require 'nokogiri'
+require 'open-uri'
+
+module SeoParams
+
+  class Yandex
+
+    def initialize(url)
+      url.match(/^(https?:\/\/)/) ? @url = url : @url = 'http://' + url
+      @host = URI(@url).host
+    end
+
+
+    def tic
+      query = Nokogiri::XML(open("http://bar-navig.yandex.ru/u?ver=2&show=32&url=#{@url}"))
+      tic = query.xpath('//@value')
+      tic.to_s.to_i
+    end
+
+    def yandex_pages
+      pages = ask_yandex(@url)
+      (pages.is_a? String) ? (url = pages; pages = ask_yandex(url); ) : pages
+      pages
+    end
+
+    def yandex_position(user, key, lr, keywords, num)
+
+      uri = URI.parse "http://xmlsearch.yandex.ru/xmlsearch?user=#{user}&key=#{key}&lr=#{lr}"
+
+      h = Hash.new
+
+      EventMachine.synchrony do
+        EM::Synchrony::FiberIterator.new(keywords, keywords.size).each do |keyword|
+          request = EventMachine::HttpRequest.new(uri)
+          response = request.post(:body => xml_request(keyword, num))
+
+          result = parse_results(response)
+
+          (result.is_a? Hash) ? (h.merge! result) : (h[keyword] = result)
+
+        end
+
+        EventMachine.stop
+      end
+
+      h
+
+    end
+
+
+    private
+      def ask_yandex(url)
+
+        doc = Nokogiri::HTML(open("http://webmaster.yandex.ua/check.xml?hostname=#{url}"))
+
+        if doc.css('div.error-message').length > 0
+          if doc.css('div.error-message').to_s.scan('Сайт является зеркалом')
+            url_regexp = /\?hostname\=([a-zA-Z\.:\/0-9]{1,})/
+            new_url = doc.css('div.error-message').to_s.scan(url_regexp)[0][0]
+            index = new_url
+          end
+
+        else
+          doc.css('div.header div').each do |link|
+              pages = /\d{1,}/
+              @pages_in_index = link.content.scan(pages)
+              index = @pages_in_index[0].to_i
+          end
+        end
+
+        index
+      end
+
+      def xml_request keyword, num
+        "<?xml version=\"1.0\" encoding=\"UTF-8\"?>
+          <request>
+            <query>#{keyword}</query>
+            <groupings>
+              <groupby attr=\"d\" mode=\"deep\" groups-on-page=\"#{num}\"  docs-in-group=\"1\" />
+            </groupings>
+          </request>"
+      end
+
+      def parse_results response
+        h_err = Hash.new
+        pos = 0
+        i = 1
+        doc = Nokogiri::XML(response.response)
+
+        if doc.xpath('//error')
+          doc.xpath("//error").map do |err|
+            h_err["error_code"] = err['code']
+            h_err["error_message"] = err.text()
+          end
+        end
+
+
+        doc.xpath('//url').each do |link|
+          if link.to_s[/#{Regexp.escape(@host)}/]
+            pos = i
+            break
+          else
+            i = i + 1
+          end
+        end
+
+
+        (h_err.length != 0) ? h_err : pos
+
+      end
+  end
+
+end

data/lib/seo_params.rb

@@ -0,0 +1,56 @@
+# -*- encoding: utf-8 -*-
+
+require File.expand_path('../seo_params/google', __FILE__)
+require File.expand_path('../seo_params/yandex', __FILE__)
+
+module SeoParams
+
+  class << self
+
+    def all(url)
+      h = Hash.new
+      h["pr"] = pr(url)
+      h["gp"] = gp(url)
+      h["tic"] = tic(url)
+      h["yap"] = yap(url)
+      h
+    end
+
+    def pr(url)
+      Google.new(url).pagerank
+    end
+
+    def tic(url)
+      Yandex.new(url).tic
+    end
+
+    def yap(url)
+      Yandex.new(url).yandex_pages
+    end
+
+    def gp(url)
+      Google.new(url).google_pages
+    end
+
+    def yaposition(url, user, key, keywords, options = {})
+      options[:lr]  ||= 187
+      options[:num] ||= 100
+
+      (keywords.is_a? Array) ? keywords : keywords = Array.new.push(keywords)
+
+      Yandex.new(url).yandex_position(user, key, options[:lr], keywords, options[:num])
+    end
+
+    def gposition(url, keywords, options = {})
+      options[:hl]  ||= "ru"
+      options[:cr]  ||= "countryUA"
+      options[:num] ||= 100
+
+      (keywords.is_a? Array) ? keywords : keywords = Array.new.push(keywords)
+
+      Google.new(url).google_position(options[:hl], options[:cr], keywords, options[:num])
+    end
+
+  end
+
+end

data/seo_params.gemspec

@@ -0,0 +1,22 @@
+# -*- encoding: utf-8 -*-
+require File.expand_path('../lib/seo_params/version', __FILE__)
+
+Gem::Specification.new do |gem|
+  gem.authors       = ["Alexander Podkidyshev aka none"]
+  gem.email         = ["alexander.podkidyshev@gmail.com"]
+  gem.description   = %q{"Easy way to retrieve main SEO parameters."}
+  gem.summary       = %q{"Easy way to retrieve main SEO parameters: Google PageRank, Yandex tIC, backlinks, SE site positions and etc."}
+  gem.homepage      = "https://github.com/n0ne/seo_params"
+
+  gem.add_runtime_dependency "nokogiri",                  ">= 1.5.0"
+  gem.add_runtime_dependency "page_rankr",                ">= 3.2.1"
+  gem.add_runtime_dependency "em-http-request",           ">= 1.0.3"
+  gem.add_runtime_dependency "em-synchrony",              ">= 1.0.2"
+
+  gem.files         = `git ls-files`.split($\)
+  gem.executables   = gem.files.grep(%r{^bin/}).map{ |f| File.basename(f) }
+  gem.test_files    = gem.files.grep(%r{^(test|spec|features)/})
+  gem.name          = "seo_params"
+  gem.require_paths = ["lib"]
+  gem.version       = SeoParams::VERSION
+end

metadata

@@ -0,0 +1,120 @@
+--- !ruby/object:Gem::Specification
+name: seo_params
+version: !ruby/object:Gem::Version
+  version: 0.0.1
+  prerelease: 
+platform: ruby
+authors:
+- Alexander Podkidyshev aka none
+autorequire: 
+bindir: bin
+cert_chain: []
+date: 2012-11-16 00:00:00.000000000 Z
+dependencies:
+- !ruby/object:Gem::Dependency
+  name: nokogiri
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.5.0
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.5.0
+- !ruby/object:Gem::Dependency
+  name: page_rankr
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 3.2.1
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 3.2.1
+- !ruby/object:Gem::Dependency
+  name: em-http-request
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.0.3
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.0.3
+- !ruby/object:Gem::Dependency
+  name: em-synchrony
+  requirement: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.0.2
+  type: :runtime
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    none: false
+    requirements:
+    - - ! '>='
+      - !ruby/object:Gem::Version
+        version: 1.0.2
+description: ! '"Easy way to retrieve main SEO parameters."'
+email:
+- alexander.podkidyshev@gmail.com
+executables: []
+extensions: []
+extra_rdoc_files: []
+files:
+- .gitignore
+- Gemfile
+- LICENSE
+- README.md
+- Rakefile
+- lib/seo_params.rb
+- lib/seo_params/google.rb
+- lib/seo_params/version.rb
+- lib/seo_params/yandex.rb
+- seo_params.gemspec
+homepage: https://github.com/n0ne/seo_params
+licenses: []
+post_install_message: 
+rdoc_options: []
+require_paths:
+- lib
+required_ruby_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+required_rubygems_version: !ruby/object:Gem::Requirement
+  none: false
+  requirements:
+  - - ! '>='
+    - !ruby/object:Gem::Version
+      version: '0'
+requirements: []
+rubyforge_project: 
+rubygems_version: 1.8.24
+signing_key: 
+specification_version: 3
+summary: ! '"Easy way to retrieve main SEO parameters: Google PageRank, Yandex tIC,
+  backlinks, SE site positions and etc."'
+test_files: []