baiduserp 2.0.5 → 2.0.8

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA1:
-  metadata.gz: 1138e5971b4ff973f8abd7096fe76997887f7f9d
-  data.tar.gz: 9582079945b7ad67a128107429927ea1831bbfb6
+  metadata.gz: 7e641beffc3596d64307f569ee35549c9fd8df39
+  data.tar.gz: 9c3ead060412c5a70fa07a01fedf8b903f0fce8f
 SHA512:
-  metadata.gz: 5e8e18d34e4820b2e1b7e2d55c23d15b562c5e8b14a7084c15657fe1dd718de50b35897c0e5642d551acb3e342a7b824e91eae462bcd8448c1a8944e30d3d174
-  data.tar.gz: dea624fc9c1231affe3d0a9e7344dc6ae325ab0c9b6f1c77ecd4caa9d8eff2909b1d911ab3f16574a453617683d759e7c0da92746370965b9bfae4814ec02b08
+  metadata.gz: 9a76f5a9d0cd00076bc2c30367626239b77a820ec6a268180efc9b3e1d063b5ecb658efb8ec140df0959b3a794d576b2af92f12c4707f9bdf7d8510f45e13037
+  data.tar.gz: 08a5df964b5b9c88c1a540e6049708743d4f8be88804829d0ebad80540ff35db6ce8c7f8607616563d2a998d1ded4324c5b9d7912645468b7870e35593ad24f9

data/README.md

@@ -1,8 +1,26 @@
 # Baiduserp
 
-此gem的目的是专门用来解析百度的搜索结果页.最主要的功能是能得到十个自然结果的排名和URL,以及左侧/右侧的广告的排名/URL.
+此gem的目的是专门用来解析百度的搜索结果页.并以最大限度获取SERP结果页面所能拿到的信息为目的.
+(注意目前这并不是一个批量处理关键词排名的程序, 但可以作为一个批量排名查询软件中解析百度SERP页面的模块)
 
-目前这个只是一个基本能用的版本,可能会有各种各样的问题,欢迎提BUG.
+## 特点
+### 解析SERP结果尽量全面
+众所周知百度的SERP页面现在越来越复杂,左侧的各种新样式层出不穷.右侧也增加了很多内容.
+这个GEM的功能就是把SERP页面解析成ruby中的数据结构.
+
+做SERP页面分析的时候有可能会想要分析页面上各种信息, 如SEO排名, SEM排名, 竞争对手排名, 
+标题/描述文字, 还有相关关键词, 右侧相关推荐信息, 是否有百度开放平台等...
+
+此gem会把上述各种各样的信息都解析出来, 供后续分析使用. 并且如果使用量越来越大, 或者百度又出新产品的话, 
+也可以增加新模块的解析.
+
+
+### 提供命令行接口(既可以测试用,也可以以JSON格式输出)
+除了提供ruby调用外, 使用其他编程语言的也可以用命令行的接口, 使用JSON格式输出结果数据.
+详细使用说明见下文.
+
+### 已知问题 
+目前这个只是一个基本能用的版本,可能会有各种各样的问题,欢迎提BUG. [已知问题列表](https://github.com/semseo/baiduserp/issues).
 
 ## Installation
 
@@ -12,7 +30,7 @@ Linux或Mac. Linux最好使用新版本的Ubuntu或Fedora系列.
 
  2 安装ruby环境
 
-只支持ruby1.9及以上. 最好的安装ruby的方法是通过[RVM](https://rvm.io/),RVM的使用方法可以搜索一下,有很多教程.
+只支持ruby1.9及以上. 最好的安装ruby的方法是通过[RVM](https://rvm.io/),RVM的使用方法可以参考这个页面[http://ruby-china.org/wiki/install_ruby_guide](http://ruby-china.org/wiki/install_ruby_guide), 虽然同时安装了一些不需要的rails相关的软件, 但是介绍很详细.
 
 在最新的Ubuntu或Fedora系列的Linux中,也可以通过apt-get或yum安装ruby1.9.
 
@@ -64,120 +82,140 @@ Usage:
     -f, --file File                  Parse Local File
 ```
 
-最终结果采用了哈希表和数组相互嵌套的数据结构,其中,每条结果的paid值中,0代表自然结果,1代表左侧广告,2代表右侧广告.结果示例如下:
+最终结果采用了哈希表和数组相互嵌套的数据结构.结果示例如下:
 
 ```
 $ baiduserp -s 香港
-{:serp_results=>
-  [{:rank=>4001,
-    :url=>"Agoda.Com.Cn/Hong_Kong",
-    :title=>nil,
-    :content=>"Agoda.Com.Cn/Hong_Kong",
-    :paid=>1},
-   {:rank=>4002,
-    :url=>"WWW.XKCITS.COM///TEL0755-86178888",
-    :title=>nil,
-    :content=>"WWW.XKCITS.COM///TEL0755-86178888",
-    :paid=>1},
-   {:rank=>1,
-    :url=>"baike.baidu.com/view/2607.htm",
-    :title=>"香港_百度百科",
-    :content=>
-     "香港是一座繁华的国际化大都市。地处珠江以东，与广东省深圳市相接。1842年至1997年，香港曾经是英国的殖民地，1997年7月1日，依据中英政府共同签...   简介   - 历史概况   - 大事年表   - 地名来源   - 地理环境    - 更多>>  baike.baidu.com/view/2607.htm  2013-2-5 ",
-    :paid=>0,
-    :mu=>"http://baike.baidu.com/view/2607.htm"},
+{:ads_right=>
+  [{:rank=>1,
+    :title=>"预订香港酒店上携程,全景图..",
+    :content=>"订香港酒店,享受有房保障,服务好,折扣低,返现高达201元,订香港酒店上携程超划算!",
+    :site=>"www.ctrip.com"},
    {:rank=>2,
-    :url=>"www.gov.hk/",
-    :title=>"GovHK 香港政府一站通:本港居民",
-    :content=>
-     "香港政府为当地居民提供的资讯和服务,内容包括通讯及科技、文化、康乐及运动、教育及培训、就业、环境、政府、法律及治安、保健及医疗服务、房屋及社会服务、入境事务...",
-    :paid=>0},
+    :title=>"香港-去哪儿网度假频道,聪明..",
+    :content=>"香港-去哪儿网度假频道,比价首选!180000条报价实时更新,先比价后出行!",
+    :site=>"dujia.qunar.com"},
    {:rank=>3,
-    :url=>"www.mafengwo.cn/travel-scenic-spot/m...",
-    :title=>"2013香港旅游攻略,香港自助游攻略,蚂蜂窝香港出游攻略游记 - 蚂蜂窝",
-    :content=>
-     "2013香港旅游攻略,介绍了香港旅游景点、线路、美食、住宿、地图等香港旅游攻略信息,了解香港旅游如莎莎化妆品、迪士尼、购物天堂和美食小吃等自助游攻略信息来蚂蜂窝...",
-    :paid=>0},
+    :title=>"香港香港怎么玩最划算?",
+    :content=>"深圳旅行社香港,香港旅游您的超值之选!天天出团港澳游专线,缤纷全程绝不",
+    :site=>"www.sztygl128.com"},
    {:rank=>4,
-    :url=>"image.baidu.com",
-    :title=>"香港_百度图片 - 举报图片",
-    :content=>"相关推荐：香港电影金像奖提名香港地图香港迪士尼乐园香港海洋公园",
-    :paid=>0,
-    :mu=>
-     "http://image.baidu.com/i?tn=baiduimage&ct=201326592&lm=-1&cl=2&fr=ala1&word=%CF%E3%B8%DB"},
+    :title=>"香港旅游攻略-150000条点评",
+    :content=>"还没来过香港?507个景点都玩遍?到到网告诉你网友怎么玩(150000张游记照片)!",
+    :site=>"www.daodao.com"},
    {:rank=>5,
-    :url=>"lvyou.baidu.com/",
-    :title=>"    香港旅游攻略_百度旅游",
-    :content=>
-     " 香港是亚洲繁华的大都市，地区及国际金融中心之一，条件优越的天然深水港，1842年至1997年是英国的殖民地，1997年7月1日回归中国。\n作为“...,旅游旺季为秋冬季节最佳...",
-    :paid=>0,
-    :mu=>"http://lvyou.baidu.com/scene/view/79c0adc41efa15d8330ab4f5"},
+    :title=>"香港旅游首选北京青年旅行社..",
+    :content=>"北京青年旅行社专业香港旅游旅行社,高品质服务,天天折扣价,",
+    :site=>"www.hqly8.com"},
    {:rank=>6,
-    :url=>"map.baidu.com/",
-    :title=>"香港特别行政区地图",
+    :title=>"香港旅游特价啦!香港旅游价..",
+    :content=>"本社邀你一起体验超值香港旅游,全程绝无强制购物,行程安排合理.",
+    :site=>"www.cctbj.net"},
+   {:rank=>7,
+    :title=>"香港旅游线路",
+    :content=>"北京旅行社提供香港咨询服务,多条精品旅游线路供您选择.",
+    :site=>"www.ctslyw.com"},
+   {:rank=>8,
+    :title=>"全新香港旅游报价,香港旅游..",
+    :content=>"北京国际旅行社,精选多条香港旅游线路,信誉保证,全程无隐性消费.",
+    :site=>"www.quly8.net"}],
+ :ads_top=>
+  [{:rank=>1,
+    :title=>"香港酒店预订 在Agoda立享1-7折",
+    :content=>"香港酒店预订,尽在Agoda,网上订购低价回馈,为您节省75%.",
+    :site=>"www.agoda.com"},
+   {:rank=>2,
+    :title=>"香港酒店预订 在Agoda立享1-7折",
+    :content=>"香港酒店预订,尽在Agoda,网上订购低价回馈,为您节省75%.",
+    :site=>"www.agoda.com"}],
+ :pinpaizhuanqu=>false,
+ :ranks=>
+  [{:rank=>1,
+    :url=>
+     "http://baike.baidu.com/link?url=Ujomxkw-4Whq7C7TI6do9nxHr3G0sO6ywJ3SZfr-lX4qQiht-2rnuGomrclwc4bJ",
+    :title=>"香港_百度百科",
+    :content=>nil,
+    :mu=>"http://baike.baidu.com/view/2607.htm",
+    :baiduopen=>false},
+   {:rank=>2,
+    :url=>"http://lvyou.baidu.com/xianggang/",
+    :title=>"2013香港旅游攻略_香港景点线路游记_百度旅游",
+    :content=>nil,
+    :mu=>"http://lvyou.baidu.com/xianggang/",
+    :baiduopen=>false},
+   {:rank=>3,
+    :url=>
+     "http://image.baidu.com/i?tn=baiduimage&ct=201326592&lm=-1&cl=2&fr=ala1&word=%CF%E3%B8%DB",
+    :title=>"香港_百度图片 - 举报图片",
     :content=>nil,
-    :paid=>0,
     :mu=>
-     "http://map.baidu.com/?newmap=1&s=s%26wd%3D%25E9%25A6%2599%25E6%25B8%25AF%25E7%2589%25B9%25E5%2588%25AB%25E8%25A1%258C%25E6%2594%25BF%25E5%258C%25BA%26c%3D2912&fr=alac0&from=alamap"},
+     "http://image.baidu.com/i?tn=baiduimage&ct=201326592&lm=-1&cl=2&fr=ala1&word=%CF%E3%B8%DB",
+    :baiduopen=>false},
+   {:rank=>4,
+    :url=>"http://www.gov.hk/sc/residents/",
+    :title=>"GovHK 香港政府一站通:本港居民",
+    :content=>
+     "香港政府为当地居民提供的资讯和服务,内容包括通讯及科技、文化、康乐及运动、教育及培训、就业、环境、政府、法律及治安、保健及医疗服务、房屋及社会服务、入境事务...",
+    :mu=>nil,
+    :baiduopen=>false},
+   {:rank=>5,
+    :url=>"http://www.baidu.com/s?rtt=2&tn=baiduwb&rn=20&cl=2&wd=%CF%E3%B8%DB",
+    :title=>"香港的最新微博结果",
+    :content=>nil,
+    :mu=>"http://www.baidu.com/s?rtt=2&tn=baiduwb&rn=20&cl=2&wd=%CF%E3%B8%DB",
+    :baiduopen=>false},
+   {:rank=>6,
+    :url=>"http://tieba.baidu.com/f?kw=%CF%E3%B8%DB&fr=ala0",
+    :title=>"香港吧 百度贴吧",
+    :content=>
+     "月活跃用户：38万人  累计发贴：202万       图片(1856)    |   视频(61)    |   精品贴(335)       香港和上海的夜景那个美？？？？？？？？？？     点击：439 回复：259  最近怎么那么多自以为漂亮的S！B女求认证啊。     点击：303 回复：69  什么时候，去香港不必签证，和去北京上海一样容...     点击：839 回复：187  查看更多香港吧内容>> tieba.baidu.com/香港?fr=ala0 2013-10-18",
+    :mu=>"http://tieba.baidu.com/f?kw=%CF%E3%B8%DB&fr=ala0",
+    :baiduopen=>false},
    {:rank=>7,
-    :url=>nil,
-    :title=>"    中国香港天气预报_春节期间未来一周天气_中国天气网 - 最近访问：",
+    :url=>"http://www.weather.com.cn/html/weather/101320101.shtml",
+    :title=>"香港天气预报_一周天气预报_中国天气网 - 最近访问：",
     :content=>nil,
-    :paid=>0,
-    :baiduopen=>1,
-    :mu=>"http://www.weather.com.cn/weather/101320101.shtml"},
+    :mu=>"http://www.weather.com.cn/html/weather/101320101.shtml",
+    :baiduopen=>true},
    {:rank=>8,
-    :url=>nil,
-    :title=>" 香港吧 百度贴吧 ",
-    :content=>nil,
-    :paid=>0,
-    :mu=>"http://tieba.baidu.com/f?kw=%CF%E3%B8%DB&fr=ala0"},
+    :url=>"http://www.mafengwo.cn/travel-scenic-spot/mafengwo/10189.html",
+    :title=>"2013香港旅游攻略,香港自助游攻略,蚂蜂窝香港出游攻略游记 - 蚂蜂窝",
+    :content=>
+     "在香港寻吃完全就是一场舌尖的盛宴,从街边小吃到世界顶级的米其林餐厅任您选择,茶餐厅、早茶、烧腊和及甜品极具港式风味,世界各地的美食料理也一个不落单。 香港...",
+    :mu=>nil,
+    :baiduopen=>false},
    {:rank=>9,
-    :url=>nil,
-    :title=>"香港的最新相关信息",
+    :url=>"http://hongkong.cncn.com/",
+    :title=>"香港旅游攻略_香港香港旅游景点_香港旅游网",
     :content=>
-     "联手香港知名餐企 陶然居赴港开店 新华网重庆频道 2小时前今后,在香港也能吃到正宗芋儿鸡和田螺了。严琦昨日告诉商报记者,陶然居已与香港金百加集团达成合作协议——今年,金百加到重庆发展,陶然居则在香港开...",
-    :paid=>0,
-    :mu=>"http://www.baidu.com/s?tn=baidurt&rtt=1&bsst=1&wd=%CF%E3%B8%DB"},
+     "香港欣欣旅游网,提供香港香港旅游景点推荐、10月香港旅游攻略、香港旅行社、香港旅游线路、香港酒店预订、香港旅游地图等出行指南及旅游服务●欣欣旅游网 CNCN.com ...",
+    :mu=>nil,
+    :baiduopen=>false},
    {:rank=>10,
-    :url=>"hongkong.cncn.com/",
-    :title=>"香港旅游攻略2011_香港旅游网",
-    :content=>
-     "香港欣欣旅游网,提供香港香港旅游景点推荐、2月香港旅游攻略、香港旅行社、香港旅游线路、香港酒店预订、香港旅游地图等出行指南及旅游服务●欣欣旅游网 CNCN.com 更新...",
-    :paid=>0},
-   {:paid=>2,
-    :rank=>1,
-    :url=>"www.gay688.com",
-    :title=>"香港旅游",
-    :content=>"在深圳找香港旅游,我亲身体验,觉着最好的还是国际香港旅游,"},
-   {:paid=>2,
-    :rank=>2,
-    :url=>"www.galy678.com",
-    :title=>"国旅香港 港澳游四天三晚贵..",
-    :content=>"香港深圳国旅,2013特别港澳游观光路线,港澳游天天出团.贴心的"},
-   {:paid=>2,
-    :rank=>3,
-    :url=>"Hotel.Qunar.Com",
-    :title=>"香港-香港订酒店,就上去哪儿..",
-    :content=>"订香港酒店?驴友真实点评,酒店高清图,实时优惠价格,出行乐无忧!"},
-   {:paid=>2,
-    :rank=>4,
-    :url=>"DaoDao.com",
-    :title=>"去香港,看真实用户点评,来Da..",
-    :content=>"来DaoDao.com看香港相关的380家酒店,更有120000条旅客真实点评."}],
- :result_num=>100000000,
- :baidubrand=>0,
+    :url=>"http://www.baidu.com/s?tn=baidurt&rtt=1&bsst=1&wd=%CF%E3%B8%DB",
+    :title=>"香港的最新相关信息",
+    :content=>nil,
+    :mu=>"http://www.baidu.com/s?tn=baidurt&rtt=1&bsst=1&wd=%CF%E3%B8%DB",
+    :baiduopen=>false}],
  :related_keywords=>
-  ["香港天气",
-   "香港电影",
+  ["香港电影",
+   "香港旅游",
+   "香港天气",
    "香港大学",
+   "香港购物",
    "香港地图",
-   "苹果香港官网",
+   "香港电视剧",
    "香港地铁",
    "香港中文大学",
-   "香港电视剧",
-   "香港旅游",
-   "苹果香港"]}
+   "香港苹果官网"],
+ :result_num=>100000000,
+ :right_hotel=>nil,
+ :right_personinfo=>nil,
+ :right_relaperson=>
+  [{:title=>"香港特别行政区行政区划", :names=>["油尖旺区", "九龙城区", "湾仔", "元朗区", "西贡区"]},
+   {:title=>"全球性国际金融中心", :names=>["新加坡", "纽约", "东京", "伦敦"]},
+   {:title=>"其他人还搜", :names=>["台北", "上海", "直布罗陀", "海南", "深圳"]}],
+ :right_weather=>nil}
 ```
 
 ## Contributing

data/lib/baiduserp.rb

@@ -2,8 +2,8 @@ require "baiduserp/version"
 require 'baiduserp/parser'
 
 module Baiduserp
-  def self.search(keyword)
-    Parser.new.search keyword
+  def self.search(keyword,page=1)
+    Parser.new.search(keyword,page)
   end
 
   def self.parse(html)

data/lib/baiduserp/client.rb

@@ -2,8 +2,29 @@ require 'httparty'
 
 module Baiduserp
   class Client
+    AllUserAgents = YAML.load(open(File.expand_path('../user_agents.yml',__FILE__)))
+
+    def self.rand_ua
+      AllUserAgents[rand(AllUserAgents.size)]
+    end
+
     include HTTParty
     base_uri 'www.baidu.com'
     follow_redirects false
+    headers "User-Agent" => self.rand_ua
+
+    def self.get_serp(url, retries = 6)
+      if retries > 0
+        response = self.get(url)
+        if response.code == 301
+          sleep(rand(60)+60)
+          response = self.get_serp(url,retries - 1)
+        end
+        return response.body
+      else
+        return nil
+      end
+    end
+
   end
 end

data/lib/baiduserp/parser.rb

@@ -26,15 +26,18 @@ module Baiduserp
       @serp
     end
 
-    def search(keyword)
-      parse_file("http://www.baidu.com/s?wd=#{keyword}")
+    def search(keyword,page=1)
+      keyword = keyword.gsub(" ","+")
+      page = page.to_i > 1 ? "&pn=#{page.to_i-1}0" : ""
+      serp_url = URI.escape("http://www.baidu.com/s?wd=#{keyword}#{page}&ie=utf-8")
+      parse_file(serp_url)
     end
 
     def parse_file(file_path)
       if File.exists? file_path
         html = open(file_path).read
       else
-        html = Client.get(URI.escape(file_path)).body
+        html = Client.get_serp(file_path)
       end
       parse html
     end

data/lib/baiduserp/user_agents.yml

@@ -0,0 +1,402 @@
+---
+- 'Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/527  (KHTML, like Gecko,
+  Safari/419.3) Arora/0.6 (Change: )'
+- Mozilla/5.0 (Windows; U; ; en-NZ) AppleWebKit/527  (KHTML, like Gecko, Safari/419.3)
+  Arora/0.8.0
+- Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 5.1; Avant Browser; Avant Browser;
+  .NET CLR 1.0.3705; .NET CLR 1.1.4322; Media Center PC 4.0; .NET CLR 2.0.50727; .NET
+  CLR 3.0.04506.30)
+- Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.8 (KHTML, like Gecko) Beamrise/17.2.0.9
+  Chrome/17.0.939.0 Safari/535.8
+- Mozilla/5.0 (Windows NT 6.1) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/18.6.872.0
+  Safari/535.2 UNTRUSTED/1.0 3gpp-gba UNTRUSTED/1.0
+- Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.3 (KHTML, like Gecko) Chrome/19.0.1061.1
+  Safari/536.3
+- Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1092.0
+  Safari/536.6
+- Mozilla/5.0 (Windows NT 6.2) AppleWebKit/536.6 (KHTML, like Gecko) Chrome/20.0.1090.0
+  Safari/536.6
+- Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML, like Gecko) Chrome/22.0.1207.1
+  Safari/537.1
+- Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/28.0.1469.0
+  Safari/537.36
+- Mozilla/5.0 (Windows NT 6.2; WOW64) AppleWebKit/537.36 (KHTML like Gecko) Chrome/28.0.1469.0
+  Safari/537.36
+- Mozilla/5.0 (Windows NT 6.1; rv:12.0) Gecko/20120403211507 Firefox/12.0
+- Mozilla/5.0 (Windows NT 6.0; rv:14.0) Gecko/20100101 Firefox/14.0.1
+- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:15.0) Gecko/20120427 Firefox/15.0a1
+- Mozilla/5.0 (Windows NT 6.2; Win64; x64; rv:16.0) Gecko/16.0 Firefox/16.0
+- Mozilla/5.0 (Windows NT 6.2; rv:19.0) Gecko/20121129 Firefox/19.0
+- Mozilla/5.0 (Windows NT 6.2; rv:20.0) Gecko/20121202 Firefox/20.0
+- Mozilla/5.0 (Windows NT 6.1; rv:21.0) Gecko/20130401 Firefox/21.0
+- Mozilla/5.0 (compatible; Konqueror/4.5; Windows) KHTML/4.5.4 (like Gecko)
+- Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/4.0; SLCC2; .NET CLR
+  2.0.50727; .NET CLR 3.5.30729; .NET CLR 3.0.30729; Media Center PC 6.0; Maxthon
+  2.0)
+- Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/533.1 (KHTML, like Gecko)
+  Maxthon/3.0.8.2 Safari/533.1
+- Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/537.1 (KHTML like Gecko) Maxthon/4.0.0.2000
+  Chrome/22.0.1229.79 Safari/537.1
+- Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.1)
+- Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0)
+- Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 5.1; Trident/4.0; .NET CLR 2.0.50727;
+  .NET CLR 3.0.04506.648; .NET CLR 3.5.21022; .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729)
+- Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.1; Trident/4.0)
+- Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Trident/4.0)
+- Mozilla/4.0 (compatible; MSIE 8.0; Windows NT 6.0; Trident/4.0)
+- Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.0; Trident/5.0)
+- Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0)
+- Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.2; Trident/5.0)
+- Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.2; WOW64; Trident/5.0)
+- Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; WOW64; Trident/5.0; SLCC2; Media
+  Center PC 6.0; InfoPath.3; MS-RTC LM 8; Zune 4.7)
+- Mozilla/5.0 (compatible; MSIE 10.0; Windows NT 6.1; WOW64; Trident/6.0)
+- Mozilla/4.0 (compatible; MSIE 7.0; Windows NT 6.1; Trident/6.0)
+- Mozilla/5.0 (compatible; MSIE 10.6; Windows NT 6.1; Trident/5.0; InfoPath.2; SLCC1;
+  .NET CLR 3.0.4506.2152; .NET CLR 3.5.30729; .NET CLR 2.0.50727) 3gpp-gba UNTRUSTED/1.0
+- Opera/9.25 (Windows NT 6.0; U; en)
+- Opera/9.80 (Windows NT 5.2; U; en) Presto/2.2.15 Version/10.10
+- Opera/9.80 (Windows NT 5.1; U; ru) Presto/2.7.39 Version/11.00
+- Opera/9.80 (Windows NT 6.1; U; en) Presto/2.7.62 Version/11.01
+- Opera/9.80 (Windows NT 5.1; U; zh-tw) Presto/2.8.131 Version/11.10
+- Opera/9.80 (Windows NT 6.1; U; es-ES) Presto/2.9.181 Version/12.00
+- Opera/9.80 (Windows NT 6.0) Presto/2.12.388 Version/12.14
+- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/531.21.8 (KHTML, like
+  Gecko) Version/4.0.4 Safari/531.21.10
+- Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/533.17.8 (KHTML, like
+  Gecko) Version/5.0.1 Safari/533.17.8
+- Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/533.19.4 (KHTML, like
+  Gecko) Version/5.0.2 Safari/533.18.5
+- Mozilla/5.0 (Windows; U; Windows NT 6.2; es-US ) AppleWebKit/540.0 (KHTML like Gecko)
+  Version/6.0 Safari/8900.00
+- Mozilla/5.0 (Windows; U; Windows NT 6.1; en-GB; rv:1.9.1.17) Gecko/20110123 (like
+  Firefox/3.x) SeaMonkey/2.0.12
+- Mozilla/5.0 (Windows NT 5.2; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 SeaMonkey/2.7.1
+- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:12.0) Gecko/20120422 Firefox/12.0 SeaMonkey/2.9
+- Avant Browser/1.2.789rel1 (http://www.avantbrowser.com)
+- Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/532.5 (KHTML, like Gecko)
+  Chrome/4.0.249.0 Safari/532.5
+- Mozilla/5.0 (Windows; U; Windows NT 5.2; en-US) AppleWebKit/532.9 (KHTML, like Gecko)
+  Chrome/5.0.310.0 Safari/532.9
+- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US) AppleWebKit/534.7 (KHTML, like Gecko)
+  Chrome/7.0.514.0 Safari/534.7
+- Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US) AppleWebKit/534.14 (KHTML, like
+  Gecko) Chrome/9.0.601.0 Safari/534.14
+- Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.14 (KHTML, like
+  Gecko) Chrome/10.0.601.0 Safari/534.14
+- Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US) AppleWebKit/534.20 (KHTML, like
+  Gecko) Chrome/11.0.672.2 Safari/534.20
+- Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/534.27 (KHTML, like Gecko) Chrome/12.0.712.0
+  Safari/534.27
+- Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.24
+  Safari/535.1
+- Mozilla/5.0 (Windows NT 6.0) AppleWebKit/535.2 (KHTML, like Gecko) Chrome/15.0.874.120
+  Safari/535.2
+- Mozilla/5.0 (Windows NT 6.1; WOW64) AppleWebKit/535.7 (KHTML, like Gecko) Chrome/16.0.912.36
+  Safari/535.7
+- Mozilla/5.0 (Windows; U; Windows NT 6.0 x64; en-US; rv:1.9pre) Gecko/2008072421
+  Minefield/3.0.2pre
+- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.9.0.10) Gecko/2009042316 Firefox/3.0.10
+- Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.9.0.11) Gecko/2009060215 Firefox/3.0.11
+  (.NET CLR 3.5.30729)
+- Mozilla/5.0 (Windows; U; Windows NT 6.0; en-US; rv:1.9.1.6) Gecko/20091201 Firefox/3.5.6
+  GTB5
+- Mozilla/5.0 (Windows; U; Windows NT 5.1; tr; rv:1.9.2.8) Gecko/20100722 Firefox/3.6.8
+  ( .NET CLR 3.5.30729; .NET4.0E)
+- Mozilla/5.0 (Windows NT 6.1; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
+- Mozilla/5.0 (Windows NT 6.1; Win64; x64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
+- Mozilla/5.0 (Windows NT 5.1; rv:5.0) Gecko/20100101 Firefox/5.0
+- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:6.0a2) Gecko/20110622 Firefox/6.0a2
+- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:7.0.1) Gecko/20100101 Firefox/7.0.1
+- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:10.0.1) Gecko/20100101 Firefox/10.0.1
+- Mozilla/5.0 (Windows NT 6.1; WOW64; rv:2.0b4pre) Gecko/20100815 Minefield/4.0b4pre
+- Mozilla/4.0 (compatible; MSIE 5.5; Windows NT 5.0 )
+- Mozilla/4.0 (compatible; MSIE 5.5; Windows 98; Win 9x 4.90)
+- Mozilla/5.0 (Windows; U; Windows XP) Gecko MultiZilla/1.6.1.0a
+- Mozilla/2.02E (Win95; U)
+- Mozilla/3.01Gold (Win95; I)
+- Mozilla/4.8 [en] (Windows NT 5.1; U)
+- Mozilla/5.0 (Windows; U; Win98; en-US; rv:1.4) Gecko Netscape/7.1 (ax)
+- Opera/7.50 (Windows XP; U)
+- Opera/7.50 (Windows ME; U) [en]
+- Opera/7.51 (Windows NT 5.1; U) [en]
+- Mozilla/4.0 (compatible; MSIE 6.0; Windows NT 5.0; en) Opera 8.0
+- Mozilla/5.0 (Windows; U; WinNT4.0; en-US; rv:1.2b) Gecko/20021001 Phoenix/0.2
+- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; rv:1.8.1.23) Gecko/20090825 SeaMonkey/1.1.18
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
+  Camino/2.2.1
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0b6pre) Gecko/20100907 Firefox/4.0b6pre
+  Camino/2.2a1pre
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_0) AppleWebKit/536.3 (KHTML, like Gecko)
+  Chrome/19.0.1063.0 Safari/536.3
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_2) AppleWebKit/537.4 (KHTML like Gecko)
+  Chrome/22.0.1229.79 Safari/537.4
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_8_4) AppleWebKit/537.31 (KHTML like Gecko)
+  Chrome/26.0.1410.63 Safari/537.31
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 1083) AppleWebKit/537.36 (KHTML like Gecko)
+  Chrome/28.0.1469.0 Safari/537.36
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2; rv:10.0.1) Gecko/20100101 Firefox/10.0.1
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:16.0) Gecko/20120813 Firefox/16.0
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.7; rv:20.0) Gecko/20100101 Firefox/20.0
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.8; rv:21.0) Gecko/20100101 Firefox/21.0
+- iTunes/4.2 (Macintosh; U; PPC Mac OS X 10.2)
+- iTunes/9.0.3 (Macintosh; U; Intel Mac OS X 10_6_2; en-ca)
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X; en-US) AppleWebKit/528.16 (KHTML, like
+  Gecko, Safari/528.16) OmniWeb/v622.8.0.112941
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_6; en-US) AppleWebKit/528.16 (KHTML,
+  like Gecko, Safari/528.16) OmniWeb/v622.8.0
+- Opera/9.20 (Macintosh; Intel Mac OS X; U; en)
+- Opera/9.64 (Macintosh; PPC Mac OS X; U; en) Presto/2.1.1
+- Opera/9.80 (Macintosh; Intel Mac OS X; U; en) Presto/2.6.30 Version/10.61
+- Opera/9.80 (Macintosh; Intel Mac OS X 10.4.11; U; en) Presto/2.7.62 Version/11.00
+- Opera/9.80 (Macintosh; Intel Mac OS X 10.6.8; U; fr) Presto/2.9.168 Version/11.52
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_2; en-us) AppleWebKit/531.21.8 (KHTML,
+  like Gecko) Version/4.0.4 Safari/531.21.10
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; de-de) AppleWebKit/534.15  (KHTML,
+  like Gecko) Version/5.0.3 Safari/533.19.4
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_6; en-us) AppleWebKit/533.20.25 (KHTML,
+  like Gecko) Version/5.0.4 Safari/533.20.27
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_7; en-us) AppleWebKit/534.20.8 (KHTML,
+  like Gecko) Version/5.1 Safari/534.20.8
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_3) AppleWebKit/534.55.3 (KHTML, like
+  Gecko) Version/5.1.3 Safari/534.53.10
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_5) AppleWebKit/536.26.17 (KHTML like
+  Gecko) Version/6.0.2 Safari/536.26.17
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.5; rv:10.0.1) Gecko/20100101 Firefox/10.0.1
+  SeaMonkey/2.7.1
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_5_8; en-US) AppleWebKit/532.8 (KHTML,
+  like Gecko) Chrome/4.0.302.2 Safari/532.8
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_4; en-US) AppleWebKit/534.3 (KHTML,
+  like Gecko) Chrome/6.0.464.0 Safari/534.3
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_5; en-US) AppleWebKit/534.13 (KHTML,
+  like Gecko) Chrome/9.0.597.15 Safari/534.13
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_2) AppleWebKit/535.1 (KHTML, like Gecko)
+  Chrome/14.0.835.186 Safari/535.1
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.2 (KHTML, like Gecko)
+  Chrome/15.0.874.54 Safari/535.2
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10_6_8) AppleWebKit/535.7 (KHTML, like Gecko)
+  Chrome/16.0.912.36 Safari/535.7
+- 'Mozilla/5.0 (Macintosh; U; Mac OS X Mach-O; en-US; rv:2.0a) Gecko/20040614 Firefox/3.0.0 '
+- Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.0.3) Gecko/2008092414
+  Firefox/3.0.3
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:1.9.1) Gecko/20090624
+  Firefox/3.5
+- Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.6; en-US; rv:1.9.2.14) Gecko/20110218
+  AlexaToolbar/alxf-2.0 Firefox/3.6.14
+- Mozilla/5.0 (Macintosh; U; PPC Mac OS X 10.5; en-US; rv:1.9.2.15) Gecko/20110303
+  Firefox/3.6.15
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:5.0) Gecko/20100101 Firefox/5.0
+- Mozilla/5.0 (Macintosh; Intel Mac OS X 10.6; rv:9.0) Gecko/20100101 Firefox/9.0
+- Mozilla/4.0 (compatible; MSIE 5.15; Mac_PowerPC)
+- Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en-US) AppleWebKit/125.4 (KHTML, like Gecko,
+  Safari) OmniWeb/v563.15
+- Opera/9.0 (Macintosh; PPC Mac OS X; U; en)
+- Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.2 (KHTML, like Gecko)
+  Safari/85.8
+- Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/125.2 (KHTML, like Gecko)
+  Safari/125.8
+- Mozilla/5.0 (Macintosh; U; PPC Mac OS X; fr-fr) AppleWebKit/312.5 (KHTML, like Gecko)
+  Safari/312.3
+- Mozilla/5.0 (Macintosh; U; PPC Mac OS X; en) AppleWebKit/418.8 (KHTML, like Gecko)
+  Safari/419.3
+- Mozilla/5.0 (X11; U; Linux; en-US) AppleWebKit/527  (KHTML, like Gecko, Safari/419.3)
+  Arora/0.10.1
+- Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/536.5 (KHTML, like Gecko) Chrome/19.0.1084.9
+  Safari/536.5
+- Mozilla/5.0 (X11; CrOS i686 2268.111.0) AppleWebKit/536.11 (KHTML, like Gecko) Chrome/20.0.1132.57
+  Safari/536.11
+- Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.4 (KHTML like Gecko) Chrome/22.0.1229.56
+  Safari/537.4
+- Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/28.0.1478.0
+  Safari/537.36
+- Mozilla/5.0 (X11; Linux i686) AppleWebKit/537.22 (KHTML like Gecko) Ubuntu Chromium/25.0.1364.160
+  Chrome/25.0.1364.160 Safari/537.22
+- Mozilla/4.0 (compatible; Dillo 3.0)
+- Mozilla/5.0 (X11; U; Linux i686; en-us) AppleWebKit/528.5  (KHTML, like Gecko, Safari/528.5
+  ) lt-GtkLauncher
+- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.16) Gecko/20120421 Gecko Firefox/11.0
+- 'Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20100101 Firefox/12.0 '
+- Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:14.0) Gecko/20100101 Firefox/14.0.1
+- Mozilla/5.0 (X11; Linux i686; rv:16.0) Gecko/20100101 Firefox/16.0
+- Mozilla/5.0 (X11; U; Linux i686; rv:19.0) Gecko/20100101 Slackware/13 Firefox/19.0
+- Mozilla/5.0 (X11; Ubuntu; Linux x86_64; rv:20.0) Gecko/20100101 Firefox/20.0
+- Mozilla/5.0 (X11; Linux i686; rv:20.0) Gecko/20100101 Firefox/20.0
+- Mozilla/5.0 (X11; Linux i686; rv:21.0) Gecko/20100101 Firefox/21.0
+- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.8) Gecko Galeon/2.0.6 (Ubuntu 2.0.6-2)
+- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1.16) Gecko/20080716 (Gentoo) Galeon/2.0.6
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.13) Gecko/20100916 Iceape/2.0.8
+- Mozilla/5.0 (X11; Linux i686; rv:14.0) Gecko/20100101 Firefox/14.0.1 Iceweasel/14.0.1
+- Mozilla/5.0 (X11; Linux x86_64; rv:15.0) Gecko/20120724 Debian Iceweasel/15.02
+- Mozilla/5.0 (X11; Linux x86_64; rv:19.0) Gecko/20100101 Firefox/19.0 Iceweasel/19.0.2
+- Mozilla/5.0 (compatible; Konqueror/4.2; Linux) KHTML/4.2.4 (like Gecko) Slackware/13.0
+- Mozilla/5.0 (compatible; Konqueror/4.3; Linux) KHTML/4.3.1 (like Gecko) Fedora/4.3.1-3.fc11
+- Mozilla/5.0 (compatible; Konqueror/4.4; Linux) KHTML/4.4.1 (like Gecko) Fedora/4.4.1-1.fc12
+- Mozilla/5.0 (compatible; Konqueror/4.4; Linux 2.6.32-22-generic; X11; en_US) KHTML/4.4.3
+  (like Gecko) Kubuntu
+- Mozilla/5.0 (compatible; Konqueror/4.4; Linux 2.6.32-22-generic; X11; en_US) KHTML/4.4.3
+  (like Gecko) Kubuntu
+- Mozilla/5.0 (X11; Linux 3.8-6.dmz.1-liquorix-686) KHTML/4.8.4 (like Gecko) Konqueror/4.8
+- 'Midori/0.1.10 (X11; Linux i686; U; en-us) WebKit/(531).(2) '
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.0.3) Gecko/2008092814 (Debian-3.0.1-1)
+- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9a3pre) Gecko/20070330
+- Opera/9.64 (X11; Linux i686; U; Linux Mint; nb) Presto/2.1.1
+- Opera/9.80 (X11; Linux i686; U; en) Presto/2.2.15 Version/10.10
+- Opera/9.80 (X11; Linux x86_64; U; pl) Presto/2.7.62 Version/11.00
+- Mozilla/5.0 (X11; Linux i686) AppleWebKit/534.34 (KHTML, like Gecko) QupZilla/1.2.0
+  Safari/534.34
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.17) Gecko/20110123 SeaMonkey/2.0.12
+- Mozilla/5.0 (X11; Linux i686; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 SeaMonkey/2.7.1
+- Mozilla/5.0 (X11; Linux i686; rv:12.0) Gecko/20120502 Firefox/12.0 SeaMonkey/2.9.1
+- Mozilla/5.0 (X11; U; Linux x86_64; us; rv:1.9.1.19) Gecko/20110430 shadowfox/7.0
+  (like Firefox/7.0
+- Mozilla/5.0 (X11; U; Linux i686; it; rv:1.9.2.3) Gecko/20100406 Firefox/3.6.3 (Swiftfox)
+- Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.4 (KHTML, like Gecko) Chrome/4.0.237.0
+  Safari/532.4 Debian
+- Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/532.8 (KHTML, like Gecko) Chrome/4.0.277.0
+  Safari/532.8
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/532.9 (KHTML, like Gecko)
+  Chrome/5.0.309.0 Safari/532.9
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.7 (KHTML, like Gecko)
+  Chrome/7.0.514.0 Safari/534.7
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/540.0 (KHTML, like Gecko)
+  Ubuntu/10.10 Chrome/9.1.0.0 Safari/540.0
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US) AppleWebKit/534.15 (KHTML, like Gecko)
+  Chrome/10.0.613.0 Safari/534.15
+- Mozilla/5.0 (X11; U; Linux i686; en-US) AppleWebKit/534.15 (KHTML, like Gecko) Ubuntu/10.10
+  Chromium/10.0.613.0 Chrome/10.0.613.0 Safari/534.15
+- Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/534.24 (KHTML, like Gecko) Ubuntu/10.10
+  Chromium/12.0.703.0 Chrome/12.0.703.0 Safari/534.24
+- Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/535.1 (KHTML, like Gecko) Chrome/13.0.782.20
+  Safari/535.1
+- Mozilla/5.0 Slackware/13.37 (X11; U; Linux x86_64; en-US) AppleWebKit/535.1 (KHTML,
+  like Gecko) Chrome/13.0.782.41
+- Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.1 (KHTML, like Gecko) Ubuntu/11.04
+  Chromium/14.0.825.0 Chrome/14.0.825.0 Safari/535.1
+- Mozilla/5.0 (X11; Linux i686) AppleWebKit/535.2 (KHTML, like Gecko) Ubuntu/11.10
+  Chromium/15.0.874.120 Chrome/15.0.874.120 Safari/535.2
+- Mozilla/5.0 (X11; U; Linux; i686; en-US; rv:1.6) Gecko Epiphany/1.2.5
+- Mozilla/5.0 (X11; U; Linux i586; en-US; rv:1.7.3) Gecko/20040924 Epiphany/1.4.4
+  (Ubuntu)
+- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.6) Gecko/20040614 Firefox/0.8
+- Mozilla/5.0 (X11; U; Linux x86_64; sv-SE; rv:1.8.1.12) Gecko/20080207 Ubuntu/7.10
+  (gutsy) Firefox/2.0.0.12
+- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.0.11) Gecko/2009060309 Ubuntu/9.10
+  (karmic) Firefox/3.0.11
+- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.9.1.2) Gecko/20090803 Ubuntu/9.04 (jaunty)
+  Shiretoko/3.5.2
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.5) Gecko/20091107 Firefox/3.5.5
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.1.3) Gecko/20091020 Linux Mint/8
+  (Helena) Firefox/3.5.3
+- Mozilla/5.0 (X11; U; Linux x86_64; en-US; rv:1.9.2.9) Gecko/20100915 Gentoo Firefox/3.6.9
+- Mozilla/5.0 (X11; U; Linux i686; pl-PL; rv:1.9.0.2) Gecko/20121223 Ubuntu/9.25 (jaunty)
+  Firefox/3.8
+- Mozilla/5.0 (X11; Linux i686; rv:2.0b6pre) Gecko/20100907 Firefox/4.0b6pre
+- Mozilla/5.0 (X11; Linux i686 on x86_64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
+- Mozilla/5.0 (X11; Linux i686; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
+- Mozilla/5.0 (X11; Linux x86_64; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
+- Mozilla/5.0 (X11; Linux x86_64; rv:2.2a1pre) Gecko/20100101 Firefox/4.2a1pre
+- Mozilla/5.0 (X11; Linux i686; rv:5.0) Gecko/20100101 Firefox/5.0
+- Mozilla/5.0 (X11; Linux i686; rv:6.0) Gecko/20100101 Firefox/6.0
+- Mozilla/5.0 (X11; Linux x86_64; rv:7.0a1) Gecko/20110623 Firefox/7.0a1
+- Mozilla/5.0 (X11; Linux i686; rv:8.0) Gecko/20100101 Firefox/8.0
+- Mozilla/5.0 (X11; Linux x86_64; rv:10.0.1) Gecko/20100101 Firefox/10.0.1
+- Mozilla/5.0 (X11; U; Linux; i686; en-US; rv:1.6) Gecko Galeon/1.3.14
+- Mozilla/5.0 (X11; U; Linux ppc; en-US; rv:1.8.1.13) Gecko/20080313 Iceape/1.1.9
+  (Debian-1.1.9-5)
+- Mozilla/5.0 (X11; U; Linux i686; pt-PT; rv:1.9.2.3) Gecko/20100402 Iceweasel/3.6.3
+  (like Firefox/3.6.3) GTB7.0
+- Mozilla/5.0 (X11; Linux x86_64; rv:5.0) Gecko/20100101 Firefox/5.0 Iceweasel/5.0
+- Mozilla/5.0 (X11; Linux i686; rv:6.0a2) Gecko/20110615 Firefox/6.0a2 Iceweasel/6.0a2
+- Konqueror/3.0-rc4; (Konqueror/3.0-rc4; i686 Linux;;datecode)
+- Mozilla/5.0 (compatible; Konqueror/3.3; Linux 2.6.8-gentoo-r3; X11;
+- Mozilla/5.0 (compatible; Konqueror/3.5; Linux 2.6.30-7.dmz.1-liquorix-686; X11)
+  KHTML/3.5.10 (like Gecko) (Debian package 4:3.5.10.dfsg.1-1 b1)
+- Mozilla/5.0 (compatible; Konqueror/3.5; Linux; en_US) KHTML/3.5.6 (like Gecko) (Kubuntu)
+- Mozilla/5.0 (X11; Linux x86_64; en-US; rv:2.0b2pre) Gecko/20100712 Minefield/4.0b2pre
+- Mozilla/5.0 (X11; U; Linux; i686; en-US; rv:1.6) Gecko Debian/1.6-7
+- MSIE (MSIE 6.0; X11; Linux; i686) Opera 7.23
+- Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.8.1) Gecko/20061024 Firefox/2.0 (Swiftfox)
+- Mozilla/5.0 (Unknown; U; UNIX BSD/SYSV system; C -) AppleWebKit/527  (KHTML, like
+  Gecko, Safari/419.3) Arora/0.10.2
+- Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/536.5 (KHTML like Gecko) Chrome/19.0.1084.56
+  Safari/536.5
+- Mozilla/5.0 (X11; FreeBSD amd64) AppleWebKit/537.4 (KHTML like Gecko) Chrome/22.0.1229.79
+  Safari/537.4
+- Mozilla/5.0 (X11; U; OpenBSD arm; en-us) AppleWebKit/531.2  (KHTML, like Gecko)
+  Safari/531.2  Epiphany/2.30.0
+- Mozilla/5.0 (X11; U; FreeBSD amd64; en-us) AppleWebKit/531.2  (KHTML, like Gecko)
+  Safari/531.2  Epiphany/2.30.0
+- Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.9.1b3) Gecko/20090429 Firefox/3.1b3
+- Mozilla/5.0 (X11; U; OpenBSD i386; en-US; rv:1.9.1) Gecko/20090702 Firefox/3.5
+- Mozilla/5.0 (X11; U; FreeBSD i386; de-CH; rv:1.9.2.8) Gecko/20100729 Firefox/3.6.8
+- Mozilla/5.0 (X11; FreeBSD amd64; rv:5.0) Gecko/20100101 Firefox/5.0
+- Mozilla/5.0 (compatible; Konqueror/4.1; DragonFly) KHTML/4.1.4 (like Gecko)
+- Mozilla/5.0 (compatible; Konqueror/4.1; OpenBSD) KHTML/4.1.4 (like Gecko)
+- Mozilla/5.0 (compatible; Konqueror/4.5; NetBSD 5.0.2; X11; amd64; en_US) KHTML/4.5.4
+  (like Gecko)
+- Mozilla/5.0 (compatible; Konqueror/4.5; FreeBSD) KHTML/4.5.4 (like Gecko)
+- Mozilla/5.0 (X11; U; NetBSD amd64; en-US; rv:1.9.2.15) Gecko/20110308 Namoroka/3.6.15
+- NetSurf/1.2 (NetBSD; amd64)
+- Opera/9.80 (X11; FreeBSD 8.1-RELEASE i386; Edition Next) Presto/2.12.388 Version/12.10
+- Mozilla/5.0 (X11; U; SunOS i86pc; en-US; rv:1.8.1.12) Gecko/20080303 SeaMonkey/1.1.8
+- Mozilla/5.0 (X11; U; FreeBSD i386; en-US) AppleWebKit/532.0 (KHTML, like Gecko)
+  Chrome/4.0.207.0 Safari/532.0
+- Mozilla/5.0 (X11; U; OpenBSD i386; en-US) AppleWebKit/533.3 (KHTML, like Gecko)
+  Chrome/5.0.359.0 Safari/533.3
+- Mozilla/5.0 (X11; U; FreeBSD x86_64; en-US) AppleWebKit/534.16 (KHTML, like Gecko)
+  Chrome/10.0.648.204 Safari/534.16
+- Mozilla/5.0 (X11; U; SunOS sun4m; en-US; rv:1.4b) Gecko/20030517 Mozilla Firebird/0.6
+- Mozilla/5.0 (X11; U; FreeBSD i386; en-US; rv:1.6) Gecko/20040406 Galeon/1.3.15
+- Mozilla/5.0 (compatible; Konqueror/3.5; NetBSD 4.0_RC3; X11) KHTML/3.5.7 (like Gecko)
+- Mozilla/5.0 (compatible; Konqueror/3.5; SunOS) KHTML/3.5.1 (like Gecko)
+- Mozilla/5.0 (X11; U; FreeBSD; i386; en-US; rv:1.7) Gecko
+- Mozilla/4.77 [en] (X11; I; IRIX;64 6.5 IP30)
+- Mozilla/4.8 [en] (X11; U; SunOS; 5.7 sun4u)
+- Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US; BOLT/2.800) AppleWebKit/534.6 (KHTML,
+  like Gecko) Version/5.0 Safari/534.6.3
+- Mozilla/4.0 (compatible; MSIE 6.0; Windows CE; IEMobile 6.12; Microsoft ZuneHD 4.3)
+- 'Mozilla/4.0 (compatible; MSIE 6.0; Windows CE; IEMobile 7.11) '
+- Mozilla/4.0 (compatible; MSIE 7.0; Windows Phone OS 7.0; Trident/3.1; IEMobile/7.0)
+  Asus;Galaxy6
+- Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0)
+- Mozilla/5.0 (compatible; MSIE 9.0; Windows Phone OS 7.5; Trident/5.0; IEMobile/9.0)
+- 'Mozilla/5.0 (compatible; MSIE 10.0; Windows Phone 8.0; Trident/6.0; IEMobile/10.0;
+  ARM; Touch) '
+- Mozilla/1.22 (compatible; MSIE 5.01; PalmOS 3.0) EudoraWeb 2.1
+- Mozilla/5.0 (WindowsCE 6.0; rv:2.0.1) Gecko/20100101 Firefox/4.0.1
+- Mozilla/5.0 (X11; U; Linux armv61; en-US; rv:1.9.1b2pre) Gecko/20081015 Fennec/1.0a1
+- Mozilla/5.0 (Maemo; Linux armv7l; rv:2.0.1) Gecko/20100101 Firefox/4.0.1 Fennec/2.0.1
+- Mozilla/5.0 (Maemo; Linux armv7l; rv:10.0.1) Gecko/20100101 Firefox/10.0.1 Fennec/10.0.1
+- Mozilla/5.0 (Windows; U; Windows CE 5.1; rv:1.8.1a3) Gecko/20060610 Minimo/0.016
+- Mozilla/5.0 (X11; U; Linux armv6l; rv 1.8.1.5pre) Gecko/20070619 Minimo/0.020
+- Mozilla/5.0 (X11; U; Linux arm7tdmi; rv:1.8.1.11) Gecko/20071130 Minimo/0.025
+- Mozilla/4.0 (PDA; PalmOS/sony/model prmr/Revision:1.1.54 (en)) NetFront/3.0
+- Opera/9.51 Beta (Microsoft Windows; PPC; Opera Mobi/1718; U; en)
+- Opera/9.60 (J2ME/MIDP; Opera Mini/4.1.11320/608; U; en) Presto/2.2.0
+- Opera/9.60 (J2ME/MIDP; Opera Mini/4.2.14320/554; U; cs) Presto/2.2.0
+- Opera/9.80 (S60; SymbOS; Opera Mobi/499; U; ru) Presto/2.4.18 Version/10.00
+- Opera/10.61 (J2ME/MIDP; Opera Mini/5.1.21219/19.999; en-US; rv:1.9.3a5) WebKit/534.5
+  Presto/2.6.30
+- POLARIS/6.01 (BREW 3.1.5; U; en-us; LG; LX265; POLARIS/6.01/WAP) MMP/2.0 profile/MIDP-2.1
+  Configuration/CLDC-1.1
+- Mozilla/5.0 (Linux; U; Android 2.0; en-us; Droid Build/ESD20) AppleWebKit/530.17
+  (KHTML, like Gecko) Version/4.0 Mobile Safari/530.17
+- Mozilla/5.0 (iPad; U; CPU OS 4_2_1 like Mac OS X; ja-jp) AppleWebKit/533.17.9 (KHTML,
+  like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5
+- Mozilla/5.0 (iPhone; U; CPU iPhone OS 4_2_1 like Mac OS X; da-dk) AppleWebKit/533.17.9
+  (KHTML, like Gecko) Version/5.0.2 Mobile/8C148 Safari/6533.18.5
+- Mozilla/5.0 (iPad; CPU OS 6_0 like Mac OS X) AppleWebKit/536.26 (KHTML, like Gecko)
+  Version/6.0 Mobile/10A5355d Safari/8536.25
+- Mozilla/5.0 (compatible; MSIE 9.0; Windows NT 6.1; Trident/5.0; XBLWP7; ZuneWP7)
+  UCBrowser/2.9.0.263
+- Mozilla/5.0 (Linux; U; Android 2.3.3; en-us ; LS670 Build/GRI40) AppleWebKit/533.1
+  (KHTML, like Gecko) Version/4.0 Mobile Safari/533.1/UCBrowser/8.6.1.262/145/355
+- Mozilla/3.0 (compatible; NetPositive/2.1.1; BeOS)
+- Mozilla/5.0 (BeOS; U; BeOS BePC; en-US; rv:1.9a1) Gecko/20060702 SeaMonkey/1.5a
+- Mozilla/5.0 (OS/2; Warp 4.5; rv:10.0.12) Gecko/20100101 Firefox/10.0.12
+- Mozilla/5.0 (OS/2; Warp 4.5; rv:10.0.12) Gecko/20130108 Firefox/10.0.12 SeaMonkey/2.7.2
+- 'Mozilla/5.0 (OS/2; U; OS/2; en-US) AppleWebKit/533.3 (KHTML, like Gecko) Arora/0.11.0
+  Safari/533.3 '
+- 'Mozilla/5.0 (OS/2; U; OS/2; en-US) AppleWebKit/533.3 (KHTML, like Gecko) QupZilla/1.3.1
+  Safari/533.3 '

data/lib/baiduserp/version.rb

@@ -1,3 +1,3 @@
 module Baiduserp
-  VERSION = "2.0.5"
+  VERSION = "2.0.8"
 end

data/lib/parsers/ranks.rb

@@ -1,10 +1,10 @@
 class Baiduserp::Parser
   def _parse_ranks(file)
     result = []
-    file[:doc].search("//table").each do |table|
+    file[:doc].search("div[@id='content_left']").first.children.each do |table|
       next if table.nil?
       id = table['id'].to_i
-      next unless id > 0
+      next unless id > 0 && id < 3000
       r = {:rank => id}
 
       url = table.search('h3/a').first

data/lib/parsers/right_relaperson.rb

@@ -5,7 +5,8 @@ class Baiduserp::Parser
 
     result = []
     relapersons.each do |rr|
-      title = rr.search('span.opr-relaperson-subtitle-tip').first.content
+      title = rr.search('div.cr-title/span').first
+      title = title.content unless title.nil?
       r = []
       rr.search('p.opr-relaperson-name/a').each do |p|
         r << p['title']

metadata

@@ -1,14 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: baiduserp
 version: !ruby/object:Gem::Version
-  version: 2.0.5
+  version: 2.0.8
 platform: ruby
 authors:
 - MingQian Zhang
 autorequire: 
 bindir: bin
 cert_chain: []
-date: 2013-09-23 00:00:00.000000000 Z
+date: 2013-11-01 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: nokogiri
@@ -63,7 +63,8 @@ files:
 - lib/parsers/right_weather.rb
 - bin/baiduserp
 - README.md
-homepage: https://github.com/mqzhang/baiduserp
+- lib/baiduserp/user_agents.yml
+homepage: https://github.com/semseo/baiduserp
 licenses: []
 metadata: {}
 post_install_message: 
@@ -82,7 +83,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
       version: '0'
 requirements: []
 rubyforge_project: 
-rubygems_version: 2.0.3
+rubygems_version: 2.0.0
 signing_key: 
 specification_version: 4
 summary: Baidu SERP