browser 3.0.1 → 4.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 53decb60a500c2a7d9aa08ecb38764898be5deb813849131ef883e9944c6951e
-  data.tar.gz: 5aad1b2edf63615f946c20b14dc292a7f172a6379baceb6eaeb83205f5bb04fc
+  metadata.gz: d83e06fd8c03def4d4c2af17fbbd94580594eeb689aa6f4d0fda3d7c94be4c8c
+  data.tar.gz: 4e8b416860701c61d4a1cf40d2a51316c28431e60d24716d8092746faff88c70
 SHA512:
-  metadata.gz: 0fe86685cdf72e8706872ce2e64534460bd0989123b896aa6d3430cf6c3ccb9676fa782553ff3ff3aa7f5adc3d60e6b92214721551972798a4a1f761ba57a82c
-  data.tar.gz: d5749c3672b07fa937958586deb29f6447087f7a7fdd169323e35921de42260292ae7098f765119cb2ce9fdd47c83acb54a86ea2b57f652ffea46ca6743d9bc8
+  metadata.gz: 86f29ee60621611d8abb82837eac44594b50739e7b39991bd8ce73c7b701f7630e314d3ac22fdf80567d3c2a44985ad4a632204d14805d7015e18b856d357dfc
+  data.tar.gz: 9bb2826023b3e7d6b7a7ca7c5f0d14b0871bcb39764d3c3c05ecba7784cc4458949533dfc16cc3de7b5f3844faf254d6faf127613ca27547d52ea07040e7ccfa

data/.github/ISSUE_TEMPLATE.md

@@ -27,4 +27,4 @@
 
 ## Report
 
-[Visit <http://user-agent.herokuapp.com> and paste the URL here]
+[Visit <https://user-agent.herokuapp.com> and paste the URL here]

data/.gitignore

@@ -4,3 +4,4 @@ doc
 log
 *.lock
 bin/
+.bundle

data/.prettierignore

@@ -0,0 +1 @@
+*.yml

data/.rubocop.yml

@@ -27,3 +27,12 @@ Style/Alias:
 
 Lint/RedundantCopDisableDirective:
   Enabled: false
+
+Metrics/AbcSize:
+  Enabled: false
+
+Metrics/CyclomaticComplexity:
+  Enabled: false
+
+Metrics/PerceivedComplexity:
+  Enabled: false

data/.travis.yml

@@ -1,10 +1,10 @@
 ---
 language: ruby
-sudo: false
 cache: bundler
 rvm:
-  - 2.6.0
-  - 2.5.0
+  - 2.7
+  - 2.6
+  - 2.5
 gemfile:
   - Gemfile
   - gemfiles/rails5.gemfile

data/CHANGELOG.md

@@ -2,7 +2,35 @@
 
 ## Unreleased
 
-- Nothing yet.
+- Fix Chrome Lighthouse detection.
+- Add Skype to bot list.
+
+## 4.1.0
+
+- Add Samsung browser.
+- Add Google Image Proxy to the bot list.
+- Add The Knowledge AI bot to the bot list.
+- Add Go HttpClient to the bot list.
+- Fix Microsoft Edge detection on Android and iOS.
+- Fix MicroMessenger detection on Android
+
+## 4.0.0
+
+- Add Chrome Lighthouse to bot list.
+- Add SeobilityBot to the bot list.
+- Detect Mac-based platforms differently, depending on the version; "Mac OS X"
+  will be returned for versions prior to 10.12, and "macOS" for newer versions.
+- Remove `Browser.modern_rules` and `Browser::Base#modern?`.
+- Add DuckDuckGo browser.
+
+# 3.0.3
+
+- Deprecate `Browser.modern_rules` and `Browser::Base#modern?`. Theses methods
+  will be removed on the next major released, or by June 1st 2020.
+
+## 3.0.2
+
+- Remove .bundle directory from package.
 
 ## 3.0.1
 
@@ -13,23 +41,32 @@
 - Add ArchiveTeam's ArchiveBot to the bot list.
 - Fix QQ Browser detection.
 - Update modern rules.
-- You can now define new bot matchers by adding a callable object to `Browser::Bot.matchers`.
+- You can now define new bot matchers by adding a callable object to
+  `Browser::Bot.matchers`.
 - Fix `browser.yandex?` and `browser.sputnik?`.
-- [BREAKING CHANGE] Removed methods to enable the bot's empty user agent detection (`Browser::Bot.detect_empty_ua!` and `Browser::Bot.detect_empty_ua?`).
-- [BREAKING CHANGE] Bot detection is now more aggressive by default. It matches empty user agents, anything that matches `crawl|fetch|search|monitoring|spider|bot`, and anything listed under https://github.com/fnando/browser/blob/master/bots.yml.
+- [BREAKING CHANGE] Removed methods to enable the bot's empty user agent
+  detection (`Browser::Bot.detect_empty_ua!` and
+  `Browser::Bot.detect_empty_ua?`).
+- [BREAKING CHANGE] Bot detection is now more aggressive by default. It matches
+  empty user agents, anything that matches
+  `crawl|fetch|search|monitoring|spider|bot`, and anything listed under
+  https://github.com/fnando/browser/blob/master/bots.yml.
 - Add Jaunt to the bot list.
 
 ## 2.7.1
 
-- Handle Snapchat user agents that have a space or an empty string instead of a slash before the version.
+- Handle Snapchat user agents that have a space or an empty string instead of a
+  slash before the version.
 - Fix iOS 10+ version detection.
-- Add fallback versions for instagram and snapchat to avoid NoMethodErrors on unexpected user agents.
+- Add fallback versions for instagram and snapchat to avoid NoMethodErrors on
+  unexpected user agents.
 
 ## 2.7.0
 
 - Add more Slack bots.
 - Handle instagram user agents that have a slash instead of a space.
-- Add `Browser::Bot.why?(ua)` to help debugging why a user agent is considered bot.
+- Add `Browser::Bot.why?(ua)` to help debugging why a user agent is considered
+  bot.
 - Promote Snapchat to a browser (it was detected as a bot previously).
 - Detect Edge based on Chrome correctly.
 - Improve Yandex detection.
@@ -45,7 +82,8 @@
 
 - Add GarlikCrawler, ImplisenseBot and WikiDo bots.
 - Add Mastodon URL expander bot.
-- Add eZ Publish Link Validator, GermCrawler, Pu_iN Crawler, ZoomBot, and ZoominfoBot bots.
+- Add eZ Publish Link Validator, GermCrawler, Pu_iN Crawler, ZoomBot, and
+  ZoominfoBot bots.
 - Add Datanyze bot.
 - Add support for Instagram in-app browser.
 - Add Updown.io monitor bot.
@@ -67,13 +105,13 @@
 - Add Jooble bot.
 - Add Fyre bot.
 - Drop Rails 4 official support.
-- Fix accept-language sorting (If HTTP-header has value `en,fr`—without qualities—the first language should be `en` instead of `fr`).
+- Fix accept-language sorting (If HTTP-header has value `en,fr`—without
+  qualities—the first language should be `en` instead of `fr`).
 - Ignore malformed strings when comparing versions.
 - Fix Facebook detection on newer apps.
 - Change precedence for bot detection when common libs are used.
 - Add Yandex's search browser to the exception list.
 
-
 ## v2.5.3
 
 - Add Google Site Verification to the bot list.
@@ -114,8 +152,10 @@
 - Add Google Drive API, Proximic Spider, NewRelic pinger and SocialRank bots.
 - Add Pinboard in-app browser to the bot exception list.
 - All browser detection methods can now compare versions.
-- All platform detection methods can now compare versions (except `#linux?` and `#firefox_os?`).
-- Add `browser/aliases`, so you can have methods on the base object (e.g. `browser.mobile?`). See README for instructions.
+- All platform detection methods can now compare versions (except `#linux?` and
+  `#firefox_os?`).
+- Add `browser/aliases`, so you can have methods on the base object (e.g.
+  `browser.mobile?`). See README for instructions.
 - Remove official support for Rails 3 and Ruby 2.1.
 
 ## v2.3.0
@@ -146,7 +186,8 @@
 
 ## v2.0.3
 
-- Fix issue with version detection when no actual version is provided (i.e. the user agent doesn't have any version information).
+- Fix issue with version detection when no actual version is provided (i.e. the
+  user agent doesn't have any version information).
 
 ## v2.0.2
 
@@ -159,9 +200,12 @@
 
 ## v2.0.0
 
-- `Browser#platform` now returns instance of `Browser::Platform`, instead of a `String`. It contains information about the platform (software).
-- `Browser#device` was added. It returns information about the device (hardware).
-- `Browser#accept_language` now returns a list of `Browser::AcceptLanguage` objects.
+- `Browser#platform` now returns instance of `Browser::Platform`, instead of a
+  `String`. It contains information about the platform (software).
+- `Browser#device` was added. It returns information about the device
+  (hardware).
+- `Browser#accept_language` now returns a list of `Browser::AcceptLanguage`
+  objects.
 - `Browser#bot` now returns a `Browser::Bot` instance.
 - Safari running as web app mode is not recognized as Safari anymore.
 - ruby-2.3+ will always activate frozen strings.

data/README.md

@@ -25,6 +25,7 @@ browser = Browser.new("Some User Agent", accept_language: "en-us")
 browser.bot?
 browser.chrome?
 browser.core_media?
+browser.duck_duck_go?
 browser.edge?                # Newest MS browser
 browser.electron?            # Electron Framework
 browser.firefox?
@@ -34,7 +35,6 @@ browser.ie?(6)               # detect specific IE version
 browser.ie?([">8", "<10"])   # detect specific IE (IE9).
 browser.known?               # has the browser been successfully detected?
 browser.meta                 # an array with several attributes
-browser.modern?              # Webkit, Firefox 17+, IE 9+ and Opera 12+
 browser.name                 # readable browser name
 browser.nokia?
 browser.opera?
@@ -43,6 +43,7 @@ browser.phantom_js?
 browser.quicktime?
 browser.safari?
 browser.safari_webapp_mode?
+browser.samsung_browser?
 browser.to_s            # the meta info joined by space
 browser.uc_browser?
 browser.version         # major version number
@@ -145,23 +146,26 @@ browser.mobile? #=> false
 - For a list of device detections, check [lib/browser/device.rb](https://github.com/fnando/browser/blob/master/lib/browser/device.rb)
 - For a list of bot detections, check [bots.yml](https://github.com/fnando/browser/blob/master/bots.yml)
 
-### What defines a modern browser?
+### Detecting modern browsers
 
-The current rules that define a modern browser are pretty loose.
-
-* Chrome 65+
-* Safari 10+
-* Firefox 52+
-* IE11+
-* Microsoft Edge 39+
-* Opera 50+
-
-You can define your own rules. A rule must be a proc/lambda or any object that implements the method === and accepts the browser object. To redefine all rules, clear the existing rules before adding your own.
+To detect whether a browser can be considered as modern or not, create a method that abstracts your versioning constraints. The following example will consider any of the following browsers as a modern:
 
 ```ruby
-# Only Google Chrome 79+ is considered modern.
-Browser.modern_rules.clear
-Browser.modern_rules << -> b { b.chrome? && b.version.to_i >= 79 }
+# Expects an Browser instance,
+# like in `Browser.new(user_agent, accept_language: language)`.
+def modern_browser?(browser)
+  [
+    browser.chrome?(">= 65"),
+    browser.safari?(">= 10"),
+    browser.firefox?(">= 52"),
+    browser.ie?(">= 11") && !browser.compatibility_view?,
+    browser.edge?(">= 15"),
+    browser.opera?(">= 50"),
+    browser.facebook?
+      && browser.safari_webapp_mode?
+      && browser.webkit_full_version.to_i >= 602
+  ].any?
+end
 ```
 
 ### Rails integration
@@ -240,9 +244,6 @@ browser.msie_full_version
 
 browser.compatibility_view?
 #=> true
-
-browser.modern?
-#=> false
 ```
 
 This behavior changed in `v1.0.0`; previously there wasn't a way of getting the real browser version.
@@ -298,7 +299,7 @@ You can use the `Browser::Middleware` to redirect user agents.
 
 ```ruby
 use Browser::Middleware do
-  redirect_to "/upgrade" unless browser.modern?
+  redirect_to "/upgrade" if browser.ie?
 end
 ```
 
@@ -306,17 +307,7 @@ If you're using Rails, you can use the route helper methods. Just add something
 
 ```ruby
 Rails.configuration.middleware.use Browser::Middleware do
-  redirect_to upgrade_path unless browser.modern?
-end
-```
-
-Notice that you can have multiple conditionals.
-
-```ruby
-Rails.configuration.middleware.use Browser::Middleware do
-  next if browser.bot.search_engine?
-  redirect_to upgrade_path(browser: "oldie") if browser.ie? && !browser.modern?
-  redirect_to upgrade_path(browser: "oldfx") if browser.firefox? && !browser.modern?
+  redirect_to upgrade_path if browser.ie?
 end
 ```
 
@@ -324,25 +315,10 @@ If you need access to the `Rack::Request` object (e.g. to exclude a path), you c
 
 ```ruby
 Rails.configuration.middleware.use Browser::Middleware do
-  redirect_to upgrade_path unless browser.modern? || request.env["PATH_INFO"] == "/exclude_me"
+  redirect_to upgrade_path if browser.ie? && request.env["PATH_INFO"] != "/exclude_me"
 end
 ```
 
-### Migrating to v2
-
-#### Troubleshooting
-
-##### `TypeError: no implicit conversion of Hash into String`
-
-The class constructor now has a different signature. Change the instantiation from `Browser.new(options)` to `Browser.new(ua, options)`, where:
-
-- `ua`: must be a string representing the user agent.
-- `options`: must be a hash (for now it only accepts the `accept_language` option).
-
-##### `NoMethodError: undefined method 'user_agent'`
-
-`.ua` can now be used to retrieve the full User Agent string.
-
 ## Development
 
 ### Versioning

data/bots.yml

@@ -1,299 +1,305 @@
 ---
-200pleasebot: "200PleaseBot"
-360spider: "360Spider"
-abot: "CrawlDaddy, abot"
-addthis: "AddThis"
-adldxbot: "Microsoft Bing Ads"
-admantx: "ADmantX Platform Semantic Analyzer"
-adsbot-google: "Google Adwords"
-adstxtcrawler: "AdsTxtCrawler"
-advbot: "AdvBot"
-ahrefsbot: "Ahrefs backlinks research tool"
-alexa: "Alexa Crawler"
-anderspink: "AndersPinkBot"
-apache-httpclient: "Java http library"
-apachebench: "ApacheBench (ab)"
+200pleasebot: 200PleaseBot
+360spider: 360Spider
+abot: CrawlDaddy, abot
+addthis: AddThis
+adldxbot: Microsoft Bing Ads
+admantx: ADmantX Platform Semantic Analyzer
+adsbot-google: Google Adwords
+adstxtcrawler: AdsTxtCrawler
+advbot: AdvBot
+ahrefsbot: Ahrefs backlinks research tool
+alexa: Alexa Crawler
+anderspink: AndersPinkBot
+apache-httpclient: Java http library
+apachebench: ApacheBench (ab)
 apis-google: APIs-Google
-appengine-google: "Google App Engine"
-applebot: "Apple Bot"
-archive.org_bot: "Internet Archive (archive.org)"
-archiveteam archivebot: "ArchiveTeam ArchiveBot"
-ask jeeves: "Ask Jeeves"
-asynchttpclient: "Java http and WebSocket client library"
-awe.sm: "Awe.sm URL expander"
-baidu: "Baidu"
-barkrowler: "Barkrowler"
-bdcbot: "Big Data Corp"
-bingbot: "Microsoft Bing"
-bingpreview: "Microsoft Bing preview"
-bitlybot: "bit.ly bot"
-blekkobot: "Blekkobot"
-blexbot: "BLEXBot (webmeup)"
-bot@linkfluence.net: "Linkfluence bot"
-bubing: "BUbiNG"
-bufferbot: "BufferBot"
-buibui-checkbot: "buibui"
-butterfly: "Topsy Labs"
-buzzbot: "Buzzbot"
-buzztalk: "buzztalk"
-catchbot: "CatchBot (catchbot.com)"
-check_http: "Nagios monitor"
-cipacrawler: "CipaCrawler"
-cliqzbot: "Cliqzbot"
-cloudflare: "CloudFlare-AlwaysOnline"
-cmradar/0.1: "CMRadar/0.1"
-coldfusion: "ColdFusion http library"
-commoncrawl: "CCBot"
-comodo ssl checker: "COMODO SSL Checker"
-comodo-webinspector-crawler: "Comodo"
-copypants: "BotPants"
-crowsnest: "Crowsnest"
-curabot: "cura.yt"
-curl: "curl unix CLI http client"
-dap/nethttp: "DAP/NetHTTP"
-datafeedwatch: "DataFeedWatch"
-datagnionbot: "datagnion.com/bot.html"
-datanyze: "Datanyze"
-daumoa: "Korean portal and search engine indexing bot"
-developers.google.com/+/web/snippet/: "Google Plus"
-diffbot: "Diffbot"
-digitalpersona fingerprint software: "HP Fingerprint scanner"
-domain re-animator bot: "Domain Re-Animator Bot"
-domainsbot: "DomainsBot"
-domaintunocrawler: "DomainTuno"
-dotbot: "Dot Bot"
-duckduck: "Duck Duck Go"
-elb-healthchecker: "AWS ELB HealthChecker"
-embedly: "Embedly"
-eoaagent: "EOAAgent"
-everyonesocialbot: "EveryoneSocial"
-evrinid: "Evri bot"
-exabot: "Exalead's bot"
-exaleadcloudview: "ExaleadCloudView"
-ez publish: "eZ Publish Link Validator"
-facebookexternalhit: "Facebook Bot"
-facebot: "Facebook Bot"
-feedburner: "RSS bot"
-feedfetcher-google: "Google Feedfetcher"
-findxbot: "Findxbot"
-flipboardproxy: "FlipboardProxy"
-friendfeedbot: "FriendFeed"
-fyrebot: "Fyrebot"
-garlik: "GarlikCrawler"
-genieo: "Genieo Web filter bot"
-germcrawler: "GermCrawler"
-getprismatic.com: "getprismatic.com"
-gigabot: "Gigabot spider"
-gimme60bot: "Gimme60 (gimme60.com)"
-gimmeusabot: "Gimme60 (gimme60.com)"
-go http package: "Go http library"
-google page speed insights: "Google Page Speed Insights"
-google Web Preview: "Google Instant Previews crawler"
-google-site-verification: Google-Site-Verification
-google-structured-data-testing-tool: "Google-StructuredDataTestingTool"
-google-structureddatatestingtool: "Google-StructuredDataTestingTool"
-google-xrawler: "Google Shopping"
-googlebot: "Google Bot"
-googlestackdrivermonitoring-uptimechecks: "GoogleStackdriverMonitoring-UptimeChecks"
-grapeshotcrawler: "GrapeshotCrawler"
-gravitybot: "Gravity Bot"
-hatena::bookmark: "Hatena::Bookmark"
-heritrix: "heritrix"
-https://developers.google.com/+/web/snippet: "Google+ Snippet Fetcher"
-httrack: "HTTrack"
-hubspot: "HubSpot"
-ia_archiver: "Internet Archive (WayBackMachine)"
-icoreservice: "iCoreService"
-idmarch: "idmarch.org/bot.html"
-implisensebot: "ImplisenseBot"
-inagist: "URL resolver"
-insieve: "Insieve Bot"
-insitesbot: "Insitesbot"
-instapaper: "Instapaper"
-istellabot: "IstellaBot"
-jaunt: "Jaunt - Java Web Scraping & JSON Querying"
-jetslide: "Jetslide"
-jobseeker: "jobseeker.com.au/bot.html"
-jooble: "Jooble"
-js-kit: "URL resolver"
-kemvibot: "Kemvi"
-kimengi: "Kimengi Bot"
-knows.is: "knows.is"
-kojitsubot: "Kojitsubot"
-komodiabot: "KomodiaBot"
-kraken: "kraken"
-laconica: "Laconica"
-lijit crawler: "Lijit"
-linkdexbot: "Linkdex Bot"
-linkedinbot: "LinkedIn"
-linkscrawler: "LinksCrawler"
-linode: "Linode Longview"
-lipperhey: "Lipperhey"
-livelapbot: "Livelapbot"
-loadtimebot: "Load Time Bot"
-longurl: "URL expander service"
-ltx71: "ltx71.com"
-lumibot: "Lumibot"
-magpie-crawler: "magpie-crawler"
-mail.ru_bot: "Mail.ru Bot"
-mappydata: "Mappy"
-mastodon: "Mastodon URL expander"
-mauibot: "MauiBot"
-meanpathbot: "meanpath"
-mediapartners-google: "Google Adsense bot"
-megaindex.ru: "MegaIndex"
-memorybot: "mignify.com/bot.html"
-metauri: "MetaURI"
-mfe_expand: "Mcafee spider"
-mir web crawler: "MIR web crawler"
-mj12bot: "Majestic-12 spider"
-mojeekbot: "Mojeek UK search crawler"
-ms search 6.0 robot: "MS Search 6.0 Robot"
-msnbot-media: "Microsoft media bot"
-msnbot: "Microsoft bot"
-nerdybot: "NerdyBot"
-netcraft: "Netcraft"
-netstate: "netEstate NE Crawler"
-netvibes: "Personalized dashboard bot"
-netzcheckbot: "netzcheck"
-newrelicmonitor: "NewRelic monitor"
-newrelicpinger: "NewRelicPinger"
-newsme: "newsme"
-niki-bot: "niki-bot"
-ning: "NING - Yet Another Twitter Swarmer"
-nutch: "Apache search spider"
-openhosebot: "OpenHoseBot"
-orangebot: "OrangeBot"
-paessler: "paessler.com - PRTG Network Monitor"
-pagesinventory: "pagesinventory.com"
-panopta: "Monitoring service"
-paperlibot: "PaperLi"
-peerindex: "peerindex"
-percolatecrawler: "PercolateCrawler"
-perfectmarketkwtbot: "PerfectMarket"
-phantomjs: "PhantomJS"
-pingdom: "Pingdom monitoring"
-pinterest: "Pinterest"
-plukkie: "botje.com/plukkie.htm"
-pr-cy.ru: "PR-CY.RU"
-privacyawarebot: "PrivacyAwareBot"
-proximic: "Proximic Spider"
-psbot-page: "Picsearch"
-pu_in: "Pu_iN Crawler"
-publiclibraryarchive.org: "publiclibraryarchive.org"
-pycurl: "Python http library"
-python-httplib2: "Python-httplib2"
-python-requests: "Python http library"
-python-urllib: "Python http library"
-queryseeker: "QuerySeekerSpider"
-quick-crawler: "Quick-Crawler"
-quicklook: "QuickLook"
-re-animator: "Domain Re-Animator Bot"
-readability: "Readability"
-rebelmouse: "RebelMouse"
-redditbot: "Reddit Bot"
-relateiq: "RelateIQ"
-riddler: "Riddler Bot"
-rogerbot: "SeoMoz spider"
-rssmicro: "RSS/Atom Feed Robot (rssmicro.com)"
-scouturlmonitor: "ScoutURLMonitor"
-scrapy: "Scrapy"
+appengine-google: Google App Engine
+applebot: Apple Bot
+archive.org_bot: Internet Archive (archive.org)
+archiveteam archivebot: ArchiveTeam ArchiveBot
+ask jeeves: Ask Jeeves
+asynchttpclient: Java http and WebSocket client library
+awe.sm: Awe.sm URL expander
+baidu: Baidu
+barkrowler: Barkrowler
+bdcbot: Big Data Corp
+bingbot: Microsoft Bing
+bingpreview: Microsoft Bing preview
+bitlybot: bit.ly bot
+blekkobot: Blekkobot
+blexbot: BLEXBot (webmeup)
+bot@linkfluence.net: Linkfluence bot
+bubing: BUbiNG
+bufferbot: BufferBot
+buibui-checkbot: buibui
+butterfly: Topsy Labs
+buzzbot: Buzzbot
+buzztalk: buzztalk
+catchbot: CatchBot (catchbot.com)
+check_http: Nagios monitor
+chrome-lighthouse: Chrome-Lighthouse
+cipacrawler: CipaCrawler
+cliqzbot: Cliqzbot
+cloudflare: CloudFlare-AlwaysOnline
+cmradar/0.1: CMRadar/0.1
+coldfusion: ColdFusion http library
+commoncrawl: CCBot
+comodo ssl checker: COMODO SSL Checker
+comodo-webinspector-crawler: Comodo
+copypants: BotPants
+crowsnest: Crowsnest
+curabot: cura.yt
+curl: curl unix CLI http client
+dap/nethttp: DAP/NetHTTP
+datafeedwatch: DataFeedWatch
+datagnionbot: datagnion.com/bot.html
+datanyze: Datanyze
+daumoa: Korean portal and search engine indexing bot
+developers.google.com/+/web/snippet/: Google Plus
+diffbot: Diffbot
+digitalpersona fingerprint software: HP Fingerprint scanner
+domain re-animator bot: Domain Re-Animator Bot
+domainsbot: DomainsBot
+domaintunocrawler: DomainTuno
+dotbot: Dot Bot
+duckduck: Duck Duck Go
+elb-healthchecker: AWS ELB HealthChecker
+embedly: Embedly
+eoaagent: EOAAgent
+everyonesocialbot: EveryoneSocial
+evrinid: Evri bot
+exabot: Exalead's bot
+exaleadcloudview: ExaleadCloudView
+ez publish: eZ Publish Link Validator
+facebookexternalhit: Facebook Bot
+facebot: Facebook Bot
+feedburner: RSS bot
+feedfetcher-google: Google Feedfetcher
+findxbot: Findxbot
+flipboardproxy: FlipboardProxy
+friendfeedbot: FriendFeed
+fyrebot: Fyrebot
+garlik: GarlikCrawler
+genieo: Genieo Web filter bot
+germcrawler: GermCrawler
+getprismatic.com: getprismatic.com
+gigabot: Gigabot spider
+gimme60bot: Gimme60 (gimme60.com)
+gimmeusabot: Gimme60 (gimme60.com)
+go http package: Go http library
+go-http-client: Go http client
+google page speed insights: Google Page Speed Insights
+google web preview: Google Instant Previews crawler
+google-site-verification: Google Site Verification
+google-structured-data-testing-tool: Google Structured Data Testing Tool
+google-structureddatatestingtool: Google Structured Data Testing Tool
+google-xrawler: Google Shopping
+googlebot: Google Bot
+googleimageproxy: Google Image Proxy
+googlestackdrivermonitoring-uptimechecks: Google Stackdriver Monitoring - Uptime Checks
+grapeshotcrawler: GrapeshotCrawler
+gravitybot: Gravity Bot
+hatena::bookmark: Hatena::Bookmark
+heritrix: heritrix
+https://developers.google.com/+/web/snippet: Google+ Snippet Fetcher
+httrack: HTTrack
+hubspot: HubSpot
+ia_archiver: Internet Archive (WayBackMachine)
+icoreservice: iCoreService
+idmarch: idmarch.org/bot.html
+implisensebot: ImplisenseBot
+inagist: URL resolver
+insieve: Insieve Bot
+insitesbot: Insitesbot
+instapaper: Instapaper
+istellabot: IstellaBot
+jaunt: Jaunt - Java Web Scraping & JSON Querying
+jetslide: Jetslide
+jobseeker: jobseeker.com.au/bot.html
+jooble: Jooble
+js-kit: URL resolver
+kemvibot: Kemvi
+kimengi: Kimengi Bot
+knows.is: knows.is
+kojitsubot: Kojitsubot
+komodiabot: KomodiaBot
+kraken: kraken
+laconica: Laconica
+lijit crawler: Lijit
+linkdexbot: Linkdex Bot
+linkedinbot: LinkedIn
+linkscrawler: LinksCrawler
+linode: Linode Longview
+lipperhey: Lipperhey
+livelapbot: Livelapbot
+loadtimebot: Load Time Bot
+longurl: URL expander service
+ltx71: ltx71.com
+lumibot: Lumibot
+magpie-crawler: magpie-crawler
+mail.ru_bot: Mail.ru Bot
+mappydata: Mappy
+mastodon: Mastodon URL expander
+mauibot: MauiBot
+meanpathbot: meanpath
+mediapartners-google: Google Adsense bot
+megaindex.ru: MegaIndex
+memorybot: mignify.com/bot.html
+metauri: MetaURI
+mfe_expand: Mcafee spider
+mir web crawler: MIR web crawler
+mj12bot: Majestic-12 spider
+mojeekbot: Mojeek UK search crawler
+ms search 6.0 robot: MS Search 6.0 Robot
+msnbot-media: Microsoft media bot
+msnbot: Microsoft bot
+nerdybot: NerdyBot
+netcraft: Netcraft
+netstate: netEstate NE Crawler
+netvibes: Personalized dashboard bot
+netzcheckbot: netzcheck
+newrelicmonitor: NewRelic monitor
+newrelicpinger: NewRelicPinger
+newsme: newsme
+niki-bot: niki-bot
+ning: NING - Yet Another Twitter Swarmer
+nutch: Apache search spider
+openhosebot: OpenHoseBot
+orangebot: OrangeBot
+paessler: paessler.com - PRTG Network Monitor
+pagesinventory: pagesinventory.com
+panopta: Monitoring service
+paperlibot: PaperLi
+peerindex: peerindex
+percolatecrawler: PercolateCrawler
+perfectmarketkwtbot: PerfectMarket
+phantomjs: PhantomJS
+pingdom: Pingdom monitoring
+pinterest: Pinterest
+plukkie: botje.com/plukkie.htm
+pr-cy.ru: PR-CY.RU
+privacyawarebot: PrivacyAwareBot
+proximic: Proximic Spider
+psbot-page: Picsearch
+pu_in: Pu_iN Crawler
+publiclibraryarchive.org: publiclibraryarchive.org
+pycurl: Python http library
+python-httplib2: Python-httplib2
+python-requests: Python http library
+python-urllib: Python http library
+queryseeker: QuerySeekerSpider
+quick-crawler: Quick-Crawler
+quicklook: QuickLook
+re-animator: Domain Re-Animator Bot
+readability: Readability
+rebelmouse: RebelMouse
+redditbot: Reddit Bot
+relateiq: RelateIQ
+riddler: Riddler Bot
+rogerbot: SeoMoz spider
+rssmicro: RSS/Atom Feed Robot (rssmicro.com)
+scouturlmonitor: ScoutURLMonitor
+scrapy: Scrapy
 screaming frog seo spider: Screaming Frog SEO Spider
-searchmetricsbot: "SearchmetricsBot"
-semanticbot: "Semanticbot"
-semrushbot: "SEO analysis bot"
-seo-audit: "seo-audit-check-bot"
-seodiver: "SEOdiver"
-seokicks: "SEOKicks"
-seznambot: "SeznamBot"
-shopwiki: "ShopWiki"
-shortlinktranslate: "Link shortener"
-showyoubot: "Showyou iOS app spider"
-siege: "Joe Dog Siege"
-sistrix: "SISTRIX"
-sitecheck: "SiteCheck sitecrawl"
-siteuptime: "Site monitoring services"
-slack: "Slackbot-LinkExpanding"
-slackbot: "Slackbot 1.0(+https://api.slack.com/robots)"
-slack-imgproxy: "Slack-ImgProxy 0.19 (+https://api.slack.com/robots)"
-slack-linkexpanding: "Slackbot-LinkExpanding 1.0 (+https://api.slack.com/robots)"
-slurp: "Yahoo spider"
-smtbot: "SimilarTech"
-snapchat: "Snapchat"
-socialrank: "SocialRankIOBot"
-sogou: "Chinese search engine"
-spbot: "OpenLinkProfiler"
-spinn3r: "Spinn3r aggregator"
-sputnikbot: "SputnikBot"
-squider: "Squider"
-statuscake: "StatusCake"
-stripe: "Stripe"
-swiftbot: "Swiftype Bot"
-tangibleebot: "TangibleeBot"
-teeraid: "TeeRaidBot"
-test certificate info: "C http library?"
-tineye: "TinEye Bot"
-traackr: "Traackr Bot"
-trendictionbot: "Trendiction Search"
-trendsmap: "Trendsmap Resolver"
-turnitinbot: "TurnitinBot"
-tweetedtimes: "The Tweeted Times"
-tweetmemebot: "TweetMeMe Crawler"
-twikle: "Social web search bot"
-twitjobsearch: "TwitJobSearch"
-twitmunin: "Twitmunin"
-twitterbot: "Twitter URL expander"
-twurly: "Twurly"
-typhoeus: "Typhoeus"
-umbot: "uberMetrics"
-unwindfetch: "Gnip"
-updown: "Updown.io monitor"
-uptimerobot: "Uptime Robot"
-vagabondo: "Vagabondo"
-vb project: "Visual Basic"
-vigil: "Vigil"
-vkshare: "VKontake Sharer"
-voilabot: "VoilaBot"
-vrcrawler: "Venture Radar"
-wasalive-bot: "Wasalive Bots"
-watchsumo: "WatchSumo"
-wbsearchbot: "Ware Bay Best Buys"
-webceo: "online-webceo-bot"
-webscout: "Webscout"
-wesee: "WeSEE"
-wget: "wget unix CLI http client"
-whatsapp: "WhatsApp"
-wikido: "WikiDo"
-woorank: "WooRank"
-wordpress: "WordPress spider"
-woriobot: "woriobot"
-wormly: "WormlyBot"
-wotbox: "Wotbox"
-xenu link sleuth: "Xenu Link Sleuth"
-xing-contenttabreceiver: "Xing bot"
-xovibot: "XoviBot"
-yacybot: "YaCy"
-yahoo-ad-monitoring: "Yahoo Ad monitoring"
-yandex: "Yandex"
-yanga: "Yanga WorldSearch Bot"
-yeti: "Naver Corp"
-yourls: "YOURLS"
-zabbix: 'Zabbix'
-zelist.ro: "feed parser"
-zibb: "ZIBB spider"
-zitebot: "Zite"
-zoombot: "ZoomBot"
-zoominfobot: "ZoominfoBot"
-zyborg: "Zyborg"
+searchmetricsbot: SearchmetricsBot
+semanticbot: Semanticbot
+semrushbot: SEO analysis bot
+seo-audit: seo-audit-check-bot
+seobilitybot: SeobilityBot
+seodiver: SEOdiver
+seokicks: SEOKicks
+seznambot: SeznamBot
+shopwiki: ShopWiki
+shortlinktranslate: Link shortener
+showyoubot: Showyou iOS app spider
+siege: Joe Dog Siege
+sistrix: SISTRIX
+sitecheck: SiteCheck sitecrawl
+siteuptime: Site monitoring services
+skypeuripreview: SkypeUriPreview
+slack-imgproxy: Slack Image Proxy
+slack-linkexpanding: Slack Link Expanding
+slack: Slack Link Expanding
+slackbot: Slackbot
+slurp: Yahoo spider
+smtbot: SimilarTech
+snapchat: Snapchat
+socialrank: SocialRankIOBot
+sogou: Chinese search engine
+spbot: OpenLinkProfiler
+spinn3r: Spinn3r aggregator
+sputnikbot: SputnikBot
+squider: Squider
+statuscake: StatusCake
+stripe: Stripe
+swiftbot: Swiftype Bot
+tangibleebot: TangibleeBot
+teeraid: TeeRaidBot
+test certificate info: C http library?
+the knowledge ai: Knowledge AI Bot
+tineye: TinEye Bot
+traackr: Traackr Bot
+trendictionbot: Trendiction Search
+trendsmap: Trendsmap Resolver
+turnitinbot: TurnitinBot
+tweetedtimes: The Tweeted Times
+tweetmemebot: TweetMeMe Crawler
+twikle: Social web search bot
+twitjobsearch: TwitJobSearch
+twitmunin: Twitmunin
+twitterbot: Twitter URL expander
+twurly: Twurly
+typhoeus: Typhoeus
+umbot: uberMetrics
+unwindfetch: Gnip
+updown: Updown.io monitor
+uptimerobot: Uptime Robot
+vagabondo: Vagabondo
+vb project: Visual Basic
+vigil: Vigil
+vkshare: VKontake Sharer
+voilabot: VoilaBot
+vrcrawler: Venture Radar
+wasalive-bot: Wasalive Bots
+watchsumo: WatchSumo
+wbsearchbot: Ware Bay Best Buys
+webceo: online-webceo-bot
+webscout: Webscout
+wesee: WeSEE
+wget: wget unix CLI http client
+whatsapp: WhatsApp
+wikido: WikiDo
+woorank: WooRank
+wordpress: WordPress spider
+woriobot: woriobot
+wormly: WormlyBot
+wotbox: Wotbox
+xenu link sleuth: Xenu Link Sleuth
+xing-contenttabreceiver: Xing bot
+xovibot: XoviBot
+yacybot: YaCy
+yahoo-ad-monitoring: Yahoo Ad monitoring
+yandex: Yandex
+yanga: Yanga WorldSearch Bot
+yeti: Naver Corp
+yourls: YOURLS
+zabbix: Zabbix
+zelist.ro: feed parser
+zibb: ZIBB spider
+zitebot: Zite
+zoombot: ZoomBot
+zoominfobot: ZoominfoBot
+zyborg: Zyborg
 
 # Generic lib user agents go here.
-eventmachine httpclient: "Ruby http library"
-go 1.1 package http: "Go 1.1 package http"
-htmlparser: "HTMLParser"
-http_request2: "HTTP_Request2"
-httpclient: "HTTPClient"
-jakarta commons: "Jakarta Commons HttpClient"
-java: "Generic Java http library"
-libwww-perl: "Perl client-server library"
-lwp-trivial: "Another Perl library"
-ruby: "Ruby"
+eventmachine httpclient: Ruby http library
+go 1.1 package http: Go 1.1 package http
+htmlparser: HTMLParser
+http_request2: HTTP_Request2
+httpclient: HTTPClient
+jakarta commons: Jakarta Commons HttpClient
+java: Generic Java http library
+libwww-perl: Perl client-server library
+lwp-trivial: Another Perl library
+ruby: Ruby