bot_detection 1.0.1 → 1.0.2
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +4 -0
- data/README.md +29 -6
- data/lib/bot_detection/bot_user_agents.rb +7 -1
- data/lib/bot_detection/search_engine_user_agents.rb +1 -0
- data/lib/bot_detection/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA1:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: e95e799dda05201285a7785ebcdbd4bfac871278
|
|
4
|
+
data.tar.gz: a33bfd5449247970885c9687a3b7d9fbd7c14d0e
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: d6fb0bbbc3591a8cc295286b7cc2534fe38130cb9a6e422542aecf3aecbe642bb426a2fc222fd4a15e45ab6e72c6dc9ef10da2d9da4d44cbb512eeee79ba84bd
|
|
7
|
+
data.tar.gz: 89aeb6ec92cc70e203e390ceea71cb49d3edd523e11fb0f3fbd0a65a82e8e6ef28bc082772966c1deacfc62b6ceed720e8e05090fc12383499ebc088ac2e40d6
|
data/CHANGELOG.md
CHANGED
data/README.md
CHANGED
|
@@ -1,20 +1,31 @@
|
|
|
1
1
|
# BotDetection
|
|
2
2
|
|
|
3
|
-
|
|
3
|
+
BotDetection will help you identify users on your site as search engine crawlers or other bots like Facebook spiders.
|
|
4
|
+
Currently supported Bots are:
|
|
5
|
+
|
|
6
|
+
- Search Engines:
|
|
7
|
+
- all known Google-Bots
|
|
8
|
+
- Yahoo
|
|
9
|
+
- MSN
|
|
10
|
+
- Bing
|
|
11
|
+
- Yandex
|
|
12
|
+
- Baidu
|
|
13
|
+
- Bots
|
|
14
|
+
- Facebook
|
|
4
15
|
|
|
5
16
|
## Installation
|
|
6
17
|
|
|
7
18
|
Add this line to your application's Gemfile:
|
|
8
19
|
|
|
9
|
-
|
|
20
|
+
gem 'bot_detection'
|
|
10
21
|
|
|
11
22
|
And then execute:
|
|
12
23
|
|
|
13
|
-
|
|
24
|
+
$ bundle
|
|
14
25
|
|
|
15
26
|
Or install it yourself as:
|
|
16
27
|
|
|
17
|
-
|
|
28
|
+
$ gem install bot_detection
|
|
18
29
|
|
|
19
30
|
## Changes
|
|
20
31
|
|
|
@@ -22,11 +33,23 @@ See the [CHANGELOG.md](CHANGELOG.md) file for details.
|
|
|
22
33
|
|
|
23
34
|
## Usage
|
|
24
35
|
|
|
25
|
-
|
|
36
|
+
After installing the GEM you can use within your controllers or helpers the following methods
|
|
37
|
+
|
|
38
|
+
# this will return true if user agent and reverse DNS lookup will match one bot out of the database otherwise false is returned
|
|
39
|
+
is_search_engine_crawler?
|
|
40
|
+
|
|
41
|
+
# this will return true if user agent will match one bot out of the database otherwise false is returned, a reverse dns lookup isn't executed
|
|
42
|
+
is_search_engine_crawler? reverse_lookup: false
|
|
43
|
+
|
|
44
|
+
# will return true if the current user agent is a known bot otherwise false is returned
|
|
45
|
+
is_known_crawler?
|
|
46
|
+
|
|
47
|
+
# this will return true if the current user agent is associated with google and the reverse dns lookup will match google.com or googlebot.com
|
|
48
|
+
is_google?
|
|
26
49
|
|
|
27
50
|
## Contributing
|
|
28
51
|
|
|
29
|
-
1. Fork it ( https://github.com/
|
|
52
|
+
1. Fork it ( https://github.com/sumy/bot_detection/fork )
|
|
30
53
|
2. Create your feature branch (`git checkout -b my-new-feature`)
|
|
31
54
|
3. Commit your changes (`git commit -am 'Add some feature'`)
|
|
32
55
|
4. Push to the branch (`git push origin my-new-feature`)
|
|
@@ -3,5 +3,11 @@ BotDetection::BotUserAgents = [
|
|
|
3
3
|
"FacebookExternalHit/1.0",
|
|
4
4
|
"facebookexternalhit/1.0 (+http://www.facebook.com/externalhit_uatext.php)",
|
|
5
5
|
"facebookexternalhit/1.1 (+http://www.facebook.com/externalhit_uatext.php)",
|
|
6
|
-
"facebookplatform/1.0 (+http://developers.facebook.com)"
|
|
6
|
+
"facebookplatform/1.0 (+http://developers.facebook.com)",
|
|
7
|
+
"Mozilla/5.0 (compatible; MJ12bot/v1.4.5; http://www.majestic12.co.uk/bot.php?+)",
|
|
8
|
+
"Mozilla/5.0 (compatible; SISTRIX Crawler; http://crawler.sistrix.net/)",
|
|
9
|
+
"ShowyouBot (http://showyou.com/crawler)",
|
|
10
|
+
"Mozilla/5.0 (compatible; AhrefsBot/5.0; +http://ahrefs.com/robot/)",
|
|
11
|
+
"Mozilla/5.0 (compatible; proximic; +http://www.proximic.com/info/spider.php)",
|
|
12
|
+
"Mozilla/5.0 (Windows; U; Windows NT 6.0; en-GB; rv:1.0; trendictionbot0.5.0; trendiction search; http://www.trendiction.de/bot; please let us know of any problems; web at trendiction.com) Gecko/20071127 Firefox/3.0.0.11",
|
|
7
13
|
]
|