public_suffix_service 0.9.0 → 0.9.1
Sign up to get free protection for your applications and to get access to all the features.
- data/.gitignore +0 -1
- data/.travis.yml +11 -0
- data/.yardopts +1 -1
- data/CHANGELOG.md +11 -0
- data/Gemfile.lock +6 -4
- data/README.md +8 -135
- data/Rakefile +23 -31
- data/lib/public_suffix.rb +134 -0
- data/lib/{public_suffix_service → public_suffix}/definitions.txt +282 -1
- data/lib/{public_suffix_service → public_suffix}/domain.rb +50 -50
- data/lib/{public_suffix_service → public_suffix}/errors.rb +12 -12
- data/lib/{public_suffix_service/rule_list.rb → public_suffix/list.rb} +54 -55
- data/lib/{public_suffix_service → public_suffix}/rule.rb +29 -29
- data/lib/public_suffix/rule_list.rb +14 -0
- data/lib/{public_suffix_service → public_suffix}/version.rb +4 -4
- data/lib/public_suffix_service.rb +5 -121
- data/public_suffix_service.gemspec +18 -15
- data/test/acceptance_test.rb +3 -3
- data/test/test_helper.rb +2 -6
- data/test/{public_suffix_service → unit}/domain_test.rb +11 -11
- data/test/unit/errors_test.rb +23 -0
- data/test/unit/list_test.rb +193 -0
- data/test/unit/public_suffix_test.rb +85 -0
- data/test/{public_suffix_service → unit}/rule_test.rb +22 -22
- metadata +66 -59
- data/test/public_suffix_service/errors_test.rb +0 -23
- data/test/public_suffix_service/rule_list_test.rb +0 -193
- data/test/public_suffix_service_test.rb +0 -85
data/.travis.yml
ADDED
data/.yardopts
CHANGED
@@ -1,2 +1,2 @@
|
|
1
1
|
--readme README.rdoc
|
2
|
-
--title 'Public Suffix
|
2
|
+
--title 'Ruby Public Suffix API Documentation'
|
data/CHANGELOG.md
CHANGED
@@ -1,6 +1,17 @@
|
|
1
1
|
# Changelog
|
2
2
|
|
3
3
|
|
4
|
+
## Release 0.9.1
|
5
|
+
|
6
|
+
* CHANGED: Renamed PublicSuffixService::RuleList to PublicSuffixService::List.
|
7
|
+
|
8
|
+
* CHANGED: Renamed PublicSuffixService::List#list to PublicSuffixService::List#rules.
|
9
|
+
|
10
|
+
* CHANGED: Renamed PublicSuffixService to PublicSuffix.
|
11
|
+
|
12
|
+
* CHANGED: Updated definitions.
|
13
|
+
|
14
|
+
|
4
15
|
## Release 0.9.0
|
5
16
|
|
6
17
|
* CHANGED: Minimum Ruby version increased to Ruby 1.8.7.
|
data/Gemfile.lock
CHANGED
@@ -1,18 +1,20 @@
|
|
1
1
|
PATH
|
2
2
|
remote: .
|
3
3
|
specs:
|
4
|
-
public_suffix_service (0.
|
4
|
+
public_suffix_service (0.9.1)
|
5
5
|
|
6
6
|
GEM
|
7
7
|
remote: http://rubygems.org/
|
8
8
|
specs:
|
9
|
-
|
10
|
-
|
9
|
+
mocha (0.9.12)
|
10
|
+
rake (0.9.2)
|
11
|
+
yard (0.7.2)
|
11
12
|
|
12
13
|
PLATFORMS
|
13
14
|
ruby
|
14
15
|
|
15
16
|
DEPENDENCIES
|
17
|
+
mocha
|
16
18
|
public_suffix_service!
|
17
|
-
|
19
|
+
rake
|
18
20
|
yard
|
data/README.md
CHANGED
@@ -1,144 +1,17 @@
|
|
1
|
-
# Public Suffix
|
1
|
+
# Public Suffix List - Ruby implementation
|
2
2
|
|
3
|
-
*Public Suffix Service* is a Ruby domain name parser based on the [Public Suffix List](http://publicsuffix.org).
|
3
|
+
*Public Suffix* (formerly *Public Suffix Service*) is a Ruby domain name parser based on the [Public Suffix List](http://publicsuffix.org).
|
4
4
|
|
5
5
|
|
6
|
-
##
|
6
|
+
## Important
|
7
7
|
|
8
|
-
The
|
8
|
+
The `public_suffix_list` gem is now known as `public_suffix`.
|
9
|
+
Please update your applications to use the new gem.
|
9
10
|
|
10
|
-
|
11
|
-
|
12
|
-
A "public suffix" is one under which Internet users can directly register names. Some examples of public suffixes are ".com", ".co.uk" and "pvt.k12.wy.us". The Public Suffix List is a list of all known public suffixes.
|
13
|
-
|
14
|
-
Source: http://publicsuffix.org
|
15
|
-
|
16
|
-
|
17
|
-
## Why the Public Suffix List is better than any available Regular Expression parser?
|
18
|
-
|
19
|
-
Previously, browsers used an algorithm which basically only denied setting wide-ranging cookies for top-level domains with no dots (e.g. com or org). However, this did not work for top-level domains where only third-level registrations are allowed (e.g. co.uk). In these cases, websites could set a cookie for co.uk which will be passed onto every website registered under co.uk.
|
20
|
-
|
21
|
-
Clearly, this was a security risk as it allowed websites other than the one setting the cookie to read it, and therefore potentially extract sensitive information.
|
22
|
-
|
23
|
-
Since there is no algorithmic method of finding the highest level at which a domain may be registered for a particular top-level domain (the policies differ with each registry), the only method is to create a list of all top-level domains and the level at which domains can be registered. This is the aim of the effective TLD list.
|
24
|
-
|
25
|
-
As well as being used to prevent cookies from being set where they shouldn't be, the list can also potentially be used for other applications where the registry controlled and privately controlled parts of a domain name need to be known, for example when grouping by top-level domains.
|
26
|
-
|
27
|
-
Source: https://wiki.mozilla.org/Public_Suffix_List
|
28
|
-
|
29
|
-
Not convinced yet? Check out a real world example:
|
30
|
-
http://stackoverflow.com/questions/288810/get-the-subdomain-from-a-url
|
31
|
-
|
32
|
-
|
33
|
-
## Requirements
|
34
|
-
|
35
|
-
* Ruby >= 1.8.7
|
36
|
-
|
37
|
-
Public Suffix Service >= 0.9.0 requires Ruby 1.8.7 or newer.
|
38
|
-
For older versions of Ruby, see the CHANGELOG.rdoc file.
|
39
|
-
|
40
|
-
Successfully tested with the following interpreters
|
41
|
-
|
42
|
-
* Ruby 1.8.7 / 1.9.2
|
43
|
-
* MacRuby
|
44
|
-
* Ruby Enterprise Edition
|
45
|
-
|
46
|
-
|
47
|
-
## Installation
|
48
|
-
|
49
|
-
The best way to install Public Suffix Service is via [RubyGems](http://www.rubygems.org).
|
50
|
-
|
51
|
-
$ gem install public_suffix_service
|
52
|
-
|
53
|
-
You might need administrator privileges on your system to install the Gem.
|
54
|
-
|
55
|
-
|
56
|
-
## Basic Usage
|
57
|
-
|
58
|
-
Example domain without subdomains.
|
59
|
-
|
60
|
-
domain = PublicSuffixService.parse("google.com")
|
61
|
-
# => #<PublicSuffixService::Domain>
|
62
|
-
domain.tld
|
63
|
-
# => "com"
|
64
|
-
domain.sld
|
65
|
-
# => "google"
|
66
|
-
domain.trd
|
67
|
-
# => nil
|
68
|
-
domain.domain
|
69
|
-
# => "google.com"
|
70
|
-
domain.subdomain
|
71
|
-
# => nil
|
72
|
-
|
73
|
-
Example domain with subdomains.
|
74
|
-
|
75
|
-
domain = PublicSuffixService.parse("www.google.com")
|
76
|
-
# => #<PublicSuffixService::Domain>
|
77
|
-
domain.tld
|
78
|
-
# => "com"
|
79
|
-
domain.sld
|
80
|
-
# => "google"
|
81
|
-
domain.trd
|
82
|
-
# => "www"
|
83
|
-
domain.domain
|
84
|
-
# => "google.com"
|
85
|
-
domain.subdomain
|
86
|
-
# => "google.com"
|
87
|
-
|
88
|
-
Simple validation example.
|
89
|
-
|
90
|
-
PublicSuffixService.valid?("google.com")
|
91
|
-
# => true
|
92
|
-
|
93
|
-
PublicSuffixService.valid?("www.google.com")
|
94
|
-
# => true
|
95
|
-
|
96
|
-
PublicSuffixService.valid?("x.yz")
|
97
|
-
# => false
|
98
|
-
|
99
|
-
## Fully Qualified Domain Names
|
100
|
-
|
101
|
-
This library automatically recognizes Fully Qualified Domain Names. A FQDN is a domain name that end with a trailing dot.
|
102
|
-
|
103
|
-
# Parse a standard domain name
|
104
|
-
domain = PublicSuffixService.parse("www.google.com")
|
105
|
-
# => #<PublicSuffixService::Domain>
|
106
|
-
domain.tld
|
107
|
-
# => "com"
|
108
|
-
|
109
|
-
# Parse a fully qualified domain name
|
110
|
-
domain = PublicSuffixService.parse("www.google.com.")
|
111
|
-
# => #<PublicSuffixService::Domain>
|
112
|
-
domain.tld
|
113
|
-
# => "com"
|
114
|
-
|
115
|
-
|
116
|
-
## Credits
|
117
|
-
|
118
|
-
* **Author:** [Simone Carletti](http://www.simonecarletti.com/)
|
119
|
-
|
120
|
-
|
121
|
-
## FeedBack and Bug reports
|
122
|
-
|
123
|
-
If you use this library and find yourself missing any functionality I have missed, please [let me know](mailto:weppos@weppos.net).
|
124
|
-
|
125
|
-
Bug reports and Feature suggestions [are welcomed](http://github.com/weppos/public_suffix_service/issues).
|
126
|
-
|
127
|
-
|
128
|
-
## More
|
129
|
-
|
130
|
-
* [Homepage](http://www.simonecarletti.com/code/public_suffix_service)
|
131
|
-
* [Repository](http://github.com/weppos/public_suffix_service)
|
132
|
-
* [API Documentation](http://www.simonecarletti.com/code/public_suffix_service/api/) (RDoc)
|
133
|
-
* [Introducing the Public Suffix List library for Ruby](http://www.simonecarletti.com/blog/2010/06/public-suffix-list-library-for-ruby/)
|
134
|
-
|
135
|
-
|
136
|
-
## Changelog
|
137
|
-
|
138
|
-
See the CHANGELOG.md file for details.
|
11
|
+
$ gem install public_suffix
|
139
12
|
|
140
13
|
|
141
14
|
## License
|
142
15
|
|
143
|
-
|
144
|
-
|
16
|
+
*Public Suffix* is copyright (c) 2009-2011 Simone Carletti.
|
17
|
+
This is Free Software distributed under the MIT license.
|
data/Rakefile
CHANGED
@@ -1,18 +1,15 @@
|
|
1
1
|
require 'rubygems'
|
2
|
+
require 'rubygems/package_task'
|
2
3
|
require 'bundler'
|
3
4
|
require 'rake/testtask'
|
4
|
-
require 'rubygems/package_task'
|
5
|
-
require 'yard'
|
6
|
-
require 'yard/rake/yardoc_task'
|
7
|
-
|
8
5
|
|
9
6
|
$:.unshift(File.dirname(__FILE__) + "/lib")
|
10
|
-
require '
|
7
|
+
require 'public_suffix'
|
11
8
|
|
12
9
|
|
13
10
|
# Common package properties
|
14
|
-
PKG_NAME = ENV['PKG_NAME'] ||
|
15
|
-
PKG_VERSION = ENV['PKG_VERSION'] ||
|
11
|
+
PKG_NAME = ENV['PKG_NAME'] || PublicSuffix::GEM
|
12
|
+
PKG_VERSION = ENV['PKG_VERSION'] || PublicSuffix::VERSION
|
16
13
|
RUBYFORGE_PROJECT = nil
|
17
14
|
|
18
15
|
if ENV['SNAPSHOT'].to_i == 1
|
@@ -20,15 +17,8 @@ if ENV['SNAPSHOT'].to_i == 1
|
|
20
17
|
end
|
21
18
|
|
22
19
|
|
23
|
-
# Run all the tests in the /test folder
|
24
|
-
Rake::TestTask.new do |t|
|
25
|
-
t.libs << "test"
|
26
|
-
t.test_files = FileList["test/**/*_test.rb"]
|
27
|
-
t.verbose = true
|
28
|
-
end
|
29
|
-
|
30
20
|
# Run test by default.
|
31
|
-
task :default =>
|
21
|
+
task :default => :test
|
32
22
|
|
33
23
|
|
34
24
|
# This builds the actual gem. For details of what all these options
|
@@ -40,13 +30,13 @@ spec = Gem::Specification.new do |s|
|
|
40
30
|
s.name = PKG_NAME
|
41
31
|
s.version = PKG_VERSION
|
42
32
|
s.summary = "Domain name parser based in the Public Suffix List."
|
43
|
-
s.description = "PublicSuffixService
|
33
|
+
s.description = "PublicSuffixService gem is now known as PublicSuffix."
|
44
34
|
|
45
35
|
s.required_ruby_version = ">= 1.8.7"
|
46
36
|
|
47
37
|
s.author = "Simone Carletti"
|
48
38
|
s.email = "weppos@weppos.net"
|
49
|
-
s.homepage = "http://www.simonecarletti.com/code/
|
39
|
+
s.homepage = "http://www.simonecarletti.com/code/public_suffix"
|
50
40
|
|
51
41
|
# Add any extra files to include in the gem (like your README)
|
52
42
|
s.files = `git ls-files`.split("\n")
|
@@ -58,14 +48,11 @@ spec = Gem::Specification.new do |s|
|
|
58
48
|
# s.add_dependency("some_other_gem", "~> 0.1.0")
|
59
49
|
|
60
50
|
# If your tests use any gems, include them here
|
61
|
-
s.add_development_dependency("
|
51
|
+
s.add_development_dependency("rake")
|
52
|
+
s.add_development_dependency("mocha")
|
62
53
|
s.add_development_dependency("yard")
|
63
54
|
end
|
64
55
|
|
65
|
-
# This task actually builds the gem. We also regenerate a static
|
66
|
-
# .gemspec file, which is useful if something (i.e. GitHub) will
|
67
|
-
# be automatically building a gem for this project. If you're not
|
68
|
-
# using GitHub, edit as appropriate.
|
69
56
|
Gem::PackageTask.new(spec) do |pkg|
|
70
57
|
pkg.gem_spec = spec
|
71
58
|
end
|
@@ -88,18 +75,23 @@ desc "Package the library and generates the gemspec"
|
|
88
75
|
task :package => [:gemspec]
|
89
76
|
|
90
77
|
|
78
|
+
# Run all the tests in the /test folder
|
79
|
+
Rake::TestTask.new do |t|
|
80
|
+
t.libs << "test"
|
81
|
+
t.test_files = FileList["test/**/*_test.rb"]
|
82
|
+
t.verbose = !!ENV["VERBOSE"]
|
83
|
+
t.warning = !!ENV["WARNING"]
|
84
|
+
end
|
85
|
+
|
86
|
+
|
87
|
+
require 'yard'
|
88
|
+
require 'yard/rake/yardoc_task'
|
89
|
+
|
91
90
|
YARD::Rake::YardocTask.new(:yardoc) do |y|
|
92
91
|
y.options = ["--output-dir", "yardoc"]
|
93
92
|
end
|
94
93
|
|
95
94
|
namespace :yardoc do
|
96
|
-
desc "Publish YARD documentation to the site"
|
97
|
-
task :publish => ["yardoc:clobber", "yardoc"] do
|
98
|
-
ENV["username"] || raise(ArgumentError, "Missing ssh username")
|
99
|
-
sh "rsync -avz --delete yardoc/ #{ENV["username"]}@code:/var/www/apps/code/#{PKG_NAME}/api"
|
100
|
-
end
|
101
|
-
|
102
|
-
desc "Remove YARD products"
|
103
95
|
task :clobber do
|
104
96
|
rm_r "yardoc" rescue nil
|
105
97
|
end
|
@@ -110,7 +102,7 @@ task :clobber => "yardoc:clobber"
|
|
110
102
|
|
111
103
|
desc "Open an irb session preloaded with this library"
|
112
104
|
task :console do
|
113
|
-
sh "irb -rubygems -I lib -r
|
105
|
+
sh "irb -rubygems -I lib -r public_suffix.rb"
|
114
106
|
end
|
115
107
|
|
116
108
|
|
@@ -123,7 +115,7 @@ task :download_definitions do
|
|
123
115
|
|
124
116
|
DEFINITION_URL = "http://mxr.mozilla.org/mozilla-central/source/netwerk/dns/effective_tld_names.dat?raw=1"
|
125
117
|
|
126
|
-
File.open("lib/
|
118
|
+
File.open("lib/public_suffix/definitions.txt", "w+") do |f|
|
127
119
|
response = Net::HTTP.get_response(URI.parse(DEFINITION_URL))
|
128
120
|
f.write(response.body)
|
129
121
|
end
|
@@ -0,0 +1,134 @@
|
|
1
|
+
#--
|
2
|
+
# Public Suffix
|
3
|
+
#
|
4
|
+
# Domain name parser based on the Public Suffix List.
|
5
|
+
#
|
6
|
+
# Copyright (c) 2009-2011 Simone Carletti <weppos@weppos.net>
|
7
|
+
#++
|
8
|
+
|
9
|
+
|
10
|
+
require 'public_suffix/domain'
|
11
|
+
require 'public_suffix/version'
|
12
|
+
require 'public_suffix/errors'
|
13
|
+
require 'public_suffix/rule'
|
14
|
+
require 'public_suffix/list'
|
15
|
+
|
16
|
+
|
17
|
+
module PublicSuffix
|
18
|
+
|
19
|
+
NAME = "Public Suffix"
|
20
|
+
GEM = "public_suffix_service"
|
21
|
+
AUTHORS = ["Simone Carletti <weppos@weppos.net>"]
|
22
|
+
|
23
|
+
|
24
|
+
# Backwards compatibility
|
25
|
+
autoload :RuleList, 'public_suffix/rule_list'
|
26
|
+
|
27
|
+
|
28
|
+
# Parses +domain+ and returns the
|
29
|
+
# {PublicSuffix::Domain} instance.
|
30
|
+
#
|
31
|
+
# Parsing uses the default {PublicSuffix::List}.
|
32
|
+
#
|
33
|
+
# @param [String, #to_s] domain
|
34
|
+
# The domain name or fully qualified domain name to parse.
|
35
|
+
#
|
36
|
+
# @return [PublicSuffix::Domain]
|
37
|
+
#
|
38
|
+
# @example Parse a valid domain
|
39
|
+
# PublicSuffix.parse("google.com")
|
40
|
+
# # => #<PublicSuffix::Domain ...>
|
41
|
+
#
|
42
|
+
# @example Parse a valid subdomain
|
43
|
+
# PublicSuffix.parse("www.google.com")
|
44
|
+
# # => #<PublicSuffix::Domain ...>
|
45
|
+
#
|
46
|
+
# @example Parse a fully qualified domain
|
47
|
+
# PublicSuffix.parse("google.com.")
|
48
|
+
# # => #<PublicSuffix::Domain ...>
|
49
|
+
#
|
50
|
+
# @example Parse a fully qualified domain (subdomain)
|
51
|
+
# PublicSuffix.parse("www.google.com.")
|
52
|
+
# # => #<PublicSuffix::Domain ...>
|
53
|
+
#
|
54
|
+
# @example Parse an invalid domain
|
55
|
+
# PublicSuffix.parse("x.yz")
|
56
|
+
# # => PublicSuffix::DomainInvalid
|
57
|
+
#
|
58
|
+
# @example Parse an URL (not supported, only domains)
|
59
|
+
# PublicSuffix.parse("http://www.google.com")
|
60
|
+
# # => PublicSuffix::DomainInvalid
|
61
|
+
#
|
62
|
+
# @raise [PublicSuffix::Error]
|
63
|
+
# If domain is not a valid domain.
|
64
|
+
# @raise [PublicSuffix::DomainNotAllowed]
|
65
|
+
# If a rule for +domain+ is found, but the rule
|
66
|
+
# doesn't allow +domain+.
|
67
|
+
#
|
68
|
+
def self.parse(domain)
|
69
|
+
rule = List.default.find(domain)
|
70
|
+
if rule.nil?
|
71
|
+
raise DomainInvalid, "`#{domain}' is not a valid domain"
|
72
|
+
end
|
73
|
+
if !rule.allow?(domain)
|
74
|
+
raise DomainNotAllowed, "`#{domain}' is not allowed according to Registry policy"
|
75
|
+
end
|
76
|
+
|
77
|
+
left, right = rule.decompose(domain)
|
78
|
+
|
79
|
+
parts = left.split(".")
|
80
|
+
# If we have 0 parts left, there is just a tld and no domain or subdomain
|
81
|
+
# If we have 1 part left, there is just a tld, domain and not subdomain
|
82
|
+
# If we have 2 parts left, the last part is the domain, the other parts (combined) are the subdomain
|
83
|
+
tld = right
|
84
|
+
sld = parts.empty? ? nil : parts.pop
|
85
|
+
trd = parts.empty? ? nil : parts.join(".")
|
86
|
+
|
87
|
+
Domain.new(tld, sld, trd)
|
88
|
+
end
|
89
|
+
|
90
|
+
# Checks whether +domain+ is assigned and allowed,
|
91
|
+
# without actually parsing it.
|
92
|
+
#
|
93
|
+
# This method doesn't care whether domain is a domain or subdomain.
|
94
|
+
# The validation is performed using the default {PublicSuffix::List}.
|
95
|
+
#
|
96
|
+
# @param [String, #to_s] domain
|
97
|
+
# The domain name or fully qualified domain name to validate.
|
98
|
+
#
|
99
|
+
# @return [Boolean]
|
100
|
+
#
|
101
|
+
# @example Validate a valid domain
|
102
|
+
# PublicSuffix.valid?("example.com")
|
103
|
+
# # => true
|
104
|
+
#
|
105
|
+
# @example Validate a valid subdomain
|
106
|
+
# PublicSuffix.valid?("www.example.com")
|
107
|
+
# # => true
|
108
|
+
#
|
109
|
+
# @example Validate a not-assigned domain
|
110
|
+
# PublicSuffix.valid?("example.zip")
|
111
|
+
# # => false
|
112
|
+
#
|
113
|
+
# @example Validate a not-allowed domain
|
114
|
+
# PublicSuffix.valid?("example.do")
|
115
|
+
# # => false
|
116
|
+
# PublicSuffix.valid?("www.example.do")
|
117
|
+
# # => true
|
118
|
+
#
|
119
|
+
# @example Validate a fully qualified domain
|
120
|
+
# PublicSuffix.valid?("google.com.")
|
121
|
+
# # => true
|
122
|
+
# PublicSuffix.valid?("www.google.com.")
|
123
|
+
# # => true
|
124
|
+
#
|
125
|
+
# @example Check an URL (which is not a valid domain)
|
126
|
+
# PublicSuffix.valid?("http://www.example.com")
|
127
|
+
# # => false
|
128
|
+
#
|
129
|
+
def self.valid?(domain)
|
130
|
+
rule = List.default.find(domain)
|
131
|
+
!rule.nil? && rule.allow?(domain)
|
132
|
+
end
|
133
|
+
|
134
|
+
end
|