maxmind-db 1.1.1 → 1.3.2

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
Files changed (62) hide show
  1. checksums.yaml +4 -4
  2. data/CHANGELOG.md +23 -0
  3. data/Gemfile.lock +58 -0
  4. data/README.dev.md +2 -35
  5. data/README.md +2 -3
  6. data/Rakefile +1 -0
  7. data/bin/mmdb-benchmark.rb +3 -3
  8. data/lib/maxmind/db/decoder.rb +3 -0
  9. data/lib/maxmind/db/file_reader.rb +29 -11
  10. data/lib/maxmind/db.rb +9 -10
  11. data/maxmind-db.gemspec +5 -4
  12. data/test/data/MaxMind-DB-spec.md +15 -11
  13. data/test/data/bad-data/maxminddb-python/bad-unicode-in-map-key.mmdb +0 -0
  14. data/test/data/source-data/GeoIP2-Anonymous-IP-Test.json +1 -0
  15. data/test/data/source-data/GeoIP2-City-Test.json +81 -48
  16. data/test/data/source-data/GeoIP2-Connection-Type-Test.json +20 -0
  17. data/test/data/source-data/GeoIP2-Country-Test.json +46 -58
  18. data/test/data/source-data/GeoIP2-Enterprise-Test.json +78 -6
  19. data/test/data/source-data/GeoIP2-ISP-Test.json +13 -1
  20. data/test/data/source-data/GeoIP2-Precision-Enterprise-Test.json +200 -13
  21. data/test/data/source-data/GeoLite2-ASN-Test.json +4091 -8
  22. data/test/data/source-data/GeoLite2-City-Test.json +12804 -0
  23. data/test/data/source-data/GeoLite2-Country-Test.json +11280 -0
  24. data/test/data/test-data/GeoIP2-Anonymous-IP-Test.mmdb +0 -0
  25. data/test/data/test-data/GeoIP2-City-Test-Broken-Double-Format.mmdb +0 -0
  26. data/test/data/test-data/GeoIP2-City-Test-Invalid-Node-Count.mmdb +0 -0
  27. data/test/data/test-data/GeoIP2-City-Test.mmdb +0 -0
  28. data/test/data/test-data/GeoIP2-Connection-Type-Test.mmdb +0 -0
  29. data/test/data/test-data/GeoIP2-Country-Test.mmdb +0 -0
  30. data/test/data/test-data/GeoIP2-DensityIncome-Test.mmdb +0 -0
  31. data/test/data/test-data/GeoIP2-Domain-Test.mmdb +0 -0
  32. data/test/data/test-data/GeoIP2-Enterprise-Test.mmdb +0 -0
  33. data/test/data/test-data/GeoIP2-ISP-Test.mmdb +0 -0
  34. data/test/data/test-data/GeoIP2-Precision-Enterprise-Test.mmdb +0 -0
  35. data/test/data/test-data/GeoIP2-Static-IP-Score-Test.mmdb +0 -0
  36. data/test/data/test-data/GeoIP2-User-Count-Test.mmdb +0 -0
  37. data/test/data/test-data/GeoLite2-ASN-Test.mmdb +0 -0
  38. data/test/data/test-data/GeoLite2-City-Test.mmdb +0 -0
  39. data/test/data/test-data/GeoLite2-Country-Test.mmdb +0 -0
  40. data/test/data/test-data/MaxMind-DB-no-ipv4-search-tree.mmdb +0 -0
  41. data/test/data/test-data/MaxMind-DB-string-value-entries.mmdb +0 -0
  42. data/test/data/test-data/MaxMind-DB-test-broken-pointers-24.mmdb +0 -0
  43. data/test/data/test-data/MaxMind-DB-test-broken-search-tree-24.mmdb +0 -0
  44. data/test/data/test-data/MaxMind-DB-test-decoder.mmdb +0 -0
  45. data/test/data/test-data/MaxMind-DB-test-ipv4-24.mmdb +0 -0
  46. data/test/data/test-data/MaxMind-DB-test-ipv4-28.mmdb +0 -0
  47. data/test/data/test-data/MaxMind-DB-test-ipv4-32.mmdb +0 -0
  48. data/test/data/test-data/MaxMind-DB-test-ipv6-24.mmdb +0 -0
  49. data/test/data/test-data/MaxMind-DB-test-ipv6-28.mmdb +0 -0
  50. data/test/data/test-data/MaxMind-DB-test-ipv6-32.mmdb +0 -0
  51. data/test/data/test-data/MaxMind-DB-test-metadata-pointers.mmdb +0 -0
  52. data/test/data/test-data/MaxMind-DB-test-mixed-24.mmdb +0 -0
  53. data/test/data/test-data/MaxMind-DB-test-mixed-28.mmdb +0 -0
  54. data/test/data/test-data/MaxMind-DB-test-mixed-32.mmdb +0 -0
  55. data/test/data/test-data/MaxMind-DB-test-nested.mmdb +0 -0
  56. data/test/data/test-data/MaxMind-DB-test-pointer-decoder.mmdb +0 -0
  57. data/test/data/test-data/README.md +3 -3
  58. data/test/data/test-data/write-test-data.pl +77 -22
  59. data/test/test_decoder.rb +8 -10
  60. data/test/test_reader.rb +22 -14
  61. metadata +13 -10
  62. data/test/data/MaxMind-DB-test-metadata-pointers.mmdb +0 -0
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: 485d10049eaba35df9d49f7aa4637857a3fe84f38dec3e2a163c675fc8f2e474
4
- data.tar.gz: 964008df8d8bc5bc8c558fb004cbb7eda6843295cb3abd5fea3fc193da58981d
3
+ metadata.gz: 9b642dbfca79a5abf8c5c5ab4a2ceb07051affb7148dd005514d0bc1a9950c3b
4
+ data.tar.gz: 6cb14299dec849aef6f054ca39cbf139d3e552ed593960ebfbd72a09a6ce5f27
5
5
  SHA512:
6
- metadata.gz: 92181b893b438f63a874d915817f4e325aa4a20fb12e8e373e6d6f4885235472353391db4508f313f7d4b5b9ab33e6f09cd05a5b88e718b59dc352dc18f415be
7
- data.tar.gz: a03201ffac72cac0228545ce2f1a3a5e3d84ad807d33fac18bfffd3da4a5be69d0bceee96c5c87429b394410ed3e3efc17f20893700b7811558c7a383d4773e4
6
+ metadata.gz: 232d60717fed7da9d7253edc041316eec8f71782b9fca2e72cdbd5ad5414e7253504d423b3bab105cd87c5fd1c62e6df2fe7b8a2d70ea19f97a5fc5fc493a572
7
+ data.tar.gz: c40ce6eec502c4f2f45d8b5036b179b9f498e680771571430228845f0a588a564e3a4adf66193f56440bb8c21940793cba5ee073f3eb444dbe9f303db9f6fe21
data/CHANGELOG.md CHANGED
@@ -1,5 +1,28 @@
1
1
  # Changelog
2
2
 
3
+ ## 1.3.2 (2025-04-03)
4
+
5
+ * Re-release to fix a release script problem. There are no code changes.
6
+
7
+ ## 1.3.1 (2025-04-03)
8
+
9
+ * Re-release to fix a release script problem. There are no code changes.
10
+
11
+ ## 1.3.0 (2025-04-03)
12
+
13
+ * Ruby 3.0+ is now required. If you're using Ruby 2.5, 2.6, or 2.7, please
14
+ use version 1.2.0 of this gem.
15
+
16
+ ## 1.2.0 (2023-11-22)
17
+
18
+ * Ruby 2.4 is no longer supported. If you're using Ruby 2.4, please use
19
+ version 1.1.1 of this gem.
20
+ * `Object#respond_to?` is no longer called on every read. Pull request by
21
+ Jean byroot Boussier. GitHub #65.
22
+ * The `get` and `get_prefix_length` methods now accept the IP addresses as
23
+ `IPAddr` objects. Strings are still accepted too. Pull request by Eddie
24
+ Lebow. GitHub #69.
25
+
3
26
  ## 1.1.1 (2020-06-23)
4
27
 
5
28
  * Fixed the memory reader's inspect method to no longer attempt to modify a
data/Gemfile.lock ADDED
@@ -0,0 +1,58 @@
1
+ PATH
2
+ remote: .
3
+ specs:
4
+ maxmind-db (1.3.2)
5
+
6
+ GEM
7
+ remote: https://rubygems.org/
8
+ specs:
9
+ ast (2.4.3)
10
+ json (2.10.2)
11
+ language_server-protocol (3.17.0.4)
12
+ lint_roller (1.1.0)
13
+ minitest (5.25.5)
14
+ parallel (1.26.3)
15
+ parser (3.3.7.4)
16
+ ast (~> 2.4.1)
17
+ racc
18
+ prism (1.4.0)
19
+ racc (1.8.1)
20
+ rainbow (3.1.1)
21
+ rake (13.2.1)
22
+ regexp_parser (2.10.0)
23
+ rubocop (1.75.2)
24
+ json (~> 2.3)
25
+ language_server-protocol (~> 3.17.0.2)
26
+ lint_roller (~> 1.1.0)
27
+ parallel (~> 1.10)
28
+ parser (>= 3.3.0.2)
29
+ rainbow (>= 2.2.2, < 4.0)
30
+ regexp_parser (>= 2.9.3, < 3.0)
31
+ rubocop-ast (>= 1.44.0, < 2.0)
32
+ ruby-progressbar (~> 1.7)
33
+ unicode-display_width (>= 2.4.0, < 4.0)
34
+ rubocop-ast (1.44.0)
35
+ parser (>= 3.3.7.2)
36
+ prism (~> 1.4)
37
+ rubocop-performance (1.25.0)
38
+ lint_roller (~> 1.1)
39
+ rubocop (>= 1.75.0, < 2.0)
40
+ rubocop-ast (>= 1.38.0, < 2.0)
41
+ ruby-progressbar (1.13.0)
42
+ unicode-display_width (3.1.4)
43
+ unicode-emoji (~> 4.0, >= 4.0.4)
44
+ unicode-emoji (4.0.4)
45
+
46
+ PLATFORMS
47
+ ruby
48
+ x86_64-linux
49
+
50
+ DEPENDENCIES
51
+ maxmind-db!
52
+ minitest
53
+ rake
54
+ rubocop
55
+ rubocop-performance
56
+
57
+ BUNDLED WITH
58
+ 2.6.2
data/README.dev.md CHANGED
@@ -1,37 +1,4 @@
1
1
  # How to release
2
- * Ensure tests pass: `rake`
3
- * Update changelog: Set version and release date
4
- * Set version in `maxmind-db.gemspec`
5
- * Add them: `git add -p`
6
- * Commit: `git commit -m v1.0.0`
7
- * Tag: `git tag -a v1.0.0 -m v1.0.0`
8
- * Clean up to be sure nothing stray gets into gem: `git clean -dxff`
9
- * Create `.gem` file: `gem build maxmind-db.gemspec`
10
- * Complete prerequisites (see below)
11
- * You only need to do this if `~/.gem/credentials` is missing
12
- `:rubygems_api_key`.
13
- * Upload to rubygems.org: `gem push maxmind-db-1.0.0.gem`
14
- * Push: `git push`
15
- * Push tag: `git push --tags`
16
- * Double check it looks okay at https://rubygems.org/gems/maxmind-db and
17
- https://www.rubydoc.info/gems/maxmind-db
18
2
 
19
-
20
- # Prerequisites
21
-
22
- ## Step 1
23
- Sign up for an account at rubygems.org if you don't have one.
24
-
25
- Enable multi factor authentication.
26
-
27
-
28
- ## Step 2
29
- Ask someone who is an owner of the gem to add you as one.
30
-
31
- They do this by using the `gem owner` command
32
- ([docs](https://guides.rubygems.org/command-reference/#gem-owner)).
33
-
34
-
35
- ## Step 3
36
- Go to your rubygems.org profile and find the curl command to run to
37
- download your API key.
3
+ See
4
+ [here](https://github.com/maxmind/minfraud-api-ruby/blob/main/README.dev.md).
data/README.md CHANGED
@@ -6,7 +6,6 @@ This is the Ruby API for reading [MaxMind
6
6
  DB](https://maxmind.github.io/MaxMind-DB/) files. MaxMind DB is a binary
7
7
  file format that stores data indexed by IP address subnets (IPv4 or IPv6).
8
8
 
9
-
10
9
  ## Installation
11
10
 
12
11
  ```
@@ -36,7 +35,7 @@ For more information see the
36
35
 
37
36
  ## Requirements
38
37
 
39
- This code requires Ruby version 2.4 or higher.
38
+ This code requires Ruby version 3.0 or higher.
40
39
 
41
40
  ## Contributing
42
41
 
@@ -57,7 +56,7 @@ This library uses [Semantic Versioning](https://semver.org/).
57
56
 
58
57
  ## Copyright and License
59
58
 
60
- This software is Copyright (c) 2018 - 2020 by MaxMind, Inc.
59
+ This software is Copyright (c) 2018 - 2024 by MaxMind, Inc.
61
60
 
62
61
  This is free software, licensed under the [Apache License, Version
63
62
  2.0](LICENSE-APACHE) or the [MIT License](LICENSE-MIT), at your option.
data/Rakefile CHANGED
@@ -1,5 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require 'bundler/gem_tasks'
3
4
  require 'rake/testtask'
4
5
  require 'rubocop/rake_task'
5
6
 
@@ -29,9 +29,9 @@ end
29
29
 
30
30
  def print_usage
31
31
  # rubocop:disable Style/StderrPuts
32
- STDERR.puts "Usage: #{$PROGRAM_NAME} <MMDB file> <IP file>"
33
- STDERR.puts
34
- STDERR.puts 'Benchmark by reading IPs from the IP file and looking up each one in the MMDB file.'
32
+ $stderr.puts "Usage: #{$PROGRAM_NAME} <MMDB file> <IP file>"
33
+ $stderr.puts
34
+ $stderr.puts 'Benchmark by reading IPs from the IP file and looking up each one in the MMDB file.'
35
35
  # rubocop:enable Style/StderrPuts
36
36
  end
37
37
 
@@ -12,6 +12,8 @@ module MaxMind
12
12
  #
13
13
  # @!visibility private
14
14
  class Decoder
15
+ # rubocop:disable Style/OptionalBooleanParameter
16
+
15
17
  # Create a +Decoder+.
16
18
  #
17
19
  # +io+ is the DB. It must provide a +read+ method. It must be opened in
@@ -28,6 +30,7 @@ module MaxMind
28
30
  @pointer_base = pointer_base
29
31
  @pointer_test = pointer_test
30
32
  end
33
+ # rubocop:enable Style/OptionalBooleanParameter
31
34
 
32
35
  private
33
36
 
@@ -6,10 +6,36 @@ module MaxMind
6
6
  class DB
7
7
  # @!visibility private
8
8
  class FileReader
9
+ if ::File.method_defined?(:pread)
10
+ PReadFile = ::File
11
+ else
12
+ # For Windows support
13
+ class PReadFile
14
+ def initialize(filename, mode)
15
+ @mutex = Mutex.new
16
+ @file = File.new(filename, mode)
17
+ end
18
+
19
+ def size
20
+ @file.size
21
+ end
22
+
23
+ def close
24
+ @file.close
25
+ end
26
+
27
+ def pread(size, offset)
28
+ @mutex.synchronize do
29
+ @file.seek(offset, IO::SEEK_SET)
30
+ @file.read(size)
31
+ end
32
+ end
33
+ end
34
+ end
35
+
9
36
  def initialize(filename)
10
- @fh = File.new(filename, 'rb')
37
+ @fh = PReadFile.new(filename, 'rb')
11
38
  @size = @fh.size
12
- @mutex = Mutex.new
13
39
  end
14
40
 
15
41
  attr_reader :size
@@ -21,15 +47,7 @@ module MaxMind
21
47
  def read(offset, size)
22
48
  return ''.b if size == 0
23
49
 
24
- # When we support only Ruby 2.5+, remove this and require pread.
25
- if @fh.respond_to?(:pread)
26
- buf = @fh.pread(size, offset)
27
- else
28
- @mutex.synchronize do
29
- @fh.seek(offset, IO::SEEK_SET)
30
- buf = @fh.read(size)
31
- end
32
- end
50
+ buf = @fh.pread(size, offset)
33
51
 
34
52
  raise InvalidDatabaseError, 'The MaxMind DB file contains bad data' if buf.nil? || buf.length != size
35
53
 
data/lib/maxmind/db.rb CHANGED
@@ -3,9 +3,9 @@
3
3
  require 'ipaddr'
4
4
  require 'maxmind/db/decoder'
5
5
  require 'maxmind/db/errors'
6
- require 'maxmind/db/file_reader.rb'
7
- require 'maxmind/db/memory_reader.rb'
8
- require 'maxmind/db/metadata.rb'
6
+ require 'maxmind/db/file_reader'
7
+ require 'maxmind/db/memory_reader'
8
+ require 'maxmind/db/metadata'
9
9
 
10
10
  module MaxMind
11
11
  # DB provides a way to read {MaxMind DB
@@ -130,8 +130,7 @@ module MaxMind
130
130
  #
131
131
  # If no record is found for the IP address, +get+ returns +nil+.
132
132
  #
133
- # @param ip_address [String] a string in the standard notation. It may be
134
- # IPv4 or IPv6.
133
+ # @param ip_address [String, IPAddr] IPv4 or IPv6 address.
135
134
  #
136
135
  # @raise [ArgumentError] if you attempt to look up an IPv6 address in an
137
136
  # IPv4-only database.
@@ -153,8 +152,7 @@ module MaxMind
153
152
  # If no record is found for the IP address, the record will be +nil+ and
154
153
  # the prefix length will be the value for the missing network.
155
154
  #
156
- # @param ip_address [String] a string in the standard notation. It may be
157
- # IPv4 or IPv6.
155
+ # @param ip_address [String, IPAddr] IPv4 or IPv6 address.
158
156
  #
159
157
  # @raise [ArgumentError] if you attempt to look up an IPv6 address in an
160
158
  # IPv4-only database.
@@ -163,7 +161,8 @@ module MaxMind
163
161
  #
164
162
  # @return [Array<(Object, Integer)>]
165
163
  def get_with_prefix_length(ip_address)
166
- ip = IPAddr.new(ip_address)
164
+ ip = ip_address.is_a?(IPAddr) ? ip_address : IPAddr.new(ip_address)
165
+
167
166
  # We could check the IP has the correct prefix (32 or 128) but I do not
168
167
  # for performance reasons.
169
168
 
@@ -200,7 +199,7 @@ module MaxMind
200
199
  break if depth >= bit_count || node >= node_count
201
200
 
202
201
  c = packed[depth >> 3].ord
203
- bit = 1 & (c >> 7 - (depth % 8))
202
+ bit = 1 & (c >> (7 - (depth % 8)))
204
203
  node = read_node(node, bit)
205
204
  depth += 1
206
205
  end
@@ -273,7 +272,7 @@ module MaxMind
273
272
  end
274
273
 
275
274
  def find_metadata_start
276
- metadata_max_size = @size < METADATA_MAX_SIZE ? @size : METADATA_MAX_SIZE
275
+ metadata_max_size = [@size, METADATA_MAX_SIZE].min
277
276
 
278
277
  stop_index = @size - metadata_max_size
279
278
  index = @size - METADATA_START_MARKER_LENGTH
data/maxmind-db.gemspec CHANGED
@@ -2,10 +2,10 @@
2
2
 
3
3
  Gem::Specification.new do |s|
4
4
  s.authors = ['William Storey']
5
- s.files = Dir['**/*']
5
+ s.files = Dir['**/*'].difference(Dir['.github/**/*', 'dev-bin/**/*'])
6
6
  s.name = 'maxmind-db'
7
7
  s.summary = 'A gem for reading MaxMind DB files.'
8
- s.version = '1.1.1'
8
+ s.version = '1.3.2'
9
9
 
10
10
  s.description = 'A gem for reading MaxMind DB files. MaxMind DB is a binary file format that stores data indexed by IP address subnets (IPv4 or IPv6).'
11
11
  s.email = 'support@maxmind.com'
@@ -13,12 +13,13 @@ Gem::Specification.new do |s|
13
13
  s.licenses = ['Apache-2.0', 'MIT']
14
14
  s.metadata = {
15
15
  'bug_tracker_uri' => 'https://github.com/maxmind/MaxMind-DB-Reader-ruby/issues',
16
- 'changelog_uri' => 'https://github.com/maxmind/MaxMind-DB-Reader-ruby/blob/master/CHANGELOG.md',
16
+ 'changelog_uri' => 'https://github.com/maxmind/MaxMind-DB-Reader-ruby/blob/main/CHANGELOG.md',
17
17
  'documentation_uri' => 'https://www.rubydoc.info/gems/maxmind-db',
18
18
  'homepage_uri' => 'https://github.com/maxmind/MaxMind-DB-Reader-ruby',
19
19
  'source_code_uri' => 'https://github.com/maxmind/MaxMind-DB-Reader-ruby',
20
+ 'rubygems_mfa_required' => 'true',
20
21
  }
21
- s.required_ruby_version = '>= 2.4.0'
22
+ s.required_ruby_version = '>= 3.0'
22
23
 
23
24
  s.add_development_dependency 'minitest'
24
25
  s.add_development_dependency 'rake'
@@ -26,9 +26,10 @@ not be broken by minor version changes to the format.
26
26
  The binary database is split into three parts:
27
27
 
28
28
  1. The binary search tree. Each level of the tree corresponds to a single bit
29
- in the 128 bit representation of an IPv6 address.
30
- 2. The data section. These are the values returned to the client for a
31
- specific IP address, e.g. "US", "New York", or a more complex map type made up
29
+ in the prefix of the network the IP address belongs to.
30
+ 2. The data section with the values for the networks in the binary search
31
+ tree. These values may be comprised of a single data type, e.g., the string
32
+ "US" or "New York", or they may be a more complex map or array type made up
32
33
  of multiple fields.
33
34
  3. Database metadata. Information about the database itself.
34
35
 
@@ -48,15 +49,15 @@ of this sequence.
48
49
  The maximum allowable size for the metadata section, including the marker that
49
50
  starts the metadata, is 128KiB.
50
51
 
51
- The metadata is stored as a map data structure. This structure is described
52
- later in the spec. Changing a key's data type or removing a key would
53
- constitute a major version change for this spec.
52
+ The metadata is stored as a separate data section comprised of a map data
53
+ structure starting at the beginning of that section. This structure is
54
+ described later in the spec.
54
55
 
55
56
  Except where otherwise specified, each key listed is required for the database
56
57
  to be considered valid.
57
58
 
58
- Adding a key constitutes a minor version change. Removing a key or changing
59
- its type constitutes a major version change.
59
+ Changing a key's data type or removing a key would constitute a major version
60
+ change for this spec. Adding a key constitutes a minor version change.
60
61
 
61
62
  The list of known keys for the current version of the format is as follows:
62
63
 
@@ -304,7 +305,8 @@ will point to the beginning of a field. It is illegal for a pointer to point
304
305
  to another pointer.
305
306
 
306
307
  Pointer values start from the beginning of the data section, *not* the
307
- beginning of the file.
308
+ beginning of the file. Pointers in the metadata start from the beginning of
309
+ the metadata section.
308
310
 
309
311
  ### UTF-8 string - 2
310
312
 
@@ -337,8 +339,10 @@ by the length specifier in the control byte. See below for details.
337
339
 
338
340
  A length of zero always indicates the number 0.
339
341
 
340
- When storing a signed integer, the left-most bit is the sign. A 1 is negative
341
- and a 0 is positive.
342
+ When storing a signed integer, fields shorter than the maximum byte length
343
+ are always positive. When the field is the maximum length, e.g., 4 bytes for
344
+ 32-bit integers, the left-most bit is the sign. A 1 is negative and a 0 is
345
+ positive.
342
346
 
343
347
  The type numbers for our integer types are:
344
348
 
@@ -18,6 +18,7 @@
18
18
  "is_anonymous_vpn" : true,
19
19
  "is_hosting_provider" : true,
20
20
  "is_public_proxy" : true,
21
+ "is_residential_proxy" : true,
21
22
  "is_tor_exit_node" : true
22
23
  }
23
24
  },