maxmind-db-rust 0.2.1 → 0.3.0
This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
- checksums.yaml +4 -4
- data/CHANGELOG.md +19 -0
- data/README.md +30 -28
- data/ext/maxmind_db_rust/Cargo.toml +2 -1
- data/ext/maxmind_db_rust/src/lib.rs +206 -289
- metadata +5 -8
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA256:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 1c9ce85a0f2ba6a7852b6b3b12ffacc517f2a484f41fb686b131df4e7c7685fb
|
|
4
|
+
data.tar.gz: 85f03d7a87b5b855cdbe66e00106503692d9ef13270f9f99a2074210f042291a
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: adadcd0e268b04c7aafa8d9540c62dce31158388c03e4539b9b579890d9572606cba2e83275e7d5f511e7c885c9b9ee0015ed581cc194b93618a5a8b89f1437c
|
|
7
|
+
data.tar.gz: b6c8eb6ddba493e62d145d2c028db95c13a54b8a64e41aff4d7651604890d77c31aac29f3eb5370c85da46e777369fc3c281d574408d5c0d965cfef3217c3159
|
data/CHANGELOG.md
CHANGED
|
@@ -5,6 +5,25 @@ All notable changes to this project will be documented in this file.
|
|
|
5
5
|
The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
|
|
6
6
|
and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html).
|
|
7
7
|
|
|
8
|
+
## [0.3.0] - 2026-02-22
|
|
9
|
+
|
|
10
|
+
### Changed
|
|
11
|
+
|
|
12
|
+
- Improved lookup performance by using a generic bounded key cache for decoded map keys.
|
|
13
|
+
- Improved `IPAddr` lookup performance by decoding packed bytes from `IPAddr#hton` directly.
|
|
14
|
+
- Switched map-key cache hashing to `FxHashMap` for faster key-cache access.
|
|
15
|
+
- Switched map-key cache roots to a Ruby-owned cache array with Rust key-to-index lookups.
|
|
16
|
+
- Refactored duplicated prefix and `within` decode paths in the Rust reader for simpler maintenance.
|
|
17
|
+
- Refactored duplicate database file-open error handling shared by MMAP and MEMORY modes.
|
|
18
|
+
- Updated Rust and Ruby dependencies.
|
|
19
|
+
- Added Ruby 4.0 coverage to CI workflows.
|
|
20
|
+
|
|
21
|
+
### Fixed
|
|
22
|
+
|
|
23
|
+
- Made extension initialization idempotent across `MaxMind::DB` class/module loading modes to avoid typed-data incompatibility when the extension is loaded more than once.
|
|
24
|
+
- When loaded with the official `MaxMind::DB` class, `MaxMind::DB::Rust` now uses anonymous module creation to preserve canonical module naming.
|
|
25
|
+
- Scoped Rust dependency cache per Ruby version in CI tests and stopped caching `target/` in the test workflow to avoid cross-version artifact contamination.
|
|
26
|
+
|
|
8
27
|
## [0.2.1] - 2025-12-18
|
|
9
28
|
|
|
10
29
|
### Changed
|
data/README.md
CHANGED
|
@@ -3,18 +3,19 @@
|
|
|
3
3
|
[](https://github.com/oschwald/maxmind-db-rust-ruby/actions/workflows/test.yml)
|
|
4
4
|
[](https://github.com/oschwald/maxmind-db-rust-ruby/actions/workflows/lint.yml)
|
|
5
5
|
|
|
6
|
-
A
|
|
6
|
+
A Ruby gem for reading MaxMind DB files, implemented in Rust.
|
|
7
|
+
It keeps the API close to the official `maxmind-db` gem while adding Rust-backed performance.
|
|
7
8
|
|
|
8
9
|
> **Note:** This is an unofficial library and is not endorsed by MaxMind. For the official Ruby library, see [maxmind-db](https://github.com/maxmind/MaxMind-DB-Reader-ruby).
|
|
9
10
|
|
|
10
11
|
## Features
|
|
11
12
|
|
|
12
|
-
-
|
|
13
|
-
-
|
|
14
|
-
-
|
|
15
|
-
-
|
|
16
|
-
-
|
|
17
|
-
-
|
|
13
|
+
- Rust implementation focused on fast lookups
|
|
14
|
+
- API modeled after the official `maxmind-db` gem
|
|
15
|
+
- Thread-safe lookups
|
|
16
|
+
- Supports MMAP and in-memory modes
|
|
17
|
+
- Includes network iteration support
|
|
18
|
+
- Accepts both `String` and `IPAddr` inputs
|
|
18
19
|
|
|
19
20
|
## Installation
|
|
20
21
|
|
|
@@ -277,30 +278,31 @@ Metadata attributes:
|
|
|
277
278
|
|
|
278
279
|
## Comparison with Official Gem
|
|
279
280
|
|
|
280
|
-
| Feature | maxmind-db (official) | maxmind-db-rust (this gem)
|
|
281
|
-
| ---------------- | --------------------- |
|
|
282
|
-
| Implementation | Pure Ruby | Rust with Ruby bindings
|
|
283
|
-
| Performance | Baseline |
|
|
284
|
-
| API | MaxMind::DB | MaxMind::DB::Rust
|
|
285
|
-
| MODE_FILE | ✓ | ✗
|
|
286
|
-
| MODE_MEMORY | ✓ | ✓
|
|
287
|
-
| MODE_AUTO | ✓ | ✓
|
|
288
|
-
| MODE_MMAP | ✗ | ✓
|
|
289
|
-
| Iterator support | ✗ | ✓
|
|
290
|
-
| Thread-safe | ✓ | ✓
|
|
281
|
+
| Feature | maxmind-db (official) | maxmind-db-rust (this gem) |
|
|
282
|
+
| ---------------- | --------------------- | ------------------------------------------ |
|
|
283
|
+
| Implementation | Pure Ruby | Rust with Ruby bindings |
|
|
284
|
+
| Performance | Baseline | Faster lookup throughput in our benchmarks |
|
|
285
|
+
| API | MaxMind::DB | MaxMind::DB::Rust |
|
|
286
|
+
| MODE_FILE | ✓ | ✗ |
|
|
287
|
+
| MODE_MEMORY | ✓ | ✓ |
|
|
288
|
+
| MODE_AUTO | ✓ | ✓ |
|
|
289
|
+
| MODE_MMAP | ✗ | ✓ |
|
|
290
|
+
| Iterator support | ✗ | ✓ |
|
|
291
|
+
| Thread-safe | ✓ | ✓ |
|
|
291
292
|
|
|
292
293
|
## Performance
|
|
293
294
|
|
|
294
|
-
|
|
295
|
+
Lookup performance depends on hardware, Ruby version, database, and workload.
|
|
295
296
|
|
|
296
|
-
-
|
|
297
|
-
-
|
|
298
|
-
-
|
|
299
|
-
-
|
|
297
|
+
- In this project’s random-lookup benchmarks, this gem is consistently faster than the official Ruby implementation.
|
|
298
|
+
- On `/var/lib/GeoIP/GeoIP2-City.mmdb` in this environment, random lookup throughput was about `47x` higher than the official gem.
|
|
299
|
+
- `MODE_MMAP` and `MODE_MEMORY` both perform well; which is faster can vary by environment.
|
|
300
|
+
- For reproducible numbers on your own data, run `benchmark/compare_lookups.rb` against your database.
|
|
301
|
+
- Safe for concurrent lookups across threads.
|
|
300
302
|
|
|
301
303
|
## Development
|
|
302
304
|
|
|
303
|
-
|
|
305
|
+
See [CONTRIBUTING.md](CONTRIBUTING.md) for developer documentation, including:
|
|
304
306
|
|
|
305
307
|
- Development setup and prerequisites
|
|
306
308
|
- Building and testing the extension
|
|
@@ -321,11 +323,11 @@ bundle exec rake test
|
|
|
321
323
|
|
|
322
324
|
## Contributing
|
|
323
325
|
|
|
324
|
-
1. Fork
|
|
325
|
-
2. Create
|
|
326
|
-
3. Commit your changes (`git commit -am '
|
|
326
|
+
1. Fork the repository
|
|
327
|
+
2. Create a feature branch (`git checkout -b my-new-feature`)
|
|
328
|
+
3. Commit your changes (`git commit -am 'Describe your change'`)
|
|
327
329
|
4. Push to the branch (`git push origin my-new-feature`)
|
|
328
|
-
5.
|
|
330
|
+
5. Open a Pull Request
|
|
329
331
|
|
|
330
332
|
## License
|
|
331
333
|
|
|
@@ -12,9 +12,10 @@ name = "maxmind_db_rust"
|
|
|
12
12
|
crate-type = ["cdylib"]
|
|
13
13
|
|
|
14
14
|
[dependencies]
|
|
15
|
-
arc-swap = "1.
|
|
15
|
+
arc-swap = "1.8"
|
|
16
16
|
ipnetwork = "0.21"
|
|
17
17
|
magnus = "0.8"
|
|
18
18
|
maxminddb = { version = "0.27", features = ["unsafe-str-decode"] }
|
|
19
19
|
memmap2 = "0.9"
|
|
20
|
+
rustc-hash = "2.1"
|
|
20
21
|
serde = "1.0"
|
|
@@ -5,14 +5,15 @@ use ::maxminddb as maxminddb_crate;
|
|
|
5
5
|
use arc_swap::{ArcSwapOption, Guard};
|
|
6
6
|
use ipnetwork::IpNetwork;
|
|
7
7
|
use magnus::{
|
|
8
|
-
error::Error, prelude::*, scan_args::get_kwargs, scan_args::scan_args,
|
|
9
|
-
|
|
8
|
+
error::Error, prelude::*, scan_args::get_kwargs, scan_args::scan_args, ExceptionClass,
|
|
9
|
+
IntoValue, RArray, RClass, RHash, RModule, RString, Symbol, Value,
|
|
10
10
|
};
|
|
11
11
|
use maxminddb_crate::{MaxMindDbError, Reader as MaxMindReader, Within};
|
|
12
12
|
use memmap2::Mmap;
|
|
13
|
+
use rustc_hash::FxHashMap;
|
|
13
14
|
use serde::de::{self, Deserialize, DeserializeSeed, Deserializer, MapAccess, SeqAccess, Visitor};
|
|
14
15
|
use std::{
|
|
15
|
-
|
|
16
|
+
cell::RefCell,
|
|
16
17
|
collections::BTreeMap,
|
|
17
18
|
fmt,
|
|
18
19
|
fs::File,
|
|
@@ -31,176 +32,58 @@ const ERR_CLOSED_DB: &str = "Attempt to read from a closed MaxMind DB.";
|
|
|
31
32
|
const ERR_BAD_DATA: &str =
|
|
32
33
|
"The MaxMind DB file's data section contains bad data (unknown data type or corrupt data)";
|
|
33
34
|
|
|
34
|
-
|
|
35
|
-
|
|
36
|
-
|
|
37
|
-
|
|
38
|
-
|
|
39
|
-
|
|
40
|
-
|
|
41
|
-
|
|
42
|
-
|
|
43
|
-
|
|
44
|
-
|
|
45
|
-
|
|
46
|
-
|
|
47
|
-
|
|
48
|
-
|
|
49
|
-
|
|
50
|
-
b"ja" => Some(ruby.get_inner(&$crate::JA_KEY).as_value()),
|
|
51
|
-
b"ru" => Some(ruby.get_inner(&$crate::RU_KEY).as_value()),
|
|
52
|
-
b"AF" => Some(ruby.get_inner(&$crate::AF_KEY).as_value()),
|
|
53
|
-
b"AN" => Some(ruby.get_inner(&$crate::AN_KEY).as_value()),
|
|
54
|
-
b"AS" => Some(ruby.get_inner(&$crate::AS_KEY).as_value()),
|
|
55
|
-
b"EU" => Some(ruby.get_inner(&$crate::EU_KEY).as_value()),
|
|
56
|
-
b"NA" => Some(ruby.get_inner(&$crate::NA_KEY).as_value()),
|
|
57
|
-
b"OC" => Some(ruby.get_inner(&$crate::OC_KEY).as_value()),
|
|
58
|
-
b"SA" => Some(ruby.get_inner(&$crate::SA_KEY).as_value()),
|
|
59
|
-
b"US" => Some(ruby.get_inner(&$crate::US_VAL).as_value()),
|
|
60
|
-
b"CN" => Some(ruby.get_inner(&$crate::CN_VAL).as_value()),
|
|
61
|
-
b"JP" => Some(ruby.get_inner(&$crate::JP_VAL).as_value()),
|
|
62
|
-
b"DE" => Some(ruby.get_inner(&$crate::DE_VAL).as_value()),
|
|
63
|
-
b"IN" => Some(ruby.get_inner(&$crate::IN_VAL).as_value()),
|
|
64
|
-
b"GB" => Some(ruby.get_inner(&$crate::GB_VAL).as_value()),
|
|
65
|
-
b"FR" => Some(ruby.get_inner(&$crate::FR_VAL).as_value()),
|
|
66
|
-
b"BR" => Some(ruby.get_inner(&$crate::BR_VAL).as_value()),
|
|
67
|
-
b"IT" => Some(ruby.get_inner(&$crate::IT_VAL).as_value()),
|
|
68
|
-
b"CA" => Some(ruby.get_inner(&$crate::CA_VAL).as_value()),
|
|
69
|
-
b"RU" => Some(ruby.get_inner(&$crate::RU_VAL).as_value()),
|
|
70
|
-
b"KR" => Some(ruby.get_inner(&$crate::KR_VAL).as_value()),
|
|
71
|
-
b"AU" => Some(ruby.get_inner(&$crate::AU_VAL).as_value()),
|
|
72
|
-
b"ES" => Some(ruby.get_inner(&$crate::ES_VAL).as_value()),
|
|
73
|
-
b"MX" => Some(ruby.get_inner(&$crate::MX_VAL).as_value()),
|
|
74
|
-
b"ID" => Some(ruby.get_inner(&$crate::ID_VAL).as_value()),
|
|
75
|
-
b"TR" => Some(ruby.get_inner(&$crate::TR_VAL).as_value()),
|
|
76
|
-
_ => None,
|
|
77
|
-
},
|
|
78
|
-
4 => match key.as_bytes() {
|
|
79
|
-
b"city" => Some(ruby.get_inner(&$crate::CITY_KEY).as_value()),
|
|
80
|
-
b"code" => Some(ruby.get_inner(&$crate::CODE_KEY).as_value()),
|
|
81
|
-
_ => None,
|
|
82
|
-
},
|
|
83
|
-
5 => match key.as_bytes() {
|
|
84
|
-
b"names" => Some(ruby.get_inner(&$crate::NAMES_KEY).as_value()),
|
|
85
|
-
b"pt-BR" => Some(ruby.get_inner(&$crate::PT_BR_KEY).as_value()),
|
|
86
|
-
b"zh-CN" => Some(ruby.get_inner(&$crate::ZH_CN_KEY).as_value()),
|
|
87
|
-
_ => None,
|
|
88
|
-
},
|
|
89
|
-
6 => match key.as_bytes() {
|
|
90
|
-
b"postal" => Some(ruby.get_inner(&$crate::POSTAL_KEY).as_value()),
|
|
91
|
-
b"traits" => Some(ruby.get_inner(&$crate::TRAITS_KEY).as_value()),
|
|
92
|
-
_ => None,
|
|
93
|
-
},
|
|
94
|
-
7 => match key.as_bytes() {
|
|
95
|
-
b"country" => Some(ruby.get_inner(&$crate::COUNTRY_KEY).as_value()),
|
|
96
|
-
b"network" => Some(ruby.get_inner(&$crate::NETWORK_KEY).as_value()),
|
|
97
|
-
_ => None,
|
|
98
|
-
},
|
|
99
|
-
8 => match key.as_bytes() {
|
|
100
|
-
b"location" => Some(ruby.get_inner(&$crate::LOCATION_KEY).as_value()),
|
|
101
|
-
b"iso_code" => Some(ruby.get_inner(&$crate::ISO_CODE_KEY).as_value()),
|
|
102
|
-
b"latitude" => Some(ruby.get_inner(&$crate::LATITUDE_KEY).as_value()),
|
|
103
|
-
_ => None,
|
|
104
|
-
},
|
|
105
|
-
9 => match key.as_bytes() {
|
|
106
|
-
b"continent" => Some(ruby.get_inner(&$crate::CONTINENT_KEY).as_value()),
|
|
107
|
-
b"longitude" => Some(ruby.get_inner(&$crate::LONGITUDE_KEY).as_value()),
|
|
108
|
-
b"time_zone" => Some(ruby.get_inner(&$crate::TIME_ZONE_KEY).as_value()),
|
|
109
|
-
_ => None,
|
|
110
|
-
},
|
|
111
|
-
10 => match key.as_bytes() {
|
|
112
|
-
b"geoname_id" => Some(ruby.get_inner(&$crate::GEONAME_ID_KEY).as_value()),
|
|
113
|
-
b"metro_code" => Some(ruby.get_inner(&$crate::METRO_CODE_KEY).as_value()),
|
|
114
|
-
b"confidence" => Some(ruby.get_inner(&$crate::CONFIDENCE_KEY).as_value()),
|
|
115
|
-
_ => None,
|
|
116
|
-
},
|
|
117
|
-
12 => match key.as_bytes() {
|
|
118
|
-
b"subdivisions" => Some(ruby.get_inner(&$crate::SUBDIVISIONS_KEY).as_value()),
|
|
119
|
-
_ => None,
|
|
120
|
-
},
|
|
121
|
-
15 => match key.as_bytes() {
|
|
122
|
-
b"accuracy_radius" => Some(ruby.get_inner(&$crate::ACCURACY_RADIUS_KEY).as_value()),
|
|
123
|
-
_ => None,
|
|
124
|
-
},
|
|
125
|
-
18 => match key.as_bytes() {
|
|
126
|
-
b"registered_country" => Some(ruby.get_inner(&$crate::REGISTERED_COUNTRY_KEY).as_value()),
|
|
127
|
-
b"population_density" => Some(ruby.get_inner(&$crate::POPULATION_DENSITY_KEY).as_value()),
|
|
128
|
-
_ => None,
|
|
129
|
-
},
|
|
130
|
-
19 => match key.as_bytes() {
|
|
131
|
-
b"represented_country" => Some(ruby.get_inner(&$crate::REPRESENTED_COUNTRY_KEY).as_value()),
|
|
132
|
-
b"is_anonymous_proxy" => Some(ruby.get_inner(&$crate::IS_ANONYMOUS_PROXY_KEY).as_value()),
|
|
133
|
-
_ => None,
|
|
134
|
-
},
|
|
135
|
-
21 => match key.as_bytes() {
|
|
136
|
-
b"is_satellite_provider" => Some(ruby.get_inner(&$crate::IS_SATELLITE_PROVIDER_KEY).as_value()),
|
|
137
|
-
_ => None,
|
|
138
|
-
},
|
|
139
|
-
_ => None,
|
|
140
|
-
}
|
|
35
|
+
thread_local! {
|
|
36
|
+
static MAP_KEY_CACHE: RefCell<MapKeyCache> = RefCell::new(MapKeyCache::new());
|
|
37
|
+
}
|
|
38
|
+
|
|
39
|
+
const MAP_KEY_CACHE_MAX: usize = 256;
|
|
40
|
+
const MAP_KEY_ROOTS_CONST: &str = "__MAP_KEY_ROOTS__";
|
|
41
|
+
|
|
42
|
+
struct MapKeyCache {
|
|
43
|
+
key_to_index: FxHashMap<String, usize>,
|
|
44
|
+
}
|
|
45
|
+
|
|
46
|
+
impl MapKeyCache {
|
|
47
|
+
#[inline]
|
|
48
|
+
fn new() -> Self {
|
|
49
|
+
Self {
|
|
50
|
+
key_to_index: FxHashMap::default(),
|
|
141
51
|
}
|
|
142
|
-
}
|
|
52
|
+
}
|
|
143
53
|
}
|
|
144
54
|
|
|
145
|
-
|
|
146
|
-
|
|
147
|
-
|
|
148
|
-
|
|
149
|
-
|
|
150
|
-
|
|
151
|
-
|
|
152
|
-
|
|
153
|
-
|
|
154
|
-
|
|
155
|
-
|
|
156
|
-
|
|
157
|
-
|
|
158
|
-
|
|
159
|
-
|
|
160
|
-
|
|
161
|
-
|
|
162
|
-
|
|
163
|
-
|
|
164
|
-
|
|
165
|
-
|
|
166
|
-
|
|
167
|
-
|
|
168
|
-
|
|
169
|
-
|
|
170
|
-
|
|
171
|
-
|
|
172
|
-
|
|
173
|
-
|
|
174
|
-
|
|
175
|
-
|
|
176
|
-
|
|
177
|
-
// Continent codes
|
|
178
|
-
AF_KEY => "AF",
|
|
179
|
-
AN_KEY => "AN",
|
|
180
|
-
AS_KEY => "AS",
|
|
181
|
-
EU_KEY => "EU",
|
|
182
|
-
NA_KEY => "NA",
|
|
183
|
-
OC_KEY => "OC",
|
|
184
|
-
SA_KEY => "SA",
|
|
185
|
-
// Major Country ISO codes
|
|
186
|
-
US_VAL => "US",
|
|
187
|
-
CN_VAL => "CN",
|
|
188
|
-
JP_VAL => "JP",
|
|
189
|
-
DE_VAL => "DE",
|
|
190
|
-
IN_VAL => "IN",
|
|
191
|
-
GB_VAL => "GB",
|
|
192
|
-
FR_VAL => "FR",
|
|
193
|
-
BR_VAL => "BR",
|
|
194
|
-
IT_VAL => "IT",
|
|
195
|
-
CA_VAL => "CA",
|
|
196
|
-
RU_VAL => "RU", // Already defined above as RU_KEY? No, RU_KEY is "ru" (lang), this is "RU" (country)
|
|
197
|
-
KR_VAL => "KR",
|
|
198
|
-
AU_VAL => "AU",
|
|
199
|
-
ES_VAL => "ES", // "ES" (country) vs "es" (lang). ES_KEY is "es".
|
|
200
|
-
MX_VAL => "MX",
|
|
201
|
-
ID_VAL => "ID",
|
|
202
|
-
TR_VAL => "TR",
|
|
203
|
-
);
|
|
55
|
+
#[inline]
|
|
56
|
+
fn map_key_roots_array(ruby: &magnus::Ruby) -> RArray {
|
|
57
|
+
let rust = rust_module(ruby);
|
|
58
|
+
let roots = rust
|
|
59
|
+
.const_get::<_, Value>(MAP_KEY_ROOTS_CONST)
|
|
60
|
+
.expect("map key roots constant should exist");
|
|
61
|
+
RArray::from_value(roots).expect("map key roots constant should be an array")
|
|
62
|
+
}
|
|
63
|
+
|
|
64
|
+
#[inline]
|
|
65
|
+
fn cached_map_key(ruby: &magnus::Ruby, key: &str) -> Value {
|
|
66
|
+
MAP_KEY_CACHE.with(|cache| {
|
|
67
|
+
let mut cache = cache.borrow_mut();
|
|
68
|
+
let roots = map_key_roots_array(ruby);
|
|
69
|
+
if let Some(index) = cache.key_to_index.get(key) {
|
|
70
|
+
return roots
|
|
71
|
+
.entry::<Value>(*index as isize)
|
|
72
|
+
.expect("cached map key index should be valid");
|
|
73
|
+
}
|
|
74
|
+
|
|
75
|
+
let interned = ruby.str_new(key).to_interned_str();
|
|
76
|
+
let value = interned.as_value();
|
|
77
|
+
if cache.key_to_index.len() < MAP_KEY_CACHE_MAX {
|
|
78
|
+
let index = roots.len();
|
|
79
|
+
roots
|
|
80
|
+
.push(value)
|
|
81
|
+
.expect("map key roots array push should succeed");
|
|
82
|
+
cache.key_to_index.insert(key.to_owned(), index);
|
|
83
|
+
}
|
|
84
|
+
value
|
|
85
|
+
})
|
|
86
|
+
}
|
|
204
87
|
|
|
205
88
|
/// Wrapper that owns the Ruby value produced by deserializing a MaxMind record
|
|
206
89
|
#[derive(Clone)]
|
|
@@ -331,18 +214,18 @@ impl<'de, 'ruby> Visitor<'de> for RubyValueVisitor<'ruby> {
|
|
|
331
214
|
where
|
|
332
215
|
E: de::Error,
|
|
333
216
|
{
|
|
334
|
-
|
|
335
|
-
|
|
336
|
-
|
|
217
|
+
Ok(RubyDecodedValue::new(
|
|
218
|
+
self.ruby.str_new(value).into_value_with(self.ruby),
|
|
219
|
+
))
|
|
337
220
|
}
|
|
338
221
|
|
|
339
222
|
fn visit_string<E>(self, value: String) -> Result<Self::Value, E>
|
|
340
223
|
where
|
|
341
224
|
E: de::Error,
|
|
342
225
|
{
|
|
343
|
-
|
|
344
|
-
|
|
345
|
-
|
|
226
|
+
Ok(RubyDecodedValue::new(
|
|
227
|
+
self.ruby.str_new(&value).into_value_with(self.ruby),
|
|
228
|
+
))
|
|
346
229
|
}
|
|
347
230
|
|
|
348
231
|
fn visit_bytes<E>(self, value: &[u8]) -> Result<Self::Value, E>
|
|
@@ -386,10 +269,9 @@ impl<'de, 'ruby> Visitor<'de> for RubyValueVisitor<'ruby> {
|
|
|
386
269
|
Some(cap) => self.ruby.hash_new_capa(cap),
|
|
387
270
|
None => self.ruby.hash_new(),
|
|
388
271
|
};
|
|
389
|
-
while let Some(key) = map.next_key
|
|
272
|
+
while let Some(key) = map.next_key::<&'de str>()? {
|
|
273
|
+
let key_val = cached_map_key(self.ruby, key);
|
|
390
274
|
let value = map.next_value_seed(RubyValueSeed { ruby: self.ruby })?;
|
|
391
|
-
let key_val = interned_key(self.ruby, key.as_ref())
|
|
392
|
-
.unwrap_or_else(|| self.ruby.str_new(key.as_ref()).into_value_with(self.ruby));
|
|
393
275
|
hash.aset(key_val, value.into_value())
|
|
394
276
|
.map_err(|e| de::Error::custom(e.to_string()))?;
|
|
395
277
|
}
|
|
@@ -420,35 +302,10 @@ impl ReaderSource {
|
|
|
420
302
|
&self,
|
|
421
303
|
ip: IpAddr,
|
|
422
304
|
) -> Result<(Option<RubyDecodedValue>, usize), maxminddb_crate::MaxMindDbError> {
|
|
423
|
-
|
|
424
|
-
ReaderSource::Mmap(reader) =>
|
|
425
|
-
|
|
426
|
-
|
|
427
|
-
let prefix = network.prefix();
|
|
428
|
-
|
|
429
|
-
let prefix_len = if ip.is_ipv4() && network.is_ipv6() {
|
|
430
|
-
0
|
|
431
|
-
} else {
|
|
432
|
-
prefix as usize
|
|
433
|
-
};
|
|
434
|
-
|
|
435
|
-
(result.decode()?, prefix_len)
|
|
436
|
-
}
|
|
437
|
-
ReaderSource::Memory(reader) => {
|
|
438
|
-
let result = reader.lookup(ip)?;
|
|
439
|
-
let network = result.network()?;
|
|
440
|
-
let prefix = network.prefix();
|
|
441
|
-
|
|
442
|
-
let prefix_len = if ip.is_ipv4() && network.is_ipv6() {
|
|
443
|
-
0
|
|
444
|
-
} else {
|
|
445
|
-
prefix as usize
|
|
446
|
-
};
|
|
447
|
-
|
|
448
|
-
(result.decode()?, prefix_len)
|
|
449
|
-
}
|
|
450
|
-
};
|
|
451
|
-
Ok((result, prefix_len))
|
|
305
|
+
match self {
|
|
306
|
+
ReaderSource::Mmap(reader) => lookup_prefix_for_reader(reader, ip),
|
|
307
|
+
ReaderSource::Memory(reader) => lookup_prefix_for_reader(reader, ip),
|
|
308
|
+
}
|
|
452
309
|
}
|
|
453
310
|
|
|
454
311
|
#[inline]
|
|
@@ -490,40 +347,54 @@ enum ReaderWithin {
|
|
|
490
347
|
impl ReaderWithin {
|
|
491
348
|
fn next(&mut self) -> Option<Result<(IpNetwork, RubyDecodedValue), MaxMindDbError>> {
|
|
492
349
|
match self {
|
|
493
|
-
ReaderWithin::Mmap(iter) =>
|
|
494
|
-
|
|
495
|
-
|
|
496
|
-
|
|
497
|
-
|
|
498
|
-
|
|
499
|
-
|
|
500
|
-
|
|
501
|
-
|
|
502
|
-
|
|
503
|
-
|
|
504
|
-
|
|
505
|
-
|
|
506
|
-
|
|
507
|
-
|
|
508
|
-
|
|
509
|
-
|
|
510
|
-
|
|
511
|
-
|
|
512
|
-
|
|
513
|
-
|
|
514
|
-
|
|
515
|
-
|
|
516
|
-
|
|
517
|
-
|
|
518
|
-
|
|
519
|
-
|
|
520
|
-
|
|
521
|
-
|
|
522
|
-
|
|
523
|
-
|
|
524
|
-
|
|
350
|
+
ReaderWithin::Mmap(iter) => next_within_result(iter),
|
|
351
|
+
ReaderWithin::Memory(iter) => next_within_result(iter),
|
|
352
|
+
}
|
|
353
|
+
}
|
|
354
|
+
}
|
|
355
|
+
|
|
356
|
+
#[inline]
|
|
357
|
+
fn lookup_prefix_for_reader<S: AsRef<[u8]>>(
|
|
358
|
+
reader: &MaxMindReader<S>,
|
|
359
|
+
ip: IpAddr,
|
|
360
|
+
) -> Result<(Option<RubyDecodedValue>, usize), maxminddb_crate::MaxMindDbError> {
|
|
361
|
+
let result = reader.lookup(ip)?;
|
|
362
|
+
let network = result.network()?;
|
|
363
|
+
let prefix_len = prefix_len_for_ip_network(ip, network);
|
|
364
|
+
Ok((result.decode()?, prefix_len))
|
|
365
|
+
}
|
|
366
|
+
|
|
367
|
+
#[inline]
|
|
368
|
+
// prefix_len_for_ip_network uses 0 as a sentinel for ip.is_ipv4() && network.is_ipv6().
|
|
369
|
+
// In this case, 0 is not a real prefix length; it signals an IPv4-in-IPv6 mapping path,
|
|
370
|
+
// and callers must treat it specially (distinct from "no network found").
|
|
371
|
+
fn prefix_len_for_ip_network(ip: IpAddr, network: IpNetwork) -> usize {
|
|
372
|
+
if ip.is_ipv4() && network.is_ipv6() {
|
|
373
|
+
0
|
|
374
|
+
} else {
|
|
375
|
+
network.prefix() as usize
|
|
376
|
+
}
|
|
377
|
+
}
|
|
378
|
+
|
|
379
|
+
#[inline]
|
|
380
|
+
fn next_within_result<S: AsRef<[u8]>>(
|
|
381
|
+
iter: &mut Within<'static, S>,
|
|
382
|
+
) -> Option<Result<(IpNetwork, RubyDecodedValue), MaxMindDbError>> {
|
|
383
|
+
loop {
|
|
384
|
+
match iter.next() {
|
|
385
|
+
None => return None,
|
|
386
|
+
Some(Err(e)) => return Some(Err(e)),
|
|
387
|
+
Some(Ok(lookup_result)) => {
|
|
388
|
+
let network = match lookup_result.network() {
|
|
389
|
+
Ok(n) => n,
|
|
390
|
+
Err(e) => return Some(Err(e)),
|
|
391
|
+
};
|
|
392
|
+
match lookup_result.decode::<RubyDecodedValue>() {
|
|
393
|
+
Ok(Some(data)) => return Some(Ok((network, data))),
|
|
394
|
+
Ok(None) => continue, // Skip networks without data
|
|
395
|
+
Err(e) => return Some(Err(e)),
|
|
525
396
|
}
|
|
526
|
-
}
|
|
397
|
+
}
|
|
527
398
|
}
|
|
528
399
|
}
|
|
529
400
|
}
|
|
@@ -935,6 +806,32 @@ fn parse_ip_address_fast(value: Value, ruby: &magnus::Ruby) -> Result<IpAddr, Er
|
|
|
935
806
|
}
|
|
936
807
|
|
|
937
808
|
// Slow path: Try as IPAddr object
|
|
809
|
+
if let Ok(ipaddr_class) = ruby.class_object().const_get::<_, RClass>("IPAddr") {
|
|
810
|
+
if value.is_kind_of(ipaddr_class) {
|
|
811
|
+
let packed: Value = value.funcall("hton", ())?;
|
|
812
|
+
if let Some(packed_str) = RString::from_value(packed) {
|
|
813
|
+
// SAFETY: `bytes` is used immediately and `packed`/`packed_str` stay alive and
|
|
814
|
+
// unmodified through the end of this match. This block must not introduce calls
|
|
815
|
+
// that could move, collect, or mutate the Ruby string between `as_slice()` and
|
|
816
|
+
// the final byte-pattern match handling.
|
|
817
|
+
let bytes = unsafe { packed_str.as_slice() };
|
|
818
|
+
return match bytes {
|
|
819
|
+
[a, b, c, d] => Ok(IpAddr::from([*a, *b, *c, *d])),
|
|
820
|
+
[a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, a13, a14, a15] => {
|
|
821
|
+
Ok(IpAddr::from([
|
|
822
|
+
*a0, *a1, *a2, *a3, *a4, *a5, *a6, *a7, *a8, *a9, *a10, *a11, *a12,
|
|
823
|
+
*a13, *a14, *a15,
|
|
824
|
+
]))
|
|
825
|
+
}
|
|
826
|
+
_ => Err(Error::new(
|
|
827
|
+
ruby.exception_arg_error(),
|
|
828
|
+
format!("'{}' does not appear to be an IPv4 or IPv6 address", value),
|
|
829
|
+
)),
|
|
830
|
+
};
|
|
831
|
+
}
|
|
832
|
+
}
|
|
833
|
+
}
|
|
834
|
+
|
|
938
835
|
if let Ok(ipaddr_obj) = value.funcall::<_, _, String>("to_s", ()) {
|
|
939
836
|
return IpAddr::from_str(&ipaddr_obj).map_err(|_| {
|
|
940
837
|
Error::new(
|
|
@@ -964,24 +861,7 @@ fn ipv6_in_ipv4_error(ip: &IpAddr) -> String {
|
|
|
964
861
|
/// Open a MaxMind DB using memory-mapped I/O (MODE_MMAP)
|
|
965
862
|
fn open_database_mmap(path: &str) -> Result<Reader, Error> {
|
|
966
863
|
let ruby = magnus::Ruby::get().expect("Ruby VM should be available in Ruby context");
|
|
967
|
-
|
|
968
|
-
let file = File::open(Path::new(path)).map_err(|e| match e.kind() {
|
|
969
|
-
std::io::ErrorKind::NotFound => {
|
|
970
|
-
let errno = ruby
|
|
971
|
-
.class_object()
|
|
972
|
-
.const_get::<_, RModule>("Errno")
|
|
973
|
-
.expect("Errno module should exist");
|
|
974
|
-
let enoent = errno
|
|
975
|
-
.const_get::<_, RClass>("ENOENT")
|
|
976
|
-
.expect("Errno::ENOENT should exist");
|
|
977
|
-
Error::new(
|
|
978
|
-
ExceptionClass::from_value(enoent.as_value())
|
|
979
|
-
.expect("ENOENT should convert to ExceptionClass"),
|
|
980
|
-
e.to_string(),
|
|
981
|
-
)
|
|
982
|
-
}
|
|
983
|
-
_ => Error::new(ruby.exception_io_error(), e.to_string()),
|
|
984
|
-
})?;
|
|
864
|
+
let file = open_database_file(path, &ruby)?;
|
|
985
865
|
|
|
986
866
|
let mmap = unsafe { Mmap::map(&file) }.map_err(|e| {
|
|
987
867
|
Error::new(
|
|
@@ -1007,24 +887,7 @@ fn open_database_mmap(path: &str) -> Result<Reader, Error> {
|
|
|
1007
887
|
/// Open a MaxMind DB by loading entire file into memory (MODE_MEMORY)
|
|
1008
888
|
fn open_database_memory(path: &str) -> Result<Reader, Error> {
|
|
1009
889
|
let ruby = magnus::Ruby::get().expect("Ruby VM should be available in Ruby context");
|
|
1010
|
-
|
|
1011
|
-
let mut file = File::open(Path::new(path)).map_err(|e| match e.kind() {
|
|
1012
|
-
std::io::ErrorKind::NotFound => {
|
|
1013
|
-
let errno = ruby
|
|
1014
|
-
.class_object()
|
|
1015
|
-
.const_get::<_, RModule>("Errno")
|
|
1016
|
-
.expect("Errno module should exist");
|
|
1017
|
-
let enoent = errno
|
|
1018
|
-
.const_get::<_, RClass>("ENOENT")
|
|
1019
|
-
.expect("Errno::ENOENT should exist");
|
|
1020
|
-
Error::new(
|
|
1021
|
-
ExceptionClass::from_value(enoent.as_value())
|
|
1022
|
-
.expect("ENOENT should convert to ExceptionClass"),
|
|
1023
|
-
e.to_string(),
|
|
1024
|
-
)
|
|
1025
|
-
}
|
|
1026
|
-
_ => Error::new(ruby.exception_io_error(), e.to_string()),
|
|
1027
|
-
})?;
|
|
890
|
+
let mut file = open_database_file(path, &ruby)?;
|
|
1028
891
|
|
|
1029
892
|
let mut buffer = Vec::new();
|
|
1030
893
|
file.read_to_end(&mut buffer).map_err(|e| {
|
|
@@ -1048,21 +911,51 @@ fn open_database_memory(path: &str) -> Result<Reader, Error> {
|
|
|
1048
911
|
Ok(create_reader(ReaderSource::Memory(reader)))
|
|
1049
912
|
}
|
|
1050
913
|
|
|
914
|
+
fn open_database_file(path: &str, ruby: &magnus::Ruby) -> Result<File, Error> {
|
|
915
|
+
File::open(Path::new(path)).map_err(|e| {
|
|
916
|
+
if e.kind() == std::io::ErrorKind::NotFound {
|
|
917
|
+
open_not_found_error(ruby, e)
|
|
918
|
+
} else {
|
|
919
|
+
Error::new(ruby.exception_io_error(), e.to_string())
|
|
920
|
+
}
|
|
921
|
+
})
|
|
922
|
+
}
|
|
923
|
+
|
|
924
|
+
fn open_not_found_error(ruby: &magnus::Ruby, err: std::io::Error) -> Error {
|
|
925
|
+
let errno = ruby
|
|
926
|
+
.class_object()
|
|
927
|
+
.const_get::<_, RModule>("Errno")
|
|
928
|
+
.expect("Errno module should exist");
|
|
929
|
+
let enoent = errno
|
|
930
|
+
.const_get::<_, RClass>("ENOENT")
|
|
931
|
+
.expect("Errno::ENOENT should exist");
|
|
932
|
+
Error::new(
|
|
933
|
+
ExceptionClass::from_value(enoent.as_value())
|
|
934
|
+
.expect("ENOENT should convert to ExceptionClass"),
|
|
935
|
+
err.to_string(),
|
|
936
|
+
)
|
|
937
|
+
}
|
|
938
|
+
|
|
1051
939
|
/// Get the InvalidDatabaseError class
|
|
1052
940
|
fn invalid_database_error() -> RClass {
|
|
1053
941
|
let ruby = magnus::Ruby::get().expect("Ruby VM should be available in Ruby context");
|
|
942
|
+
let rust = rust_module(&ruby);
|
|
943
|
+
rust.const_get::<_, RClass>("InvalidDatabaseError")
|
|
944
|
+
.expect("InvalidDatabaseError class should exist")
|
|
945
|
+
}
|
|
946
|
+
|
|
947
|
+
fn rust_module(ruby: &magnus::Ruby) -> RModule {
|
|
1054
948
|
let maxmind = ruby
|
|
1055
949
|
.class_object()
|
|
1056
950
|
.const_get::<_, RModule>("MaxMind")
|
|
1057
951
|
.expect("MaxMind module should exist");
|
|
1058
952
|
let db = maxmind
|
|
1059
|
-
.const_get::<_,
|
|
1060
|
-
.expect("MaxMind::DB
|
|
1061
|
-
let
|
|
1062
|
-
.
|
|
1063
|
-
.expect("MaxMind::DB::Rust
|
|
1064
|
-
|
|
1065
|
-
.expect("InvalidDatabaseError class should exist")
|
|
953
|
+
.const_get::<_, Value>("DB")
|
|
954
|
+
.expect("MaxMind::DB constant should exist");
|
|
955
|
+
let rust_value = db
|
|
956
|
+
.funcall::<_, _, Value>("const_get", ("Rust",))
|
|
957
|
+
.expect("MaxMind::DB::Rust constant should exist");
|
|
958
|
+
RModule::from_value(rust_value).expect("MaxMind::DB::Rust should be a module")
|
|
1066
959
|
}
|
|
1067
960
|
|
|
1068
961
|
#[magnus::init]
|
|
@@ -1076,11 +969,26 @@ fn init(ruby: &magnus::Ruby) -> Result<(), Error> {
|
|
|
1076
969
|
let rust = match db_value {
|
|
1077
970
|
Ok(existing) if existing.is_kind_of(ruby.class_class()) => {
|
|
1078
971
|
// MaxMind::DB exists as a Class (official gem loaded first)
|
|
1079
|
-
//
|
|
1080
|
-
let
|
|
1081
|
-
|
|
1082
|
-
|
|
1083
|
-
|
|
972
|
+
// Reuse existing Rust constant if present to avoid replacing classes.
|
|
973
|
+
if let Ok(rust_value) = existing.funcall::<_, _, Value>("const_get", ("Rust", false)) {
|
|
974
|
+
RModule::from_value(rust_value).ok_or_else(|| {
|
|
975
|
+
Error::new(
|
|
976
|
+
ruby.exception_type_error(),
|
|
977
|
+
"MaxMind::DB::Rust exists but is not a module",
|
|
978
|
+
)
|
|
979
|
+
})?
|
|
980
|
+
} else {
|
|
981
|
+
// Define Rust module directly as a constant on the class.
|
|
982
|
+
let rust_value: Value = ruby.module_new().as_value();
|
|
983
|
+
let rust_mod = RModule::from_value(rust_value).ok_or_else(|| {
|
|
984
|
+
Error::new(
|
|
985
|
+
ruby.exception_type_error(),
|
|
986
|
+
"Failed to create anonymous module for MaxMind::DB::Rust",
|
|
987
|
+
)
|
|
988
|
+
})?;
|
|
989
|
+
let _ = existing.funcall::<_, _, Value>("const_set", ("Rust", rust_mod))?;
|
|
990
|
+
rust_mod
|
|
991
|
+
}
|
|
1084
992
|
}
|
|
1085
993
|
Ok(existing) => {
|
|
1086
994
|
// MaxMind::DB exists as a Module (our gem loaded first)
|
|
@@ -1096,6 +1004,15 @@ fn init(ruby: &magnus::Ruby) -> Result<(), Error> {
|
|
|
1096
1004
|
}
|
|
1097
1005
|
};
|
|
1098
1006
|
|
|
1007
|
+
if rust.const_get::<_, Value>(MAP_KEY_ROOTS_CONST).is_err() {
|
|
1008
|
+
rust.const_set(MAP_KEY_ROOTS_CONST, ruby.ary_new_capa(MAP_KEY_CACHE_MAX))?;
|
|
1009
|
+
}
|
|
1010
|
+
|
|
1011
|
+
// The extension can be loaded more than once from different paths.
|
|
1012
|
+
// Reusing previously defined classes avoids typed-data incompatibilities.
|
|
1013
|
+
if rust.const_get::<_, Value>("Reader").is_ok() {
|
|
1014
|
+
return Ok(());
|
|
1015
|
+
}
|
|
1099
1016
|
// Define InvalidDatabaseError
|
|
1100
1017
|
let runtime_error = ruby.exception_runtime_error();
|
|
1101
1018
|
rust.define_error("InvalidDatabaseError", runtime_error)?;
|
metadata
CHANGED
|
@@ -1,14 +1,13 @@
|
|
|
1
1
|
--- !ruby/object:Gem::Specification
|
|
2
2
|
name: maxmind-db-rust
|
|
3
3
|
version: !ruby/object:Gem::Version
|
|
4
|
-
version: 0.
|
|
4
|
+
version: 0.3.0
|
|
5
5
|
platform: ruby
|
|
6
6
|
authors:
|
|
7
7
|
- Gregory Oschwald
|
|
8
|
-
autorequire:
|
|
9
8
|
bindir: bin
|
|
10
9
|
cert_chain: []
|
|
11
|
-
date:
|
|
10
|
+
date: 1980-01-02 00:00:00.000000000 Z
|
|
12
11
|
dependencies:
|
|
13
12
|
- !ruby/object:Gem::Dependency
|
|
14
13
|
name: rb_sys
|
|
@@ -30,14 +29,14 @@ dependencies:
|
|
|
30
29
|
requirements:
|
|
31
30
|
- - "~>"
|
|
32
31
|
- !ruby/object:Gem::Version
|
|
33
|
-
version: '
|
|
32
|
+
version: '6.0'
|
|
34
33
|
type: :development
|
|
35
34
|
prerelease: false
|
|
36
35
|
version_requirements: !ruby/object:Gem::Requirement
|
|
37
36
|
requirements:
|
|
38
37
|
- - "~>"
|
|
39
38
|
- !ruby/object:Gem::Version
|
|
40
|
-
version: '
|
|
39
|
+
version: '6.0'
|
|
41
40
|
- !ruby/object:Gem::Dependency
|
|
42
41
|
name: rake
|
|
43
42
|
requirement: !ruby/object:Gem::Requirement
|
|
@@ -178,7 +177,6 @@ metadata:
|
|
|
178
177
|
homepage_uri: https://github.com/oschwald/maxmind-db-rust-ruby
|
|
179
178
|
source_code_uri: https://github.com/oschwald/maxmind-db-rust-ruby
|
|
180
179
|
rubygems_mfa_required: 'true'
|
|
181
|
-
post_install_message:
|
|
182
180
|
rdoc_options: []
|
|
183
181
|
require_paths:
|
|
184
182
|
- lib
|
|
@@ -193,8 +191,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
|
|
|
193
191
|
- !ruby/object:Gem::Version
|
|
194
192
|
version: '0'
|
|
195
193
|
requirements: []
|
|
196
|
-
rubygems_version:
|
|
197
|
-
signing_key:
|
|
194
|
+
rubygems_version: 4.0.3
|
|
198
195
|
specification_version: 4
|
|
199
196
|
summary: Unofficial high-performance Rust-based MaxMind DB reader for Ruby
|
|
200
197
|
test_files: []
|