ukemi 0.1.0 → 0.4.1

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a77579f014e97cc048e95dc5488e228ee19baba62d5074dbc9bfcb5c75f9d568
4
- data.tar.gz: e156b9c6de521ea49d7a69725c724176a5d1739deee7bba32aef4e662c64ddae
3
+ metadata.gz: a512ff8b644d793535d9beac64ab550b5628bbb86fd042e2cc3ab5b2cdad1ea1
4
+ data.tar.gz: d7433d7e80ff7723bf097db0bbda4a35f3feab7d2fbd4d3d7726c1ff3319c3a0
5
5
  SHA512:
6
- metadata.gz: f4e3eb6f8f4dd2223ba2eed1846ff8d95ee8d14913f5f6e682269489ee9348fea3af858c9bf849bac4d612a42a0a3c1c7bae3b27dde3461b1104c3e0e3752964
7
- data.tar.gz: 34467c49986dc11f2ac4fae646c1d96419ebe199838fe20a8f859c1619137ce2f3d0bcfc11e1f356545e14f63a6b5ff81b7ad9f39b206b70469959a8ca9a5986
6
+ metadata.gz: 9e88eefa02c8071b840e16c25ecdd52198abdd16ede41ff657692b9fc6ff2c9b21e0b8130d6562cc348eb8b101d91b0309a7feb4b5a48cb3283968518e649d1f
7
+ data.tar.gz: 6cca209ba2a055a195e862b0f10b2e431d8555d32985050b5debc6e4d20164c640af1afa30960ebde6842bc109b4e8176be206303ee0f170de5d441436339a45
@@ -0,0 +1,27 @@
1
+ name: Ruby CI
2
+
3
+ on: [pull_request]
4
+
5
+ jobs:
6
+ build:
7
+
8
+ runs-on: ubuntu-latest
9
+
10
+ strategy:
11
+ fail-fast: false
12
+ matrix:
13
+ ruby: [2.7, '3.0']
14
+
15
+ steps:
16
+ - uses: actions/checkout@v2
17
+ - name: Set up Ruby
18
+ uses: ruby/setup-ruby@v1
19
+ with:
20
+ ruby-version: ${{ matrix.ruby }}
21
+ bundler-cache: true
22
+
23
+ - name: Build and test with Rake
24
+ run: |
25
+ gem install bundler
26
+ bundle install
27
+ bundle exec rake
data/.overcommit.yml ADDED
@@ -0,0 +1,9 @@
1
+ PreCommit:
2
+ BundleCheck:
3
+ enabled: true
4
+
5
+ RuboCop:
6
+ enabled: true
7
+ required_executable: bundle
8
+ command: ["bundle", "exec", "standardrb"]
9
+ on_warn: fail
data/.standard.yml ADDED
@@ -0,0 +1,4 @@
1
+ ignore:
2
+ - "**/*":
3
+ - Layout/SpaceInsideHashLiteralBraces
4
+ - Style/RescueStandardError
data/README.md CHANGED
@@ -1,6 +1,7 @@
1
1
  # ukemi
2
2
 
3
- [![Build Status](https://travis-ci.com/ninoseki/ukemi.svg?branch=master)](https://travis-ci.com/ninoseki/ukemi)
3
+ [![Gem Version](https://badge.fury.io/rb/ukemi.svg)](https://badge.fury.io/rb/ukemi)
4
+ [![Ruby CI](https://github.com/ninoseki/ukemi/actions/workflows/test.yaml/badge.svg)](https://github.com/ninoseki/ukemi/actions/workflows/test.yaml)
4
5
  [![Coverage Status](https://coveralls.io/repos/github/ninoseki/ukemi/badge.svg?branch=master)](https://coveralls.io/github/ninoseki/ukemi?branch=master)
5
6
  [![CodeFactor](https://www.codefactor.io/repository/github/ninoseki/ukemi/badge)](https://www.codefactor.io/repository/github/ninoseki/ukemi)
6
7
 
@@ -9,13 +10,19 @@ Ukemi is a CIL tool for querying passive DNS services.
9
10
  It supports the following services.
10
11
 
11
12
  - [CIRCL passive DNS](https://www.circl.lu/services/passive-dns/)
13
+ - [DNSDB](https://api.dnsdb.info/)
14
+ - [OTX](https://otx.alienvault.com)
12
15
  - [PassiveTotal](https://community.riskiq.com/)
13
16
  - [SecurityTrails](https://securitytrails.com/)
14
17
  - [VirusTotal](http://virustotal.com)
15
18
 
16
19
  It outputs passive DNS resolutions as JSON.
17
20
 
18
- ## Instalattion
21
+ ## Requirements
22
+
23
+ - Ruby 2.7+ or Ruby 3.x
24
+
25
+ ## Installation
19
26
 
20
27
  ```bash
21
28
  gem install ukemi
@@ -29,6 +36,8 @@ Configuration is done via environment variables.
29
36
  |------------------------|----------------------------|
30
37
  | CIRCL_PASSIVE_PASSWORD | CIRCL passive DNS password |
31
38
  | CIRCL_PASSIVE_USERNAME | CIRCL passive DNS username |
39
+ | DNSDB_API_KEY | DNSDB API key |
40
+ | OTX_API_KEY | OTX API key |
32
41
  | PASSIVETOTAL_API_KEY | PassiveTotal API key |
33
42
  | PASSIVETOTAL_USERNAME | PassiveTotal username |
34
43
  | SECURITYTRAILS_API_KEY | SecurityTrails API key |
@@ -42,379 +51,146 @@ Commands:
42
51
  ukemi help [COMMAND] # Describe available commands or one specific command
43
52
  ukemi lookup [IP|DOMAIN] # Lookup passive DNS services
44
53
 
45
- $ ukemi help looup
54
+ $ ukemi help lookup
46
55
  Usage:
47
56
  ukemi lookup [IP|DOMAIN]
48
57
 
49
- Lookup passive DNS services
58
+ Options:
59
+ [--order-by=ORDER_BY] # Ordering of the passve DNS resolutions (last_seen or first_seen)
60
+ # Default: -last_seen
61
+
62
+ Lookup passive DNS servicess
50
63
  ```
51
64
 
52
65
  ```bash
53
- $ ukemi lookup circl.lu
66
+ $ ukemi lookup example.com
54
67
  {
55
- "149.13.33.14": [
56
- {
57
- "firtst_seen": "2016-10-07",
58
- "last_seen": "2018-10-26",
59
- "source": "CIRCL"
60
- },
61
- {
62
- "firtst_seen": "2017-05-26",
63
- "last_seen": "2020-03-15",
64
- "source": "SecurityTrails"
65
- },
66
- {
67
- "firtst_seen": "2019-12-04",
68
- "last_seen": "2019-12-04",
69
- "source": "VirusTotal"
70
- }
71
- ],
72
- "149.13.33.4": [
73
- {
74
- "firtst_seen": "2011-03-08",
75
- "last_seen": "2012-02-13",
76
- "source": "CIRCL"
77
- },
78
- {
79
- "firtst_seen": "2013-07-30",
80
- "last_seen": "2013-07-30",
81
- "source": "VirusTotal"
82
- }
83
- ],
84
- "194.154.205.24": [
85
- {
86
- "firtst_seen": "2011-03-03",
87
- "last_seen": "2011-03-03",
88
- "source": "CIRCL"
89
- }
90
- ]
68
+ "93.184.216.34": {
69
+ "first_seen": "2016-03-01",
70
+ "last_seen": "2020-03-16",
71
+ "sources": [
72
+ {
73
+ "first_seen": "2016-10-07",
74
+ "last_seen": "2018-10-30",
75
+ "source": "CIRCL"
76
+ },
77
+ {
78
+ "first_seen": "2016-03-01",
79
+ "last_seen": "2020-03-16",
80
+ "source": "SecurityTrails"
81
+ },
82
+ {
83
+ "first_seen": "2020-03-03",
84
+ "last_seen": "2020-03-03",
85
+ "source": "VirusTotal"
86
+ }
87
+ ]
88
+ },
89
+ ...
91
90
  }
92
91
 
93
92
  $ ukemi lookup 195.123.226.243
94
93
  {
95
- "liankt.club": [
96
- {
97
- "firtst_seen": "2020-02-15",
98
- "last_seen": "2020-03-13",
99
- "source": "PassiveTotal"
100
- },
101
- {
102
- "firtst_seen": "2020-02-16",
103
- "last_seen": "2020-02-16",
104
- "source": "VirusTotal"
105
- }
106
- ],
107
- "weidt.club": [
108
- {
109
- "firtst_seen": "2020-03-12",
110
- "last_seen": "2020-03-12",
111
- "source": "PassiveTotal"
112
- }
113
- ],
114
- "jikt.club": [
115
- {
116
- "firtst_seen": "2020-03-04",
117
- "last_seen": "2020-03-12",
118
- "source": "PassiveTotal"
119
- },
120
- {
121
- "firtst_seen": "2020-03-05",
122
- "last_seen": "2020-03-05",
123
- "source": "VirusTotal"
124
- }
125
- ],
126
- "biesi.club": [
127
- {
128
- "firtst_seen": "2020-02-15",
129
- "last_seen": "2020-03-12",
130
- "source": "PassiveTotal"
131
- },
132
- {
133
- "firtst_seen": "2020-02-20",
134
- "last_seen": "2020-02-20",
135
- "source": "VirusTotal"
136
- }
137
- ],
138
- "kaikt.club": [
139
- {
140
- "firtst_seen": "2020-02-15",
141
- "last_seen": "2020-03-12",
142
- "source": "PassiveTotal"
143
- },
144
- {
145
- "firtst_seen": "2020-02-21",
146
- "last_seen": "2020-02-21",
147
- "source": "VirusTotal"
148
- }
149
- ],
150
- "zhaokt.club": [
151
- {
152
- "firtst_seen": "2020-02-15",
153
- "last_seen": "2020-03-11",
154
- "source": "PassiveTotal"
155
- },
156
- {
157
- "firtst_seen": "2020-02-18",
158
- "last_seen": "2020-02-18",
159
- "source": "VirusTotal"
160
- }
161
- ],
162
- "yangdt.club": [
163
- {
164
- "firtst_seen": "2020-02-26",
165
- "last_seen": "2020-03-10",
166
- "source": "PassiveTotal"
167
- },
168
- {
169
- "firtst_seen": "2020-02-27",
170
- "last_seen": "2020-02-27",
171
- "source": "VirusTotal"
172
- }
173
- ],
174
- "jinkt.club": [
175
- {
176
- "firtst_seen": "2020-02-21",
177
- "last_seen": "2020-03-10",
178
- "source": "PassiveTotal"
179
- },
180
- {
181
- "firtst_seen": "2020-02-22",
182
- "last_seen": "2020-02-22",
183
- "source": "VirusTotal"
184
- }
185
- ],
186
- "taokt.club": [
187
- {
188
- "firtst_seen": "2020-03-10",
189
- "last_seen": "2020-03-10",
190
- "source": "PassiveTotal"
191
- }
192
- ],
193
- "xinkt.club": [
194
- {
195
- "firtst_seen": "2020-02-17",
196
- "last_seen": "2020-03-09",
197
- "source": "PassiveTotal"
198
- },
199
- {
200
- "firtst_seen": "2020-02-19",
201
- "last_seen": "2020-02-19",
202
- "source": "VirusTotal"
203
- }
204
- ],
205
- "mail.realty-advertising.ru": [
206
- {
207
- "firtst_seen": "2019-11-08",
208
- "last_seen": "2020-03-09",
209
- "source": "PassiveTotal"
210
- }
211
- ],
212
- "realty-advertising.ru": [
213
- {
214
- "firtst_seen": "2019-11-08",
215
- "last_seen": "2020-03-06",
216
- "source": "PassiveTotal"
217
- }
218
- ],
219
- "ns1.realty-advertising.ru": [
220
- {
221
- "firtst_seen": "2019-12-02",
222
- "last_seen": "2020-03-04",
223
- "source": "PassiveTotal"
224
- }
225
- ],
226
- "ns2.realty-advertising.ru": [
227
- {
228
- "firtst_seen": "2019-12-04",
229
- "last_seen": "2020-03-04",
230
- "source": "PassiveTotal"
231
- }
232
- ],
233
- "xiankt.club": [
234
- {
235
- "firtst_seen": "2020-02-15",
236
- "last_seen": "2020-03-03",
237
- "source": "PassiveTotal"
238
- },
239
- {
240
- "firtst_seen": "2020-02-16",
241
- "last_seen": "2020-02-16",
242
- "source": "VirusTotal"
243
- }
244
- ],
245
- "nittsu-si.com": [
246
- {
247
- "firtst_seen": "2020-02-15",
248
- "last_seen": "2020-03-03",
249
- "source": "PassiveTotal"
250
- },
251
- {
252
- "firtst_seen": "2020-02-21",
253
- "last_seen": "2020-02-21",
254
- "source": "VirusTotal"
255
- }
256
- ],
257
- "mailer.realty-advertising.ru": [
258
- {
259
- "firtst_seen": "2020-02-23",
260
- "last_seen": "2020-02-23",
261
- "source": "PassiveTotal"
262
- }
263
- ],
264
- "mail7.realty-advertising.ru": [
265
- {
266
- "firtst_seen": "2020-02-23",
267
- "last_seen": "2020-02-23",
268
- "source": "PassiveTotal"
269
- }
270
- ],
271
- "zimbra.realty-advertising.ru": [
272
- {
273
- "firtst_seen": "2020-02-23",
274
- "last_seen": "2020-02-23",
275
- "source": "PassiveTotal"
276
- }
277
- ],
278
- "relay2.realty-advertising.ru": [
279
- {
280
- "firtst_seen": "2020-02-23",
281
- "last_seen": "2020-02-23",
282
- "source": "PassiveTotal"
283
- }
284
- ],
285
- "sniper.realty-advertising.ru": [
286
- {
287
- "firtst_seen": "2020-02-22",
288
- "last_seen": "2020-02-22",
289
- "source": "PassiveTotal"
290
- }
291
- ],
292
- "mailx.realty-advertising.ru": [
293
- {
294
- "firtst_seen": "2020-02-22",
295
- "last_seen": "2020-02-22",
296
- "source": "PassiveTotal"
297
- }
298
- ],
299
- "send.realty-advertising.ru": [
300
- {
301
- "firtst_seen": "2020-02-22",
302
- "last_seen": "2020-02-22",
303
- "source": "PassiveTotal"
304
- }
305
- ],
306
- "mta.realty-advertising.ru": [
307
- {
308
- "firtst_seen": "2020-02-22",
309
- "last_seen": "2020-02-22",
310
- "source": "PassiveTotal"
311
- }
312
- ],
313
- "home.realty-advertising.ru": [
314
- {
315
- "firtst_seen": "2020-02-22",
316
- "last_seen": "2020-02-22",
317
- "source": "PassiveTotal"
318
- }
319
- ],
320
- "pbrand.realty-advertising.ru": [
321
- {
322
- "firtst_seen": "2020-02-22",
323
- "last_seen": "2020-02-22",
324
- "source": "PassiveTotal"
325
- }
326
- ],
327
- "smtpauth.realty-advertising.ru": [
328
- {
329
- "firtst_seen": "2020-02-22",
330
- "last_seen": "2020-02-22",
331
- "source": "PassiveTotal"
332
- }
333
- ],
334
- "gate.realty-advertising.ru": [
335
- {
336
- "firtst_seen": "2020-02-21",
337
- "last_seen": "2020-02-21",
338
- "source": "PassiveTotal"
339
- }
340
- ],
341
- "mx02.realty-advertising.ru": [
342
- {
343
- "firtst_seen": "2020-02-21",
344
- "last_seen": "2020-02-21",
345
- "source": "PassiveTotal"
346
- }
347
- ],
348
- "outmail.realty-advertising.ru": [
349
- {
350
- "firtst_seen": "2020-02-21",
351
- "last_seen": "2020-02-21",
352
- "source": "PassiveTotal"
353
- }
354
- ],
355
- "exchange.realty-advertising.ru": [
356
- {
357
- "firtst_seen": "2020-02-21",
358
- "last_seen": "2020-02-21",
359
- "source": "PassiveTotal"
360
- }
361
- ],
362
- "ms.realty-advertising.ru": [
363
- {
364
- "firtst_seen": "2020-02-21",
365
- "last_seen": "2020-02-21",
366
- "source": "PassiveTotal"
367
- }
368
- ],
369
- "owa.realty-advertising.ru": [
370
- {
371
- "firtst_seen": "2020-02-20",
372
- "last_seen": "2020-02-20",
373
- "source": "PassiveTotal"
374
- }
375
- ],
376
- "mail8.realty-advertising.ru": [
377
- {
378
- "firtst_seen": "2020-02-20",
379
- "last_seen": "2020-02-20",
380
- "source": "PassiveTotal"
381
- }
382
- ],
383
- "mta-sts.realty-advertising.ru": [
384
- {
385
- "firtst_seen": "2019-11-11",
386
- "last_seen": "2020-02-08",
387
- "source": "PassiveTotal"
388
- }
389
- ],
390
- "mail02.realty-advertising.ru": [
391
- {
392
- "firtst_seen": "2020-01-18",
393
- "last_seen": "2020-01-18",
394
- "source": "PassiveTotal"
395
- }
396
- ],
397
- "www.realty-advertising.ru": [
398
- {
399
- "firtst_seen": "2019-11-08",
400
- "last_seen": "2019-11-12",
401
- "source": "PassiveTotal"
402
- }
403
- ],
404
- "ln-048.rd-00003024.id-11744955.v0.tun.vpnoverdns.com": [
405
- {
406
- "firtst_seen": "2017-04-06",
407
- "last_seen": "2017-04-06",
408
- "source": "PassiveTotal"
409
- }
410
- ],
411
- "mnen6k7g.info": [
412
- {
413
- "firtst_seen": "2010-10-28",
414
- "last_seen": "2010-10-28",
415
- "source": "PassiveTotal"
416
- }
417
- ]
94
+ "example.org": {
95
+ "first_seen": "2011-04-11",
96
+ "last_seen": "2020-03-16",
97
+ "sources": [
98
+ {
99
+ "first_seen": "2011-04-11",
100
+ "last_seen": "2011-04-11",
101
+ "source": "CIRCL"
102
+ },
103
+ {
104
+ "first_seen": "2016-10-09",
105
+ "last_seen": "2018-10-28",
106
+ "source": "CIRCL"
107
+ },
108
+ {
109
+ "first_seen": "2014-12-09",
110
+ "last_seen": "2020-03-16",
111
+ "source": "PassiveTotal"
112
+ },
113
+ {
114
+ "first_seen": null,
115
+ "last_seen": null,
116
+ "source": "SecurityTrails"
117
+ }
118
+ ]
119
+ },
120
+ ...
121
+ }
122
+
123
+ # or you scan skip "lookup"
124
+ $ ukemi example.com
125
+ $ ukemi 195.123.226.243
126
+
127
+ # You can specify the order of resolutions
128
+
129
+ # Order by last_seen DESC
130
+ $ ukemi lookup example.com --order-by -last_seen
131
+
132
+ # Order by last_seen ASC
133
+ $ ukemi lookup example.com --order-by last_seen
134
+
135
+ # Order by first_seen DESC
136
+ $ ukemi lookup example.com --order-by -first_seen
137
+
138
+ # Order by first_seen ASC
139
+ $ ukemi lookup example.com --order-by first_seen
140
+ ```
141
+
142
+ ### Using with jq
143
+
144
+ [jq](https://stedolan.github.io/jq/)'s powerful processor helps to interact with the output.
145
+
146
+ ```bash
147
+ # List up resolutions only
148
+ $ ukemi lookup example.com | jq "keys"
149
+ [
150
+ "192.0.32.10",
151
+ "192.0.43.10",
152
+ "208.77.188.166",
153
+ "209.67.208.202",
154
+ "221.121.159.162",
155
+ "93.184.216.119",
156
+ "93.184.216.34"
157
+ ]
158
+
159
+ # List up the first 2 objects
160
+ $ ukemi lookup example.com | jq "to_entries | .[:2] | from_entries"
161
+ {
162
+ "93.184.216.34": {
163
+ "first_seen": "2016-03-01",
164
+ "last_seen": "2020-03-16",
165
+ "sources": [
166
+ {
167
+ "first_seen": "2016-10-07",
168
+ "last_seen": "2018-10-30",
169
+ "source": "CIRCL"
170
+ },
171
+ {
172
+ "first_seen": "2016-03-01",
173
+ "last_seen": "2020-03-16",
174
+ "source": "SecurityTrails"
175
+ },
176
+ {
177
+ "first_seen": "2020-03-03",
178
+ "last_seen": "2020-03-03",
179
+ "source": "VirusTotal"
180
+ }
181
+ ]
182
+ },
183
+ "221.121.159.162": {
184
+ "first_seen": "2019-11-04",
185
+ "last_seen": "2019-11-04",
186
+ "sources": [
187
+ {
188
+ "first_seen": "2019-11-04",
189
+ "last_seen": "2019-11-04",
190
+ "source": "VirusTotal"
191
+ }
192
+ ]
193
+ }
418
194
  }
419
195
  ```
420
196