mtr_monitor 0.13.0 → 0.13.1
Sign up to get free protection for your applications and to get access to all the features.
- checksums.yaml +4 -4
- data/Dockerfile +1 -1
- data/Gemfile.lock +1 -1
- data/README.md +64 -13
- data/lib/mtr_monitor/version.rb +1 -1
- metadata +1 -1
checksums.yaml
CHANGED
|
@@ -1,7 +1,7 @@
|
|
|
1
1
|
---
|
|
2
2
|
SHA1:
|
|
3
|
-
metadata.gz:
|
|
4
|
-
data.tar.gz:
|
|
3
|
+
metadata.gz: 67d8be5f8ee84f2af3fa561a91befbafc3c2ba05
|
|
4
|
+
data.tar.gz: 25168b4262fe49a16ed64dc6fec9639d8c9af484
|
|
5
5
|
SHA512:
|
|
6
|
-
metadata.gz:
|
|
7
|
-
data.tar.gz:
|
|
6
|
+
metadata.gz: e78240e7f8f8247058451d06a1e5f3dc6e9f0da49a37f95bd2fb1c2fc811e688661d1bc0cbdc5375b86a140331f1104a96bfa558a520b42ac8c1caa80e9a76ff
|
|
7
|
+
data.tar.gz: 747ab0245a85ff5b6ed4f78c83bf83fc5725eb56d7e5483908a914773d856947749b13a3b84693c3d2eea710062e9ae4d244ed33dbc3d5ecd6764b26cc2f1555
|
data/Dockerfile
CHANGED
data/Gemfile.lock
CHANGED
data/README.md
CHANGED
|
@@ -1,26 +1,53 @@
|
|
|
1
|
-
#
|
|
1
|
+
# MTR Monitor
|
|
2
2
|
|
|
3
3
|
[](https://semaphoreci.com/renderedtext/mtr-monitor)
|
|
4
4
|
|
|
5
|
+
In December 2017, Hetzner, our hosting provider for the Build Platform, had a
|
|
6
|
+
major network incident that lasted for almost a whole week. Our users were
|
|
7
|
+
rightly frustrated.
|
|
5
8
|
|
|
6
|
-
|
|
7
|
-
and
|
|
9
|
+
To prevent and monitor these situation in the future, we have set up a
|
|
10
|
+
transatlantic monitoring system based on MTR reports and Curl-ing important
|
|
11
|
+
vendors for our platform such are GitHub and DockerHub. This system should
|
|
12
|
+
report any issues in the network between Germany(Hetzner) and US(GitHub,
|
|
13
|
+
DockerHub).
|
|
8
14
|
|
|
9
|
-
|
|
15
|
+
This project is part of the effort to have a readily available MTR reports
|
|
16
|
+
before, during and after incidents, that we can send to Hetzner.
|
|
10
17
|
|
|
11
|
-
|
|
12
|
-
|
|
13
|
-
|
|
18
|
+
The MTR monitor is an application that generates MTR reports every 5 minutes and
|
|
19
|
+
uploads them to an S3 Bucket. It is available as a standalone Docker container,
|
|
20
|
+
and as a Ruby gem that can be injected into other Ruby applications.
|
|
14
21
|
|
|
15
|
-
|
|
22
|
+
Currently, we have the following routes covered:
|
|
23
|
+
|
|
24
|
+
- Germany(Hetzner) -> AWS US East 1 (part of Job Runner)
|
|
25
|
+
- Germany(Hetzner) -> AWS US West 1 (part of Job Runner)
|
|
26
|
+
- Germany(Hetzner) -> AWS US West 2 (part of Job Runner)
|
|
27
|
+
- Germany(Hetzner) -> GitHub (part of Job Runner)
|
|
28
|
+
- Germany(Hetzner) -> DockerHub (part of Job Runner)
|
|
29
|
+
- Germany(Hetzner) -> Stripe (part of Job Runner)
|
|
30
|
+
- Germany(Hetzner) -> SemaphoreCI (part of Job Runner)
|
|
31
|
+
- AWS US East 1 -> Builder sb1 in Hetzner (standalone AWS instance with Docker container)
|
|
32
|
+
- AWS US West 1 -> Builder sb1 in Hetzner (standalone AWS instance with Docker container)
|
|
33
|
+
- AWS US West 2 -> Builder sb1 in Hetzner (standalone AWS instance with Docker container)
|
|
34
|
+
|
|
35
|
+
Dashboards for the MTR monitor can be found on the
|
|
36
|
+
[Platform — Network](https://semaphore.grafana.net/dashboard/db/platform-network?refresh=10s&orgId=1)
|
|
37
|
+
dashboard on Grafana.
|
|
16
38
|
|
|
17
|
-
|
|
39
|
+
## Using MTR Monitor as a gem
|
|
40
|
+
|
|
41
|
+
The MTR monitor can be used as a gem and injected into existing Ruby
|
|
42
|
+
applications. Currently, we inject the MTR monitor into Job Runner.
|
|
43
|
+
|
|
44
|
+
First, add the `mtr_monitor` gem to your Gemfile:
|
|
18
45
|
|
|
19
46
|
```ruby
|
|
20
47
|
gem 'mtr_monitor'
|
|
21
48
|
```
|
|
22
49
|
|
|
23
|
-
|
|
50
|
+
Secondly, use the report class to generate a report:
|
|
24
51
|
|
|
25
52
|
``` ruby
|
|
26
53
|
name = "google"
|
|
@@ -37,9 +64,33 @@ report = MtrMonitor::Report.new(name,
|
|
|
37
64
|
aws_secret_access_key)
|
|
38
65
|
|
|
39
66
|
report.generate
|
|
40
|
-
report.submit_metrics
|
|
41
67
|
```
|
|
42
68
|
|
|
43
|
-
|
|
69
|
+
This above snippet will :
|
|
70
|
+
- generate an MTR report on your local system under the `/var/log/mtr` directory
|
|
71
|
+
- upload the report to the provided S3 bucket
|
|
72
|
+
- submit metrics via Watchman and generate a metric "pulse"
|
|
73
|
+
|
|
74
|
+
If you want to generate reports continuously, create a CRON task that will call
|
|
75
|
+
the above code. To monitor if the CRON task is running as expected, you should
|
|
76
|
+
set up an alert on Grafana based on the "pulse" metric.
|
|
77
|
+
|
|
78
|
+
The pulse metric has the format `network.mtr.pulse` and is tagged with the
|
|
79
|
+
hostname of the server where the MTR monitor is running and with the name of the
|
|
80
|
+
metric.
|
|
81
|
+
|
|
82
|
+
MTR hops are also submitted to Grafana. Based on these metrics you can observe
|
|
83
|
+
the packet loss, avg, best, and worst latency on the network. For more
|
|
84
|
+
information read the code in `lib/mtr_monitor/metrics.rb`.
|
|
85
|
+
|
|
86
|
+
## Using MTR Monitor as a standalone Docker container
|
|
87
|
+
|
|
88
|
+
``` bash
|
|
89
|
+
docker run -d -v /var/log/mtr:/var/log/mtr -e NAME=<> -e DOMAIN=<> -e MTR_OPTIONS=<> -e S3_BUCKET=<> -e AWS_ACCESS_KEY_ID=<> -e AWS_SECRET_ACCESS_KEY=<> -e SLEEP_TIME=<> renderedtext/mtr_monitor
|
|
90
|
+
```
|
|
91
|
+
|
|
92
|
+
### Generate an MTR report from Ruby
|
|
93
|
+
|
|
94
|
+
|
|
95
|
+
Invoke mtr generation:
|
|
44
96
|
|
|
45
|
-
The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
|
data/lib/mtr_monitor/version.rb
CHANGED