yabeda-http_requests 0.1.0

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml ADDED
@@ -0,0 +1,7 @@
1
+ ---
2
+ SHA256:
3
+ metadata.gz: 01bace4735a867891820dd93841a8644b1f3937e0d51680cc7179ba508bea8d3
4
+ data.tar.gz: 4bea13b8f7843f97673dbaaf529b27f16de4fce6d548271e3dbd70e64a4142cd
5
+ SHA512:
6
+ metadata.gz: d9d17867e1a4c0a2405526c0f5f8e0f6a715bf2256781128e58f0a6b894a80772350138537ac6260770554204798b913684853edaa9cee5049e9123b36e66c8e
7
+ data.tar.gz: 48df8e0f9ac3f1d5715464ec3756dd4fb9989412abae0407e489a4dc2ade86436eb33eb6df814971d40db8abd67e2a41fc198e181b4c6ec7062b8e3fe1998dce
data/.gitignore ADDED
@@ -0,0 +1,11 @@
1
+ /.bundle/
2
+ /.yardoc
3
+ /_yardoc/
4
+ /coverage/
5
+ /doc/
6
+ /pkg/
7
+ /spec/reports/
8
+ /tmp/
9
+
10
+ # rspec failure tracking
11
+ .rspec_status
data/.rspec ADDED
@@ -0,0 +1,3 @@
1
+ --format documentation
2
+ --color
3
+ --require spec_helper
data/.rubocop.yml ADDED
@@ -0,0 +1,3 @@
1
+ Style/Documentation:
2
+ Exclude:
3
+ - 'example/**/*'
data/.travis.yml ADDED
@@ -0,0 +1,6 @@
1
+ ---
2
+ language: ruby
3
+ cache: bundler
4
+ rvm:
5
+ - 2.7.0
6
+ before_install: gem install bundler -v 2.1.2
data/Gemfile ADDED
@@ -0,0 +1,13 @@
1
+ # frozen_string_literal: true
2
+
3
+ source 'https://rubygems.org'
4
+
5
+ gemspec
6
+
7
+ gem 'byebug'
8
+ gem 'rake', '~> 12.0'
9
+ gem 'rspec', '~> 3.0'
10
+ gem 'rubocop'
11
+
12
+ gem 'faraday'
13
+ gem 'sniffer', '>= 0.4.0'
data/LICENSE.txt ADDED
@@ -0,0 +1,21 @@
1
+ The MIT License (MIT)
2
+
3
+ Copyright (c) 2020 TODO: Write your name
4
+
5
+ Permission is hereby granted, free of charge, to any person obtaining a copy
6
+ of this software and associated documentation files (the "Software"), to deal
7
+ in the Software without restriction, including without limitation the rights
8
+ to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
9
+ copies of the Software, and to permit persons to whom the Software is
10
+ furnished to do so, subject to the following conditions:
11
+
12
+ The above copyright notice and this permission notice shall be included in
13
+ all copies or substantial portions of the Software.
14
+
15
+ THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
16
+ IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
17
+ FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
18
+ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
19
+ LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
20
+ OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
21
+ THE SOFTWARE.
data/README.md ADDED
@@ -0,0 +1,83 @@
1
+ <a href="https://amplifr.com/?utm_source=yabeda-http_requests">
2
+ <img width="100" height="140" align="right"
3
+ alt="Sponsored by Amplifr" src="https://amplifr-direct.s3-eu-west-1.amazonaws.com/social_images/image/37b580d9-3668-4005-8d5a-137de3a3e77c.png" />
4
+ </a>
5
+
6
+ # Yabeda::HttpRequests
7
+
8
+ Built-in metrics for external services HTTP calls! This gem is a Part of the [yabeda](https://github.com/yabeda-rb/yabeda) suite.
9
+
10
+ ## Metrics
11
+
12
+ Works as the Puma plugin and provides following metrics:
13
+ - `http_request_total` - the number of external HTTP request attempts (by host, port, method)
14
+ - `http_response_total` - the number of made external HTTP requeusts (by host, port, method, status)
15
+ - `http_response_duuration` - the histogram of response duration (by host, port, method, status)
16
+
17
+ ## Installation
18
+
19
+ Add this line to your application's Gemfile:
20
+
21
+ ```ruby
22
+ gem 'yabeda-http_requests'
23
+ ```
24
+
25
+ And then execute:
26
+
27
+ $ bundle install
28
+
29
+ Or install it yourself as:
30
+
31
+ $ gem install yabeda-http_requests
32
+
33
+ ## Usage
34
+
35
+ After plugin the gem, you just have to set up metrics exporting with [yabeda-prometheus](https://github.com/yabeda-rb/yabeda-prometheus) gem.
36
+
37
+ The metrics page will look like this:
38
+
39
+ ```
40
+ # TYPE http_requests_total_count counter
41
+ # HELP http_requests_total_count A counter of the total number of external HTTP requests.
42
+ http_request_total{host="twitter.com",port="443",method="GET",query="/dsalahutdinov1"} 149.0
43
+ http_request_total{host="dev.to",port="443",method="GET",query="/amplifr"} 145.0
44
+ ...
45
+ ```
46
+
47
+ To simple set up Grafana, try the [sample dashboard](https://github.com/yabeda-rb/yabeda-http_requests/blob/master/example/grafana/provisioning/dashboards/yabeda-http_requests.json).
48
+
49
+ ## Sample application
50
+
51
+ Sample application aims to show how Ruby web-application, this gem and Prometheus/Grafana work togather.
52
+ Get into `example` directory and run docker compose:
53
+
54
+ ```sh
55
+ $ cd example
56
+ $ docker-compose up
57
+ ```
58
+
59
+ After docker image builds and all the services get up, you can browse application endpoints:
60
+ - Ruby web-application runs on [http://localhost:9292/](http://localhost:9292/). Everytime you request page, it schedules random web-scrapping job into sidekiq.
61
+ - Sidekiq exposes prometheus metrics on [http://localhost:9394/metrics](http://localhost:9394/metrics). This endpoint is scrapped by prometheus exporter every 5 seconds.
62
+ - Grafana runs on [http://localhost:3000/](http://localhost:3000/). Use `admin/foobar` as login and password to get in. Grafana already has specific dashboard with data visualisation.
63
+
64
+ Follow the [yabeda-external-http-requests](http://localhost:3000/d/OGd-oEXWz/yabeda-external-http-requests?orgId=1&refresh=5s) dashboard in Grafana.
65
+ Finally, after a couple of minutes when data collected you will see the following:
66
+ ![Monitor external HTTP calls with Grafana](docs/http_requests_rate_by_host.png)
67
+
68
+ ## Development with Docker
69
+
70
+ Get local development environment working and tests running is very easy with docker-compose:
71
+ ```bash
72
+ docker-compose run app bundle
73
+ docker-compose run app bundle exec rspec
74
+ ```
75
+
76
+ ## Contributing
77
+
78
+ Bug reports and pull requests are welcome on GitHub at https://github.com/yabeda-rb/yabeda-http_requests.
79
+
80
+
81
+ ## License
82
+
83
+ The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).
data/Rakefile ADDED
@@ -0,0 +1,11 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'bundler/gem_tasks'
4
+ require 'rspec/core/rake_task'
5
+ require 'rubocop/rake_task'
6
+
7
+ RuboCop::RakeTask.new
8
+
9
+ RSpec::Core::RakeTask.new(:spec)
10
+
11
+ task default: %i[rubocop spec]
data/bin/console ADDED
@@ -0,0 +1,15 @@
1
+ #!/usr/bin/env ruby
2
+ # frozen_string_literal: true
3
+
4
+ require 'bundler/setup'
5
+ require 'yabeda/http'
6
+
7
+ # You can add fixtures and/or initialization code here to make experimenting
8
+ # with your gem easier. You can also use a different console, if you like.
9
+
10
+ # (If you use this, don't forget to add pry to your Gemfile!)
11
+ # require "pry"
12
+ # Pry.start
13
+
14
+ require 'irb'
15
+ IRB.start(__FILE__)
data/bin/setup ADDED
@@ -0,0 +1,8 @@
1
+ #!/usr/bin/env bash
2
+ set -euo pipefail
3
+ IFS=$'\n\t'
4
+ set -vx
5
+
6
+ bundle install
7
+
8
+ # Do any other automated setup that you need to do here
@@ -0,0 +1,18 @@
1
+ version: '3.4'
2
+
3
+ services:
4
+ app:
5
+ image: ruby:latest
6
+ environment:
7
+ - BUNDLE_PATH=/bundle
8
+ - BUNDLE_CONFIG=/app/.bundle/config
9
+ command: bash
10
+ working_dir: /app
11
+ volumes:
12
+ - .:/app:cached
13
+ - bundler_data:/bundle
14
+ tmpfs:
15
+ - /tmp
16
+
17
+ volumes:
18
+ bundler_data:
@@ -0,0 +1,42 @@
1
+ # frozen_string_literal: true
2
+
3
+ require 'yabeda/http_requests/version'
4
+ require 'yabeda/http_requests/sniffer'
5
+ require 'yabeda'
6
+ require 'sniffer'
7
+
8
+ module Yabeda
9
+ # Common module
10
+ module HttpRequests
11
+ Yabeda.configure do
12
+ group :http
13
+
14
+ LONG_RUNNING_REQUEST_BUCKETS = [
15
+ 0.5, 1, 2.5, 5, 10, 25, 50, 100, 250, 500, 1000, # standard
16
+ 30_000, 60_000, 120_000, 300_000, 600_000 # slow queries
17
+ ].freeze
18
+
19
+ counter :request_total,
20
+ comment: 'A counter of the total number of external HTTP \
21
+ requests.',
22
+ tags: %i[host port method]
23
+ counter :response_total,
24
+ comment: 'A counter of the total number of external HTTP \
25
+ responses.',
26
+ tags: %i[host port method status]
27
+
28
+ histogram :response_duration, tags: %i[host port method status],
29
+ unit: :milliseconds,
30
+ buckets: LONG_RUNNING_REQUEST_BUCKETS,
31
+ comment: "A histogram of the response \
32
+ duration (milliseconds)."
33
+
34
+ ::Sniffer.config do |c|
35
+ c.enabled = true
36
+ c.middleware do |chain|
37
+ chain.add(Yabeda::HttpRequests::Sniffer)
38
+ end
39
+ end
40
+ end
41
+ end
42
+ end
@@ -0,0 +1,52 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Yabeda
4
+ module HttpRequests
5
+ # Middleware for sniffer gem
6
+ class Sniffer
7
+ def request(data_item)
8
+ Yabeda.http_request_total.increment(
9
+ host: data_item.request.host,
10
+ port: data_item.request.port,
11
+ method: data_item.request.method
12
+ )
13
+ end
14
+
15
+ def response(data_item)
16
+ log_http_response_total(data_item)
17
+ log_http_response_duration(data_item)
18
+ end
19
+
20
+ private
21
+
22
+ def log_http_response_total(data_item)
23
+ Yabeda.http_response_total.increment(
24
+ host: data_item.request.host,
25
+ port: data_item.request.port,
26
+ method: data_item.request.method,
27
+ status: data_item.response.status
28
+ )
29
+ end
30
+
31
+ def log_http_response_duration(data_item)
32
+ labels = {
33
+ host: data_item.request.host,
34
+ port: data_item.request.port,
35
+ method: data_item.request.method,
36
+ status: data_item.response.status
37
+ }
38
+
39
+ Yabeda.http_response_duration.measure(
40
+ labels, duration_in_milliseconds(data_item)
41
+ )
42
+ end
43
+
44
+ def duration_in_milliseconds(data_item)
45
+ seconds = data_item.response&.timing
46
+ return nil if seconds.nil?
47
+
48
+ (seconds * 1000).round
49
+ end
50
+ end
51
+ end
52
+ end
@@ -0,0 +1,7 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Yabeda
4
+ module HttpRequests
5
+ VERSION = '0.1.0'
6
+ end
7
+ end
@@ -0,0 +1,31 @@
1
+ # frozen_string_literal: true
2
+
3
+ require_relative 'lib/yabeda/http_requests/version'
4
+
5
+ Gem::Specification.new do |spec|
6
+ spec.name = 'yabeda-http_requests'
7
+ spec.version = Yabeda::HttpRequests::VERSION
8
+ spec.authors = ['Dmitry Salahutdinov']
9
+ spec.email = ['dsalahutdinov@gmail.com']
10
+
11
+ spec.summary = 'Monitoring of external services HTTP/HTTPS calls'
12
+ spec.description = 'Extends Yabeda metrics to collect external calls'
13
+ spec.homepage = 'https://github.com/yabeda-rb/yabeda-http_requests'
14
+ spec.license = 'MIT'
15
+ spec.required_ruby_version = Gem::Requirement.new('>= 2.3.0')
16
+
17
+ spec.metadata['homepage_uri'] = spec.homepage
18
+ spec.metadata['source_code_uri'] = 'https://github.com/yabeda-rb/yabeda-http_requests'
19
+
20
+ spec.files = Dir.chdir(File.expand_path(__dir__)) do
21
+ `git ls-files -z`.split("\x0").reject do |f|
22
+ f.match(%r{^(test|spec|features|example|docs)/})
23
+ end
24
+ end
25
+ spec.bindir = 'exe'
26
+ spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
27
+ spec.require_paths = ['lib']
28
+
29
+ spec.add_runtime_dependency 'sniffer'
30
+ spec.add_runtime_dependency 'yabeda'
31
+ end
metadata ADDED
@@ -0,0 +1,88 @@
1
+ --- !ruby/object:Gem::Specification
2
+ name: yabeda-http_requests
3
+ version: !ruby/object:Gem::Version
4
+ version: 0.1.0
5
+ platform: ruby
6
+ authors:
7
+ - Dmitry Salahutdinov
8
+ autorequire:
9
+ bindir: exe
10
+ cert_chain: []
11
+ date: 2020-03-19 00:00:00.000000000 Z
12
+ dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: sniffer
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - ">="
18
+ - !ruby/object:Gem::Version
19
+ version: '0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - ">="
25
+ - !ruby/object:Gem::Version
26
+ version: '0'
27
+ - !ruby/object:Gem::Dependency
28
+ name: yabeda
29
+ requirement: !ruby/object:Gem::Requirement
30
+ requirements:
31
+ - - ">="
32
+ - !ruby/object:Gem::Version
33
+ version: '0'
34
+ type: :runtime
35
+ prerelease: false
36
+ version_requirements: !ruby/object:Gem::Requirement
37
+ requirements:
38
+ - - ">="
39
+ - !ruby/object:Gem::Version
40
+ version: '0'
41
+ description: Extends Yabeda metrics to collect external calls
42
+ email:
43
+ - dsalahutdinov@gmail.com
44
+ executables: []
45
+ extensions: []
46
+ extra_rdoc_files: []
47
+ files:
48
+ - ".gitignore"
49
+ - ".rspec"
50
+ - ".rubocop.yml"
51
+ - ".travis.yml"
52
+ - Gemfile
53
+ - LICENSE.txt
54
+ - README.md
55
+ - Rakefile
56
+ - bin/console
57
+ - bin/setup
58
+ - docker-compose.yml
59
+ - lib/yabeda/http_requests.rb
60
+ - lib/yabeda/http_requests/sniffer.rb
61
+ - lib/yabeda/http_requests/version.rb
62
+ - yabeda-http_requests.gemspec
63
+ homepage: https://github.com/yabeda-rb/yabeda-http_requests
64
+ licenses:
65
+ - MIT
66
+ metadata:
67
+ homepage_uri: https://github.com/yabeda-rb/yabeda-http_requests
68
+ source_code_uri: https://github.com/yabeda-rb/yabeda-http_requests
69
+ post_install_message:
70
+ rdoc_options: []
71
+ require_paths:
72
+ - lib
73
+ required_ruby_version: !ruby/object:Gem::Requirement
74
+ requirements:
75
+ - - ">="
76
+ - !ruby/object:Gem::Version
77
+ version: 2.3.0
78
+ required_rubygems_version: !ruby/object:Gem::Requirement
79
+ requirements:
80
+ - - ">="
81
+ - !ruby/object:Gem::Version
82
+ version: '0'
83
+ requirements: []
84
+ rubygems_version: 3.1.2
85
+ signing_key:
86
+ specification_version: 4
87
+ summary: Monitoring of external services HTTP/HTTPS calls
88
+ test_files: []