yabeda-datadog 0.2.0 → 0.3.0.rc1

Sign up to get free protection for your applications and to get access to all the features.
checksums.yaml CHANGED
@@ -1,7 +1,7 @@
1
1
  ---
2
2
  SHA256:
3
- metadata.gz: a61ee9350adf2ceab80125e940127a919d1fcef9ce84444a41644d689182df0e
4
- data.tar.gz: 9f941017b4fa519c18a3ed0c4351c76990464a7b9098ab6031fc7f008da8537d
3
+ metadata.gz: a2600b842315a1b8dfde3a722c1c6b75b0c88a66146a439d3d2d59e48553bbaa
4
+ data.tar.gz: df24c470280a87e6fefdc68d90296a33fc7ce1bdcae900e530cb59106b099ab9
5
5
  SHA512:
6
- metadata.gz: 97ba2ef03616154f4ae5ff48bad10cda63e8625bd25bc706babd3ab63d26d0b733cdbc831ea5b0ab84c06ea073571e468faab8a7ff2933ebb9188a27ba4709f6
7
- data.tar.gz: 63cf3ba1a667d8b044fd69004397d7e7579a4a31fd2fff247ef8cb781e0de091928c55660426fe611754b809a48005784e52269a2a7092149fe1ccfcf0f654b3
6
+ metadata.gz: edec9182d5535f62cbd2e38efd3a1f5b2ab11f33f76616d0a8706c8ff79a109cd00e6933a19a20737686891f5c6ea919e253657017c5c1fac05d54bd30dd98d7
7
+ data.tar.gz: 67ecae6205e8f9286a563caeb98813359d7e3888d15b02b0ff23711835f429cddbf18096395d9e97d37728d8716de009616ba2f8bf4f16ac968b6d006b632482
data/.gitignore CHANGED
@@ -12,3 +12,6 @@
12
12
 
13
13
  # environment files
14
14
  .datadog-agent.env
15
+
16
+ # Editors
17
+ /.vscode/
@@ -0,0 +1,55 @@
1
+ # Contributing
2
+
3
+ 1. Fork the repo.
4
+
5
+ 1. Run `./bin/setup`.
6
+
7
+ 1. Run the tests. We only take pull requests with passing tests, and it's great to know that you have a clean slate: `rake spec`.
8
+
9
+ 1. Make sure your editor has [rubocop gem](https://github.com/rubocop-hq/rubocop) integration or using rubocop as CLI tool and you are not violating code style rules.
10
+
11
+ 1. Add a test for your change. Only refactoring and documentation changes require no new tests. If you are adding functionality or fixing a bug, we need a test!
12
+
13
+ 1. Make the test pass.
14
+
15
+ 1. Write a [good commit message][commit].
16
+
17
+ 1. Push to your fork and submit a pull request.
18
+
19
+ Others will give constructive feedback. This is a time for discussion and improvements, and making the necessary changes will be required before we can merge the contribution.
20
+
21
+ ## Start Application in Development
22
+
23
+ After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
24
+
25
+ You can run a Datadog agent in a docker container with the following command:
26
+
27
+ $ bin/dev
28
+
29
+ Beware that the agent will collect metrics (a lot) from docker itself and your OS and all launched docker containers. You have to provide `DD_API_KEY` in `.datadog-agent.env` file. You can put additional environment variable for Datadog agent container into this file
30
+
31
+ Example of `.datadog-agent.env` file:
32
+
33
+ ```shell
34
+ # required
35
+ DD_API_KEY=<your Datadog API key>
36
+ DD_DOGSTATSD_NON_LOCAL_TRAFFIC=true
37
+ # optinal
38
+ DD_HOSTNAME=my-development-computer
39
+ ```
40
+
41
+ As a real usage example you can run:
42
+
43
+ $ YABEDA_DATADOG_API_KEY=<your Datadog API key> YABEDA_DATADOG_APP_KEY=<your Datadog App key> ruby examples/script.rb
44
+
45
+ To install this gem onto your local machine, run:
46
+
47
+ $ bundle exec rake install
48
+
49
+ To release a new version, update the version number in `version.rb`, and then run:
50
+
51
+ $ bundle exec rake release
52
+
53
+ Which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
54
+
55
+ [commit]: http://tbaggery.com/2008/04/19/a-note-about-git-commit-messages.html
@@ -1,7 +1,8 @@
1
1
  PATH
2
2
  remote: .
3
3
  specs:
4
- yabeda-datadog (0.1.0)
4
+ yabeda-datadog (0.3.0.rc1)
5
+ anyway_config (~> 1.0)
5
6
  dogapi
6
7
  dogstatsd-ruby
7
8
  yabeda
@@ -9,6 +10,7 @@ PATH
9
10
  GEM
10
11
  remote: https://rubygems.org/
11
12
  specs:
13
+ anyway_config (1.4.2)
12
14
  coderay (1.1.2)
13
15
  concurrent-ruby (1.1.4)
14
16
  diff-lcs (1.3)
@@ -50,4 +52,4 @@ DEPENDENCIES
50
52
  yabeda-datadog!
51
53
 
52
54
  BUNDLED WITH
53
- 1.17.2
55
+ 1.17.3
data/README.md CHANGED
@@ -4,7 +4,7 @@
4
4
 
5
5
  ## Prerequisites
6
6
 
7
- Have an instance of Datadog agent and dogstats-d running. For other installation options of Datadog agent please refer to [Datadog agent documentation](https://docs.datadoghq.com/agent/).
7
+ Have an instance of Datadog agent and dogstats-d running. For installation options of Datadog agent please refer to [Datadog agent documentation](https://docs.datadoghq.com/agent/).
8
8
 
9
9
  ## Installation
10
10
 
@@ -20,43 +20,101 @@ And then execute:
20
20
 
21
21
  ## Usage
22
22
 
23
- Configure Yabeda metrics. Refer to [Yabeda documentation](https://github.com/yabeda-rb/yabeda) for instruction on how to configure and use Yabeda metrics.
23
+ ### Define metrics
24
24
 
25
- Please note when configuring Yabeda you have to use Datadog units. Refer to [Datadog unit for metrics documentation](https://docs.datadoghq.com/developers/metrics/#units).
26
- If a unit of a metric is not supported by Datadog, this unit will not be automatically updated. But you always have the ability to update it manually in Datadog metrics dashboard or by calling API by yourself.
25
+ Define Yabeda metrics to collect. Refer to [Yabeda documentation](https://github.com/yabeda-rb/yabeda) for instruction on how to configure and use Yabeda metrics.
26
+
27
+ Please note when configuring Yabeda you have to use [Datadog units](https://docs.datadoghq.com/developers/metrics/#units).
28
+ If a unit of a metric is not supported by Datadog, unit information will not be submitted to Datadog. However, the rest of the metric information will be updated.
29
+ You always have the ability to update it manually in Datadog metrics dashboard or by calling API by yourself.
27
30
 
28
31
  Refer to [Datadog metrics documentation](https://docs.datadoghq.com/graphing/metrics/) for working with your metrics in Datadog dashboard.
29
32
 
30
- Start your application. You have to set your `DATADOG_API_KEY` and `DATADOG_APP_KEY` as environment variables.
33
+ ### Configure the adapter
34
+
35
+ You can configure with either `APP_ROOT/config/yabeda_datadog.yml` file or with environment variables.
36
+ Rails 5.1 users able to use encrypted rails secrets `Rails.application.secrets.yabeda_datadog.*`.
37
+
38
+ Example of `yabeda_datadog.yml` file:
39
+
40
+ ```yaml
41
+ # required
42
+ api_key: <your Datadog API key>
43
+ app_key: <your Datadog App key>
44
+
45
+ # optional, default values used as an example
46
+ # how many queued metrics metrics sends in batches
47
+ batch_size: 10
48
+ # how many metrics you can queue for sending
49
+ queue_size: 1000
50
+ # threads to sends enqueued metrics
51
+ num_threads: 2
52
+ # Datadog agent host and port
53
+ agent_host: localhost
54
+ agent_port: 8125
55
+ ```
56
+
57
+ Example of environment variables:
58
+
59
+ ```shell
60
+ # required
61
+ YABEDA_DATADOG_API_KEY=<your Datadog API key>
62
+ YABEDA_DATADOG_APP_KEY=<your Datadog App key>
63
+
64
+ # optional, default values used as an example
65
+ # how many queued metrics metrics sends in batches
66
+ YABEDA_DATADOG_BATCH_SIZE=10
67
+ # how many metrics you can queue for sending
68
+ YABEDA_DATADOG_QUEUE_SIZE=1000
69
+ # threads to sends enqueued metrics
70
+ YABEDA_DATADOG_NUM_THREADS=2
71
+ # Datadog agent host and port
72
+ YABEDA_DATADOG_AGENT_HOST=localhost
73
+ YABEDA_DATADOG_AGENT_PORT=8125
74
+ ```
75
+
31
76
  You can obtain your Datadog API keys in [Datadog dashboard](https://app.datadoghq.com/account/settings#api).
32
77
 
33
- You may specify `DATADOG_AGENT_HOST` and/or `DATADOG_AGENT_PORT` environment variables if your Datadog agent is run not in the same host as an app/code that you collection metrics.
78
+ Please note, when filling the queue (queue size option), your application will be blocked by waiting for a place in the queue.
34
79
 
35
- You may specify `YABEDA_DATADOG_COLLECT_INTERVAL` environment variable to change default interval to call Yabeda collect blocks (aka collectors).
80
+ You may specify `DATADOG_AGENT_HOST` and/or `DATADOG_AGENT_PORT` environment variables if your Datadog agent is running not on the same host as an app/code that collects metrics.
36
81
 
37
- ### Limitations
82
+ ### Start the adapter
38
83
 
39
- On the first run of your application no metrics metadata will be updated. This is happening because metrics have not yet been collected, thus not been created, and there is nothing to update.
84
+ To start collecting and sending your metrics to Datadog agent run:
40
85
 
41
- ### Example
86
+ ```ruby
87
+ Yabeda::Datadog.start
88
+ ```
42
89
 
43
- [yabeda-datadog-sidekiq-example](https://github.com/shvetsovdm/yabeda-datadog-sidekiq-example)
90
+ To star collecting Yabeda collect blocks (aka collectors) run the command:
91
+
92
+ ```ruby
93
+ Yabeda::Datadog.start_exporter
94
+
95
+ # optionaly you can pass collect_interval argument
96
+
97
+ TEN_SECONDS = 10
98
+ Yabeda::Datadog.start_exporter(collect_interval: TEN_SECONDS)
99
+ ```
44
100
 
45
- ## Development
101
+ ### Limitations
46
102
 
47
- After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
103
+ On the first run of your application no metrics metadata will be updated. This is happening because metrics have not yet been collected, thus not been created, and there is nothing to update.
48
104
 
49
- In can run a dogstats-d instance in a docker container with the following command:
105
+ ### Example
50
106
 
51
- $ bin/dev
107
+ [yabeda-datadog-sidekiq-example](https://github.com/shvetsovdm/yabeda-datadog-sidekiq-example)
52
108
 
53
- Beware that the agent will collect metrics (a lot) from docker itself and all launched docker containers.
109
+ ### Alternatives
54
110
 
55
- To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and tags, and push the `.gem` file to [rubygems.org](https://rubygems.org).
111
+ Using [Prometheus support for Datadog Agent 6](https://www.datadoghq.com/blog/monitor-prometheus-metrics/) with [yabeda-prometheus](https://github.com/yabeda-rb/yabeda-prometheus).
56
112
 
57
113
  ## Contributing
58
114
 
59
- Bug reports and pull requests are welcome on GitHub at https://github.com/shvetsovdm/yabeda-datadog.
115
+ Please see [CONTRIBUTING guide](/CONTRIBUTING.md).
116
+
117
+ Bug reports and pull requests are welcome on GitHub at [https://github.com/shvetsovdm/yabeda-datadog](https://github.com/shvetsovdm/yabeda-datadog).
60
118
 
61
119
  ## License
62
120
 
@@ -5,22 +5,36 @@ require "yabeda/datadog"
5
5
 
6
6
  Thread.abort_on_exception = true
7
7
 
8
+ yabeda_datadog = Yabeda::Datadog.start
9
+
10
+ # Make sure you start a datadog agent container with the command:
11
+ #
12
+ # $ bin/dev
13
+ #
8
14
  # To Use this script execute it directly with ruby command.
9
15
  # You have to provide DATADOG_API_KEY and DATADOG_APP_KEY
10
16
  # environment variables.
11
17
  #
12
18
  # Example:
13
19
  #
14
- # DATADOG_API_KEY=<your API key> DATADOG_APP_KEY=<your app key> ruby script.rb
20
+ # DATADOG_API_KEY=<your API key> DATADOG_APP_KEY=<your app key> ruby examples/script.rb
15
21
  #
16
22
 
17
23
  Yabeda.configure do
18
24
  group :yabeda_datadog_gem_examples_script
19
- counter :run_count, comment: "The total number of times the script was executed", unit: "execution"
25
+ counter :run_count, comment: "The total number of times the script was executed", unit: "time"
20
26
  gauge :run_time, comment: "Script execution time", unit: "second"
27
+ histogram :rand_num, comment: "Random number", buckets: [0, 20, 40, 60, 80, 100]
21
28
  end
22
29
 
23
30
  start_time = Time.now
24
31
  Yabeda.yabeda_datadog_gem_examples_script_run_count.increment(host: "dev_machine")
32
+ Yabeda.yabeda_datadog_gem_examples_script_rand_num.measure({ host: "dev_machine", rand: true }, rand(100))
33
+ Yabeda.yabeda_datadog_gem_examples_script_rand_num.measure({ host: "dev_machine", rand: true }, rand(100))
34
+ Yabeda.yabeda_datadog_gem_examples_script_rand_num.measure({ host: "dev_machine", rand: true }, rand(100))
25
35
  finish_time = Time.now
26
36
  Yabeda.yabeda_datadog_gem_examples_script_run_time.set({ host: "dev_machine" }, finish_time - start_time)
37
+
38
+ puts "Type exit for exit the script" until gets.chomp =~ /^exit$/i
39
+ puts "Stoping Yabeda::Datadog, please wait ..."
40
+ yabeda_datadog.stop
@@ -3,17 +3,40 @@
3
3
  require "yabeda"
4
4
  require "yabeda/datadog/adapter"
5
5
  require "yabeda/datadog/version"
6
+ require "yabeda/datadog/exceptions"
7
+ require "yabeda/datadog/logging"
8
+ require "yabeda/datadog/response_handler"
9
+ require "yabeda/datadog/config"
6
10
 
7
11
  module Yabeda
8
12
  # = Namespace for DataDog adapter
9
13
  module Datadog
10
14
  SECOND = 1
11
- DEFAULT_COLLECT_INTERVAL = 60 * SECOND
15
+ COLLECT_INTERVAL = 60 * SECOND
12
16
 
13
- def self.start_exporter(collect_interval: DEFAULT_COLLECT_INTERVAL)
17
+ # Gem configuration object
18
+ def self.config
19
+ @config ||= Config.new
20
+ end
21
+
22
+ # Prepare the adapter to work
23
+ def self.start
24
+ raise ApiKeyError unless config.api_key
25
+ raise AppKeyError unless config.app_key
26
+
27
+ adapter = Yabeda::Datadog::Adapter.new
28
+ Yabeda.register_adapter(:datadog, adapter)
29
+ adapter
30
+ end
31
+
32
+ # Start collection metrics from Yabeda collectors
33
+ def self.start_exporter(collect_interval: COLLECT_INTERVAL)
14
34
  Thread.new do
35
+ Logging.instance.info "initilize collectors harvest"
15
36
  loop do
37
+ Logging.instance.info "start collectors harvest"
16
38
  Yabeda.collectors.each(&:call)
39
+ Logging.instance.info "end collectors harvest"
17
40
  sleep(collect_interval)
18
41
  end
19
42
  end
@@ -1,5 +1,6 @@
1
1
  # frozen_string_literal: true
2
2
 
3
+ require "yabeda/datadog/worker"
3
4
  require "yabeda/datadog/metric"
4
5
  require "yabeda/datadog/tags"
5
6
  require "yabeda/base_adapter"
@@ -8,76 +9,62 @@ require "dogapi"
8
9
 
9
10
  module Yabeda
10
11
  module Datadog
11
- DEFAULT_AGENT_HOST = "localhost"
12
- DEFAULT_AGENT_PORT = 8125
13
-
14
12
  # = DataDog adapter.
15
13
  #
16
- # Sends yabeda metrics as custom metrics to DataDog API.
14
+ # Sends yabeda metrics as custom metrics to DataDog.
17
15
  # https://docs.datadoghq.com/integrations/ruby/
18
16
  class Adapter < BaseAdapter
17
+ def initialize(worker: Worker.start)
18
+ @worker = worker
19
+ end
20
+
19
21
  def register_counter!(counter)
20
- metric = Metric.new(counter, "counter")
21
- Thread.new { metric.update(dogapi) }
22
+ enqueue_register(Metric.new(counter, "count"))
22
23
  end
23
24
 
24
25
  def perform_counter_increment!(counter, tags, increment)
25
- metric = Metric.new(counter, "counter")
26
- dogstatsd.count(metric.name, increment, tags: Tags.build(tags))
26
+ metric = Metric.new(counter, "count")
27
+ tags = Tags.build(tags)
28
+ enqueue_send(metric, increment, tags)
27
29
  end
28
30
 
29
31
  def register_gauge!(gauge)
30
- metric = Metric.new(gauge, "gauge")
31
- Thread.new { metric.update(dogapi) }
32
+ enqueue_register(Metric.new(gauge, "gauge"))
32
33
  end
33
34
 
34
35
  def perform_gauge_set!(gauge, tags, value)
35
36
  metric = Metric.new(gauge, "gauge")
36
- dogstatsd.gauge(metric.name, value, tags: Tags.build(tags))
37
+ tags = Tags.build(tags)
38
+ enqueue_send(metric, value, tags)
37
39
  end
38
40
 
39
41
  def register_histogram!(histogram)
40
- # sending many requests in separate threads
41
- # cause rejections by Datadog API
42
- Thread.new do
43
- histogram_metrics(histogram).map do |historgam_sub_metric|
44
- historgam_sub_metric.update(dogapi)
45
- end
42
+ Metric.histogram_metrics(histogram).map do |historgam_sub_metric|
43
+ enqueue_register(historgam_sub_metric)
46
44
  end
47
45
  end
48
46
 
49
47
  def perform_histogram_measure!(historam, tags, value)
50
48
  metric = Metric.new(historam, "histogram")
51
- dogstatsd.histogram(metric.name, value, tags: Tags.build(tags))
49
+ tags = Tags.build(tags)
50
+ enqueue_send(metric, value, tags)
51
+ end
52
+
53
+ def stop
54
+ worker.stop
52
55
  end
53
56
 
54
57
  private
55
58
 
56
- def dogstatsd
57
- # consider memoization here
58
- ::Datadog::Statsd.new(
59
- ENV.fetch("DATADOG_AGENT_HOST", DEFAULT_AGENT_HOST),
60
- ENV.fetch("DATADOG_AGENT_PORT", DEFAULT_AGENT_PORT),
61
- )
62
- end
59
+ attr_reader :worker
63
60
 
64
- def dogapi
65
- # consider memoization here
66
- ::Dogapi::Client.new(ENV["DATADOG_API_KEY"], ENV["DATADOG_APP_KEY"])
61
+ def enqueue_register(metric)
62
+ worker.enqueue(:REGISTER, metric: metric)
67
63
  end
68
64
 
69
- def histogram_metrics(historgram)
70
- [
71
- Metric.new(historgram, "gauge", name_sufix: "avg"),
72
- Metric.new(historgram, "gauge", name_sufix: "max"),
73
- Metric.new(historgram, "gauge", name_sufix: "min"),
74
- Metric.new(historgram, "gauge", name_sufix: "median"),
75
- Metric.new(historgram, "gauge", name_sufix: "95percentile", unit: nil, per_unit: nil),
76
- Metric.new(historgram, "rate", name_sufix: "count", unit: nil, per_unit: nil),
77
- ]
65
+ def enqueue_send(metric, value, tags)
66
+ worker.enqueue(:SEND, metric: metric, value: value, tags: tags)
78
67
  end
79
-
80
- Yabeda.register_adapter(:datadog, new)
81
68
  end
82
69
  end
83
70
  end
@@ -0,0 +1,20 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "anyway"
4
+
5
+ module Yabeda
6
+ module Datadog
7
+ # = Describes all config variables
8
+ class Config < Anyway::Config
9
+ env_prefix :datadog
10
+ config_name :datadog
11
+ attr_config :api_key,
12
+ :app_key,
13
+ agent_host: "localhost",
14
+ agent_port: 8125,
15
+ batch_size: 10,
16
+ queue_size: 1000,
17
+ num_threads: 2
18
+ end
19
+ end
20
+ end
@@ -0,0 +1,19 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Yabeda
4
+ module Datadog
5
+ # = This error raised when no Datadog API key provided
6
+ class ApiKeyError < StandardError
7
+ def initialize(msg = "Datadog API key doesn't set")
8
+ super
9
+ end
10
+ end
11
+
12
+ # = This error raised when no Datadog application key provided
13
+ class AppKeyError < StandardError
14
+ def initialize(msg = "Datadog application key doesn't set")
15
+ super
16
+ end
17
+ end
18
+ end
19
+ end
@@ -0,0 +1,41 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "logger"
4
+ require "singleton"
5
+
6
+ module Yabeda
7
+ module Datadog
8
+ # = Perform loging
9
+ class Logging
10
+ include Singleton
11
+
12
+ def initialize
13
+ @logger = Logger.new(STDOUT)
14
+ end
15
+
16
+ def warn(message)
17
+ @logger.warn message
18
+ end
19
+
20
+ def info(message)
21
+ @logger.info message
22
+ end
23
+
24
+ def debug(message)
25
+ @logger.debug message
26
+ end
27
+
28
+ def fatal(message)
29
+ @logger.fatal message
30
+ end
31
+
32
+ def error(message)
33
+ @logger.error message
34
+ end
35
+
36
+ def level=(level)
37
+ @logger.level = level
38
+ end
39
+ end
40
+ end
41
+ end
@@ -12,6 +12,8 @@ module Yabeda
12
12
  @overides = overides
13
13
  end
14
14
 
15
+ attr_reader :type
16
+
15
17
  # Datadog API argument
16
18
  def metadata
17
19
  {
@@ -50,7 +52,21 @@ module Yabeda
50
52
 
51
53
  private
52
54
 
53
- attr_reader :metric, :type, :overides
55
+ attr_reader :metric, :overides
56
+
57
+ class << self
58
+ # Build Datadog histogram metrics from Yabeda histogram metric
59
+ def histogram_metrics(historgram)
60
+ [
61
+ new(historgram, "gauge", name_sufix: "avg"),
62
+ new(historgram, "gauge", name_sufix: "max"),
63
+ new(historgram, "gauge", name_sufix: "min"),
64
+ new(historgram, "gauge", name_sufix: "median"),
65
+ new(historgram, "gauge", name_sufix: "95percentile", unit: nil, per_unit: nil),
66
+ new(historgram, "rate", name_sufix: "count", unit: nil, per_unit: nil),
67
+ ]
68
+ end
69
+ end
54
70
  end
55
71
  end
56
72
  end
@@ -0,0 +1,29 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Yabeda
4
+ module Datadog
5
+ # = Handle response from dogapi
6
+ class ResponseHandler
7
+ class << self
8
+ def call(metric)
9
+ Logging.instance.info "Sending #{metric.name} metric"
10
+ response = yield
11
+ Logging.instance.info "Response on #{metric.name}: #{handle(response)}"
12
+ response
13
+ end
14
+
15
+ private
16
+
17
+ def handle(response)
18
+ if response.is_a? Array
19
+ return response if response.count < 2
20
+ raise response[1]["errors"].join(", ") if response[1].key?("errors")
21
+
22
+ return "status: #{response[0]}, payload: #{response[1]}"
23
+ end
24
+ response
25
+ end
26
+ end
27
+ end
28
+ end
29
+ end
@@ -2,6 +2,6 @@
2
2
 
3
3
  module Yabeda
4
4
  module Datadog
5
- VERSION = "0.2.0"
5
+ VERSION = "0.3.0.rc1"
6
6
  end
7
7
  end
@@ -0,0 +1,100 @@
1
+ # frozen_string_literal: true
2
+
3
+ require "yabeda/datadog/worker/send"
4
+ require "yabeda/datadog/worker/register"
5
+
6
+ module Yabeda
7
+ module Datadog
8
+ # = Perform actions async
9
+ class Worker
10
+ def self.start(queue_size: Yabeda::Datadog.config.queue_size)
11
+ instance = new(SizedQueue.new(queue_size))
12
+ instance.spawn_threads(Yabeda::Datadog.config.num_threads)
13
+ instance
14
+ end
15
+
16
+ def initialize(queue)
17
+ @queue = queue
18
+ @threads = []
19
+ end
20
+
21
+ def enqueue(action, payload)
22
+ Logging.instance.info "enqueue action"
23
+ queue.push([action, payload])
24
+ end
25
+
26
+ def spawn_threads(num_threads)
27
+ num_threads.times do
28
+ threads << Thread.new do
29
+ grouped_actions = Hash.new { |hash, key| hash[key] = [] }
30
+
31
+ while running? || actions_left?
32
+ batch_size = 0
33
+ # wait for actions, blocks the current thread
34
+ action_key, action_payload = wait_for_action
35
+ if action_key
36
+ grouped_actions[action_key].push(action_payload)
37
+ batch_size += 1
38
+ end
39
+
40
+ # group a batch of actions
41
+ while batch_size < Yabeda::Datadog.config.batch_size
42
+ begin
43
+ action_key, action_payload = dequeue_action
44
+ grouped_actions[action_key].push(action_payload)
45
+ batch_size += 1
46
+ rescue ThreadError
47
+ break # exit batch loop if we drain the queue
48
+ end
49
+ end
50
+
51
+ # invoke actions in batches
52
+ grouped_actions.each_pair do |group_key, group_payload|
53
+ self.class.const_get(group_key, false).call(group_payload)
54
+ end
55
+
56
+ grouped_actions.clear
57
+ end
58
+ end
59
+ end
60
+
61
+ true
62
+ end
63
+
64
+ def spawned_threads_count
65
+ threads.size
66
+ end
67
+
68
+ def stop
69
+ queue.close
70
+ threads.each(&:exit)
71
+ threads.clear
72
+ true
73
+ end
74
+
75
+ private
76
+
77
+ attr_reader :queue, :threads, :logger
78
+
79
+ def actions_left?
80
+ !queue.empty?
81
+ end
82
+
83
+ def no_acitons?
84
+ queue.empty?
85
+ end
86
+
87
+ def dequeue_action
88
+ queue.pop(true)
89
+ end
90
+
91
+ def wait_for_action
92
+ queue.pop(false)
93
+ end
94
+
95
+ def running?
96
+ !queue.closed?
97
+ end
98
+ end
99
+ end
100
+ end
@@ -0,0 +1,23 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Yabeda
4
+ module Datadog
5
+ class Worker
6
+ REGISTER = proc do |accumulated_payload|
7
+ dogapi = ::Dogapi::Client.new(Yabeda::Datadog.config.api_key, Yabeda::Datadog.config.app_key)
8
+
9
+ accumulated_payload.each do |payload|
10
+ metric = payload.fetch(:metric)
11
+
12
+ begin
13
+ ResponseHandler.call(metric) do
14
+ metric.update(dogapi)
15
+ end
16
+ rescue StandardError => e
17
+ Logging.instance.fatal "Metric sending failed: #{e.message}"
18
+ end
19
+ end
20
+ end
21
+ end
22
+ end
23
+ end
@@ -0,0 +1,24 @@
1
+ # frozen_string_literal: true
2
+
3
+ module Yabeda
4
+ module Datadog
5
+ class Worker
6
+ SEND = proc do |accumulated_payload|
7
+ dogstatsd = ::Datadog::Statsd.new(
8
+ ENV.fetch("DATADOG_AGENT_HOST", Yabeda::Datadog.config.agent_host),
9
+ ENV.fetch("DATADOG_AGENT_PORT", Yabeda::Datadog.config.agent_port),
10
+ )
11
+
12
+ dogstatsd.batch do |stats|
13
+ accumulated_payload.each do |payload|
14
+ metric = payload.fetch(:metric)
15
+ value = payload.fetch(:value)
16
+ tags = payload.fetch(:tags)
17
+
18
+ stats.send(metric.type, metric.name, value, tags: tags)
19
+ end
20
+ end
21
+ end
22
+ end
23
+ end
24
+ end
@@ -27,6 +27,9 @@ Gem::Specification.new do |spec|
27
27
  spec.executables = spec.files.grep(%r{^exe/}) { |f| File.basename(f) }
28
28
  spec.require_paths = ["lib"]
29
29
 
30
+ spec.required_ruby_version = ">= 2.3.0"
31
+
32
+ spec.add_dependency "anyway_config", "~> 1.0"
30
33
  spec.add_dependency "dogapi"
31
34
  spec.add_dependency "dogstatsd-ruby"
32
35
  spec.add_dependency "yabeda"
metadata CHANGED
@@ -1,15 +1,29 @@
1
1
  --- !ruby/object:Gem::Specification
2
2
  name: yabeda-datadog
3
3
  version: !ruby/object:Gem::Version
4
- version: 0.2.0
4
+ version: 0.3.0.rc1
5
5
  platform: ruby
6
6
  authors:
7
7
  - Dmitry Shvetsov <@shvetsovdm>
8
8
  autorequire:
9
9
  bindir: exe
10
10
  cert_chain: []
11
- date: 2018-12-26 00:00:00.000000000 Z
11
+ date: 2019-01-08 00:00:00.000000000 Z
12
12
  dependencies:
13
+ - !ruby/object:Gem::Dependency
14
+ name: anyway_config
15
+ requirement: !ruby/object:Gem::Requirement
16
+ requirements:
17
+ - - "~>"
18
+ - !ruby/object:Gem::Version
19
+ version: '1.0'
20
+ type: :runtime
21
+ prerelease: false
22
+ version_requirements: !ruby/object:Gem::Requirement
23
+ requirements:
24
+ - - "~>"
25
+ - !ruby/object:Gem::Version
26
+ version: '1.0'
13
27
  - !ruby/object:Gem::Dependency
14
28
  name: dogapi
15
29
  requirement: !ruby/object:Gem::Requirement
@@ -120,6 +134,7 @@ files:
120
134
  - ".gitignore"
121
135
  - ".rspec"
122
136
  - ".rubocop.yml"
137
+ - CONTRIBUTING.md
123
138
  - Gemfile
124
139
  - Gemfile.lock
125
140
  - LICENSE.txt
@@ -131,10 +146,17 @@ files:
131
146
  - examples/script.rb
132
147
  - lib/yabeda/datadog.rb
133
148
  - lib/yabeda/datadog/adapter.rb
149
+ - lib/yabeda/datadog/config.rb
150
+ - lib/yabeda/datadog/exceptions.rb
151
+ - lib/yabeda/datadog/logging.rb
134
152
  - lib/yabeda/datadog/metric.rb
153
+ - lib/yabeda/datadog/response_handler.rb
135
154
  - lib/yabeda/datadog/tags.rb
136
155
  - lib/yabeda/datadog/units.rb
137
156
  - lib/yabeda/datadog/version.rb
157
+ - lib/yabeda/datadog/worker.rb
158
+ - lib/yabeda/datadog/worker/register.rb
159
+ - lib/yabeda/datadog/worker/send.rb
138
160
  - yabeda-datadog.gemspec
139
161
  homepage: https://github.com/shvetsovdm/yabeda-datadog
140
162
  licenses:
@@ -150,12 +172,12 @@ required_ruby_version: !ruby/object:Gem::Requirement
150
172
  requirements:
151
173
  - - ">="
152
174
  - !ruby/object:Gem::Version
153
- version: '0'
175
+ version: 2.3.0
154
176
  required_rubygems_version: !ruby/object:Gem::Requirement
155
177
  requirements:
156
- - - ">="
178
+ - - ">"
157
179
  - !ruby/object:Gem::Version
158
- version: '0'
180
+ version: 1.3.1
159
181
  requirements: []
160
182
  rubyforge_project:
161
183
  rubygems_version: 2.7.6