io_monitor 0.2.0 → 1.0.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: c6e067b726ccf26c1f0b71c6ee209c910f2ee0f7004d5aa552dfdc04276e7b89
-  data.tar.gz: 595c2ef011ddae63f5ec7e470065f163477dadbe187648fcdd9960661018972e
+  metadata.gz: 1ee7933c9976ebde6bcd8a84dd206a4020cd297edc96ad762ed9345b4a912a82
+  data.tar.gz: e17935f41005f34bda26cbc6021fa77b337ec876f0b3440a45a4b0085376a0fd
 SHA512:
-  metadata.gz: 69d47ffff8807be7991ccaa82471b085252634e7ac4db064ece0ec87ef02f3ae9d18a6ee833619d47912f8c0fc712d1b8762a86196a23a251724c20d5cd2bf6e
-  data.tar.gz: d57e63f51f61c9f86aa0683525f134cfb9be1670e95cab651b68dbbefff9af6e4b5b3486bafd0e607700ef110acb37bb95e9ba0dbb2af9cfe436204ea866f5fc
+  metadata.gz: 96ed1d20af1c8a20a101f04d389a9d9d5922c5bd9757a0460913f36347ba4a82c15abec0b6f0128079df44d3296d94d1e3e405a464883ae72a503d36c086e7dd
+  data.tar.gz: 4765a09692f9788c2fb394be13f161b9bce18a753bd3fa327874e00dcfc359b7ffd0d79f424dbbd6cb0d4df51e2e996f5cc48bf0107de8c2f498354264bc300f

data/CHANGELOG.md

@@ -2,6 +2,14 @@
 
 ## main
 
+## 1.0.0 (2023-05-06)
+
+- [PR#22](https://github.com/DmitryTsepelev/io_monitor/pull/22) Handle zero payload ([@DmitryTsepelev])
+- [PR#17](https://github.com/DmitryTsepelev/io_monitor/pull/17) Prometheus publisher ([@maxshend])
+- [PR#10](https://github.com/DmitryTsepelev/io_monitor/pull/10) Per–action monitoring ([@DmitryTsepelev])
+- [PR#15](https://github.com/DmitryTsepelev/io_monitor/pull/15) Allow configure more than one publisher ([@DmitryTsepelev])
+- [PR#9](https://github.com/DmitryTsepelev/io_monitor/pull/9) Restrict minimum Rails version to 6.1, adjust test matrix, and related changes ([@Envek])
+
 ## 0.2.0 (2022-05-29)
 
 - [PR#8](https://github.com/DmitryTsepelev/io_monitor/pull/8) Add Redis adapter ([@DmitryTsepelev])
@@ -17,3 +25,4 @@
 [@prog-supdex]: https://github.com/prog-supdex
 [@maxshend]: https://github.com/maxshend
 [@DmitryTsepelev]: https://github.com/DmitryTsepelev
+[@Envek]: https://github.com/Envek

data/README.md

@@ -1,5 +1,9 @@
 # IoMonitor
 
+[![Gem Version](https://badge.fury.io/rb/io_monitor.svg)](https://rubygems.org/gems/io_monitor)
+[![Tests status](https://github.com/DmitryTsepelev/io_monitor/actions/workflows/test.yml/badge.svg)](https://github.com/DmitryTsepelev/io_monitor/actions/workflows/test.yml)
+![](https://ruby-gem-downloads-badge.herokuapp.com/io_monitor?type=total)
+
 A gem that helps to detect potential memory bloats.
 
 When your controller loads a lot of data to the memory but returns a small response to the client it might mean that you're using the IO in the non–optimal way. In this case, you'll see the following message in your logs:
@@ -8,12 +12,6 @@ When your controller loads a lot of data to the memory but returns a small respo
 Completed 200 OK in 349ms (Views: 2.1ms | ActiveRecord: 38.7ms | ActiveRecord Payload: 866.00 B | Response Payload: 25.00 B | Allocations: 72304)
 ```
 
-<p align="center">
-  <a href="https://evilmartians.com/?utm_source=io_monitor">
-    <img src="https://evilmartians.com/badges/sponsored-by-evil-martians.svg" alt="Sponsored by Evil Martians" width="236" height="54">
-  </a>
-</p>
-
 ## Usage
 
 Add this line to your application's Gemfile:
@@ -28,7 +26,7 @@ Change configuration in an initializer if you need:
 
 ```ruby
 IoMonitor.configure do |config|
-  config.publish = :notifications # defaults to :logs
+  config.publish = [:logs, :notifications, :prometheus] # defaults to :logs
   config.warn_threshold = 0.8 # defaults to 0
   config.adapters = [:active_record, :net_http, :redis] # defaults to [:active_record]
 end
@@ -47,6 +45,31 @@ Depending on configuration when IO payload size to response payload size ratio r
 ```
 ActiveRecord I/O to response payload ratio is 0.1, while threshold is 0.8
 ```
+Prometheus metrics example:
+```
+...
+# TYPE io_monitor_ratio histogram
+# HELP io_monitor_ratio IO payload size to response payload size ratio
+io_monitor_ratio_bucket{adapter="active_record",le="0.01"} 0.0
+io_monitor_ratio_bucket{adapter="active_record",le="5"} 2.0
+io_monitor_ratio_bucket{adapter="active_record",le="10"} 2.0
+io_monitor_ratio_bucket{adapter="active_record",le="+Inf"} 2.0
+io_monitor_ratio_sum{adapter="active_record"} 0.15779381908414167
+io_monitor_ratio_count{adapter="active_record"} 2.0
+...
+```
+If you want to customize Prometheus publisher you can pass it as object:
+```ruby
+IoMonitor.configure do |config|
+  config.publish = [
+    IoMonitor::PrometheusPublisher.new(
+      registry: custom_registry, # defaults to Prometheus::Client.registry
+      aggregation: :max, # defaults to nil
+      buckets: [0.1, 5, 10] # defaults to Prometheus::Client::Histogram::DEFAULT_BUCKETS
+    )
+  ]
+end
+```
 
 In addition, if `publish` is set to logs, additional data will be logged on each request:
 
@@ -62,6 +85,18 @@ ActiveSupport::Notifications.subscribe("process_action.action_controller") do |n
 end
 ```
 
+## Per–action monitoring
+
+Since this approach can lead to false–positives or other things you don't want or cannot fix, there is a way to configure monitoring only for specific actions:
+
+```ruby
+class MyController < ApplicationController
+  include IoMonitor::Controller
+
+  monitor_io_for :index, :show
+end
+```
+
 ## Custom publishers
 
 Implement your custom publisher by inheriting from `BasePublisher`:
@@ -110,6 +145,10 @@ end
 
 After checking out the repo, run `bin/setup` to install dependencies. Then, run `rake spec` to run the tests.
 
+## Credits
+
+Initially sponsored by [Evil Martians](http://evilmartians.com).
+
 ## Contributing
 
 Bug reports and pull requests are welcome on GitHub at https://github.com/DmitryTsepelev/io_monitor.

data/lib/io_monitor/adapters/active_record_adapter.rb

@@ -17,7 +17,7 @@ module IoMonitor
         # but it makes a lot of unnecessary allocations.
         io_payload_size = rows.sum(0) do |row|
           row.sum(0) do |val|
-            (String === val ? val : val.to_s).bytesize
+            ((String === val) ? val : val.to_s).bytesize
           end
         end
 

data/lib/io_monitor/aggregator.rb

@@ -13,6 +13,12 @@ module IoMonitor
       InputPayload.active.present?
     end
 
+    def collect
+      start!
+      yield
+      stop!
+    end
+
     def start!
       InputPayload.active = true
     end

data/lib/io_monitor/configuration.rb

@@ -5,22 +5,15 @@ module IoMonitor
     DEFAULT_WARN_THRESHOLD = 0.0
 
     def initialize
-      @publisher = LogsPublisher.new
+      @publishers = [LogsPublisher.new]
       @adapters = [ActiveRecordAdapter.new]
       @warn_threshold = DEFAULT_WARN_THRESHOLD
     end
 
-    attr_reader :publisher, :adapters, :warn_threshold
+    attr_reader :publishers, :adapters, :warn_threshold
 
-    def publish=(value)
-      if value.is_a?(BasePublisher)
-        @publisher = value
-      elsif (publisher_type = resolve(IoMonitor::PUBLISHERS, value))
-        @publisher = publisher_type.new
-      else
-        supported = IoMonitor::PUBLISHERS.map(&:kind)
-        raise ArgumentError, "Only the following publishers are supported: #{supported}."
-      end
+    def publish=(values)
+      @publishers = [*values].map { |value| value_to_publisher(value) }
     end
 
     def adapters=(value)
@@ -50,5 +43,16 @@ module IoMonitor
     def resolve(list, kind)
       list.find { |p| p.kind == kind }
     end
+
+    def value_to_publisher(value)
+      if value.is_a?(BasePublisher)
+        value
+      elsif (publisher_type = resolve(IoMonitor::PUBLISHERS, value))
+        publisher_type.new
+      else
+        supported = IoMonitor::PUBLISHERS.map(&:kind)
+        raise ArgumentError, "Only the following publishers are supported: #{supported}."
+      end
+    end
   end
 end

data/lib/io_monitor/controller.rb

@@ -4,24 +4,47 @@ module IoMonitor
   module Controller
     extend ActiveSupport::Concern
 
-    def process_action(*)
-      IoMonitor.aggregator.start!
+    delegate :aggregator, to: IoMonitor
 
-      super
+    ALL_ACTIONS = Object.new
+
+    class_methods do
+      def monitor_io_for(*actions_to_monitor_io)
+        @actions_to_monitor_io = actions_to_monitor_io
+      end
 
-      IoMonitor.aggregator.stop!
+      def actions_to_monitor_io
+        @actions_to_monitor_io || ALL_ACTIONS
+      end
+    end
+
+    def process_action(*)
+      if monitors_action?(action_name)
+        aggregator.collect { super }
+      else
+        super
+      end
     end
 
     def append_info_to_payload(payload)
       super
 
+      return unless monitors_action?(action_name)
+
       data = payload[IoMonitor::NAMESPACE] = {}
 
-      IoMonitor.aggregator.sources.each do |source|
-        data[source] = IoMonitor.aggregator.get(source)
+      aggregator.sources.each do |source|
+        data[source] = aggregator.get(source)
       end
 
-      data[:response] = payload[:response]&.body&.bytesize || 0
+      data[:response] = payload[:response].body.bytesize
+    end
+
+    private
+
+    def monitors_action?(action_name)
+      actions = self.class.actions_to_monitor_io
+      actions == ALL_ACTIONS || actions.include?(action_name.to_sym)
     end
   end
 end

data/lib/io_monitor/patches/action_controller_base_patch.rb

@@ -4,7 +4,7 @@ module IoMonitor
   module ActionControllerBasePatch
     def log_process_action(payload)
       super.tap do |messages|
-        next unless IoMonitor.config.publisher.is_a?(LogsPublisher)
+        next unless IoMonitor.config.publishers.any? { |publisher| publisher.is_a?(LogsPublisher) }
 
         data = payload[IoMonitor::NAMESPACE]
         next unless data

data/lib/io_monitor/publishers/base_publisher.rb

@@ -16,15 +16,15 @@ module IoMonitor
       (payload.keys - [:response]).each do |source|
         ratio = ratio(payload[:response], payload[source])
 
-        if ratio < IoMonitor.config.warn_threshold
-          publish(source, ratio)
-        end
+        publish(source, ratio) if ratio < IoMonitor.config.warn_threshold
       end
     end
 
     private
 
     def ratio(response_size, io_size)
+      return 0 if io_size.to_f.zero?
+
       response_size.to_f / io_size.to_f
     end
   end

data/lib/io_monitor/publishers/prometheus_publisher.rb

@@ -0,0 +1,40 @@
+# frozen_string_literal: true
+
+module IoMonitor
+  class PrometheusPublisher < BasePublisher
+    HELP_MESSAGE = "IO payload size to response payload size ratio"
+
+    def initialize(registry: nil, aggregation: nil, buckets: nil)
+      registry ||= ::Prometheus::Client.registry
+      @metric = registry.histogram(
+        "#{IoMonitor::NAMESPACE}_ratio".to_sym,
+        labels: %i[adapter],
+        buckets: buckets || ::Prometheus::Client::Histogram::DEFAULT_BUCKETS,
+        store_settings: store_settings(aggregation),
+        docstring: HELP_MESSAGE
+      )
+    end
+
+    def self.kind
+      :prometheus
+    end
+
+    def publish(source, ratio)
+      metric.observe(ratio, labels: {adapter: source})
+    end
+
+    private
+
+    attr_reader :metric
+
+    # From https://github.com/yabeda-rb/yabeda-prometheus/blob/v0.8.0/lib/yabeda/prometheus/adapter.rb#L101
+    def store_settings(aggregation)
+      case ::Prometheus::Client.config.data_store
+      when ::Prometheus::Client::DataStores::Synchronized, ::Prometheus::Client::DataStores::SingleThreaded
+        {} # Default synchronized store doesn't allow to pass any options
+      when ::Prometheus::Client::DataStores::DirectFileStore, ::Object # Anything else
+        {aggregation: aggregation}.compact
+      end
+    end
+  end
+end

data/lib/io_monitor/railtie.rb

@@ -15,7 +15,7 @@ module IoMonitor
         payload = args.last[IoMonitor::NAMESPACE]
         next unless payload
 
-        IoMonitor.config.publisher.process_action(payload)
+        IoMonitor.config.publishers.each { |publisher| publisher.process_action(payload) }
       end
     end
   end

data/lib/io_monitor/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module IoMonitor
-  VERSION = "0.2.0"
+  VERSION = "1.0.0"
 end

data/lib/io_monitor.rb

@@ -12,6 +12,7 @@ require "io_monitor/adapters/net_http_adapter"
 require "io_monitor/publishers/base_publisher"
 require "io_monitor/publishers/logs_publisher"
 require "io_monitor/publishers/notifications_publisher"
+require "io_monitor/publishers/prometheus_publisher"
 
 require "io_monitor/railtie"
 
@@ -26,7 +27,7 @@ module IoMonitor
   end
   ADAPTERS = adapters.freeze
 
-  PUBLISHERS = [LogsPublisher, NotificationsPublisher].freeze
+  PUBLISHERS = [LogsPublisher, NotificationsPublisher, PrometheusPublisher].freeze
 
   class << self
     def aggregator

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: io_monitor
 version: !ruby/object:Gem::Version
-  version: 0.2.0
+  version: 1.0.0
 platform: ruby
 authors:
 - baygeldin
@@ -11,7 +11,7 @@ authors:
 autorequire:
 bindir: bin
 cert_chain: []
-date: 2022-05-29 00:00:00.000000000 Z
+date: 2023-05-06 00:00:00.000000000 Z
 dependencies:
 - !ruby/object:Gem::Dependency
   name: rails
@@ -19,14 +19,14 @@ dependencies:
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '6.0'
+        version: '6.1'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - ">="
       - !ruby/object:Gem::Version
-        version: '6.0'
+        version: '6.1'
 - !ruby/object:Gem::Dependency
   name: redis
   requirement: !ruby/object:Gem::Requirement
@@ -41,6 +41,20 @@ dependencies:
     - - ">="
       - !ruby/object:Gem::Version
         version: '4.0'
+- !ruby/object:Gem::Dependency
+  name: prometheus-client
+  requirement: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
+  type: :development
+  prerelease: false
+  version_requirements: !ruby/object:Gem::Requirement
+    requirements:
+    - - ">="
+      - !ruby/object:Gem::Version
+        version: '0'
 description:
 email:
 - dmitry.a.tsepelev@gmail.com
@@ -67,6 +81,7 @@ files:
 - lib/io_monitor/publishers/base_publisher.rb
 - lib/io_monitor/publishers/logs_publisher.rb
 - lib/io_monitor/publishers/notifications_publisher.rb
+- lib/io_monitor/publishers/prometheus_publisher.rb
 - lib/io_monitor/railtie.rb
 - lib/io_monitor/version.rb
 homepage: https://github.com/DmitryTsepelev/io_monitor
@@ -86,14 +101,14 @@ required_ruby_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
-      version: 2.6.0
+      version: 2.7.0
 required_rubygems_version: !ruby/object:Gem::Requirement
   requirements:
   - - ">="
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubygems_version: 3.2.33
+rubygems_version: 3.2.15
 signing_key:
 specification_version: 4
 summary: A gem that helps to detect potential memory bloats