neeto-monitor-ruby 1.0.42

checksums.yaml

@@ -0,0 +1,7 @@
+---
+SHA256:
+  metadata.gz: adb80970d221b0ba7554aefd49426eb7018de773f7148200ba1fe588f4ccbc20
+  data.tar.gz: 7b080f2c0979e732dafa40243b63d56c23c4ccf6636f21a14959bf062cd9bb62
+SHA512:
+  metadata.gz: a9e3eafe53eb9b3a117926a0c8b1d59e83816a86156888212827adc949f463f0efe35a01496e7a3d98c61a8c93ee86daf6df44ac2318055d63d8d76f40981b00
+  data.tar.gz: b07f078fae394db5cad93fde00a8871b24535006dc803ac2404f8584a964be0355d099fb6f9ce586d3b07405c98aa2183630f22ab144fc5f902f00a60ccfdd31

data/LICENSE

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2022 BigBinary
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

data/README.md

@@ -0,0 +1,288 @@
+# neeto-monitor-ruby
+
+neetoMonitor - A ruby gem for monitoring application's uptime and performance.
+
+Installation:
+
+##### 1. Add the gem
+
+```bash
+gem "neeto-monitor-ruby", git: "https://github.com/bigbinary/neeto-monitor-ruby.git"
+```
+
+##### 2. Install the gems
+
+```bash
+bundle install
+```
+
+##### 3. Add configuration
+
+The neetoMonitor can be configured at multiple levels. i.e. using
+`env variables`, adding a `config file` or manually passing the configuration
+hash to `NeetoMonitorRuby.init!`.
+
+###### 3.1 Setting the config via `NeetoMonitorRuby.init!`
+
+```ruby
+NeetoMonitorRuby.init!({
+  api_key: your_app_api_key,
+  environment: production
+
+  ...
+
+})
+```
+
+###### 3.2 Using a config file
+
+```yaml
+# config/neetomonitor.yml
+---
+api_key: [API_KEY]
+ping_timeout: 5
+```
+
+Following are the available configuration options with their default values.
+
+```yaml
+base_url: (default: https://neetomonitor.com)
+# The monitor server base URL. In `development` or `staging` we can set this to local/staging URL.
+
+api_key: (default: nil)
+# A unique api key for an organization.
+
+environment: (default: nil)
+# The monitoring environment name for an application
+
+ping_timeout: (default: 8 seconds)
+# The request timeout for ping requests.
+
+sidekiq_enabled:
+# Enables sidekiq plugin to report worker/job events.
+
+config_path: (default: "config/neetomonitor.yml")
+# The config file path
+
+key_prefix_enabled: (default: true)
+# This allows to enable/disable prefix for monitor key.
+
+key_prefix: (default: application's name e.g. NeetoKb if integrated in neetoKb)
+# The prefix added to the monitor key. e.g. if the check name is ArticleWorker, the result will be NeetoKb::ArticleWorker.
+
+monitors:
+  # more on this in next section
+```
+
+###### 3.3 Using ENV variables:
+
+The `ENV` variables must start with `NEETO_MONITOR` e.g.
+`NEETO_MONITOR_API_KEY`.
+
+Similarly, other ENV variables can also be set by prefixing with
+`NEETO_MONITOR_`.
+
+The precedence of the configuration is as follows:
+
+```
+Options passed to NeetoMonitorRuby.init! > Config file > ENV variables > default values
+```
+
+##### 4. Configuring monitors
+
+The monitors can be added in bulk using a config file. e.g.
+
+```yaml
+  api_key: [API_KEY]
+  monitors:
+    checks:
+      - name: "neeto-monitor"
+        request_attributes:
+          endpoint: "http://app.neetomonitor.test/health_check/"
+          kind: "http"
+          verb: "get"
+          interval: 10
+          timeout: 20
+        assertions_attributes:
+          - kind: "response_code"
+            verb: "equal"
+            value: 200
+    jobs:
+      - name: "neeto-job"
+        assertions_attributes:
+        - kind: max_duration
+          verb: less_than
+          value: 10
+      - name: "hard-job",
+        schedule: "* * * * *"
+    heartbeats:
+      - name: "neeto-heartbeat"
+        schedule: "* * * * *"
+```
+
+The above config will create total four monitors i.e. 2 jobs and 1 check and 1
+heartbeat.
+
+###### 4.1 Checks monitors (Checks)
+
+    The checks are used for monitoring website's uptime. The website requests are made at regular intervals which is configured via `interval` attribute.
+
+```yaml
+name: # The name of check monitor e.g. "neeto-auth"
+request_attributes: # This specifies configuring options for the website request.
+  endpoint: # The request URL endpoint
+
+  kind: # The request protocol
+
+  verb: # The request verb/method i.e. get/post/head etc.
+
+  interval: # The request frequency
+
+  timeout: # The request timeout
+
+assertions_attributes: # This specifies the assertion to be made when a request is made.
+  kind:# The kind of assertion to be made
+    # available options => response_code, response_time, response_body, response_json, response_header, ssl_expiry, min_duration, max_duration
+
+  verb:# The operation verb for assertion
+    # available options => less_than, less_than_equal, equal, not_equal, greater_than, greater_than_equal, contains, not_contains
+
+  key:# The key from the response body json or headers
+    # e.g. if response body = { "success": true }, key can be "success"
+
+  value: # Any value to be checked for the assertion
+```
+
+###### 4.2 Heartbeat monitors (Heartbeats)
+
+The heartbeats are used for monitoring the website/service uptime. The basic
+difference between `checks` and `heartbeats` is that the heartbeats requests are
+sent from the host application. Where as the checks requests are made from the
+`neetoMonitor` server application.
+
+The heartbeats can be added via the config file. When it is mentioned in the
+config, the heartbeats will be created in neetoMonitor upon the app
+initialization.
+
+When `schedule` is defined for the heartbeats, the gem will send the heartbeats
+using the schedule frequency and neetoMonitor can send alerts when heartbeats are not received in this frequency.
+
+The heartbeats events are sent using the `HeartbeatRunnerJob`. This sends the ping at time intervals based on the schedule. E.g. following heartbeat sends a ping every minute.
+
+```ruby
+monitors:
+  heartbeats:
+    - name: "neeto-heartbeat"
+      schedule: "* * * * *" # heartbeat frequency
+```
+
+  **Trigger heartbeats from application manually**
+    The heartbeats can be triggered manually using a periodic job. If you are using `sidekiq`, you can use `sidekiq-cron` to trigger heartbeats at desired frequency.
+
+  Here is an example of how you can configure heartbeats manually.
+
+  ```ruby
+    # Worker for sending heartbeat
+    class HeartbeatsWorker < BaseWorker
+      sidekiq_options neeto_monitor_disabled: true
+
+      def perform
+        super
+
+        NeetoMonitor::Monitor.new("test-heartbeat").ping
+      end
+    end
+
+    # config/schedule.yml
+    heartbeat:
+      cron: "* * * * *"
+      class: "HeartbeatsWorker"
+  ```
+###### 4.3 Job monitors (Jobs)
+
+The `jobs` are used for checking reliability and performance of background
+workers.
+
+All the monitors can be added either via the web application or from the
+mentioned config file.
+
+The `jobs` can be created by other methods as well i.e. without adding `jobs` to
+config section or web application UI. This is done using `Sidekiq` plugin
+included in the gem. When any new worker starts execution e.g. `ArticleJob`, the
+job monitor's `run` event is sent automatically. The `complete` event is sent
+when the job execution is successfully completed. Similarly, `fail` event is
+sent when job execution fails. This `ArticleJob` monitor is created at
+`neetoMonitor` application if it is not created already. If we want `schedule`
+field i.e. periodic job events to be sent to neetoMonitor application, we must
+add such jobs to a config file. e.g.
+
+```yaml
+monitors:
+  jobs:
+    - name: "NeetoWorker"
+      schedule: "* * * * *" # frequency of periodic jobs
+    - name: "neeto-job"
+      assertions_attributes:
+        - kind:
+          # available values => max_duration, min_duration
+          verb:
+          # available values => less_than_or_equal, less_than, greater_than, greater_than_or_equal, equal
+          value:
+          # allowed values => integer/float value
+```
+
+The job events are automatically sent from the `sidekiq` plugin when
+`sidekiq_enabled` is set to true. This flag is set to `true` by default. The job
+events are not sent when this plugin is not enabled. The gem currently supports
+only `Sidekiq` for background job monitoring. If we are using any other library
+for background jobs, we can send the events manually. E.g.
+
+```ruby
+  class ArticleJob
+    ..
+
+    ...
+
+    def perform
+      job_monitor = NeetoMonitorRuby::Monitor.new("ArticleJob")
+      series_stamp = Time.now.utc.to_f
+      job_monitor.job_ping(state: "run", series: series_stamp)
+
+      process_article_job # job business logic
+
+      job_monitor.job_ping(state: "complete", series: series_stamp)
+    rescue StandardError => error
+      job_monitor.job_ping(state: "fail", series: series_stamp, message: error.message)
+    end
+
+    private
+
+    ..
+  end
+```
+
+Sometimes we don't want to monitor some `Sidekiq` workers. We have an option to
+disable the monitoring for such workers.
+
+```ruby
+class MonitorDisabledWorker
+  include Sidekiq::Worker
+  sidekiq_options neeto_monitor_disabled: true
+
+  ...
+
+end
+```
+
+The default `monitor key` for `jobs` are the class name of the `Sidekiq` worker
+itself. We can change the `key` of the monitored job by setting
+`neeto_monitor_key` in `sidekiq_options`.
+
+```ruby
+class CustomMonitorWorker
+  include Sidekiq::Worker
+  sidekiq_options neeto_monitor_key: "super_worker"
+
+  ...
+end
+```

data/Rakefile

@@ -0,0 +1,14 @@
+# frozen_string_literal: true
+
+require "bundler/setup"
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.pattern = "test/**/*_test.rb"
+  t.verbose = false
+  t.warning = false
+end
+
+task default: :test

data/lib/neeto-monitor-ruby.rb

@@ -0,0 +1,7 @@
+# frozen_string_literal: true
+
+if defined?(::Rails::Railtie)
+  require "neeto_monitor_ruby/init/rails"
+else
+  require "neeto_monitor_ruby/init/ruby"
+end

data/lib/neeto_monitor_ruby/configuration.rb

@@ -0,0 +1,92 @@
+# frozen_string_literal: true
+
+require "yaml"
+
+module NeetoMonitorRuby
+  class ConfigurationError < StandardError; end
+
+  class Configuration
+    BASE_URL = "https://neetomonitor.com".freeze
+    DEFAULT_PING_TIMEOUT = 8
+    DEFAULT_CONFIG_PATH = "config/neetomonitor.yml"
+
+    class << self
+      attr_accessor :config
+
+      def configure!(options = {}, reset = false)
+        return config if !config.nil? && !reset
+
+        reset! if reset
+
+        config_options = options.dup
+
+        config_path = if options[:config_path]
+          options[:config_path]
+        else
+          file_path = ENV.fetch("NEETO_MONITOR_CONFIG_PATH", DEFAULT_CONFIG_PATH)
+
+          defined?(Rails) && Rails.respond_to?(:root) ? Rails.root.join(file_path) : file_path
+        end
+
+        config_options = load_config(config_path).merge(config_options) if config_path && load_config?(config_path)
+
+        self.config = Configuration.new(config_options)
+        config
+      end
+
+      def reset!
+        self.config = Configuration.new
+
+        config
+      end
+
+      def load_config?(config_path)
+        File.exist?(config_path)
+      end
+
+      def load_config(file_path)
+        config = YAML.load(ERB.new(File.read(file_path)).result(binding))
+
+        JSON.parse(JSON[config], symbolize_names: true)
+
+      rescue Errno::ENOENT
+        raise ConfigurationError.new("#{file_path} not found")
+      rescue Psych::SyntaxError => e
+        raise ConfigurationError.new("#{file_path} is malformed: #{e.message}")
+      end
+    end
+
+    attr_accessor :base_url, :api_key, :environment, :ping_timeout, :logger, :config_path, :monitors,
+      :sidekiq_enabled, :key_prefix_enabled, :key_prefix
+
+    attr_reader :options
+
+    def initialize(options = {})
+      @options = options
+      @base_url = fetch_or_set_value(:base_url, BASE_URL)
+      @api_key = fetch_or_set_value(:api_key)
+      @environment = fetch_or_set_value(:environment)
+      @ping_timeout = fetch_or_set_value(:ping_timeout, DEFAULT_PING_TIMEOUT)
+      @logger = Logger.new($stdout)
+      @sidekiq_enabled = fetch_or_set_value(:sidekiq_enabled, true)
+      @config_path = fetch_or_set_value(:config_path)
+      @monitors = options[:monitors]
+      @key_prefix_enabled = fetch_or_set_value(:key_prefix_enabled, true)
+      @key_prefix = fetch_or_set_value(:key_prefix, app_key_prefix)
+    end
+
+    private
+
+      def fetch_or_set_value(key, default_value = nil)
+        env_or_default_value = ENV.fetch("NEETO_MONITOR_#{key.to_s.upcase}", default_value)
+
+        options.fetch(key, env_or_default_value)
+      end
+
+      def app_key_prefix
+        if defined?(Rails) && Rails.respond_to?(:application)
+          Rails.application.class.to_s.split("::").first
+        end
+      end
+  end
+end

data/lib/neeto_monitor_ruby/heartbeat_runner_job.rb

@@ -0,0 +1,22 @@
+# frozen_string_literal: true
+
+require "fugit"
+require "active_job"
+
+module NeetoMonitorRuby
+  class HeartbeatRunnerJob < ActiveJob::Base
+    queue_as :default
+
+    def perform(heartbeat_name, schedule)
+      NeetoMonitorRuby::Monitor.new(heartbeat_name).ping
+
+      fugit = Fugit.do_parse_cronish(schedule)
+      next_time_string = fugit.next_time(Time.now).to_s
+      next_time = Time.parse(next_time_string)
+
+      NeetoMonitorRuby::HeartbeatRunnerJob.set(wait_until: next_time).perform_later(heartbeat_name, schedule)
+    rescue Date::Error, ArgumentError => exception
+      NeetoMonitor.logger.error "Error while parsing fugit date time: #{error.message}"
+    end
+  end
+end

data/lib/neeto_monitor_ruby/init/common.rb

@@ -0,0 +1,13 @@
+# frozen_string_literal: true
+
+require "logger"
+require "json"
+require "httparty"
+require "socket"
+require "time"
+require "yaml"
+
+require "neeto_monitor_ruby/version"
+require "neeto_monitor_ruby/monitor_utils"
+require "neeto_monitor_ruby/monitor"
+require "neeto_monitor_ruby/neeto_monitor"

data/lib/neeto_monitor_ruby/init/rails.rb

@@ -0,0 +1,27 @@
+# frozen_string_literal: true
+
+require "rails"
+
+require_relative "common"
+
+module NeetoMonitorRuby
+  module Init
+    module Rails
+      class Railtie < ::Rails::Railtie
+        config.before_initialize do
+          NeetoMonitorRuby.init!(
+            {
+              environment: ::Rails.env,
+              logger: ::Rails.logger,
+              config_path: ::Rails.root.join(Configuration::DEFAULT_CONFIG_PATH)
+            })
+        end
+
+        config.after_initialize do
+          NeetoMonitorRuby.load_plugins!
+          NeetoMonitorRuby.load_monitors!
+        end
+      end
+    end
+  end
+end

data/lib/neeto_monitor_ruby/init/ruby.rb

@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+require_relative "common"
+
+NeetoMonitorRuby.init!(
+  {
+    environment: ENV["RUBY_ENV"] || ENV["RACK_ENV"]
+  })
+
+NeetoMonitorRuby.load_plugins!

data/lib/neeto_monitor_ruby/monitor.rb

@@ -0,0 +1,158 @@
+# frozen_string_literal: true
+
+require "httparty"
+require "sidekiq/api"
+require_relative "heartbeat_runner_job"
+
+module NeetoMonitorRuby
+  class Monitor
+    include MonitorUtils
+
+    REQUEST_HEADERS = {
+      'Content-Type': "application/json",
+      'Accept': "application/json",
+      'User-Agent': "neeto-monitor-ruby"
+    }.freeze
+
+    MAX_RETRY_COUNT = 3
+    BULK_CREATE_PATH = "/api/v1/clients/bulk_checks".freeze
+
+    class << self
+      def load_monitors!
+        return unless NeetoMonitorRuby.config.api_key
+
+        config_monitors = NeetoMonitorRuby.config.monitors&.except(:heartbeats)
+
+        if !config_monitors.blank?
+          environment = NeetoMonitorRuby.config.environment
+
+          all_monitors = config_monitors.each_with_object([]) do |(type, monitors), result|
+            monitors.each { |m| result << m.merge(kind: type.to_s.chop, environment:) }
+          end
+
+          make_bulk_create_request({ checks: all_monitors, api_key: NeetoMonitorRuby.config.api_key })
+        else
+          NeetoMonitorRuby.logger.debug "No monitors found in config"
+        end
+      end
+
+      def make_bulk_create_request(check_params)
+        bulk_monitors_url = "#{NeetoMonitorRuby.config.base_url}#{BULK_CREATE_PATH}"
+
+        response = HTTParty.post(
+          bulk_monitors_url,
+          body: check_params.to_json,
+          headers: REQUEST_HEADERS
+        )
+
+        result = response.code >= 200 && response.code < 300
+
+        NeetoMonitor.logger.error "Bulk create response code: #{response.code}" unless result
+
+        load_heartbeats! if defined?(ActiveJob::Base)
+
+        result
+      rescue StandardError => exception
+        NeetoMonitor.logger.error exception.message
+        NeetoMonitor.logger.error exception.backtrace.join("\n")
+      end
+
+      def load_heartbeats!
+        NeetoMonitor.logger.debug "Loading heartbeats!!!"
+        NeetoMonitor.logger.debug "NeetoMonitor.config.monitors #{NeetoMonitor.config.monitors}"
+        heartbeats = NeetoMonitor.config.monitors[:heartbeats]
+        prefix = NeetoMonitor.config.key_prefix_enabled ? NeetoMonitor.config.key_prefix : nil
+
+        heartbeats.each do |heartbeat|
+          heartbeat_name = prefix ? [prefix, heartbeat[:name]].join("::") : heartbeat[:name]
+
+          delete_old_heartbeat_jobs!
+
+          HeartbeatRunnerJob.perform_later(heartbeat_name, heartbeat[:schedule])
+        end
+      end
+
+      def delete_old_heartbeat_jobs!
+        if ActiveJob::Base.queue_adapter.respond_to?(:enqueued_jobs)
+          ActiveJob::Base.queue_adapter.enqueued_jobs.reject! do |job|
+            job["job_class"] == "NeetoMonitorRuby::HeartbeatRunnerJob"
+          end
+        elsif ActiveJob::Base.queue_adapter.class == ActiveJob::QueueAdapters::SidekiqAdapter
+          Sidekiq::ScheduledSet.new.scan("NeetoMonitorRuby::HeartbeatRunnerJob").each do |job|
+            job.delete if job.display_class == "NeetoMonitorRuby::HeartbeatRunnerJob"
+          end
+        end
+      end
+    end
+
+    attr_reader :api_key, :environment, :monitor_key
+
+    def initialize(monitor_key, api_key: nil, environment: nil)
+      @monitor_key = monitor_key
+      @api_key = api_key || NeetoMonitorRuby.config.api_key
+      @environment = environment || NeetoMonitorRuby.config.environment
+    end
+
+    def ping(params = {})
+      if api_key.nil?
+        log_error "No API key is provided, please pass the api_key or configure."
+        return false
+      end
+
+      retry_count = params[:retry_count] || 0
+      response = make_ping_request(params)
+      response.code >= 200 && response.code < 300
+    rescue StandardError => exception
+      return false if retry_count >= MAX_RETRY_COUNT
+
+      retry_ping(params, retry_count)
+    end
+
+    def job_ping(params)
+      ping(params.merge(kind: MONITOR_TYPES[:job]))
+    end
+
+    private
+
+      def make_ping_request(params)
+        response = HTTParty.get(
+          monitor_ping_url,
+          query: sanitize_params(params),
+          headers: REQUEST_HEADERS,
+          timeout: NeetoMonitorRuby.config.ping_timeout
+        )
+      end
+
+      def retry_ping(params, retry_count)
+        retry_count += 1
+        sleep(retry_count)
+        ping(params.merge(retry_count:))
+      end
+
+      def monitor_ping_url
+        "#{NeetoMonitorRuby.config.base_url}/tm/#{api_key}/#{CGI.escape(monitor_key)}"
+      end
+
+      def sanitize_params(params)
+        {
+          state: params.fetch(:state, nil),
+          kind: params.fetch(:kind, MONITOR_TYPES[:heartbeat]),
+          message: params.fetch(:message, nil),
+          series: params.fetch(:series, nil),
+          host: params.fetch(:host, Socket.gethostname),
+          metrics: params.fetch(:metrics, nil),
+          stamp: generate_stamp,
+          env: params.fetch(:env, environment)
+        }
+      end
+
+      def log_exception(exception)
+        log_error exception.message
+        log_error exception.backtrace.join("\n")
+      end
+
+      def log_error(text)
+        NeetoMonitorRuby.logger.error text
+      end
+  end
+end