closeyourit-ruby 0.10.2 → 0.12.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 866fd0b6c34af299e0e538ddb181b1f6b3fd961da1e638226a81ad8b8ea15c0b
-  data.tar.gz: eaf0d4530e9015f7353790aeffe104f04a2f80d46c4c7622ad7dad3302bd75fe
+  metadata.gz: e8e0fc4eed9289e8fbb3ac2b93e6bcf71c950200b24483be93ca9b03cbb5edf5
+  data.tar.gz: bec0fc92de3a19b85caa500e41953726a1693ca8f757cd98022b8b5287e83f76
 SHA512:
-  metadata.gz: b4193d36f1123b89bdf11998ebcd5ea445b0b84c325ccae654d434d2d94158d6dd9c920b0e879167f0fba5cbaa7615f42a3d2cd11f3f61d8ade9f3eaee18917f
-  data.tar.gz: 7a4d366fbf67b5720d878e2e1febbcda5500c6063d8f654e450c10b68789964a7c8b15390e23d4516d9220ea6cb2ffbde619900ec2871fe7b8b20fd23c967b8b
+  metadata.gz: f39ecd3584260bf7257fb8efc7fcf12222a2adec2fa2d1ba7ff3a59e368ae6c6e52921d5c83dda516c588946d718c4407aa934d6f495cfd9269621226fd58555
+  data.tar.gz: a634e8af76d32b12d1ffe210e73d3070c8059d15a55953caed9406d2c8db7fbeaac0f34bf800ee049bf9b4cdcb737d0239f3968978b970bb1175f42dfca3f0c5

data/README.md

@@ -52,7 +52,7 @@ In CloseYourIt, area **Member → Project → tokens**, crea un token: ottieni i
 ```ruby
 # config/initializers/closeyourit.rb
 CloseYourIt.init do |c|
-  c.endpoint_url = ENV["CLOSEYOURIT_ENDPOINT_URL"]   # es. https://www.closeyour.it (host canonico)
+  c.endpoint_url = ENV["CLOSEYOURIT_ENDPOINT_URL"]   # es. https://app.closeyour.it (host canonico)
   c.token        = ENV["CLOSEYOURIT_TOKEN"]          # Bearer secret del Projects::Token
   c.project_id   = ENV["CLOSEYOURIT_PROJECT_ID"]     # UUID del progetto su CloseYourIt
   c.environment  = Rails.env
@@ -62,9 +62,8 @@ end
 **Senza `endpoint_url` / `token` / `project_id` la gemma è no-op** (nessun invio, nessun overhead). In
 `production` un `endpoint_url` `http://` viene rifiutato (no-op + warning): il token viaggerebbe in chiaro.
 
-> **Endpoint:** usa l'host canonico **`https://www.closeyour.it`**. La gemma segue fino a 2 redirect su POST
-> (preservando metodo e body), quindi anche l'apex `https://closeyour.it` (301 → www) funziona — ma puntare
-> direttamente a www risparmia un hop per ogni evento.
+> **Endpoint:** usa l'host canonico **`https://app.closeyour.it`**. Le installazioni già configurate su
+> `https://www.closeyour.it` continuano a funzionare: l'ingest su `www` resta inoltrato all'app.
 
 ### Opzioni
 
@@ -84,7 +83,9 @@ end
 | `slow_query_threshold_ms` | `100` | Soglia query lente |
 | `excluded_query_patterns` | `solid_queue_`, `solid_cache_`, `solid_cable_` | Query da NON misurare come rallentamento — **Regexp** (pattern) o **String** (testo letterale). Vale solo per la misura: breadcrumb e profiling N+1 vedono tutto |
 | `slow_method_threshold_ms` | `200` | Soglia metodi lenti |
+| `capture_method_arguments` | `false` | Allega gli argomenti di `measure`/`monitor` ai `slow_method`. **Attenzione**: gli argomenti sono spesso dati personali; la gemma redige chiavi sensibili, credenziali e `scrub_message_patterns`, non il resto |
 | `capture_request` | `true` | Cattura il contesto HTTP della richiesta (method/url/header allowlist) |
+| `capture_request_body` | `true` | Allega i parametri del body (`request.data`) all'errore, scrubbati. Vale **solo con `send_pii = true`**: con la privacy spenta il body non parte mai |
 | `request_header_allowlist` | `Accept`, `Content-Type`, `User-Agent`, `Referer` | Header inviati (mai Authorization/Cookie) |
 | `breadcrumbs_enabled` | `true` | Cronologia (query offuscate + `add_breadcrumb`) allegata all'errore |
 | `max_breadcrumbs` | `100` | Dimensione max del ring buffer breadcrumbs |
@@ -92,7 +93,7 @@ end
 | `report_active_job_errors` | `true` | Cattura gli errori dei job ActiveJob/Solid Queue/Sidekiq |
 | `send_pii` | `false` | Master switch PII |
 | `obfuscate_sql` | `true` | Maschera i literal nello SQL |
-| `filter_parameters` | `[]` | Chiavi extra da redarre (mergiate con quelle di Rails) |
+| `filter_parameters` | `[]` | Chiavi extra da redarre, in aggiunta a `Rails.application.config.filter_parameters` (unite automaticamente dalla Railtie) |
 | `scrub_message_patterns` | `[]` | Regexp **aggiuntive** da redarre da messaggi d'eccezione **e da `message` dei log**; le credenziali `Bearer`/`Basic`/`Token` sono già redatte di serie |
 | `logs_enabled` | `true` | Master switch dello stream di log strutturato |
 | `logs_min_level` | `:info` | Soglia minima: i log sotto (es. `debug`) non vengono inviati |
@@ -109,6 +110,7 @@ end
 | `slow_request_threshold_ms` | `1000` | Durata totale della richiesta (ms) oltre cui = `slow_request` |
 | `slow_external_threshold_ms` | `1000` | Durata di una singola HTTP esterna (ms) oltre cui = `slow_external_http` |
 | `capture_external_http` | `true` | Strumenta `Net::HTTP` per rilevare le HTTP esterne (effettivo solo con `detect_performance_issues`) |
+| `job_lifecycle_logs` | `false` | Log informativo per esito di tentativo osservato, opt-in. |
 | `monitor_jobs` | `true` | Misura durata e attesa in coda dei background job (ActiveJob + Sidekiq) → metriche `slow_job`/`job_queue_latency` (vedi [Metriche dei background job](#metriche-dei-background-job)) |
 | `slow_job_threshold_ms` | `5000` | Durata del job (ms) oltre cui = `slow_job` |
 | `job_queue_latency_threshold_ms` | `60000` | Attesa in coda (enqueue→esecuzione, ms) oltre cui = `job_queue_latency` |
@@ -154,7 +156,9 @@ end
 
 Un Rack middleware allega a ogni evento il contesto HTTP della richiesta: **method**, **url** (senza
 query string) e gli **header dell'allowlist** (`request_header_allowlist`, mai Authorization/Cookie).
-Query string e IP solo con `send_pii`. Lo scope è resettato a fine richiesta (niente bleed tra request).
+Query string, IP e **parametri del body** (`request.data`, scrubbati, estratti solo quando scoppia
+l'errore; `capture_request_body` li spegne) solo con `send_pii`. Lo scope è resettato a fine richiesta
+(niente bleed tra request).
 
 ### Background job + errori handled (automatico, con Rails)
 
@@ -311,7 +315,10 @@ end
 ```
 
 Viene inviato un `slow_method` solo se la durata supera `slow_method_threshold_ms`. **Gli argomenti
-del metodo non vengono mai inviati** (solo label, durata, file:riga).
+del metodo non vengono inviati** (solo label, durata, file:riga) salvo `capture_method_arguments = true`:
+in quel caso partono con `inspect` troncato, redatti per chiave sensibile (kwargs), credenziali
+`Bearer`/`Basic`/`Token` e `scrub_message_patterns`. Tutto il resto arriva in chiaro: accendilo solo
+su metodi che non ricevono dati personali.
 
 ## Performance issues (opt-in)
 
@@ -359,7 +366,7 @@ sulla pipeline metriche (`/api/v1/projects/:id/metrics`). Funziona con **ActiveJ
 Due `subtype`:
 
 - **`slow_job`** — la durata del `perform` supera `slow_job_threshold_ms` (default 5000 ms).
-- **`job_queue_latency`** — l'attesa in coda (dall'enqueue all'inizio dell'esecuzione) supera
+- **`job_queue_latency`** — l'attesa dopo la scadenza effettiva (`max(enqueued_at, scheduled_at)`) supera
   `job_queue_latency_threshold_ms` (default 60000 ms): coda intasata o worker insufficienti.
 
 Ogni metrica porta la **label** (nome della classe del job), la **queue**, l'**adapter**
@@ -380,6 +387,20 @@ CloseYourIt.init do |c|
 end
 ```
 
+Il contesto additivo `contexts.job` usa il contratto `schema_version: 1`: separa durata monotona,
+attesa in coda e ritardo programmato. Un orologio mancante resta `null`; un'attesa negativa viene
+azzerata con `clock_skew: true`. Sidekiq 8 usa timestamp in millisecondi, le versioni precedenti
+secondi: la conversione segue la versione del framework, senza euristiche sulla grandezza.
+
+Per osservare anche gli esiti dei tentativi veloci, abilita `c.job_lifecycle_logs = true` (default
+`false`). Emette un log `info` con messaggio fisso `Background job attempt finished` e
+`attributes.contexts.job`, senza argomenti, risultati o header. `retry` non è terminale;
+`discarded` è esplicito; un errore con retry dell'adapter non noto ha `terminal: null`.
+Un `rescue_from` applicativo non identificabile resta `unknown`, non diventa successo.
+ActiveJob dentro Sidekiq è osservato una sola volta quando il subscriber ActiveJob è installato.
+Sampling, disabilitazione o perdita del transport impediscono di considerare questi log un censimento.
+Il raggruppamento backend v1 separa framework, nome, coda e misura; i gruppi legacy restano intatti.
+
 ## Privacy & PII
 
 Privacy-by-default (`send_pii = false`). In sintesi:
@@ -391,8 +412,10 @@ Privacy-by-default (`send_pii = false`). In sintesi:
   (`Authorization: Bearer …`, `Basic …`, `Token …`) sono redatte **di serie**, senza configurazione:
   chiave e schema restano leggibili, sparisce il valore → `Authorization: Bearer [FILTERED]`. Il resto
   è redigibile con `scrub_message_patterns` / `before_send`.
-- **Mai inviati**: variabili locali dei frame, argomenti dei metodi, IP/cookie/Authorization, token (che
-  viaggia solo nell'header su HTTPS).
+- **Solo con `send_pii`**: query string, IP, email/username dell'utente e parametri del body
+  (`capture_request_body`). Con la privacy spenta non partono, qualunque sia il resto della configurazione.
+- **Mai inviati**: variabili locali dei frame, cookie/Authorization, token (che viaggia solo nell'header
+  su HTTPS). Gli argomenti dei metodi solo con `capture_method_arguments` (opt-in, vedi sopra).
 
 > Rischio residuo: `exception.value` e i nomi tabella/colonna nello SQL possono contenere dati di
 > dominio. Usa `before_send`/`scrub_message_patterns` per azzerarli. Dettaglio in [`PDR.md` §9](PDR.md).

data/lib/closeyourit/configuration.rb

@@ -41,7 +41,7 @@ module CloseYourIt
                   :query_time_threshold_ms, :slow_request_threshold_ms, :slow_external_threshold_ms,
                   :capture_external_http, :trap_signals,
                   :monitor_jobs, :slow_job_threshold_ms, :job_queue_latency_threshold_ms,
-                  :jobs_sample_rate, :propagate_trace_context,
+                  :jobs_sample_rate, :job_lifecycle_logs, :propagate_trace_context,
                   :usage_enabled, :usage_flush_interval, :usage_max_symbols
     attr_writer :release, :project_root
     attr_reader :excluded_exceptions, :excluded_log_patterns, :excluded_query_patterns,
@@ -94,7 +94,8 @@ module CloseYourIt
 
       # Request body params in the event (`request.data`), extracted LAZILY only when the error
       # happens, sanitized and scrubbed (denylist + filter_parameters). The backend scrubs again
-      # defensively. Uploads → placeholder, 64 KB cap.
+      # defensively. Uploads → placeholder, 64 KB cap. Gated by send_pii: with it OFF the body
+      # is never sent, whatever this flag says (CYRB-28).
       @capture_request_body = true
 
       # Breadcrumbs: history (obfuscated queries, custom events) attached to the error.
@@ -157,6 +158,7 @@ module CloseYourIt
       # (normal jobs under threshold = no metric) and by sampling. The label is the job class name;
       # arguments are NEVER sent.
       @monitor_jobs                   = true
+      @job_lifecycle_logs             = false
       @slow_job_threshold_ms          = 5000     # perform duration beyond which = slow_job
       @job_queue_latency_threshold_ms = 60_000   # enqueue→execution wait beyond which = job_queue_latency
       @jobs_sample_rate               = 1.0      # fraction of over-threshold candidates actually sent
@@ -199,6 +201,13 @@ module CloseYourIt
       @filter_parameters = Array(list)
     end
 
+    # Adds the host app's `Rails.application.config.filter_parameters` (strings/symbols/regexps) to the
+    # gem list, without duplicates. Rails filter procs are dropped: they mutate values in place, while
+    # the Scrubber matches keys only (CYRB-28).
+    def merge_filter_parameters(list)
+      @filter_parameters |= Array(list).reject { |item| item.respond_to?(:call) }
+    end
+
     # Destinations allowed to receive the W3C trace headers. String = exact host (case-insensitive
     # match), Regexp = pattern (for subdomains/host families). Empty list = none.
     def trace_propagation_allowlist=(list)

data/lib/closeyourit/events/job_metric_event.rb

@@ -33,6 +33,9 @@ module CloseYourIt
         "queue" => @attrs[:queue],
         "adapter" => @attrs[:adapter],
         "attempt" => @attrs[:attempt],
+        "contexts" => (@attrs[:job_context] && { "job" => @attrs[:job_context].merge(
+          "measurement" => (@attrs[:subtype] == "slow_job" ? "duration" : "queue_wait")
+        ) }),
         "sdk" => sdk
       )
     end

data/lib/closeyourit/events/slow_method_event.rb

@@ -42,7 +42,9 @@ module CloseYourIt
     private
 
     # Arguments — ONLY with capture_method_arguments (opt-in). Positional by index; kwargs by name with
-    # sensitive key scrubbing (denylist password/token/…). Value = truncated inspect (JSON-safe).
+    # sensitive key scrubbing (denylist password/token/…). Value = truncated inspect (JSON-safe),
+    # passed through the message scrubber (auth schemes + scrub_message_patterns) like exception
+    # messages, so a token inside a positional argument never leaves the app (CYRB-28).
     def arguments
       return nil unless @configuration.capture_method_arguments
 
@@ -54,7 +56,7 @@ module CloseYourIt
     end
 
     def safe_value(value)
-      inspected = value.inspect.to_s
+      inspected = @scrubber.scrub_message(value.inspect.to_s)
       inspected.length > 120 ? "#{inspected[0, 117]}…" : inspected
     end
   end

data/lib/closeyourit/job_context.rb

@@ -0,0 +1,61 @@
+# frozen_string_literal: true
+
+require "time"
+require "date"
+
+module CloseYourIt
+  # Versioned attempt metadata. Missing clocks stay unknown, never an invented zero.
+  module JobContext
+    def self.build(framework:, name:, queue: nil, job_id: nil, attempt: nil, retry_count: :derive,
+                   enqueued_at: nil, scheduled_at: nil, started_at: nil)
+      enqueued, scheduled, started = [ enqueued_at, scheduled_at, started_at ].map { |value| instant(value) }
+      due = enqueued && scheduled ? [ enqueued, scheduled ].max : enqueued
+      wait = (started - due) * 1000.0 if started && due
+      delay = (scheduled - enqueued) * 1000.0 if enqueued && scheduled
+      attempt = nil unless attempt.is_a?(Integer) && attempt.positive?
+      retry_count = attempt && attempt - 1 if retry_count == :derive
+      retry_count = nil unless retry_count.is_a?(Integer) && retry_count >= 0
+      {
+        "schema_version" => 1, "framework" => framework, "name" => bounded_text(name, 200) || "unknown",
+        "queue" => bounded_text(queue, 128), "job_id" => bounded_text(job_id, 256), "attempt" => attempt,
+        "retry_count" => retry_count, "attempt_outcome" => "unknown", "terminal" => nil,
+        "enqueued_at" => enqueued&.iso8601(6), "scheduled_at" => scheduled&.iso8601(6),
+        "started_at" => started&.iso8601(6), "duration_ms" => nil,
+        "queue_wait_ms" => wait && [ wait, 0.0 ].max,
+        "scheduled_delay_ms" => delay && [ delay, 0.0 ].max, "clock_skew" => !!(wait && wait.negative?)
+      }
+    end
+
+    def self.bounded_text(value, limit)
+      value if value.is_a?(String) && value.length.between?(1, limit) && !value.match?(/[\x00-\x1f\x7f]/)
+    end
+
+    def self.sidekiq_time(value, version:)
+      return value unless value.is_a?(Numeric)
+
+      version.to_s.split(".").first.to_i >= 8 ? value / 1000.0 : value
+    end
+
+    def self.instant(value)
+      case value
+      when Time then value.getutc
+      when Numeric then Time.at(value).utc if value.finite?
+      when String
+        if value.match?(/(?:Z|[+-]\d{2}:\d{2})\z/)
+          DateTime.rfc3339(value)
+          Time.iso8601(value).utc
+        end
+      end
+    rescue ArgumentError, RangeError
+      nil
+    end
+
+    def self.log(context, configuration)
+      return unless configuration.monitor_jobs && configuration.job_lifecycle_logs
+
+      CloseYourIt.log(:info, "Background job attempt finished", logger: "closeyourit.jobs", contexts: { job: context })
+    rescue StandardError
+      CloseYourIt.internal_logger.warn("CloseYourIt job lifecycle emission failed")
+    end
+  end
+end

data/lib/closeyourit/rails/active_job_extension.rb

@@ -31,6 +31,9 @@ module CloseYourIt
       def self.included(base)
         base.around_perform do |job, block|
           CloseYourIt::Rails::ActiveJobExtension.monitor(job) { block.call }
+        rescue Exception # rubocop:disable Lint/RescueException
+          job.instance_variable_set(:@__closeyourit_job_outcome, :raised)
+          raise
         end
 
         return unless base.respond_to?(:after_discard)

data/lib/closeyourit/rails/railtie.rb

@@ -72,25 +72,9 @@ module CloseYourIt
         end
       end
 
-      # Measures duration and queue wait of ActiveJob jobs via ActiveSupport notifications:
-      # `perform_start` gives the wait (now - enqueued_at) as soon as the job starts, `perform` gives
-      # the execution duration at the end. Beyond threshold → slow_job / job_queue_latency metrics.
-      # No-op when monitor_jobs is OFF.
+      # Observe the completed attempt after ActiveJob has decided retry/discard.
       initializer "closeyourit.subscribe_active_job_performance" do
-        jobs = CloseYourIt::Subscribers::JobPerformance.new
-
-        ActiveSupport::Notifications.subscribe("perform_start.active_job") do |*args|
-          job = ActiveSupport::Notifications::Event.new(*args).payload[:job]
-          jobs.active_job_started(job) if job
-        end
-
-        ActiveSupport::Notifications.subscribe("perform.active_job") do |*args|
-          event = ActiveSupport::Notifications::Event.new(*args)
-          job = event.payload[:job]
-          # CYSK-29 — jobs also declare they ran: kind `job`, symbol = the class.
-          CloseYourIt.usage_registry.record("job", job.class.name) if job && CloseYourIt.enabled?
-          jobs.active_job_performed(job, event.duration) if job
-        end
+        CloseYourIt::Subscribers::JobPerformance.new.install
       end
 
       # Captures HANDLED errors reported via Rails.error.report (Rails 7+).
@@ -116,6 +100,13 @@ module CloseYourIt
         end
       end
 
+      # Keys the app declared sensitive via config.filter_parameters are scrubbed by the gem too.
+      # `after: :load_config_initializers`: CloseYourIt.init (config/initializers/closeyourit.rb)
+      # replaces the configuration, so an earlier merge would be lost (CYRB-28).
+      initializer "closeyourit.merge_rails_filter_parameters", after: :load_config_initializers do |app|
+        CloseYourIt.configuration.merge_filter_parameters(app.config.filter_parameters)
+      end
+
       # Captures Sidekiq job errors + measures duration/wait via server middleware (only when Sidekiq
       # is present). The middleware is effectively a no-op when monitor_jobs is OFF (the decision
       # lives in JobPerformance#record).

data/lib/closeyourit/scope.rb

@@ -139,8 +139,10 @@ module CloseYourIt
       data ? @request.merge("data" => data) : @request
     end
 
+    # Body params are user input (PII): sent only with send_pii AND capture_request_body (CYRB-28).
     def request_body_data
-      return nil unless CloseYourIt.configuration.capture_request_body
+      config = CloseYourIt.configuration
+      return nil unless config.send_pii && config.capture_request_body
       return nil if @rack_env.nil?
 
       Rails::RequestBody.extract(@rack_env)

data/lib/closeyourit/scrubber.rb

@@ -69,8 +69,9 @@ module CloseYourIt
     # credential: here the prose guard is needed, the word "bearer" shows up in messages.
     AUTH_CREDENTIAL = /(\b#{AUTH_SCHEME}[ \t]+)#{NOT_PROSE}#{CREDENTIAL_TOKEN}/
 
+    DOLLAR_LITERAL = /\$([A-Za-z_][A-Za-z0-9_]*|)\$.*?(?:\$\1\$|\z)/m
     STRING_LITERAL = /'(?:[^']|'')*'/
-    NUMERIC_LITERAL = /\b\d+(?:\.\d+)?\b/
+    NUMERIC_LITERAL = /(?<![\w$])\d+(?:\.\d+)?\b/
 
     def initialize(configuration)
       @configuration = configuration
@@ -94,7 +95,7 @@ module CloseYourIt
     def obfuscate_sql(sql)
       return sql if sql.nil? || !@configuration.obfuscate_sql
 
-      sql.to_s.gsub(STRING_LITERAL, "?").gsub(NUMERIC_LITERAL, "?")
+      sql.to_s.gsub(DOLLAR_LITERAL, "?").gsub(STRING_LITERAL, "?").gsub(NUMERIC_LITERAL, "?")
     end
 
     # Built-in rule (authentication schemes) BEFORE the user patterns: the two protections add up,

data/lib/closeyourit/sidekiq/job_metrics_middleware.rb

@@ -16,27 +16,50 @@ module CloseYourIt
       end
 
       def call(_worker, job, queue)
+        return yield if active_job_owned?(job)
+
         started = Process.clock_gettime(Process::CLOCK_MONOTONIC)
-        # The queue wait is known at the START (now - enqueued_at); Sidekiq stores enqueued_at as epoch
-        # seconds. Computed before the yield so it does not include the job duration.
-        latency = Subscribers::JobPerformance.latency_ms(job["enqueued_at"], now: Time.now.utc)
-        yield
+        context = observe(job, queue)
+        value = yield
+        succeeded = true
+        value
       ensure
-        emit(job, queue, started, latency)
+        emit(job, queue, started, context, succeeded) if started
       end
 
       private
 
-      def emit(job, queue, started, latency)
+      def active_job_owned?(job)
+        %w[ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper Sidekiq::ActiveJob::Wrapper].include?(job["class"]) &&
+          Subscribers::JobPerformance.active_job_installed?
+      end
+
+      def observe(job, queue)
+        # Sidekiq 8 changed the timestamp wire unit; do not infer it from magnitude.
+        version = ::Sidekiq::VERSION if defined?(::Sidekiq::VERSION)
+        enqueued = JobContext.sidekiq_time(job["enqueued_at"], version: version)
+        JobContext.build(framework: "sidekiq", name: job["wrapped"] || job["class"],
+          queue: queue || job["queue"], job_id: job["jid"], attempt: attempt(job), retry_count: job["retry_count"],
+          enqueued_at: enqueued, started_at: Time.now.utc)
+      rescue StandardError
+        nil
+      end
+
+      def emit(job, queue, started, context, succeeded)
         duration_ms = (Process.clock_gettime(Process::CLOCK_MONOTONIC) - started) * 1000.0
+        if context
+          context = context.merge("duration_ms" => duration_ms, "attempt_outcome" => succeeded ? "success" : "error",
+            "terminal" => succeeded ? true : nil)
+          JobContext.log(context, CloseYourIt.configuration)
+        end
         subscriber.record(
           job_class: job["wrapped"] || job["class"],
           queue: queue || job["queue"],
           adapter: "sidekiq",
           duration_ms: duration_ms,
-          queue_latency_ms: latency,
+          queue_latency_ms: context && context["queue_wait_ms"],
           attempt: attempt(job),
-          trace_id: job["jid"]
+          trace_id: job["jid"], job_context: context
         )
       rescue StandardError => e
         CloseYourIt.internal_logger.error("CloseYourIt job metrics: #{e.class}: #{e.message}")

data/lib/closeyourit/subscribers/job_performance.rb

@@ -2,6 +2,7 @@
 
 require "time"
 require_relative "../events/job_metric_event"
+require_relative "../job_context"
 
 module CloseYourIt
   module Subscribers
@@ -12,10 +13,43 @@ module CloseYourIt
     # wiring to ActiveSupport::Notifications and the Sidekiq middleware lives elsewhere (Railtie /
     # JobMetricsMiddleware). Honors the `monitor_jobs` master switch, the thresholds and `jobs_sample_rate`.
     class JobPerformance
+      REGISTRY_LOCK = Mutex.new
+      ACTIVE_JOB_SUBSCRIBERS = []
+
+      def self.active_job_installed?
+        REGISTRY_LOCK.synchronize { !ACTIVE_JOB_SUBSCRIBERS.empty? }
+      end
       def initialize(configuration = nil)
         @configuration = configuration
       end
 
+      def install
+        REGISTRY_LOCK.synchronize do
+          return self if @subscriptions || !ACTIVE_JOB_SUBSCRIBERS.empty?
+
+          @subscriptions = []
+          { "enqueue_retry" => :retry, "retry_stopped" => :exhausted, "discard" => :discarded }.each do |name, outcome|
+            subscribe(name) { |event| event.payload[:job]&.instance_variable_set(:@__closeyourit_job_outcome, outcome) }
+          end
+          subscribe("perform_start") { |event| active_job_started(event.payload[:job]) if event.payload[:job] }
+          subscribe("perform") do |event|
+            job = event.payload[:job]
+            if job
+              CloseYourIt.usage_registry.record("job", job.class.name) if CloseYourIt.enabled?
+              active_job_performed(job, event.duration, payload: event.payload)
+            end
+          end
+          ACTIVE_JOB_SUBSCRIBERS << self
+          self
+        end
+      end
+
+      def uninstall
+        Array(@subscriptions).each { |subscription| ActiveSupport::Notifications.unsubscribe(subscription) }
+        @subscriptions = nil
+        REGISTRY_LOCK.synchronize { ACTIVE_JOB_SUBSCRIBERS.delete(self) }
+      end
+
       # Single decision point: from the measured values (duration and/or wait) it builds 0..2 metrics,
       # applies the thresholds (strict `>`, so X makes no noise and X+1 does) and sampling, then sends
       # them fire-and-forget. `duration_ms` and `queue_latency_ms` are optional: ActiveJob provides them
@@ -23,11 +57,11 @@ module CloseYourIt
       # Sampling is applied ONLY to candidates already beyond the threshold (normal jobs neither
       # consume nor generate anything).
       def record(job_class:, queue: nil, adapter: nil, duration_ms: nil, queue_latency_ms: nil,
-                 attempt: nil, trace_id: nil)
+                 attempt: nil, trace_id: nil, job_context: nil)
         config = configuration
         return unless config.monitor_jobs
 
-        common = { job_class: job_class, queue: queue, adapter: adapter, attempt: attempt, trace_id: trace_id }
+        common = { job_class: job_class, queue: queue, adapter: adapter, attempt: attempt, trace_id: trace_id, job_context: job_context }
         events = []
         events << build(config, "slow_job", duration_ms, common) if slow?(config, duration_ms)
         events << build(config, "job_queue_latency", queue_latency_ms, common) if waited?(config, queue_latency_ms)
@@ -41,26 +75,51 @@ module CloseYourIt
       # `perform_start.active_job` hook: the queue wait is known as soon as the job starts (now - the
       # moment it was due to run).
       def active_job_started(job, now: Time.now.utc)
+        context = JobContext.build(framework: "active_job", name: job.class.name,
+          queue: (job.queue_name if job.respond_to?(:queue_name)), job_id: (job.job_id if job.respond_to?(:job_id)),
+          attempt: (job.executions if job.respond_to?(:executions)), enqueued_at: enqueued_at(job),
+          scheduled_at: (job.scheduled_at if job.respond_to?(:scheduled_at)), started_at: now)
+        job.instance_variable_set(:@__closeyourit_job_context, context)
         record(
           job_class: job.class.name,
           queue: (job.queue_name if job.respond_to?(:queue_name)),
           adapter: "active_job",
-          queue_latency_ms: self.class.latency_ms(due_at(job), now: now),
+          queue_latency_ms: context["queue_wait_ms"],
           attempt: (job.executions if job.respond_to?(:executions)),
-          trace_id: (job.job_id if job.respond_to?(:job_id))
+          trace_id: (job.job_id if job.respond_to?(:job_id)), job_context: context
         )
       end
 
       # `perform.active_job` hook: at the end of execution the duration is `event.duration` (ms).
-      def active_job_performed(job, duration_ms)
+      def active_job_performed(job, duration_ms, payload: {})
+        context = job.instance_variable_get(:@__closeyourit_job_context)
+        if context
+          outcome, terminal = active_job_outcome(job, payload)
+          context = context.merge("duration_ms" => duration_ms, "attempt_outcome" => outcome, "terminal" => terminal)
+          JobContext.log(context, configuration)
+        end
         record(
           job_class: job.class.name,
           queue: (job.queue_name if job.respond_to?(:queue_name)),
           adapter: "active_job",
           duration_ms: duration_ms,
           attempt: (job.executions if job.respond_to?(:executions)),
-          trace_id: (job.job_id if job.respond_to?(:job_id))
+          trace_id: (job.job_id if job.respond_to?(:job_id)), job_context: context
         )
+      ensure
+        job.remove_instance_variable(:@__closeyourit_job_context) if job.instance_variable_defined?(:@__closeyourit_job_context)
+        job.remove_instance_variable(:@__closeyourit_job_outcome) if job.instance_variable_defined?(:@__closeyourit_job_outcome)
+      end
+
+      def active_job_outcome(job, payload)
+        outcome = job.instance_variable_get(:@__closeyourit_job_outcome)
+        return [ "error", nil ] if payload[:exception_object] || payload[:exception]
+        return [ "retry", false ] if outcome == :retry
+        return [ "discarded", true ] if outcome == :discarded
+        return [ "error", nil ] if outcome == :exhausted
+        return [ "unknown", nil ] if outcome == :raised || payload[:aborted]
+
+        [ "success", true ]
       end
 
       # Normalizes `enqueued_at` (Time, Numeric epoch in seconds like Sidekiq, or ISO8601 String) into
@@ -91,6 +150,14 @@ module CloseYourIt
 
       private
 
+      def subscribe(name, &callback)
+        @subscriptions << ActiveSupport::Notifications.monotonic_subscribe("#{name}.active_job") do |*args|
+          callback.call(ActiveSupport::Notifications::Event.new(*args))
+        rescue StandardError
+          CloseYourIt.internal_logger.warn("CloseYourIt job observation failed")
+        end
+      end
+
       def enqueued_at(job)
         job.enqueued_at if job.respond_to?(:enqueued_at)
       end

data/lib/closeyourit/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module CloseYourIt
-  VERSION = "0.10.2"
+  VERSION = "0.12.1"
 end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: closeyourit-ruby
 version: !ruby/object:Gem::Version
-  version: 0.10.2
+  version: 0.12.1
 platform: ruby
 authors:
 - Alessio Bussolari
@@ -62,6 +62,7 @@ files:
 - lib/closeyourit/events/slow_method_event.rb
 - lib/closeyourit/events/slow_query_event.rb
 - lib/closeyourit/instrumenter.rb
+- lib/closeyourit/job_context.rb
 - lib/closeyourit/line_cache.rb
 - lib/closeyourit/log_buffer.rb
 - lib/closeyourit/log_device.rb
@@ -89,12 +90,12 @@ files:
 - lib/closeyourit/transport.rb
 - lib/closeyourit/usage_registry.rb
 - lib/closeyourit/version.rb
-homepage: https://github.com/bussolabs/closeyourit-ruby
+homepage: https://github.com/bussolabs/closeyourit-ruby-community
 licenses:
 - MIT
 metadata:
-  homepage_uri: https://github.com/bussolabs/closeyourit-ruby
-  source_code_uri: https://github.com/bussolabs/closeyourit-ruby
+  homepage_uri: https://github.com/bussolabs/closeyourit-ruby-community
+  source_code_uri: https://github.com/bussolabs/closeyourit-ruby-community
 rdoc_options: []
 require_paths:
 - lib