forge_ops_tracker 0.13.0 → 0.15.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: ccbeed25a3172994d20455c19fa79a8336f3c73324ed667fd2f31d2cc05a19f5
-  data.tar.gz: 3cb7e1562daf2ccb8f09226bc915b7f00f6e8486ebbf361f8adf0a16c9851afe
+  metadata.gz: 2c05c6f94c7c080f7c148cc315097b8e177d34dec772d580e21cf89a7120f1b3
+  data.tar.gz: d486796b0f26b752872118cfaad2c9c184671bc596bd6c32a04cfb6a754dc171
 SHA512:
-  metadata.gz: 6287470f8bc768e95d1d1afe55f2c7a0e640a4d66ded8a0e3c518e0248031a7f5f76621adf0436d2de3bc7da029733b9debab07e31d2f1b37c7867cfd3aee7e5
-  data.tar.gz: b7c996fbf89c2c0a90f6c39c848542cde734242a3fa4efaf450a36b1e7362815322417ceafc89d05a71a453a9965d7b4ab0c4deb2eedad0fdafd6b21808ad790
+  metadata.gz: e99dddf991705420ceb4d69f21ad8d5e39885f5f2f782e0747976aa2c8cb2463a4d8f56652f23f6944c40d38bc89e95475e61ecaf2079b0f106ca401d4e25d06
+  data.tar.gz: 1e5e0e69a95ddecb9411a024dcbf1817b688312131bbffc7359cb917f86d416e7d2811b1215feeb355b4ff4df3b1a2757bafe747ddc425e057e3248bb3572287

data/CHANGELOG.md

@@ -1,5 +1,28 @@
 # Changelog
 
+## 0.15.0 (2026-09-29)
+
+- SQL masking now also catches strings with a backslash-escaped quote (`'o\'brien'`,
+  `E'o\'brien'`), prefixed strings (`E''`, `X''`, `N''`, `B''`, `U&''`, prefix included), hex,
+  binary and exponent numbers (`0x1F`, `0b101`, `3e10`, `1.5E-3`, `.5`), and, on MySQL and
+  MariaDB, `"double-quoted"` strings. `SqlStatement.mask` takes an optional `system:`, and
+  database spans (including `ForgeOpsTracker.span`'s `db_system:`) pass their `db.system` to it.
+
+## 0.14.0 (2026-09-25)
+
+- Every Active Record database span now carries the query's SQL (`db.statement`), masked so every
+  string and number is `?` and bind values are never sent, and the database it ran on
+  (`db.system`: `postgresql`, `mysql`, `sqlite`, `mssql`, `oracle`, or the adapter's own name).
+- `ForgeOpsTracker.span` takes optional `statement:` and `db_system:` on a `kind: "database"`
+  span, masked the same way.
+- New opt-in `config.explain_slow_queries` (default false) and `config.explain_threshold_ms`
+  (default 500). On PostgreSQL, a single plain `SELECT` at least that slow gets a plain
+  `EXPLAIN (FORMAT JSON)` (never `EXPLAIN ANALYZE`), run on a background thread with its own
+  pooled connection, in a read-only transaction with a 2 second statement timeout that is always
+  rolled back. The plan is sent with every string in it masked. Limited to once per 10 minutes per
+  distinct query and 10 per minute per process; plans over 64 KB are dropped; any failure is
+  logged at debug and never raised.
+
 ## 0.13.0 (2026-09-25)
 
 - New `ForgeOpsTracker.record_change(kind:, title:, details: {}, environment: nil, service: nil,

data/README.md

@@ -411,6 +411,73 @@ Requires a ForgeOps plan that includes distributed tracing; on a plan that doesn
 simply rejected server-side and dropped, exactly like any other delivery failure. The `trace_id`
 on error events is sent either way.
 
+### The SQL behind each database span
+
+Every Active Record query span carries the query's SQL and the database it ran on, so a slow trace
+shows exactly which query was slow. The SQL is masked before it leaves your process: every string
+and number is replaced by `?`, so `WHERE email = 'a@b.co' AND id = 42` is sent as
+`WHERE email = ? AND id = ?`, and bind values are never sent at all. That includes escaped and
+prefixed strings (`E'o\'brien'`, `X'DEADBEEF'`), hex, binary and exponent numbers (`0x1F`,
+`0b101`, `1.5E-3`), and, on MySQL and MariaDB (where `"double quotes"` hold a string, not a name),
+double-quoted strings. ForgeOps masks it again on arrival. The database comes from the
+connection's adapter (`postgresql`, `mysql`, `sqlite`, `mssql`, `oracle`, or the adapter's own
+name otherwise).
+
+Spans you create yourself can carry SQL too. Pass `statement:` (and optionally `db_system:`) on a
+`kind: "database"` span; it's masked the same way:
+
+```ruby
+sql = ActiveRecord::Base.sanitize_sql_array(
+  [ "SELECT * FROM orders WHERE customer_id = ? AND status = ?", customer.id, "open" ]
+)
+
+ForgeOpsTracker.span("Load open orders", kind: "database", statement: sql, db_system: "postgresql") do
+  ReportingDatabase.connection.select_all(sql)
+end
+# Sent as "SELECT * FROM orders WHERE customer_id = ? AND status = ?"
+```
+
+### Query plans for slow queries (PostgreSQL)
+
+Opt in, and a slow `SELECT` also sends Postgres's plan for it, so ForgeOps can show why it was
+slow (a sequential scan over a big table, a missing index, a bad join order):
+
+```ruby
+ForgeOpsTracker.configure do |config|
+  config.explain_slow_queries = true # default false
+  config.explain_threshold_ms = 500  # default 500; only queries at least this slow
+end
+```
+
+What it does:
+
+- For a query that took at least `explain_threshold_ms`, runs `EXPLAIN (FORMAT JSON)` with the
+  same SQL and bind values, and sends the plan with the query's masked SQL. Every string inside the
+  plan (a filter like `(email = 'a@b.co'::text)`) is masked the same way before it's sent. A plan
+  over 64 KB is dropped rather than sent.
+- Runs on the gem's own background thread, on a connection it checks out of your app's pool itself,
+  inside its own `READ ONLY` transaction with a 2 second `statement_timeout`, always rolled back.
+  Never on the connection your request is using, and never inside your request's transaction.
+- At most once per 10 minutes for each distinct (masked) query, and at most 10 per minute per
+  process overall.
+
+What it doesn't do:
+
+- It never runs `EXPLAIN ANALYZE`, so your query is planned again but never executed again.
+- It only explains a single plain `SELECT`: nothing that starts with anything else (including
+  `WITH`), nothing with a second statement, no `FOR UPDATE`/`FOR SHARE` (or `FOR NO KEY UPDATE`/
+  `FOR KEY SHARE`), and nothing with `INSERT`, `UPDATE`, `DELETE` or `MERGE` anywhere in it.
+- It only runs on PostgreSQL. Other databases are left alone.
+- It never raises into your app. A timeout, a permission error, an exhausted connection pool or
+  anything else is logged at debug level and skipped.
+- The original SQL and its bind values are only held in memory until the `EXPLAIN` runs. Only the
+  masked SQL and the masked plan are sent.
+
+It does use one pooled connection, briefly, for each plan. With the limits above that's at most 10
+short checkouts a minute, but on a pool that's already at capacity, leave it off or raise
+`explain_threshold_ms`. Plans are part of performance monitoring; on a plan without it, they're
+rejected server-side and dropped.
+
 ## What changed
 
 ForgeOps can show what changed in your system next to the errors and slowdowns that followed it.

data/lib/forge_ops_tracker/client.rb

@@ -71,6 +71,12 @@ module ForgeOpsTracker
       post(configuration.change_snapshots_uri, payload)
     end
 
+    # Same again; see Configuration#query_plans_uri and QueryExplainer. One plan per call. A plan
+    # without performance monitoring answers 403, which is just a false here.
+    def deliver_query_plan(payload)
+      post(configuration.query_plans_uri, payload)
+    end
+
     private
       attr_reader :configuration
 

data/lib/forge_ops_tracker/configuration.rb

@@ -17,6 +17,7 @@ module ForgeOpsTracker
     attr_accessor :metric_flush_interval, :infrastructure_metric_flush_interval
     attr_accessor :gauge_poll_interval
     attr_accessor :track_tracing, :trace_capture_threshold_ms
+    attr_accessor :explain_slow_queries, :explain_threshold_ms
     attr_accessor :propagate_traces, :trace_propagation_targets
     attr_accessor :detect_changes, :track_env_var_names
 
@@ -126,6 +127,13 @@ module ForgeOpsTracker
       # PerformanceSample-derived threshold elsewhere in this gem: those describe a trend across
       # many requests, this describes a single one being outright slow.
       @trace_capture_threshold_ms = 1_000
+      # Opt-in: for a SELECT on PostgreSQL that took at least explain_threshold_ms, ask Postgres for
+      # its plan (plain EXPLAIN, never EXPLAIN ANALYZE, so the query is never run again) and send
+      # it, masked, so ForgeOps can show why the query was slow. Off by default because it runs one
+      # extra statement against the app's own database. See QueryExplainer for exactly what it
+      # does and doesn't do, and its rate limits.
+      @explain_slow_queries = false
+      @explain_threshold_ms = 500
       # Adds a W3C `traceparent` header (https://www.w3.org/TR/trace-context/) to every outbound
       # Net::HTTP request made during a request, so the service on the other end can continue this
       # request's trace instead of starting its own (see Integrations::NetHTTP). On by default, the
@@ -241,6 +249,10 @@ module ForgeOpsTracker
       uri
     end
 
+    def query_plans_uri
+      sibling_uri("/query_plans")
+    end
+
     def changes_uri
       sibling_uri("/changes")
     end

data/lib/forge_ops_tracker/performance_instrumentation.rb

@@ -70,6 +70,39 @@ module ForgeOpsTracker
       name.nil? || name.empty? ? "SQL" : name
     end
 
+    # Active Record's adapter names, as ForgeOps' "db.system" values. Anything not listed here is
+    # sent as its own adapter name, lowercased.
+    DB_SYSTEMS = {
+      "postgresql" => "postgresql", "postgis" => "postgresql",
+      "mysql" => "mysql", "mysql2" => "mysql", "trilogy" => "mysql",
+      "sqlite" => "sqlite", "sqlite3" => "sqlite",
+      "sqlserver" => "mssql",
+      "oracle" => "oracle", "oracleenhanced" => "oracle"
+    }.freeze
+
+    # The "data" a database span carries: the query's SQL, masked by SqlStatement (every string
+    # and number literal becomes "?", so no value the app queried with is ever sent), and which
+    # database it ran on, from the connection's adapter. Either is left out when it isn't known.
+    def database_span_data(payload)
+      system = db_system(payload[:connection])
+      {
+        "db.statement" => SqlStatement.mask(payload[:sql].to_s, system: system),
+        "db.system" => system
+      }.compact
+    end
+
+    # payload[:connection] is the adapter instance the query ran on (Rails 6.0 and later).
+    def db_system(connection)
+      return nil unless connection.respond_to?(:adapter_name)
+
+      name = connection.adapter_name.to_s.downcase
+      return nil if name.empty?
+
+      DB_SYSTEMS.fetch(name, name)
+    rescue StandardError
+      nil
+    end
+
     def job_transaction_name(payload)
       payload[:job].class.name
     end

data/lib/forge_ops_tracker/query_explainer.rb

@@ -0,0 +1,250 @@
+require "json"
+require "thread"
+require "time"
+
+module ForgeOpsTracker
+  # Opt-in (configuration.explain_slow_queries, default false): asks PostgreSQL for the plan of a
+  # slow SELECT and sends it, masked, to ForgeOps so a slow query shows why it was slow.
+  #
+  # What it never does, by design:
+  # - EXPLAIN ANALYZE. Plain EXPLAIN only plans the statement, it never executes it.
+  # - Touch the app's own connection or transaction. #consider only reads the notification payload
+  #   and remembers the connection's pool; the EXPLAIN runs later, on this class's own background
+  #   thread, on a connection it checks out of that pool itself, inside its own READ ONLY
+  #   transaction with a 2 second statement_timeout, always rolled back.
+  # - Explain anything but a single plain SELECT (see .explainable?).
+  # - Raise. Every failure (a timeout, a permission error, an exhausted pool, a statement Postgres
+  #   can't parse) is logged at debug and dropped.
+  # - Send the original statement or its bind values anywhere. Both are held in memory only until
+  #   the EXPLAIN runs; what goes out is the masked statement (exactly as sent on the span) and the
+  #   plan with every string in it masked the same way, since plan conditions quote literals
+  #   ("Filter": "(email = 'a@b.co'::text)").
+  #
+  # Rate limited per process: each distinct masked statement at most once per 10 minutes, and at
+  # most 10 EXPLAINs per minute overall. Both are reserved when a query is accepted, not when its
+  # EXPLAIN succeeds, so a statement whose EXPLAIN keeps failing still can't be retried more often.
+  class QueryExplainer
+    STATEMENT_INTERVAL = 600.0
+    GLOBAL_WINDOW = 60.0
+    GLOBAL_LIMIT = 10
+    MAX_PLAN_BYTES = 64 * 1024
+    STATEMENT_TIMEOUT = "2s".freeze
+    CHECKOUT_TIMEOUT = 1
+    QUEUE_SIZE = GLOBAL_LIMIT
+    # Past this many remembered statements, the ones whose 10 minutes are up are forgotten, so an
+    # app with endless distinct statements can't grow this without bound.
+    MAX_REMEMBERED_STATEMENTS = 1_000
+
+    # One pass, left to right, so a quote inside a comment or comment syntax inside a string can't
+    # confuse the other: string literals (single-quoted and dollar-quoted) become "?", comments
+    # become a space.
+    LITERALS_AND_COMMENTS = %r{
+      '(?:[^']|'')*(?:'|\z)
+      | (?<tag>\$[A-Za-z_]*\$).*?(?:\k<tag>|\z)
+      | /\*.*?(?:\*/|\z)
+      | --[^\n]*
+    }mx
+    LOCKING_CLAUSE = /\bFOR\s+(?:NO\s+KEY\s+UPDATE|KEY\s+SHARE|UPDATE|SHARE)\b/i
+    DATA_MODIFYING = /\b(?:INSERT|UPDATE|DELETE|MERGE)\b/i
+
+    Job = Struct.new(:sql, :binds, :pool, :statement, :duration_ms, :captured_at, keyword_init: true)
+
+    class << self
+      # True on this class's own worker thread while it's running an EXPLAIN. Railtie's
+      # sql.active_record subscriber skips those queries entirely, so the EXPLAIN (and its BEGIN,
+      # SET LOCAL and ROLLBACK) never shows up as the app's own query timing, breadcrumbs or spans.
+      def explaining?
+        Thread.current[:forge_ops_tracker_explaining] == true
+      end
+
+      # Whether statement is a single plain SELECT: after dropping comments and string literals,
+      # it starts with SELECT, has no ";" other than one trailing one, no locking clause (FOR
+      # UPDATE, FOR SHARE, FOR NO KEY UPDATE, FOR KEY SHARE), and no INSERT, UPDATE, DELETE or
+      # MERGE anywhere (so no data-modifying CTE either).
+      def explainable?(statement)
+        return false unless statement.is_a?(String)
+
+        sql = statement.gsub(LITERALS_AND_COMMENTS) { |match| match.start_with?("'", "$") ? "?" : " " }.strip
+        return false unless sql.match?(/\ASELECT\b/i)
+        return false if sql.delete_suffix(";").include?(";")
+
+        !sql.match?(LOCKING_CLAUSE) && !sql.match?(DATA_MODIFYING)
+      end
+
+      # Masks every String inside a parsed EXPLAIN (FORMAT JSON) result with SqlStatement, leaving
+      # keys, numbers, booleans, arrays and the structure itself exactly as they were.
+      def mask_plan(value)
+        case value
+        when Hash then value.transform_values { |inner| mask_plan(inner) }
+        when Array then value.map { |inner| mask_plan(inner) }
+        when String then SqlStatement.mask(value) || value
+        else value
+        end
+      end
+    end
+
+    def initialize(configuration, client: Client.new(configuration), clock: -> { Process.clock_gettime(Process::CLOCK_MONOTONIC) })
+      @configuration = configuration
+      @client = client
+      @clock = clock
+      @queue = SizedQueue.new(QUEUE_SIZE)
+      @mutex = Mutex.new
+      @thread = nil
+      @last_explained = {}
+      @recent = []
+    end
+
+    # Called from the sql.active_record subscriber with that event's payload and duration. Cheap
+    # checks first, so a fast query costs one comparison. Returns true only when an EXPLAIN was
+    # queued; never raises.
+    def consider(payload, duration_ms:)
+      return false unless configuration.explain_slow_queries
+      return false unless duration_ms && duration_ms >= configuration.explain_threshold_ms.to_f
+
+      connection = payload[:connection]
+      system = PerformanceInstrumentation.db_system(connection)
+      return false unless system == "postgresql"
+
+      sql = payload[:sql]
+      return false unless self.class.explainable?(sql)
+
+      pool = connection.respond_to?(:pool) ? connection.pool : nil
+      return false unless pool
+
+      statement = SqlStatement.mask(sql, system: system)
+      return false unless reserve(statement)
+
+      enqueue(Job.new(
+        sql: sql.dup, binds: bind_values(payload), pool: pool, statement: statement,
+        duration_ms: duration_ms.to_f.round(1), captured_at: Time.now.utc.iso8601(3)
+      ))
+    rescue StandardError => e
+      log { "EXPLAIN skipped: #{e.class}: #{e.message}" }
+      false
+    end
+
+    # Runs one queued EXPLAIN and delivers its plan. Normally called on the worker thread; public so
+    # a test can run a job synchronously. Never raises.
+    def perform(job)
+      plan = explain(job)
+      return false unless plan
+
+      masked_plan = self.class.mask_plan(plan)
+      if JSON.generate(masked_plan).bytesize > MAX_PLAN_BYTES
+        log { "EXPLAIN plan over #{MAX_PLAN_BYTES} bytes, dropped" }
+        return false
+      end
+
+      client.deliver_query_plan(
+        statement: job.statement, db_system: "postgresql", plan: masked_plan,
+        duration_ms: job.duration_ms, environment: configuration.environment, captured_at: job.captured_at
+      )
+    rescue StandardError => e
+      log { "EXPLAIN failed: #{e.class}: #{e.message}" }
+      false
+    end
+
+    private
+      attr_reader :configuration, :client
+
+      # Both rate limits at once, under one lock, so two threads finishing the same slow query
+      # together still only explain it once.
+      def reserve(statement)
+        return false if statement.nil?
+
+        @mutex.synchronize do
+          now = @clock.call
+          @recent.reject! { |at| now - at >= GLOBAL_WINDOW }
+          return false if @recent.size >= GLOBAL_LIMIT
+
+          last = @last_explained[statement]
+          return false if last && now - last < STATEMENT_INTERVAL
+
+          if @last_explained.size >= MAX_REMEMBERED_STATEMENTS
+            @last_explained.reject! { |_, at| now - at >= STATEMENT_INTERVAL }
+          end
+          @last_explained[statement] = now
+          @recent << now
+          true
+        end
+      end
+
+      # The values the app's own query was run with, already type cast for the database. Rails
+      # passes these either as an Array or as a lambda that builds one.
+      def bind_values(payload)
+        binds = payload[:type_casted_binds]
+        binds = binds.call if binds.respond_to?(:call)
+        Array(binds).dup
+      end
+
+      def enqueue(job)
+        ensure_worker_started
+        @queue.push(job, true)
+        true
+      rescue ThreadError
+        log { "EXPLAIN queue full, dropping" }
+        false
+      end
+
+      def ensure_worker_started
+        return if @thread&.alive?
+
+        @mutex.synchronize do
+          return if @thread&.alive?
+
+          @thread = Thread.new { run }
+          @thread.abort_on_exception = false
+        end
+      end
+
+      def run
+        Thread.current[:forge_ops_tracker_explaining] = true
+        loop do
+          perform(@queue.pop)
+        rescue StandardError => e
+          log { "EXPLAIN thread error: #{e.class}: #{e.message}" }
+        end
+      end
+
+      # The parsed plan (the JSON array Postgres returns), or nil. The connection is checked out
+      # explicitly rather than borrowed with with_connection, so it's never one this thread (or
+      # any other) already holds. Checked back in only once its transaction is rolled back; if
+      # even the ROLLBACK fails, the connection is removed from the pool and closed instead, so a
+      # connection in an unknown state is never handed to the app.
+      def explain(job)
+        connection = job.pool.checkout(CHECKOUT_TIMEOUT)
+        begin
+          connection.execute("BEGIN READ ONLY")
+          connection.execute("SET LOCAL statement_timeout = '#{STATEMENT_TIMEOUT}'")
+          result = connection.exec_query("EXPLAIN (FORMAT JSON) #{job.sql}", "EXPLAIN", job.binds)
+          value = result.rows.first&.first
+          value.is_a?(String) ? JSON.parse(value) : value
+        ensure
+          release(job.pool, connection, rollback(connection))
+        end
+      end
+
+      def rollback(connection)
+        connection.execute("ROLLBACK")
+        true
+      rescue StandardError => e
+        log { "EXPLAIN rollback failed: #{e.class}: #{e.message}" }
+        false
+      end
+
+      def release(pool, connection, healthy)
+        if healthy
+          pool.checkin(connection)
+        else
+          pool.remove(connection)
+          connection.disconnect!
+        end
+      rescue StandardError => e
+        log { "EXPLAIN connection release failed: #{e.class}: #{e.message}" }
+      end
+
+      def log
+        configuration.logger&.debug { "[ForgeOpsTracker] #{yield}" }
+      end
+  end
+end

data/lib/forge_ops_tracker/railtie.rb

@@ -28,6 +28,11 @@ module ForgeOpsTracker
       def span_queue
         @span_queue ||= ForgeOpsTracker::SpanQueue.new(ForgeOpsTracker.configuration)
       end
+
+      # One per process, so its rate limits (see QueryExplainer) are per process too.
+      def query_explainer
+        @query_explainer ||= ForgeOpsTracker::QueryExplainer.new(ForgeOpsTracker.configuration)
+      end
     end
 
     initializer "forge_ops_tracker.subscribe_error_reporter" do |app|
@@ -167,6 +172,9 @@ module ForgeOpsTracker
       end
 
       ActiveSupport::Notifications.subscribe("sql.active_record") do |*args|
+        # QueryExplainer's own EXPLAIN (and its BEGIN/SET LOCAL/ROLLBACK), on its own thread: not
+        # the app's query, so never timed, traced or explained itself.
+        next if QueryExplainer.explaining?
         next unless configuration.enabled?
 
         event = ActiveSupport::Notifications::Event.new(*args)
@@ -184,13 +192,19 @@ module ForgeOpsTracker
         # A leaf span, parented onto whatever's currently open (the controller root, or a
         # manually-wrapped ForgeOpsTracker.span block around it): a query has no children of its
         # own to nest anything under, so unlike the controller root above, this never touches the
-        # stack itself, just records against whatever's already on top of it.
+        # stack itself, just records against whatever's already on top of it. Carries the masked
+        # SQL and the database system (see PerformanceInstrumentation.database_span_data).
         if configuration.track_tracing && (buffer = Thread.current[:forge_ops_tracker_spans])
           buffer.record(
             span_id: SecureRandom.hex(8), name: transaction_name, kind: "database",
-            started_at: Time.at(event.time), duration_ms: event.duration
+            started_at: Time.at(event.time), duration_ms: event.duration,
+            data: PerformanceInstrumentation.database_span_data(event.payload)
           )
         end
+
+        # Opt-in, PostgreSQL only; a no-op unless config.explain_slow_queries is on. Only reads the
+        # payload here; the EXPLAIN itself runs later on QueryExplainer's own thread and connection.
+        Railtie.query_explainer.consider(event.payload, duration_ms: event.duration) if configuration.explain_slow_queries
       end
 
       ActiveSupport::Notifications.subscribe("perform.active_job") do |*args|

data/lib/forge_ops_tracker/sql_statement.rb

@@ -15,11 +15,19 @@ module ForgeOpsTracker
     MAX_NAME_LENGTH = 200
     MAX_CAUSE_DEPTH = 5
 
-    LITERAL = /
-      '(?:[^']|'')*(?:'|\z)                      # string literal, or one cut off by truncation
-      | (?<tag>\$[A-Za-z_]*\$).*?(?:\k<tag>|\z)  # dollar-quoted string
-      | (?<![\w$.])\d+(?:\.\d+)?(?!\w)           # number, not part of an identifier or placeholder
-    /mx
+    # db.system values where "double quotes" are a string, not a name.
+    DOUBLE_QUOTED_STRING_SYSTEMS = %w[mysql mariadb].freeze
+
+    # The prefix only counts as one when it isn't the end of a word, but the quote always does:
+    # LIKE'%x%', with no space, is still a string. A backslash can be the statement's last character
+    # ('secret\ cut off mid-escape), and the string is still masked to the end.
+    STRING = /(?:(?<![\w$])(?:[EeXxNnBb]|[Uu]&))?'(?:[^'\\]|\\(?:.|\z)|'')*(?:'|\z)/m
+    DOUBLE_QUOTED_STRING = /"(?:[^"\\]|\\(?:.|\z)|"")*(?:"|\z)/m
+    DOLLAR_QUOTED = /(?<tag>\$[A-Za-z_]*\$).*?(?:\k<tag>|\z)/m
+    NUMBER = /(?<![\w$.])(?:0[xX]\h+|0[bB][01]+|(?:\d+(?:\.\d+)?|\.\d+)(?:[eE][+-]?\d+)?)(?!\w)/
+
+    LITERAL = Regexp.union(STRING, DOLLAR_QUOTED, NUMBER)
+    LITERAL_WITH_DOUBLE_QUOTES = Regexp.union(STRING, DOUBLE_QUOTED_STRING, DOLLAR_QUOTED, NUMBER)
 
     PART = /(?:[\w$#@]+|"[^"]+"|\[[^\]]+\]|`[^`]+`)/
     NAME = /#{PART}(?:\.#{PART})*/
@@ -51,10 +59,13 @@ module ForgeOpsTracker
       nil
     end
 
-    def mask(statement)
+    # system is the query's db.system, when known: on MySQL and MariaDB "double quotes" are a
+    # string too, so they're masked there and left alone everywhere else.
+    def mask(statement, system: nil)
       return nil if statement.nil? || statement.strip.empty?
 
-      masked = statement.gsub(LITERAL, MASK)
+      pattern = DOUBLE_QUOTED_STRING_SYSTEMS.include?(system.to_s.downcase) ? LITERAL_WITH_DOUBLE_QUOTES : LITERAL
+      masked = statement.gsub(pattern, MASK)
       masked.length > MAX_LENGTH ? "#{masked[0, MAX_LENGTH]}..." : masked
     end
 

data/lib/forge_ops_tracker/version.rb

@@ -1,3 +1,3 @@
 module ForgeOpsTracker
-  VERSION = "0.13.0"
+  VERSION = "0.15.0"
 end

data/lib/forge_ops_tracker.rb

@@ -20,6 +20,7 @@ require "forge_ops_tracker/periodic_poller"
 require "forge_ops_tracker/breadcrumb_buffer"
 require "forge_ops_tracker/span_buffer"
 require "forge_ops_tracker/span_queue"
+require "forge_ops_tracker/query_explainer"
 require "forge_ops_tracker/trace_parent"
 require "forge_ops_tracker/request_state"
 require "forge_ops_tracker/middleware/session_tracking"
@@ -128,12 +129,18 @@ module ForgeOpsTracker
     # a standalone buffer for itself: a lone span with no request-level trace around it, and no
     # middleware left running to ever flush it, would just accumulate in this thread's memory
     # forever with nothing to ever send it, which is worse than never recording it at all.
-    def span(name, kind: "service", data: {})
+    #
+    # A span with kind: "database" can also carry the SQL it ran and which database it ran on, e.g.
+    # `ForgeOpsTracker.span("Load orders", kind: "database", statement: sql, db_system: "postgresql")`.
+    # The statement is masked by SqlStatement before it's kept (every string and number becomes
+    # "?"), so no value it was run with is ever sent. Both are ignored on any other kind of span.
+    def span(name, kind: "service", data: {}, statement: nil, db_system: nil)
       return yield unless configuration.enabled? && configuration.track_tracing
 
       buffer = Thread.current[:forge_ops_tracker_spans]
       return yield unless buffer
 
+      data = database_span_data(data, statement, db_system) if kind.to_s == "database"
       span_id = SecureRandom.hex(8)
       started_at = Time.now.utc
       buffer.push(span_id)
@@ -147,6 +154,15 @@ module ForgeOpsTracker
     end
 
     private
+      def database_span_data(data, statement, db_system)
+        system = (db_system.to_s.strip.downcase unless db_system.nil? || db_system.to_s.strip.empty?)
+        extra = {
+          "db.statement" => (SqlStatement.mask(statement.to_s, system: system) unless statement.nil?),
+          "db.system" => system
+        }.compact
+        extra.empty? ? data : data.merge(extra)
+      end
+
       def change_queue
         @change_queue ||= DeliveryQueue.new(configuration, deliver_with: :deliver_change)
       end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: forge_ops_tracker
 version: !ruby/object:Gem::Version
-  version: 0.13.0
+  version: 0.15.0
 platform: ruby
 authors:
 - ForgeOps
@@ -29,14 +29,14 @@ dependencies:
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '7.0'
+        version: '8.1'
   type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: '7.0'
+        version: '8.1'
 - !ruby/object:Gem::Dependency
   name: puma
   requirement: !ruby/object:Gem::Requirement
@@ -116,6 +116,7 @@ files:
 - lib/forge_ops_tracker/performance_instrumentation.rb
 - lib/forge_ops_tracker/periodic_poller.rb
 - lib/forge_ops_tracker/pii_scrubber.rb
+- lib/forge_ops_tracker/query_explainer.rb
 - lib/forge_ops_tracker/railtie.rb
 - lib/forge_ops_tracker/request_state.rb
 - lib/forge_ops_tracker/session_flusher.rb