rails-profiler 0.31.4 → 0.31.6

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3ca8756c9efed6c4619db0cbc1226f104deb5f7287f972b137f9a8bf291a40c4
-  data.tar.gz: dcb61c33b3262c081493beb1ee783923b3c7b3ac8d4cd26ef9195ea4e3ff182d
+  metadata.gz: f8c9d9f31f3f32dc5ada20eb78eb6bc205c0985e084942edf18b283cfafe81be
+  data.tar.gz: 49fb9afa0dd2fa52033f151b077ec971db08fe45818fe64448559ba95f4face3
 SHA512:
-  metadata.gz: 2576d031d1b4d3e6e462392ed5ca00bc83c12eaed23b9ec07ea842bd3365e373d7eff1b4c53ae73e779366fc947ad70f01f15e8b1108c8fb36e85409986ebd22
-  data.tar.gz: 4b4d5914c82c06e58fd1d8e14157a28b2d0a9bbc00d15f574a5a5bdb0e782c0e9957042f25c38ba2d6cbc93c159b0b4d2f921667d552e397f37fed896bcc2245
+  metadata.gz: 79bd6095dd2deb184ee662c398183d5e63b9b9af30f5bda76b1806353adce5137627c2e33afc5c95aadc451dd540a9d243c1cc1df3eb72eccc2001331362581f
+  data.tar.gz: bd488d9da999d2f54be42b4002030f88bb51b069c956b78c85f0aadc4e87faa3f6ec3a2bb26c9921b73bea0fbaa84e16fd8e3f5967cfc8276ada75fa44a9b331

data/CHANGELOG.md

@@ -19,6 +19,109 @@ every commit of every tag interval is accounted for one way or the other.
 
 ## [Unreleased]
 
+## [0.31.6] - 2026-10-05
+
+<!-- stamped -->
+
+### Fixed
+
+- **Storage:** With `config.compress_bodies` (on by default), a text body larger than
+  `config.compress_body_threshold` (10 KB) is now stored gzip+base64, as the README said: the
+  profile was decompressed again before every save, so the file, Redis and memory stores kept the
+  body as text, next to the compressed copy the request collector keeps, and the compression only
+  cost time. A 132 KB HTML page now takes 86 KB in those stores instead of 171 KB (273 KB without
+  compression); SQLite, which only keeps that copy, 44 KB instead of 36 KB at the faster level.
+  Compressing it costs 1.6 ms on the request instead of 4.1 ms (the fastest zlib level), saving
+  the profile 0.2 ms instead of 1.2 ms; the bodies are decompressed only when a profile is shown,
+  about 0.7 ms. The profile page, its API, the job, console and test APIs, the lists the cluster
+  proxy reads and the MCP tools still give the bodies as text; the toolbar, which shows no body,
+  gets them as stored (86 KB instead of 171 KB for that page).
+- **Storage:** The profile list summaries kept by the file, SQLite and Redis stores no longer
+  carry the first 500 characters of the request and response bodies. Summaries written by 0.31.4
+  and 0.31.5 keep them until their profiles are evicted; no list shows them.
+- **MCP:** The `curl` command of `get_profile_detail` carries the request body as it was sent: a
+  body larger than `compress_body_threshold` came out as its compressed base64 form, and a binary
+  body as base64 text. A binary body is now left out of the command.
+- **Upgrading:** the profiles already stored, all as text, are read as before, with no migration.
+  An earlier version reading a store written by this one shows the compressed bodies as text too,
+  and a cluster master and its slaves may run either version: the profile API they read still
+  gives the bodies as text. A body that cannot be decompressed is shown as it is stored, never as
+  an error. To keep the bodies readable as text in the stored files or Redis keys, set
+  `config.compress_bodies = false`.
+
+## [0.31.5] - 2026-10-05
+
+<!-- stamped -->
+
+### Fixed
+
+- **Logging:** Send the profiler's own errors and warnings to the application's log instead of
+  `$stderr`. A profile that could not be saved, a collector that failed, a store that could not be
+  read, a cluster node out of reach: about thirty places wrote to `$stderr`, which is not where
+  `log/development.log` is, so these errors went unseen. They now go through one entry point,
+  `Profiler.log_error`, to `config.logger` when the application sets one (read on each message),
+  otherwise `Rails.logger`, otherwise `$stderr`. Every line starts with `[Profiler]` (the cluster's
+  lines, `[Profiler Cluster]` before, now start with `[Profiler] Cluster:`), carries the error's
+  class and message, masked then cut at 1,000 bytes, and has the cluster secret masked. A logger that
+  raises never fails a request, a job or the boot: the line then goes to `$stderr`. A slave whose
+  `Rails.logger` was still nil raised instead of logging. Several failures that left no trace at
+  all are now logged, once per place and error class for the life of the process: a store lookup or broadcast, the Redis
+  version read of the toolbar, the MCP body cache, the mailer collector reading a mail, the route
+  lookup of the Request tab, a file profile that cannot be read to rebuild the index. The request middleware saves its
+  profiles through `Profiler.save_profile`, like jobs, console commands and tests; a page whose
+  profile could not be saved gets neither the `X-Profiler-Token` header nor the toolbar, which
+  would point to a missing profile. The profiler's own lines no longer appear in the Logs tab of
+  the request during which they are written, nor count as its errors. The same holds while the store is
+  unavailable, for pages and for streamed responses, which no longer send `X-Profiler-Token` for a
+  profile that will be dropped. The storage warnings (a store that cannot be created, said once
+  per cause; a Redis profile that cannot be indexed; a SQLite summary that cannot be read) go to
+  the application's log too, and the `run_tests` MCP tool answers an unavailable store as an error
+  instead of a run without profiles.
+- **Outbound HTTP:** Record the calls to services on the same machine. `127.0.0.1`, `localhost`
+  and `::1` were always left out, so in development the calls to a neighbouring service never
+  showed. `config.http_skip_hosts` is now the whole list, empty by default. The profiler's own
+  calls are never recorded, whatever the list says: a slave's registration and heartbeats, and the
+  master's requests to its slaves (the slave proxy, the fan-out of the profile pages and the MCP
+  tools that query a slave), marked where they are made, and any call to the host and port of
+  `config.master_url`. `NetHttpInstrumentation::SKIP_HOSTS` stays, deprecated, with its former value
+  (`%w[127.0.0.1 localhost ::1]`); nothing reads it any more.
+- **Outbound HTTP:** Keep the bodies of outbound `Net::HTTP` calls to the first
+  `config.max_captured_body_bytes`, sent and received, instead of copying them whole; a compressed
+  answer is inflated only up to that size, give or take one 16 KB buffer. A `body_stream` is read
+  only when it can be put back where it was (a `StringIO`, a file), up to that size and no further
+  than its `Content-Length`: a pipe or a socket was read whole and then sent empty, it is now sent
+  unread and its body is not captured. A `body_stream` without `pos`, such as a Faraday multipart
+  body or a `RestClient::Payload`, is not captured either, where earlier versions replaced it with
+  a copy read whole. The size is the `Content-Length` or the stream's own size, otherwise a
+  minimum. The HTTP tab and the `get_profile_http` MCP tool say when a body was cut or not
+  captured; each call carries `request_body_truncated`, `response_body_truncated`,
+  `request_body_not_captured` and `request_size_is_minimum`. Text bodies read in part stay UTF-8.
+- **Middleware:** Cap the `params` a profile keeps at `config.max_captured_body_bytes`: long values
+  are cut and the entries past it left out, with a `"[profiler]"` key saying so, and the profile
+  carries `params_truncated`. The application still gets all of its params. With the log line
+  capped below, a 5 MB JSON request no longer takes about 15 MB in its profile.
+- **Logs:** Cap what a profile keeps of its request's log lines at the new
+  `config.max_captured_log_bytes` (1 MB by default). A request, or a stream held open, that logged
+  without end grew its thread's buffer without bound; a single line, such as the
+  `Parameters: ...` line of a large request, is cut at the cap. The Logs tab ends with the number
+  of lines left out, and the collected data carries `truncated` and `dropped`.
+- **Dump:** `Profiler.dump` returns its argument when the profiler is disabled too; it returned
+  `nil`, so a call left in the code changed the value of the expression in production.
+- **Engine:** Stop failing the boot of an application that loads `ActionController::API` (an
+  `api_only` application, or a single API controller): the engine called `helper` on it, which
+  `ActionController::API` does not have (`NoMethodError`).
+- **Toolbar:** Find where the toolbar goes about four times faster on a page of many tags (0.8 ms
+  instead of 4 ms for a page of 20 KB and 4,000 tags, 10 ms instead of 25 ms for 320 KB of tags
+  with attributes): ordinary text and tags are skipped in one pass. The place is the same, and a
+  `</body>` in an attribute, a comment, a script or a CDATA section, an unclosed quote or a
+  `<plaintext>` are handled as before.
+- **Upgrading:** to get the previous behaviour back, in `config/initializers/profiler.rb`:
+  - the profiler's messages on `$stderr`: `config.logger = Logger.new($stderr)`;
+  - the local hosts left out of the HTTP tab:
+    `config.http_skip_hosts += Profiler::Configuration::LOCAL_HTTP_HOSTS`;
+  - whole outbound bodies and params: `config.max_captured_body_bytes = nil` (also keeps the
+    incoming bodies whole); every log line: `config.max_captured_log_bytes = nil`.
+
 ## [0.31.4] - 2026-10-05
 
 <!-- stamped -->

data/app/assets/builds/profiler.js

@@ -4414,6 +4414,13 @@
     parts.push(`  ${JSON.stringify(req.url)}`);
     return parts.join(" \\\n");
   }
+  function bodyNotes(req) {
+    const notes = [];
+    if (req.request_body_not_captured) notes.push("Request body not captured: a stream that cannot be rewound is sent unread.");
+    if (req.request_body_truncated) notes.push("Request body truncated: only the beginning was kept (max_captured_body_bytes).");
+    if (req.response_body_truncated) notes.push("Response body truncated: only the beginning was kept (max_captured_body_bytes).");
+    return notes;
+  }
   function HttpRequestDetail({ req, index, threshold }) {
     const [open, setOpen] = d2(false);
     const isError = req.status >= 400 || req.status === 0;
@@ -4436,12 +4443,13 @@
       /* @__PURE__ */ u3("div", { class: "profiler-ajax-card__row", children: [
         /* @__PURE__ */ u3("span", { class: "profiler-text--xs profiler-text--muted", children: [
           "\u2191 ",
-          formatBytes(req.request_size),
+          req.request_size == null ? "size unknown" : `${req.request_size_is_minimum ? "at least " : ""}${formatBytes(req.request_size)}`,
           " \xB7 \u2193 ",
           formatBytes(req.response_size)
         ] }),
         req.backtrace && req.backtrace.length > 0 && /* @__PURE__ */ u3("span", { class: "profiler-text--xs profiler-text--muted", style: "margin-left:12px", children: req.backtrace[0] })
       ] }),
+      bodyNotes(req).map((note) => /* @__PURE__ */ u3("div", { class: "profiler-ajax-card__row", children: /* @__PURE__ */ u3("span", { class: "profiler-text--xs profiler-text--muted", children: note }) })),
       req.error && /* @__PURE__ */ u3("div", { class: "profiler-ajax-card__row", children: /* @__PURE__ */ u3("span", { class: "profiler-text--xs profiler-text--error", children: req.error }) }),
       open && /* @__PURE__ */ u3(
         HttpReqRespDetail,

data/app/controllers/profiler/api/console_controller.rb

@@ -14,7 +14,7 @@ module Profiler
           return render json: { error: "Console profile not found" }, status: :not_found
         end
 
-        render json: profile.to_h
+        render json: profile.to_h(decode_bodies: true)
       end
 
       def destroy

data/app/controllers/profiler/api/jobs_controller.rb

@@ -14,7 +14,7 @@ module Profiler
           return render json: { error: "Job profile not found" }, status: :not_found
         end
 
-        render json: profile.to_h.merge(
+        render json: profile.to_h(decode_bodies: true).merge(
           child_jobs: build_child_jobs(profile),
           parent_profile: build_parent_summary(profile)
         )

data/app/controllers/profiler/api/profiles_controller.rb

@@ -28,7 +28,7 @@ module Profiler
         # The route table and ENV are the process's, not stored in the profile
         Profiler::ProcessSnapshot.hydrate(profile)
 
-        render json: profile.to_h.merge(
+        render json: profile.to_h(decode_bodies: true).merge(
           child_jobs: build_child_jobs(profile),
           parent_profile: build_parent_summary(profile)
         )

data/app/controllers/profiler/api/tests_controller.rb

@@ -14,7 +14,7 @@ module Profiler
           return render json: { error: "Test profile not found" }, status: :not_found
         end
 
-        render json: profile.to_h
+        render json: profile.to_h(decode_bodies: true)
       end
 
       def destroy

data/app/controllers/profiler/api/toolbar_controller.rb

@@ -19,6 +19,7 @@ module Profiler
         # The route table and ENV are the process's, not stored in the profile
         Profiler::ProcessSnapshot.hydrate(profile)
 
+        # The bodies as stored (gzip+base64 past compress_body_threshold): the toolbar shows none.
         render json: {
           profile: profile.to_h.merge(child_jobs: build_child_jobs(profile)),
           events_cursor: events_cursor

data/app/controllers/profiler/application_controller.rb

@@ -74,9 +74,10 @@ module Profiler
       render_page(Profiler.storage.list(limit: limit + 1, offset: offset, type: type, summary: summary), limit, offset)
     end
 
+    # The bodies as text: the cluster proxy lists whole profiles (all_types), a summary has none.
     def render_page(profiles, limit, offset)
       render json: {
-        profiles: profiles.first(limit).map(&:to_h),
+        profiles: profiles.first(limit).map { |profile| profile.to_h(decode_bodies: true) },
         limit:    limit,
         offset:   offset,
         has_more: profiles.size > limit

data/app/controllers/profiler/profiles_controller.rb

@@ -25,7 +25,7 @@ module Profiler
       # The route table and ENV are the process's, not stored in the profile
       Profiler::ProcessSnapshot.hydrate(@profile)
 
-      @profile_data = @profile.to_h.merge(
+      @profile_data = @profile.to_h(decode_bodies: true).merge(
         child_jobs: build_child_jobs(@profile),
         parent_profile: build_parent_summary(@profile)
       )
@@ -100,7 +100,7 @@ module Profiler
               return profile
             end
           rescue => e
-            Rails.logger.warn("[Profiler] Cached slave #{cached_slave} failed for #{token}: #{e.message}")
+            Profiler.log_warn("Cluster: cached slave #{cached_slave} failed for #{token}", e)
           end
         end
         Profiler.token_cache.invalidate(token)
@@ -130,7 +130,7 @@ module Profiler
               end
             end
           rescue => e
-            Rails.logger.warn("[Profiler] Fan-out error for slave #{slave_name} (#{token}): #{e.message}")
+            Profiler.log_warn("Cluster: fan-out error for slave #{slave_name} (#{token})", e)
           end
         end
       end

data/lib/profiler/cluster/master_client.rb

@@ -14,7 +14,7 @@ module Profiler
           register!
           @registered = true
         rescue => e
-          log_warn("Could not register with master at startup: #{e.message} — will retry in heartbeat loop")
+          log_warn("could not register with master at startup, will retry in the heartbeat loop", e)
         end
         start_heartbeat_thread
       end
@@ -25,7 +25,7 @@ module Profiler
         config = Profiler.configuration
         uri = master_uri("register")
         body = { name: config.resolved_name, url: config.self_url }.to_json
-        resp = Net::HTTP.post(uri, body, request_headers)
+        resp = Profiler.untracked_http { Net::HTTP.post(uri, body, request_headers) }
         unless resp.code.to_i.between?(200, 299)
           raise "Master returned #{resp.code}: #{resp.body.to_s.slice(0, 200)}"
         end
@@ -37,7 +37,7 @@ module Profiler
         config = Profiler.configuration
         uri = master_uri("heartbeat")
         body = { name: config.resolved_name }.to_json
-        resp = Net::HTTP.post(uri, body, request_headers)
+        resp = Profiler.untracked_http { Net::HTTP.post(uri, body, request_headers) }
         raise "Heartbeat rejected #{resp.code}" unless resp.code.to_i.between?(200, 299)
       end
 
@@ -74,25 +74,17 @@ module Profiler
             end
           rescue => e
             @registered = false
-            log_warn("Cluster communication failed: #{e.message} — will retry")
+            log_warn("communication with the master failed, will retry", e)
           end
         end
       end
 
       def log_info(msg)
-        if defined?(Rails)
-          Rails.logger.info("[Profiler Cluster] #{msg}")
-        else
-          $stderr.puts("[Profiler Cluster] #{msg}")
-        end
+        Profiler.log_info("Cluster: #{msg}")
       end
 
-      def log_warn(msg)
-        if defined?(Rails)
-          Rails.logger.warn("[Profiler Cluster] #{msg}")
-        else
-          $stderr.puts("[Profiler Cluster] WARN #{msg}")
-        end
+      def log_warn(msg, error = nil)
+        Profiler.log_warn("Cluster: #{msg}", error)
       end
     end
   end

data/lib/profiler/cluster/security.rb

@@ -44,9 +44,9 @@ module Profiler
           return unless config.cluster_master? || config.slave?
           return unless (problem = secret_problem) && secret_required?
 
-          logger.warn("[Profiler Cluster] #{problem}: registrations, heartbeats and proxied requests " \
-                      "are refused. Generate one with `ruby -rsecurerandom -e 'puts SecureRandom.hex(32)'` " \
-                      "and give the same value to every node.")
+          Profiler.log_warn("Cluster: #{problem}: registrations, heartbeats and proxied requests " \
+                            "are refused. Generate one with `ruby -rsecurerandom -e 'puts SecureRandom.hex(32)'` " \
+                            "and give the same value to every node.", logger: logger)
         end
 
         # Constant-time comparison of the secret a request carries with the configured one.

data/lib/profiler/cluster/slave_proxy.rb

@@ -115,10 +115,13 @@ module Profiler
         Security.outgoing_headers.each { |name, value| req[name] = value }
         # Net::HTTP never follows a redirect; a 3xx is reported, without its body, rather than
         # handed back as if the slave had answered.
-        resp = Net::HTTP.start(uri.hostname, uri.port,
-                               open_timeout: @open_timeout, read_timeout: @read_timeout,
-                               use_ssl: uri.scheme == "https") do |http|
-          http.request(req)
+        # The profiler's own call, never recorded in the profile of the request that makes it.
+        resp = Profiler.untracked_http do
+          Net::HTTP.start(uri.hostname, uri.port,
+                          open_timeout: @open_timeout, read_timeout: @read_timeout,
+                          use_ssl: uri.scheme == "https") do |http|
+            http.request(req)
+          end
         end
         return {} if resp.code == "204"
         if resp.code.to_s.start_with?("3")

data/lib/profiler/collectors/lifecycle.rb

@@ -13,7 +13,7 @@ module Profiler
         collectors.each { |collector| collector.subscribe if collector.respond_to?(:subscribe) }
         true
       rescue => e
-        warn "Profiler #{label}: Collector subscribe failed: #{e.message}"
+        Profiler.log_error("#{label}: collector subscribe failed", e)
         release_all(collectors)
         false
       end
@@ -23,7 +23,7 @@ module Profiler
         collectors&.each do |collector|
           collector.unsubscribe if collector.respond_to?(:unsubscribe)
         rescue => e
-          warn "Profiler: Collector #{collector.class} release failed: #{e.message}"
+          Profiler.log_error("collector #{collector.class} release failed", e)
         end
       end
     end

data/lib/profiler/collectors/log_collector.rb

@@ -8,6 +8,43 @@ module Profiler
     class LogCollector < BaseCollector
       SEVERITY_LABELS = %w[DEBUG INFO WARN ERROR FATAL UNKNOWN].freeze
 
+      # The lines of one request, up to max_captured_log_bytes of messages: past it, a line is cut
+      # to what is left, and the next ones are only counted. A request, or a stream held open,
+      # that logs without end no longer grows its thread's buffer without bound.
+      class Buffer < Array
+        attr_reader :bytes, :dropped
+
+        def initialize(limit)
+          super()
+          @limit = limit
+          @bytes = 0
+          @dropped = 0
+          @truncated = false
+        end
+
+        def truncated?
+          @truncated
+        end
+
+        def record(entry)
+          message = entry[:message]
+          if @limit
+            room = @limit - @bytes
+            if room <= 0
+              @dropped += 1
+              @truncated = true
+              return
+            end
+            if message.bytesize > room
+              message = Redaction.cut_bytes(message, room).scrub("")
+              @truncated = true
+            end
+          end
+          @bytes += message.bytesize
+          self << entry.merge(message: message)
+        end
+      end
+
       # Records into the thread's buffer, which only exists while a LogCollector of this thread
       # is subscribed: lines logged by other threads, or after release, are not kept. A logger
       # bound to one collector's buffer records only while that buffer is the thread's current
@@ -21,17 +58,20 @@ module Profiler
         def add(severity, message = nil, progname = nil)
           logs = Thread.current[:profiler_logs]
           return true unless logs
+          # The profiler's own messages (Profiler.log) are not the application's.
+          return true if Thread.current[:profiler_logging]
           return true if @buffer && !logs.equal?(@buffer)
 
           msg = message || progname
           msg = yield if block_given? && msg.nil?
           return true if msg.nil?
 
-          logs << {
+          entry = {
             level: SEVERITY_LABELS[severity] || "UNKNOWN",
             message: msg.to_s.strip,
             timestamp: Time.now.iso8601(3)
           }
+          logs.respond_to?(:record) ? logs.record(entry) : logs << entry
           true
         end
 
@@ -78,7 +118,7 @@ module Profiler
       end
 
       def subscribe
-        @logs = claim_thread_slot(:profiler_logs, [])
+        @logs = claim_thread_slot(:profiler_logs, Buffer.new(Profiler.configuration.max_captured_log_bytes))
 
         if defined?(Rails) && Rails.logger
           if Rails.logger.respond_to?(:broadcast_to)
@@ -90,7 +130,7 @@ module Profiler
           end
         end
       rescue => e
-        warn "LogCollector#subscribe failed: #{e.message}"
+        Profiler.log_error("LogCollector: subscribe failed", e)
       end
 
       # Collect reads only what the collector gathered itself.
@@ -113,12 +153,21 @@ module Profiler
 
         errors   = logs.count { |l| %w[ERROR FATAL].include?(l[:level]) }
         warnings = logs.count { |l| l[:level] == "WARN" }
+        dropped = logs.respond_to?(:dropped) ? logs.dropped : 0
+        truncated = logs.respond_to?(:truncated?) && logs.truncated?
+        lines = logs.to_a
+        if dropped.positive?
+          lines += [{ level: "WARN", timestamp: Time.now.iso8601(3),
+                      message: "[Profiler] #{dropped} more log lines were not recorded (max_captured_log_bytes)" }]
+        end
 
         store_data({
-          count: logs.size,
+          count: logs.size + dropped,
           errors: errors,
           warnings: warnings,
-          logs: logs
+          logs: lines,
+          truncated: truncated,
+          dropped: dropped
         })
       end
 

data/lib/profiler/collectors/mailer_collector.rb

@@ -213,7 +213,8 @@ module Profiler
             serialize_assign(filtered)
           end
         end
-      rescue StandardError
+      rescue StandardError => e
+        Profiler.log_error_once(:mailer_assigns, "MailerCollector: could not read the mailer's assigns", e)
         {}
       end
 
@@ -232,7 +233,8 @@ module Profiler
           text = Profiler::Redaction.truncate(mail.body.decoded, MAX_BODY_SIZE, "")
         end
         [html.nil? || html.empty? ? nil : html, text.nil? || text.empty? ? nil : text]
-      rescue StandardError
+      rescue StandardError => e
+        Profiler.log_error_once(:mailer_body, "MailerCollector: could not read the mail's body", e)
         [nil, nil]
       end
 
@@ -250,7 +252,8 @@ module Profiler
         return nil unless mail.is_a?(String)
 
         Mail.new(mail)
-      rescue StandardError
+      rescue StandardError => e
+        Profiler.log_error_once(:mailer_parse, "MailerCollector: could not parse the mail", e)
         nil
       end
 
@@ -291,7 +294,8 @@ module Profiler
             "filename" => att.filename.to_s,
             "size" => att.body.decoded.bytesize
           }
-        rescue StandardError
+        rescue StandardError => e
+          Profiler.log_error_once(:mailer_attachment, "MailerCollector: could not read an attachment", e)
           { "filename" => att.filename.to_s, "size" => 0 }
         end
       end

data/lib/profiler/collectors/request_collector.rb

@@ -33,6 +33,7 @@ module Profiler
           allocated_objects: @profile.allocated_objects,
           memory: @profile.memory, # deprecated, see Profile#memory
           params: @profile.params,
+          params_truncated: @profile.params_truncated || false,
           headers: @profile.headers,
           response_headers: @profile.response_headers,
           request_body: @profile.request_body,
@@ -103,7 +104,11 @@ module Profiler
           route_params:      Profiler::Redaction.filter_hash(route_params),
           controller_action: controller_action
         }
-      rescue StandardError
+      rescue StandardError => e
+        # A path no route matches (a 404) has no route to show.
+        unless defined?(ActionController::RoutingError) && e.is_a?(ActionController::RoutingError)
+          Profiler.log_error_once(:request_route, "RequestCollector: could not read the matched route", e)
+        end
         {}
       end
 

data/lib/profiler/collectors/scoped_notifications.rb

@@ -219,7 +219,7 @@ module Profiler
               handler.call(name, started, finished, id, payload)
             rescue StandardError => e
               # Never into the application's query, nor at the expense of the other collectors.
-              warn "Profiler: a collector failed on #{event}: #{e.class}: #{e.message}"
+              Profiler.log_error("collector failed on #{event}", e)
             end
           end
         end

data/lib/profiler/configuration.rb

@@ -33,6 +33,10 @@ module Profiler
 
     BACKEND_LOCK = Mutex.new
 
+    # The hosts earlier versions always left out of the outbound HTTP tab. Add them back with
+    # config.http_skip_hosts += Profiler::Configuration::LOCAL_HTTP_HOSTS.
+    LOCAL_HTTP_HOSTS = [/\A127\.0\.0\.1\z/, /\Alocalhost\z/i, /\A::1\z/].freeze
+
     attr_accessor :storage_options, :collectors,
                   :skip_paths, :slow_query_threshold, :max_queries_warning, :sql_backtrace,
                   :track_memory, :allocated_objects_warning_threshold,
@@ -46,12 +50,12 @@ module Profiler
                   :apply_env_overrides_when_disabled, :restrict_storage_permissions,
                   :test_runner_allow_undiscovered_files,
                   :track_mailers, :capture_mail_body, :sanitize_mailer_recipients, :mailer_skip_actions,
-                  :compress_bodies, :compress_body_threshold, :max_captured_body_bytes,
+                  :compress_bodies, :compress_body_threshold, :max_captured_body_bytes, :max_captured_log_bytes,
                   :redact_sensitive_data, :filter_parameters, :env_allowlist,
                   :name, :master_url, :self_url,
                   :cluster_heartbeat_interval, :cluster_offline_threshold,
                   :cluster_master, :cluster_secret, :cluster_require_secret,
-                  :cluster_allow_insecure_http
+                  :cluster_allow_insecure_http, :logger
 
     attr_reader :authorize_block, :enabled, :track_tests, :max_profiles
 
@@ -89,6 +93,8 @@ module Profiler
       @ajax_skip_paths = [/^\/_profiler/]
       @track_http = true
       @slow_http_threshold = 500 # milliseconds
+      # Every host whose outbound calls are left out (Strings and Regexps matched against the host).
+      # The profiler's own calls (cluster, slave proxy) are never recorded, whatever this says.
       @http_skip_hosts = []
       @http_backtrace_depth = 40
       @track_jobs = true
@@ -107,6 +113,8 @@ module Profiler
       # The request and response bodies kept in a profile stop at this many bytes; the
       # application still reads and sends all of them. nil keeps whole bodies.
       @max_captured_body_bytes = 256 * 1024
+      # What a profile keeps of the lines its request logs, in bytes. nil keeps every line.
+      @max_captured_log_bytes = 1024 * 1024
       @redact_sensitive_data = true
       @filter_parameters = DEFAULT_FILTER_PARAMETERS.dup
       @env_allowlist = DEFAULT_ENV_ALLOWLIST.dup
@@ -125,6 +133,9 @@ module Profiler
       @cluster_require_secret = true
       @cluster_allowed_slave_urls = []
       @cluster_allow_insecure_http = false
+      # Where the profiler's own errors go (Profiler.log): Rails.logger when nil, or $stderr when
+      # there is none.
+      @logger = nil
     end
 
     # Always a Pathname, whether set from a String or left to its default.
@@ -146,8 +157,8 @@ module Profiler
     end
 
     def memory_warning_threshold=(bytes)
-      warn "[Profiler] memory_warning_threshold is deprecated, set allocated_objects_warning_threshold " \
-           "(a number of objects) instead"
+      Profiler.log_warn("memory_warning_threshold is deprecated, set allocated_objects_warning_threshold " \
+                        "(a number of objects) instead")
       @allocated_objects_warning_threshold = bytes && bytes / AllocationCounter::LEGACY_BYTES_PER_OBJECT
     end
 

data/lib/profiler/console_profiler.rb

@@ -76,7 +76,7 @@ module Profiler
             collector.collect if collector.respond_to?(:collect)
             profile.add_collector_metadata(collector)
           rescue => e
-            warn "Profiler ConsoleProfiler: Collector #{collector.class} failed: #{e.message}"
+            Profiler.log_error("ConsoleProfiler: collector #{collector.class} failed", e)
           end
         end
 

data/lib/profiler/engine.rb

@@ -9,8 +9,9 @@ module Profiler
     config.profiler = ActiveSupport::OrderedOptions.new
 
     initializer "profiler.helpers" do
+      # The hook runs for ActionController::API as well, which has no view helpers.
       ActiveSupport.on_load(:action_controller) do
-        helper Profiler::Engine.helpers
+        helper Profiler::Engine.helpers if respond_to?(:helper)
       end
     end
   end

data/lib/profiler/env_override_store.rb

@@ -108,10 +108,10 @@ module Profiler
 
       @boot_warning_logged = true
       noun, verb = count == 1 ? %w[override was] : %w[overrides were]
-      logger&.warn("[Profiler] #{count} persisted environment #{noun} in #{override_file_path} " \
-                   "#{verb} not applied: #{BLOCKED_REASONS.fetch(reason)}.")
+      Profiler.log_warn("#{count} persisted environment #{noun} in #{override_file_path} " \
+                        "#{verb} not applied: #{BLOCKED_REASONS.fetch(reason)}.", logger: logger)
     rescue => e
-      warn "[Profiler] EnvOverrideStore: failed to check overrides at boot: #{e.message}"
+      Profiler.log_error("EnvOverrideStore: failed to check overrides at boot", e)
     end
 
     # Set by the test runner in the environment of the test process, to which it already gave
@@ -158,7 +158,7 @@ module Profiler
         end
       end
     rescue => e
-      warn "[Profiler] EnvOverrideStore: failed to apply overrides: #{e.message}"
+      Profiler.log_error("EnvOverrideStore: failed to apply overrides", e)
     end
 
     # Returns active overrides (excludes RESTORE entries, already being reverted)