logsy 0.1.0 → 0.2.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: eee92d3c7715964e65bd1a5f5efb80e07328bd8fdf3de735d07664c429f19623
-  data.tar.gz: 3a5a149095b25d6cb00e662f96c5d235c1e35dcb1e728f9977676a49ed039e6b
+  metadata.gz: 8d9d026e66871c418c7315bbec120182f0aa3a2e4caa247e9fa533a4a6b99361
+  data.tar.gz: caf2ad615f13cd2f75954e43dfcab5d61e93ec40b11c004bf9723bd8c4736721
 SHA512:
-  metadata.gz: 6035309f83d88ad2a07745d01331d38ecff83254685e9b76d204de8f65b2cba0e86246b192dea878ee1231b8e7f98a0d435cd0d80166e1f37422046ab588db2a
-  data.tar.gz: 2fd3061b4df673dd3bc186d0f61bc4e4ac945de8d1f45a6b1d4cf24025f95db834a65a3e453e6aefdc088bcc079a06c0fe60e2635a2b510e830449dcee470a6d
+  metadata.gz: 9bed718fb7a7618cd420220579e7edd7ce92b44d46f538a0ef2459644014fc207febe0c09cc51a3b72624d083930dc78a37e00752bab2ef6fd34f4291c6677f0
+  data.tar.gz: 2e2d8bd7a250600dedbf8e704e87227cd5ff590c8f9fee39276a1a909b8b1973d51db5f09e6a415ccd375a5222b9bef87039958e80a9c44b3718d4c5724c7d05

data/CHANGELOG.md

@@ -1,5 +1,15 @@
 # Changelog
 
+## [0.2.0] - 2026-06-07
+
+- **`Logsy::RackMiddleware` + Railtie**: request_id is now captured in Rack middleware (auto-inserted after `ActionDispatch::RequestId`) instead of a controller `before_action`. Every log line of a request now carries `request_id` — including "Started GET ..." and "Processing by ..." which a controller callback fires too late to tag. Falls back to `X-Request-Id`, then a generated UUID, so the tag is always present.
+- **Breaking-ish**: `Logsy::ControllerHooks` no longer registers the request-id `before_action` (the middleware supersedes it). It still emits the wide event.
+- `JsonFormatter`: `file` and `line` merged into a single `file` field (`"app/models/order.rb:42"`).
+- `JsonFormatter`: caller-location now skips gem/stdlib frames and attributes the line to the nearest *app* frame (under `Rails.root`, excluding `vendor/bundle`) — an ActiveRecord SQL line points at your controller/model code, not `active_record/log_subscriber.rb`. Omitted when no app frame exists.
+- `JsonFormatter`: String messages are cleaned — ANSI color codes stripped (ActiveRecord SQL highlighting) and surrounding whitespace removed.
+- `JsonFormatter`: caller-location lookup walks the stack in batches of 16 frames and gives up after `caller_location_max_depth` frames (default 64, configurable) instead of capturing the full 100+-frame Rails stack on every line. Typical app log line: ~7µs; framework-only line: bounded by the cap.
+- `ControllerHooks`: the wide event no longer reports a wrong `status` when the action raised (the final status is decided by `rescue_from` after the hook unwinds); `status` is omitted on error, with `error` carrying the class/message.
+
 ## [0.1.0] - Unreleased
 
 - Initial release.

data/README.md

@@ -25,7 +25,11 @@ config.logger = ActiveSupport::Logger.new($stdout)
   .tap { |l| l.formatter = Logsy::JsonFormatter.new }
 ```
 
-### 2. Include the controller hooks
+### 2. Request id — automatic
+
+In a Rails app there is nothing to do: a Railtie inserts `Logsy::RackMiddleware` right after `ActionDispatch::RequestId`, so `Logsy[:request_id]` is set before *any* request log line is emitted — including Rails' own "Started GET ..." and "Processing by ..." lines. It uses `X-Request-Id` when the caller (or your proxy) sends one, otherwise the UUID Rails generated, otherwise generates one itself — the tag is always present.
+
+### 3. Include the controller hooks (wide event)
 
 ```ruby
 # app/controllers/application_controller.rb
@@ -34,11 +38,9 @@ class ApplicationController < ActionController::API
 end
 ```
 
-This gives you:
-- `Logsy[:request_id]` automatically populated from `request.request_id` (or `X-Request-Id` header) on every request
-- One "wide event" log line at end of each request with `event: "request"`, method, path, controller, action, status, duration_ms, and any error class/message — plus every tag you set during the request
+This emits one "wide event" log line at end of each request with `event: "request"`, method, path, controller, action, status, duration_ms, and any error class/message — plus every tag you set during the request. (On an unhandled error, `status` is omitted — the final status is decided by `rescue_from` after the hook unwinds — and `error` carries the class/message.)
 
-### 3. Set tags wherever they become known
+### 4. Set tags wherever they become known
 
 ```ruby
 # In a controller, model, service, anywhere:
@@ -51,7 +53,7 @@ end
 
 That's it. No `Current` model to define, no attribute declarations — just key/value.
 
-### 4. (Optional) Background job propagation
+### 5. (Optional) Background job propagation
 
 If you use Sidekiq:
 
@@ -92,9 +94,8 @@ A regular log line:
 {
   "ts":"2026-05-02T10:00:00.123Z",
   "level":"INFO",
-  "msg":"Calling SPG gateway",
-  "file":"app/lib/gateways/spg_gateway.rb",
-  "line":437,
+  "msg":"Calling payment gateway",
+  "file":"app/services/billing_service.rb:437",
   "request_id":"abc-123",
   "user_id":"u-1",
   "message_id":"m-42",
@@ -124,6 +125,11 @@ The wide event at end of request:
 
 Search your log store by `message_id` to find every request from that message. Pivot from a request's `request_id` to see every breadcrumb log line emitted while it ran.
 
+Notes on the fields:
+
+- **`file`** points at the nearest *app* frame — gem, stdlib, and logging-machinery frames are skipped, so an ActiveRecord SQL line is attributed to the controller/model code that ran the query, not to `active_record/log_subscriber.rb`. Framework-only lines with no app frame (e.g. "Started GET ...") omit `file` rather than show a useless gem path.
+- **`msg`** is cleaned: ANSI color codes (ActiveRecord's SQL highlighting) are stripped and surrounding whitespace removed, so the text is plain and searchable.
+
 ## API reference
 
 ```ruby
@@ -167,6 +173,11 @@ Logsy.configure do |c|
   # Disable if you measure overhead. Default: true
   c.include_caller_location = true
 
+  # How many stack frames to inspect before giving up on attributing a log
+  # line to app code. Bounds the cost of framework-only lines that have no
+  # app frame anywhere in the stack. Default: 64
+  c.caller_location_max_depth = 64
+
   # Regex patterns for caller frames to skip. Defaults already cover Logger,
   # ActiveSupport, lograge, sprockets, quiet_assets, and Logsy itself.
   c.ignored_caller_paths += [%r{/my_internal_logger/}]

data/lib/logsy/configuration.rb

@@ -16,13 +16,19 @@ module Logsy
     # these). The middleware that does the actual carrying is job-runner
     # specific (Logsy ships one for Sidekiq).
     attr_accessor :job_propagated_keys, :ignored_caller_paths,
-                  :include_caller_location, :request_summary_event_name
+                  :include_caller_location, :request_summary_event_name,
+                  :caller_location_max_depth
 
     def initialize
       @job_propagated_keys        = [:request_id]
       @ignored_caller_paths       = DEFAULT_IGNORED_CALLER_PATHS.dup
       @include_caller_location    = true
       @request_summary_event_name = 'request'
+      # How many stack frames to inspect before giving up on attributing
+      # the log line to app code. App frames sit within a few dozen frames
+      # of the logger; framework-only lines (no app frame at all) would
+      # otherwise pay a full walk of a 100+ deep Rails stack on every line.
+      @caller_location_max_depth  = 64
     end
   end
 end

data/lib/logsy/controller_hooks.rb

@@ -3,13 +3,14 @@
 require 'active_support/concern'
 
 module Logsy
-  # Mix into your ApplicationController (or specific controllers) to:
+  # Mix into your ApplicationController (or specific controllers) to emit
+  # a single "wide event" log line at end of request with the method,
+  # path, status, duration_ms, controller, action, error, and every Logsy
+  # tag that was set during the request.
   #
-  #   1. Capture the request's UUID into Logsy.context.request_id, so every
-  #      log line emitted during the request carries it.
-  #   2. Emit a single "wide event" log line at end of request with the
-  #      method, path, status, duration_ms, controller, action, error, and
-  #      every Current.* attribute that was set during the request.
+  # (Request-id capture lives in Logsy::RackMiddleware, inserted
+  # automatically by the Railtie — not here. A controller callback fires
+  # too late to tag the "Started GET" / "Processing by" lines.)
   #
   # Usage:
   #
@@ -27,16 +28,11 @@ module Logsy
     extend ActiveSupport::Concern
 
     included do
-      before_action :_logsy_capture_request_id
       around_action :_logsy_emit_request_summary
     end
 
     private
 
-    def _logsy_capture_request_id
-      Logsy[:request_id] = request.request_id || request.headers['X-Request-Id']
-    end
-
     def _logsy_emit_request_summary
       started_at = Process.clock_gettime(Process::CLOCK_MONOTONIC)
       error = nil
@@ -52,7 +48,11 @@ module Logsy
         path: request.path,
         controller: controller_name,
         action: action_name,
-        status: response&.status,
+        # On an unhandled error the final status is decided by rescue_from /
+        # the exception app *after* this around_action unwinds, so
+        # response.status still holds the pre-error default — omit it
+        # rather than report it wrong.
+        status: error ? nil : response&.status,
         duration_ms:,
         error: error && "#{error.class}: #{error.message}"
       }.merge(logsy_request_summary_extras).compact

data/lib/logsy/json_formatter.rb

@@ -11,8 +11,18 @@ module Logsy
   #   - ts:    UTC ISO-8601 timestamp with millis
   #   - level: severity ('INFO', 'ERROR', ...)
   #   - msg:   the log message (when message is a String or Exception)
-  #   - file/line: source location of the call (when include_caller_location)
-  #   - any non-nil attributes from the configured Logsy.context
+  #   - file:  "path:line" of the nearest app frame (when
+  #            include_caller_location). Gem, stdlib, and ignored frames
+  #            are skipped, so an ActiveRecord SQL line points at the
+  #            controller/model code that ran the query — not at
+  #            active_record/log_subscriber.rb. Omitted entirely for
+  #            framework-only lines (e.g. "Started GET ..."), which have
+  #            no app frame.
+  #   - any non-nil tags from Logsy[]
+  #
+  # String messages are cleaned before emission: ANSI color codes (e.g.
+  # ActiveRecord's SQL highlighting) are stripped and surrounding
+  # whitespace removed, so log stores get plain searchable text.
   #
   # When the message is a Hash, its keys are merged directly into the
   # top-level payload (useful for "wide event" emissions like a request
@@ -20,9 +30,12 @@ module Logsy
   #
   # Example output:
   #   {"ts":"2026-05-02T10:00:00.123Z","level":"INFO","msg":"hello",
-  #    "file":"app/controllers/orders_controller.rb","line":34,
+  #    "file":"app/controllers/orders_controller.rb:34",
   #    "request_id":"abc-123","user_id":"u-1"}
   class JsonFormatter < ::Logger::Formatter
+    ANSI_ESCAPE = /\e\[[0-9;]*m/
+    FRAME_BATCH_SIZE = 16
+
     def call(severity, time, _progname, message)
       payload = {
         ts: time.utc.iso8601(3),
@@ -41,19 +54,59 @@ module Logsy
     private
 
     def add_caller_location!(payload)
-      location = caller_locations.find { |loc| user_frame?(loc.path) }
+      location = find_user_frame
       return unless location
 
-      payload[:file] = relative_path(location.path)
-      payload[:line] = location.lineno
+      payload[:file] = "#{relative_path(location.path)}:#{location.lineno}"
+    end
+
+    # Walk the stack in small batches instead of capturing it all at once:
+    # the frame we want is usually a handful of frames above the logger
+    # machinery, while Rails request stacks run 100+ frames deep —
+    # capturing everything costs ~3x more per log line. The walk is capped
+    # at caller_location_max_depth so framework-only lines (no app frame
+    # anywhere) don't pay for a full-stack scan either.
+    def find_user_frame
+      start = 2 # skip find_user_frame and add_caller_location!
+      limit = start + Logsy.configuration.caller_location_max_depth
+      while start < limit
+        batch = caller_locations(start, [FRAME_BATCH_SIZE, limit - start].min)
+        return nil if batch.nil? || batch.empty?
+
+        batch.each { |loc| return loc if user_frame?(loc.path) }
+        return nil if batch.size < FRAME_BATCH_SIZE
+
+        start += FRAME_BATCH_SIZE
+      end
+      nil
     end
 
+    # A frame worth attributing the log line to: app code, not gems,
+    # stdlib, or logging machinery. With an app root (Rails), only frames
+    # under it count (vendored gems under root excluded); without one,
+    # any non-gem frame counts.
     def user_frame?(path)
-      Logsy.configuration.ignored_caller_paths.none? { |pattern| path.match?(pattern) }
+      return false if Logsy.configuration.ignored_caller_paths.any? { |pattern| path.match?(pattern) }
+
+      if (root = app_root)
+        path.start_with?(root) && !path.include?('/vendor/bundle/')
+      else
+        !gem_frame?(path)
+      end
+    end
+
+    def gem_frame?(path)
+      path.include?('/gems/') ||
+        path.include?('/rubygems/') ||
+        path.start_with?('<internal:', RbConfig::CONFIG['rubylibdir'])
+    end
+
+    def app_root
+      defined?(Rails) && Rails.respond_to?(:root) && Rails.root ? "#{Rails.root}/" : nil
     end
 
     def relative_path(path)
-      root = defined?(Rails) && Rails.respond_to?(:root) && Rails.root ? "#{Rails.root}/" : nil
+      root = app_root
       return path unless root && path.start_with?(root)
 
       path[root.length..]
@@ -70,7 +123,7 @@ module Logsy
     def merge_message!(payload, message)
       case message
       when ::String
-        payload[:msg] = message
+        payload[:msg] = message.gsub(ANSI_ESCAPE, '').strip
       when ::Hash
         message.each { |k, v| payload[k.to_sym] = v }
       when ::Exception

data/lib/logsy/rack_middleware.rb

@@ -0,0 +1,34 @@
+# frozen_string_literal: true
+
+require 'securerandom'
+
+module Logsy
+  # Rack middleware that captures the request id into Logsy[:request_id]
+  # at the earliest useful point in the stack — before Rails::Rack::Logger
+  # emits "Started GET ..." and before ActionController's start_processing
+  # event fires — so *every* log line of the request carries it, not just
+  # the ones emitted after controller callbacks run.
+  #
+  # The Railtie inserts this automatically right after
+  # ActionDispatch::RequestId (which populates `action_dispatch.request_id`
+  # from X-Request-Id or generates a UUID). Outside Rails, falls back to
+  # the X-Request-Id header, then to a generated UUID, so the tag is
+  # always present.
+  #
+  # Tag cleanup between requests is handled by the Rails executor
+  # (ActionDispatch::Executor wraps this middleware and resets
+  # CurrentAttributes); plain Rack apps should call Logsy.reset themselves.
+  class RackMiddleware
+    def initialize(app)
+      @app = app
+    end
+
+    def call(env)
+      Logsy[:request_id] = env['action_dispatch.request_id'] ||
+                           env['HTTP_X_REQUEST_ID'] ||
+                           SecureRandom.uuid
+
+      @app.call(env)
+    end
+  end
+end

data/lib/logsy/railtie.rb

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+module Logsy
+  # Inserts the request-id capturing middleware right after
+  # ActionDispatch::RequestId, before Rails::Rack::Logger — early enough
+  # that the "Started GET ..." line is already tagged.
+  class Railtie < ::Rails::Railtie
+    initializer 'logsy.insert_rack_middleware' do |app|
+      app.middleware.insert_after(ActionDispatch::RequestId, Logsy::RackMiddleware)
+    end
+  end
+end

data/lib/logsy/version.rb

@@ -1,5 +1,5 @@
 # frozen_string_literal: true
 
 module Logsy
-  VERSION = '0.1.0'
+  VERSION = '0.2.0'
 end

data/lib/logsy.rb

@@ -5,6 +5,8 @@ require 'logsy/store'
 require 'logsy/configuration'
 require 'logsy/json_formatter'
 require 'logsy/controller_hooks'
+require 'logsy/rack_middleware'
+require 'logsy/railtie' if defined?(Rails::Railtie)
 
 module Logsy
   class Error < StandardError; end

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: logsy
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.0
 platform: ruby
 authors:
 - ruby_is_love
@@ -95,6 +95,8 @@ files:
 - lib/logsy/configuration.rb
 - lib/logsy/controller_hooks.rb
 - lib/logsy/json_formatter.rb
+- lib/logsy/rack_middleware.rb
+- lib/logsy/railtie.rb
 - lib/logsy/sidekiq_middleware.rb
 - lib/logsy/sidekiq_middleware/client.rb
 - lib/logsy/sidekiq_middleware/server.rb