lognorth 0.1.0 → 0.3.0.rc.2

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: dbd243e69bd9758c158d11868e091edc0f0c629916cebc8094fdcfb74664b0b3
-  data.tar.gz: db2c6d4d5e68353da4ecb2757737595ec144c9ae002202bf3cde5abcacdfa978
+  metadata.gz: 8ee47ef92f738c9a430483e2f35f55e17155e7519b806017943e69a217649d95
+  data.tar.gz: 57995a21f2d38a165604ce4825e1ba4d924b0f13e9c02040c7eee7279997c156
 SHA512:
-  metadata.gz: 1381eee25cc094217d00d294ee381846cdb0390110e2e727c104459c5199918ede21c6157b1dfbd723b39d8fde5cc45b74998e031b352df058d76676a437a57e
-  data.tar.gz: 70a5edefba779fd982adcaf91465c34ec89082242428c953485ba4a9855ddb7852f4950c6e01ba21636ac1c2bbfb6963ef66cbbc3208f6594abc85196b1914f7
+  metadata.gz: ba3de220f46ffe6a576e2b48619d6a7b664f6bbdc80b5db9f910defb959d0b9940b81a4548bb5bd21631882324edf461f8bf3200f0892d509c0bed09a0a6360c
+  data.tar.gz: edc1a9b1795925af3f04c05feef9334a99d5ef4e133b03976f974689a21de952b0664408c3918cdaa03ef150c2c0c76adf3f8e3650af8c1dc69921a2dc01f772

data/README.md

@@ -1,115 +1,156 @@
-# LogNorth Rails SDK
+# lognorth-sdk-rails
 
-Send errors and logs from Rails to [LogNorth](https://lognorth.com) for monitoring and alerting.
+[![CI](https://github.com/karloscodes/lognorth-sdk-rails/actions/workflows/ci.yml/badge.svg)](https://github.com/karloscodes/lognorth-sdk-rails/actions/workflows/ci.yml) [![Gem Version](https://img.shields.io/gem/v/lognorth)](https://rubygems.org/gems/lognorth) [![License](https://img.shields.io/github/license/karloscodes/lognorth-sdk-rails)](LICENSE)
 
-## Installation
+Sends errors, logs, and request events from Rails and Rack apps to LogNorth.
 
-```ruby
-gem "lognorth", github: "karloscodes/lognorth-sdk-rails"
-```
-
-## Rails Setup
+[LogNorth](https://lognorth.com) monitors your app and alerts you when it fails. See the [LogNorth Rails docs](https://lognorth.com/docs/integrations/rails/).
 
-Add credentials:
+## Install
 
-```yaml
-# config/credentials.yml.enc
-lognorth:
-  url: https://your-lognorth-instance.com
-  api_key: your_api_key
+```sh
+bundle add lognorth
 ```
 
-That's it. The gem auto-configures via Railtie.
+## Use
 
-### Manual Configuration
+With credentials set (see [Configuration](#configuration)), Rails sends request events, reported exceptions, and job events on its own. Call the SDK to send your own events:
 
 ```ruby
-# config/initializers/lognorth.rb
-LogNorth.config(
-  ENV["LOGNORTH_URL"],
-  ENV["LOGNORTH_API_KEY"]
-)
+LogNorth.log("User signed up", { user_id: 123 })
+
+begin
+  risky_operation
+rescue => e
+  LogNorth.error("Payment failed", e, { order_id: 456 })
+  raise
+end
+
+LogNorth.flush # sends the queue now, for up to 5 seconds
 ```
 
-### Options
+### Who hit the error, and in which release
+
+The SDK reads `Current.user` when your app defines it, as the Rails authentication generator does. Otherwise, set the user yourself. Use an ID, not an email:
 
 ```ruby
-# config/application.rb
-config.lognorth.enabled = true           # Default: on everywhere except development and test
-config.lognorth.middleware = true        # Log HTTP requests
-config.lognorth.error_subscriber = true  # Report exceptions (Rails 7+)
-
-# Paths to skip from request logging. Defaults to Rails' built-in
-# health-check endpoint; set to [] to log everything or replace with
-# your own list.
-config.lognorth.ignored_paths = ["/up", "/healthz"]
+class ApplicationController < ActionController::Base
+  before_action { LogNorth.user = current_user&.id }
+end
 ```
 
-Default: `["/up"]` (Rails 7.1's auto-generated health check — swamped by
-kamal-proxy and load-balancer pings otherwise). Setting `ignored_paths =
-[]` disables ignoring entirely. Matching is exact path or `path/…`
-prefix, so `/up` also covers `/up/detail`.
+The request event and its errors carry the user.
+
+Errors also carry the release. See [Configuration](#configuration) for how the release is set.
+
+## Middleware
+
+The middleware logs each request as one event. The event has the method, path, status, duration, controller, action, and route (`/users/:id`, Rails 7.1 and later). LogNorth groups requests into endpoints by the route. It also sets an `X-Trace-ID` response header. It reuses an incoming `X-Trace-ID` request header when one is set.
+
+Requests that match no route are not logged. A 5xx request also carries its user agent, and it sends the queue at once.
 
 ### Client errors are not errors
 
-An exception that Rails answers with a 4xx is the request's fault, not the app's:
-`ActiveRecord::RecordNotFound` (404), `ActionController::InvalidAuthenticityToken`
-(422), `ActionController::ParameterMissing` (400). The SDK logs the request with
-that status and does not report an error, so these never become issues or alerts.
+An exception that Rails answers with a 4xx is the request's fault, not the app's. Examples are `ActiveRecord::RecordNotFound` (404), `ActionController::InvalidAuthenticityToken` (422), and `ActionController::ParameterMissing` (400). The SDK logs the request with that status. It does not report an error, so these never become issues or alerts.
 
-Rails decides the status from `config.action_dispatch.rescue_responses`, and the SDK
-asks the same map. To report one of them as an error, map it to a 5xx:
+Rails sets the status in `config.action_dispatch.rescue_responses`. The SDK reads the same map. To report one of them as an error, map it to a 5xx:
 
 ```ruby
 config.action_dispatch.rescue_responses["ActiveRecord::RecordNotFound"] = :internal_server_error
 ```
 
-## Usage
+### Without Rails
 
-```ruby
-# Log messages (batched, sent after 5s or at 10 events)
-LogNorth.log("User signed up", { user_id: 123 })
+Use the middleware in any Rack app:
 
-# Report errors (sent at once)
-begin
-  risky_operation
-rescue => e
-  LogNorth.error("Payment failed", e, { order_id: 456 })
-  raise
-end
+```ruby
+require "lognorth"
 
-# Manual flush (called automatically at exit)
-LogNorth.flush
+LogNorth.config("https://lognorth.example.com", "api_key")
+use LogNorth::Middleware
 ```
 
-## Batching and delivery
+## How it works
 
-Logging calls never block and never raise. They add the event to a queue in memory.
-One background thread sends the queue, one request at a time.
+### Batching
 
-- The thread sends when the queue holds 10 events, when you report an error, or 5 seconds after the first event.
+Logging calls never wait for the network. Each call adds the event to a queue in memory. One background thread sends the queue, one request at a time.
+
+- The thread sends when the queue holds 10 events, when you report an error, or 5 seconds after the first queued event.
 - A request holds at most 500 events and 1 MB of JSON.
 - The queue holds at most 10,000 events or 10 MB of JSON.
-- The SDK trims an event to at most 64 KB before it enters the queue. It marks a trimmed event with `context.truncated = true`.
+- A message over 1,000 characters is cut. A long string in the context is cut.
+- An event still over 64 KB keeps only its essential context keys. The SDK sets `context.truncated` to `true` on any cut event.
+
+### Failed sends
+
+The SDK keeps the events and sends them again later. Events keep their order.
+
+- A network error, a timeout, a 408, or a 5xx waits 1, 2, 4 seconds, and so on, up to 60 seconds. Each wait varies by up to 20%.
+- A 429 or 503 waits as long as the `Retry-After` header says, from 1 to 300 seconds. Without the header, the SDK uses the normal wait.
+- A 401, 403, 404, or other unexpected status waits 60 seconds, then 120, then 240, up to 300 seconds. It writes a line to stderr when delivery starts failing. Check the URL and the API key.
+- Another 4xx, such as 400, 413, or 422, splits the batch in two and sends each half. A single event that the server still refuses is dropped.
+
+When the queue is full, the SDK drops the oldest event that is not an error. After the next successful send, it logs `LogNorth client dropped N events`. The context holds the counts of dropped events and dropped errors.
+
+### Exit
 
-When a send fails, the SDK keeps the events and tries again later. Events keep their order.
+At exit, the SDK sends the queue for up to 5 seconds. Ruby runs this exit hook on SIGTERM and SIGINT too, unless the app traps them. The SDK writes the number of unsent events to stderr.
 
-- On a network error, a timeout, or a 5xx, it waits 1 second, then 2, then 4, up to 60.
-- On a 429 or 503, it waits as long as the `Retry-After` header says.
-- On a 401, 403, or 404, it waits 60 seconds, then up to 5 minutes. It writes one line to stderr, so you can fix the url or the key.
-- On a 413 or other 4xx, it splits the batch in two and sends again. It drops a single event the server still refuses.
+### Errors and jobs
 
-When the queue is full, the SDK drops the oldest event that is not an error. It keeps errors longest.
-After the next successful send, it logs `LogNorth client dropped N events` with the counts.
+- `Rails.error` reports go to LogNorth. Each event adds `handled`, `severity`, and `source` to the context.
+- With `config.lognorth.active_job` on, each job logs `<Job> completed` or `<Job> failed`. A job carries the trace ID of the request that enqueued it.
+
+## Configuration
+
+The SDK reads the URL and API key from Rails credentials:
+
+```yaml
+# config/credentials.yml.enc
+lognorth:
+  url: https://your-lognorth-instance.com
+  api_key: your_api_key
+```
 
-At exit (including SIGTERM and SIGINT), the SDK sends what is left in the queue for up to 5 seconds.
-It writes the number of events it could not send to stderr.
+Without a URL and an API key, the SDK logs a warning at boot and sends nothing.
 
-## Rack (without Rails)
+Rails sets the environment name for you. Set `release:` yourself, or let the SDK read it from the environment. To configure the SDK by hand, call `LogNorth.config` in an initializer:
 
 ```ruby
-require "lognorth"
+# config/initializers/lognorth.rb
+LogNorth.config(
+  ENV["LOGNORTH_URL"],
+  ENV["LOGNORTH_API_KEY"],
+  environment: "production", # optional
+  release: "1.2.3"           # optional
+)
+```
 
-LogNorth.config("https://lognorth.example.com", "api_key")
-use LogNorth::Middleware
+When no `release:` is set, the SDK reads the first of these variables that is set: `LOGNORTH_RELEASE`, `GIT_SHA`, `GIT_COMMIT`, `SOURCE_COMMIT`, `KAMAL_VERSION`, `RENDER_GIT_COMMIT`, `HEROKU_SLUG_COMMIT`, `SOURCE_VERSION`, `RAILWAY_GIT_COMMIT_SHA`, `VERCEL_GIT_COMMIT_SHA`. When a release is set, the SDK logs `Release <version> started` once, when it configures.
+
+Options, in `config/application.rb`:
+
+```ruby
+config.lognorth.enabled = true           # Default: on, except in development and test
+config.lognorth.middleware = true        # Request events
+config.lognorth.error_subscriber = true  # Rails.error reports (Rails 7.0+)
+config.lognorth.active_job = true        # Job events
+
+# Requests to skip. Default: Rails' health check, which load balancers call often.
+# Set [] to log everything.
+config.lognorth.ignored_paths = ["/up"]
+```
+
+An ignored path matches the exact path, or the path followed by `/`. So `/up` also skips `/up/detail`.
+
+## Development
+
+```sh
+bundle install
+bundle exec rake test
 ```
+
+## License
+
+MIT. See [LICENSE](LICENSE).

data/lib/lognorth/client.rb

@@ -15,7 +15,12 @@ module LogNorth
   MAX_STACK_TRACE_BYTES = 16 * 1024
   MAX_STRING_BYTES = 8 * 1024
   # The context keys an event keeps when it is still too big after trimming.
-  ESSENTIAL_KEYS = %w[error error_class error_file error_line method path status environment].freeze
+  ESSENTIAL_KEYS = %w[error error_class error_file error_line method path status environment release user].freeze
+  # Where deploy tools put the version that runs. The first one set wins.
+  RELEASE_ENV = %w[
+    LOGNORTH_RELEASE GIT_SHA GIT_COMMIT SOURCE_COMMIT KAMAL_VERSION
+    RENDER_GIT_COMMIT HEROKU_SLUG_COMMIT SOURCE_VERSION RAILWAY_GIT_COMMIT_SHA VERCEL_GIT_COMMIT_SHA
+  ].freeze
 
   module Client
     # An event in the queue, with its JSON size and whether it is an error.
@@ -39,6 +44,8 @@ module LogNorth
     @endpoint = nil
     @api_key = nil
     @environment = nil
+    @release = nil
+    @announced = nil # the release this process last said it started
 
     # Wait times in seconds. Tests set them lower.
     @flush_interval = 5
@@ -53,13 +60,15 @@ module LogNorth
     class << self
       attr_accessor :debug
 
-      def config(url, key, environment: nil)
+      def config(url, key, environment: nil, release: nil)
         @mutex.synchronize do
           @endpoint = url.chomp("/")
           @api_key = key
           @environment = environment
+          @release = release || release_from_env
         end
         log_debug("configured with url=#{url} env=#{environment.inspect}")
+        announce_release
       end
 
       def configured?
@@ -82,6 +91,22 @@ module LogNorth
         Thread.current[:lognorth_trace_id] = id
       end
 
+      # The user of the request on this thread: the one set with LogNorth.user=,
+      # or else Current.user, which the Rails authentication generator defines.
+      def current_user
+        Thread.current[:lognorth_user] || user_from_current_attributes
+      end
+
+      def current_user=(id)
+        Thread.current[:lognorth_user] = id&.to_s
+      end
+
+      # Forgets the trace ID and the user when a request ends.
+      def end_request
+        Thread.current[:lognorth_trace_id] = nil
+        Thread.current[:lognorth_user] = nil
+      end
+
       def send_event(message, context = {}, trace_id: nil, duration_ms: nil, timestamp: nil)
         return unless configured?
 
@@ -89,7 +114,7 @@ module LogNorth
         event = {
           message: message,
           timestamp: (timestamp || Time.now).utc.iso8601(3),
-          context: stamp_environment(context)
+          context: stamp_user(stamp_environment(context))
         }
         event[:trace_id] = trace_id if trace_id
         event[:duration_ms] = duration_ms if duration_ms
@@ -115,14 +140,14 @@ module LogNorth
         event = {
           message: message,
           timestamp: (timestamp || Time.now).utc.iso8601(3),
-          context: stamp_environment(context.merge(
+          context: stamp_user(stamp_environment(context.merge(
             error: exception.message,
             error_class: exception.class.name,
             error_file: error_file,
             error_line: error_line,
             error_caller: error_caller,
             stack_trace: exception.backtrace&.first(20)&.join("\n")
-          ))
+          )))
         }
         event[:trace_id] = trace_id if trace_id
         event[:duration_ms] = duration_ms if duration_ms
@@ -169,8 +194,48 @@ module LogNorth
         context.merge(environment: env)
       end
 
+      # Logs "Release <version> started" once per release, when the client is
+      # configured. LogNorth takes the first start of a release as its deploy
+      # time and marks it on its charts.
+      def announce_release
+        release = @mutex.synchronize do
+          next if @release.nil? || @release == @announced
+
+          @announced = @release
+        end
+        send_event("Release #{release} started", { release: release }) if release
+      end
+
+      def stamp_user(context)
+        user = current_user
+        return context if user.nil? || fetch(context, :user)
+
+        context.merge(user: user)
+      end
+
+      def user_from_current_attributes
+        return unless defined?(::Current) && ::Current.respond_to?(:user)
+
+        ::Current.user&.id&.to_s
+      rescue StandardError
+        nil
+      end
+
+      # Errors carry the release: that is where it answers which deploy broke
+      # something. Other events stay small.
+      def stamp_release(event)
+        release = @mutex.synchronize { @release }
+        return event if release.nil? || !error_event?(event[:context]) || fetch(event[:context], :release)
+
+        event.merge(context: event[:context].merge(release: release))
+      end
+
+      def release_from_env
+        RELEASE_ENV.lazy.map { |k| ENV[k].to_s.strip }.find { |v| !v.empty? }
+      end
+
       def enqueue(event, urgent: false)
-        entry = new_entry(trim(event))
+        entry = new_entry(trim(stamp_release(event)))
         @mutex.synchronize do
           push(entry)
           @urgent = true if urgent

data/lib/lognorth/middleware.rb

@@ -9,7 +9,10 @@ module LogNorth
     end
 
     def call(env)
+      # A mounted engine or Rack app leaves PATH_INFO and SCRIPT_NAME as it
+      # saw them, so read both before the call.
       path = env["PATH_INFO"]
+      script_name = env["SCRIPT_NAME"].to_s
 
       # Skip ignored paths (health checks, metrics, etc.)
       if ignored_path?(path)
@@ -26,18 +29,19 @@ module LogNorth
 
       # Don't track requests that didn't match any route (scanner noise)
       if status == 404 && !env["action_controller.instance"]
-        LogNorth::Client.current_trace_id = nil
+        LogNorth::Client.end_request
         return [status, headers, response]
       end
 
       duration_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000).round
       headers["X-Trace-ID"] = trace_id
 
-      context = { method: env["REQUEST_METHOD"], path: env["PATH_INFO"], status: status }
-      merge_route_info!(context, env)
+      context = { method: env["REQUEST_METHOD"], path: path, status: status }
+      merge_route_info!(context, env, script_name)
+      merge_user_agent!(context, env, status)
 
       LogNorth::Client.send_event(
-        "#{env['REQUEST_METHOD']} #{env['PATH_INFO']} → #{status}",
+        "#{env['REQUEST_METHOD']} #{path} → #{status}",
         context,
         trace_id: trace_id,
         duration_ms: duration_ms,
@@ -46,7 +50,7 @@ module LogNorth
 
       LogNorth.flush if status >= 500
 
-      LogNorth::Client.current_trace_id = nil
+      LogNorth::Client.end_request
       [status, headers, response]
     rescue StandardError => e
       duration_ms = ((Process.clock_gettime(Process::CLOCK_MONOTONIC) - start) * 1000).round
@@ -54,23 +58,23 @@ module LogNorth
         # Rails turns this into a 4xx further out. Log the request with that
         # status, like any other request, instead of an error.
         status = LogNorth.response_status_for(e)
-        context = { method: env["REQUEST_METHOD"], path: env["PATH_INFO"], status: status }
-        merge_route_info!(context, env)
+        context = { method: env["REQUEST_METHOD"], path: path, status: status }
+        merge_route_info!(context, env, script_name)
         LogNorth::Client.send_event(
-          "#{env['REQUEST_METHOD']} #{env['PATH_INFO']} → #{status}", context,
+          "#{env['REQUEST_METHOD']} #{path} → #{status}", context,
           trace_id: trace_id, duration_ms: duration_ms, timestamp: start_time
         )
-        LogNorth::Client.current_trace_id = nil
+        LogNorth::Client.end_request
         raise
       end
       LogNorth::Client.send_error_event(
-        "Request failed: #{env['REQUEST_METHOD']} #{env['PATH_INFO']}", e,
-        { method: env["REQUEST_METHOD"], path: env["PATH_INFO"] },
+        "Request failed: #{env['REQUEST_METHOD']} #{path}", e,
+        merge_user_agent!({ method: env["REQUEST_METHOD"], path: path }, env, 500),
         trace_id: trace_id,
         duration_ms: duration_ms,
         timestamp: start_time
       )
-      LogNorth::Client.current_trace_id = nil
+      LogNorth::Client.end_request
       raise
     end
 
@@ -83,10 +87,18 @@ module LogNorth
       ignored_paths.any? { |p| path == p || path.start_with?("#{p}/") }
     end
 
+    # The user agent tells a bot from a browser on a failed request. Only
+    # failed ones carry it, to keep every other event small.
+    def merge_user_agent!(context, env, status)
+      agent = env["HTTP_USER_AGENT"].to_s
+      context[:user_agent] = agent if status.to_i >= 500 && !agent.empty?
+      context
+    end
+
     # Rails populates action_controller.instance after dispatch so the
-    # controller/action names — and the named route when available —
-    # can be pulled straight off the env.
-    def merge_route_info!(context, env)
+    # controller/action names — and the route — can be pulled straight off
+    # the env.
+    def merge_route_info!(context, env, script_name)
       controller = env["action_controller.instance"]
       return unless controller
 
@@ -94,6 +106,21 @@ module LogNorth
       action = controller.respond_to?(:action_name) ? controller.action_name : nil
       context[:controller] = klass if klass
       context[:action] = action if action
+      route = route_of(env, script_name)
+      context[:route] = route if route
+    end
+
+    # The route as `bin/rails routes` prints it (Rails 7.1+), without the
+    # optional format: "/users/:id". An engine's route is relative to its
+    # mount point, which Rails added to SCRIPT_NAME.
+    def route_of(env, script_name)
+      return unless defined?(ActionDispatch::Request) && ActionDispatch::Request.method_defined?(:route_uri_pattern)
+
+      pattern = ActionDispatch::Request.new(env).route_uri_pattern
+      return unless pattern
+
+      mount = env["SCRIPT_NAME"].to_s.delete_prefix(script_name)
+      mount + pattern.delete_suffix("(.:format)")
     end
   end
 end

data/lib/lognorth.rb

@@ -7,8 +7,17 @@ require_relative "lognorth/railtie" if defined?(Rails::Railtie)
 
 module LogNorth
   class << self
-    def config(url, key, environment: nil)
-      Client.config(url, key, environment: environment)
+    def config(url, key, environment: nil, release: nil)
+      Client.config(url, key, environment: environment, release: release)
+    end
+
+    # Names the user of the current request: an ID, not an email. The request
+    # event and its errors carry it, so an issue shows how many users it hit.
+    # Without it, LogNorth reads Current.user when the app defines one.
+    #
+    #   before_action { LogNorth.user = current_user&.id }
+    def user=(id)
+      Client.current_user = id
     end
 
     def log(message, context = {})

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: lognorth
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.3.0.rc.2
 platform: ruby
 authors:
 - LogNorth