rails_tracepoint_stack 0.3.4 → 0.4.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: a3045ee154f570e6a780af9aff0edfcd18822b3394a0f268ee9a4e07e6d5769f
-  data.tar.gz: 9865f094f0a70ffcb474148aaab7ee6712ac63450326bf7e3385dfc55e1c123f
+  metadata.gz: 612ce09f08d0dfebf77cfddc8dfbe9712598aaf324579f688bac6148354c92fb
+  data.tar.gz: aec94f6a1a148f16199b6835932a5b1fe91ff458a2915fab60efe937244b7cac
 SHA512:
-  metadata.gz: 6c5482941d4e5119b88133eca986cdb87c490d0ed6e69ed0df5a09ebe1d2efebff0c4abbf9b554abf44cbab80038b96b3bf53cb8c86b1052b967d7ad307573d8
-  data.tar.gz: 699757a6965a6101ee9b80e5039580840a5e34de1271538a286d95c6ea5d2f0e678e666c8f098b2a93df5896868fc821182258f4d07616684bace6857087e7b8
+  metadata.gz: de2e858cfad1406ee8becdb5d3be65b956286c802a1f95023d8698eaa80500569ee603be1b28e06404845a41eebb25e364ea936cdd886fcdab02b14113caaf10
+  data.tar.gz: 17b4beea42cb83b2058b32652acf0ddeec66a1b2fabe32e61a07fcf446738fd4c92b238dd6ce603d5426d68af4e1bbe397e23c7afd7c739c4385ed3d8cf61177

data/README.md

@@ -0,0 +1,174 @@
+# rails_tracepoint_stack
+
+**[carlosdanielpohlod.github.io/rails_tracepoint_stack](https://carlosdanielpohlod.github.io/rails_tracepoint_stack/)**
+
+See what your Rails code actually did at runtime: which of *your* methods ran,
+what arguments they got, what each one returned, and where an exception was
+first raised — as a call tree, with gems, the framework and stdlib filtered out.
+
+```ruby
+session = RailsTracepointStack.capture(max_depth: 4) do
+  Order.find(42).recalculate!
+end
+
+puts session.to_tree
+```
+
+```
+Order#recalculate! (app/models/order.rb:88) {}
+  Order#apply_discount (app/models/order.rb:102) {"total":200.0}
+    Discount#rate_for (app/models/discount.rb:12) {"order":"#<Order id: 42>"}
+      -> null
+    -> 200.0
+  -> 200.0
+3 calls, 3 returns, 0 raises, 2 classes
+```
+
+`rate_for` returned nil. No `puts`, no log file, no restart.
+
+A whole Rails request comes out about this long, because everything the
+framework runs underneath is filtered away:
+
+```
+RealEstateAgenciesController#index (app/controllers/real_estate_agencies_controller.rb:14) {}
+  IpLocation.guess_state_uf_and_city_name (app/services/ip_location.rb:15) {"ip":"127.0.0.1"}
+    -> [null,null]
+  -> null
+render app/views/real_estate_agencies/index.html.erb {}
+  -> "<!-- BEGIN app/views/real_estate_agencies/index.html.erb\n-->… (2507 chars)"
+4 calls, 4 returns, 0 raises, 3 classes
+```
+
+That is 4 kept traces out of 13869 the tracer saw.
+
+## Install
+
+```ruby
+# Gemfile
+gem "rails_tracepoint_stack"
+```
+
+## Debugging with an AI agent
+
+Agents reach for `puts` and log lines because they do not know this exists.
+Install the packaged skill into your app and yours will use the gem instead:
+
+```bash
+bin/rails generate rails_tracepoint_stack:install
+```
+
+That writes `.claude/skills/debug-with-tracepoint/SKILL.md`, which tells the
+agent when tracing beats reading code, how to keep the output inside its
+context window, and how to read the tree. Pass `--force` to overwrite an
+existing copy.
+
+## Capturing
+
+`capture` traces the block, returns a session and re-raises anything the block
+raised. It watches only the calling thread, so it stays clean under Puma.
+
+```ruby
+session = RailsTracepointStack.capture { Order.find(42).recalculate! }
+
+session.to_tree    # the indented call tree above
+session.as_json    # the same data, structured, one entry per trace
+session.summary    # {calls:, returns:, raises:, classes:, truncated:}
+session.result     # what the block returned
+session.error      # the exception that escaped, if any
+session.traces     # the raw TraceRecord list
+```
+
+To keep the traces when the block blows up, take the session from the block
+argument:
+
+```ruby
+session = nil
+begin
+  RailsTracepointStack.capture { |s| session = s; thing_that_blows_up }
+rescue => error
+  puts session.to_tree
+end
+```
+
+### Keeping the output small
+
+A real request produces tens of thousands of traces, so captures are bounded.
+
+| Option | Default | What it does |
+|---|---|---|
+| `max_depth` | none | Drops traces nested deeper than this |
+| `max_traces` | 5000 | Stops collecting; `session.truncated?` becomes true |
+| `max_string_length` | 200 | Shortens long strings |
+| `max_collection_size` | 20 | Shortens long arrays and hashes |
+| `capture_params` | `true` | Set false to show only the flow |
+| `capture_return` | `true` | Set false to show only the calls |
+| `threads` | `:current` | `:all` also records background threads |
+
+```ruby
+RailsTracepointStack.capture(max_depth: 3, capture_return: false) { ... }
+```
+
+## Tracing the whole process
+
+For code you cannot wrap in a block — boot, a rake task, a request handled by a
+running server — enable the tracer globally instead. Output goes to
+`log/rails_tracepoint_stack.log`.
+
+```bash
+RAILS_TRACEPOINT_STACK_ENABLED=true bin/rails server
+```
+
+```
+called: Bar#perform in /path/to/app/services/bar.rb:12 with params: {}
+```
+
+`RailsTracepointStack.enable_trace { ... }` does the same for one block, also
+writing to the log rather than returning a session.
+
+## Configuration
+
+```ruby
+# config/initializers/rails_tracepoint_stack.rb
+
+RailsTracepointStack.configure do |config|
+  config.file_path_to_filter_patterns << %r{app/services/}
+  config.ignore_patterns << %r{app/models/concerns/}
+  config.log_format = :json
+  config.log_external_sources = false
+  config.logger = Rails.logger
+end
+```
+
+| Configuration | Description |
+|---|---|
+| `file_path_to_filter_patterns` | Trace **only** files whose path matches one of these patterns |
+| `ignore_patterns` | Skip traces whose file path matches one of these patterns |
+| `log_format` | `:text` (default) or `:json`. Applies to the log, not to `capture` |
+| `log_external_sources` | Include gems, bundler and stdlib. Default `false` |
+| `logger` | Your own logger. Defaults to `log/rails_tracepoint_stack.log` |
+
+A method missing from a trace is usually one the filters dropped as external —
+add its path to `file_path_to_filter_patterns` to force it in.
+
+## Worth knowing
+
+- TracePoint makes the traced code noticeably slower — around 8x on a real
+  Rails request. Fine for an investigation, not for something left running in
+  production.
+- Ruby >= 3.0.
+- A `-> null` line directly under a `!!` line is the frame unwinding, not a
+  method that returned nil.
+- An empty tree says `no app code ran` along with how much was filtered. That
+  usually means the block ran entirely inside gems — `Order.where(...).map(&:name)`
+  calls no method you wrote, since `name` is generated by ActiveRecord.
+
+## Links
+
+- [Website](https://carlosdanielpohlod.github.io/rails_tracepoint_stack/) — the
+  same story with real captured output, including a whole Rails request
+- [RubyGems](https://rubygems.org/gems/rails_tracepoint_stack)
+- [Changelog](changelog.md)
+
+## License
+
+MIT

data/changelog.md

@@ -0,0 +1,132 @@
+# Changelog
+
+## 0.4.0
+
+**Features**
+
+- `RailsTracepointStack.capture` traces a block and returns the traces in
+  memory, with no env var, log file or restart involved
+- Traces now carry return values and raised exceptions, not just calls and params
+- Each trace knows its call depth, so a session renders as a call tree
+- `session.to_tree`, `session.as_json` and `session.summary` render a capture
+- Captures are bounded by `max_depth`, `max_traces`, `max_string_length` and
+  `max_collection_size`, and report `truncated?` when a limit cut them short
+- Captures watch only the calling thread by default; `threads: :all` opts out
+- `rails g rails_tracepoint_stack:install` writes an agent skill into the app,
+  so AI coding agents know when and how to trace it
+
+**BugFix**
+
+- Params reported every local the method body declared, not just the arguments
+  it received, so unassigned locals showed up as arguments passed as nil.
+  `TracePoint#parameters` now decides what is read from the binding
+- `GemPath` referenced `Bundler` without requiring it, raising `NameError` on
+  the first trace outside a bundle
+- A raise coming out of a C method landed one level too deep in the tree
+
+**Changes**
+
+- The tracer writes to a sink; the default one keeps the previous logging
+  behaviour, so global tracing is unchanged
+- Class methods render as `Foo.bar` rather than `#<Class:Foo>#bar`
+- Compiled templates render as `render app/views/…` with their locals, instead
+  of the generated method name and the rendering internals
+- An empty capture reports how many traces the filters dropped, so "nothing
+  ran" is distinguishable from "nothing of yours ran"
+- Serialized string values are frozen copies, so a snapshot no longer follows
+  later mutations of the traced object
+
+## 0.3.5
+
+**BugFix**
+
+- Sanitize traced params before formatting so JSON logging does not overflow on recursive or problematic objects [Issue #37](https://github.com/carlosdanielpohlod/rails_tracepoint_stack/issues/37)
+- Stabilize text log param rendering across Ruby versions
+
+## 0.3.4
+
+**Changes**
+
+- Fix flaky tests by @danielmbrasil [PR #27](https://github.com/carlosdanielpohlod/rails_tracepoint_stack/pull/27)
+
+- Huge refactor of filters class, separating in modules by filter type [PR #18](https://github.com/carlosdanielpohlod/rails_tracepoint_stack/pull/18)
+
+- Some other code refactors
+
+## 0.3.3
+
+**Changes**
+
+- Add autoload of all lib files on test_helper
+- Fix some tests
+
+**BugFix**
+
+- Fixed the Configuration module not loading the default value for the configuration attributes
+
+## 0.3.1
+
+**Changes:**
+
+- Add the ability to include the external sources to the log using `config.log_external_sources = true`
+
+## 0.3.0
+
+**Changes:**
+
+- Refactor classes, formatting a trace using a value object class `RailsTracepointStack::Trace`
+- include configuration `log_format` option, allowing choose an output as `text` or `json`
+- Include configuration `file_path_to_filter_patterns`, allowing filter traces only when the origin file path matches a pattern
+- Improve test coverage
+
+## 0.2.1
+
+**Changes:**
+
+- Update the ENV enable to be more semantic
+- Add the VERSION constant module
+- Sorted the files inside of gemspec
+- Fix the depencies on the gemspec
+- Add the "log_format" configuration support for text and json formats
+
+## 0.2.0
+
+**Changes:**
+
+- Refactor by separating Logger and Filter into their own classes.
+
+- Introduce `RailsTracepointStack.configure`, which allows ignoring traces with a custom pattern and customizing the logs output. Example:
+
+```ruby
+RailsTracepointStack.configure do |config|
+  config.ignore_patterns << /services\/foo.rb/
+  config.logger = YourLogger
+end
+```
+
+The default log destination is a file located on `log/rails_tracepoint_stack.log`
+
+- Add The possibility of enable the tracer locally, by calling:
+
+```ruby
+class Foo
+  def bar
+    RailsTracepointStack.enable_trace do
+      p "your code"
+    end
+  end
+end
+```
+
+- Add Rspec and Rake development dependencies, and add partial test coverage.
+
+## 0.1.4
+
+**Changes:**
+
+- Ignore logs containing `gems/bundler`.
+- Require ruby >= 3.0.
+
+**Breaking Changes:**
+
+- To enable logs catch, it is necessary to set `RAILS_TRACEPOINT_STACK` as `true`.

data/lib/generators/rails_tracepoint_stack/install/install_generator.rb

@@ -0,0 +1,29 @@
+require "rails/generators/base"
+require "rails_tracepoint_stack/skill_installer"
+
+module RailsTracepointStack
+  module Generators
+    # Thin wrapper over SkillInstaller so the behaviour stays testable without
+    # booting Rails.
+    class InstallGenerator < Rails::Generators::Base
+      desc "Installs the debug-with-tracepoint skill so agents working in " \
+           "this app know how to trace it"
+
+      def install_agent_skill
+        installer = RailsTracepointStack::SkillInstaller.new(
+          destination: destination_root,
+          force: options[:force]
+        )
+        written = installer.install
+
+        if written
+          say_status :create, relative_to_original_destination_root(written), :green
+        else
+          say_status :skip,
+            "#{relative_to_original_destination_root(installer.target_path)} already exists (--force to replace)",
+            :yellow
+        end
+      end
+    end
+  end
+end

data/lib/rails_tracepoint_stack/depth_tracker.rb

@@ -0,0 +1,54 @@
+module RailsTracepointStack
+  # Turns raw stack positions into app-level nesting.
+  #
+  # Counting :call and :return events would not work here, for two reasons.
+  # Most frames between two traced methods belong to gems and get filtered out,
+  # so an event counter would indent app code by the depth of the framework
+  # underneath it. And the tracer does not always watch :return at all - the
+  # global tracer only subscribes to :call - so nothing would ever pop.
+  #
+  # Reading the real stack position of each kept trace avoids both. A frame
+  # recorded at an equal or deeper position has necessarily finished, whether
+  # it returned, raised, or was abandoned, so it gets dropped on the next
+  # event rather than waiting for one that may never arrive.
+  class DepthTracker
+    def initialize
+      @stack = []
+    end
+
+    def enter(raw_position)
+      drop_finished_frames(raw_position, inclusive: true)
+      depth = @stack.size
+      @stack.push(raw_position)
+      depth
+    end
+
+    def leave(raw_position)
+      drop_finished_frames(raw_position, inclusive: false)
+      return @stack.size unless @stack.last == raw_position
+
+      @stack.pop
+      @stack.size
+    end
+
+    # A raise belongs to the innermost frame being tracked. It may report a
+    # position deeper than that frame when the exception comes out of a C
+    # method the app called, such as an arithmetic coercion, and there is no
+    # tracked frame down there to attribute it to.
+    def raised(raw_position)
+      drop_finished_frames(raw_position, inclusive: false)
+
+      [@stack.size - 1, 0].max
+    end
+
+    private
+
+    def drop_finished_frames(raw_position, inclusive:)
+      if inclusive
+        @stack.pop while @stack.last && @stack.last >= raw_position
+      else
+        @stack.pop while @stack.last && @stack.last > raw_position
+      end
+    end
+  end
+end

data/lib/rails_tracepoint_stack/filter/custom_trace_selector_filter.rb

@@ -4,7 +4,7 @@ module RailsTracepointStack
       def is_a_trace_required_to_watch_by_the_custom_configs?(trace:)
         return false unless RailsTracepointStack.configuration.file_path_to_filter_patterns.any?
 
-        !filter_match_a_custom_pattern_to_be_not_ignored?(trace)
+        filter_match_a_custom_pattern_to_be_not_ignored?(trace)
       end
 
       private

data/lib/rails_tracepoint_stack/filter/gem_path.rb

@@ -1,8 +1,27 @@
 module RailsTracepointStack
   module Filter
     class GemPath
+      # Bundler gives the tightest answer, but it is not always there: a plain
+      # `ruby -e` or an irb session against an installed gem has no bundle, and
+      # asking Bundler then raises instead of filtering. Falling back to every
+      # installed gem is wider than needed and still filters out the code the
+      # developer did not write.
       def self.full_gem_path
-        @full_gem_path ||= Bundler.load.specs.map(&:full_gem_path)
+        @full_gem_path ||= bundled_gem_paths || gem_install_roots
+      end
+
+      def self.bundled_gem_paths
+        return nil unless defined?(::Bundler)
+
+        ::Bundler.load.specs.map(&:full_gem_path)
+      rescue
+        nil
+      end
+
+      # Without a bundle there is no resolved list to ask for, but everything
+      # installed lives under these roots, which is all the filter needs.
+      def self.gem_install_roots
+        Gem.path.map { |dir| File.join(dir, "gems") }
       end
     end
   end

data/lib/rails_tracepoint_stack/filter/rb_config.rb

@@ -2,7 +2,7 @@ module RailsTracepointStack
   module Filter
     class RbConfig
       def self.ruby_lib_path
-        @ruby_lib_path ||= ::RbConfig::CONFIG['rubylibdir']
+        @ruby_lib_path ||= ::RbConfig::CONFIG["rubylibdir"]
       end
     end
   end

data/lib/rails_tracepoint_stack/limits.rb

@@ -0,0 +1,41 @@
+module RailsTracepointStack
+  # How much a capture is allowed to keep. The defaults aim at an output a
+  # person or an agent can read in one go rather than at completeness: a
+  # single Rails request can easily produce tens of thousands of traces.
+  class Limits
+    DEFAULT_MAX_TRACES = 5_000
+    DEFAULT_MAX_STRING_LENGTH = 200
+    DEFAULT_MAX_COLLECTION_SIZE = 20
+
+    attr_reader :max_depth,
+      :max_traces,
+      :max_string_length,
+      :max_collection_size,
+      :capture_params,
+      :capture_return
+
+    def initialize(
+      max_depth: nil,
+      max_traces: DEFAULT_MAX_TRACES,
+      max_string_length: DEFAULT_MAX_STRING_LENGTH,
+      max_collection_size: DEFAULT_MAX_COLLECTION_SIZE,
+      capture_params: true,
+      capture_return: true
+    )
+      @max_depth = max_depth
+      @max_traces = max_traces
+      @max_string_length = max_string_length
+      @max_collection_size = max_collection_size
+      @capture_params = capture_params
+      @capture_return = capture_return
+    end
+
+    def too_deep?(depth)
+      !max_depth.nil? && !depth.nil? && depth > max_depth
+    end
+
+    def room_for?(count)
+      max_traces.nil? || count < max_traces
+    end
+  end
+end

data/lib/rails_tracepoint_stack/log_formatter.rb

@@ -1,3 +1,5 @@
+require "json"
+
 module RailsTracepointStack
   module LogFormatter
     def self.message(trace)
@@ -10,17 +12,187 @@ module RailsTracepointStack
     end
 
     def self.text(trace)
-      "called: #{trace.class_name}##{trace.method_name} in #{trace.file_path}:#{trace.line_number} with params: #{trace.params}"
+      "called: #{trace.class_name}##{trace.method_name} in #{trace.file_path}:#{trace.line_number} with params: #{text_value(trace.params)}"
     end
 
     def self.json(trace)
-      {
-        class: trace.class_name,
-        method_name: trace.method_name,
-        path: trace.file_path,
+      JSON.generate(
+        class: stringify(trace.class_name),
+        method_name: stringify(trace.method_name),
+        path: stringify(trace.file_path),
         line: trace.line_number,
-        params: trace.params
-      }.to_json
+        params: safe_value(trace.params)
+      )
+    end
+
+    def self.stringify(value)
+      return nil if value.nil?
+
+      value.to_s
+    rescue SystemStackError, StandardError => error
+      inspect_fallback(value, error)
+    end
+
+    def self.safe_value(value, ancestry = {})
+      case value
+      when nil, true, false, Numeric
+        value
+      when String
+        # Callers may keep the result around after the traced code moved on,
+        # so a live reference to a mutable string would drift.
+        value.frozen? ? value : value.dup.freeze
+      when Symbol
+        value.to_s
+      when Array
+        safe_array(value, ancestry)
+      when Hash
+        safe_hash(value, ancestry)
+      else
+        safe_json_object(value)
+      end
+    rescue SystemStackError, StandardError => error
+      inspect_fallback(value, error)
+    end
+
+    def self.safe_array(value, ancestry)
+      object_id = value.__id__
+      return recursive_placeholder(value) if ancestry.key?(object_id)
+
+      ancestry[object_id] = true
+      value.map { |item| safe_value(item, ancestry) }
+    ensure
+      ancestry.delete(object_id)
+    end
+
+    def self.safe_hash(value, ancestry)
+      object_id = value.__id__
+      return recursive_placeholder(value) if ancestry.key?(object_id)
+
+      ancestry[object_id] = true
+      value.each_with_object({}) do |(key, item), result|
+        result[safe_hash_key(key, ancestry)] = safe_value(item, ancestry)
+      end
+    ensure
+      ancestry.delete(object_id)
+    end
+
+    def self.safe_hash_key(key, ancestry = {})
+      case key
+      when String
+        key
+      when Symbol, Numeric
+        key.to_s
+      when true, false, nil
+        key.inspect
+      else
+        stringify_hash_key(key, ancestry)
+      end
+    end
+
+    def self.safe_json_object(value)
+      json_value = JSON.parse(JSON.generate(value))
+
+      return json_value unless default_string_representation?(value, json_value)
+
+      safe_object_string(value)
+    rescue SystemStackError, StandardError
+      safe_object_string(value)
+    end
+
+    def self.default_string_representation?(value, json_value)
+      json_value.is_a?(String) &&
+        value.method(:to_s).owner == Kernel &&
+        json_value == value.to_s
+    rescue SystemStackError, StandardError
+      false
+    end
+
+    def self.stringify_hash_key(key, ancestry)
+      normalized_key = safe_value(key, ancestry)
+
+      case normalized_key
+      when String
+        normalized_key
+      when nil
+        "null"
+      when true, false, Numeric
+        normalized_key.to_s
+      else
+        JSON.generate(normalized_key)
+      end
+    rescue SystemStackError, StandardError
+      safe_object_string(key)
+    end
+
+    def self.safe_object_string(value)
+      value.inspect
+    rescue SystemStackError, StandardError => error
+      inspect_fallback(value, error)
+    end
+
+    def self.inspect_fallback(value, error)
+      class_name = value.nil? ? "NilClass" : value.class.to_s
+      "#<#{class_name} unserializable: #{error.class}: #{error.message}>"
+    end
+
+    def self.recursive_placeholder(value)
+      "[recursive #{value.class}]"
+    end
+
+    def self.text_value(value, ancestry = {})
+      case value
+      when nil
+        "nil"
+      when true, false, Numeric
+        value.to_s
+      when String
+        value.inspect
+      when Symbol
+        ":#{value}"
+      when Array
+        text_array(value, ancestry)
+      when Hash
+        text_hash(value, ancestry)
+      else
+        safe_object_string(value)
+      end
+    rescue SystemStackError, StandardError => error
+      inspect_fallback(value, error)
+    end
+
+    def self.text_array(value, ancestry)
+      object_id = value.__id__
+      return recursive_placeholder(value) if ancestry.key?(object_id)
+
+      ancestry[object_id] = true
+      "[#{value.map { |item| text_value(item, ancestry) }.join(", ")}]"
+    ensure
+      ancestry.delete(object_id)
+    end
+
+    def self.text_hash(value, ancestry)
+      object_id = value.__id__
+      return recursive_placeholder(value) if ancestry.key?(object_id)
+
+      ancestry[object_id] = true
+      pairs = value.map do |key, item|
+        "#{text_hash_key(key, ancestry)}=>#{text_value(item, ancestry)}"
+      end
+
+      "{#{pairs.join(", ")}}"
+    ensure
+      ancestry.delete(object_id)
+    end
+
+    def self.text_hash_key(key, ancestry = {})
+      case key
+      when Symbol
+        ":#{key}"
+      when String
+        key.inspect
+      else
+        text_value(key, ancestry)
+      end
     end
   end
 end