rails_tracepoint_stack 0.3.5 → 0.5.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f67c982828a7ad060513cbb50daf9f88399229a534a43cf12a674571b6ae11ae
-  data.tar.gz: fd80c178bf9de58408e769742d7640b0ccdbcc99a4722ec9ccbd6e68d115ff51
+  metadata.gz: 7853cf9027a5db8f962c4cf6753ae98cac10e3cdd0344b618fac59a112837e6d
+  data.tar.gz: 29beeae6b2e5eacbca3121a90bef1670e759477b5cfc48aa837dba6ebcb48fe0
 SHA512:
-  metadata.gz: 57b7125a7a242bf59f15096fffeef7de3c8a7543162fa7851566a07256d2ed790d39ff9c9dfcd1f3c3f8ca54714b797ed5f5015f2eb0d6f0adbe57e3ad582412
-  data.tar.gz: cd33cf3e53fe35fe62ba3867cfe8ff56047cceae59e2139c55898fcb0ed0e44d8dccf57a0b66954f1543797c1cb1070b6c68f479f76ce80f0c15282518c50110
+  metadata.gz: 49af2952c9312ea180cbf2afde354b1d9c68a0a4f6d94a991534e8f19b6fed4f477afda01cdb4d93e766751a9195f183f12f93c0fdd51705b3c984127271cb7c
+  data.tar.gz: 79ff4005947c9a73f08dc03261894e6044653c3e8e1979043546e56425d0fb718587562707957c868c55930ad02606fca3abe5edd29d19b9d1b57d0b4b69938c

data/README.md

@@ -0,0 +1,204 @@
+# rails_tracepoint_stack
+
+**[carlosdanielpohlod.github.io/rails_tracepoint_stack](https://carlosdanielpohlod.github.io/rails_tracepoint_stack/)**
+
+See what your Rails code actually did at runtime: which of *your* methods ran,
+what arguments they got, what each one returned, and where an exception was
+first raised — as a call tree, with gems, the framework and stdlib filtered out.
+
+```ruby
+session = RailsTracepointStack.capture(max_depth: 4) do
+  Order.find(42).recalculate!
+end
+
+puts session.to_tree
+```
+
+```
+Order#recalculate! (app/models/order.rb:88) {}
+  Order#apply_discount (app/models/order.rb:102) {"total":200.0}
+    Discount#rate_for (app/models/discount.rb:12) {"order":"#<Order id: 42>"}
+      -> null
+    -> 200.0
+  -> 200.0
+3 calls, 3 returns, 0 raises, 2 classes
+```
+
+`rate_for` returned nil. No `puts`, no log file, no restart.
+
+A whole Rails request comes out about this long, because everything the
+framework runs underneath is filtered away:
+
+```
+RealEstateAgenciesController#index (app/controllers/real_estate_agencies_controller.rb:14) {}
+  IpLocation.guess_state_uf_and_city_name (app/services/ip_location.rb:15) {"ip":"127.0.0.1"}
+    -> [null,null]
+  -> null
+render app/views/real_estate_agencies/index.html.erb {}
+  -> "<!-- BEGIN app/views/real_estate_agencies/index.html.erb\n-->… (2507 chars)"
+4 calls, 4 returns, 0 raises, 3 classes
+```
+
+That is 4 kept traces out of 13869 the tracer saw.
+
+## Install
+
+```ruby
+# Gemfile
+gem "rails_tracepoint_stack"
+```
+
+## Debugging with an AI agent
+
+Agents reach for `puts` and log lines because they do not know this exists.
+Install the packaged skill into your app and yours will use the gem instead:
+
+```bash
+bin/rails generate rails_tracepoint_stack:install
+```
+
+That writes `.claude/skills/debug-with-tracepoint/SKILL.md`, which tells the
+agent when tracing beats reading code, how to keep the output inside its
+context window, and how to read the tree. Pass `--force` to overwrite an
+existing copy.
+
+## Capturing
+
+`capture` traces the block, returns a session and re-raises anything the block
+raised. It watches only the calling thread, so it stays clean under Puma.
+
+```ruby
+session = RailsTracepointStack.capture { Order.find(42).recalculate! }
+
+session.to_tree    # the indented call tree above
+session.as_json    # the same data, structured, one entry per trace
+session.summary    # {calls:, returns:, raises:, classes:, truncated:}
+session.result     # what the block returned
+session.error      # the exception that escaped, if any
+session.traces     # the raw TraceRecord list
+```
+
+To keep the traces when the block blows up, take the session from the block
+argument:
+
+```ruby
+session = nil
+begin
+  RailsTracepointStack.capture { |s| session = s; thing_that_blows_up }
+rescue => error
+  puts session.to_tree
+end
+```
+
+### Keeping the output small
+
+A real request produces tens of thousands of traces, so captures are bounded.
+
+| Option | Default | What it does |
+|---|---|---|
+| `max_depth` | none | Drops traces nested deeper than this |
+| `max_traces` | 5000 | Stops collecting; `session.truncated?` becomes true |
+| `max_string_length` | 200 | Shortens long strings |
+| `max_collection_size` | 20 | Shortens long arrays and hashes |
+| `max_value_length` | 1000 | Replaces any single value larger than this with its size |
+| `capture_params` | `true` | Set false to show only the flow |
+| `capture_return` | `true` | Set false to show only the calls |
+| `threads` | `:current` | `:all` also records background threads |
+| `blocks` | `false` | `true` also traces blocks — see below |
+
+```ruby
+RailsTracepointStack.capture(max_depth: 3, capture_return: false) { ... }
+```
+
+### Blocks and scopes
+
+A Rails scope is a lambda, so the logic inside it is not a method call and does
+not appear by default. `blocks: true` traces block entry and exit as well:
+
+```ruby
+RailsTracepointStack.capture(blocks: true) { News.latest(User.current) }
+```
+
+```
+News.latest (app/models/news.rb:88) {"user":"Anonymous"}
+  block { } (app/models/news.rb:46) {"user":"Anonymous"}
+    Project.allowed_to_condition (app/models/project.rb:189) {"permission":"view_news"}
+      ↻ 40× AccessControl.permission { } (lib/redmine/access_control.rb:37) {"p":"#<Permission …>"}
+        -> false
+```
+
+It is off by default because a block written inside a loop runs once per
+element: in Redmine, one call produced 48 block events and 43 of them came from
+a single `detect` predicate. Runs of the same block at the same depth collapse
+into one line with a `↻ N×` count, which keeps only the first run's arguments —
+if you need every iteration, leave blocks off and read the loop instead.
+
+Blocks are named after the method they were written in. A block with no
+enclosing method — a scope, a lambda held in a constant — has neither a class
+nor a method name to show, so it renders as `block { }` plus its location.
+```
+
+## Tracing the whole process
+
+For code you cannot wrap in a block — boot, a rake task, a request handled by a
+running server — enable the tracer globally instead. Output goes to
+`log/rails_tracepoint_stack.log`.
+
+```bash
+RAILS_TRACEPOINT_STACK_ENABLED=true bin/rails server
+```
+
+```
+called: Bar#perform in /path/to/app/services/bar.rb:12 with params: {}
+```
+
+`RailsTracepointStack.enable_trace { ... }` does the same for one block, also
+writing to the log rather than returning a session.
+
+## Configuration
+
+```ruby
+# config/initializers/rails_tracepoint_stack.rb
+
+RailsTracepointStack.configure do |config|
+  config.file_path_to_filter_patterns << %r{app/services/}
+  config.ignore_patterns << %r{app/models/concerns/}
+  config.log_format = :json
+  config.log_external_sources = false
+  config.logger = Rails.logger
+end
+```
+
+| Configuration | Description |
+|---|---|
+| `file_path_to_filter_patterns` | Trace **only** files whose path matches one of these patterns |
+| `ignore_patterns` | Skip traces whose file path matches one of these patterns |
+| `log_format` | `:text` (default) or `:json`. Applies to the log, not to `capture` |
+| `log_external_sources` | Include gems, bundler and stdlib. Default `false` |
+| `logger` | Your own logger. Defaults to `log/rails_tracepoint_stack.log` |
+
+A method missing from a trace is usually one the filters dropped as external —
+add its path to `file_path_to_filter_patterns` to force it in.
+
+## Worth knowing
+
+- TracePoint makes the traced code noticeably slower — around 8x on a real
+  Rails request. Fine for an investigation, not for something left running in
+  production.
+- Ruby >= 3.0.
+- A `-> null` line directly under a `!!` line is the frame unwinding, not a
+  method that returned nil.
+- An empty tree says `no app code ran` along with how much was filtered. That
+  usually means the block ran entirely inside gems — `Order.where(...).map(&:name)`
+  calls no method you wrote, since `name` is generated by ActiveRecord.
+
+## Links
+
+- [Website](https://carlosdanielpohlod.github.io/rails_tracepoint_stack/) — the
+  same story with real captured output, including a whole Rails request
+- [RubyGems](https://rubygems.org/gems/rails_tracepoint_stack)
+- [Changelog](changelog.md)
+
+## License
+
+MIT

data/changelog.md

@@ -0,0 +1,156 @@
+# Changelog
+
+## 0.5.0
+
+**Features**
+
+- `capture(blocks: true)` traces block entry and exit, which is what makes a
+  Rails scope, a lambda held in a constant, or any yielded block visible.
+  Off by default: a block inside a loop runs once per element. Runs of the
+  same block at the same depth collapse into one record with a `↻ N×` count
+- `max_value_length` (default 1000) caps any single value, whatever its shape
+
+**BugFix**
+
+- `to_tree` raised `Encoding::CompatibilityError` whenever a traced value held
+  bytes that are not text — a file read in binary mode, a digest, a response
+  body. Strings are now converted to UTF-8 with invalid bytes replaced
+- A class method on an ActiveRecord model rendered as its singleton class,
+  which prints the model's entire schema for a single call. The class name now
+  comes from the module's own name
+- Per-string and per-collection limits still let one value through at thousands
+  of characters, since twenty items truncated to two hundred each is four
+  thousand
+- The block handed to `capture` was traced as part of the result and consumed a
+  level of depth, indenting everything under it
+
+## 0.4.0
+
+**Features**
+
+- `RailsTracepointStack.capture` traces a block and returns the traces in
+  memory, with no env var, log file or restart involved
+- Traces now carry return values and raised exceptions, not just calls and params
+- Each trace knows its call depth, so a session renders as a call tree
+- `session.to_tree`, `session.as_json` and `session.summary` render a capture
+- Captures are bounded by `max_depth`, `max_traces`, `max_string_length` and
+  `max_collection_size`, and report `truncated?` when a limit cut them short
+- Captures watch only the calling thread by default; `threads: :all` opts out
+- `rails g rails_tracepoint_stack:install` writes an agent skill into the app,
+  so AI coding agents know when and how to trace it
+
+**BugFix**
+
+- Params reported every local the method body declared, not just the arguments
+  it received, so unassigned locals showed up as arguments passed as nil.
+  `TracePoint#parameters` now decides what is read from the binding
+- `GemPath` referenced `Bundler` without requiring it, raising `NameError` on
+  the first trace outside a bundle
+- A raise coming out of a C method landed one level too deep in the tree
+
+**Changes**
+
+- The tracer writes to a sink; the default one keeps the previous logging
+  behaviour, so global tracing is unchanged
+- Class methods render as `Foo.bar` rather than `#<Class:Foo>#bar`
+- Compiled templates render as `render app/views/…` with their locals, instead
+  of the generated method name and the rendering internals
+- An empty capture reports how many traces the filters dropped, so "nothing
+  ran" is distinguishable from "nothing of yours ran"
+- Serialized string values are frozen copies, so a snapshot no longer follows
+  later mutations of the traced object
+
+## 0.3.5
+
+**BugFix**
+
+- Sanitize traced params before formatting so JSON logging does not overflow on recursive or problematic objects [Issue #37](https://github.com/carlosdanielpohlod/rails_tracepoint_stack/issues/37)
+- Stabilize text log param rendering across Ruby versions
+
+## 0.3.4
+
+**Changes**
+
+- Fix flaky tests by @danielmbrasil [PR #27](https://github.com/carlosdanielpohlod/rails_tracepoint_stack/pull/27)
+
+- Huge refactor of filters class, separating in modules by filter type [PR #18](https://github.com/carlosdanielpohlod/rails_tracepoint_stack/pull/18)
+
+- Some other code refactors
+
+## 0.3.3
+
+**Changes**
+
+- Add autoload of all lib files on test_helper
+- Fix some tests
+
+**BugFix**
+
+- Fixed the Configuration module not loading the default value for the configuration attributes
+
+## 0.3.1
+
+**Changes:**
+
+- Add the ability to include the external sources to the log using `config.log_external_sources = true`
+
+## 0.3.0
+
+**Changes:**
+
+- Refactor classes, formatting a trace using a value object class `RailsTracepointStack::Trace`
+- include configuration `log_format` option, allowing choose an output as `text` or `json`
+- Include configuration `file_path_to_filter_patterns`, allowing filter traces only when the origin file path matches a pattern
+- Improve test coverage
+
+## 0.2.1
+
+**Changes:**
+
+- Update the ENV enable to be more semantic
+- Add the VERSION constant module
+- Sorted the files inside of gemspec
+- Fix the depencies on the gemspec
+- Add the "log_format" configuration support for text and json formats
+
+## 0.2.0
+
+**Changes:**
+
+- Refactor by separating Logger and Filter into their own classes.
+
+- Introduce `RailsTracepointStack.configure`, which allows ignoring traces with a custom pattern and customizing the logs output. Example:
+
+```ruby
+RailsTracepointStack.configure do |config|
+  config.ignore_patterns << /services\/foo.rb/
+  config.logger = YourLogger
+end
+```
+
+The default log destination is a file located on `log/rails_tracepoint_stack.log`
+
+- Add The possibility of enable the tracer locally, by calling:
+
+```ruby
+class Foo
+  def bar
+    RailsTracepointStack.enable_trace do
+      p "your code"
+    end
+  end
+end
+```
+
+- Add Rspec and Rake development dependencies, and add partial test coverage.
+
+## 0.1.4
+
+**Changes:**
+
+- Ignore logs containing `gems/bundler`.
+- Require ruby >= 3.0.
+
+**Breaking Changes:**
+
+- To enable logs catch, it is necessary to set `RAILS_TRACEPOINT_STACK` as `true`.

data/lib/generators/rails_tracepoint_stack/install/install_generator.rb

@@ -0,0 +1,29 @@
+require "rails/generators/base"
+require "rails_tracepoint_stack/skill_installer"
+
+module RailsTracepointStack
+  module Generators
+    # Thin wrapper over SkillInstaller so the behaviour stays testable without
+    # booting Rails.
+    class InstallGenerator < Rails::Generators::Base
+      desc "Installs the debug-with-tracepoint skill so agents working in " \
+           "this app know how to trace it"
+
+      def install_agent_skill
+        installer = RailsTracepointStack::SkillInstaller.new(
+          destination: destination_root,
+          force: options[:force]
+        )
+        written = installer.install
+
+        if written
+          say_status :create, relative_to_original_destination_root(written), :green
+        else
+          say_status :skip,
+            "#{relative_to_original_destination_root(installer.target_path)} already exists (--force to replace)",
+            :yellow
+        end
+      end
+    end
+  end
+end

data/lib/rails_tracepoint_stack/depth_tracker.rb

@@ -0,0 +1,54 @@
+module RailsTracepointStack
+  # Turns raw stack positions into app-level nesting.
+  #
+  # Counting :call and :return events would not work here, for two reasons.
+  # Most frames between two traced methods belong to gems and get filtered out,
+  # so an event counter would indent app code by the depth of the framework
+  # underneath it. And the tracer does not always watch :return at all - the
+  # global tracer only subscribes to :call - so nothing would ever pop.
+  #
+  # Reading the real stack position of each kept trace avoids both. A frame
+  # recorded at an equal or deeper position has necessarily finished, whether
+  # it returned, raised, or was abandoned, so it gets dropped on the next
+  # event rather than waiting for one that may never arrive.
+  class DepthTracker
+    def initialize
+      @stack = []
+    end
+
+    def enter(raw_position)
+      drop_finished_frames(raw_position, inclusive: true)
+      depth = @stack.size
+      @stack.push(raw_position)
+      depth
+    end
+
+    def leave(raw_position)
+      drop_finished_frames(raw_position, inclusive: false)
+      return @stack.size unless @stack.last == raw_position
+
+      @stack.pop
+      @stack.size
+    end
+
+    # A raise belongs to the innermost frame being tracked. It may report a
+    # position deeper than that frame when the exception comes out of a C
+    # method the app called, such as an arithmetic coercion, and there is no
+    # tracked frame down there to attribute it to.
+    def raised(raw_position)
+      drop_finished_frames(raw_position, inclusive: false)
+
+      [@stack.size - 1, 0].max
+    end
+
+    private
+
+    def drop_finished_frames(raw_position, inclusive:)
+      if inclusive
+        @stack.pop while @stack.last && @stack.last >= raw_position
+      else
+        @stack.pop while @stack.last && @stack.last > raw_position
+      end
+    end
+  end
+end

data/lib/rails_tracepoint_stack/filter/gem_path.rb

@@ -1,8 +1,27 @@
 module RailsTracepointStack
   module Filter
     class GemPath
+      # Bundler gives the tightest answer, but it is not always there: a plain
+      # `ruby -e` or an irb session against an installed gem has no bundle, and
+      # asking Bundler then raises instead of filtering. Falling back to every
+      # installed gem is wider than needed and still filters out the code the
+      # developer did not write.
       def self.full_gem_path
-        @full_gem_path ||= Bundler.load.specs.map(&:full_gem_path)
+        @full_gem_path ||= bundled_gem_paths || gem_install_roots
+      end
+
+      def self.bundled_gem_paths
+        return nil unless defined?(::Bundler)
+
+        ::Bundler.load.specs.map(&:full_gem_path)
+      rescue
+        nil
+      end
+
+      # Without a bundle there is no resolved list to ask for, but everything
+      # installed lives under these roots, which is all the filter needs.
+      def self.gem_install_roots
+        Gem.path.map { |dir| File.join(dir, "gems") }
       end
     end
   end

data/lib/rails_tracepoint_stack/limits.rb

@@ -0,0 +1,45 @@
+module RailsTracepointStack
+  # How much a capture is allowed to keep. The defaults aim at an output a
+  # person or an agent can read in one go rather than at completeness: a
+  # single Rails request can easily produce tens of thousands of traces.
+  class Limits
+    DEFAULT_MAX_TRACES = 5_000
+    DEFAULT_MAX_STRING_LENGTH = 200
+    DEFAULT_MAX_COLLECTION_SIZE = 20
+    DEFAULT_MAX_VALUE_LENGTH = 1_000
+
+    attr_reader :max_depth,
+      :max_traces,
+      :max_string_length,
+      :max_collection_size,
+      :max_value_length,
+      :capture_params,
+      :capture_return
+
+    def initialize(
+      max_depth: nil,
+      max_traces: DEFAULT_MAX_TRACES,
+      max_string_length: DEFAULT_MAX_STRING_LENGTH,
+      max_collection_size: DEFAULT_MAX_COLLECTION_SIZE,
+      max_value_length: DEFAULT_MAX_VALUE_LENGTH,
+      capture_params: true,
+      capture_return: true
+    )
+      @max_depth = max_depth
+      @max_traces = max_traces
+      @max_string_length = max_string_length
+      @max_collection_size = max_collection_size
+      @max_value_length = max_value_length
+      @capture_params = capture_params
+      @capture_return = capture_return
+    end
+
+    def too_deep?(depth)
+      !max_depth.nil? && !depth.nil? && depth > max_depth
+    end
+
+    def room_for?(count)
+      max_traces.nil? || count < max_traces
+    end
+  end
+end

data/lib/rails_tracepoint_stack/log_formatter.rb

@@ -25,18 +25,44 @@ module RailsTracepointStack
       )
     end
 
+    REPLACEMENT = "�".freeze
+
     def self.stringify(value)
       return nil if value.nil?
 
-      value.to_s
+      utf8(value.to_s)
     rescue SystemStackError, StandardError => error
       inspect_fallback(value, error)
     end
 
+    # Traced code holds bytes that are not text: a file read in binary mode, a
+    # digest, a response body. Left as they are, those strings poison every
+    # consumer downstream - JSON refuses to encode them, and joining them with
+    # the rest of the output raises Encoding::CompatibilityError. Normalizing
+    # here keeps the damage to the one value that caused it.
+    def self.utf8(value)
+      return value unless value.is_a?(String)
+
+      converted =
+        if value.encoding == Encoding::UTF_8
+          value
+        else
+          value.encode(Encoding::UTF_8, invalid: :replace, undef: :replace, replace: REPLACEMENT)
+        end
+
+      converted.valid_encoding? ? converted : converted.scrub(REPLACEMENT)
+    rescue SystemStackError, StandardError
+      value.scrub(REPLACEMENT)
+    end
+
     def self.safe_value(value, ancestry = {})
       case value
-      when nil, true, false, Numeric, String
+      when nil, true, false, Numeric
         value
+      when String
+        # Callers may keep the result around after the traced code moved on,
+        # so a live reference to a mutable string would drift.
+        utf8(value).dup.freeze
       when Symbol
         value.to_s
       when Array
@@ -121,7 +147,7 @@ module RailsTracepointStack
     end
 
     def self.safe_object_string(value)
-      value.inspect
+      utf8(value.inspect)
     rescue SystemStackError, StandardError => error
       inspect_fallback(value, error)
     end
@@ -142,7 +168,7 @@ module RailsTracepointStack
       when true, false, Numeric
         value.to_s
       when String
-        value.inspect
+        utf8(value).inspect
       when Symbol
         ":#{value}"
       when Array

data/lib/rails_tracepoint_stack/renderer/summary.rb

@@ -0,0 +1,37 @@
+module RailsTracepointStack
+  module Renderer
+    # Counts what a session holds, so a reader can tell at a glance whether
+    # the capture saw what they expected before reading the tree itself.
+    module Summary
+      def self.call(session)
+        {
+          calls: count(session, :call),
+          returns: count(session, :return),
+          raises: count(session, :raise),
+          classes: session.traces.map(&:class_name).uniq.size,
+          filtered: session.filtered_count,
+          truncated: session.truncated?
+        }
+      end
+
+      def self.line(session)
+        counts = call(session)
+
+        [
+          pluralize(counts[:calls], "call"),
+          pluralize(counts[:returns], "return"),
+          pluralize(counts[:raises], "raise"),
+          pluralize(counts[:classes], "class", "classes")
+        ].join(", ")
+      end
+
+      def self.count(session, kind)
+        session.traces.count { |record| record.kind == kind }
+      end
+
+      def self.pluralize(count, singular, plural = "#{singular}s")
+        "#{count} #{(count == 1) ? singular : plural}"
+      end
+    end
+  end
+end