orange_tap 0.1.0

data/README.md

@@ -0,0 +1,197 @@
+# OrangeTap
+
+OrangeTap hooks Ruby method calls (call/return) with `TracePoint`, assembles
+them into OpenTelemetry-style spans on a background thread inside the same
+process, and writes the result as an OTLP/JSON file. There is no central
+daemon, no shared memory, and no inter-process communication of any kind —
+tracing is entirely contained within the observed Ruby process.
+
+## Design notes
+
+### Why Ruby-level `TracePoint` instead of `rb_add_event_hook`
+
+OrangeTap intentionally uses the Ruby-level `TracePoint` API rather than the
+C-level `rb_add_event_hook`/`trace_func` mechanism. Ruby 4.0's C-level
+`trace_func` has a known bug, so this gem avoids it by design and pays the
+(small, and in practice dominated by the traced call itself) overhead of a
+Ruby-level hook instead. Target methods are narrowed with
+`TracePoint#enable(target: iseq)` so untargeted methods incur no hook
+overhead at all.
+
+### No central daemon — everything lives in one process's Thread + Queue
+
+Earlier iterations of this idea used a shared-memory ring buffer and an XPC
+daemon process. OrangeTap drops all of that: a session is just a
+`Thread::Queue` plus a background `Thread` inside the same process that
+called `OrangeTap.open`. Correlating traces across multiple OS processes is
+explicitly out of scope for this gem.
+
+### Why `open`/`stop` need no `session_id`
+
+Each call to `open` creates a brand new `Queue` and worker `Thread` dedicated
+to that session. Because a Worker only ever drains events pushed by the
+TracePoints that same `Session` enabled, the `Queue` instance itself is what
+scopes an event to a session — there is no `session_id` field anywhere, and
+running multiple sessions concurrently (multiple `OrangeTap.new.open` calls
+at once) works without any extra bookkeeping.
+
+### What isn't tracked
+
+- **Cross-thread/Fiber/Ractor causality.** Spans are grouped strictly by
+  `thread_id`, so if a traced method spawns work on another thread
+  (`Thread.new`, a Fiber, a Ractor), that work's spans are not linked back to
+  the caller as parent/child. This is out of scope for the current version.
+- **Methods defined via `define_method` or a block.** `trace_method` requires
+  a `Method`/`UnboundMethod` backed by a `def`-defined instance sequence.
+  Block-based method bodies (`define_method`, `define_singleton_method`) have
+  an ISeq of type `:block`, which `TracePoint#enable(target:)` cannot target
+  and raises `ArgumentError` — attempting to trace one will fail at `open`
+  time, not at `trace_method` time.
+- **Errors inside the Worker thread.** If the worker thread raises while
+  assembling spans, `Session#stop` re-raises that error via `Thread#value`'s
+  standard behavior. TracePoints are always disabled *before* the worker is
+  waited on, so a worker crash never leaves hooks enabled.
+- **Double-hook overhead across concurrent sessions.** If two sessions are
+  open at once and both target the same method, that method gets two
+  independent `TracePoint`s enabled on it. This is accepted for simplicity in
+  the current version.
+
+## Installation
+
+```bash
+bundle add orange_tap
+```
+
+Or, without Bundler:
+
+```bash
+gem install orange_tap
+```
+
+## Usage
+
+```ruby
+require "orange_tap"
+
+class Worker
+  def process(job)
+    # ...
+  end
+end
+
+OrangeTap.trace_method(Worker.instance_method(:process))
+
+path = OrangeTap.open do
+  Worker.new.process(job)
+end
+# => path to a written OTLP/JSON file
+```
+
+Or using the instance form:
+
+```ruby
+tape = OrangeTap.new
+tape.open
+# ...
+path = tape.stop
+```
+
+Other registration entry points:
+
+```ruby
+OrangeTap.trace_method(SomeClass.method(:some_class_method))   # class/singleton method
+OrangeTap.trace_method(some_object.method(:some_method))       # singleton method on one object
+OrangeTap.trace_all_instance_methods(SomeClass)                # all instance methods at once
+OrangeTap.untrace_method(SomeClass.instance_method(:some_method))
+
+# Register several methods in one call, and/or use "Foo.bar" / "Foo#bar"
+# notation strings instead of resolving Method/UnboundMethod objects yourself:
+OrangeTap.trace_method("SomeClass.some_class_method", "SomeClass#some_method")
+OrangeTap.untrace_method("SomeClass.some_class_method", "SomeClass#some_method")
+```
+
+Output location is configurable:
+
+```ruby
+OrangeTap.config.output_dir = "/path/to/traces"
+```
+
+### Example
+
+[`examples/order_demo.rb`](examples/order_demo.rb) is a runnable,
+self-contained example. It defines a small `Order`/`Pricing`/`Receipt` set of
+classes:
+
+```ruby
+class Order
+  def total
+    @items.sum { |item| Pricing.price_for(item) }
+  end
+
+  def checkout
+    amount = total
+    Receipt.new(amount).print
+    amount
+  end
+end
+
+module Pricing
+  def self.price_for(item) = PRICES.fetch(item, 0)
+end
+
+class Receipt
+  def print = puts "Total: #{@amount} yen"
+end
+```
+
+traces a mix of instance and module methods, and wraps the call in
+`OrangeTap.open`:
+
+```ruby
+OrangeTap.trace_method(
+  "Order#total",
+  "Order#checkout",
+  "Pricing.price_for",
+  "Receipt#print"
+)
+
+path = OrangeTap.open do
+  Order.new(%w[coffee cake tea coffee]).checkout
+end
+```
+
+Run it with:
+
+```bash
+bundle exec ruby -Ilib examples/order_demo.rb
+```
+
+This prints the OTLP/JSON file path and pretty-prints its contents. A sample
+of that output is checked in at
+[`examples/trace-example.json`](examples/trace-example.json).
+
+Importing that JSON into Jaeger (all-in-one) as an OTLP trace renders the
+following span tree, which matches the example's call graph exactly —
+`orange_tap session` → `tid=...` → `Order#checkout` → `Order#total` →
+`Pricing.price_for` ×4, plus `Receipt#print`:
+
+![Example trace visualized in Jaeger](examples/jaeger-screenshot.png)
+
+## Development
+
+After checking out the repo, run `bin/setup` to install dependencies. Then,
+run `rake test` to run the tests. You can also run `bin/console` for an
+interactive prompt that will allow you to experiment.
+
+## Contributing
+
+Bug reports and pull requests are welcome on GitHub at
+https://github.com/udzura/orange_tap. This project is intended to be a safe,
+welcoming space for collaboration, and contributors are expected to adhere to
+the [code of conduct](https://github.com/udzura/orange_tap/blob/main/CODE_OF_CONDUCT.md).
+
+## Code of Conduct
+
+Everyone interacting in the OrangeTap project's codebases, issue trackers,
+chat rooms and mailing lists is expected to follow the
+[code of conduct](https://github.com/udzura/orange_tap/blob/main/CODE_OF_CONDUCT.md).

data/Rakefile

@@ -0,0 +1,12 @@
+# frozen_string_literal: true
+
+require "bundler/gem_tasks"
+require "rake/testtask"
+
+Rake::TestTask.new(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.test_files = FileList["test/**/*_test.rb"]
+end
+
+task default: :test

data/examples/order_demo.rb

@@ -0,0 +1,66 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# Minimal end-to-end example: define some classes, register the methods you
+# want traced, wrap the code you want to observe in OrangeTap.open, and print
+# the resulting OTLP/JSON file.
+#
+#   ruby -Ilib examples/order_demo.rb
+
+require "orange_tap"
+require "json"
+
+class Order
+  def initialize(items)
+    @items = items
+  end
+
+  def total
+    @items.sum { |item| Pricing.price_for(item) }
+  end
+
+  def checkout
+    amount = total
+    Receipt.new(amount).print
+    amount
+  end
+end
+
+module Pricing
+  PRICES = { "coffee" => 400, "tea" => 350, "cake" => 500 }.freeze
+
+  def self.price_for(item)
+    PRICES.fetch(item, 0)
+  end
+end
+
+class Receipt
+  def initialize(amount)
+    @amount = amount
+  end
+
+  def print
+    puts "Total: #{@amount} yen"
+  end
+end
+
+# Register the methods we care about in one call. "Foo#bar" resolves to an
+# instance method, "Foo.bar" to a module/class (singleton) method.
+OrangeTap.trace_method(
+  "Order#total",
+  "Order#checkout",
+  "Pricing.price_for",
+  "Receipt#print"
+)
+
+path = nil
+# Avoid YJIT compile overhead in this example by running the code multiple times
+3.times do
+  path = OrangeTap.open do
+    order = Order.new(%w[coffee cake tea coffee])
+    order.checkout
+  end
+end
+
+puts "\nOTLP/JSON written to: #{path}\n\n"
+puts JSON.pretty_generate(JSON.parse(File.read(path)))

data/examples/trace-example.json

@@ -0,0 +1 @@
+{"resourceSpans":[{"resource":{"attributes":[{"key":"service.name","value":{"stringValue":"orange_tap"}}]},"scopeSpans":[{"scope":{"name":"orange_tap","version":"0.1.0"},"spans":[{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"d18866e452c53566","name":"orange_tap session","kind":1,"startTimeUnixNano":"1783516109569910000","endTimeUnixNano":"1783516109569995000","attributes":[]},{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"53fc59a2efc177df","name":"tid=704","kind":1,"startTimeUnixNano":"1783516109569910000","endTimeUnixNano":"1783516109569995000","attributes":[{"key":"thread.id","value":{"intValue":"704"}}],"parentSpanId":"d18866e452c53566"},{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"0917347853e48ab3","name":"Pricing.price_for","kind":1,"startTimeUnixNano":"1783516109569944000","endTimeUnixNano":"1783516109569945000","attributes":[{"key":"thread.id","value":{"intValue":"704"}}],"parentSpanId":"21e20b4ea2a8f28a"},{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"227bec23cab15004","name":"Pricing.price_for","kind":1,"startTimeUnixNano":"1783516109569946000","endTimeUnixNano":"1783516109569946000","attributes":[{"key":"thread.id","value":{"intValue":"704"}}],"parentSpanId":"21e20b4ea2a8f28a"},{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"43e1d5bb5d47a207","name":"Pricing.price_for","kind":1,"startTimeUnixNano":"1783516109569947000","endTimeUnixNano":"1783516109569948000","attributes":[{"key":"thread.id","value":{"intValue":"704"}}],"parentSpanId":"21e20b4ea2a8f28a"},{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"defa3633ea1bce01","name":"Pricing.price_for","kind":1,"startTimeUnixNano":"1783516109569948000","endTimeUnixNano":"1783516109569949000","attributes":[{"key":"thread.id","value":{"intValue":"704"}}],"parentSpanId":"21e20b4ea2a8f28a"},{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"21e20b4ea2a8f28a","name":"Order#total","kind":1,"startTimeUnixNano":"1783516109569944000","endTimeUnixNano":"1783516109569950000","attributes":[{"key":"thread.id","value":{"intValue":"704"}}],"parentSpanId":"007fe473ad175bc2"},{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"edfcf55ab44605d8","name":"Receipt#print","kind":1,"startTimeUnixNano":"1783516109569950000","endTimeUnixNano":"1783516109569952000","attributes":[{"key":"thread.id","value":{"intValue":"704"}}],"parentSpanId":"007fe473ad175bc2"},{"traceId":"85993b8744f30aaabce32f8f68762816","spanId":"007fe473ad175bc2","name":"Order#checkout","kind":1,"startTimeUnixNano":"1783516109569941000","endTimeUnixNano":"1783516109569953000","attributes":[{"key":"thread.id","value":{"intValue":"704"}}],"parentSpanId":"53fc59a2efc177df"}]}]}]}

data/lib/orange_tap/config.rb

@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+require "tmpdir"
+
+module OrangeTap
+  class Config
+    attr_accessor :output_dir, :service_name, :otel_converter
+
+    def initialize
+      @output_dir = File.join(Dir.tmpdir, "orange_tap")
+      @service_name = "orange_tap"
+      @otel_converter = OrangeTap::OtelConverter
+    end
+  end
+end

data/lib/orange_tap/event.rb

@@ -0,0 +1,8 @@
+# frozen_string_literal: true
+
+module OrangeTap
+  # Lightweight, immutable event created inside the TracePoint hook. Only
+  # numeric/symbol/Module references are captured here (no tp.binding,
+  # no tp.parameters, no string building) so the hook body stays cheap.
+  Event = Data.define(:type, :thread_id, :method_id, :defined_class, :timestamp_ns)
+end

data/lib/orange_tap/method_registry.rb

@@ -0,0 +1,85 @@
+# frozen_string_literal: true
+
+module OrangeTap
+  # Holds the set of methods to be traced, keyed by [owner, name] rather than
+  # by Method/UnboundMethod object identity. Method/UnboundMethod instances
+  # are freshly allocated on every `obj.method(:foo)` call, so identity-based
+  # bookkeeping would make untrace_method impossible to use correctly. Using
+  # owner (the singleton class for class/singleton methods) + name gives a
+  # stable identity for instance methods, class methods, and singleton
+  # methods on a specific object alike.
+  class MethodRegistry
+    # "Foo::Bar.baz" -> class/singleton method; "Foo::Bar#baz" -> instance method.
+    # The class-path side never contains "." or "#", so a greedy match up to
+    # the last separator is unambiguous.
+    CLASS_METHOD_NOTATION = /\A(.+)\.([^.#]+)\z/
+    INSTANCE_METHOD_NOTATION = /\A(.+)#([^.#]+)\z/
+
+    def initialize
+      @entries = {}
+      @mutex = Mutex.new
+    end
+
+    def register(*method_objs)
+      method_objs.each { |m| register_one(m) }
+      nil
+    end
+
+    def unregister(*method_objs)
+      method_objs.each { |m| unregister_one(m) }
+      nil
+    end
+
+    def register_all_instance_methods(klass)
+      klass.instance_methods(false).each { |m| register(klass.instance_method(m)) }
+      nil
+    end
+
+    # Snapshot of currently registered ISeqs, taken once per Session#open.
+    def targets
+      @mutex.synchronize { @entries.values.dup }
+    end
+
+    private
+
+    def register_one(method_obj)
+      method_obj = resolve(method_obj)
+      iseq = RubyVM::InstructionSequence.of(method_obj)
+      raise OrangeTap::UntraceableMethodError, method_obj.inspect unless iseq
+
+      @mutex.synchronize { @entries[key_for(method_obj)] = iseq }
+    end
+
+    def unregister_one(method_obj)
+      method_obj = resolve(method_obj)
+      @mutex.synchronize { @entries.delete(key_for(method_obj)) }
+    end
+
+    # Accepts a Method/UnboundMethod as-is, or a notation String ("Foo.bar"
+    # for a class/singleton method, "Foo#bar" for an instance method) that is
+    # resolved to one via Object.const_get + #method/#instance_method.
+    def resolve(method_obj)
+      return method_obj if method_obj.is_a?(Method) || method_obj.is_a?(UnboundMethod)
+
+      unless method_obj.is_a?(String)
+        raise ArgumentError,
+              "Method / UnboundMethod、または 'Foo.bar' / 'Foo#bar' 形式の文字列を渡してください: " \
+              "#{method_obj.inspect}"
+      end
+
+      if (m = CLASS_METHOD_NOTATION.match(method_obj))
+        Object.const_get(m[1]).method(m[2].to_sym)
+      elsif (m = INSTANCE_METHOD_NOTATION.match(method_obj))
+        Object.const_get(m[1]).instance_method(m[2].to_sym)
+      else
+        raise ArgumentError,
+              "'Foo.bar'（クラス/特異メソッド）または 'Foo#bar'（インスタンスメソッド）形式で指定してください: " \
+              "#{method_obj.inspect}"
+      end
+    end
+
+    def key_for(method_obj)
+      [method_obj.owner, method_obj.name]
+    end
+  end
+end

data/lib/orange_tap/otel_converter.rb

@@ -0,0 +1,67 @@
+# frozen_string_literal: true
+
+module OrangeTap
+  # Self-contained OTLP/JSON writer. Follows the wire format conventions of
+  # OTLP/JSON (resourceSpans -> scopeSpans -> spans, 32-hex traceId, 16-hex
+  # spanId, string-encoded nanosecond timestamps) but does not depend on any
+  # external gem: PendingSpan already carries hex-encoded span_id/parent_span_id
+  # (see Worker), so no int-to-hex conversion is needed here.
+  #
+  # Pluggable via Config#otel_converter so callers can swap in their own
+  # converter (e.g. to stream spans elsewhere) without touching Worker.
+  module OtelConverter
+    SPAN_KIND_INTERNAL = 1
+
+    module_function
+
+    # spans: Array<PendingSpan> holding monotonic-ns timestamps.
+    # Returns a Hash ready for JSON.generate.
+    def build_document(spans:, trace_id:, start_mono_ns:, start_unix_ns:, service_name:)
+      to_unix = ->(mono_ns) { (start_unix_ns.to_i + (mono_ns.to_i - start_mono_ns.to_i)).to_s }
+
+      {
+        resourceSpans: [
+          {
+            resource: { attributes: [str_attr("service.name", service_name)] },
+            scopeSpans: [
+              {
+                scope: { name: service_name, version: OrangeTap::VERSION },
+                spans: spans.map { |s| span_hash(s, trace_id, to_unix) }
+              }
+            ]
+          }
+        ]
+      }
+    end
+
+    def span_hash(span, trace_id, to_unix)
+      attributes = []
+      attributes << int_attr("thread.id", span.thread_id) if span.thread_id
+      attributes << bool_attr("orange_tap.incomplete", true) if span.incomplete
+
+      hash = {
+        traceId: trace_id,
+        spanId: span.span_id,
+        name: span.name,
+        kind: SPAN_KIND_INTERNAL,
+        startTimeUnixNano: to_unix.call(span.start_mono_ns),
+        endTimeUnixNano: to_unix.call(span.end_mono_ns),
+        attributes: attributes
+      }
+      hash[:parentSpanId] = span.parent_span_id if span.parent_span_id
+      hash
+    end
+
+    def str_attr(key, value)
+      { key: key, value: { stringValue: value.to_s } }
+    end
+
+    def int_attr(key, value)
+      { key: key, value: { intValue: value.to_i.to_s } }
+    end
+
+    def bool_attr(key, value)
+      { key: key, value: { boolValue: !!value } }
+    end
+  end
+end

data/lib/orange_tap/pending_span.rb

@@ -0,0 +1,21 @@
+# frozen_string_literal: true
+
+module OrangeTap
+  # Mutable, in-progress span assembled by Worker while draining the queue.
+  # Timestamps are kept in monotonic ns; OtelConverter is responsible for
+  # anchoring them to wall-clock unix ns at output time.
+  class PendingSpan
+    attr_accessor :span_id, :parent_span_id, :name, :thread_id,
+                  :start_mono_ns, :end_mono_ns, :incomplete
+
+    def initialize(span_id:, parent_span_id:, name:, thread_id:, start_mono_ns:, end_mono_ns: nil)
+      @span_id = span_id
+      @parent_span_id = parent_span_id
+      @name = name
+      @thread_id = thread_id
+      @start_mono_ns = start_mono_ns
+      @end_mono_ns = end_mono_ns
+      @incomplete = false
+    end
+  end
+end

data/lib/orange_tap/session.rb

@@ -0,0 +1,79 @@
+# frozen_string_literal: true
+
+require "securerandom"
+
+module OrangeTap
+  # Controls a single recording session's lifecycle. Each #open call creates
+  # a dedicated Queue and Worker Thread; there is no session_id, because the
+  # Queue instance itself is what scopes events to this session. This also
+  # makes concurrent sessions (multiple Session instances open at once) work
+  # without any cross-session bookkeeping: each has its own Queue, Thread,
+  # and TracePoint set.
+  class Session
+    def initialize(registry: OrangeTap.default_registry, config: OrangeTap.config)
+      @registry = registry
+      @config = config
+      @queue = nil
+      @worker_thread = nil
+      @tracepoint_targets = nil
+    end
+
+    def open
+      raise AlreadyOpenError if @queue
+
+      # Anchor monotonic time to wall-clock time once, at session start, so
+      # OtelConverter can later translate CLOCK_MONOTONIC-based timestamps
+      # (cheap, used inside the hot hook) into absolute OTLP unix ns.
+      start_mono_ns = Process.clock_gettime(Process::CLOCK_MONOTONIC, :nanosecond)
+      start_unix_ns = Process.clock_gettime(Process::CLOCK_REALTIME, :nanosecond)
+      trace_id = SecureRandom.hex(16)
+
+      @queue = Thread::Queue.new
+
+      # One TracePoint per target ISeq: whether a single TracePoint can
+      # safely enable(target:) more than one ISeq is version-dependent, so
+      # each ISeq gets its own TracePoint instance to enable/disable.
+      @tracepoint_targets = @registry.targets.map { |iseq| [build_tracepoint(@queue), iseq] }
+
+      ctx = Worker::Context.new(
+        queue: @queue, config: @config, trace_id: trace_id,
+        start_mono_ns: start_mono_ns, start_unix_ns: start_unix_ns
+      )
+      @worker_thread = Thread.new(ctx) { |worker_ctx| Worker.new(worker_ctx).run }
+
+      @tracepoint_targets.each { |tp, iseq| tp.enable(target: iseq) }
+      self
+    end
+
+    def stop
+      raise NotOpenError unless @queue
+
+      # Disable hooks before closing the queue / waiting on the worker, so
+      # that even if the worker raises, tracing has already stopped and
+      # cannot leak into whatever runs next.
+      @tracepoint_targets.each { |tp, _iseq| tp.disable }
+      @queue.close
+      path = @worker_thread.value
+      @queue = nil
+      @tracepoint_targets = nil
+      @worker_thread = nil
+      path
+    end
+
+    private
+
+    def build_tracepoint(queue)
+      TracePoint.new(:call, :return) do |tp|
+        # Hook body stays minimal: push a single Event built from cheap
+        # primitives only. No tp.binding, no tp.parameters, no string work.
+        queue << Event.new(
+          type: tp.event,
+          thread_id: Thread.current.object_id,
+          method_id: tp.method_id,
+          defined_class: tp.defined_class,
+          timestamp_ns: Process.clock_gettime(Process::CLOCK_MONOTONIC, :nanosecond)
+        )
+      end
+    end
+  end
+end

data/lib/orange_tap/version.rb

@@ -0,0 +1,5 @@
+# frozen_string_literal: true
+
+module OrangeTap
+  VERSION = "0.1.0"
+end