pf2 0.1.0 → 0.3.0

data/README.md

@@ -1,26 +1,123 @@
-# Pf2
+Pf2
+===========
 
-A sampling-based profiler for Ruby.
-Works only on a patched version of CRuby (MRI) at the moment.
+A experimental sampling-based profiler for Ruby 3.3+.
 
-## Installation
+Notable Capabilites
+--------
 
-TBD
+- Can accurately track multiple Ruby Threads' activity
+- Sampling interval can be set based on per-Thread CPU usage
 
-## Usage
+Usage
+--------
 
-TBD
+### Profiling
 
-## Development
+Pf2 will collect samples every 10 ms of wall time by default.
 
-After checking out the repo, run `bin/setup` to install dependencies. You can also run `bin/console` for an interactive prompt that will allow you to experiment.
+```ruby
+# Threads in `threads` will be tracked
+Pf2.start(threads: [Thread.current])
 
-To install this gem onto your local machine, run `bundle exec rake install`. To release a new version, update the version number in `version.rb`, and then run `bundle exec rake release`, which will create a git tag for the version, push git commits and the created tag, and push the `.gem` file to [rubygems.org](https://rubygems.org).
+your_code_here
 
-## Contributing
+# Stop profiling and save the profile for visualization
+profile = Pf2.stop
+File.write("my_program.pf2profile", profile)
+```
 
-Bug reports and pull requests are welcome on GitHub at https://github.com/osyoyu/pf2.
+Alternatively, you may provide a code block to profile.
 
-## License
+```ruby
+profile = Pf2.profile do
+  your_code_here() # will be profiled
+  Thread.new { threaded_code() } # will also be profiled
+end
+
+# Save the profile for visualization
+File.write("my_program.pf2profile", profile)
+```
+
+### Reporting / Visualization
+
+Profiles can be visualized using the [Firefox Profiler](https://profiler.firefox.com/).
+
+```console
+$ pf2 -o report.json my_program.pf2profile
+```
+
+### Configuration
+
+Pf2 accepts the following configuration keys:
+
+```rb
+Pf2.start(
+  interval_ms: 49,        # Integer: The sampling interval in milliseconds (default: 49)
+  threads: [],            # Array<Thread>: A list of Ruby Threads to be tracked (default: `Thread.list`)
+  time_mode: :cpu,        # `:cpu` or `:wall`: The sampling timer's mode
+                          # (default: `:cpu` for SignalScheduler, `:wall` for TimerThreadScheduler)
+  track_new_threads: true # Boolean: Whether to automatically track Threads spawned after profiler start
+                          # (default: false)
+)
+```
+
+
+Overhead
+--------
+
+While Pf2 aims to be non-disturbulent as much as possible, a small overhead still is incured.
+
+(TBD)
+
+Limitations
+--------
+
+Pf2 cannot properly track program activity in some known cases. I'm working to remove these limtations, so stay tuned.
+
+- Program execution in forked processes
+  - Workarounds available for Puma
+- Program execution in Fibers
+- Program execution when MaNy (`RUBY_MN_THREADS`) is enabled
+
+Internals
+--------
+
+### Sampling
+
+Pf2 is a _sampling profiler_. This means that Pf2 collects _samples_ of program execution periodically, instead of tracing every action (e.g. method invocations and returns).
+
+Pf2 uses the `rb_profile_thread_frames()` API for sampling. When to do so is controlled by _Schedulers_, described in the following section.
+
+### Schedulers
+
+Schedulers determine when to execute sample collection, based on configuration (time mode and interval). Pf2 has two schedulers available.
+
+#### SignalScheduler (Linux-only)
+
+The first is the `SignalScheduler`, based on POSIX timers. Pf2 will use this scheduler when possible. SignalScheduler creates a POSIX timer for each Ruby Thread (the underlying pthread to be more accurate) using `timer_create(3)`. This leaves the actual time-keeping to the OS, which is capable of tracking accurate per-thread CPU time usage.
+
+When the specified interval has arrived (the timer has _expired_), the OS delivers us a SIGALRM (note: Unlike `setitimer(2)`, `timer_create(3)` allows us to choose which signal to be delivered, and Pf2 uses SIGALRM regardless of time mode). This is why the scheduler is named SignalScheduler.
+
+Signals are directed to Ruby Threads' underlying pthread, effectively "pausing" the Thread's activity. This routing is done using `SIGEV_THREAD_ID`, which is a Linux-only feature. Sample collection is done in the signal handler, which is expected to be more _accurate_, capturing the paused Thread's activity.
+
+This scheduler heavily relies on Ruby's 1:N Thread model (1 Ruby Threads is strongly tied to a native pthread). It will not work properly in MaNy (`RUBY_MN_THREADS=1`).
+
+#### TimerThreadScheduler
+
+Another scheduler is the `TimerThreadScheduler`, which maintains a time-keeping thread by itself. A new native thread (pthread on Linux/macOS) will be created, and an infinite loop will be run inside. After `sleep(2)`-ing for the specified interval time, sampling will be queued using Ruby's Postponed Job API.
+
+This scheduler is wall-time only, and does not support CPU-time based profiling.
+
+Future Plans
+--------
+
+- Remove known limitations, if possible
+- Implement a "tracing" scheduler, using the C TracePoint API
+- more
+
+
+License
+--------
 
 The gem is available as open source under the terms of the [MIT License](https://opensource.org/licenses/MIT).

data/Rakefile

@@ -1,8 +1,16 @@
 require 'bundler/gem_tasks'
 require 'rake/extensiontask'
+require 'minitest/test_task'
 
 task default: %i[]
 
 Rake::ExtensionTask.new 'pf2' do |ext|
   ext.lib_dir = 'lib/pf2'
 end
+
+Minitest::TestTask.create(:test) do |t|
+  t.libs << "test"
+  t.libs << "lib"
+  t.warning = false
+  t.test_globs = ["test/**/*_test.rb"]
+end

data/crates/backtrace-sys2/.gitignore

@@ -0,0 +1 @@
+/target

data/crates/backtrace-sys2/Cargo.toml

@@ -0,0 +1,9 @@
+[package]
+name = "backtrace-sys2"
+version = "0.1.0"
+edition = "2021"
+
+# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html
+
+[build-dependencies]
+bindgen = "0.69.2"

data/crates/backtrace-sys2/build.rs

@@ -0,0 +1,48 @@
+use std::env;
+use std::path::Path;
+use std::path::PathBuf;
+use std::process::Command;
+
+fn main() {
+    let libbacktrace_src_dir = Path::new("src/libbacktrace").canonicalize().unwrap();
+
+    // Run ./configure
+    let configure_status = Command::new("./configure")
+        .current_dir(&libbacktrace_src_dir)
+        .status()
+        .expect("libbacktrace: ./configure failed");
+    if !configure_status.success() {
+        panic!("libbacktrace: ./configure failed");
+    }
+
+    // Run make
+    let make_status = Command::new("make")
+        .current_dir(&libbacktrace_src_dir)
+        .status()
+        .expect("libbacktrace: make failed");
+    if !make_status.success() {
+        panic!("libbacktrace: make failed");
+    }
+
+    // Generate bindings
+    let bindings = bindgen::Builder::default()
+        .header(format!("{}/backtrace.h", libbacktrace_src_dir.display()))
+        .allowlist_function("backtrace_.*")
+        .generate_comments(true)
+        .parse_callbacks(Box::new(bindgen::CargoCallbacks::new()))
+        .generate()
+        .expect("Failed to generate bindings");
+
+    // Output bindings to the src directory
+    let out_path = PathBuf::from(env::var("OUT_DIR").unwrap());
+    bindings
+        .write_to_file(out_path.join("backtrace_bindings.rs"))
+        .expect("Failed to write bindings");
+
+    println!("cargo:rerun-if-changed=build.rs");
+    println!(
+        "cargo:rustc-link-search=native={}",
+        libbacktrace_src_dir.join(".libs").display()
+    );
+    println!("cargo:rustc-link-lib=static=backtrace");
+}

data/crates/backtrace-sys2/src/lib.rs

@@ -0,0 +1,5 @@
+#![allow(non_upper_case_globals)]
+#![allow(non_camel_case_types)]
+#![allow(non_snake_case)]
+
+include!(concat!(env!("OUT_DIR"), "/backtrace_bindings.rs"));

data/crates/backtrace-sys2/src/libbacktrace/.gitignore

@@ -0,0 +1,15 @@
+*~
+*.o
+*.lo
+*.a
+*.la
+
+.libs
+Makefile
+backtrace-supported.h
+config.h
+config.log
+config.status
+install-debuginfo-for-buildid.sh
+libtool
+stamp-h1