rperf 0.7.0 → 0.9.0

data/docs/help.md

@@ -12,18 +12,22 @@ POSIX systems (Linux, macOS). Requires Ruby >= 3.4.0.
     rperf stat [options] command [args...]
     rperf exec [options] command [args...]
     rperf report [options] [file]
+    rperf diff [options] base target
     rperf help
+    rperf -v / --version
 
 ### record: Profile and save to file.
 
-    -o, --output PATH       Output file (default: rperf.data)
+    -o, --output PATH       Output file (default: rperf.json.gz)
     -f, --frequency HZ      Sampling frequency in Hz (default: 1000)
     -m, --mode MODE         cpu or wall (default: cpu)
-    --format FORMAT         pprof, collapsed, or text (default: auto from extension)
+    --format FORMAT         json, pprof, collapsed, or text (default: auto from extension)
     -p, --print             Print text profile to stdout
                             (same as --format=text --output=/dev/stdout)
     --signal VALUE          Timer signal (Linux only): signal number, or 'false'
                             for nanosleep thread (default: auto)
+    --no-inherit            Do not profile forked/spawned child processes
+    --no-aggregate          Disable C-level sample aggregation (raw per-sample data)
     -v, --verbose           Print sampling statistics to stderr
 
 ### stat: Run command and print performance summary to stderr.
@@ -36,12 +40,20 @@ Uses wall mode by default. No file output by default.
     --report                Include flat/cumulative profile tables in output
     --signal VALUE          Timer signal (Linux only): signal number, or 'false'
                             for nanosleep thread (default: auto)
+    --no-inherit            Do not profile forked/spawned child processes
+    --no-aggregate          Disable C-level sample aggregation (raw per-sample data)
     -v, --verbose           Print additional sampling statistics
 
 Shows: user/sys/real time, time breakdown (CPU execution, GVL blocked,
 GVL wait, GC marking, GC sweeping), GC/memory/OS stats, and profiler overhead.
+Lines are prefixed: `[Rperf]` for sampling-derived data, `[Ruby ]` for
+runtime info, `[OS   ]` for OS-level info.
 Use --report to add flat and cumulative top-50 function tables.
 
+When child processes are profiled (default), the stat output shows
+aggregated data from all processes and includes a "Ruby processes profiled"
+count. Use --no-inherit to disable child process tracking.
+
 ### exec: Run command and print full profile report to stderr.
 
 Like `stat --report`. Uses wall mode by default. No file output by default.
@@ -51,26 +63,81 @@ Like `stat --report`. Uses wall mode by default. No file output by default.
     -m, --mode MODE         cpu or wall (default: wall)
     --signal VALUE          Timer signal (Linux only): signal number, or 'false'
                             for nanosleep thread (default: auto)
+    --no-inherit            Do not profile forked/spawned child processes
+    --no-aggregate          Disable C-level sample aggregation (raw per-sample data)
     -v, --verbose           Print additional sampling statistics
 
 Shows: user/sys/real time, time breakdown, GC/memory/OS stats, profiler overhead,
 and flat/cumulative top-50 function tables.
 
-### report: Open pprof profile with go tool pprof. Requires Go.
+### report: Open profile viewer or go tool pprof.
 
     --top                   Print top functions by flat time
     --text                  Print text report
+    --html                  Output static HTML viewer to stdout
 
 Default (no flag): opens interactive web UI in browser.
-Default file: rperf.data
+Default file: rperf.json.gz
+
+`--html` generates an HTML file with profile data embedded inline.
+No server is needed — open it directly in a browser. d3 and
+d3-flamegraph are loaded from CDN, so an internet connection is
+required on first viewing. Useful for sharing or hosting on static
+sites (e.g., GitHub Pages).
+
+    rperf report --html profile.json.gz > report.html
+
+### diff: Compare two profiles (target - base). Requires Go.
 
-### diff: Compare two pprof profiles (target - base). Requires Go.
+Accepts `.json.gz` (auto-converted to pprof) or `.pb.gz` files.
 
     --top                   Print top functions by diff
     --text                  Print text diff report
 
 Default (no flag): opens diff in browser.
 
+### Multi-process profiling
+
+By default, rperf profiles forked and spawned Ruby child processes.
+Profiles from all processes are merged into a single output. Each child
+process's samples are tagged with a `%pid` label for per-process filtering.
+
+    # Profile a preforking server (Unicorn, Puma, etc.)
+    rperf stat -m wall bundle exec unicorn
+    rperf record -m wall -o profile.json.gz bundle exec unicorn
+
+    # Profile with fork
+    rperf stat ruby -e '4.times { fork { work } }; Process.waitall'
+
+    # Disable child process tracking
+    rperf stat --no-inherit ruby app.rb
+
+How it works:
+
+- On fork: `Process._fork` hook restarts profiling in the child and sets
+  a `%pid` label. When the child exits, its profile is saved to a
+  temporary session directory.
+- On spawn/system: The spawned Ruby process inherits `RUBYLIB` (pointing
+  to rperf's lib directory) and `RUBYOPT=-rrperf`, plus `RPERF_SESSION_DIR`.
+  It auto-starts profiling and writes its profile to the session directory.
+- When the root process exits, it aggregates all profiles from the
+  session directory into a single output (stat report or file).
+- The session directory is cleaned up after aggregation.
+
+Limitations:
+
+- Daemon children (Process.daemon) that outlive the parent will have
+  their profiles lost, since the parent aggregates and cleans up the
+  session directory at exit.
+- Cross-process snapshots (Rperf.snapshot) are not supported; snapshots
+  only cover the current process.
+- Only Ruby child processes are profiled; non-Ruby children (shell
+  scripts, Python, etc.) are not affected.
+- Child processes that use rperf independently (Rperf.start in their
+  own code) will conflict with the inherited auto-start session.
+  Such programs should clear RPERF_ENABLED from their environment
+  before requiring rperf.
+
 ### Examples
 
     rperf record ruby app.rb
@@ -82,6 +149,8 @@ Default (no flag): opens diff in browser.
     rperf stat ruby app.rb
     rperf stat --report ruby app.rb
     rperf stat -o profile.pb.gz ruby app.rb
+    rperf stat -m wall bundle exec unicorn
+    rperf stat --no-inherit ruby app.rb
     rperf exec ruby app.rb
     rperf exec -m cpu ruby app.rb
     rperf report
@@ -112,12 +181,17 @@ Rperf.save("profile.txt", data)
 
 ### Rperf.start parameters
 
-    frequency:  Sampling frequency in Hz (Integer, default: 1000)
+    frequency:  Sampling frequency in Hz (Integer, 1..10000, default: 1000)
     mode:       :cpu or :wall (Symbol, default: :cpu)
     output:     File path to write on stop (String or nil)
     verbose:    Print statistics to stderr (true/false, default: false)
-    format:     :pprof, :collapsed, :text, or nil for auto-detect (Symbol or nil)
+    format:     :json, :pprof, :collapsed, :text, or nil for auto-detect (Symbol or nil)
     defer:      Start with timer paused; use Rperf.profile to activate (default: false)
+    inherit:    Child process tracking: :fork (default), true (fork+spawn), false (none)
+                Note: CLI defaults to true (--no-inherit to disable)
+    signal:     Timer signal (Linux only): nil (default, auto), false (use nanosleep),
+                or a signal number (Integer)
+    aggregate:  Aggregate samples in C (default: true). false returns raw per-sample data
 
 ### Rperf.stop return value
 
@@ -126,7 +200,8 @@ nil if profiler was not running; otherwise a Hash:
 ```ruby
 { mode: :cpu,                      # or :wall
   frequency: 500,
-  sampling_count: 1234,
+  trigger_count: 1300,             # number of timer triggers
+  sampling_count: 1234,            # number of timer callbacks (may differ from trigger_count)
   sampling_time_ns: 56789,
   detected_thread_count: 4,        # threads seen during profiling
   start_time_ns: 17740...,         # CLOCK_REALTIME epoch nanos
@@ -151,7 +226,9 @@ Only works in aggregate mode (the default). Returns nil if not profiling.
 
 When `clear: true` is given, resets aggregated data after taking the snapshot.
 This enables interval-based profiling where each snapshot covers only the
-period since the last clear.
+period since the last clear. Note: the frame table is intentionally retained
+(frame IDs must stay stable for GC safety and thread data consistency), so
+`unique_frames` may accumulate across intervals.
 
 ```ruby
 Rperf.start(frequency: 1000)
@@ -215,13 +292,19 @@ In pprof output, use labels for filtering and grouping:
 
 With `defer: true`, the profiler infrastructure is set up but the sampling
 timer does not start. Use `Rperf.profile` to activate the timer for specific
-sections. Outside `profile` blocks, overhead is zero.
+sections. Outside `profile` blocks, the timer is disarmed and overhead is zero.
+
+Note: the timer is process-wide, not per-thread. While a `profile` block is
+active on one thread, other threads running at the same time will also be
+sampled. Their samples carry their own labels (not the calling thread's labels),
+so they can be distinguished in the profile. This design is intentional: it
+provides complete visibility into what the process was doing during profiled
+sections, including GVL contention and background work.
 
 ### Rperf.profile(**labels, &block)
 
-Activates the sampling timer for the block duration and applies labels.
-Designed for use with `start(defer: true)` to profile only specific
-code paths.
+Activates the sampling timer for the block duration and applies labels to
+the current thread. Designed for use with `start(defer: true)`.
 
 ```ruby
 Rperf.start(defer: true, mode: :wall)
@@ -242,9 +325,21 @@ running). Raises `RuntimeError` if not started, `ArgumentError` without block.
 
 Returns the current thread's labels as a Hash. Empty hash if none set.
 
+### Rperf.load(path)
+
+Loads a `.json.gz` or `.json` profile file (saved by `rperf record` or `Rperf.save`)
+and returns the parsed data hash (same format as `Rperf.stop` / `Rperf.snapshot`).
+Gzip is auto-detected by magic bytes, so both compressed and plain files work.
+Warns to stderr if the file was saved by a different rperf version.
+
+```ruby
+data = Rperf.load("rperf.json.gz")   # gzip compressed
+data = Rperf.load("profile.json")    # plain text JSON
+```
+
 ### Rperf.save(path, data, format: nil)
 
-Writes data to path. format: :pprof, :collapsed, or :text.
+Writes data to path. format: :json, :pprof, :collapsed, or :text.
 nil auto-detects from extension.
 
 ### Rperf::RackMiddleware (Rack)
@@ -284,6 +379,88 @@ Sidekiq.configure_server do |config|
 end
 ```
 
+### Rperf::Viewer (Rack middleware)
+
+In-browser profiling UI with flamegraph, top table, and tag breakdown.
+Requires `require "rperf/viewer"`.
+
+**Security note**: Rperf::Viewer has no built-in authentication and exposes
+profiling data (including stack traces and label values) to anyone who can
+reach the endpoint. In production, always restrict access using your
+framework's authentication — see "Access control" below. The UI loads
+d3.js and d3-flame-graph from CDNs (cdnjs.cloudflare.com, cdn.jsdelivr.net).
+
+```ruby
+# config.ru or Rails config
+require "rperf/viewer"
+use Rperf::Viewer                         # mount at /rperf/ (default)
+use Rperf::Viewer, path: "/profiler"      # custom mount path
+use Rperf::Viewer, max_snapshots: 12      # keep fewer snapshots (default: 24)
+```
+
+Take snapshots via `Rperf::Viewer.instance.take_snapshot!` or
+`Rperf::Viewer.instance.add_snapshot(data)`.
+
+#### Typical setup with RackMiddleware and periodic snapshots
+
+```ruby
+require "rperf/viewer"
+require "rperf/rack"
+
+Rperf.start(mode: :wall, frequency: 999, defer: true)
+use Rperf::Viewer
+use Rperf::RackMiddleware
+run MyApp
+
+# Take a snapshot every 60 minutes in a background thread
+Thread.new do
+  loop do
+    sleep 60 * 60
+    Rperf::Viewer.instance&.take_snapshot!
+  end
+end
+```
+
+Visit `/rperf/` in a browser. Snapshots accumulate automatically
+(up to `max_snapshots`, oldest are discarded). You can also trigger
+a snapshot manually via an endpoint or console:
+
+```ruby
+Rperf::Viewer.instance.take_snapshot!
+```
+
+#### Access control
+
+Rperf::Viewer has no built-in authentication. Restrict access using
+your framework's existing mechanisms:
+
+```ruby
+# Rails: route constraint (e.g., admin-only)
+# config/routes.rb
+require "rperf/viewer"
+constraints ->(req) { req.session[:admin] } do
+  mount Rperf::Viewer.new(nil, path: ""), at: "/rperf"
+end
+```
+
+#### UI tabs
+
+- **Flamegraph** — Interactive flamegraph (d3-flame-graph). Click to zoom.
+- **Top** — Flat/cumulative weight table. Click column headers to sort.
+- **Tags** — Label key/value breakdown with weight bars. Click a row to
+  set tagfocus and switch to Flamegraph.
+
+#### Filtering controls
+
+- **tagfocus** — Regex matched against label values. Press Enter to apply.
+- **tagignore** — Dropdown checkboxes. Select `key = (none)` to exclude
+  samples without that key (e.g., background threads without `endpoint`).
+- **tagroot** — Dropdown checkboxes for label keys. Checked keys are
+  prepended as root frames (e.g., `[endpoint: GET /users]`).
+- **tagleaf** — Same as tagroot but appended as leaf frames.
+
+Tag keys are sorted alphabetically (`%`-prefixed VM state keys appear first).
+
 ## PROFILING MODES
 
 - **cpu** — Measures per-thread CPU time via Linux thread clock.
@@ -293,15 +470,26 @@ end
 - **wall** — Measures wall-clock time (CLOCK_MONOTONIC).
   Use for: finding where wall time goes, including I/O, sleep, GVL
   contention, and off-CPU waits.
-  Includes synthetic frames (see below).
+  Includes VM state labels (see below).
 
 ## OUTPUT FORMATS
 
-### pprof (default)
+### json (default) — rperf native format
+
+JSON representation of the internal data hash
+(the same hash returned by `Rperf.stop` / `Rperf.snapshot` — see
+"Return value" above for the full structure).
+Preserves all data including labels, VM state, thread info, and statistics.
+Readable by non-Ruby tools (Python, jq, etc.).
+Extension convention: `.json.gz` (gzip-compressed, default) or `.json` (plain text).
+View with: `rperf report` (opens rperf viewer in browser, no Go required).
+Load programmatically: `data = Rperf.load("rperf.json.gz")`
+
+### pprof
 
 Gzip-compressed protobuf. Standard pprof format.
 Extension convention: `.pb.gz`
-View with: `go tool pprof`, pprof-rs, or speedscope (via import).
+View with: `go tool pprof`, pprof-rs, speedscope, or `rperf report` (requires Go).
 
 Embedded metadata:
 
@@ -358,27 +546,45 @@ Example output:
 
 Format is auto-detected from the output file extension:
 
-    .collapsed → collapsed
-    .txt       → text
-    anything else → pprof
+    .json.gz    → json (rperf native, gzip compressed, default)
+    .json       → json (plain text, readable by jq)
+    .pb.gz      → pprof
+    .collapsed  → collapsed
+    .txt        → text
 
 The `--format` flag (CLI) or `format:` parameter (API) overrides auto-detect.
 
-## SYNTHETIC FRAMES
+## VM STATE LABELS
 
-In wall mode, rperf adds synthetic frames that represent non-CPU time:
+rperf tracks GVL and GC states as **labels** (tags) on samples, not as
+stack frames. The C extension records a VM state per sample, and the Ruby
+layer merges it into the sample's label set using reserved keys `%GVL`
+and `%GC`.
 
-- **[GVL blocked]** — Time the thread spent off-GVL (I/O, sleep, C extension
+In wall mode, GVL state labels are recorded:
+
+- **%GVL=blocked** — The thread was off-GVL (I/O, sleep, C extension
   releasing GVL). Attributed to the stack at SUSPENDED.
-- **[GVL wait]** — Time the thread spent waiting to reacquire the GVL after
+- **%GVL=wait** — The thread was waiting to reacquire the GVL after
   becoming ready. Indicates GVL contention. Same stack.
 
-In both modes, GC time is tracked:
+In both modes, GC state labels are recorded:
+
+- **%GC=mark** — Time spent in GC marking phase (wall time).
+- **%GC=sweep** — Time spent in GC sweeping phase (wall time).
+
+These labels appear in `label_sets` (e.g., `{:"%GVL" => "blocked"}`,
+`{:"%GC" => "mark"}`) and are written into pprof sample labels.
+
+To add VM state as frames in flamegraphs, use pprof tag options:
 
-- **[GC marking]** — Time spent in GC marking phase (wall time).
-- **[GC sweeping]** — Time spent in GC sweeping phase (wall time).
+    go tool pprof -tagleaf=%GVL profile.pb.gz
+    go tool pprof -tagroot=%GC profile.pb.gz
 
-These always appear as the leaf (deepest) frame in a sample.
+To filter by VM state:
+
+    go tool pprof -tagfocus=%GVL=blocked profile.pb.gz
+    go tool pprof -tagfocus=%GC=mark profile.pb.gz
 
 ## INTERPRETING RESULTS
 
@@ -403,20 +609,24 @@ To convert: 1,000,000 ns = 1 ms, 1,000,000,000 ns = 1 s.
 **Problem: slow request / high latency**
 - Mode: wall
 - Look for: functions with high cum wall time.
-- If [GVL blocked] is dominant → I/O or sleep is the bottleneck.
-- If [GVL wait] is dominant → GVL contention; reduce GVL-holding work
+- If %GVL=blocked is dominant → I/O or sleep is the bottleneck.
+  Filter: `go tool pprof -tagfocus=%GVL=blocked profile.pb.gz`
+- If %GVL=wait is dominant → GVL contention; reduce GVL-holding work
   or move work to Ractors / child processes.
+  Filter: `go tool pprof -tagfocus=%GVL=wait profile.pb.gz`
 
 **Problem: GC pauses**
 - Mode: cpu or wall
-- Look for: [GC marking] and [GC sweeping] samples.
-- High [GC marking] → too many live objects; reduce allocations.
-- High [GC sweeping] → too many short-lived objects; reuse or pool.
+- Look for: samples with %GC=mark and %GC=sweep labels.
+  Filter: `go tool pprof -tagfocus=%GC profile.pb.gz`
+- High %GC=mark → too many live objects; reduce allocations.
+- High %GC=sweep → too many short-lived objects; reuse or pool.
 
 **Problem: multithreaded app slower than expected**
 - Mode: wall
-- Look for: [GVL wait] time across threads.
-- High [GVL wait] means threads are serialized on the GVL.
+- Look for: samples with %GVL=wait label across threads.
+  Filter: `go tool pprof -tagfocus=%GVL=wait profile.pb.gz`
+- High %GVL=wait means threads are serialized on the GVL.
 
 ## READING COLLAPSED STACKS PROGRAMMATICALLY
 
@@ -454,11 +664,20 @@ Used internally by the CLI to pass options to the auto-started profiler:
     RPERF_OUTPUT=path     Output file path
     RPERF_FREQUENCY=hz    Sampling frequency
     RPERF_MODE=cpu|wall   Profiling mode
-    RPERF_FORMAT=fmt      pprof, collapsed, or text
+    RPERF_FORMAT=fmt      json, pprof, collapsed, or text
     RPERF_VERBOSE=1       Print statistics
     RPERF_SIGNAL=N|false  Timer signal number or 'false' for nanosleep (Linux only)
     RPERF_STAT=1          Enable stat mode (used by rperf stat)
     RPERF_STAT_REPORT=1   Include profile tables in stat output
+    RPERF_AGGREGATE=0     Disable C-level sample aggregation (raw mode)
+    RPERF_DEFER=1         Start with timer paused; use Rperf.profile to activate
+    RPERF_TMPDIR=path     Base directory for session directories (overrides default tmpdir)
+
+Internal variables (set automatically by the CLI — not for manual use):
+
+    RPERF_SESSION_DIR=path   Session directory for multi-process profiling
+    RPERF_ROOT_PROCESS=pid   Marks the root aggregating process
+    RPERF_STAT_COMMAND=str   Command string displayed in stat output
 
 ## TIPS
 

data/docs/logo.svg

@@ -0,0 +1,25 @@
+<svg viewBox="0 0 280 160" xmlns="http://www.w3.org/2000/svg">
+  <!-- Gauge arc -->
+  <path d="M70,100 A70,70 0 0,1 210,100" fill="none" stroke="#cc342d" stroke-width="3"/>
+
+  <!-- Tick marks -->
+  <g stroke="#cc342d" stroke-width="1.5">
+    <line x1="76" y1="68" x2="82" y2="74"/>
+    <line x1="100" y1="42" x2="104" y2="50"/>
+    <line x1="140" y1="32" x2="140" y2="40"/>
+    <line x1="180" y1="42" x2="176" y2="50"/>
+    <line x1="204" y1="68" x2="198" y2="74"/>
+  </g>
+
+  <!-- Ruby gem (below 12 o'clock tick) -->
+  <g transform="translate(140,52)">
+    <polygon points="-5,-5.5 5,-5.5 7.5,-1.5 0,6.5 -7.5,-1.5" fill="none" stroke="#cc342d" stroke-width="0.9"/>
+  </g>
+
+  <!-- Needle (-30%) -->
+  <line x1="140" y1="100" x2="172" y2="69" stroke="#8a7038" stroke-width="2.5" stroke-linecap="round"/>
+  <circle cx="140" cy="100" r="4" fill="#8a7038"/>
+
+  <!-- "rperf" text -->
+  <text x="140" y="148" text-anchor="middle" font-family="'Space Mono', monospace" font-size="24" font-weight="700" fill="#8a7038" letter-spacing="3">rperf</text>
+</svg>