fiber-profiler 0.7.0 → 0.7.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 907a8aaa882e1eb3d59e994e89b6c7181b85e5a1795f7bddab05cae9bf32afe8
-  data.tar.gz: 412d12500a7bb74047b883669c70575360222fcf7df7a8b3fc6ecf5f73efc6b9
+  metadata.gz: e3d1446f213526360f5368bfb8a08884033484f4ab868160340709652349c25c
+  data.tar.gz: 844c5319c20f8b0929342978790efe2d1aeb4dd37ae260cd073ff36c82c730fd
 SHA512:
-  metadata.gz: e5347ee515d48af4c75bbe85b32b8a71019ad8f48392a641d47e213012ed41bf2ee9a6112c11cbe46ecf66da23d9682d770ead22f73180eaa2950a636c00ce39
-  data.tar.gz: dceff8570643161dc70ec0b02bf72504cc5d554c4d5db201f6c9efdc63f64e0e6b90fa3b234a5b56174d67aab4d6cb98b4b8360250466aba46752fe9f5a5c1c6
+  metadata.gz: '08e564d3dc9c9abfba57056f3ff5a7e0fce516a5969ee014820ff2dac72b60fd51603c258c21cbee80c46d686f9ca9bbd2376a11b1202635371d0201b8a3b372'
+  data.tar.gz: 88d96fba98c5b9578683ad7f13cb898d70e8da83ce11a3b756b15d29b4cbc29649a61c7ec2d589cb9a1185e9646bee4ed018bbcbb1946defdd2e7163ae951aa8

checksums.yaml.gz.sig

@@ -1,4 +1,3 @@
-����s	�I<��f�D����m:�ݬ�,Kq�rZ��^��FJ���	�?)��-��Y-��)����/sN
-���Xo��
-Z:�����
-2�^���@I|���M�IiX����B�˹o]z�[�evs�K�f�m�@��*�X,���NL��$���b]4�
+ɫq��}���ƃ�Gp�<�~O.�EgG}y������%ω�`h+���Ɉ[3�%z�K}U���2&�5J��T<.�`��
+�����Z΃aD�,S��p���K��}���F�]ި���H@�e�b�&ִ�	Q��X��.4EᱟT�Ş���-jxz���fq��w^���|�l�l��Y
F�-��2�*�H�"~4��K�/��z��-��]�w4�G���G��Y��0+���~&��gDŒta|�o�Zv�.jP��
+>�@�q�C���a�pD�fֹ�nz

data/context/capture-mode.md

@@ -0,0 +1,80 @@
+# Capture Mode
+
+This guide explains how to trace fiber execution and analyze call timings with the capture profiler.
+
+Use capture mode when you need to investigate the calls made during a stall. It records call timings for sampled fiber executions and reports after the fiber switches away. For sampled backtraces while a stall is still in progress, see [Watchdog Mode](../watchdog-mode/index).
+
+## Usage
+
+Add `fiber-profiler` to your application's bundle as described in [Getting Started](../getting-started/index), then select capture mode when starting Async or Falcon:
+
+```bash
+FIBER_PROFILER=capture bundle exec falcon serve
+```
+
+Async starts and stops capture automatically. The legacy `FIBER_PROFILER_CAPTURE=true` setting also enables capture when `FIBER_PROFILER` is unset. An explicit `FIBER_PROFILER` value takes precedence.
+
+### Manual Instrumentation
+
+To reproduce a stall without a scheduler, save this as `capture.rb`. It starts a {ruby Fiber::Profiler::Capture} and simulates a blocking operation inside a fiber:
+
+```ruby
+require "fiber/profiler"
+
+profiler = Fiber::Profiler::Capture.new
+
+begin
+	profiler.start
+	
+	# Simulate a blocking operation without a scheduler:
+	Fiber.new(blocking: false) do
+		sleep 0.1
+	end.resume
+ensure
+	profiler.stop
+end
+```
+
+```bash
+bundle exec ruby capture.rb
+```
+
+This example starts the profiler explicitly, so it does not need `FIBER_PROFILER=capture`. The report should include `Kernel#sleep` and its elapsed duration. Start and stop profiling on the same thread.
+
+## Configuration
+
+Set these environment variables before loading `fiber-profiler`; the native extension reads them when it loads:
+
+| Variable | Default | Meaning |
+| --- | --- | --- |
+| `FIBER_PROFILER_CAPTURE_STALL_THRESHOLD` | `0.01` | Minimum execution duration in seconds to exceed before reporting a stall. |
+| `FIBER_PROFILER_CAPTURE_FILTER_THRESHOLD` | 10% of the stall threshold | Filter calls shorter than this duration in seconds. |
+| `FIBER_PROFILER_CAPTURE_TRACK_CALLS` | `true` | Record call timings. Set to `false` to report stall durations without tracing calls. |
+| `FIBER_PROFILER_CAPTURE_SAMPLE_RATE` | `1.0` | Fraction of eligible fiber executions to sample: `1.0` samples all, `0.1` samples approximately 10%. |
+
+These settings apply to capture mode only. The sample rate controls selection at fiber switches; it is not a time interval between backtrace samples.
+
+For explicit instrumentation, override the defaults with `Fiber::Profiler::Capture.new(stall_threshold: 0.05, filter_threshold: 0.005, track_calls: true, sample_rate: 0.1)`. Pass `output:` to select a writable IO; the default is standard error.
+
+## Reading Reports
+
+When standard error is a terminal, capture prints a readable call log. Redirected output uses one JSON object per stall, including the execution `duration` and a `calls` array. Each retained call includes its source location, class, method, duration, and nesting information.
+
+To collect the example's reports and summarize call timings:
+
+```bash
+bundle exec ruby capture.rb 2> capture.ndjson
+bundle exec bake input --file capture.ndjson fiber:profiler:analyze output
+```
+
+The analyzer aggregates call durations by source location and sorts the summary by total duration. Feed it capture reports; unrelated application output on standard error must be separated from the JSON first. With `track_calls: false`, reports contain no call timings to aggregate.
+
+Call durations can include time spent in nested calls, so adding durations across different locations does not give total application runtime. Short or uninformative calls may be filtered from the report.
+
+## Interpretation and Limits
+
+Capture samples non-blocking fibers and excludes blocking fibers, including the event-loop fiber. It measures wall time between fiber switches, so an ordinary scheduler-aware wait that yields does not count its entire wait time as a stall. A blocking operation that keeps the same fiber executing can count toward a stall.
+
+Reports are emitted when the fiber switches away. An operation that never yields or returns can therefore prevent its capture report from appearing. Use [Watchdog Mode](../watchdog-mode/index) to investigate an ongoing stall when Ruby thread scheduling is still possible.
+
+Call tracing can substantially affect performance. Reduce the sample rate to trace fewer executions, or disable call tracking if you only need stall durations. Use the [Falcon overhead benchmark](https://github.com/socketry/fiber-profiler/tree/main/benchmark/falcon) to compare modes, and measure with your application's Ruby/JIT configuration and workload.

data/context/getting-started.md

@@ -1,6 +1,6 @@
 # Getting Started
 
-This guide explains how to detect stalls using the fiber profiler.
+This guide explains how to install the fiber profiler and choose a mode for diagnosing event-loop stalls.
 
 ## Installation
 
@@ -10,7 +10,18 @@ Add the gem to your project:
 $ bundle add fiber-profiler
 ```
 
-## Usage
+## Choose a Mode
+
+A fiber that runs for too long without yielding prevents other work on the same event loop from progressing. The profiler offers two ways to investigate it:
+
+| Mode | Use it to | Reports |
+| --- | --- | --- |
+| [Watchdog](../watchdog-mode/index) | Find where a fiber is spending time during an ongoing stall. | Periodically sampled backtraces while the fiber is still executing. |
+| [Capture](../capture-mode/index) | Investigate individual call timings during a fiber's execution. | Traced calls and their durations after the fiber switches away. |
+
+Watchdog avoids method-call tracing and is a useful starting point for observing sustained stalls. Capture provides more detail at a higher instrumentation cost. Both affect the application being measured; see the [Falcon overhead benchmark](https://github.com/socketry/fiber-profiler/tree/main/benchmark/falcon) and measure with your own workload.
+
+## Integration with Async and Falcon
 
 Select the profiling mode when starting your application:
 
@@ -29,120 +40,4 @@ With Async, including Falcon, the scheduler starts and stops the selected profil
 
 Unknown modes raise `ArgumentError`. Set environment variables before loading the gem: the native capture settings are read when the extension loads. The new mode selector and watchdog settings are read when `Fiber::Profiler.default` is called.
 
-### Manual Instrumentation
-
-Instrument your code using the default profiler:
-
-```ruby
-#!/usr/bin/env ruby
-
-require "fiber/profiler"
-
-profiler = Fiber::Profiler.default
-
-begin
-	profiler&.start
-	
-	# Your application code:
-	Fiber.new do
-		sleep 0.1
-	end.resume
-ensure
-	profiler&.stop
-end
-```
-
-Running this program will output the following:
-
-```bash
-$ FIBER_PROFILER_CAPTURE=true bundle exec ./test.rb
-Fiber stalled for 0.105 seconds
-/Users/samuel/Developer/socketry/fiber-profiler/test.rb:11 in c-call 'Kernel#sleep' (0.105s)
-```
-
-## Integration with Async
-
-The fiber profiler is optionally supported by `Async`. Enable either mode using `FIBER_PROFILER=watchdog` or `FIBER_PROFILER=capture`. The legacy `FIBER_PROFILER_CAPTURE=true` setting continues to enable capture mode when `FIBER_PROFILER` is unset.
-
-Each scheduler obtains its own profiler. Watchdog binds to the thread when profiling starts, so schedulers in separate worker processes or threads are monitored independently. After `fork`, inherited profiling is stopped in the child; a new scheduler starts a new profiler normally.
-
-## Watchdog Mode
-
-Watchdog uses a thread-specific `:fiber_switch` tracepoint to track which fiber is executing. A separate Ruby thread periodically captures the monitored thread's backtrace. It does not install method-call tracing hooks or require a heartbeat task.
-
-By default, it samples every 100 ms, retains at most five recent stacks, and reports when the same fiber execution lasts at least 500 ms. A continued stall produces at most one report per additional threshold interval. Returning to the event loop or switching to another fiber resets the sample window. This measures individual uninterrupted fiber executions, rather than starvation of a scheduled heartbeat.
-
-To reproduce a stall, save this as `stall.rb`:
-
-```ruby
-require "async"
-
-Sync do
-	deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + 2
-	while Process.clock_gettime(Process::CLOCK_MONOTONIC) < deadline
-	end
-end
-```
-
-```bash
-FIBER_PROFILER=watchdog bundle exec ruby stall.rb
-```
-
-The reports should point to the busy loop. Replacing the loop with `sleep(2)` allows Async to keep scheduling fibers and should produce no reports.
-
-Terminal output contains readable stacks. Redirected output uses one JSON object per report, with `mode: "watchdog"`, process/thread/fiber identifiers, elapsed `duration`, and a bounded `samples` array. Each sample contains its elapsed time and `backtrace`. Duration is elapsed wall time since the fiber resumed, and a report is emitted while the fiber is still executing.
-
-### Configuration
-
-| Variable | Default | Meaning |
-| --- | --- | --- |
-| `FIBER_PROFILER_WATCHDOG_STALL_THRESHOLD` | `0.5` | Minimum execution duration in seconds before reporting, and minimum interval between repeated reports. |
-| `FIBER_PROFILER_WATCHDOG_SAMPLE_INTERVAL` | `0.1` | Delay in seconds between stack samples. |
-
-Both values must be finite and positive. Actual sampling intervals depend on Ruby and operating-system scheduling. Capture-specific settings, including its sample rate, do not configure watchdog mode.
-
-For explicit instrumentation, use `Fiber::Profiler::Watchdog.new(stall_threshold: 0.5, sample_interval: 0.1, max_samples: 5, output: $stderr)` and the same `start`/`stop` lifecycle as capture. The output must support writes from the watchdog thread. Stop the profiler before closing its output.
-
-If sampling or reporting raises a `StandardError`, the watchdog disables its tracepoint and stops sampling. It retains the exception in `watchdog.error` and attempts one warning to standard error. Failure to write that warning is also contained. These errors do not escape through `stop` or replace application errors. Call `stop` normally to finish cleanup; a subsequent `start` clears the error and resumes monitoring.
-
-### Interpretation and Limits
-
-Repeated frames identify code worth investigating: optimize expensive work, add cooperative yield points, or offload suitable work to a bounded thread pool. Sampling still has overhead; measure it for your workload.
-
-The repository includes a [Falcon overhead benchmark](https://github.com/socketry/fiber-profiler/tree/main/benchmark/falcon) comparing disabled, watchdog, and capture modes. It measures throughput, latency, server CPU time, and Ruby allocations across repeated runs. Use it as a starting point for measuring your own application before enabling continuous monitoring.
-
-The watchdog requires Ruby thread scheduling. Native code that holds the GVL without allowing other threads to run can prevent sampling entirely. Short stalls can occur between samples, and normal scheduler/OS delays can extend measured execution time. A report is a diagnostic lead, not proof that every sampled frame is expensive.
-
-When profiling starts on a blocking event-loop fiber, that fiber is excluded from monitoring, including its normal idle waits. Other fibers, including blocking application fibers, are monitored. When profiling starts inside a non-blocking application fiber, blocking fibers are excluded because the event-loop fiber is not known. Start and stop profiling on the monitored thread.
-
-## Default Environment Variables
-
-The following settings apply to capture mode only.
-
-### `FIBER_PROFILER_CAPTURE`
-
-Set to `true` to enable capturing of stalled fibers.
-
-### `FIBER_PROFILER_CAPTURE_STALL_THRESHOLD`
-
-Set the threshold in seconds for reporting a stalled fiber. Default is `0.01`.
-
-### `FIBER_PROFILER_CAPTURE_TRACK_CALLS`
-
-Set to `true` to track calls within the fiber. Default is `true`. This can be disabled to reduce overhead.
-
-### `FIBER_PROFILER_CAPTURE_SAMPLE_RATE`
-
-Set the sample rate of the profiler as a percentage of all context switches. The default is 1.0 (100%).
-
-## Analyzing Logs
-
-If you collect your logs in a file (e.g. as `ndjson`) you can analyze them using the included `bake` commands:
-
-```bash
-$ bundle exec bake input --file samples.ndjson fiber:profiler:analyze output
-```
-
-This will aggregate all the call logs and generate a short summary, ordered by duration.
-
-The analyzer consumes capture-mode call timings. Watchdog reports contain sampled backtraces instead of call timings and are not aggregated by this command.
+Each scheduler obtains its own profiler, so schedulers in separate worker processes or threads are monitored independently. After `fork`, inherited profiling is stopped in the child; a new scheduler starts a new profiler normally.

data/context/index.yaml

@@ -8,4 +8,13 @@ metadata:
 files:
 - path: getting-started.md
   title: Getting Started
-  description: This guide explains how to detect stalls using the fiber profiler.
+  description: This guide explains how to install the fiber profiler and choose a
+    mode for diagnosing event-loop stalls.
+- path: watchdog-mode.md
+  title: Watchdog Mode
+  description: This guide explains how to sample ongoing fiber stalls with the watchdog
+    profiler.
+- path: capture-mode.md
+  title: Capture Mode
+  description: This guide explains how to trace fiber execution and analyze call timings
+    with the capture profiler.

data/context/watchdog-mode.md

@@ -0,0 +1,100 @@
+# Watchdog Mode
+
+This guide explains how to sample ongoing fiber stalls with the watchdog profiler.
+
+Use watchdog mode to find where a fiber is spending time while it prevents other fibers from running. It reports during the stall, so you can investigate an operation that has not returned yet. For individual call timings, see [Capture Mode](../capture-mode/index).
+
+## Usage
+
+Add `fiber-profiler` to your application's bundle as described in [Getting Started](../getting-started/index), then select watchdog mode when starting Async or Falcon:
+
+```bash
+FIBER_PROFILER=watchdog bundle exec falcon serve
+```
+
+Async starts and stops the watchdog automatically. Each scheduler gets its own profiler, which binds to the thread when profiling starts.
+
+To reproduce a stall, save this as `stall.rb`:
+
+```ruby
+require "async"
+
+Sync do
+	deadline = Process.clock_gettime(Process::CLOCK_MONOTONIC) + 2
+	while Process.clock_gettime(Process::CLOCK_MONOTONIC) < deadline
+	end
+end
+```
+
+```bash
+FIBER_PROFILER=watchdog bundle exec ruby stall.rb
+```
+
+The reports should point to the busy loop. Replacing the loop with `sleep(2)` allows Async to keep scheduling fibers and should produce no reports.
+
+## How Sampling Works
+
+Watchdog uses a thread-specific `:fiber_switch` tracepoint to track which fiber is executing. A separate Ruby thread periodically captures the monitored thread's backtrace. It does not install method-call tracing hooks or require a heartbeat task.
+
+By default, it samples every 100 ms, retains at most five recent stacks, and reports when the same fiber execution lasts at least 500 ms. A continued stall produces at most one report per additional threshold interval. Returning to the event loop or switching to another fiber resets the sample window. The measured duration covers one uninterrupted fiber execution.
+
+## Configuration
+
+Set these environment variables before starting your application. They are read when the default watchdog is constructed:
+
+| Variable | Default | Meaning |
+| --- | --- | --- |
+| `FIBER_PROFILER_WATCHDOG_STALL_THRESHOLD` | `0.5` | Minimum execution duration in seconds before reporting, and minimum interval between repeated reports. |
+| `FIBER_PROFILER_WATCHDOG_SAMPLE_INTERVAL` | `0.1` | Delay in seconds between stack samples. |
+
+Both values must be finite and positive. Actual sampling intervals depend on Ruby and operating-system scheduling. Capture-specific settings, including its sample rate, do not configure watchdog mode.
+
+### Manual Instrumentation
+
+When managing fibers directly, construct a {ruby Fiber::Profiler::Watchdog} and stop it in an `ensure` block:
+
+```ruby
+require "fiber/profiler/watchdog"
+
+watchdog = Fiber::Profiler::Watchdog.new(
+	stall_threshold: 0.5,
+	sample_interval: 0.1,
+	max_samples: 5,
+	output: $stderr
+)
+
+begin
+	watchdog.start
+	
+	# Simulate an application fiber blocked on an operation:
+	Fiber.new do
+		sleep 1
+	end.resume
+ensure
+	watchdog.stop
+end
+```
+
+This example starts the profiler explicitly, so it does not need `FIBER_PROFILER=watchdog`. Start and stop profiling on the monitored thread. `max_samples` must be a positive integer; there is no environment variable for it. The output must support writes from the watchdog thread. Stop the profiler before closing its output.
+
+When profiling starts on a blocking event-loop fiber, that fiber is excluded from monitoring, including its normal idle waits. Other fibers, including blocking application fibers, are monitored. When profiling starts inside a non-blocking application fiber, blocking fibers are excluded because the event-loop fiber is not known.
+
+If sampling or reporting raises a `StandardError`, the watchdog disables its tracepoint and stops sampling. It retains the exception in `watchdog.error` and attempts one warning to standard error. Failure to write that warning is also contained. These errors do not escape through `stop` or replace application errors. Call `stop` normally to finish cleanup; a subsequent `start` clears the error and resumes monitoring.
+
+## Reading Reports
+
+Terminal output contains readable stacks. Redirected output uses one JSON object per report:
+
+```bash
+FIBER_PROFILER=watchdog bundle exec ruby stall.rb 2> watchdog.ndjson
+```
+
+Each report contains `mode: "watchdog"`, process/thread/fiber identifiers, elapsed `duration`, and a bounded `samples` array. Each sample contains its elapsed time and `backtrace`. Duration is elapsed wall time since the fiber resumed, and a report is emitted while the fiber is still executing.
+
+Repeated frames identify code worth investigating: optimize expensive work, add cooperative yield points, or offload suitable work to a bounded thread pool. Watchdog reports contain sampled backtraces, so the capture-mode `fiber:profiler:analyze` command does not aggregate them.
+
+## Interpretation and Limits
+
+The watchdog requires Ruby thread scheduling. Native code that holds the GVL without allowing other threads to run can prevent sampling entirely. Short stalls can occur between samples, and normal scheduler/OS delays can extend measured execution time. A report is a diagnostic lead, not proof that every sampled frame is expensive.
+
+Sampling still has overhead. Use the [Falcon overhead benchmark](https://github.com/socketry/fiber-profiler/tree/main/benchmark/falcon) as a starting point for measuring your own application before enabling continuous monitoring.

data/lib/fiber/profiler/version.rb

@@ -7,6 +7,6 @@
 class Fiber
 	# @namespace
 	module Profiler
-		VERSION = "0.7.0"
+		VERSION = "0.7.1"
 	end
 end

data/readme.md

@@ -12,7 +12,11 @@ Migrating existing applications to the event loop can be tricky. One of the most
 
 Please see the [project documentation](https://socketry.github.io/fiber-profiler/) for more details.
 
-  - [Getting Started](https://socketry.github.io/fiber-profiler/guides/getting-started/index) - This guide explains how to detect stalls using the fiber profiler.
+  - [Getting Started](https://socketry.github.io/fiber-profiler/guides/getting-started/index) - This guide explains how to install the fiber profiler and choose a mode for diagnosing event-loop stalls.
+
+  - [Watchdog Mode](https://socketry.github.io/fiber-profiler/guides/watchdog-mode/index) - This guide explains how to sample ongoing fiber stalls with the watchdog profiler.
+
+  - [Capture Mode](https://socketry.github.io/fiber-profiler/guides/capture-mode/index) - This guide explains how to trace fiber execution and analyze call timings with the capture profiler.
 
 ## Releases
 

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: fiber-profiler
 version: !ruby/object:Gem::Version
-  version: 0.7.0
+  version: 0.7.1
 platform: ruby
 authors:
 - Samuel Williams
@@ -54,8 +54,10 @@ extensions:
 extra_rdoc_files: []
 files:
 - bake/fiber/profiler/analyze.rb
+- context/capture-mode.md
 - context/getting-started.md
 - context/index.yaml
+- context/watchdog-mode.md
 - ext/extconf.rb
 - ext/fiber/profiler/capture.c
 - ext/fiber/profiler/capture.h