sus-fixtures-benchmark 0.1.0 → 0.2.1

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 63078795b33d87a934f27e4b1a3499b67a5a1dbc3d4fb9c50cc83a3014dc9916
-  data.tar.gz: c2c82e5e602dfd1f3a2e9f27609cbd518fc8822b70da8bb226a606bd9f6de06a
+  metadata.gz: de94d05fdae13b52e4e260d2f2fc5fc65792436d6756409fb332e77e4bc64b81
+  data.tar.gz: c0a75a615f5021dbd5ae2f26bb6d5a1eb4ac7c9d5ae22955bffe6573106d260b
 SHA512:
-  metadata.gz: a7fb2ab5b00ab5061b5fee54bb1bdfe0587ec6e95f811eca1b49e914a3511c0f7da13dbabe861b30c797c476c65577d3eb4200243cd2557ce19083b25269ab0a
-  data.tar.gz: 96a95d1c1e20585ff56e82035b0819f0c40cb967436ef32837bc389e67524b318e7e04b2ac863f55d172f8e108151e2850de2d169d4b36b10e796514ddb626a4
+  metadata.gz: 0212ff21fd9e1dc27a2ee42d08bce8e1a2a713bdb204c869789f37a651fa828ff9047d4c2b4f4c85cf1322c822ce232fa2691da90a8d1808e1e7b94636eeb9e4
+  data.tar.gz: 72b37243db4d7c841924e17458dae86e8c028f1327f27711ece5603cb21f87831399498756fe2e1b4ad09be3119db714208c918c99d22a8742bb9efff1c47d0d

data/context/getting-started.md

@@ -0,0 +1,127 @@
+# Getting Started
+
+This guide explains how to use the `sus-fixtures-benchmark` gem to measure and benchmark code performance within your Sus test suite.
+
+## Quick Start
+
+### Project Structure
+
+We recommend organizing your benchmarks in a dedicated `benchmark/` directory:
+
+```
+your-project/
+├── benchmark/
+│   ├── database_performance.rb
+│   ├── algorithm_comparison.rb
+│   └── memory_usage.rb
+├── test/
+├── lib/
+└── Gemfile
+```
+
+This keeps your benchmarks separate from your regular tests and makes them easy to find and run.
+
+### Basic Measurement
+
+The simplest way to measure code performance is using the `measure` method:
+
+```ruby
+# benchmark/array_performance.rb
+require "sus/fixtures/benchmark"
+
+describe "Array Performance" do
+  include Sus::Fixtures::Benchmark
+  
+  let(:data) {(1..1000).to_a}
+  
+  measure "array iteration" do |repeats|
+    repeats.times do
+      data.each{|i| i * 2}
+    end
+  end
+end
+```
+
+This will automatically run your code until statistical convergence is reached and report the results.
+
+**Run with:** `sus --verbose benchmark/array_performance.rb`
+
+### Understanding the Output
+
+When you run your tests with `sus --verbose`, you'll see output like:
+
+```
+measure array iteration 34 samples, mean: 0.15ms, standard deviation: 0.02ms, standard error: 0.003ms
+```
+
+**Important:** You must run `sus --verbose` to see the benchmark output. Without the verbose flag, the benchmark results will not be displayed.
+
+This tells you:
+- **34 samples**: How many times the code was executed.
+- **mean: 0.15ms**: Average execution time.
+- **standard deviation: 0.02ms**: How much variation there was between runs.
+- **standard error: 0.003ms**: How reliable the mean estimate is.
+
+## Measurement Modes
+
+### Automatic Convergence (Default)
+
+By default, the benchmark runs until it achieves statistical convergence:
+
+```ruby
+measure "database query" do |repeats|
+  repeats.times do
+    User.where(active: true).count
+  end
+end
+```
+
+The system will automatically determine when it has enough samples to provide a reliable measurement based on:
+- **Confidence level**: 95% by default
+- **Margin of error**: 2% of the mean by default (0.02)
+- **Minimum samples**: 8 by default
+
+### Fixed Number of Iterations
+
+If you need a specific number of iterations, use the `exactly` method:
+
+```ruby
+measure "quick test" do |repeats|
+  repeats.exactly(10).times do
+    # Your code here
+  end
+end
+```
+
+This is useful for:
+- Quick smoke tests.
+- When you know the exact number of iterations needed.
+- Debugging or development scenarios.
+
+## Configuration Options
+
+### Customizing the Sampler
+
+You can customize the statistical parameters directly in the `measure` method:
+
+```ruby
+measure "high precision", minimum: 20, confidence: 0.98, margin_of_error: 0.01 do |repeats|
+  repeats.times do
+    # Your code here
+  end
+end
+```
+
+### Parameter Reference
+
+You should try to avoid deviating from the defaults.
+
+  - **`minimum`** (Integer): Minimum samples before convergence (default: 8). 
+    - **Lower `minimum`**: Quick development feedback (e.g., `minimum: 3`)
+    - **Higher `minimum`**: High-variance operations (e.g., `minimum: 30`)
+  - **`confidence`** (Float): Confidence level 0.0-1.0 (default: 0.95). High confidence (above 0.98) may take an extremely long time to converge.
+    - **Lower `confidence`**: Faster results (e.g., `confidence: 0.90`)
+    - **Higher `confidence`**: Production guarantees (e.g., `confidence: 0.98`)
+  - **`margin_of_error`** (Float): Acceptable error as a fraction of the mean (default: 0.02 = 2%).
+    - **Lower `margin_of_error`**: High precision (e.g., `margin_of_error: 0.01`)
+    - **Higher `margin_of_error`**: Rough estimates (e.g., `margin_of_error: 0.05`)

data/lib/sus/fixtures/benchmark/repeats.rb

@@ -1,6 +1,5 @@
 # frozen_string_literal: true
 
-#
 # Released under the MIT License.
 # Copyright, 2025, by Samuel Williams.
 
@@ -12,27 +11,71 @@ module Sus
 			# Represents a benchmarking helper that executes a block multiple times, collecting timing samples until statistical convergence is reached.
 			class Repeats
 				# Initializes a new {Repeats} object with a sampler to collect timing data.
-				# @parameter samples [Sampler] The sampler object to collect timing data.
-				def initialize(samples)
-					@samples = samples
+				# @parameter sampler [Sampler] The sampler object to collect timing data.
+				def initialize(sampler)
+					@sampler = sampler
 				end
 				
+				# @attribute [Sampler] The sampler that collects timing data.
+				attr :sampler
+				
 				# Samples the execution time of the given block and adds the result to the sampler.
 				# @parameter block [Proc] The block to benchmark.
 				private def sample!(block)
 					time = Benchmark::Time.measure do
 						block.call
 					end
-					@samples.add(time.real)
+					@sampler.add(time.real)
 				end
 				
 				# Repeatedly executes the block until the sampler reports convergence.
 				# @parameter block [Proc] The block to benchmark.
 				def times(&block)
-					until @samples.converged?
+					until @sampler.converged?
 						sample!(block)
 					end
 				end
+				
+				# Represents a benchmarking helper that executes a block a fixed number of times.
+				class Exactly
+					# Initializes a new {Exactly} object with a sampler and a fixed count.
+					# @parameter sampler [Sampler] The sampler object to collect timing data.
+					# @parameter count [Integer] The exact number of times to execute the block.
+					def initialize(sampler, count)
+						@sampler = sampler
+						@count = count
+					end
+					
+					# @attribute [Sampler] The sampler that collects timing data.
+					attr :sampler
+					
+					# @attribute [Integer] The number of times to execute the block.
+					attr :count
+					
+					# Samples the execution time of the given block and adds the result to the sampler.
+					# @parameter block [Proc] The block to benchmark.
+					private def sample!(block)
+						time = Benchmark::Time.measure do
+							block.call
+						end
+						@sampler.add(time.real)
+					end
+					
+					# Executes the block exactly the specified number of times.
+					# @parameter block [Proc] The block to benchmark.
+					def times(&block)
+						@count.times do
+							sample!(block)
+						end
+					end
+				end
+				
+				# Sets a fixed number of times to execute the block, returning a new {Exactly} instance.
+				# @parameter count [Integer] The exact number of times to execute the block.
+				# @returns [Exactly] A new instance that will execute the block exactly the specified number of times.
+				def exactly(count)
+					Exactly.new(@sampler, count)
+				end
 			end
 		end
 	end

data/lib/sus/fixtures/benchmark/time.rb

@@ -41,7 +41,7 @@ module Sus
 				# The real (wall clock) time in seconds.
 				# @returns [Float]
 				attr :real
-
+				
 				# Returns a string representation of the real time in seconds.
 				# @returns [String]
 				def to_s

data/lib/sus/fixtures/benchmark/version.rb

@@ -6,7 +6,7 @@
 module Sus
 	module Fixtures
 		module Benchmark
-			VERSION = "0.1.0"
+			VERSION = "0.2.1"
 		end
 	end
 end

data/lib/sus/fixtures/benchmark.rb

@@ -26,15 +26,19 @@ module Sus
 				# @parameter parent [Class] The parent test context class.
 				# @parameter description [String] The description of the measure.
 				# @parameter unique [Boolean] Whether the measure should have a unique identity.
+				# @parameter **options [Hash] Options to pass to the Sampler constructor.
 				# @parameter block [Proc] The block to execute for the measure.
 				# @returns [Class] The new measure class.
-				def self.build(parent, description, unique: true, &block)
+				def self.build(parent, description, unique: true, **options, &block)
 					base = Class.new(parent)
 					base.extend(self)
 					base.description = description
 					base.identity = Identity.nested(parent.identity, base.description, unique: unique)
 					base.set_temporary_name("#{self}[#{description}]")
 					
+					# Store sampler options for later use
+					base.define_singleton_method(:sampler_options) {options}
+					
 					if block_given?
 						base.define_method(:run, &block)
 					end
@@ -67,7 +71,8 @@ module Sus
 					assertions.nested(self, identity: self.identity, isolated: true, measure: true) do |assertions|
 						instance = self.new(assertions)
 						
-						samples = Sampler.new
+						# Create sampler with options
+						samples = Sampler.new(**self.sampler_options)
 						repeats = Repeats.new(samples)
 						
 						instance.around do

data/readme.md

@@ -12,11 +12,19 @@ bundle add sus-fixtures-benchmark
 
 ## Usage
 
-Please see the [project documentation](https://suspecting.github.io/sus-fixtures-benchmark/) for more details.
+Please see the [project documentation](https://socketry.github.io/sus-fixtures-benchmark/) for more details.
 
 ## Releases
 
-Please see the [project releases](https://suspecting.github.io/sus-fixtures-benchmark/releases/index) for all releases.
+Please see the [project releases](https://socketry.github.io/sus-fixtures-benchmark/releases/index) for all releases.
+
+### v0.2.1
+
+  - Fix links and add context.
+
+### v0.2.0
+
+  - Added `exactly(count)` method to `Sus::Fixtures::Benchmark::Repeats` which returns an `Exactly` instance for fixed-count benchmarking.
 
 ### v0.1.0
 

data/releases.md

@@ -1,5 +1,13 @@
 # Releases
 
+## v0.2.1
+
+  - Fix links and add context.
+
+## v0.2.0
+
+  - Added `exactly(count)` method to `Sus::Fixtures::Benchmark::Repeats` which returns an `Exactly` instance for fixed-count benchmarking.
+
 ## v0.1.0
 
   - Added `Sus::Fixtures::Benchmark::Repeats` which is not an integer, but an instance that allows the block to be executed multiple times until the benchmark converges.

metadata

@@ -1,7 +1,7 @@
 --- !ruby/object:Gem::Specification
 name: sus-fixtures-benchmark
 version: !ruby/object:Gem::Version
-  version: 0.1.0
+  version: 0.2.1
 platform: ruby
 authors:
 - Samuel Williams
@@ -57,6 +57,7 @@ extensions: []
 extra_rdoc_files: []
 files:
 - agent.md
+- context/getting-started.md
 - lib/sus/fixtures/benchmark.rb
 - lib/sus/fixtures/benchmark/repeats.rb
 - lib/sus/fixtures/benchmark/sampler.rb
@@ -65,13 +66,13 @@ files:
 - license.md
 - readme.md
 - releases.md
-homepage: https://github.com/suspecting/sus-fixtures-benchmark
+homepage: https://github.com/socketry/sus-fixtures-benchmark
 licenses:
 - MIT
 metadata:
-  documentation_uri: https://suspecting.github.io/sus-fixtures-benchmark/
+  documentation_uri: https://socketry.github.io/sus-fixtures-benchmark/
   funding_uri: https://github.com/sponsors/ioquatix/
-  source_code_uri: https://github.com/suspecting/sus-fixtures-benchmark.git
+  source_code_uri: https://github.com/socketry/sus-fixtures-benchmark.git
 rdoc_options: []
 require_paths:
 - lib