RubyGems - benchmark-sweet - Versions diffs - 0.2.2 → 0.3.0 - Mend

benchmark-sweet 0.2.2 → 0.3.0

This diff represents the content of publicly available package versions that have been released to one of the supported registries. The information contained in this diff is provided for informational purposes only and reflects changes between package versions as they appear in their respective public registries.

Files changed (13) hide show

checksums.yaml +4 -4
data/CHANGELOG.md +13 -2
data/README.md +209 -121
data/benchmark-sweet.gemspec +9 -11
data/examples/benchmark_simple_database.rb +20 -4
data/lib/benchmark/sweet/comparison.rb +1 -1
data/lib/benchmark/sweet/job.rb +17 -24
data/lib/benchmark/sweet/memory.rb +3 -3
data/lib/benchmark/sweet/queries.rb +35 -14
data/lib/benchmark/sweet/version.rb +1 -1
data/lib/benchmark/sweet.rb +20 -12
metadata +23 -43
data/.travis.yml +0 -7

checksums.yaml CHANGED Viewed

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: d244cd69d99d599b4c6b7ae175ebcfa59b1c826ef66f643662bac4424d884f1b
-  data.tar.gz: 8350da53d68b067996858b52bdfbd5b9e7525d88042d099f0d68d443a00dc3c6
+  metadata.gz: 54cd4724f37f8b493ebb7ebcfd949d7a05fec9e09d6fcf8402a01bea4a92e1b5
+  data.tar.gz: b1853dfaf463323d2796ef194105a6ff7ac0b01260be29c279ab5a3d6b03e9ba
 SHA512:
-  metadata.gz: 129add152f798180f6d7f3d76c294b7a8c78e8aa917a7c838c713a2177d339d3396b9155e0a253b96643207366e20377d9c61c219dc7f9088b0708356fa628c4
-  data.tar.gz: d618214ba5e14cf06c6c1ae323e7836514af96ebd39602fbf1c682be091ee8e20f1f755bc70a0756adddcddf3e4e5a794c45b14c2ce8a2ce8bdaac4933913de3
+  metadata.gz: ae9f7595d95ee066fc849d856f7deda2aa9d5276d47e053a914da4736c1b5e62a52efc45137093704a2d72ed64b9a277837e21055294846ee887122bb57e8e4d
+  data.tar.gz: 12549ad3a023a1cf50a03972d315e49e360c8995157e853d50eb56c0f509fd14822a01bf0779f39ee148c90e3cc368981f4cf9a17ce0b6d79063a859d1dcf887

data/CHANGELOG.md CHANGED Viewed

@@ -6,12 +6,22 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 ## [Unreleased]
+## [0.3.0] - 2026-02-07
 ### Added
+- Tests for Comparison, Job, Item, and table rendering
 ### Changed
+- Removed activesupport runtime dependency (label values normalized to strings automatically)
+- Loosened dependency pins for benchmark-ips (~> 2.8) and memory_profiler (~> 0.9)
+- Improved README with single-example walkthrough showing how report_with reshapes output
 ### Fixed
-- properly indent tables with iso color codes (thanks @d-m-u)
+- Fixed @instance typo in QueryCounter#callback (was crashing on cached queries)
+- Fixed table rendering crash when header is wider than values
+- Fixed table column alignment with ANSI color codes
+- Properly indent tables with ANSI color codes (thanks @d-m-u)
+- Removed dead code (labels_have_symbols!, unused symbol_value check)
 ## [0.2.1] - 2020-06-24
@@ -38,7 +48,8 @@ and this project adheres to [Semantic Versioning](http://semver.org/).
 ### Added
 - good stuff
-[Unreleased]: https://github.com/kbrock/benchmark-sweet/compare/v0.2.2...HEAD
+[Unreleased]: https://github.com/kbrock/benchmark-sweet/compare/v0.3.0...HEAD
+[0.3.0]: https://github.com/kbrock/benchmark-sweet/compare/v0.2.2...v0.3.0
 [0.2.2]: https://github.com/kbrock/benchmark-sweet/compare/v0.2.1...v0.2.2
 [0.2.1]: https://github.com/kbrock/benchmark-sweet/compare/v0.2.0...v0.2.1
 [0.2.0]: https://github.com/kbrock/benchmark-sweet/compare/v0.0.1...v0.2.0

data/README.md CHANGED Viewed

@@ -24,17 +24,68 @@ Or install it yourself as:
     $ gem install benchmark-sweet
-## Example 1
+## How it works
-### Code
+### Labels
+Every benchmark item has a **label** — a Hash of metadata that identifies it. Labels are built
+from `metadata` and the name passed to `report`:
+```ruby
+x.metadata version: RUBY_VERSION
+x.metadata data: "nil" do
+  x.report("to_s.split") { ... }
+end
+# produces label: {version: "3.4.8", data: "nil", method: "to_s.split"}
+```
+These label keys (`version`, `data`, `method`) are not reserved words — they are whatever
+metadata you choose. The same keys are then used with `compare_by` and `report_with` to
+control comparisons and display.
+### `compare_by` — which items to compare
+`compare_by` partitions results into comparison groups. Within each group, items are ranked
+best to worst and slowdown factors are calculated. Items in different groups are never
+compared against each other.
+Method names and metrics are always part of the partition implicitly.
+```ruby
+x.compare_by :data
+```
+**Why this matters:** splitting a nil is much faster than splitting a string. Without
+`compare_by :data`, the string case would show as "slower" — but it's a different workload,
+not a worse implementation.
+### `report_with` — how to display the results
+`report_with` controls the table layout using four parameters:
+- **`grouping`** — a separate table is generated for each unique value (default: one table for everything)
+- **`row`** — what goes on the left side of each row (default: the full label)
+- **`column`** — what goes across the top as column headers (default: `:metric`)
+- **`value`** — what appears in each cell (default: `:comp_short` — value with slowdown)
+Each accepts a Symbol (label key), an Array of Symbols (joined to make compound headers),
+or a lambda for custom formatting.
+The rest of this README shows one example data set displayed multiple ways by changing
+only `report_with`, so you can see how each parameter affects the output.
+## Example: same data, different views
+### The benchmark
+Two split implementations, tested on both nil and string data, measuring ips and memsize:
 ```ruby
 require "benchmark/sweet"
-require "active_support/all"
 NSTRING = nil
-DELIMITER='/'.freeze
-STRING="ab/cd/ef/gh".freeze
+DELIMITER = '/'.freeze
+STRING = "ab/cd/ef/gh".freeze
 Benchmark.items(metrics: %w(ips memsize)) do |x|
   x.metadata data: "nil" do
@@ -46,72 +97,123 @@ Benchmark.items(metrics: %w(ips memsize)) do |x|
     x.report("?split:[]")  { STRING ? STRING.split(DELIMITER) : [] }
   end
-  # partition the data by data value (nil vs string)
-  # that way we're not comparing a split on a nil vs a split on a populated string
-  compare_by :data
-  # each row is a different method (via `row: :method`)
-  # each column is by data type (via `column: :data` - specified via `metadata data: "nil"`)
-  x.report_with grouping: :metric, sort: true, row: :method, column: :data
+  x.compare_by :data
+  x.report_with ...  # <-- we'll vary this part below
 end
 ```
-#### `compare_by`
+This produces 8 data points (2 methods x 2 data types x 2 metrics). The `compare_by :data`
+ensures nil-vs-nil and str-vs-str comparisons — never nil-vs-str.
-The code takes a different amount of time to process a `nil` vs a string. So the values
-are given metadata with the appropriate data used (i.e.: `metadata data: "nil"`).
-The benchmark is then told that the results need to be partitioned by `data` (i.e.: `compare_by :data`).
+Now let's see how `report_with` changes the presentation of these same 8 data points.
-The multipliers are only comparing values with the same data value and do not compare `"string"` values with `"nil"` values.
+---
-Values for method labels (e.g.: `"to_s.split"`) and metrics (e.g.: `ips`) are already part of the partition.
+### View 1: a table per metric
-#### `grouping`
+```ruby
+x.report_with grouping: :metric, row: :method, column: :data, sort: true
+```
-In this example, each metric is considered distinct so each metric is given a
-unique table (i.e.: `grouping: :metric`)
+- **`grouping: :metric`** — one table for `ips`, another for `memsize`
+- **`row: :method`** — each method gets its own row
+- **`column: :data`** — nil and str become column headers
-Metrics of `ips` and `memsize` are calculated (i.e.: `metrics: %w(ips memsize)`)
+#### metric ips
-#### `row`
+```
+     method |                   nil |           str
+------------|------------------------|---------------
+ ?split:[]  | 7090539.2 i/s          | 1322010.9 i/s
+ to_s.split | 3703981.6 i/s - 1.91x  | 1311153.9 i/s
+```
-A different method is given per row (i.e. `row: :method`)
+#### metric memsize
-#### `column`
+```
+     method |                nil |         str
+------------|--------------------|--------------
+ ?split:[]  | 40.0 bytes         | 360.0 bytes
+ to_s.split | 80.0 bytes - 2.00x | 360.0 bytes
+```
-The other axis for the columns is the type of data passed (i.e.: `column: :data`)
-This is not a native value, it is specified when the items are specified (e.g.:`metadata data: "nil"`)
+Each table is small and focused. The slowdown `1.91x` only compares methods within the same
+data type (nil column), because `compare_by :data` keeps them separate.
-### example output
+---
-#### metric ips
+### View 2: everything in one table
- method     | nil                   | str
-------------|-----------------------|---------------
-`?split:[]` | 7090539.2 i/s         | 1322010.9 i/s
-`to_s.split`| 3703981.6 i/s - 1.91x | 1311153.9 i/s
+```ruby
+x.report_with row: :method, column: [:data, :metric], sort: true
+```
-#### metric memsize
+- **no `grouping`** — all data in one table
+- **`column: [:data, :metric]`** — combines data and metric into compound column headers like `nil_ips`, `str_memsize`
- method     | nil                | str
-------------|--------------------|-------------
-`?split:[]` | 40.0 bytes         | 360.0 bytes
-`to_s.split`| 80.0 bytes - 2.00x | 360.0 bytes
+```
+     method |             nil_ips |           str_ips |        nil_memsize |     str_memsize
+------------|---------------------|-------------------|--------------------|----------------
+ ?split:[]  | 7090539.2 i/s       | 1322010.9 i/s     | 40.0 bytes         | 360.0 bytes
+ to_s.split | 3703981.6 i/s - 1.91x | 1311153.9 i/s   | 80.0 bytes - 2.00x | 360.0 bytes
+```
+One wide table — useful for comparing all dimensions at a glance.
-## Example 2
+---
-### Code
+### View 3: a table per data type
 ```ruby
-require "benchmark/sweet"
-require "active_support/all"
+x.report_with grouping: :data, row: :method, column: :metric, sort: true
+```
-NSTRING = nil
-DELIMITER='/'.freeze
-STRING="ab/cd/ef/gh".freeze
+- **`grouping: :data`** — one table for nil, another for str
+- **`column: :metric`** — ips and memsize become columns
-Benchmark.items(metrics: %w(ips memsize), memory: 3, warmup: 1, time: 1, quiet: false, force: ENV["FORCE"] == "true") do |x|
+#### data nil
+```
+     method |                   ips |            memsize
+------------|-----------------------|-------------------
+ ?split:[]  | 7090539.2 i/s         | 40.0 bytes
+ to_s.split | 3703981.6 i/s - 1.91x | 80.0 bytes - 2.00x
+```
+#### data str
+```
+     method |           ips |     memsize
+------------|---------------|------------
+ ?split:[]  | 1322010.9 i/s | 360.0 bytes
+ to_s.split | 1311153.9 i/s | 360.0 bytes
+```
+Now each table shows "how do these methods compare for this particular input?" —
+all metrics side by side for the same workload.
+---
+### Summary
+The same data, three different views — controlled entirely by `report_with`:
+| View | `grouping` | `row` | `column` | Result |
+|------|-----------|-------|----------|--------|
+| 1 | `:metric` | `:method` | `:data` | table per metric, data across top |
+| 2 | *(none)* | `:method` | `[:data, :metric]` | one wide table |
+| 3 | `:data` | `:method` | `:metric` | table per data type, metrics across top |
+The rule: every label key must appear in exactly one of `grouping`, `row`, or `column`
+(`:metric` is included automatically if not specified). Changing which key goes where
+reshapes the table.
+## Cross-version comparisons with `save_file`
+Add `version` metadata and use `save_file` to accumulate results across multiple runs:
+```ruby
+Benchmark.items(metrics: %w(ips memsize)) do |x|
   x.metadata version: RUBY_VERSION
   x.metadata data: "nil" do
     x.report("to_s.split") { NSTRING.to_s.split(DELIMITER)           }
@@ -122,112 +224,98 @@ Benchmark.items(metrics: %w(ips memsize), memory: 3, warmup: 1, time: 1, quiet:
     x.report("?split:[]")  { STRING ? STRING.split(DELIMITER) : [] }
   end
-  # partition the data by ruby version and data present
-  # that way we're not comparing a split on a nil vs a split on a populated string
   x.compare_by :version, :data
-if ENV["CONDENSED"].to_s == "true"
-  x.report_with grouping: :version, sort: true, row: :method, column: [:data, :metric]
-else
   x.report_with grouping: [:version, :metric], sort: true, row: :method, column: :data
-end
-  x.save_file $PROGRAM_NAME.sub(/\.rb$/, '.json')
+  x.save_file "split_results.json"
 end
 ```
 #### `save_file`
-Creates a json save file which saves the timings across multiple runs.
-This is used along with the `version` metadata to record different results per ruby version.
-Another common use is to record ActiveRecord version or a gem's version.
-This is run with two different versions of ruby.
-Interim values are stored in the save_file.
+Creates a JSON file that persists results across runs. Run once with Ruby 3.3, then again
+with Ruby 3.4 — the file accumulates both. Another common use is recording `ActiveRecord.version`
+to compare across gem versions.
-Running with environment variable `FORCE` will force running this again. (i.e.: `force: ENV["force"] == true`)
-Depending upon the environment variable `CONDENSED`, there are two types of output.
+Running with `force: true` will re-run and overwrite previously saved data for the same metadata.
 #### `compare_by`
-We introduce `version` as metadata for the tests. Adding `version` to the comparison
-says that we should only compare values for the same version of ruby (along with the same data).
-If you note `version` and `data` are not reserved words, instead, they are just what metadata we
-decided to pass in.
+Adding `:version` to `compare_by` means Ruby 3.3 results are only compared against other 3.3
+results, and 3.4 against 3.4. Note that `version` and `data` are not reserved words — they are
+just the metadata keys we chose.
-### Example 2 output
+#### Output — one table per version and metric
-#### [:version, :metric] 2.3.7_ips
+##### [:version, :metric] 3.3.0_ips
- method     | nil                   | str
-------------|-----------------------|---------------
- `?split:[]`| 10146134.2 i/s        | 1284159.2 i/s
-`to_s.split`| 4232772.3 i/s - 2.40x | 1258665.8 i/s
-#### [:version, :metric] 2.3.7_memsize
- method     | nil                | str
-------------|--------------------|-------------
- `?split:[]`| 40.0 bytes         | 360.0 bytes
-`to_s.split`| 80.0 bytes - 2.00x | 360.0 bytes
-#### [:version, :metric] 2.4.6_ips
- method     | nil                   | str
+```
+     method |                   nil |           str
 ------------|-----------------------|---------------
- `?split:[]`| 10012873.4 i/s        | 1377320.5 i/s
-`to_s.split`| 4557456.3 i/s - 2.20x | 1350562.6 i/s
+ ?split:[]  | 10146134.2 i/s        | 1284159.2 i/s
+ to_s.split | 4232772.3 i/s - 2.40x | 1258665.8 i/s
+```
-#### [:version, :metric] 2.4.6_memsize
+##### [:version, :metric] 3.3.0_memsize
- method     | nil                | str
-------------|--------------------|-------------
- `?split:[]`| 40.0 bytes         | 360.0 bytes
-`to_s.split`| 80.0 bytes - 2.00x | 360.0 bytes
+```
+     method |                nil |         str
+------------|--------------------|--------------
+ ?split:[]  | 40.0 bytes         | 360.0 bytes
+ to_s.split | 80.0 bytes - 2.00x | 360.0 bytes
+```
-#### [:version, :metric] 2.5.5_ips
+##### [:version, :metric] 3.4.0_ips
- method     | nil                   | str
+```
+     method |                   nil |           str
 ------------|-----------------------|---------------
- `?split:[]`| 7168109.1 i/s         | 1357046.0 i/s
-`to_s.split`| 3779969.3 i/s - 1.90x | 1328072.4 i/s
-#### [:version, :metric] 2.5.5_memsize
+ ?split:[]  | 10012873.4 i/s        | 1377320.5 i/s
+ to_s.split | 4557456.3 i/s - 2.20x | 1350562.6 i/s
+```
- method     | nil                | str
-------------|--------------------|-------------
- `?split:[]`| 40.0 bytes         | 360.0 bytes
-`to_s.split`| 80.0 bytes - 2.00x | 360.0 bytes
+#### Condensed alternative
+To see everything in fewer tables, combine data and metric into columns and group by version only:
-### Example 2 CONDENSED output
+```ruby
+x.report_with grouping: :version, sort: true, row: :method, column: [:data, :metric]
+```
-running with `CONDENSED=true` calls with a different `report_with`
+##### version 3.3.0
-As you might notice, the number of objects created for the runs are the same.
-But for some reason, the `nil` split case is slower for ruby 2.5.5.
+```
+     method |             nil_ips |           str_ips |        nil_memsize |     str_memsize
+------------|---------------------|-------------------|--------------------|----------------
+ ?split:[]  | 10146134.2 i/s      | 1284159.2 i/s     | 40.0 bytes         | 360.0 bytes
+ to_s.split | 4232772.3 i/s - 2.40x | 1258665.8 i/s   | 80.0 bytes - 2.00x | 360.0 bytes
+```
-#### version 2.3.7
+As you might notice, the number of objects created for the runs are the same across versions.
+But the ips numbers may vary — letting you spot performance regressions.
- method     | nil_ips               | str_ips       | nil_memsize        | str_memsize
-------------|-----------------------|---------------|--------------------|-------------
- `?split:[]`| 10146134.2 i/s        | 1284159.2 i/s | 40.0 bytes         | 360.0 bytes
-`to_s.split`| 4232772.3 i/s - 2.40x | 1258665.8 i/s | 80.0 bytes - 2.00x | 360.0 bytes
+## Custom value formatting
-#### version 2.4.6
+Use the `value` parameter to customize cell display. This adds ANSI colors (green for best, red for worst):
- method     | nil_ips               | str_ips       | nil_memsize        | str_memsize
-------------|-----------------------|---------------|--------------------|-------------
- `?split:[]`| 10012873.4 i/s        | 1377320.5 i/s | 40.0 bytes         | 360.0 bytes
-`to_s.split`| 4557456.3 i/s - 2.20x | 1350562.6 i/s | 80.0 bytes - 2.00x | 360.0 bytes
+```ruby
+VALUE_TO_S = ->(m) { m.comp_short("\e[#{m.color}m#{m.central_tendency.round(1)} #{m.units}\e[0m") }
+x.report_with row: :method, column: :metric, value: VALUE_TO_S
+```
-#### version 2.5.5
+## Options
- method     | nil_ips               | str_ips       | nil_memsize        | str_memsize
-------------|-----------------------|---------------|--------------------|-------------
- `?split:[]`| 7168109.1 i/s         | 1357046.0 i/s | 40.0 bytes         | 360.0 bytes
-`to_s.split`| 3779969.3 i/s - 1.90x | 1328072.4 i/s | 80.0 bytes - 2.00x | 360.0 bytes
+```ruby
+Benchmark.items(
+  metrics: %w(ips memsize),  # which metrics to measure (default: %w(ips))
+  memory: 3,                 # number of memory profiling runs (default: 1)
+  warmup: 1,                 # IPS warmup seconds
+  time: 3,                   # IPS measurement seconds
+  quiet: false,              # suppress interim output
+  force: true,               # re-run even if saved data exists
+) do |x|
+  # ...
+end
+```
 ## Development

data/benchmark-sweet.gemspec CHANGED Viewed

@@ -11,11 +11,10 @@ Gem::Specification.new do |spec|
   spec.summary       = %q{Suite to run multiple benchmarks}
   spec.description   = <<-EOF
-  Benchmark Sweet is a suite to run multiple kinds of metrics.
-  It can be configured to run memory, sql query, and ips benchmarks on a common set of code.
-  This data can be collected across multiple runs of the code, to support multiple ruby or
-  gem versions.
-  This also generates more complex comparisons
+  Benchmark Sweet is a suite to run multiple kinds of metrics that allows for the generation
+  of complex comparisons. It can be configured to run memory, sql query, and ips benchmarks
+  on a common set of code. These data can be collected across multiple runs with numerous
+  supported ruby or gem versions.
 EOF
   spec.homepage      = "https://github.com/kbrock/benchmark-sweet"
@@ -25,8 +24,8 @@ EOF
   # to allow pushing to a single host or delete this section to allow pushing to any host.
   if spec.respond_to?(:metadata)
     spec.metadata["homepage_uri"] = spec.homepage
-    spec.metadata["source_code_uri"] = "http://github.com/kbrock/benchmark-sweet"
-    spec.metadata["changelog_uri"] = "http://github.com/kbrock/benchmark-sweet/CHANGELOG.md"
+    spec.metadata["source_code_uri"] = "https://github.com/kbrock/benchmark-sweet"
+    spec.metadata["changelog_uri"] = "https://github.com/kbrock/benchmark-sweet/blob/master/CHANGELOG.md"
   end
   # Specify which files should be added to the gem when it is released.
@@ -36,12 +35,11 @@ EOF
   end
   spec.require_paths = ["lib"]
-  spec.add_development_dependency "bundler", "~> 2.1.4"
   spec.add_development_dependency "rake", ">= 12.3.3"
   spec.add_development_dependency "rspec", "~> 3.0"
   spec.add_development_dependency "activerecord"
+  spec.add_development_dependency "sqlite3"
-  spec.add_runtime_dependency "benchmark-ips", "~> 2.8.2"
-  spec.add_runtime_dependency "memory_profiler", "~> 0.9.0"
-  spec.add_runtime_dependency "activesupport"         # for {}.symbolize_keys! - want to remove
+  spec.add_runtime_dependency "benchmark-ips", "~> 2.14"
+  spec.add_runtime_dependency "memory_profiler", "~> 1.1"
 end

data/examples/benchmark_simple_database.rb CHANGED Viewed

@@ -7,7 +7,8 @@ require "active_record"
 # User.all.to_a.first | 977.1 i/s - 3.26x | 1.0 objs | 100.0 objs - 100.00x
-ActiveRecord::Base.establish_connection(ENV.fetch('DATABASE_URL') { "postgres://localhost/user_benchmark" })
+#ActiveRecord::Base.establish_connection(ENV.fetch('DATABASE_URL') { "postgres://localhost/user_benchmark" })
+ActiveRecord::Base.establish_connection(ENV.fetch('DATABASE_URL') { "sqlite3::memory:" })
 ActiveRecord::Migration.verbose = false
 ActiveRecord::Schema.define do
@@ -15,19 +16,34 @@ ActiveRecord::Schema.define do
     t.string :name
   end
   #add_index :users, :name
+  create_table :dogs, force: true do |t|
+    t.string :name
+  end
 end
 class User < ActiveRecord::Base ;end
+class Dog < ActiveRecord::Base ;end
 if User.count == 0
   puts "Creating 100 users"
   100.times { |i| User.create name: "user #{i}" }
+  puts "Creating 1 dog"
+  Dog.create name: "the dog"
 end
 Benchmark.items(metrics: %w(ips queries rows)) do |x|
-  x.report("User.all.first")
-  x.report("User.all.to_a.first")
+  x.metadata version: ActiveRecord.version.to_s
+  x.report(model: "User", method: "all.first") { User.all.first }
+  x.report(model: "User", method: "all.to_a.first") { User.all.to_a.first }
+  x.report(model: "Dog", method: "all.first") { Dog.all.first }
+  x.report(model: "Dog", method: "all.to_a.first") { Dog.all.to_a.first }
-  x.report_with row: :method, column: :metric
+  # metric names across the top (column)
+  # method names down the side (rows)
+  x.report_with row: [:method], column: [:metric], grouping: :model, sort: true
+  # pass SAVE_FILE=true, to save results to a file (true chooses name, or pass actual file name)
   x.save_file (ENV["SAVE_FILE"] == "true") ? $0.sub(/\.rb$/, '.json') : ENV["SAVE_FILE"] if ENV["SAVE_FILE"]
 end

data/lib/benchmark/sweet/comparison.rb CHANGED Viewed

@@ -46,7 +46,7 @@ module Benchmark
           stats.overlaps?(@worst)
         else
           slowdown == Float::INFINITY || (total.to_i - 1 == offset.to_i && slowdown.to_i > 1)
-         end
+        end
       end
       def slowdown

data/lib/benchmark/sweet/job.rb CHANGED Viewed

@@ -37,7 +37,7 @@ module Benchmark
       def initialize(options = {})
         @options = options
-        @options[:metrics] ||= IPS_METRICS + %w()
+        @options[:metrics] ||= IPS_METRICS.dup
         validate_metrics(@options[:metrics])
         @items = []
         @entries = {}
@@ -57,26 +57,26 @@ module Benchmark
       end
       # @returns [Boolean] true to run iterations per second tests
-      def ips? ; !(relevant_metric_names & IPS_METRICS).empty? ; end
+      def ips? ; (relevant_metric_names & IPS_METRICS).any?; end
       # @returns [Boolean] true to run memory tests
-      def memory? ; !(relevant_metric_names & MEMORY_METRICS).empty? ; end
+      def memory? ; (relevant_metric_names & MEMORY_METRICS).any?; end
       # @returns [Boolean] true to run database queries tests
-      def database? ; !(relevant_metric_names & DATABASE_METRICS).empty? ; end
+      def database? ; (relevant_metric_names & DATABASE_METRICS).any?; end
       # @returns  [Boolean] true to suppress the display of interim test calculations
-      def quiet? ; options[:quiet] ; end
+      def quiet?; options[:quiet]; end
       # @returns  [Boolean] true to run tests for data that has already been processed
-      def force? ; options[:force] ; end
+      def force?; options[:force]; end
       # @returns [Array<String>] List of metrics to compare
-      def relevant_metric_names ; options[:metrics] ; end
+      def relevant_metric_names; options[:metrics]; end
       # items to run (typical benchmark/benchmark-ips use case)
       def item(label, action = nil, &block)
         # could use Benchmark::IPS::Job::Entry
         current_meta = label.kind_of?(Hash) ? @meta.merge(label) : @meta.merge(method: label)
-        @items << Item.new(current_meta, action || block)
+        @items << Item.new(normalize_label(current_meta), action || block)
       end
       alias report item
@@ -102,7 +102,7 @@ module Benchmark
       #   x.compare_by :data
       #
       def compare_by(*symbol, &block)
-        @grouping = symbol.empty? ? block : Proc.new { |label, value| symbol.map { |s| label[s] } }
+        @grouping = symbol.empty? ? block : Proc.new { |label, _value| symbol.map { |s| label[s] } }
       end
       # Setup the testing framework
@@ -123,10 +123,6 @@ module Benchmark
         end
       end
-      # if we are using symbols as keys for our labels
-      def labels_have_symbols!
-      end
       # report results
       def add_entry(label, metric, stat)
         (@entries[metric] ||= {})[label] = stat.respond_to?(:central_tendency) ? stat : create_stats(stat)
@@ -147,8 +143,7 @@ module Benchmark
         require "json"
         JSON.load(IO.read(filename)).each do |v|
-          n = v["name"]
-          n.symbolize_keys!
+          n = normalize_label(v["name"].transform_keys(&:to_sym))
           add_entry n, v["metric"], v["samples"]
         end
@@ -158,23 +153,16 @@ module Benchmark
         return unless filename
         require "json"
-        # sanity checking
-        symbol_value = false
         data = @entries.flat_map do |metric_name, metric_values|
           metric_values.map do |label, stat|
-            # warnings
-            symbol_values ||= label.kind_of?(Hash) && label.values.detect { |v| v.nil? || v.kind_of?(Symbol) }
             {
               'name'    => label,
               'metric'  => metric_name,
               'samples' => stat.samples,
-              # extra data like measured_us, iter, and others?
             }
           end
         end
-        puts "", "Warning: Please use strings or numbers for label hash values (not nils or symbols). Symbols are not JSON friendly." if symbol_value
         IO.write(filename, JSON.pretty_generate(data) << "\n")
       end
@@ -213,8 +201,8 @@ module Benchmark
         relevant_entries.flat_map do |metric_name, metric_entries|
           # TODO: map these to Comparison(metric_name, label, stats) So we only have 1 type of lambda
           partitioned_metrics = grouping ? metric_entries.group_by(&grouping) : {nil => metric_entries}
-          partitioned_metrics.flat_map do |grouping_name, grouped_metrics|
-            sorted = grouped_metrics.sort_by { |n, e| e.central_tendency }
+          partitioned_metrics.flat_map do |_grouping_name, grouped_metrics|
+            sorted = grouped_metrics.sort_by { |_n, e| e.central_tendency }
             sorted.reverse! if HIGHER_BETTER.include?(metric_name)
             _best_label, best_stats = sorted.first
@@ -236,6 +224,11 @@ module Benchmark
         end
       end
+      # Normalize label hash values to strings so labels match after JSON round-trip
+      def normalize_label(label)
+        label.transform_values(&:to_s)
+      end
       def create_stats(samples)
         Benchmark::IPS::Stats::SD.new(Array(samples))
       end

data/lib/benchmark/sweet/memory.rb CHANGED Viewed

@@ -17,7 +17,7 @@ module Benchmark
         #   add_entry e.label, "string_retained",  e.measurement.string.retained
         # end
         require 'memory_profiler'
-        puts "Memory Profiling----------" unless quiet?
+        $stdout.puts "Memory Profiling----------" unless quiet?
         items.each do |entry|
           name = entry.label
@@ -26,8 +26,8 @@ module Benchmark
           rpts = (options[:memory] || 1).times.map { MemoryProfiler.report(&entry.block) }
           tot_stat  = add_entry(name, "memsize",          rpts.map(&:total_allocated_memsize))
           totr_stat = add_entry name, "memsize_retained", rpts.map(&:total_retained_memsize)
-          obj_stat  = add_entry name, "objects",          rpts.map(&:total_allocated) ## ? size
-          objr_stat = add_entry name, "objects_retained", rpts.map(&:total_retained)
+          add_entry name, "objects",          rpts.map(&:total_allocated)
+          add_entry name, "objects_retained", rpts.map(&:total_retained)
           str_stat  = add_entry(name, "strings",          rpts.map { |rpt| rpt.strings_allocated.size })
           strr_stat = add_entry(name, "strings_retained", rpts.map { |rpt| rpt.strings_retained.size })

data/lib/benchmark/sweet/queries.rb CHANGED Viewed

@@ -2,16 +2,20 @@ module Benchmark
   module Sweet
     module Queries
       def run_queries
-        items.each do |entry|
-          values = ::Benchmark::Sweet::Queries::QueryCounter.count(&entry.block) # { entry.call_times(1) }
-          add_entry entry.label, "rows",    values[:instance_count]
-          add_entry entry.label, "queries", values[:sql_count]
-          add_entry entry.label, "ignored", values[:ignored_count]
-          add_entry entry.label, "cached",  values[:cache_count]
-          unless options[:quiet]
-            printf "%20s: %3d queries %5d ar_objects", entry.label, values[:sql_count], values[:instance_count]
-            printf " (%d ignored)", values[:ignored_count] if values[:ignored_count] > 0
-            puts
+        cntr = ::Benchmark::Sweet::Queries::QueryCounter.new
+        cntr.sub do
+          items.each do |entry|
+            entry.block.call
+            values = cntr.get_clear
+            add_entry entry.label, "rows",    values[:instance_count]
+            add_entry entry.label, "queries", values[:sql_count]
+            add_entry entry.label, "ignored", values[:ignored_count]
+            add_entry entry.label, "cached",  values[:cache_count]
+            unless options[:quiet]
+              printf "%20s: %3d queries %5d ar_objects", entry.label, values[:sql_count], values[:instance_count]
+              printf " (%d ignored)", values[:ignored_count] if values[:ignored_count] > 0
+              puts
+            end
           end
         end
       end
@@ -30,10 +34,14 @@ module Benchmark
         IGNORED_STATEMENTS = %w(CACHE SCHEMA).freeze
         IGNORED_QUERIES    = /^(?:ROLLBACK|BEGIN|COMMIT|SAVEPOINT|RELEASE)/.freeze
+        def initialize
+          clear
+        end
         def callback(_name, _start, _finish, _id, payload)
           if payload[:sql]
             if payload[:name] == CACHE_STATEMENT
-              @instance[:cache_count] += 1
+              @instances[:cache_count] += 1
             elsif IGNORED_STATEMENTS.include?(payload[:name]) || IGNORED_QUERIES.match(payload[:sql])
               @instances[:ignored_count] += 1
             else
@@ -48,12 +56,25 @@ module Benchmark
           lambda(&method(:callback))
         end
-        # TODO: possibly setup a single subscribe and use a context/thread local to properly count metrics
-        def count(&block)
+        def clear
           @instances = {cache_count: 0, ignored_count: 0, sql_count: 0, instance_count: 0}
-          ActiveSupport::Notifications.subscribed(callback_proc, /active_record/, &block)
+        end
+        def get_clear; @instances.tap { clear }; end
+        def get; @instances; end
+        # either use 10.times { value = count(&block) }
+        # or use
+        # sub { 10.times { block.call; value = get_clear } }
+        def count(&block)
+          clear
+          sub(&block)
           @instances
         end
+        def sub(&block)
+          ActiveSupport::Notifications.subscribed(callback_proc, /active_record/, &block)
+        end
       end
     end
   end

data/lib/benchmark/sweet/version.rb CHANGED Viewed

@@ -1,5 +1,5 @@
 module Benchmark
   module Sweet
-    VERSION = "0.2.2"
+    VERSION = "0.3.0"
   end
 end

data/lib/benchmark/sweet.rb CHANGED Viewed

@@ -40,7 +40,7 @@ module Benchmark
       grouping = symbol_to_proc(grouping)
-      label_records = base.group_by(&grouping).select { |value, comparisons| !value.nil? }
+      label_records = base.group_by(&grouping).select { |value, _comparisons| !value.nil? }
       label_records = label_records.sort_by(&:first) if sort
       label_records.each(&block)
@@ -89,22 +89,30 @@ module Benchmark
     end
     def self.to_table(arr)
-      field_sizes = Hash.new
-      arr.each { |row| field_sizes.merge!(row => row.map { |iterand| iterand[1].to_s.gsub(/\e\[[^m]+m/, '').length } ) }
+      return if arr.empty?
-      column_sizes = arr.reduce([]) do |lengths, row|
-        row.each_with_index.map { |iterand, index| [lengths[index] || 0, field_sizes[row][index]].max }
-      end
+      strip_ansi = ->(s) { s.to_s.gsub(/\e\[[^m]*m/, '') }
+      # collect headers from the first row's keys
+      headers = arr[0].keys
-      format = column_sizes.collect {|n| "%#{n}s" }.join(" | ")
-      format += "\n"
+      # compute visible width for each column (max of header and all values)
+      column_sizes = headers.each_with_index.map do |header, i|
+        values_max = arr.map { |row| strip_ansi.call(row.values[i]).length }.max
+        [strip_ansi.call(header).length, values_max].max
+      end
-      printf format, *arr[0].each_with_index.map { |el, i| " "*(column_sizes[i] - field_sizes[arr[0]][i] ) + el[0].to_s }
+      # right-align with ANSI-aware padding
+      pad = ->(str, width) {
+        visible = strip_ansi.call(str).length
+        " " * [width - visible, 0].max + str.to_s
+      }
-      printf format, *column_sizes.collect { |w| "-" * w }
+      puts headers.each_with_index.map { |h, i| pad.call(h.to_s, column_sizes[i]) }.join(" | ")
+      puts column_sizes.map { |w| "-" * w }.join("-|-")
-      arr[0..arr.count].each do |line|
-        printf format, *line.each_with_index.map { |el, i| " "*(column_sizes[i] - field_sizes[line][i] ) + el[1].to_s }
+      arr.each do |row|
+        puts row.values.each_with_index.map { |v, i| pad.call(v.to_s, column_sizes[i]) }.join(" | ")
       end
     end
   end

metadata CHANGED Viewed

@@ -1,29 +1,14 @@
 --- !ruby/object:Gem::Specification
 name: benchmark-sweet
 version: !ruby/object:Gem::Version
-  version: 0.2.2
+  version: 0.3.0
 platform: ruby
 authors:
 - Keenan Brock
-autorequire:
 bindir: bin
 cert_chain: []
-date: 2020-07-15 00:00:00.000000000 Z
+date: 1980-01-02 00:00:00.000000000 Z
 dependencies:
-- !ruby/object:Gem::Dependency
-  name: bundler
-  requirement: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: 2.1.4
-  type: :development
-  prerelease: false
-  version_requirements: !ruby/object:Gem::Requirement
-    requirements:
-    - - "~>"
-      - !ruby/object:Gem::Version
-        version: 2.1.4
 - !ruby/object:Gem::Dependency
   name: rake
   requirement: !ruby/object:Gem::Requirement
@@ -67,53 +52,52 @@ dependencies:
       - !ruby/object:Gem::Version
         version: '0'
 - !ruby/object:Gem::Dependency
-  name: benchmark-ips
+  name: sqlite3
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 2.8.2
-  type: :runtime
+        version: '0'
+  type: :development
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - "~>"
+    - - ">="
       - !ruby/object:Gem::Version
-        version: 2.8.2
+        version: '0'
 - !ruby/object:Gem::Dependency
-  name: memory_profiler
+  name: benchmark-ips
   requirement: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.9.0
+        version: '2.14'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
     - - "~>"
       - !ruby/object:Gem::Version
-        version: 0.9.0
+        version: '2.14'
 - !ruby/object:Gem::Dependency
-  name: activesupport
+  name: memory_profiler
   requirement: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '1.1'
   type: :runtime
   prerelease: false
   version_requirements: !ruby/object:Gem::Requirement
     requirements:
-    - - ">="
+    - - "~>"
       - !ruby/object:Gem::Version
-        version: '0'
+        version: '1.1'
 description: |2
-    Benchmark Sweet is a suite to run multiple kinds of metrics.
-    It can be configured to run memory, sql query, and ips benchmarks on a common set of code.
-    This data can be collected across multiple runs of the code, to support multiple ruby or
-    gem versions.
-    This also generates more complex comparisons
+    Benchmark Sweet is a suite to run multiple kinds of metrics that allows for the generation
+    of complex comparisons. It can be configured to run memory, sql query, and ips benchmarks
+    on a common set of code. These data can be collected across multiple runs with numerous
+    supported ruby or gem versions.
 email:
 - keenan@thebrocks.net
 executables: []
@@ -122,7 +106,6 @@ extra_rdoc_files: []
 files:
 - ".gitignore"
 - ".rspec"
-- ".travis.yml"
 - CHANGELOG.md
 - Gemfile
 - LICENSE.txt
@@ -150,9 +133,8 @@ licenses:
 - MIT
 metadata:
   homepage_uri: https://github.com/kbrock/benchmark-sweet
-  source_code_uri: http://github.com/kbrock/benchmark-sweet
-  changelog_uri: http://github.com/kbrock/benchmark-sweet/CHANGELOG.md
-post_install_message:
+  source_code_uri: https://github.com/kbrock/benchmark-sweet
+  changelog_uri: https://github.com/kbrock/benchmark-sweet/blob/master/CHANGELOG.md
 rdoc_options: []
 require_paths:
 - lib
@@ -167,9 +149,7 @@ required_rubygems_version: !ruby/object:Gem::Requirement
     - !ruby/object:Gem::Version
       version: '0'
 requirements: []
-rubyforge_project:
-rubygems_version: 2.7.6.2
-signing_key:
+rubygems_version: 4.0.6
 specification_version: 4
 summary: Suite to run multiple benchmarks
 test_files: []

data/.travis.yml DELETED Viewed

@@ -1,7 +0,0 @@
----
-sudo: false
-language: ruby
-cache: bundler
-rvm:
-  - 2.4.4
-before_install: gem install bundler -v 1.16.6