cumo 0.5.8 → 0.5.10

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 12027799f8203c1e08409de1fdec1b550e26511547bbaee2a938488ecd19e079
-  data.tar.gz: 399daea2a183febf70ceeb1e6492479fd3a34ae29e994265c4da9f89644c3c4d
+  metadata.gz: bcb51638cb488e5abcb0bf885744ea1478f4daefd4945f6af8becf252bc85f29
+  data.tar.gz: 75aed5ea382f37a823992e29ef2f3fe9ee31a7b7c11350d2541e5f4fc0c0fb59
 SHA512:
-  metadata.gz: d12b0fa56578c8ffd309aa314ab40c98d3b983375d7ddd33b0909ac47092d52b48ea1b53173ea843701587a0fb50442a907b459bd558a8c92e4525eedd1531fd
-  data.tar.gz: ba1b2bac6afb811d790c3f8d90906f3e7f571cbbdb8dc59e3ff6141fa54aa5b8f8a8d6e597ff3045e001c9ae5ff0cab6d9e6679ee6b8f5bbdb1642f203ba19b1
+  metadata.gz: ee03167a0966008c99598d90623757772060c09bad859af35d313dda831975461cba49c8da251a4793e6a9b933cd5ec75588038381d55e492398d0b4e2686559
+  data.tar.gz: 2e2b3e1a6f0f304be8b2375be48061ba73926973d4ec966e49d0b50f666d88d7078cec3c59ea797401bd72b91ec96a4f115bf2ed49485dc1d7ca3b6a29fb1c48

data/3rd_party/mkmf-cu/lib/mkmf-cu/cli.rb

@@ -40,7 +40,9 @@ module MakeMakefileCuda
       s = MakeMakefileCuda::Nvcc.generate(argv)
       cmd = "nvcc #{s}"
       if ENV['CUMO_NVCC_GENERATE_CODE']
-        cmd << " --generate-code=#{ENV['CUMO_NVCC_GENERATE_CODE']}"
+        ENV['CUMO_NVCC_GENERATE_CODE'].split.each do |code|
+          cmd << " --generate-code=#{code}"
+        end
       else
         capability = nil
         if find_executable('nvidia-smi')

data/CHANGELOG.md

@@ -1,3 +1,79 @@
+# 0.5.10 (2026/08/23)
+
+Breaking changes:
+
+* An indexed assignment to a frozen view raises instead of writing through it; `store` and `fill` already raised (PR #306)
+* The cuDNN and cuBLAS entry points turn away arguments they used to read or write out of bounds: `pooling_backward` requires `y` and `gy` to carry its own dtype and the pooling output shape of `x`, `gemm` requires a contiguous `c`, `batch_norm` requires the reduced size to cover x's channels, and `conv_grad_w` raises on a mismatched `gy` rather than asserting (PR #303, PR #302, PR #300, PR #278)
+* `seq` on the unsigned types wraps a negative start and a start past 2**63 the way `fill` and `cast` do, where they collapsed to zero and clamped to INT64_MAX (PR #295, PR #294)
+* A negative power of an unsigned array answers zero instead of spinning the device until SIGKILL (PR #293)
+* A subscript that is a Cumo::RObject subclass, and an `expand_dims` past `CUMO_NA_MAX_DIMENSION`, raise instead of being taken (PR #272, PR #270)
+
+Fixes:
+
+* Fix `[]=` writing through a frozen view, the store going to a derived view whose data object is the root rather than the receiver (PR #306)
+* Fix `at()` reading the accumulator in place of the subscript on a reversed view, which addressed far in front of the buffer (PR #305)
+* Fix `cumo_cuda_cudnn_CreateBNTensorDescriptor` returning before it derived the descriptor, an inverted status check that no caller reached (PR #304)
+* Fix `batch_norm`, `fixed_batch_norm` and `batch_norm_backward` reading and writing past their parameters, the sizes being checked against the reduced shape while cuDNN reaches x's channel count (PR #303)
+* Fix `gemm` writing past its allocation when `c` is a non-contiguous inplace view, and over whatever the pool put next to it (PR #302)
+* Fix `pooling_backward` reading past `y` and `gy`, and returning what it read in `gx` (PR #300)
+* Fix `pow_int` negating INT32_MIN, which overflows and leaves the loop unbounded (PR #299)
+* Fix the device free running for a subscript that allocated no index array (PR #297)
+* Fix a NULL dereference from `at()` given a scalar subscript (PR #271)
+* Fix an out-of-bounds write through an unchecked cuDNN output array (PR #269)
+* Fix `gemm` reading past an operand that carries fewer batch dimensions (PR #268)
+
+Changes:
+
+* Pass a numeric operand to the kernel instead of casting it to a 0-dimensional array first, in the operators, `clip`, `pow`, the comparisons, the NMath functions and the coerced left-hand side; GPT-2 124M decode drops from 787 to 677 kernel launches a token (PR #296, PR #291, PR #290, PR #289, PR #288, PR #287)
+* Run the NMath binary functions through the indexer loop instead of one kernel launch per row; 11x on a transposed operand and up to 60x on the cheap kernels (PR #292)
+* Copy a reduction operand only when it carries an index array, where any non-contiguous view was copied whole (PR #286)
+* Run the nan-aware reductions, the nan-aware index reductions, `kahan_sum` and `Bit#mask` on the GPU (PR #277, PR #276, PR #275, PR #273)
+* Take a transposed batch of matrices with the cuBLAS transpose flag rather than duplicating it, which saves a temporary the size of the operand (PR #283)
+* Emit indexer accessors up to eight dimensions (PR #284)
+* Drop a free-list bin from the pool arena when `Malloc` empties it (PR #279)
+* Check the cuDNN header and version, not just the library (PR #280)
+* Document every method that returns a 0-dimensional NArray, and what keeping scalars on the device buys (PR #298, PR #282)
+* Build two GPU architectures in CI instead of every one (PR #285)
+* Add `fiddle` to the Gemfile (PR #281)
+* Remove TODO comments whose questions have been answered (PR #274)
+* Cover the Int32 array exponent in the INT_MIN power test (PR #301)
+
+# 0.5.9 (2026/08/20)
+
+Breaking changes:
+
+* `sort_index` resolves ties to the lowest index, the GPU sort being stable, where the host quicksort left them in whatever order it produced (PR #261)
+
+Fixes:
+
+* Fix `a[[2, 1], true].store(array)` writing every row onto row 0, on every dtype: ndloop read the index array on the host before the kernel that fills it had run (PR #265)
+* Fix a segmentation fault storing a Ruby Array into a reversed `Cumo::Bit` view that starts past the first word (PR #266)
+* Fix `sort` in place of a view backed by an index array reading row 0 for every row (PR #262)
+* Fix a `Cumo::RObject` store reading an index array before the kernel writes it, which answered zeroes from the second store onwards (PR #258)
+* Fix a store of an Array of narrays into an indexed destination writing one element per row (PR #257)
+* Fix the zero fill of a row stored from a shorter narray starting one element too far (PR #256)
+* Fix grid-stride kernels hanging from about 2**32 elements, the step wrapping to zero (PR #240)
+* Fix `Cumo::Bit` views with a negative step reading out of bounds (PR #238)
+* Fix index reductions answering a later index on ties, where `Numo::NArray` answers the first (PR #236)
+* Fix wrong values from reductions with `nan: true` (PR #234)
+
+Changes:
+
+* Run `sort`, `sort_index` and `median` on the GPU with `cub::DeviceSegmentedRadixSort`; 1M `SFloat` `sort` 115.7 -> 0.47 ms, `median` 107.8 -> 0.13 ms, `sort_index` 115.8 -> 1.69 ms (PR #259, PR #260, PR #261)
+* Run `poly` on the GPU, which had called its iterator once per element behind a synchronization; 2^20 `DFloat` 163.8 -> 0.021 ms (PR #263)
+* Run `bincount` on the GPU; 1M `Int32` 1.62 -> 0.05 ms (PR #255)
+* Run `cumsum` and `cumprod` on the GPU (PR #242)
+* Run `minmax`, `abs`, `isnan` and the rest of `cond_unary`, `modf` and `frexp` on the GPU (PR #230, PR #231, PR #232, PR #233)
+* Run the `Cumo::Bit` operators, reductions, `where`, `where2`, `fill` and the stores on the GPU (PR #237, PR #239, PR #241, PR #252, PR #254, PR #264, PR #266)
+* Run complex `real=` and `imag=` on the GPU; 2^22 `DComplex` 4.63 -> 0.45 ms (PR #267)
+* Run the accumulating reductions and `mulsum` on more than one GPU thread, where they had been running device-side thrust in a single thread (PR #243, PR #244)
+* Address a whole non-contiguous view in one kernel launch instead of one per row, in the elementwise templates, the stores, the copies and the Bit operands (PR #245, PR #246, PR #249, PR #251, PR #252)
+* Read a flat reduction operand without the indexer, and reduce over an outer axis with host-side addressing; `sum(axis: 0)` on 4096x4096 2.01 -> 0.017 ms (PR #248, PR #253)
+* Split a reduction with too few outputs across more blocks (PR #235)
+* Build the C++ and CUDA sources with optimization: mkmf leaves `$(optflags)` out of `CXXFLAGS`, so every `.cpp` and the host half of every `.cu` had been built at `-O0`; `a + b` 2.86 -> 2.00 us (PR #250)
+* Warn about the synchronization `a[idx]` and `inspect` perform (PR #247)
+* Add benchmark scripts under `bench/` for a transformer block, k-means, a 2D Ising model, conjugate gradient, a particle simulation and the CPU/GPU crossover, and two probes for calls that synchronize or launch one kernel per row
+
 # 0.5.8 (2026/08/17)
 
 Breaking changes:

data/Gemfile

@@ -5,6 +5,7 @@ source "https://rubygems.org"
 gemspec
 
 gem 'extconf_compile_commands_json'
+gem 'fiddle'
 gem 'rake-compiler'
 gem 'test-unit'
 gem 'yard'

data/README.md

@@ -9,6 +9,7 @@ Cumo (pronounced "koomo") is a CUDA-aware, GPU-optimized numerical library that
 * Ruby 3.0 or later
 * NVIDIA GPU Compute Capability 3.5 (Kepler) or later
 * CUDA 11.0 or later
+* cuDNN 8.0 or later (optional, for the cuDNN features)
 
 ## Preparation
 
@@ -97,17 +98,31 @@ a = xm::DFloat.new(3,5).seq
 
 ### Incompatibility With Numo
 
-The following methods behave incompatibly with Numo by default for performance reasons:
+Numo returns a Ruby numeric object wherever a result is 0-dimensional, while Cumo returns the 0-dimensional NArray itself.
+Cumo differs in this way to avoid synchronization and minimize CPU ⇄ GPU data transfer.
+That is not only a cost of the port; see [Keeping Scalars On The Device](#keeping-scalars-on-the-device) for what it buys.
 
-* `extract`
-* `[]`
-* `count_true`
-* `count_false`
+The methods affected are:
 
-Numo returns a Ruby numeric object for 0-dimensional NArray, while Cumo returns the 0-dimensional NArray instead of a Ruby numeric object.
-Cumo differs in this way to avoid synchronization and minimize CPU ⇄ GPU data transfer.
+* `[]` and `extract`
+* `count_true` and `count_false`
+* reductions down to a single value: `sum`, `prod`, `mean`, `stddev`, `var`, `rms`, `min`, `max`, `ptp`, `minmax`, `median`, `mulsum`, `dot`, `inner`
+* index reductions: `max_index`, `min_index`, `argmax`, `argmin`
+
+A 0-dimensional `Cumo::Bit` is truthy even when it holds 0, because Ruby treats every object but `nil` and `false` as true.
+Comparing two scalars therefore takes the wrong branch without raising anything:
+
+```ruby
+a = Cumo::SFloat[5.0]
+a[0] < 1.0                  #=> Cumo::Bit#shape=[] holding 0
+(a[0] < 1.0) ? :yes : :no   #=> :yes, where Numo gives :no
+```
+
+`assert_operator(a[0], :<, 1.0)` passes for the same reason, so a test suite written for Numo can stay green against Cumo while asserting nothing.
+Read the value back to the host before branching on it, or run under `compatible_mode`.
 
 Set the `CUMO_COMPATIBLE_MODE` environment variable to `ON` to force Numo NArray compatibility (for worse performance).
+Running a Numo test suite that way keeps its assertions meaningful.
 
 You may enable or disable `compatible_mode` as:
 
@@ -126,6 +141,43 @@ You can also use the following methods which behave like Numo's NArray methods.
 * `count_true_cpu`
 * `count_false_cpu`
 
+```ruby
+a.aref_cpu(0) < 1.0   #=> false in either mode
+Float(a.sum)          #=> 7.0 in either mode
+```
+
+They are methods on an NArray, so chaining one onto a result that `compatible_mode` has already turned into a Ruby object, as in `a.sum.extract_cpu`, raises `NoMethodError` while the mode is on.
+`Kernel#Float` and `Kernel#Integer` read either representation, and read Numo's too, so they are what code that runs against both libraries wants.
+
+### Keeping Scalars On The Device
+
+The 0-dimensional return is what lets an iterative loop stay on the GPU.
+Reading a scalar back to the host waits for everything queued behind it, so every read caps how far ahead the GPU is allowed to run.
+What a read costs is not a fixed price either: it is however much work happens to be queued when it is taken.
+
+`bench/cg_bench.rb` prices this with a conjugate gradient solve, 200 iterations over a 512x512 grid on an RTX 5070 Ti Laptop:
+
+```
+scalars        convergence test    us/iter    readbacks/iter
+Ruby Floats    every iteration       136.7        2.02
+Ruby Floats    never                 134.4        2.02
+0-dim NArray   every iteration       142.6        1.02
+0-dim NArray   every 20th             65.6        0.06
+0-dim NArray   never                  60.3        0.02
+```
+
+Written with Ruby Floats the loop reads back twice an iteration whatever the convergence test does, since `alpha` needs `pap` and `beta` needs `rs_new` as Floats.
+Thinning the test cannot get under that floor, and keeping the scalars as 0-dimensional NArrays buys nothing on its own.
+The two only pay together, and together they are worth 2.1x.
+The relative residual is identical in every row.
+
+```ruby
+alpha = rs_old / pap   # a 0-dimensional NArray, divided on the device
+x += p_dir * alpha     # and consumed there, without crossing the bus
+```
+
+Read the value back once the loop is done, or every k iterations if it has to test something.
+
 ### Select a GPU device ID
 
 Set the `CUDA_VISIBLE_DEVICES=id` environment variable, or
@@ -201,7 +253,14 @@ ln -sf "$HOME/opt/ccache/bin/ccache" "$HOME/opt/ccache/bin/nvcc"
 bundle exec env CUMO_NVCC_GENERATE_CODE=arch=compute_60,code=sm_60 rake compile
 ```
 
+Separate the entries with a space to build for more than one architecture:
+
+```
+bundle exec env CUMO_NVCC_GENERATE_CODE="arch=compute_75,code=sm_75 arch=compute_121,code=sm_121" rake compile
+```
+
 This is useful even on development because it makes it possible to skip JIT compilation of PTX to cubin during runtime.
+Without it, and without an `nvidia-smi` to read the local compute capability from, the build covers every architecture the CUDA version supports.
 
 ### Run tests with gdb
 

data/bench/cg_bench.rb

@@ -0,0 +1,271 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# ---------------------------------------------------------------------------
+# Conjugate gradient, to price what a host readback costs.
+#
+#   ruby cg_bench.rb
+#   GPU=1 ruby cg_bench.rb                        # the plain way: readbacks every iteration
+#   GPU=1 SCALAR=device ruby cg_bench.rb          # keep alpha and beta on the device
+#   GPU=1 SCALAR=device CHECK=20 ruby cg_bench.rb # and test convergence rarely
+#   GPU=1 OP=dense N=4096 ruby cg_bench.rb        # heavier arithmetic per iteration
+#
+# CG makes two scalars an iteration, alpha and beta, and multiplies arrays by
+# them. Written plainly that reads them back to the host every iteration, and
+# the convergence test reads a third, each one waiting for the queue to drain.
+# An iteration is not much arithmetic, so the waiting is what shows.
+#
+#   SCALAR=host   alpha and beta become Ruby Floats, as one would write it
+#   SCALAR=device they stay as the 0-dimensional NArray sum answers, and are
+#                 never read (Cumo returns 0-dim; on Numo sum is a Float
+#                 already, so the setting makes no difference there)
+#   CHECK=k       test convergence every k iterations, 0 to never test
+#
+# A readback does not cost a fixed amount. What it costs is however much work
+# is queued when it happens, so it is better read as a cap on how deep the
+# pipeline is allowed to get than as a price per call. Several shallow drains
+# an iteration cost little each; the last one costs a lot, because until it is
+# gone no iteration can overlap the next.
+#
+# On the host path the floor is two readbacks an iteration whatever CHECK is
+# set to, since alpha needs pap and beta needs rs_new as Floats. Keeping the
+# scalars on the device is what makes fewer than that possible, and thinning
+# the convergence test only pays once they are there.
+#
+# Correctness is checked against a known solution: x_true is chosen first and
+# b = A x_true built from it, so the answer has something to be compared to.
+#
+# Timings vary by a factor of a few between runs. Take the best of several.
+# ---------------------------------------------------------------------------
+
+GPU = !%w[0 false].include?(ENV['GPU'].to_s.downcase) && !ENV['GPU'].to_s.empty?
+
+if GPU
+  require 'cumo/narray'
+  XM = Cumo
+else
+  require 'numo/narray'
+  XM = Numo
+end
+
+OP      = (ENV['OP']      || 'stencil')  # stencil | dense
+GRID    = (ENV['GRID']    || 512).to_i   # side of the grid, for stencil
+N       = (ENV['N']       || 2048).to_i  # number of unknowns, for dense
+MAXITER = (ENV['MAXITER'] || 200).to_i
+SCALAR  = (ENV['SCALAR']  || 'host')     # host | device
+CHECK   = (ENV['CHECK']   || 1).to_i     # test convergence every CHECK, 0 to never
+TOL     = (ENV['TOL']     || 1.0e-6).to_f
+DOT     = (ENV['DOT']     || 'sum')      # sum | blas
+DETAIL  = ENV['DETAIL'] != '0'
+REPEAT  = (ENV['REPEAT']  || 3).to_i     # solve this many times, keep the fastest
+SEED    = (ENV['SEED'] || 42).to_i
+
+TIMES = Hash.new(0.0)
+READBACKS = [0]
+
+def sync
+  XM::CUDA::Runtime.cudaDeviceSynchronize if GPU
+end
+
+# A readback, which is where the host waits. Counted, since that is the point.
+def scalar(v)
+  READBACKS[0] += 1
+  return v if v.is_a?(Numeric)
+
+  v.respond_to?(:extract_cpu) ? v.extract_cpu : v.extract
+end
+
+def timed(key)
+  return yield unless DETAIL
+
+  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+  r = yield
+  sync
+  TIMES[key] += Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0
+  r
+end
+
+def dotp(a, b)
+  if DOT == 'blas'
+    a.reshape(a.size).dot(b.reshape(b.size))
+  else
+    (a * b).sum
+  end
+end
+
+# --- the operator ----------------------------------------------------------
+
+# The five-point Laplacian, negated to be positive definite. The border stays
+# zero and is never touched.
+def apply_stencil(p)
+  out = XM::SFloat.zeros(GRID, GRID)
+  out[1..-2, 1..-2] =
+    4.0 * p[1..-2, 1..-2] -
+    p[0..-3, 1..-2] - p[2..-1, 1..-2] -
+    p[1..-2, 0..-3] - p[1..-2, 2..-1]
+  out
+end
+
+# A diagonally dominant symmetric matrix. The identity is built with
+# arithmetic rather than a fancy index.
+def build_dense
+  r = XM::SFloat.new(N, N).rand * 2.0 - 1.0
+  sym = (r + r.transpose) * 0.5
+  i = XM::SFloat.new(N, 1).seq
+  j = XM::SFloat.new(1, N).seq
+  off = (XM::SFloat.zeros(N, N) + (i - j)).abs.clip(0.0, 1.0).ceil # 0 on the diagonal
+  sym + (1.0 - off) * (N * 1.0)
+end
+
+# --- setup -----------------------------------------------------------------
+
+version = begin
+  XM::NArray::VERSION
+rescue StandardError
+  'unknown'
+end
+
+begin
+  XM::NArray.srand(SEED)
+rescue StandardError
+  nil
+end
+
+if OP == 'dense'
+  amat = build_dense
+  x_true = XM::SFloat.new(N).rand * 2.0 - 1.0
+  apply = ->(p) { amat.dot(p) }
+  unknowns = N
+else
+  x_true = XM::SFloat.zeros(GRID, GRID)
+  x_true[1..-2, 1..-2] = XM::SFloat.new(GRID - 2, GRID - 2).rand * 2.0 - 1.0
+  apply = ->(p) { apply_stencil(p) }
+  unknowns = (GRID - 2) * (GRID - 2)
+end
+
+b = apply.call(x_true)
+
+puts "ruby      : #{RUBY_VERSION} (#{RUBY_PLATFORM})"
+puts "backend   : #{XM} #{version}"
+puts "problem   : OP=#{OP} unknowns=#{unknowns} MAXITER=#{MAXITER}"
+puts "scalar    : SCALAR=#{SCALAR} CHECK=#{CHECK} DOT=#{DOT}"
+puts
+
+# Warm up
+warm = XM::SFloat.new(64, 64).seq
+warm_sum = (warm * warm).sum
+warm_sum = warm_sum.extract if warm_sum.respond_to?(:extract)
+sync
+GC.start
+READBACKS[0] = 0
+
+# --- CG --------------------------------------------------------------------
+
+# Each distinct scalar is read at most once an iteration. Reading rs_old in
+# both alpha and beta, and rs_new in both the convergence test and beta, would
+# wait twice for a value already in hand.
+def solve(apply, b, rs0, shape)
+  x = XM::SFloat.zeros(*shape)
+  r = b + 0.0 # a copy
+  p_dir = r + 0.0
+  rs_old = timed(:dot) { dotp(r, r) }
+  rs_old_h = SCALAR == 'device' ? nil : rs0
+  iters = 0
+  converged = nil
+
+  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+  MAXITER.times do |it|
+    iters = it + 1
+
+    ap = timed(:apply) { apply.call(p_dir) }
+    pap = timed(:dot) { dotp(p_dir, ap) }
+
+    alpha = timed(:alpha) do
+      SCALAR == 'device' ? rs_old / pap : rs_old_h / scalar(pap).to_f
+    end
+
+    timed(:axpy) do
+      x += p_dir * alpha
+      r -= ap * alpha
+    end
+
+    rs_new = timed(:dot) { dotp(r, r) }
+    rs_new_h = SCALAR == 'device' ? nil : scalar(rs_new).to_f
+
+    if CHECK.positive? && (iters % CHECK).zero?
+      rel = Math.sqrt((rs_new_h || scalar(rs_new).to_f) / rs0)
+      if rel < TOL
+        converged = rel
+        break
+      end
+    end
+
+    beta = timed(:beta) do
+      SCALAR == 'device' ? rs_new / rs_old : rs_new_h / rs_old_h
+    end
+
+    timed(:update_p) { p_dir = r + p_dir * beta }
+    rs_old = rs_new
+    rs_old_h = rs_new_h
+  end
+  sync
+  { x: x, iters: iters, converged: converged,
+    elapsed: Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0 }
+end
+
+shape = OP == 'dense' ? [N] : [GRID, GRID]
+rs0 = scalar(dotp(b, b)).to_f # the reference for the relative residual, read once
+
+# Solve it a few times and keep the fastest. A single timing here varies by a
+# factor of a few, which is enough to invent a trend that is not there.
+best = nil
+REPEAT.times do
+  TIMES.clear
+  READBACKS[0] = 0
+  run = solve(apply, b, rs0, shape)
+  run[:times] = TIMES.dup
+  run[:readbacks] = READBACKS[0]
+  best = run if best.nil? || run[:elapsed] < best[:elapsed]
+end
+x = best[:x]
+iters = best[:iters]
+elapsed = best[:elapsed]
+TIMES.replace(best[:times])
+READBACKS[0] = best[:readbacks]
+puts format('  converged at iter %d, relative residual %.3e', iters, best[:converged]) if best[:converged]
+
+# --- results ---------------------------------------------------------------
+
+res = apply.call(x) - b
+rel_res = Math.sqrt(scalar((res * res).sum).to_f / rs0)
+err = x - x_true
+rel_err = Math.sqrt(scalar((err * err).sum).to_f) /
+          Math.sqrt(scalar((x_true * x_true).sum).to_f)
+
+puts format('  iterations     : %d', iters)
+puts format('  rel. residual  : %.3e', rel_res)
+puts format('  rel. error     : %.3e  (against the known x_true)', rel_err)
+puts format('  time           : %.4f s  (%.1f us/iter, best of %d)',
+            elapsed, elapsed / iters * 1e6, REPEAT)
+puts format('  readbacks      : %d  (%.2f per iteration)',
+            READBACKS[0], READBACKS[0].to_f / iters)
+
+if DETAIL
+  puts
+  measured = TIMES.values.sum
+  puts 'breakdown (per iteration):'
+  TIMES.sort_by { |_, v| -v }.each do |key, v|
+    puts format('  %-10s %9.1f us  %5.1f%%', key, v / iters * 1e6, 100.0 * v / measured)
+  end
+  puts '  (these sections synchronize, so they add up to more than us/iter above,'
+  puts '   and DETAIL=1 is much noisier than DETAIL=0 because of it)'
+end
+
+puts
+puts '  What to compare:'
+puts '    SCALAR=host   CHECK=1   the plain way, several readbacks an iteration'
+puts '    SCALAR=device CHECK=1   alpha and beta stay on the device, one readback'
+puts '    SCALAR=device CHECK=20  the convergence test thinned out too'
+puts '    SCALAR=device CHECK=0   no readback at all inside the loop'
+puts '  Thinning the test on its own changes nothing: the host path still reads'
+puts '  pap and rs_new every iteration. The two settings only pay together.'