cumo 0.5.7 → 0.5.9

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 3ddc7b756681afdf9fccd45652c422d2f67accfd1cf238d1fd05f6827fb5029d
-  data.tar.gz: 47848503fd8a388b7dfee79d2da0e627032fd4d8c8717645da70d8f09770f599
+  metadata.gz: f211eb1858b22b8418a8494efb431afeca60f023de409d74e6bdb1ae1fc3753a
+  data.tar.gz: '085be7ea42f37e235945e62783c51522c206c22eb539557b965812cc71ce422d'
 SHA512:
-  metadata.gz: f59ddc27c8f96a5eceec7fb4ae6665d1fae6dd20ef039a83563c0e0f5a76780fd2ab9b5ebbe06e116a3f01cd59c7493eee06dc6c1166a0bb82ba546c43331f03
-  data.tar.gz: 16543c06d74f79089799442abc30741ffb2a26556da953410023b19711fd25e15adb05548f9664c79bf8430a3be9c8f15c0813dccfae4b5e918ffc55a5fc68c0
+  metadata.gz: 19a6ca3c65b190e2347624232fa33302f4485e84d90df7de6972d727d76353e20a15a00bf8a7c8c96d281a97bbd7f8084016b8c2961e6781e4d94b00b17287d1
+  data.tar.gz: 2df512d0a5e2b201cd82cc91d4dc258f4d3463097e2969250ab0aa81ba591abaaa89ea0c6d1e818d8b85f32d698a6b9cc529d961a8e432be540fb7f4144f2e70

data/CHANGELOG.md

@@ -1,3 +1,88 @@
+# 0.5.9 (2026/08/20)
+
+Breaking changes:
+
+* `sort_index` resolves ties to the lowest index, the GPU sort being stable, where the host quicksort left them in whatever order it produced (PR #261)
+
+Fixes:
+
+* Fix `a[[2, 1], true].store(array)` writing every row onto row 0, on every dtype: ndloop read the index array on the host before the kernel that fills it had run (PR #265)
+* Fix a segmentation fault storing a Ruby Array into a reversed `Cumo::Bit` view that starts past the first word (PR #266)
+* Fix `sort` in place of a view backed by an index array reading row 0 for every row (PR #262)
+* Fix a `Cumo::RObject` store reading an index array before the kernel writes it, which answered zeroes from the second store onwards (PR #258)
+* Fix a store of an Array of narrays into an indexed destination writing one element per row (PR #257)
+* Fix the zero fill of a row stored from a shorter narray starting one element too far (PR #256)
+* Fix grid-stride kernels hanging from about 2**32 elements, the step wrapping to zero (PR #240)
+* Fix `Cumo::Bit` views with a negative step reading out of bounds (PR #238)
+* Fix index reductions answering a later index on ties, where `Numo::NArray` answers the first (PR #236)
+* Fix wrong values from reductions with `nan: true` (PR #234)
+
+Changes:
+
+* Run `sort`, `sort_index` and `median` on the GPU with `cub::DeviceSegmentedRadixSort`; 1M `SFloat` `sort` 115.7 -> 0.47 ms, `median` 107.8 -> 0.13 ms, `sort_index` 115.8 -> 1.69 ms (PR #259, PR #260, PR #261)
+* Run `poly` on the GPU, which had called its iterator once per element behind a synchronization; 2^20 `DFloat` 163.8 -> 0.021 ms (PR #263)
+* Run `bincount` on the GPU; 1M `Int32` 1.62 -> 0.05 ms (PR #255)
+* Run `cumsum` and `cumprod` on the GPU (PR #242)
+* Run `minmax`, `abs`, `isnan` and the rest of `cond_unary`, `modf` and `frexp` on the GPU (PR #230, PR #231, PR #232, PR #233)
+* Run the `Cumo::Bit` operators, reductions, `where`, `where2`, `fill` and the stores on the GPU (PR #237, PR #239, PR #241, PR #252, PR #254, PR #264, PR #266)
+* Run complex `real=` and `imag=` on the GPU; 2^22 `DComplex` 4.63 -> 0.45 ms (PR #267)
+* Run the accumulating reductions and `mulsum` on more than one GPU thread, where they had been running device-side thrust in a single thread (PR #243, PR #244)
+* Address a whole non-contiguous view in one kernel launch instead of one per row, in the elementwise templates, the stores, the copies and the Bit operands (PR #245, PR #246, PR #249, PR #251, PR #252)
+* Read a flat reduction operand without the indexer, and reduce over an outer axis with host-side addressing; `sum(axis: 0)` on 4096x4096 2.01 -> 0.017 ms (PR #248, PR #253)
+* Split a reduction with too few outputs across more blocks (PR #235)
+* Build the C++ and CUDA sources with optimization: mkmf leaves `$(optflags)` out of `CXXFLAGS`, so every `.cpp` and the host half of every `.cu` had been built at `-O0`; `a + b` 2.86 -> 2.00 us (PR #250)
+* Warn about the synchronization `a[idx]` and `inspect` perform (PR #247)
+* Add benchmark scripts under `bench/` for a transformer block, k-means, a 2D Ising model, conjugate gradient, a particle simulation and the CPU/GPU crossover, and two probes for calls that synchronize or launch one kernel per row
+
+# 0.5.8 (2026/08/17)
+
+Breaking changes:
+
+* Random numbers are generated on the GPU, so `srand` no longer reproduces `Numo::NArray`'s values for the same seed. Reproducibility within Cumo is unchanged and no longer depends on how the calls are split (PR #223)
+* Integer division by zero raises `ZeroDivisionError` instead of returning whatever the hardware produced, matching `Numo::NArray` (PR #218)
+* A numeric index into an unallocated NArray raises instead of returning a view nothing ever reads (PR #225)
+
+Fixes:
+
+* Fix `rb_raise` from inside a C++ catch leaking the exception object (PR #229)
+* Fix a failed Array store leaking its staging buffer, as much as the destination holds per failure (PR #228)
+* Fix the GC free hook raising when a device free fails, which surfaced at whatever line triggered the collection (PR #227)
+* Fix `from_binary` and `store_binary` writing managed memory while a kernel was still reading it (PR #226)
+* Fix integer `sum` and `prod` truncating every partial the reduction merged (PR #224)
+* Fix `minmax` ignoring compatible mode and returning zero-dimensional NArrays (PR #220)
+* Fix `nan` poisoning `max` and an all-`nan` `min` answering `DBL_MAX` (PR #219)
+* Fix `reshape!` and `marshal_load` mutating the array before they validate their arguments (PR #217)
+* Fix `ptp` answering 1 whatever the input (PR #216)
+* Fix `mulsum` ignoring the accumulator and the operand strides, which faulted the GPU on 8-bit dtypes (PR #215)
+* Fix `dot` handing `gemm` operands of a different dtype, an out-of-bounds read (PR #214)
+* Fix view offsets being added in the wrong unit (PR #213)
+* Fix interpreter abort on a zero-dimensional view (PR #212)
+* Fix `Cumo::RObject#logseq` returning `Infinity` (PR #210)
+* Fix `to_i`, `to_f` and `to_c` recursing forever on a one-element array (PR #208)
+* Fix complex `log2` and `log10` discarding the logarithm on the GPU (PR #207)
+* Fix segfault in `max_index` and `min_index` with `nan: true` (PR #206)
+* Fix `poly` failing on every input from an uninitialized `ndfunc` dimension (PR #205)
+* Fix out-of-bounds writes in `bincount` from an overflowing length and a stale scan (PR #204)
+* Fix `bincount` raising `TypeError` for every input (PR #203)
+* Fix host loops reading device memory without synchronizing, which silently corrupted `minmax`, the `nan: true` reductions, `kahan_sum`, `modf`, `frexp` and `set_imag` (PR #202)
+* Fix out-of-bounds access from three unchecked `size_t` multiplications (PR #201)
+* Fix segfaults from an unvalidated marshal payload (PR #200)
+* Fix `store_array` reading a source Array that its own elements rewrite (PR #198)
+* Fix CUDA initialisation statuses being ignored at `require` time (PR #196)
+* Fix kernel launch errors being discarded, which let a rejected launch return an untouched buffer as success (PR #195)
+* Fix wrong `Cumo::RObject` results from driving host memory with asynchronous CUDA work (PR #194)
+* Fix device memory being invisible to the GC, which let a program churning temporaries run the GPU out of memory (PR #221)
+
+Changes:
+
+* Run `rand` and `rand_norm` on the GPU (PR #223)
+* Run `clip` on the GPU (PR #222)
+* Correct spelling in messages, docs and comments (PR #211)
+* Add the math, extra and narray test suites `numo-narray-alt` has and cumo lacked (PR #207, PR #208, PR #209)
+* Add a `store_array` regression test for a shrunk source Array (PR #199)
+* Stop the ccache cache growing past the 10 GB limit in CI (PR #197)
+* Add a benchmark script under `bench/` (commit 3127f84)
+
 # 0.5.7 (2026/08/13)
 
 Breaking changes:

data/bench/bench.rb

@@ -0,0 +1,235 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# ---------------------------------------------------------------------------
+# Numo / Cumo comparison benchmark
+#
+#   ruby bench.rb                  # CPU: numo-narray
+#   GPU=1 ruby bench.rb            # GPU: cumo
+#   SIZE=4096 STEPS=500 ITER=3 ruby bench.rb
+#   PGM=0 ruby bench.rb            # skip the PGM output
+#
+# To compare against numo-narray-alt (yoshoku's fork, SIMD enabled), swap the
+# gem; this file needs no change, since the namespace stays Numo:
+#   gem uninstall numo-narray && gem install numo-narray-alt
+#
+# Environment variables that matter on the Cumo side:
+#   CUMO_SHOW_WARNING=ON     warn wherever a CPU/GPU synchronization happens
+#   CUDA_LAUNCH_BLOCKING=1   synchronize on every launch (for profiling; slow)
+#   CUMO_MEMORY_POOL=OFF     turn the memory pool off to measure allocation too
+#   CUDA_VISIBLE_DEVICES=0   device selection
+#
+# Numo::Bit, masked assignment and fancy indexing are deliberately unused. The
+# divergence test is written with clip and floor arithmetic alone, so Numo and
+# Cumo take exactly the same code path.
+# ---------------------------------------------------------------------------
+
+require 'benchmark'
+
+GPU = !%w[0 false].include?(ENV['GPU'].to_s.downcase) && !ENV['GPU'].to_s.empty?
+
+if GPU
+  require 'cumo/narray'
+  XM = Cumo
+else
+  require 'numo/narray'
+  XM = Numo
+end
+
+SIZE  = (ENV['SIZE']  || 1024).to_i   # side of the mandelbrot / diffusion grid
+STEPS = (ENV['STEPS'] || 200).to_i    # number of iterations
+NBODY = (ENV['NBODY'] || 2048).to_i   # number of n-body particles
+MC    = (ENV['MC']    || 10_000_000).to_i
+ITER  = (ENV['ITER']  || 3).to_i      # repeats per measurement (3-5 to steady it)
+PGM   = ENV['PGM'] != '0'
+
+# --- helpers ---------------------------------------------------------------
+
+# Cumo launches kernels asynchronously, so every measurement boundary syncs.
+def sync
+  XM::CUDA::Runtime.cudaDeviceSynchronize if GPU
+end
+
+# Cumo returns a zero-dimensional NArray from sum / min / max / count_true,
+# where Numo returns a Numeric.
+def scalar(v)
+  return v if v.is_a?(Numeric)
+  v.respond_to?(:extract_cpu) ? v.extract_cpu : v.extract
+end
+
+def free(*arrays)
+  return unless GPU
+
+  arrays.each { |a| a.free if a.respond_to?(:free) }
+end
+
+def report(label)
+  result = nil
+  total = Benchmark.realtime do
+    ITER.times do
+      result = yield
+      sync
+    end
+  end
+  puts format('  %-12s %9.3f s total   %9.3f s/iter', label, total, total / ITER)
+  result
+end
+
+def write_pgm(path, arr)
+  return unless PGM
+
+  lo = scalar(arr.min).to_f
+  hi = scalar(arr.max).to_f
+  hi = lo + 1.0 if hi <= lo
+  img = XM::UInt8.cast((arr - lo) / (hi - lo) * 255.0)
+  h, w = img.shape
+  bytes =
+    begin
+      img.to_binary
+    rescue StandardError
+      # fallback for a Cumo without to_binary (device to host copy)
+      img.to_a.flatten.pack('C*')
+    end
+  File.open(path, 'wb') do |f|
+    f.write("P5\n#{w} #{h}\n255\n")
+    f.write(bytes)
+  end
+  free(img)
+  puts "  -> #{path}"
+end
+
+# --- 1. mandelbrot set (pure elementwise arithmetic) ------------------------
+#
+# Build an indicator that is 1.0 once diverged and 0.0 while still alive using
+# clip and floor alone, then accumulate the survival count. No branches and no
+# masks, so it maps straight onto the GPU.
+def mandelbrot(n, steps)
+  lim = 1.0e10 # bound zr and zi here so neither Inf nor NaN appears
+  cr = XM::SFloat.new(1, n).seq / n * 3.0 - 2.0 # real axis [-2.0, 1.0)
+  ci = XM::SFloat.new(n, 1).seq / n * 3.0 - 1.5 # imaginary axis [-1.5, 1.5)
+  zr = XM::SFloat.zeros(n, n)
+  zi = XM::SFloat.zeros(n, n)
+  cnt = XM::SFloat.zeros(n, n)
+  cr = zr + cr # materialize the broadcast
+  ci = zr + ci
+
+  steps.times do
+    zr2 = zr * zr
+    zi2 = zi * zi
+    escaped = ((zr2 + zi2).clip(0.0, 4.0) / 4.0).floor # 1.0 once |z|^2 >= 4
+    cnt += 1.0 - escaped
+    zi = (2.0 * zr * zi + ci).clip(-lim, lim)
+    zr = (zr2 - zi2 + cr).clip(-lim, lim)
+  end
+  free(cr, ci, zr, zi)
+  cnt
+end
+
+# --- 2. 2-D heat diffusion (stencil / slice arithmetic) ---------------------
+def diffusion(n, steps)
+  u = XM::SFloat.zeros(n, n)
+  h = [n / 8, 1].max
+  u[h...(2 * h), h...(2 * h)] = 255.0
+  u[(5 * h)...(7 * h), (3 * h)...(4 * h)] = 255.0
+  k = 0.2
+
+  steps.times do
+    lap = u[0..-3, 1..-2] + u[2..-1, 1..-2] +
+          u[1..-2, 0..-3] + u[1..-2, 2..-1] - 4.0 * u[1..-2, 1..-2]
+    u[1..-2, 1..-2] = u[1..-2, 1..-2] + k * lap
+  end
+  u
+end
+
+# --- 3. n-body (2-D, O(N^2) through broadcasting) ---------------------------
+def nbody(n, steps)
+  dt = 1.0e-3
+  eps = 1.0e-3
+  x = XM::SFloat.new(n).rand * 2.0 - 1.0
+  y = XM::SFloat.new(n).rand * 2.0 - 1.0
+  vx = XM::SFloat.zeros(n)
+  vy = XM::SFloat.zeros(n)
+
+  steps.times do
+    dx = x.reshape(1, n) - x.reshape(n, 1) # [n, n]
+    dy = y.reshape(1, n) - y.reshape(n, 1)
+    r2 = dx * dx + dy * dy + eps
+    inv3 = 1.0 / (r2 * XM::NMath.sqrt(r2))
+    vx += (dx * inv3).sum(axis: 1) * dt
+    vy += (dy * inv3).sum(axis: 1) * dt
+    free(dx, dy, r2, inv3)
+    x += vx * dt
+    y += vy * dt
+  end
+  free(y, vx, vy)
+  x
+end
+
+# --- 4. monte carlo pi (random numbers + reduction) -------------------------
+# The sum is SFloat, so a large n costs pi its accuracy to rounding. Speed is
+# what this measures.
+def monte_carlo_pi(n)
+  x = XM::SFloat.new(n).rand
+  y = XM::SFloat.new(n).rand
+  r2 = x * x + y * y
+  inside = (1.0 - r2.clip(0.0, 1.0)).ceil # 1.0 where r2 < 1
+  hits = scalar(inside.sum).to_f
+  free(x, y, r2, inside)
+  4.0 * hits / n
+end
+
+# --- 5. compatibility probe (eyeball the incompatibilities the README lists) -
+def compat_probe
+  a = XM::SFloat[1.0, 2.0, 3.0]
+  bit = (a > 1.5)
+  count = bit.respond_to?(:count_true_cpu) ? bit.count_true_cpu : bit.count_true
+  puts "  (a > 1.5).class     : #{bit.class}"
+  puts "  count_true          : #{count.inspect}"
+  puts "  a.sum.class         : #{a.sum.class}"   # Numo: Float / Cumo: zero-dimensional NArray
+  puts "  a.max.class         : #{a.max.class}"
+  puts "  a[0].class          : #{a[0].class}"
+  free(a)
+end
+
+# --- main ------------------------------------------------------------------
+
+version = begin
+  XM::NArray::VERSION
+rescue StandardError
+  'unknown'
+end
+
+puts "ruby      : #{RUBY_VERSION} (#{RUBY_PLATFORM})"
+puts "backend   : #{XM} #{version}"
+puts "params    : SIZE=#{SIZE} STEPS=#{STEPS} NBODY=#{NBODY} MC=#{MC} ITER=#{ITER}"
+puts
+
+# warm up (keep GPU initialization and kernel JIT out of the measurement)
+warm = XM::SFloat.new(64, 64).seq
+scalar((warm * warm + warm).sum)
+sync
+free(warm)
+GC.start
+
+puts 'compat:'
+compat_probe
+puts
+
+puts 'bench:'
+cnt = report('mandelbrot') { mandelbrot(SIZE, STEPS) }
+write_pgm('mandelbrot.pgm', cnt)
+free(cnt)
+GC.start
+
+u = report('diffusion') { diffusion(SIZE, STEPS) }
+write_pgm('diffusion.pgm', u)
+free(u)
+GC.start
+
+pos = report('nbody') { nbody(NBODY, 10) }
+free(pos)
+GC.start
+
+pi = nil
+report('monte_carlo') { pi = monte_carlo_pi(MC) }
+puts format('  pi ~= %.6f', pi)

data/bench/cg_bench.rb

@@ -0,0 +1,271 @@
+#!/usr/bin/env ruby
+# frozen_string_literal: true
+
+# ---------------------------------------------------------------------------
+# Conjugate gradient, to price what a host readback costs.
+#
+#   ruby cg_bench.rb
+#   GPU=1 ruby cg_bench.rb                        # the plain way: readbacks every iteration
+#   GPU=1 SCALAR=device ruby cg_bench.rb          # keep alpha and beta on the device
+#   GPU=1 SCALAR=device CHECK=20 ruby cg_bench.rb # and test convergence rarely
+#   GPU=1 OP=dense N=4096 ruby cg_bench.rb        # heavier arithmetic per iteration
+#
+# CG makes two scalars an iteration, alpha and beta, and multiplies arrays by
+# them. Written plainly that reads them back to the host every iteration, and
+# the convergence test reads a third, each one waiting for the queue to drain.
+# An iteration is not much arithmetic, so the waiting is what shows.
+#
+#   SCALAR=host   alpha and beta become Ruby Floats, as one would write it
+#   SCALAR=device they stay as the 0-dimensional NArray sum answers, and are
+#                 never read (Cumo returns 0-dim; on Numo sum is a Float
+#                 already, so the setting makes no difference there)
+#   CHECK=k       test convergence every k iterations, 0 to never test
+#
+# A readback does not cost a fixed amount. What it costs is however much work
+# is queued when it happens, so it is better read as a cap on how deep the
+# pipeline is allowed to get than as a price per call. Several shallow drains
+# an iteration cost little each; the last one costs a lot, because until it is
+# gone no iteration can overlap the next.
+#
+# On the host path the floor is two readbacks an iteration whatever CHECK is
+# set to, since alpha needs pap and beta needs rs_new as Floats. Keeping the
+# scalars on the device is what makes fewer than that possible, and thinning
+# the convergence test only pays once they are there.
+#
+# Correctness is checked against a known solution: x_true is chosen first and
+# b = A x_true built from it, so the answer has something to be compared to.
+#
+# Timings vary by a factor of a few between runs. Take the best of several.
+# ---------------------------------------------------------------------------
+
+GPU = !%w[0 false].include?(ENV['GPU'].to_s.downcase) && !ENV['GPU'].to_s.empty?
+
+if GPU
+  require 'cumo/narray'
+  XM = Cumo
+else
+  require 'numo/narray'
+  XM = Numo
+end
+
+OP      = (ENV['OP']      || 'stencil')  # stencil | dense
+GRID    = (ENV['GRID']    || 512).to_i   # side of the grid, for stencil
+N       = (ENV['N']       || 2048).to_i  # number of unknowns, for dense
+MAXITER = (ENV['MAXITER'] || 200).to_i
+SCALAR  = (ENV['SCALAR']  || 'host')     # host | device
+CHECK   = (ENV['CHECK']   || 1).to_i     # test convergence every CHECK, 0 to never
+TOL     = (ENV['TOL']     || 1.0e-6).to_f
+DOT     = (ENV['DOT']     || 'sum')      # sum | blas
+DETAIL  = ENV['DETAIL'] != '0'
+REPEAT  = (ENV['REPEAT']  || 3).to_i     # solve this many times, keep the fastest
+SEED    = (ENV['SEED'] || 42).to_i
+
+TIMES = Hash.new(0.0)
+READBACKS = [0]
+
+def sync
+  XM::CUDA::Runtime.cudaDeviceSynchronize if GPU
+end
+
+# A readback, which is where the host waits. Counted, since that is the point.
+def scalar(v)
+  READBACKS[0] += 1
+  return v if v.is_a?(Numeric)
+
+  v.respond_to?(:extract_cpu) ? v.extract_cpu : v.extract
+end
+
+def timed(key)
+  return yield unless DETAIL
+
+  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+  r = yield
+  sync
+  TIMES[key] += Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0
+  r
+end
+
+def dotp(a, b)
+  if DOT == 'blas'
+    a.reshape(a.size).dot(b.reshape(b.size))
+  else
+    (a * b).sum
+  end
+end
+
+# --- the operator ----------------------------------------------------------
+
+# The five-point Laplacian, negated to be positive definite. The border stays
+# zero and is never touched.
+def apply_stencil(p)
+  out = XM::SFloat.zeros(GRID, GRID)
+  out[1..-2, 1..-2] =
+    4.0 * p[1..-2, 1..-2] -
+    p[0..-3, 1..-2] - p[2..-1, 1..-2] -
+    p[1..-2, 0..-3] - p[1..-2, 2..-1]
+  out
+end
+
+# A diagonally dominant symmetric matrix. The identity is built with
+# arithmetic rather than a fancy index.
+def build_dense
+  r = XM::SFloat.new(N, N).rand * 2.0 - 1.0
+  sym = (r + r.transpose) * 0.5
+  i = XM::SFloat.new(N, 1).seq
+  j = XM::SFloat.new(1, N).seq
+  off = (XM::SFloat.zeros(N, N) + (i - j)).abs.clip(0.0, 1.0).ceil # 0 on the diagonal
+  sym + (1.0 - off) * (N * 1.0)
+end
+
+# --- setup -----------------------------------------------------------------
+
+version = begin
+  XM::NArray::VERSION
+rescue StandardError
+  'unknown'
+end
+
+begin
+  XM::NArray.srand(SEED)
+rescue StandardError
+  nil
+end
+
+if OP == 'dense'
+  amat = build_dense
+  x_true = XM::SFloat.new(N).rand * 2.0 - 1.0
+  apply = ->(p) { amat.dot(p) }
+  unknowns = N
+else
+  x_true = XM::SFloat.zeros(GRID, GRID)
+  x_true[1..-2, 1..-2] = XM::SFloat.new(GRID - 2, GRID - 2).rand * 2.0 - 1.0
+  apply = ->(p) { apply_stencil(p) }
+  unknowns = (GRID - 2) * (GRID - 2)
+end
+
+b = apply.call(x_true)
+
+puts "ruby      : #{RUBY_VERSION} (#{RUBY_PLATFORM})"
+puts "backend   : #{XM} #{version}"
+puts "problem   : OP=#{OP} unknowns=#{unknowns} MAXITER=#{MAXITER}"
+puts "scalar    : SCALAR=#{SCALAR} CHECK=#{CHECK} DOT=#{DOT}"
+puts
+
+# Warm up
+warm = XM::SFloat.new(64, 64).seq
+warm_sum = (warm * warm).sum
+warm_sum = warm_sum.extract if warm_sum.respond_to?(:extract)
+sync
+GC.start
+READBACKS[0] = 0
+
+# --- CG --------------------------------------------------------------------
+
+# Each distinct scalar is read at most once an iteration. Reading rs_old in
+# both alpha and beta, and rs_new in both the convergence test and beta, would
+# wait twice for a value already in hand.
+def solve(apply, b, rs0, shape)
+  x = XM::SFloat.zeros(*shape)
+  r = b + 0.0 # a copy
+  p_dir = r + 0.0
+  rs_old = timed(:dot) { dotp(r, r) }
+  rs_old_h = SCALAR == 'device' ? nil : rs0
+  iters = 0
+  converged = nil
+
+  t0 = Process.clock_gettime(Process::CLOCK_MONOTONIC)
+  MAXITER.times do |it|
+    iters = it + 1
+
+    ap = timed(:apply) { apply.call(p_dir) }
+    pap = timed(:dot) { dotp(p_dir, ap) }
+
+    alpha = timed(:alpha) do
+      SCALAR == 'device' ? rs_old / pap : rs_old_h / scalar(pap).to_f
+    end
+
+    timed(:axpy) do
+      x += p_dir * alpha
+      r -= ap * alpha
+    end
+
+    rs_new = timed(:dot) { dotp(r, r) }
+    rs_new_h = SCALAR == 'device' ? nil : scalar(rs_new).to_f
+
+    if CHECK.positive? && (iters % CHECK).zero?
+      rel = Math.sqrt((rs_new_h || scalar(rs_new).to_f) / rs0)
+      if rel < TOL
+        converged = rel
+        break
+      end
+    end
+
+    beta = timed(:beta) do
+      SCALAR == 'device' ? rs_new / rs_old : rs_new_h / rs_old_h
+    end
+
+    timed(:update_p) { p_dir = r + p_dir * beta }
+    rs_old = rs_new
+    rs_old_h = rs_new_h
+  end
+  sync
+  { x: x, iters: iters, converged: converged,
+    elapsed: Process.clock_gettime(Process::CLOCK_MONOTONIC) - t0 }
+end
+
+shape = OP == 'dense' ? [N] : [GRID, GRID]
+rs0 = scalar(dotp(b, b)).to_f # the reference for the relative residual, read once
+
+# Solve it a few times and keep the fastest. A single timing here varies by a
+# factor of a few, which is enough to invent a trend that is not there.
+best = nil
+REPEAT.times do
+  TIMES.clear
+  READBACKS[0] = 0
+  run = solve(apply, b, rs0, shape)
+  run[:times] = TIMES.dup
+  run[:readbacks] = READBACKS[0]
+  best = run if best.nil? || run[:elapsed] < best[:elapsed]
+end
+x = best[:x]
+iters = best[:iters]
+elapsed = best[:elapsed]
+TIMES.replace(best[:times])
+READBACKS[0] = best[:readbacks]
+puts format('  converged at iter %d, relative residual %.3e', iters, best[:converged]) if best[:converged]
+
+# --- results ---------------------------------------------------------------
+
+res = apply.call(x) - b
+rel_res = Math.sqrt(scalar((res * res).sum).to_f / rs0)
+err = x - x_true
+rel_err = Math.sqrt(scalar((err * err).sum).to_f) /
+          Math.sqrt(scalar((x_true * x_true).sum).to_f)
+
+puts format('  iterations     : %d', iters)
+puts format('  rel. residual  : %.3e', rel_res)
+puts format('  rel. error     : %.3e  (against the known x_true)', rel_err)
+puts format('  time           : %.4f s  (%.1f us/iter, best of %d)',
+            elapsed, elapsed / iters * 1e6, REPEAT)
+puts format('  readbacks      : %d  (%.2f per iteration)',
+            READBACKS[0], READBACKS[0].to_f / iters)
+
+if DETAIL
+  puts
+  measured = TIMES.values.sum
+  puts 'breakdown (per iteration):'
+  TIMES.sort_by { |_, v| -v }.each do |key, v|
+    puts format('  %-10s %9.1f us  %5.1f%%', key, v / iters * 1e6, 100.0 * v / measured)
+  end
+  puts '  (these sections synchronize, so they add up to more than us/iter above,'
+  puts '   and DETAIL=1 is much noisier than DETAIL=0 because of it)'
+end
+
+puts
+puts '  What to compare:'
+puts '    SCALAR=host   CHECK=1   the plain way, several readbacks an iteration'
+puts '    SCALAR=device CHECK=1   alpha and beta stay on the device, one readback'
+puts '    SCALAR=device CHECK=20  the convergence test thinned out too'
+puts '    SCALAR=device CHECK=0   no readback at all inside the loop'
+puts '  Thinning the test on its own changes nothing: the host path still reads'
+puts '  pap and rs_new every iteration. The two settings only pay together.'