cumo 0.5.9 → 0.5.11

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f211eb1858b22b8418a8494efb431afeca60f023de409d74e6bdb1ae1fc3753a
-  data.tar.gz: '085be7ea42f37e235945e62783c51522c206c22eb539557b965812cc71ce422d'
+  metadata.gz: b345c4e036212d334600041165fe79e533ef0e1dd450e22b1288f5e6d6214ffa
+  data.tar.gz: af60f78cdd2f39a96e8a183eff02a26c26d8977560ba148643c55df5961c2bf2
 SHA512:
-  metadata.gz: 19a6ca3c65b190e2347624232fa33302f4485e84d90df7de6972d727d76353e20a15a00bf8a7c8c96d281a97bbd7f8084016b8c2961e6781e4d94b00b17287d1
-  data.tar.gz: 2df512d0a5e2b201cd82cc91d4dc258f4d3463097e2969250ab0aa81ba591abaaa89ea0c6d1e818d8b85f32d698a6b9cc529d961a8e432be540fb7f4144f2e70
+  metadata.gz: 361042e17c2f498b8e4f5ddc76bbd119688a6a3a3ac176307973fd51c75b332d4d831a82c0fc0e8a3b28be98bfbe50d507ec67d517c609c2342caf57d94d9c41
+  data.tar.gz: 53da42c447fe31812654e1ab0566f5e553f781daaf75b97bb1c36630b8dc3ed195704002990bc4a2a9f4fdd6e26e92c2d5905ff64da0e0f058b01dae20ae3dec

data/.gitignore

@@ -29,3 +29,4 @@ t.rb
 .DS_Store
 /vendor/
 .kernel_cache/
+bench/cumo_probe_baseline.json

data/3rd_party/mkmf-cu/lib/mkmf-cu/cli.rb

@@ -40,7 +40,9 @@ module MakeMakefileCuda
       s = MakeMakefileCuda::Nvcc.generate(argv)
       cmd = "nvcc #{s}"
       if ENV['CUMO_NVCC_GENERATE_CODE']
-        cmd << " --generate-code=#{ENV['CUMO_NVCC_GENERATE_CODE']}"
+        ENV['CUMO_NVCC_GENERATE_CODE'].split.each do |code|
+          cmd << " --generate-code=#{code}"
+        end
       else
         capability = nil
         if find_executable('nvidia-smi')

data/CHANGELOG.md

@@ -1,3 +1,70 @@
+# 0.5.11 (2026/08/29)
+
+Breaking changes:
+
+* `conv` and `pooling` raise `ArgumentError` on a stride of zero, where the assert meant to catch it never ran under `NDEBUG` and the division by it raised SIGFPE (PR #307)
+
+Fixes:
+
+* Fix a numeric subscript into a view built from an index array reading the parent's index before the kernel that fills it, which answered with whatever the memory held (PR #333)
+* Fix `mulsum` answering from the wrong bits when an operand is a `Cumo::Bit` view carrying an index array, `cumo_na_copy` moving whole bytes where a Bit element is one bit (PR #326)
+
+Changes:
+
+* Read a view's index array through one synchronize instead of one per row of an md-loop, one per element read, or one per one-element subscript; with twenty kernels queued, `rand` over a 2000-row gather goes from 11.2ms to 3.4ms, a second `a[idx, true][0, 0]` from 54.2us to 0.2us, and `a[[0], true] * 2.0` from 104.3us to 3.5us (PR #334, PR #332, PR #331)
+* Gather a word of a `Cumo::Bit` operand from its rows rather than a bit at a time, and fold a Bit reduction by words within a row rather than only across a flat group; on a 512x2048 column slice `dst.store(slice)` goes from 18.0us to 8.9us and `count_true` from 22.2us to 8.8us (PR #328, PR #327)
+* Reduce `Bit#count_true`, `count_false`, `all?` and `any?` in one kernel instead of one launch per output element; 512x2048 `count_true(axis: 1)` goes from 904.0us to 10.0us and `all?(axis: 1)` from 498.7us to 12.7us (PR #324, PR #323)
+* Address a store's index array from its kernel instead of staging it through a contiguous buffer; a 512x2048 SFloat store into an indexed destination goes from 30.1us to 12.4us (PR #322)
+* Copy an ndloop buffer one element per thread instead of one row; `a[idx, true] = b` on a 1024x1024 SFloat goes from 1567.3us to 67.5us (PR #319)
+* Build `flatten`'s index array with a kernel instead of a host loop; a 512x2048 SFloat column slice goes from 2448.5us to 529.1us (PR #316)
+* Reduce `mulsum` in one kernel instead of one per output element; 512x2048 SFloat `mulsum(axis: 1)` goes from 1758.8us to 13.6us (PR #315)
+* Scatter one element per lane in the compaction behind `where`, `where2` and `mask`; at 4M elements `mask` goes from 193.8us to 106.2us and `where` from 131.4us to 74.1us (PR #313)
+* Build a contiguous Bit output a word at a time with `__ballot_sync`, whatever the operand's layout; at 4M elements `a > 0.5` goes from 459.0us to 31.5us and `view & view` from 481.5us to 42.6us (PR #312, PR #311)
+* Count bits a word at a time instead of one atomicAdd per element; 16M bits all set go from 227.0us to 17.9us (PR #309)
+* Document that a Ruby Float in `Cumo::NMath` promotes the result to double, and that passing a 0-dimensional array keeps it single (PR #325)
+* Drop the contiguity assert the cuDNN descriptors do not need (PR #308)
+* Sweep every kernel family in `bench/cumo_probe.rb`, measure the kernel rather than the allocation it sits above, and read a run against a saved baseline (PR #330, PR #329, PR #321, PR #320, PR #318, PR #317, PR #314, PR #310)
+
+# 0.5.10 (2026/08/23)
+
+Breaking changes:
+
+* An indexed assignment to a frozen view raises instead of writing through it; `store` and `fill` already raised (PR #306)
+* The cuDNN and cuBLAS entry points turn away arguments they used to read or write out of bounds: `pooling_backward` requires `y` and `gy` to carry its own dtype and the pooling output shape of `x`, `gemm` requires a contiguous `c`, `batch_norm` requires the reduced size to cover x's channels, and `conv_grad_w` raises on a mismatched `gy` rather than asserting (PR #303, PR #302, PR #300, PR #278)
+* `seq` on the unsigned types wraps a negative start and a start past 2**63 the way `fill` and `cast` do, where they collapsed to zero and clamped to INT64_MAX (PR #295, PR #294)
+* A negative power of an unsigned array answers zero instead of spinning the device until SIGKILL (PR #293)
+* A subscript that is a Cumo::RObject subclass, and an `expand_dims` past `CUMO_NA_MAX_DIMENSION`, raise instead of being taken (PR #272, PR #270)
+
+Fixes:
+
+* Fix `[]=` writing through a frozen view, the store going to a derived view whose data object is the root rather than the receiver (PR #306)
+* Fix `at()` reading the accumulator in place of the subscript on a reversed view, which addressed far in front of the buffer (PR #305)
+* Fix `cumo_cuda_cudnn_CreateBNTensorDescriptor` returning before it derived the descriptor, an inverted status check that no caller reached (PR #304)
+* Fix `batch_norm`, `fixed_batch_norm` and `batch_norm_backward` reading and writing past their parameters, the sizes being checked against the reduced shape while cuDNN reaches x's channel count (PR #303)
+* Fix `gemm` writing past its allocation when `c` is a non-contiguous inplace view, and over whatever the pool put next to it (PR #302)
+* Fix `pooling_backward` reading past `y` and `gy`, and returning what it read in `gx` (PR #300)
+* Fix `pow_int` negating INT32_MIN, which overflows and leaves the loop unbounded (PR #299)
+* Fix the device free running for a subscript that allocated no index array (PR #297)
+* Fix a NULL dereference from `at()` given a scalar subscript (PR #271)
+* Fix an out-of-bounds write through an unchecked cuDNN output array (PR #269)
+* Fix `gemm` reading past an operand that carries fewer batch dimensions (PR #268)
+
+Changes:
+
+* Pass a numeric operand to the kernel instead of casting it to a 0-dimensional array first, in the operators, `clip`, `pow`, the comparisons, the NMath functions and the coerced left-hand side; GPT-2 124M decode drops from 787 to 677 kernel launches a token (PR #296, PR #291, PR #290, PR #289, PR #288, PR #287)
+* Run the NMath binary functions through the indexer loop instead of one kernel launch per row; 11x on a transposed operand and up to 60x on the cheap kernels (PR #292)
+* Copy a reduction operand only when it carries an index array, where any non-contiguous view was copied whole (PR #286)
+* Run the nan-aware reductions, the nan-aware index reductions, `kahan_sum` and `Bit#mask` on the GPU (PR #277, PR #276, PR #275, PR #273)
+* Take a transposed batch of matrices with the cuBLAS transpose flag rather than duplicating it, which saves a temporary the size of the operand (PR #283)
+* Emit indexer accessors up to eight dimensions (PR #284)
+* Drop a free-list bin from the pool arena when `Malloc` empties it (PR #279)
+* Check the cuDNN header and version, not just the library (PR #280)
+* Document every method that returns a 0-dimensional NArray, and what keeping scalars on the device buys (PR #298, PR #282)
+* Build two GPU architectures in CI instead of every one (PR #285)
+* Add `fiddle` to the Gemfile (PR #281)
+* Remove TODO comments whose questions have been answered (PR #274)
+* Cover the Int32 array exponent in the INT_MIN power test (PR #301)
+
 # 0.5.9 (2026/08/20)
 
 Breaking changes:

data/Gemfile

@@ -5,6 +5,7 @@ source "https://rubygems.org"
 gemspec
 
 gem 'extconf_compile_commands_json'
+gem 'fiddle'
 gem 'rake-compiler'
 gem 'test-unit'
 gem 'yard'

data/README.md

@@ -9,6 +9,7 @@ Cumo (pronounced "koomo") is a CUDA-aware, GPU-optimized numerical library that
 * Ruby 3.0 or later
 * NVIDIA GPU Compute Capability 3.5 (Kepler) or later
 * CUDA 11.0 or later
+* cuDNN 8.0 or later (optional, for the cuDNN features)
 
 ## Preparation
 
@@ -97,17 +98,31 @@ a = xm::DFloat.new(3,5).seq
 
 ### Incompatibility With Numo
 
-The following methods behave incompatibly with Numo by default for performance reasons:
+Numo returns a Ruby numeric object wherever a result is 0-dimensional, while Cumo returns the 0-dimensional NArray itself.
+Cumo differs in this way to avoid synchronization and minimize CPU ⇄ GPU data transfer.
+That is not only a cost of the port; see [Keeping Scalars On The Device](#keeping-scalars-on-the-device) and [Ruby Floats In NMath Promote To Double](#ruby-floats-in-nmath-promote-to-double) for what it buys.
 
-* `extract`
-* `[]`
-* `count_true`
-* `count_false`
+The methods affected are:
 
-Numo returns a Ruby numeric object for 0-dimensional NArray, while Cumo returns the 0-dimensional NArray instead of a Ruby numeric object.
-Cumo differs in this way to avoid synchronization and minimize CPU ⇄ GPU data transfer.
+* `[]` and `extract`
+* `count_true` and `count_false`
+* reductions down to a single value: `sum`, `prod`, `mean`, `stddev`, `var`, `rms`, `min`, `max`, `ptp`, `minmax`, `median`, `mulsum`, `dot`, `inner`
+* index reductions: `max_index`, `min_index`, `argmax`, `argmin`
+
+A 0-dimensional `Cumo::Bit` is truthy even when it holds 0, because Ruby treats every object but `nil` and `false` as true.
+Comparing two scalars therefore takes the wrong branch without raising anything:
+
+```ruby
+a = Cumo::SFloat[5.0]
+a[0] < 1.0                  #=> Cumo::Bit#shape=[] holding 0
+(a[0] < 1.0) ? :yes : :no   #=> :yes, where Numo gives :no
+```
+
+`assert_operator(a[0], :<, 1.0)` passes for the same reason, so a test suite written for Numo can stay green against Cumo while asserting nothing.
+Read the value back to the host before branching on it, or run under `compatible_mode`.
 
 Set the `CUMO_COMPATIBLE_MODE` environment variable to `ON` to force Numo NArray compatibility (for worse performance).
+Running a Numo test suite that way keeps its assertions meaningful.
 
 You may enable or disable `compatible_mode` as:
 
@@ -126,6 +141,87 @@ You can also use the following methods which behave like Numo's NArray methods.
 * `count_true_cpu`
 * `count_false_cpu`
 
+```ruby
+a.aref_cpu(0) < 1.0   #=> false in either mode
+Float(a.sum)          #=> 7.0 in either mode
+```
+
+They are methods on an NArray, so chaining one onto a result that `compatible_mode` has already turned into a Ruby object, as in `a.sum.extract_cpu`, raises `NoMethodError` while the mode is on.
+`Kernel#Float` and `Kernel#Integer` read either representation, and read Numo's too, so they are what code that runs against both libraries wants.
+
+### Keeping Scalars On The Device
+
+The 0-dimensional return is what lets an iterative loop stay on the GPU.
+Reading a scalar back to the host waits for everything queued behind it, so every read caps how far ahead the GPU is allowed to run.
+What a read costs is not a fixed price either: it is however much work happens to be queued when it is taken.
+
+`bench/cg_bench.rb` prices this with a conjugate gradient solve, 200 iterations over a 512x512 grid on an RTX 5070 Ti Laptop:
+
+```
+scalars        convergence test    us/iter    readbacks/iter
+Ruby Floats    every iteration       136.7        2.02
+Ruby Floats    never                 134.4        2.02
+0-dim NArray   every iteration       142.6        1.02
+0-dim NArray   every 20th             65.6        0.06
+0-dim NArray   never                  60.3        0.02
+```
+
+Written with Ruby Floats the loop reads back twice an iteration whatever the convergence test does, since `alpha` needs `pap` and `beta` needs `rs_new` as Floats.
+Thinning the test cannot get under that floor, and keeping the scalars as 0-dimensional NArrays buys nothing on its own.
+The two only pay together, and together they are worth 2.1x.
+The relative residual is identical in every row.
+
+```ruby
+alpha = rs_old / pap   # a 0-dimensional NArray, divided on the device
+x += p_dir * alpha     # and consumed there, without crossing the bus
+```
+
+Read the value back once the loop is done, or every k iterations if it has to test something.
+
+### Ruby Floats In NMath Promote To Double
+
+`Cumo::NMath` picks the module it dispatches to from every argument it is given, and a Ruby `Float` counts as a `DFloat` there.
+A single-precision array therefore comes back doubled whenever a plain Float rides along, even though the arithmetic operators leave it alone:
+
+```ruby
+Cumo::NMath.atan2(a, 2.0)   #=> Cumo::DFloat
+Cumo::NMath.atan2(a, b)     #=> Cumo::SFloat
+a + 2.0                     #=> Cumo::SFloat
+```
+
+Numo promotes the same way, and on a CPU it costs nothing: Numo's single-precision math computes in double and narrows the result anyway.
+On a GeForce card, whose double-precision rate is a sixty-fourth of its single-precision one, it costs a great deal.
+512x2048 elements in place on an RTX 5070 Ti Laptop:
+
+```
+                SFloat     DFloat
+a * 2.0         11.8 us    12.8 us
+sqrt            11.0 us    43.0 us
+sin             11.5 us    97.5 us
+atan            11.1 us   122.9 us
+atan2           12.5 us   192.4 us
+```
+
+Only the transcendentals pay for the promotion; a double multiply runs at the speed of a single one.
+The methods a Float can reach as a second argument are `atan2`, `hypot` and `ldexp`.
+`ldexp` pays a different way, since scaling by a power of two is cheap in either precision: `Cumo::NMath.ldexp(a, 2.0)` takes 268.0 us against 12.6 us for `Cumo::NMath.ldexp(a, 2)`, and the difference there is the doubled arrays it has to allocate rather than the arithmetic.
+
+Pass a 0-dimensional array instead of a Float and the call stays single precision.
+That is what `[]` hands back, so a scalar taken out of an array is already in the right form:
+
+```ruby
+two = Cumo::SFloat[2.0][0]     # a 0-dimensional Cumo::SFloat
+Cumo::NMath.atan2(a, two)      #=> Cumo::SFloat, 14.9 us against 219.3 us
+```
+
+Naming the module directly works too, under both libraries:
+
+```ruby
+Cumo::SFloat::Math.atan2(a, 2.0)   #=> Cumo::SFloat
+```
+
+The 0-dimensional form has no effect under Numo, where `[]` returns a Ruby Float.
+
 ### Select a GPU device ID
 
 Set the `CUDA_VISIBLE_DEVICES=id` environment variable, or
@@ -201,7 +297,14 @@ ln -sf "$HOME/opt/ccache/bin/ccache" "$HOME/opt/ccache/bin/nvcc"
 bundle exec env CUMO_NVCC_GENERATE_CODE=arch=compute_60,code=sm_60 rake compile
 ```
 
+Separate the entries with a space to build for more than one architecture:
+
+```
+bundle exec env CUMO_NVCC_GENERATE_CODE="arch=compute_75,code=sm_75 arch=compute_121,code=sm_121" rake compile
+```
+
 This is useful even on development because it makes it possible to skip JIT compilation of PTX to cubin during runtime.
+Without it, and without an `nvidia-smi` to read the local compute capability from, the build covers every architecture the CUDA version supports.
 
 ### Run tests with gdb