cumo 0.9.0 → 0.11.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 0f27ebdaa195fa967c144d6178ad3f9fbeca551162f8edacac205d5edd5d0fb6
-  data.tar.gz: 3d87faa55ab7c4c0e52bee3afdb0116d9d463fa778c408a46e200bff0cabdf81
+  metadata.gz: d50aabaf6d47d63e97fc6c68726cac94af2ecb8b737a3441de618860d2f5ab6a
+  data.tar.gz: 9d7bfc02015586d0ef533802be1d48558abf93fddf362ce445c55b0b4713ab7b
 SHA512:
-  metadata.gz: 4b498a760e59725c15aa3e651ecbada01e52517833fd3132a860dbfe459e354f4afba47207cc33f2c94963f58f19a8a1d7e52ce722e51065876d637c8ddfc94b
-  data.tar.gz: bbda42b2ef7ce7ea882982a015bec06ff058ad42a2b72e4c8953e75facfcb64657c8da4b69d63666cf4c06d3c265dc7e0acf54c431450f181b39cbeebed9560e
+  metadata.gz: 99f47260f8dbbb9320d2f74984262e328fd42c2dac0321300b6087712dd129d03f80d65a2089f12f624ea2814e59eb8400748c9bf08f3937b09c0bceb452f417
+  data.tar.gz: a3ccc77b320c0bfdaa64abed339cc22ce5099638f8a52e9abc5a44a1e144295f56581b43ebf99f5a431d54bbe7bed2fe590975abd59cb6b7e1a9008878c91f23

data/CHANGELOG.md

@@ -1,3 +1,96 @@
+# 0.11.0 (2026/09/27)
+
+Breaking changes:
+
+* `sort`, `sort_index`, `cumsum` and `cumprod` raise `ArgumentError` on `keepdims:`, which they accepted and ignored, since they keep the shape of their input. They take `axis:` and `nan:`, and `nan:` stays on every type as it does for `sum` and `median`. Numo still accepts `keepdims:` here without a word (PR #552)
+* `reverse` raises `ArgumentError` on `keepdims:` and `nan:`, which it accepted and ignored, and takes `axis:` only. Numo still accepts both without a word (PR #551)
+
+Fixes:
+
+* Fix a store from an overlapping part of the same array, such as `a[0...-k] = a[k..]`, writing wrong values, since threads read elements other threads had already overwritten. With 10 million SFloat elements and a shift of 1024 every run was wrong, on every path and every dtype. The source is now copied first when it shares an element with the destination, which gives numpy's answer (PR #553, PR #542)
+* Fix `ElementwiseKernel` and `ReductionKernel` racing when an input shares memory with an output but is not read element for element, such as a contiguous input shifted against its output. Such an input is copied first (PR #535)
+* Fix a leak of the new shape when `expand_dims` cannot allocate its strides (PR #563)
+
+Changes:
+
+* `cumsum` and `cumprod` down axis 0, or along a transposed view, gather and scatter their buffer through a shared-memory tile: on a 2048 x 2048 SFloat, `a.cumsum(axis: 0)` spends 133 us on the GPU where it spent 315, and the gather for `a.transpose.cumsum(axis: 1)` takes 51 us where it took 154 (PR #565)
+* An elementwise op on a view built on an index array gathers it into a buffer a typed element at a time rather than with a memcpy of a run-time size: `a[idx, true] + 1` on a 512 x 512 SFloat spends 2.75 us in the gather where it spent 5.2 (PR #564)
+* Say in the API docs of `MemoryPool.used_bytes` and `total_bytes` what each counts and when, and in the README that `used_bytes` waits for the garbage collector where CuPy's `used_bytes()` and PyTorch's `memory_allocated()` do not (PR #562)
+* `ElementwiseKernel` and `ReductionKernel` accept `Cumo::HFloat` and `Cumo::BFloat` as `float16` and `bfloat16`, CUDA's `__half` and `__nv_bfloat16`. A Ruby number handed to one is rounded on the host the way `Cumo::HFloat[x]` and `Cumo::BFloat[x]` round it (PR #561)
+* CI builds against CUDA 13.4.1 and 12.9.2 where it built against 13.2.1, and keeps 12.8.1 (PR #560)
+* `concatenate`, and `hstack`, `vstack` and `dstack` through it, write every part in one kernel launch when every part is a contiguous array of the result's class, and no longer fill the result with zeros first: `hstack` of six SFloat parts of 1500 x 64 takes 5.2 us where it took 18.3 (PR #559)
+* Elementwise ops, bias adds and math functions such as `gelu` move 16 bytes of each operand a thread when the last axis is contiguous in every operand: at 1500 x 384 and 1500 x 1536 SFloat, an add takes 4.18 us where it took 5.67, a bias add 15.1 where it took 27.4, and `gelu` 16.7 where it took 23.7. An HFloat add is 2.3 times faster and an Int8 add 3.6 times. Integer `div`, `mod` and `reciprocal`, and DComplex, keep the old loop (PR #558)
+* Say in the README and in comments why one spelling is faster than another rather than the timings of one GPU, which read as wrong on another card (PR #557, PR #556)
+* `layer_norm` and `rms_norm` hold a row in registers when it divides into sixteen-byte vectors and fits in eight of them a thread, and read it once: `layer_norm` over 1500 x 384 takes 6.7 us where it took 18.5. `layer_norm` takes the variance about the mean in a second pass, which is also more accurate, and a row whose sum overflows near the top of the range stays finite (PR #555)
+* `softmax` holds a row of up to 16 elements a thread in registers and reads it once: 1500 x 1500 takes 18.8 us where it took 34.4. SFloat and DFloat answers move by a few ulp, and HFloat and BFloat come closer to the reference, since `exp` is no longer rounded to half precision before the scale (PR #554)
+* `sort`, `sort_index`, `cumsum`, `cumprod` and `reverse` accept an empty array and answer an empty array of the same shape, where they raised. A missing axis still raises `DimensionError` (PR #550, PR #549)
+* A copy that the wide byte copy cannot take, as reductions, `argmax` and `sort` make of a non-contiguous input, goes a typed element at a time: `argmax` over a column slice runs 1.9 times faster and a copy of a column slice 1.3 times (PR #548)
+* A kernel over a view of 2 to 4 dimensions whose offsets fit an int32 does its address arithmetic in 32 bits: a store with a cast, `abs` or `clip` on such a view runs 1.17 to 1.32 times faster (PR #547)
+* A DFloat `logseq` of base 10 or 2 calls `exp10` or `exp2` rather than `pow`: 200,000 elements take 20 us where they took 84 (PR #546)
+* The rank switch of every generated kernel launcher comes from one helper in the kernel generator, and the generated sources are token for token what they were (PR #545)
+* `logseq` writes a strided, index or transposed view in one launch rather than one per row: a 6250 x 32 DFloat view takes 87 us where it took 14 ms (PR #544)
+* A reduction along a last axis of 16 elements or fewer gives a row that spans more than one 32-byte sector two threads: `Cumo::SFloat.ones(1536, 64)[true, 0...16].sum(axis: 1)` takes 2.18 us where it took 4.20, and `max`, `mean`, `mulsum` and `argmax` over short rows run 1.2 to 1.8 times faster (PR #543)
+* A store between views of the same type moves 16, 8, 4 or 2 bytes a thread over a run contiguous and aligned on both sides, through one set of kernels the copy behind `dup` shares: the store of an im2col unfold runs about 2 times faster (PR #541)
+* `cumsum` and `cumprod` no longer wait for the GPU. A 1-D scan shorter than 8192 elements ran on the host after a device synchronize, and every other shape waited for the stream to drain: `x = a * 1; x.cumsum + 1` takes about 9 us where it took 0.3 to 0.8 ms (PR #540)
+* `rand`, `rand_norm` and `seq` write a view built on an index array directly rather than through a buffer copied out and back, and so do `real=` and `imag=`: `rand` into a `[6250, 1024][true, idx64]` SFloat view takes 12.7 us where it took 80.7. An axis of length one merges with its neighbour, so a `[N, 1]` column runs what a flat array runs. A seed draws the same values as before (PR #539)
+* `ElementwiseKernel` and `ReductionKernel` read a reversed or stepped view, or a frozen array, where it is rather than copying it on every call, and `ReductionKernel` reads a transposed view the same way: a reduction along the first axis of a transposed 4000 by 4000 array no longer pays a copy that took longer than the reduction. An `ElementwiseKernel` input that is transposed is still copied, in tiles, which beats reading it across rows (PR #538, PR #535)
+* `ReductionKernel` over a leading axis gives a block up to 32 outputs that lie end to end, so a warp reads neighbouring outputs: a sum over axis 0 of a [4000, 4000] SFloat takes 0.169 ms where it took 0.413, what Cumo's own `sum` takes (PR #537)
+* `ElementwiseKernel` and `ReductionKernel` walk neighbouring dimensions that every array lays out end to end as one: a `ReductionKernel` sum over everything of a [160, 100, 1000] SFloat takes 0.162 ms where it took 0.193 (PR #536)
+* Fix a stream test that failed about once in a thousand runs, since its fill on a non-blocking stream was not ordered before its launch on the null stream (PR #534)
+* Say in the README where `CUMO_ALLOW_TF32` pays and where it does not: it speeds up matrix products and convolutions, buys nothing where reading the weights takes the time, and its rounding can push a training step past a tolerance single precision meets (PR #533)
+* A number stored with `a[...] = x`, `a[] = x` or `store(x)` goes through `fill`, one kernel launch with nothing reserved, where it filled a 0-dimensional array first. `fill` writes a view built on an index array or a mask directly rather than through a copy of the whole view (PR #532)
+
+# 0.10.0 (2026/09/22)
+
+Breaking changes:
+
+* `CUMO_COMPATIBLE_MODE`, `CUMO_SHOW_WARNING`, `CUMO_SHOW_WARNING_ONCE` and `CUMO_MEMORY_POOL` read `off`, `false` and `no` as a no, in any case, where anything but `OFF`, `0` and `NO` used to turn the flag on without a word. A value they do not recognise keeps the flag's default and warns, and an empty value reads as unset (PR #514)
+* An empty array keeps its shape through everything that goes through ndloop: `Cumo::Int32.new(0, 3).copy` and `(a + 1)` answer `[0, 3]` where they answered `[0, 0]`. `to_a` nests down to the first zero and no further, the way numpy's `tolist` does, so `(3, 0)` is `[[], [], []]` and `(0, 3, 4)` is `[]` (PR #513)
+* `Marshal.dump` of a subclass of `Cumo::RObject` writes its objects, where it wrote the raw addresses the buffer holds. Data the old dump wrote for such a subclass no longer loads, and what it held was addresses from a process that has since exited (PR #509)
+
+Fixes:
+
+* Fix `all?(axis: [])` answering a zero-dimensional `Cumo::Bit`, which is truthy whatever it holds, for an array it is false for. Whether an axis was named is now read off the reduction rather than off the list (PR #512)
+* Fix `all?`, `any?` and `none?` answering a zero-dimensional `Cumo::Bit` when a default keyword such as `axis: nil` or `keepdims: false` is written out, so `Cumo::Bit[1, 1, 0].all?(axis: nil)` read as true (PR #510)
+* Fix an axis reduction of an empty `Cumo::Bit` answering an empty array: `Cumo::Bit.new(0, 3).all?(axis: 0)` answers `[1, 1, 1]` and `count_true(axis: 0)` answers `[0, 0, 0]` (PR #508)
+* Fix the five `Cumo::Bit` reductions answering false, or an Integer, for an empty array before reading their arguments. `Cumo::Bit[].all?` is true and `any?` false, as Ruby's are, `count_true` answers a `UInt64`, and an axis that is not there raises `DimensionError` (PR #507)
+* Fix `CastError` for a subclass of a dtype, which broke `store`, arithmetic and the reductions, and put an element of one into an RObject array as one object repeated (PR #506)
+* Fix `Device#with` running its block on the device it was already on, whichever device it was given (PR #499)
+* Fix `reshape` writing the new shape over whatever the receiver's `dup` answered, which could leave an array claiming a size its buffer cannot hold. The answer `dup` gives has to be the same class, own its data and not be frozen (PR #494)
+* Fix `gemm` walking a copy of a non-contiguous operand with the original's dimensions when `dup` answered something else (PR #493)
+* Fix `marshal_dump` and `to_binary` handing out memory past the end of the array when a view's `dup` answered another shape, class or layout (PR #492)
+* Fix `median` and `sort` ignoring the axes a view marked with `:sum`, so `f[:sum, true].median` answered the whole array. `median` keeps its receiver rather than standing its copy in its place, and no longer copies an operand it only reads, so the pool holds less afterwards (PR #489, PR #488, PR #487)
+* Fix `Cumo::Bit#copy` answering its own receiver when the receiver is inplace, which let a write to the copy reach the original and gave every reduction that copies an index-backed view first the wrong rows (PR #486)
+* Fix the interpreter abort when `all?` or `any?` reduces a `[:sum, true]` view (PR #485)
+
+Changes:
+
+* `compile_with_cache` answers the same module for the same source, options and device for the rest of the process, so it can be called wherever the kernel is launched: a repeated call costs 0.0009 ms where it read the cubin back and loaded a new module for 0.033 ms, nine times the launch itself. A module nobody holds any more is unloaded when Ruby collects it, and `Cumo::CUDA::Compiler.clear_modules` makes the next call read the disk cache or compile again (PR #531)
+* The ports of the CuPy examples live in `examples/`, one Ruby file per original, with `backend.rb` running the CPU side on Numo and the GPU side on Cumo under `GPU=1`. The gem leaves them out (PR #530)
+* Drop a dead first definition of `Compiler#save_cache` that `ruby -W` warned about (PR #529)
+* Reading a result back from the device copies the range it needs into pinned host memory rather than reading managed memory in place, which faulted the page over and back once per read where a small block shares its page with others: `Float(r.sum)` takes 0.012 ms where it took 0.885, `to_a` of eight elements 0.016 ms where it took 0.97, and `to_binary` of 8 MB 2.3 ms where it took 7.6. `to_a`, `each`, `each_with_index`, `map_with_index`, `inspect`, `format`, `extract_cpu`, `aref_cpu`, `to_binary` and the RObject marshal all take the copy, and a block that writes the array it is walking still sees what it wrote (PR #528)
+* `rand` and `rand_norm` fill an array whose last axis has length one in one kernel launch rather than one per row: a `[1048576, 1]` column takes 0.018 ms where it took 1799. A seed produces the same values as before whatever the layout (PR #527)
+* A module answers a template kernel such as `kernel<float>` by its name expression: `NVRTCProgram.new` and `compile_with_cache` take `name_expressions:`, the mangled names are cached beside the cubin, and `Module#get_function` looks an expression up before asking the driver. Arrays of structs are arrays whose trailing axes are the fields, and a struct by value is a packed String. The cubin cache had been writing empty files where `/tmp` and the cache directory are different file systems, so every `compile_with_cache` compiled again; it now writes next to its place (PR #526)
+* Add `Cumo::CUDA::PinnedMemory`, a page-locked host buffer read and written as bytes, and `NArray#set` and `NArray#get`, which copy through it asynchronously on the current stream or on `stream:` the way `cupy.ndarray.set` and `get` do. A copy in flight keeps both ends until it is done (PR #525)
+* Add `Cumo::CUDA::Stream` and `Cumo::CUDA::Event`, after `cupy.cuda`: `Stream.new`, `with`, `use`, `synchronize`, `record`, `wait_event`, `Stream.null`, `Stream.current`, `Event.new`, `record`, `synchronize`, `done?` and `Cumo::CUDA.get_elapsed_time`, and `Function#launch` takes `stream:`. `Stream#with` orders the block after what the previous stream had queued and waits for the stream on the way out (PR #524)
+* Every kernel, copy, cuBLAS and cuDNN call, thrust and CUB call goes to the current stream of the thread, which is the null stream until a `Stream` is used, so a stream a caller picks reaches all of it. `Runtime.cudaStreamCreateWithFlags`, `cudaStreamDestroy`, `cudaStreamSynchronize` and `current_stream=` are bound (PR #523)
+* Add `Runtime.cudaDeviceCanAccessPeer` and `Device#can_access_peer?`. A CUDA error raised for a device that is not there no longer stays as the runtime's last error for the next kernel launch to read as its own (PR #521)
+* The CI ccache ceiling is 2 GB, since one build no longer fit in 1 GB and the CUDA 12 jobs compiled again what had been evicted (PR #522)
+* Add `Cumo::CUDA::ReductionKernel`, a reduction written as a map, a reduce and a post expression with an identity, called with `axis:` and `keepdims:`, after CuPy's. This is the last half of #106 (PR #520)
+* Add `Cumo::CUDA::ElementwiseKernel`, a kernel written as one expression over a parameter list, with one-letter types resolved from the arguments and broadcasting, compiled once per set of types, after CuPy's. Bit, half precision and complex arguments are refused for now (PR #519)
+* A kernel compiled with NVRTC can be launched: `Module#get_function` answers a `Cumo::CUDA::Function` whose `launch` takes the arguments, the grid, the block and the dynamic shared memory. An NArray hands over its device pointer, an Integer goes as a `long long`, a Float as a `double`, and a String as its bytes. On CUDA 12.4 or later the count and the sizes of the arguments are checked against the kernel before the launch (PR #518, PR #500)
+* `CUMO_ALLOW_TF32` puts single precision `gemm` on the tensor cores as well as convolutions: a `[4096, 4096]` SFloat gemm goes from 17.8 to 28.4 TFLOP/s, and its answer moves from 4e-07 to 3e-04 of a double precision reference. Off, which is the default, answers bit for bit what it did. The flag is read in `cumo.c` as `Cumo.allow_tf32?` rather than in `cudnn.c`, so a build without cuDNN has it too (PR #517, PR #516)
+* Every dtype's `gemm` goes through `cublasGemmStridedBatchedEx`, the entry point that can name a compute type. The answers are the same bytes, and no change in speed could be told apart from the noise (PR #515)
+* `all?`, `any?` and `none?` on an empty `Cumo::Bit` with no axis named answer the identity without reserving anything or waiting on the device: 0.045 us a call where it took 12.3 and 512 bytes (PR #511)
+* `sort_index`, and `sort` and `median` over a view, address by rank rather than through the run-time accessor: `sort_index` over a 4096 by 1024 SFloat takes 2.2 ms where it took 5.6, and an inplace `sort` over a column slice 2.0 ms where it took 6.5 (PR #505)
+* `cumsum` and `cumprod` along an axis send every row to one scan keyed on its row rather than starting a scan per row, and a view no single stride reaches is gathered by a kernel that addresses by rank: over a `[512, 2048]` SFloat a column slice takes 25.7 us where 0.9.0 took 50.7, and a transposed view 45.2 us where it took 308.3. A reversed view is read through its stride and costs what a flat one does (PR #504, PR #503, PR #498, PR #496)
+* The default cuDNN workspace ceiling is 128 MB rather than 8 MB, which reaches the faster convolution algorithms: the twenty convolutions of a ResNet-18 forward pass at batch 16 take 5.23 ms where they took 9.80, and a half precision convolution reaches its tensor core algorithms. The search reserves the whole ceiling while it runs, so the peak grows with it and goes back to the pool afterwards (PR #502)
+* A single precision convolution stays off the tensor cores unless `CUMO_ALLOW_TF32` says yes, where cuDNN's default let the algorithm search move it there and round the operands to a 10 bit significand (PR #501)
+* Say in the README that `contiguous?` answers true for a view whose length-one axis Numo counts as breaking contiguity, that copying a transposed operand beats handing it to cuBLAS as it is for most shapes, what the memory numbers of the pool and of `nvidia-smi` mean, where a bfloat16 integer index goes past 256, and that `quantize_symmetric` breaks ties away from zero where numpy and torch round to even (PR #497, PR #495, PR #491, PR #484)
+* Say what routing `silu` through `sigmoid` would change, rather than a fraction that no dtype matches (PR #483)
+* Add `Cumo::NMath.sigmoid`, which keeps a value where `1 / (1 + exp(-x))` written out gives out: single reaches zero at -104 rather than -89, double at -745 rather than -710 (PR #482)
+* Add `Cumo::NMath.softplus`, which does not overflow where `log(1 + exp(x))` written out does, and answers `x` to the last bit past 11.09 in half and 88.7 in single (PR #481)
+
 # 0.9.0 (2026/09/19)
 
 Breaking changes: