cumo 0.14.0 → 0.15.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: f09e6c09816ea16f1a27425e44170c7ea191b771f0a3b26b2b7298a1e53b6e6c
-  data.tar.gz: f4739b41e92d5f5920ea694393f3ab71241e9b428a29416db71e91d5fcb0f13e
+  metadata.gz: d6fcb0462dc2821749f43c243267e8eef32c5362d2a647769c3f93ccc048ddd0
+  data.tar.gz: 131ec1045f331591fb3f1c522824ea1e14c56fad40091bccacde73bb7f64a649
 SHA512:
-  metadata.gz: d58332736efabf28caeec996882ceb99435885e41d017a40911c6a85346a4f03428e32cb7cae32e8dae9633e47f7497dc35075309ec32d61d607504ef5f61fb1
-  data.tar.gz: d276cc4b4d19203ab34bfae56e96dd6e3220a780f07d3da8934e9c9241336fcb31c2040e013a7e461b47ec93461dcff21ca126597742c184e52221c0112bf196
+  metadata.gz: dfbb78f6f9ea625ae2848210722aa58106e6efd80af31d38e239585def253b7fb0e00426d18804e4177040492469d3498eaf61dca0cb7880ee482bd1c9669d0e
+  data.tar.gz: 80c944b3b2a544c593a221c9600c2a8cc441b273e37dc77f3ca1bbb738e381e1fcc0db42cbc66265f6d75832ee0ca396b008ffec7ff666549db313d386b82e81

data/CHANGELOG.md

@@ -1,8 +1,42 @@
+# 0.15.0 (2026/11/02)
+
+Breaking changes:
+
+* A subscript of an empty array is read as it is for any other array, where every subscript of an empty array raised `Cumo::NArray::ShapeError`. `a[true, 1]` of a 0 x 3 array answers an empty view, and a subscript that names an element raises as it does for any array, so `a[0]` raises `IndexError`. An empty range now selects nothing when it starts where an element could be inserted, as with `Array#[]`, so `b[2...2, true]` of a 2 x 3 array answers a 0 x 3 view where it raised `RangeError`. The README lists where this differs from Numo (PR #641)
+* `Cumo::RObject` no longer has `store_binary`, `to_string`, `hton` or `Cumo::RObject.from_string`, and `set` with a String raises `NoMethodError` for it. They read or wrote the object references of an RObject as raw bytes, so the next garbage collection could crash, and `to_string` gave out heap addresses. Calling the methods of `Cumo::NArray` on an RObject through `bind_call` raises `TypeError` (PR #638)
+
+Fixes:
+
+* Fix `require "cumo"` under `ruby -w` warning "method redefined; discarding old %". Cumo no longer defines `Range#%` again, which Ruby has had since 2.6 and which works the same (PR #643)
+* Fix a crash, possible since 0.7.0, when storing an Array whose NArray row of another type is followed by a row that raises, such as one with the wrong number of dimensions. The garbage collector could free the cast copy of the row before the store let go of it (PR #642)
+* Fix a range with a negative step and an excluded or missing end selecting the wrong elements: `(4...1).step(-1)` selected 5 elements of a 5-element array where it should select 3, and `(3..).step(-1)` none. Fix a stepped range that selects nothing returning elements, as `(1..0).step(2)` returned one and `(3...3).step(-1)` two (PR #641)
+* Fix `to_a` and `format_to_a` aborting when Cumo is compiled with `RUBY_DEBUG` against a Ruby whose headers check the index of `RARRAY_AREF`, as Ruby master does since ruby/ruby@61261bf5c4 (PR #640)
+
+Changes:
+
+* CI builds against CUDA 13.4.3 on Ubuntu 26.04 where it built against 13.4.1 on Ubuntu 24.04, and the CUDA 12.9.2 job builds for sm_121 where it built for sm_120 (PR #639)
+* The first `sort`, `median` or `sort_index` of a type loads the sort kernels of that type alone, where the first of a process loaded those of all twelve types. On SFloat the first `sort` is about 5.7 times faster, `median` about 9 times and `sort_index` about 3 times (PR #636)
+* The first `cumsum` or `cumprod` of a process no longer loads every sort kernel, which makes the first SFloat `cumsum` about 2.1 times faster and keeps the peak RSS 57 MB lower (PR #635)
+* `sum`, `mean`, `max`, `min` and the `dot` of two vectors have a kernel module of their own, so the first of them on a type no longer loads every reduction of the type. The first SFloat `sum` or vector `dot` of a process is about 8.6 times faster (PR #634)
+
+# 0.14.1 (2026/10/22)
+
+Fixes:
+
+* Fix a host loop such as `each` yielding the old values of an array after its block wrote the array through `store_binary`, an in-place `cumsum` or `cumprod`, or an in-place `map` or `map_with_index`, also one paused by an Enumerator (PR #631, PR #630)
+* Fix an in-place `map_with_index` over a view that reaches an element more than once, such as `x[[0, 0, 0]]`, reading the element as it was before its own write. It now reads its own write, as `map` and numo do (PR #631)
+* Fix the release gem recording the tag of the previous release as `Cumo::REVISION`, as the 0.14.0 gem does with `"v0.13.0-11-gafd8ead9"`. `rake release` now makes the tag before it builds the gem, so this gem gives `"v0.14.1"` (PR #626)
+
+Changes:
+
+* `+`, `-`, `*` and `/` between a complex array and a real array of the same precision, in either order, read the real array as it is rather than casting it to a complex array first. On 4M elements this makes SComplex with SFloat about 1.7 to 1.8 times faster, and DComplex divided by DFloat about 1.4 times. The answers are the same but for the sign of a NaN (PR #632)
+* A host loop such as `each` whose block writes to the device copies the rest of the row, or of the array, again at once when copying the elements one at a time would cost more. With one write early in an `each_with_index` over 100,000 elements, the loop is about 90 times faster and about as fast as with no write. With a write at every 100th element it is about 12 times faster, and with a write at every other element about 16% slower (PR #631)
+
 # 0.14.0 (2026/10/18)
 
 Breaking changes:
 
-* `cumo.h` no longer defines `CUMO_VERSION` or `CUMO_VERSION_CODE`. `Cumo::VERSION` and `Cumo::NArray::VERSION` are defined in Ruby, and the new `Cumo::REVISION` tells a release gem from one built from a checkout: it gives `"v0.14.0"` for this release and something like `"v0.14.0-3-g1a2b3c4"` for a gem built at a later commit (PR #617)
+* `cumo.h` no longer defines `CUMO_VERSION` or `CUMO_VERSION_CODE`. `Cumo::VERSION` and `Cumo::NArray::VERSION` are defined in Ruby. The new `Cumo::REVISION` tells a release gem from one built from a checkout. A release gives its tag, and a gem built at a later commit gives something like `"v0.14.0-3-g1a2b3c4d"`. The 0.14.0 gem was built before its tag was made, so it gives `"v0.13.0-11-gafd8ead9"`. `rake release` makes the tag before the gem from the next release on (PR #617, PR #626)
 
 Fixes:
 

data/README.md

@@ -198,6 +198,30 @@ Cumo::DFloat.new(1, 3).seq.transpose.contiguous?               #=> true
 Code that runs against both libraries should not read `contiguous?` and expect the same answer.
 Where it wants a contiguous array it can ask for one, since `dup` answers one in either library.
 
+#### Subscripts That Select Nothing
+
+Numo raises `ShapeError` for every subscript of an empty array.
+Cumo returns the empty view numpy does, and raises only for a subscript that names an element.
+
+```ruby
+a = Cumo::DFloat.new(0, 3)
+a[true, 1].shape   #=> [0], where Numo raises ShapeError
+a[0]               # raises IndexError, where Numo raises ShapeError
+```
+
+A range that selects nothing only has to start where an element could be inserted, as with `Array#[]`.
+Numo raises `RangeError` when such a range starts at either end of an axis.
+
+```ruby
+b = Cumo::DFloat.new(2, 3).seq
+b[2...2, true].shape   #=> [0, 3], where Numo raises RangeError
+b[3...3, true]         # raises RangeError in both
+```
+
+A stepped range that selects nothing also selects nothing in Cumo.
+Numo returns one element for `(1..0).step(2)` and two for `(3...3).step(-1)`.
+`compatible_mode` does not change any of this.
+
 ### Keeping Scalars On The Device
 
 The 0-dimensional return is what lets an iterative loop stay on the GPU.

data/Rakefile

@@ -2,6 +2,35 @@
 
 require "bundler/gem_tasks"
 require "rake/testtask"
+require "rubygems/package"
+
+Rake::Task["release"].clear
+desc "Tag the version, build the gem at the tag, and push the branch, the tag and the gem"
+task :release, [:remote] do |_, args|
+  git = ->(*argv) { IO.popen(["git", *argv], err: File::NULL, &:read).strip }
+  version = File.read(File.join(__dir__, "lib/cumo/version.rb"))[/VERSION = "([^"]+)"/, 1]
+  tag = "v#{version}"
+  gem_file = "pkg/cumo-#{version}.gem"
+  remote = args[:remote] || "origin"
+  raise "There are files that need to be committed first." unless system("git", "diff", "--quiet") && system("git", "diff-index", "--quiet", "--cached", "HEAD")
+
+  tagged = git.call("rev-parse", "-q", "--verify", "#{tag}^{commit}")
+  raise "#{tag} is on #{tagged[0, 8]}, not on HEAD." unless tagged.empty? || tagged == git.call("rev-parse", "HEAD")
+
+  sh "git", "tag", "-m", "Version #{version}", tag if tagged.empty?
+  begin
+    mkdir_p "pkg"
+    sh "gem", "build", "cumo.gemspec", "--output", gem_file
+    revision = Gem::Package.new(gem_file).spec.metadata["cumo_revision"]
+    raise "#{gem_file} records #{revision.inspect} as cumo_revision, not #{tag.inspect}." unless revision == tag
+  rescue StandardError
+    sh "git", "tag", "-d", tag if tagged.empty?
+    raise
+  end
+  sh "git", "push", remote, "HEAD"
+  sh "git", "push", remote, "refs/tags/#{tag}"
+  sh "gem", "push", gem_file unless ENV["gem_push"] == "no"
+end
 
 Rake::TestTask.new(:test) do |t|
   t.libs << "test"
@@ -10,7 +39,9 @@ Rake::TestTask.new(:test) do |t|
 end
 
 require "rake/extensiontask"
-Rake::ExtensionTask.new("cumo")
+ext = Rake::ExtensionTask.new("cumo")
+# The Makefile lists an object for each kernel part, so a new part needs extconf.rb to run again.
+file "#{ext.tmp_dir}/#{ext.platform}/#{ext.name}/#{RUBY_VERSION}/Makefile" => "ext/cumo/narray/gen/kernel_parts.rb"
 
 desc 'Build the C++ memory pool test harness without running it (running it needs a GPU)'
 task :build_ctest do

data/ext/cumo/cuda/cub.cu

@@ -0,0 +1,26 @@
+#include <cub/util_device.cuh>
+#include "cumo/template_kernel.h"
+
+namespace {
+
+__global__ void cumo_cub_ptx_version_probe() {}
+
+}  // namespace
+
+// CUB learns each device's PTX version once, from the attributes of an empty
+// kernel. Each translation unit has its own empty kernel, but the link keeps
+// one copy of the code that asks. The query loads the module that copy came
+// from, which can be a large one the caller never runs. This module has one
+// kernel built with the same flags, so it gives the same version at little cost.
+extern "C" void
+cumo_cuda_cub_prime_ptx_version(void)
+{
+    cub::GetPerDeviceAttributeCache<cub::PtxVersionCacheTag>()(
+        [](int& ptx_version) {
+            cudaFuncAttributes attrs = {};
+            cudaError_t status = cudaFuncGetAttributes(&attrs, (const void*)cumo_cub_ptx_version_probe);
+            ptx_version = attrs.ptxVersion * 10;
+            return status;
+        },
+        cub::CurrentDevice());
+}

data/ext/cumo/extconf.rb

@@ -55,7 +55,6 @@ narray/ndloop
 narray/ndloop_kernel
 narray/data
 narray/data_kernel
-narray/sort_kernel
 narray/math
 narray/SFMT
 narray/struct
@@ -70,6 +69,7 @@ cuda/cudnn
 cuda/cudnn_impl
 cuda/cusolver
 cuda/cufft
+cuda/cub
 )
 %w[bit int8 int16 int32 int64 uint8 uint16 uint32 uint64 hfloat bfloat sfloat dfloat scomplex dcomplex robject].each do |type_name|
   srcs << "narray/types/#{type_name}"

data/ext/cumo/include/cumo/cuda/runtime.h

@@ -33,10 +33,11 @@ extern size_t cumo_cuda_sync_epoch;
 // and reads again once the count moves.
 extern size_t cumo_cuda_launch_epoch;
 
-static inline void
+// Answers the count before the write.
+static inline size_t
 cumo_cuda_runtime_note_device_write(void)
 {
-    cumo_atomic_size_fetch_add(&cumo_cuda_launch_epoch, 1);
+    return cumo_atomic_size_fetch_add(&cumo_cuda_launch_epoch, 1);
 }
 
 // A pinned host buffer for reading device memory back: a copy into pinned

data/ext/cumo/include/cumo/ndloop.h

@@ -104,6 +104,13 @@ typedef struct {
 // it will read next again if device memory was written since it was staged.
 void cumo_na_ndloop_refresh_next(cumo_na_loop_t *lp, const void *next, size_t bytes);
 
+// For a function that yields to Ruby and writes an array that was there
+// before it, which cumo_na_ndloop_writes_existing answers: counts the element
+// it just wrote, so that a loop it yields to before it ends reads it. A staged
+// copy of its own has to hold the element already.
+int cumo_na_ndloop_writes_existing(cumo_na_loop_t *lp);
+void cumo_na_ndloop_wrote(cumo_na_loop_t *lp);
+
 // spec of user function
 typedef struct {
     cumo_na_iter_func_t func;    // user function

data/ext/cumo/{narray/sort_kernel.cu → include/cumo/sort_kernel.h}

@@ -1,3 +1,6 @@
+#ifndef CUMO_SORT_KERNEL_H
+#define CUMO_SORT_KERNEL_H
+
 #include "cumo/narray_kernel.h"
 #include "cumo/indexer.h"
 #include "cumo/template_kernel.h"
@@ -146,6 +149,7 @@ cudaError_t sort_pairs(const Key* kin, Key* kout, const Value* vin, Value* vout,
                        int64_t total, int64_t n_rows, int64_t row_len) {
     size_t bytes = 0;
     cudaError_t st = cudaSuccess;
+    cumo_cuda_cub_prime_ptx_version();
     if (n_rows == 1) {
         st = cub::DeviceRadixSort::SortPairs(nullptr, bytes, kin, kout, vin, vout, total, 0, sizeof(Key) * 8, cumo_cuda_stream());
         if (st != cudaSuccess) { return st; }
@@ -168,6 +172,7 @@ template <typename Key>
 cudaError_t sort_keys(const Key* kin, Key* kout, int64_t total, int64_t n_rows, int64_t row_len) {
     size_t bytes = 0;
     cudaError_t st = cudaSuccess;
+    cumo_cuda_cub_prime_ptx_version();
     if (n_rows == 1) {
         st = cub::DeviceRadixSort::SortKeys(nullptr, bytes, kin, kout, total, 0, sizeof(Key) * 8, cumo_cuda_stream());
         if (st != cudaSuccess) { return st; }
@@ -329,6 +334,12 @@ __global__ void iota_kernel(I* out, int64_t n) {
     }
 }
 
+// The positions go to CUB as this rather than as a bare integer, so that
+// SortPairs is a function of this file alone. With a bare integer it is the
+// same function for every type whose key has the same width, and the link
+// keeps one, so SFloat would launch the kernels of the UInt32 module.
+template <typename I> struct position { I value; };
+
 // perm[i] is the position the i-th smallest element sat at, and the index
 // array holds the number to answer for that position.
 template <int NDIM, typename I>
@@ -373,13 +384,13 @@ void sort_index_rows(cumo_na_iarray_t* a, cumo_na_indexer_t* indexer, cumo_na_ia
         key_t* kout = (key_t*)cumo_cuda_runtime_malloc(sizeof(key_t) * total);
         float_key_kernel<T><<<grid_dim, block_dim, 0, cumo_cuda_stream()>>>(data, kin, total);
         cumo_check_launch_holding(kin, kout, pin, pout, gathered);
-        cumo_check_status_holding(sort_pairs(kin, kout, pin, pout, total, n_rows, row_len),
+        cumo_check_status_holding(sort_pairs(kin, kout, (const position<I>*)pin, (position<I>*)pout, total, n_rows, row_len),
                              kin, kout, pin, pout, gathered);
         cumo_cuda_runtime_free((char*)kout);
         cumo_cuda_runtime_free((char*)kin);
     } else {
         T* kout = (T*)cumo_cuda_runtime_malloc(sizeof(T) * total);
-        cumo_check_status_holding(sort_pairs(data, kout, pin, pout, total, n_rows, row_len),
+        cumo_check_status_holding(sort_pairs(data, kout, (const position<I>*)pin, (position<I>*)pout, total, n_rows, row_len),
                              kout, pin, pout, gathered);
         cumo_cuda_runtime_free((char*)kout);
     }
@@ -396,7 +407,7 @@ void sort_index_rows(cumo_na_iarray_t* a, cumo_na_indexer_t* indexer, cumo_na_ia
 
 } // namespace
 
-// float_key is only defined for the two float types, so the integer entry
+// float_key is only defined for the float types, so the integer entry
 // points must not instantiate it; the bool picks the branch at compile time.
 #define CUMO_DEF_SORT(name, type, is_float)                                                     \
     extern "C" void cumo_##name##_sort_kernel_launch(                                           \
@@ -420,15 +431,4 @@ void sort_index_rows(cumo_na_iarray_t* a, cumo_na_indexer_t* indexer, cumo_na_ia
         }                                                                                       \
     }
 
-CUMO_DEF_SORT(int8, int8_t, false)
-CUMO_DEF_SORT(int16, int16_t, false)
-CUMO_DEF_SORT(int32, int32_t, false)
-CUMO_DEF_SORT(int64, int64_t, false)
-CUMO_DEF_SORT(uint8, u_int8_t, false)
-CUMO_DEF_SORT(uint16, u_int16_t, false)
-CUMO_DEF_SORT(uint32, u_int32_t, false)
-CUMO_DEF_SORT(uint64, u_int64_t, false)
-CUMO_DEF_SORT(hfloat, cumo_half, true)
-CUMO_DEF_SORT(bfloat, cumo_bfloat, true)
-CUMO_DEF_SORT(sfloat, float, true)
-CUMO_DEF_SORT(dfloat, double, true)
+#endif /* ifndef CUMO_SORT_KERNEL_H */

data/ext/cumo/include/cumo/template_kernel.h

@@ -186,6 +186,8 @@ void cumo_cuda_runtime_check_kernel_launch(void);
 // The stream every launch and copy goes to. It is 0 until Stream#with says
 // otherwise, and thread local, so one thread's choice is not another's.
 struct CUstream_st* cumo_cuda_stream(void);
+// Call before any CUB or Thrust algorithm. Defined in cuda/cub.cu, which says why.
+void cumo_cuda_cub_prime_ptx_version(void);
 
 // The forms to call while scratch is held, which free what they are given
 // before they raise. Pass every buffer that is still outstanding, including the

data/ext/cumo/narray/data.c

@@ -158,6 +158,9 @@ cumo_na_swap_byte(VALUE self)
                           CUMO_STRIDE_LOOP|CUMO_NDF_INDEXER_LOOP|CUMO_NDF_ACCEPT_BYTESWAP,
                           1, 1, ain, aout };
 
+    if (rb_obj_is_kind_of(self, cumo_cRObject)) {
+        rb_raise(rb_eTypeError, "cannot swap bytes of %s", rb_obj_classname(self));
+    }
     // Cumo::Bit's element is one bit, so swapping the bytes of an element
     // changes nothing, and a loop stepping by the element stride would walk
     // one byte per bit, eight times past the end of the packed buffer. Only

data/ext/cumo/narray/gen/cogen_kernel.rb

@@ -41,14 +41,14 @@ erb_dir.unshift("tmpl_bit") if (type_name == "bit")
 erb_dir.map! { |d| File.join(thisdir, d) }
 
 TEMPLATE_KERNEL_PARTS = {
-  "accum" => %w[reduce reduce_nan],
-  "accum_index" => %w[reduce reduce_nan],
-  "accum_arg" => %w[reduce reduce_nan],
-  "accum_binary" => %w[reduce reduce_nan reduce_from],
-  "cum" => %w[reduce],
-  "layer_norm" => %w[reduce],
-  "rms_norm" => %w[reduce],
-  "softmax" => %w[reduce],
+  "accum" => %w[reduce reduce_rare reduce_nan],
+  "accum_index" => %w[reduce reduce_rare reduce_nan],
+  "accum_arg" => %w[reduce reduce_rare reduce_nan],
+  "accum_binary" => %w[reduce reduce_rare reduce_nan reduce_from],
+  "cum" => %w[reduce_rare],
+  "layer_norm" => %w[reduce_rare],
+  "rms_norm" => %w[reduce_rare],
+  "softmax" => %w[reduce_rare],
   "bit_count" => %w[reduce],
   "bit_reduce" => %w[reduce],
   "bit_stat" => %w[reduce],
@@ -59,10 +59,14 @@ TEMPLATE_KERNEL_PARTS = {
   "poly" => %w[misc],
   "quantize_symmetric" => %w[misc],
   "unary_ret2" => %w[misc],
+  "sort" => %w[sort],
 }.freeze
 
 MATH_TEMPLATES = %w[unary_s binary_s ternary_s frexp].freeze
 COMMON_MATH = %w[sqrt log log2 log10 exp exp2 sin cos tan tanh erf gelu gelu_tanh silu sigmoid softplus].freeze
+# mulsum is here for dot of two vectors.
+COMMON_REDUCE = %w[sum mean max min mulsum].freeze
+RARE_REDUCE = %w[prod ptp minmax var stddev rms abs_sum abs_max abs_min kahan_sum min_index max_index argmin argmax].freeze
 
 abort "unknown kernel part: #{$kernel_part}" unless KERNEL_PARTS.include?($kernel_part)
 undeclared_math = COMMON_MATH - File.read(File.join(thisdir, "spec.rb")).scan(/^\s*math "(\w+)"/).flatten
@@ -83,8 +87,13 @@ class ErbPP
     @children.select { |c| c.kernel_parts.include?($kernel_part) }
   end
 
-  def kernel_part?(nan: false, from: nil)
-    $kernel_part == (from ? "reduce_from" : nan ? "reduce_nan" : "reduce")
+  def kernel_part?(op = nil, nan: false, from: nil)
+    return $kernel_part == "reduce_from" if from
+    return $kernel_part == "reduce_nan" if nan
+    return $kernel_part == "reduce" if COMMON_REDUCE.include?(op)
+    return $kernel_part == "reduce_rare" if RARE_REDUCE.include?(op)
+
+    raise ArgumentError, "#{op.inspect} is in neither COMMON_REDUCE nor RARE_REDUCE"
   end
 
   def indexer_dims(narrow = false)

data/ext/cumo/narray/gen/kernel_parts.rb

@@ -1,14 +1,16 @@
 # frozen_string_literal: true
 
-KERNEL_PARTS = %w[main reduce reduce_nan reduce_from math math_rare compare cast misc].freeze
+KERNEL_PARTS = %w[main reduce reduce_nan reduce_from reduce_rare math math_rare compare cast misc sort].freeze
 MATH_TYPES = %w[hfloat bfloat sfloat dfloat scomplex dcomplex].freeze
+SORT_TYPES = %w[int8 int16 int32 int64 uint8 uint16 uint32 uint64 hfloat bfloat sfloat dfloat].freeze
 
 def kernel_parts_of(type_name)
-  return KERNEL_PARTS.first(4) if type_name == "robject"
+  return %w[main reduce reduce_nan reduce_from] if type_name == "robject"
 
   KERNEL_PARTS.reject do |part|
     (%w[math math_rare].include?(part) && !MATH_TYPES.include?(type_name)) ||
-      (%w[compare misc].include?(part) && type_name == "bit")
+      (%w[compare misc reduce_rare].include?(part) && type_name == "bit") ||
+      (part == "sort" && !SORT_TYPES.include?(type_name))
   end
 end
 

data/ext/cumo/narray/gen/spec.rb

@@ -100,8 +100,12 @@ end
 
 if is_object
   undef_singleton_method "from_binary"
+  undef_singleton_method "from_string"
+  undef_method "store_binary"
   undef_method "to_binary"
+  undef_method "to_string"
   undef_method "swap_byte"
+  undef_method "hton"
   undef_method "to_network"
   undef_method "to_vacs"
   undef_method "to_host"

data/ext/cumo/narray/gen/tmpl/accum_arg_kernel.cu

@@ -119,7 +119,7 @@ extern "C" {
 #endif
 #endif
 
-<% if kernel_part? %>
+<% if kernel_part?("argmin") %>
 void cumo_<%=type_name%>_argmin_int<%=i%>_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_arg_split<dtype, idx_t, cumo_<%=type_name%>_argmin_int<%=i%>_impl>(*arg, cumo_<%=type_name%>_argmin_int<%=i%>_impl{});
@@ -133,7 +133,7 @@ void cumo_<%=type_name%>_argmin_nan_int<%=i%>_kernel_launch(cumo_na_reduction_ar
 }
 <% end %>
 
-<% if kernel_part? %>
+<% if kernel_part?("argmax") %>
 void cumo_<%=type_name%>_argmax_int<%=i%>_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_arg_split<dtype, idx_t, cumo_<%=type_name%>_argmax_int<%=i%>_impl>(*arg, cumo_<%=type_name%>_argmax_int<%=i%>_impl{});

data/ext/cumo/narray/gen/tmpl/accum_binary_kernel.cu

@@ -63,7 +63,7 @@ extern "C" {
 //<% (is_float ? ["","_nan"] : [""]).each do |nan| %>
 //<% ([nil] + from_types).each do |from| %>
 //<% sfx = from ? "_from_#{from[:id]}" : "" %>
-//<% if kernel_part?(nan: nan == "_nan", from: from) %>
+//<% if kernel_part?(name, nan: nan == "_nan", from: from) %>
 void <%="cumo_#{type_name}_#{name}#{nan}#{sfx}_kernel_launch"%>(cumo_na_reduction_arg_t* arg, cumo_na_iarray_t* in2)
 {
     cumo_reduce_zip_split<dtype, <%= from ? from[:ctype] : "dtype" %>, dtype, <%="cumo_#{type_name}_#{name}#{nan}#{sfx}_impl"%>>(*arg, *in2, <%="cumo_#{type_name}_#{name}#{nan}#{sfx}_impl"%>{});

data/ext/cumo/narray/gen/tmpl/accum_index_kernel.cu

@@ -119,7 +119,7 @@ extern "C" {
 #endif
 #endif
 
-<% if kernel_part? %>
+<% if kernel_part?("min_index") %>
 void cumo_<%=type_name%>_min_index_int<%=i%>_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_split<dtype, idx_t, cumo_<%=type_name%>_min_index_int<%=i%>_impl>(*arg, cumo_<%=type_name%>_min_index_int<%=i%>_impl{});
@@ -133,7 +133,7 @@ void cumo_<%=type_name%>_min_index_nan_int<%=i%>_kernel_launch(cumo_na_reduction
 }
 <% end %>
 
-<% if kernel_part? %>
+<% if kernel_part?("max_index") %>
 void cumo_<%=type_name%>_max_index_int<%=i%>_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_split<dtype, idx_t, cumo_<%=type_name%>_max_index_int<%=i%>_impl>(*arg, cumo_<%=type_name%>_max_index_int<%=i%>_impl{});

data/ext/cumo/narray/gen/tmpl/binary.c

@@ -10,6 +10,13 @@
 
 <% unless type_name == 'robject' %>
 void <%="cumo_#{c_iter}_kernel_launch"%>(cumo_na_iarray_t* a1, cumo_na_iarray_t* a2, cumo_na_iarray_t* a3, cumo_na_indexer_t* indexer, int* divzero);
+//<% if is_complex && %w[add sub mul div].include?(name) %>
+void <%="cumo_#{c_iter}_r_kernel_launch"%>(cumo_na_iarray_t* a1, cumo_na_iarray_t* a2, cumo_na_iarray_t* a3, cumo_na_indexer_t* indexer, int real_is_left);
+//<% end %>
+//<% complex_of = {'sfloat' => ['scomplex', 'cumo_cSComplex'], 'dfloat' => ['dcomplex', 'cumo_cDComplex']}[type_name] if %w[add sub mul div].include?(name) %>
+//<% if complex_of %>
+VALUE <%="cumo_#{complex_of[0]}_#{name}_real_left"%>(VALUE real, VALUE complex);
+//<% end %>
 //<% if %w[add sub mul div mod bit_and bit_or bit_xor left_shift right_shift copysign].include?(name) %>
 void <%="cumo_#{c_iter}_s_kernel_launch"%>(cumo_na_iarray_t* a1, dtype sv, cumo_na_iarray_t* a3, cumo_na_indexer_t* indexer, int* divzero, int scalar_is_left);
 //<% end %>
@@ -171,6 +178,40 @@ static void
 //<% end %>
 #undef check_intdivzero
 
+//<% if is_complex && %w[add sub mul div].include?(name) %>
+// The operands stay in the order they were written in, so that a shape
+// mismatch names them as it always did.
+static void
+<%=c_iter%>_r(cumo_na_loop_t *const lp)
+{
+    int real_is_left = *(int*)(lp->opt_ptr);
+    cumo_na_iarray_t a1 = cumo_na_make_iarray(&lp->args[real_is_left ? 1 : 0]);
+    cumo_na_iarray_t a2 = cumo_na_make_iarray(&lp->args[real_is_left ? 0 : 1]);
+    cumo_na_iarray_t a3 = cumo_na_make_iarray(&lp->args[2]);
+    cumo_na_indexer_t indexer = cumo_na_make_indexer(&lp->args[0]);
+
+    <%="cumo_#{c_iter}_r_kernel_launch"%>(&a1,&a2,&a3,&indexer,real_is_left);
+}
+
+// A real operand of this precision is read as it is rather than cast to a
+// complex array first.
+static VALUE
+<%=c_func%>_real(VALUE lhs, VALUE rhs, int real_is_left)
+{
+    cumo_ndfunc_arg_in_t ain[2] = {{real_is_left ? cRT : cT,0},{real_is_left ? cT : cRT,0}};
+    cumo_ndfunc_arg_out_t aout[1] = {{cT,0}};
+    cumo_ndfunc_t ndf = { <%=c_iter%>_r, CUMO_STRIDE_LOOP|CUMO_NDF_INDEXER_LOOP|CUMO_NDF_ANY_ORDER, 2, 1, ain, aout };
+
+    return cumo_na_ndloop3(&ndf, &real_is_left, 2, lhs, rhs);
+}
+
+VALUE
+<%="cumo_#{type_name}_#{name}_real_left"%>(VALUE real, VALUE self)
+{
+    return <%=c_func%>_real(real, self, 1);
+}
+//<% end %>
+
 static VALUE
 <%=c_func%>_self(VALUE self, VALUE other)
 {
@@ -202,6 +243,11 @@ static VALUE
         return cumo_na_ndloop3(&ndf_s, &y, 1, self);
     }
     //<% end %>
+    //<% if is_complex && %w[add sub mul div].include?(name) %>
+    if (rb_obj_class(other) == cRT) {
+        return <%=c_func%>_real(self, other, 0);
+    }
+    //<% end %>
     cumo_ndfunc_arg_in_t ain[2] = {{cT,0},{cT,0}};
     cumo_ndfunc_arg_out_t aout[1] = {{cT,0}};
     <% if type_name == 'robject' %>
@@ -230,6 +276,10 @@ static VALUE
     klass = cumo_na_upcast(rb_obj_class(self),rb_obj_class(other));
     if (klass==cT) {
         return <%=c_func%>_self(self, other);
+    //<% if complex_of %>
+    } else if (rb_obj_class(other) == <%=complex_of[1]%> && rb_obj_class(self) == cT) {
+        return <%="cumo_#{complex_of[0]}_#{name}_real_left"%>(self, other);
+    //<% end %>
     } else {
         v = rb_funcall(klass, cumo_id_cast, 1, self);
         return rb_funcall(v, <%=cumo_id_op%>, 1, other);

data/ext/cumo/narray/gen/tmpl/binary_kernel.cu

@@ -214,5 +214,29 @@ void <%="cumo_#{c_iter}_kernel_launch"%>(cumo_na_iarray_t* a1, cumo_na_iarray_t*
     cumo_cuda_runtime_check_kernel_launch();
 }
 //<% end %>
+
+//<% if is_complex && %w[add sub mul div].include?(name) %>
+// A real operand of the same precision is made complex as it is read, which is
+// what casting it first did, without the array the cast wrote.
+<% indexer_dims(true).each do |idim| %>
+__global__ void <%="cumo_#{c_iter}_r_kernel_dim#{idim}"%>(cumo_na_iarray_t a1, cumo_na_iarray_t a2, cumo_na_iarray_t a3, cumo_na_indexer_t indexer, int real_is_left)
+{
+    for (uint64_t i = blockIdx.x * blockDim.x + threadIdx.x; i < indexer.total_size; i += blockDim.x * gridDim.x) {
+        cumo_na_indexer_set_dim<%=idim%>(&indexer, i);
+        dtype x = *(dtype*)(cumo_na_iarray_at_dim<%=idim%>(&a1, &indexer));
+        dtype y = m_from_real(*(rtype*)(cumo_na_iarray_at_dim<%=idim%>(&a2, &indexer)));
+        *(dtype*)(cumo_na_iarray_at_dim<%=idim%>(&a3, &indexer)) = real_is_left ? m_<%=name%>(y, x) : m_<%=name%>(x, y);
+    }
+}
+<% end %>
+
+void <%="cumo_#{c_iter}_r_kernel_launch"%>(cumo_na_iarray_t* a1, cumo_na_iarray_t* a2, cumo_na_iarray_t* a3, cumo_na_indexer_t* indexer, int real_is_left)
+{
+    size_t grid_dim = cumo_get_grid_dim(indexer->total_size);
+    size_t block_dim = cumo_get_block_dim(indexer->total_size);
+    <%= indexer_switch("cumo_#{c_iter}_r_kernel", "*a1,*a2,*a3,*indexer,real_is_left", narrow: %w[a1 a2 a3]) %>
+    cumo_cuda_runtime_check_kernel_launch();
+}
+//<% end %>
 #undef cumo_check_intdivzero
 <% end %>

data/ext/cumo/narray/gen/tmpl/complex_accum_kernel.cu

@@ -215,33 +215,43 @@ extern "C" {
 #endif
 #endif
 
-<% if kernel_part? %>
+<% if kernel_part?("sum") %>
 void cumo_<%=type_name%>_sum_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_split<dtype, <%=dtype%>, cumo_<%=type_name%>_sum_impl>(*arg, cumo_<%=type_name%>_sum_impl{});
 }
+<% end %>
 
+<% if kernel_part?("prod") %>
 void cumo_<%=type_name%>_prod_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_split<dtype, <%=dtype%>, cumo_<%=type_name%>_prod_impl>(*arg, cumo_<%=type_name%>_prod_impl{});
 }
+<% end %>
 
+<% if kernel_part?("mean") %>
 void cumo_<%=type_name%>_mean_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     rtype n = (rtype)(arg->in_indexer.total_size / arg->out_indexer.total_size);
     cumo_reduce_split<dtype, dtype, cumo_<%=type_name%>_mean_impl>(*arg, cumo_<%=type_name%>_mean_impl{n});
 }
+<% end %>
 
+<% if kernel_part?("var") %>
 void cumo_<%=type_name%>_var_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_split<dtype, rtype, cumo_<%=type_name%>_var_impl>(*arg, cumo_<%=type_name%>_var_impl{});
 }
+<% end %>
 
+<% if kernel_part?("stddev") %>
 void cumo_<%=type_name%>_stddev_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_split<dtype, rtype, cumo_<%=type_name%>_stddev_impl>(*arg, cumo_<%=type_name%>_stddev_impl{});
 }
+<% end %>
 
+<% if kernel_part?("rms") %>
 void cumo_<%=type_name%>_rms_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     rtype n = (rtype)(arg->in_indexer.total_size / arg->out_indexer.total_size);
@@ -280,25 +290,29 @@ void cumo_<%=type_name%>_rms_nan_kernel_launch(cumo_na_reduction_arg_t* arg)
     cumo_reduce_split<dtype, rtype, cumo_<%=type_name%>_rms_nan_impl>(*arg, cumo_<%=type_name%>_rms_nan_impl{});
 }
 <% end %>
-<% if kernel_part? %>
+<% if kernel_part?("abs_sum") %>
 void cumo_<%=type_name%>_abs_sum_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     cumo_reduce_split<dtype, rtype, cumo_<%=type_name%>_abs_sum_impl>(*arg, cumo_<%=type_name%>_abs_sum_impl{});
 }
+<% end %>
 
+<% if kernel_part?("abs_max") %>
 void cumo_<%=type_name%>_abs_max_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     using impl = cumo_<%=type_name%>_abs_extremum_of<cumo_<%=type_name%>_abs_extremum_rules::Max>;
     cumo_reduce_split<dtype, rtype, impl>(*arg, impl{});
 }
+<% end %>
 
+<% if kernel_part?("abs_min") %>
 void cumo_<%=type_name%>_abs_min_kernel_launch(cumo_na_reduction_arg_t* arg)
 {
     using impl = cumo_<%=type_name%>_abs_extremum_of<cumo_<%=type_name%>_abs_extremum_rules::Min>;
     cumo_reduce_split<dtype, rtype, impl>(*arg, impl{});
 }
 <% end %>
-<% if is_double_precision && kernel_part? %>
+<% if is_double_precision && kernel_part?("kahan_sum") %>
 
 void cumo_<%=type_name%>_kahan_sum_kernel_launch(cumo_na_reduction_arg_t* arg)
 {