cumo 0.12.0 → 0.13.0

checksums.yaml

@@ -1,7 +1,7 @@
 ---
 SHA256:
-  metadata.gz: 48de893a0d01a5e8376c2a9cda402be28dd47835661f22bd69e7ed17561abed8
-  data.tar.gz: c403ffc7d25b80a0b50e8513b81439ee46f775cbd8c126667f83656077488a79
+  metadata.gz: ad04bee56b44e50dfc5df236d806a034f1adef98415e6e78ffcdd73546047dfd
+  data.tar.gz: c202fc43967dc292dddeea61274b89be6095f8724a7cfb7ac37d299cbc56cc7b
 SHA512:
-  metadata.gz: '00189a16920241771a70155a98d60c5d53b6fcd23bfcddbaed3094bbbe5d3476b2d6b5ce70f9d8d899e090a9bf3be2c4249c1b98b6d14360736a4a5ba1aa5594'
-  data.tar.gz: bd6a1c8b78837b387f93f5b57f13c26917c71c123a1441deff8e3afb7250a42bc799ded9271b9ad9d46d4c3b928660c38653f85427b024bd00797d6786535734
+  metadata.gz: 00b7285a170c241c968f2bba0ab4e80013f971c01964d8d092eefa4eb177ea06d68616326bf457b1e9a6a9c08ca8906b85456ba2ef91acbb13334292959b10a0
+  data.tar.gz: eba6aed6cdb916d1abee4d856a5496892ef5e33d3bb55cf4124fb01c3066a0324301baefe5d095798e21a0066af5d87c33ceca51142a00fec24936450b9f5946

data/CHANGELOG.md

@@ -1,3 +1,34 @@
+# 0.13.0 (2026/10/12)
+
+Breaking changes:
+
+* An integer array raised to a negative integer power raises `Cumo::NArray::ValueError`, where it answered 0 even for a base of 1 or -1. numo-narray-alt now raises here too, as NumPy does. A signed integer exponent held in an array is checked on the device, so such a `**` now waits for its kernel, as integer division by an array does. An exponent array that holds `2**31` or more is no longer narrowed to 32 bits, so `Cumo::Int64[1] ** Cumo::Int64[2**31]` answers `[1]` where it answered `[0]` (PR #614)
+
+Fixes:
+
+* Fix `inspect` printing `(empty)` for a scalar cast to an array, such as `Cumo::DFloat.cast(5.0)`, and for its views, which it did since 0.5.10. Its value and the operators on it were right. Fix such an array that is frozen, as `Ractor.make_shareable` leaves it, raising `FrozenError` when read, and then answering 0 to some operators and its value to others (PR #616)
+* Fix a crash, or a read of the wrong memory, when a 0-dimensional array is indexed with nothing but `false`, `:rest` or `:~`, such as `Cumo::DFloat.cast(5.0)[false]` (PR #615)
+* Fix the cuDNN algorithm caches and the counters that deferred host reads and index fills wait on, which two Ractors could update at once. They now take a lock or advance atomically (PR #613)
+* Fix two Ractors calling `rand` or `rand_norm` at the same time drawing the same numbers. A single thread draws what it drew before, so `srand` still reproduces a sequence (PR #612)
+* Fix wrong results, an illegal memory access, or a process stuck at full GPU load when two Ractors call cuSOLVER at once, as `Cumo::Linalg.svdvals` and `eigvals` do. Each thread keeps its own workspace and frees it when it ends (PR #611)
+* Fix `Cumo::Linalg.expm` of a triangular matrix whose elements off the diagonal are far larger than those on it answering 1 on the diagonal. The diagonal and the first off-diagonal are now reset to their exact values after each squaring, as scipy does (PR #609)
+* Fix `Cumo::Linalg.expm` raising `FloatDomainError: Infinity` for a finite matrix whose 1-norm overflows. It still refuses such a matrix, and now says that its 1-norm overflowed (PR #603)
+* Fix `Cumo::Linalg.lu` and `matrix_power(a, 0)` raising `NoMethodError` for a Bit matrix. `P` and the identity are Bit matrices (PR #602)
+* Fix `Cumo::Linalg.lu` raising `ShapeError` for an empty matrix. It answers `P`, `L` and `U` of m x m, m x 0 and 0 x n, the shapes a nonempty matrix gets. numo-linalg-alt raises here (PR #601)
+
+Changes:
+
+* The first elementwise call on a type loads the kernels of one family rather than every elementwise kernel of the type. The families are arithmetic, common math functions, rare math functions, comparisons, stores from other types, and random numbers with polynomials. A first SFloat add takes 28 MB of host memory where it took 86 MB (PR #610)
+* `require "cumo"` no longer loads NVRTC. It is loaded from the CUDA that cumo was built against the first time `ElementwiseKernel`, `ReductionKernel` or `RawKernel` builds a kernel, which takes about 34 MB off the RSS after `require` (PR #607, PR #606)
+* `Cumo::Linalg.norm` of ord 1, -1, Infinity and -Infinity takes the magnitudes as it reduces rather than through `abs`, which makes it about 2.4 times faster on a 4096 x 4096 DFloat (PR #605)
+* `Cumo::Linalg.norm` of ord 2, -2 and `"nuc"` over stacked matrices of up to 32 rows and columns computes them in one batched call, about 7 times faster for 1000 DFloat matrices of 8 x 8. A stacked matrix that holds NaN or Infinity answers NaN, as the matrices past 32 do (PR #604)
+* `Cumo::Linalg.expm` of a small matrix reads its three norms in one wait, which makes it about 3% faster on a 64 x 64 DFloat (PR #600)
+* The factorizations in `Cumo::Linalg` free their column-major copies once used, so a small call reuses memory the GPU has touched: `lu_solve` of a 256 x 256 SFloat is about 8% faster (PR #599)
+* `repeat` builds its index in one Array: `repeat(2)` of a million elements is 3.4 times faster, and `repeat` with an Array of counts 2.2 times. The development setup checks `lib` with rubocop-performance (PR #598, PR #597)
+* `Cumo::Linalg.eig` and `svd` give cuSOLVER pinned host memory to work in, kept for the process up to 256 MiB: `eig` of a 4096 x 4096 SFloat is about 8% faster, and `svd` of an 8192 x 2048 SFloat about 5% (PR #596)
+* `Cumo::Linalg.expm` frees its temporary matrices as it goes rather than leaving about 40 a call to the garbage collector, which makes it about 15% faster on a 1024 x 1024 SFloat of large norm (PR #595)
+* The API docs show the parameters of overloaded methods such as `cast`, `store_binary` and `layer_norm`, where YARD warned 1,419 times (PR #608)
+
 # 0.12.0 (2026/10/04)
 
 Fixes:
@@ -503,7 +534,7 @@ Fixes:
 * Remove unnecessary debug code
 * Fix capability list
 * Build only with supported capabilities to reduce compilation time
-* Fix SEGV when calling {mean, var, stddev, rms} on a single-element array (#154)
+* Fix SEGV when calling `mean`, `var`, `stddev` or `rms` on a single-element array (#154)
 * Suppress warning message for deprecated declarations
 * Fix variable typo in complex log2 and log10 functions (#152)
 
@@ -545,7 +576,7 @@ Fixes:
 * Backport: fix macro: STORE_BIT STORE_BIT_STEP: requires mask to leave the lowest bit
 * Backport: fix NArray::Bit#any?,all?: empty array should return false
 * Backport: fix NArray::Bit#count_true/false: empty array should return zero
-* Backport: bug in NArray::Bit; fix bit operation in tmpl_bit/{store_bit,unary,binary}.c
+* Backport: bug in NArray::Bit; fix bit operation in `tmpl_bit/{store_bit,unary,binary}.c`
 * Fix typo
 * Backport 135: Make all empty arrays equal
 * Backport: minor fixes in na_get_result_dimension(), check_index_count()

data/README.md

@@ -312,6 +312,7 @@ Cumo differs from numo-linalg-alt in a few places on purpose:
 * `expm` scales and squares the way scipy does, after Al-Mohy and Higham, which stays accurate where the norm of the matrix is large. Passing the order of the Pade approximant, as in `expm(a, 3)`, keeps numo-linalg-alt's algorithm.
 * `norm` takes the 2-norm and the nuclear norm of matrices stacked along other axes, which numo-linalg-alt refuses.
 * `cond` checks `ord` before it inverts the matrix, so an invalid `ord` raises `ArgumentError` even for a singular matrix, where numo-linalg-alt raises `LapackError`.
+* `lu` answers for an empty matrix and for a `Bit` matrix, where numo-linalg-alt raises. P keeps the shape and the class it has for any other matrix: an m x m identity for an m x 0 matrix, where scipy answers a 0 x 0 one, and a `Bit` for a `Bit` matrix. `matrix_power(a, 0)` of a `Bit` matrix likewise answers a `Bit` identity.
 
 A function waits for the current stream when it answers a Ruby number, or when it reads back whether cuSOLVER could do its work.
 `solve`, `inv` and `det` read back whether the matrix was singular, `cho_fact` whether it was positive definite, and `eigh` and `svd` whether they converged.

data/ext/cumo/cuda/cudnn_impl.cpp

@@ -11,6 +11,7 @@
 #include "cumo/cuda/runtime.h"
 #include "cumo/cuda/memory_pool.h"
 
+#include <mutex>
 #include <unordered_map>
 
 #if defined(__cplusplus)
@@ -387,13 +388,32 @@ struct AlgoCacheEntry {
 // Nothing may reach the C caller: it has no handler, so an escaping exception
 // is std::terminate. The cache only saves a later search, so one that cannot
 // grow leaves the answer this call already has.
+static std::mutex algo_cache_mutex_;
+
 template <typename Map>
 void StoreAlgoCache(Map& map, const AlgoCacheKey& key, const typename Map::mapped_type& entry) {
+    std::lock_guard<std::mutex> lock{algo_cache_mutex_};
     try {
         map[key] = entry;
     } catch (...) {
     }
 }
+
+template <typename Map, typename Perf>
+bool LookupAlgoCache(Map& map, const AlgoCacheKey& key, Perf* perf_result) {
+    std::lock_guard<std::mutex> lock{algo_cache_mutex_};
+    auto it = map.find(key);
+    if (it == map.end()) {
+        return false;
+    }
+    auto entry = it->second;
+    // clear the fields the search would have filled but the cache drops
+    *perf_result = {};
+    perf_result->algo = entry.algo;
+    perf_result->memory = entry.memory;
+    perf_result->mathType = entry.math_type;
+    return true;
+}
 }
 
 using FwdAlgoCacheMap = std::unordered_map<AlgoCacheKey, AlgoCacheEntry<cudnnConvolutionFwdAlgo_t>, AlgoCacheKeyHash>;
@@ -456,15 +476,7 @@ cumo_cuda_cudnn_FindConvolutionForwardAlgorithm(
                                 cudnn_dtype, max_workspace_size);
 
     auto& algo_cache_map = fwd_algo_cache_map_;
-    // TODO: thread-safe
-    auto it = algo_cache_map.find(key);
-    if (it != algo_cache_map.end()) {
-        auto entry = it->second;
-        // clear the fields the search would have filled but the cache drops
-        *perf_result = {};
-        perf_result->algo = entry.algo;
-        perf_result->memory = entry.memory;
-        perf_result->mathType = entry.math_type;
+    if (LookupAlgoCache(algo_cache_map, key, perf_result)) {
         return CUDNN_STATUS_SUCCESS;
     }
 
@@ -495,7 +507,6 @@ cumo_cuda_cudnn_FindConvolutionForwardAlgorithm(
     if (returned_algo_count < 1) return CUDNN_STATUS_NOT_SUPPORTED;
     if (perf_result->status != CUDNN_STATUS_SUCCESS) return perf_result->status;
 
-    // TODO: thread-safe
     StoreAlgoCache(algo_cache_map, key, {perf_result->algo, perf_result->memory, perf_result->mathType});
     return status;
 }
@@ -527,15 +538,7 @@ cumo_cuda_cudnn_FindConvolutionBackwardDataAlgorithm(
                                 cudnn_dtype, max_workspace_size);
 
     auto& algo_cache_map = bwd_data_algo_cache_map_;
-    // TODO: thread-safe
-    auto it = algo_cache_map.find(key);
-    if (it != algo_cache_map.end()) {
-        auto entry = it->second;
-        // clear the fields the search would have filled but the cache drops
-        *perf_result = {};
-        perf_result->algo = entry.algo;
-        perf_result->memory = entry.memory;
-        perf_result->mathType = entry.math_type;
+    if (LookupAlgoCache(algo_cache_map, key, perf_result)) {
         return CUDNN_STATUS_SUCCESS;
     }
 
@@ -566,7 +569,6 @@ cumo_cuda_cudnn_FindConvolutionBackwardDataAlgorithm(
     if (returned_algo_count < 1) return CUDNN_STATUS_NOT_SUPPORTED;
     if (perf_result->status != CUDNN_STATUS_SUCCESS) return perf_result->status;
 
-    // TODO: thread-safe
     StoreAlgoCache(algo_cache_map, key, {perf_result->algo, perf_result->memory, perf_result->mathType});
     return status;
 }
@@ -598,15 +600,7 @@ cumo_cuda_cudnn_FindConvolutionBackwardFilterAlgorithm(
                                 cudnn_dtype, max_workspace_size);
 
     auto& algo_cache_map = bwd_filter_algo_cache_map_;
-    // TODO: thread-safe
-    auto it = algo_cache_map.find(key);
-    if (it != algo_cache_map.end()) {
-        auto entry = it->second;
-        // clear the fields the search would have filled but the cache drops
-        *perf_result = {};
-        perf_result->algo = entry.algo;
-        perf_result->memory = entry.memory;
-        perf_result->mathType = entry.math_type;
+    if (LookupAlgoCache(algo_cache_map, key, perf_result)) {
         return CUDNN_STATUS_SUCCESS;
     }
 
@@ -637,7 +631,6 @@ cumo_cuda_cudnn_FindConvolutionBackwardFilterAlgorithm(
     if (returned_algo_count < 1) return CUDNN_STATUS_NOT_SUPPORTED;
     if (perf_result->status != CUDNN_STATUS_SUCCESS) return perf_result->status;
 
-    // TODO: thread-safe
     StoreAlgoCache(algo_cache_map, key, {perf_result->algo, perf_result->memory, perf_result->mathType});
     return status;
 }

data/ext/cumo/cuda/cusolver.c

@@ -94,6 +94,8 @@ rb_cusolver_version(VALUE self)
 typedef struct {
     char *work;
     size_t size;
+    char *host_work;
+    size_t host_size;
     int *info;
     cudaEvent_t done;
 } cusolver_scratch_t;
@@ -105,19 +107,18 @@ typedef struct {
 } cusolver_context_t;
 
 #define CUSOLVER_KEPT_WORK_MAX ((size_t)32 << 20)
+#define CUSOLVER_KEPT_HOST_WORK_MAX ((size_t)256 << 20)
+
+static cumo_cuda_thread_local_t scratches;
 
 static cusolver_scratch_t*
 cusolver_scratch(void)
 {
-    static cusolver_scratch_t *scratch = 0;
-    cusolver_scratch_t *s;
-
-    if (scratch == 0) {
-        scratch = ZALLOC_N(cusolver_scratch_t, cumo_cuda_runtime_get_device_count());
+    cusolver_scratch_t *s = (cusolver_scratch_t*)cumo_cuda_thread_local_get(&scratches);
+    if (s->done == NULL) {
+        cumo_cuda_runtime_check_status(cudaEventCreateWithFlags(&s->done, cudaEventDisableTiming));
     }
-    s = &scratch[cumo_cuda_runtime_get_device()];
     if (s->info == NULL) {
-        cumo_cuda_runtime_check_status(cudaEventCreateWithFlags(&s->done, cudaEventDisableTiming));
         s->info = (int*)cumo_cuda_runtime_malloc(sizeof(int));
     } else {
         cumo_cuda_runtime_check_status(cudaStreamWaitEvent(cumo_cuda_stream(), s->done, 0));
@@ -125,6 +126,21 @@ cusolver_scratch(void)
     return s;
 }
 
+static void
+cusolver_scratch_release(void *entry)
+{
+    cusolver_scratch_t *s = (cusolver_scratch_t*)entry;
+    if (s->done != NULL) {
+        cudaEventSynchronize(s->done);
+        cudaEventDestroy(s->done);
+    }
+    if (s->host_work != NULL) { cudaFreeHost(s->host_work); }
+    if (s->work != NULL) { cumo_cuda_runtime_free_no_raise(s->work); }
+    if (s->info != NULL) { cumo_cuda_runtime_free_no_raise((char*)s->info); }
+    cudaGetLastError();
+    memset(s, 0, sizeof(*s));
+}
+
 static cumo_cuda_thread_local_t contexts;
 
 static cusolver_context_t
@@ -199,11 +215,38 @@ typedef struct {
     char *d_work;
     char *loose_work;
     void *h_work;
+    bool h_pinned;
+    gesvdjInfo_t gesvdj;
     int *d_info;
     int info;
     int skip_info;
 } cusolver_call_t;
 
+static void*
+cusolver_host_work(cusolver_call_t *c, size_t size)
+{
+    cusolver_scratch_t *s = c->ctx.scratch;
+
+    if (size <= CUSOLVER_KEPT_HOST_WORK_MAX && s->host_size < size) {
+        size_t grown = s->host_size * 2 > size ? s->host_size * 2 : size;
+        if (grown > CUSOLVER_KEPT_HOST_WORK_MAX) grown = CUSOLVER_KEPT_HOST_WORK_MAX;
+        cudaFreeHost(s->host_work);
+        s->host_work = NULL;
+        s->host_size = 0;
+        if (cudaHostAlloc((void**)&s->host_work, grown, cudaHostAllocPortable) == cudaSuccess) {
+            s->host_size = grown;
+        } else {
+            s->host_work = NULL;
+            cudaGetLastError();
+        }
+    }
+    if (size <= s->host_size) {
+        c->h_pinned = true;
+        return s->host_work;
+    }
+    return ruby_xmalloc(size);
+}
+
 static char*
 cusolver_work(cusolver_call_t *c, size_t size)
 {
@@ -243,7 +286,7 @@ getrf_body(VALUE arg)
             c->ctx.handle, c->ctx.params, c->m, c->n, c->dtype, c->a, c->m, c->dtype, &d_size, &h_size));
     c->d_info = c->ctx.scratch->info;
     if (d_size > 0) c->d_work = cusolver_work(c, d_size);
-    if (h_size > 0) c->h_work = ruby_xmalloc(h_size);
+    if (h_size > 0) c->h_work = cusolver_host_work(c, h_size);
     cumo_cuda_cusolver_check_status(cusolverDnXgetrf(
             c->ctx.handle, c->ctx.params, c->m, c->n, c->dtype, c->a, c->m, c->ipiv,
             c->dtype, c->d_work, d_size, c->h_work, h_size, c->d_info));
@@ -274,7 +317,7 @@ potrf_body(VALUE arg)
             c->ctx.handle, c->ctx.params, c->uplo, c->n, c->dtype, c->a, c->n, c->dtype, &d_size, &h_size));
     c->d_info = c->ctx.scratch->info;
     if (d_size > 0) c->d_work = cusolver_work(c, d_size);
-    if (h_size > 0) c->h_work = ruby_xmalloc(h_size);
+    if (h_size > 0) c->h_work = cusolver_host_work(c, h_size);
     cumo_cuda_cusolver_check_status(cusolverDnXpotrf(
             c->ctx.handle, c->ctx.params, c->uplo, c->n, c->dtype, c->a, c->n,
             c->dtype, c->d_work, d_size, c->h_work, h_size, c->d_info));
@@ -347,7 +390,7 @@ syevd_body(VALUE arg)
                 c->dtype, c->a, c->n, wtype, c->w, c->dtype, &d_size, &h_size));
     }
     if (d_size > 0) c->d_work = cusolver_work(c, d_size);
-    if (h_size > 0) c->h_work = ruby_xmalloc(h_size);
+    if (h_size > 0) c->h_work = cusolver_host_work(c, h_size);
     if (c->range) {
         cumo_cuda_cusolver_check_status(cusolverDnXsyevdx(
                 c->ctx.handle, c->ctx.params, c->jobz, CUSOLVER_EIG_RANGE_I, CUBLAS_FILL_MODE_UPPER, c->n,
@@ -421,7 +464,7 @@ gesvd_body(VALUE arg)
             stype, c->w, c->dtype, c->u, c->m, c->dtype, c->vt, ldvt, c->dtype, &d_size, &h_size));
     c->d_info = c->ctx.scratch->info;
     if (d_size > 0) c->d_work = cusolver_work(c, d_size);
-    if (h_size > 0) c->h_work = ruby_xmalloc(h_size);
+    if (h_size > 0) c->h_work = cusolver_host_work(c, h_size);
     cumo_cuda_cusolver_check_status(cusolverDnXgesvd(
             c->ctx.handle, c->ctx.params, c->job, c->job, c->m, c->n, c->dtype, c->a, c->m,
             stype, c->w, c->dtype, c->u, c->m, c->dtype, c->vt, ldvt, c->dtype,
@@ -430,6 +473,91 @@ gesvd_body(VALUE arg)
     return Qnil;
 }
 
+#define CUSOLVER_BATCHED_SVD_MAX 32
+#define CUSOLVER_BATCHED_SVD_UV_MAX ((size_t)64 << 20)
+
+static size_t
+align_up(size_t size)
+{
+    return (size + 255) & ~(size_t)255;
+}
+
+static VALUE
+gesvdj_batched_body(VALUE arg)
+{
+    cusolver_call_t *c = (cusolver_call_t*)arg;
+    cusolverDnHandle_t h = c->ctx.handle;
+    int m = (int)c->m;
+    int n = (int)c->n;
+    int64_t k = m < n ? m : n;
+    int64_t done;
+    int chunk;
+    int lwork = 0;
+    size_t elsize;
+    size_t welsize;
+    size_t work_bytes;
+    size_t u_bytes;
+    size_t v_bytes;
+    char *buf;
+    int *d_infos;
+    int *infos;
+    int64_t i;
+
+    switch (c->dtype) {
+    case CUDA_R_32F: elsize = sizeof(float); welsize = sizeof(float); break;
+    case CUDA_R_64F: elsize = sizeof(double); welsize = sizeof(double); break;
+    case CUDA_C_32F: elsize = sizeof(cuComplex); welsize = sizeof(float); break;
+    default: elsize = sizeof(cuDoubleComplex); welsize = sizeof(double); break;
+    }
+    chunk = (int)(CUSOLVER_BATCHED_SVD_UV_MAX / (elsize * ((size_t)m * m + (size_t)n * n)));
+    if (chunk < 1) chunk = 1;
+    if ((int64_t)chunk > c->k) chunk = (int)c->k;
+
+    check_call(cusolverDnCreateGesvdjInfo(&c->gesvdj));
+    switch (c->dtype) {
+#define GESVDJ_BUFFER_SIZE(fn, type, rtype)                                                          \
+        check_call(cusolverDn##fn##_bufferSize(h, CUSOLVER_EIG_MODE_NOVECTOR, m, n, (type*)c->a, m,  \
+                (rtype*)c->w, NULL, m, NULL, n, &lwork, c->gesvdj, chunk));                          \
+        break
+    case CUDA_R_32F: GESVDJ_BUFFER_SIZE(SgesvdjBatched, float, float);
+    case CUDA_R_64F: GESVDJ_BUFFER_SIZE(DgesvdjBatched, double, double);
+    case CUDA_C_32F: GESVDJ_BUFFER_SIZE(CgesvdjBatched, cuComplex, float);
+    default: GESVDJ_BUFFER_SIZE(ZgesvdjBatched, cuDoubleComplex, double);
+#undef GESVDJ_BUFFER_SIZE
+    }
+    work_bytes = align_up(elsize * (size_t)(lwork > 0 ? lwork : 1));
+    u_bytes = align_up(elsize * (size_t)m * m * chunk);
+    v_bytes = align_up(elsize * (size_t)n * n * chunk);
+    buf = cusolver_work(c, work_bytes + u_bytes + v_bytes + sizeof(int) * (size_t)c->k);
+    d_infos = (int*)(buf + work_bytes + u_bytes + v_bytes);
+    for (done = 0; done < c->k; done += chunk) {
+        int batch = (int)(c->k - done < chunk ? c->k - done : chunk);
+        char *a = (char*)c->a + elsize * (size_t)m * n * done;
+        char *w = (char*)c->w + welsize * (size_t)k * done;
+        switch (c->dtype) {
+#define GESVDJ_BATCHED(fn, type, rtype)                                                              \
+            cumo_cuda_cusolver_check_status(cusolverDn##fn(h, CUSOLVER_EIG_MODE_NOVECTOR, m, n,      \
+                    (type*)a, m, (rtype*)w, (type*)(buf + work_bytes), m,                            \
+                    (type*)(buf + work_bytes + u_bytes), n, (type*)buf, lwork, d_infos + done,       \
+                    c->gesvdj, batch));                                                              \
+            break
+        case CUDA_R_32F: GESVDJ_BATCHED(SgesvdjBatched, float, float);
+        case CUDA_R_64F: GESVDJ_BATCHED(DgesvdjBatched, double, double);
+        case CUDA_C_32F: GESVDJ_BATCHED(CgesvdjBatched, cuComplex, float);
+        default: GESVDJ_BATCHED(ZgesvdjBatched, cuDoubleComplex, double);
+#undef GESVDJ_BATCHED
+        }
+    }
+    infos = ruby_xmalloc(sizeof(int) * (size_t)c->k);
+    c->h_work = infos;
+    cumo_cuda_runtime_check_status(cumo_cuda_runtime_memcpy_to_host(infos, d_infos, sizeof(int) * (size_t)c->k));
+    c->info = 0;
+    for (i = 0; i < c->k && c->info == 0; i++) {
+        c->info = infos[i];
+    }
+    return Qnil;
+}
+
 static VALUE
 geqrf_body(VALUE arg)
 {
@@ -442,7 +570,7 @@ geqrf_body(VALUE arg)
             &d_size, &h_size));
     c->d_info = c->ctx.scratch->info;
     if (d_size > 0) c->d_work = cusolver_work(c, d_size);
-    if (h_size > 0) c->h_work = ruby_xmalloc(h_size);
+    if (h_size > 0) c->h_work = cusolver_host_work(c, h_size);
     cumo_cuda_cusolver_check_status(cusolverDnXgeqrf(
             c->ctx.handle, c->ctx.params, c->m, c->n, c->dtype, c->a, c->m, c->dtype, c->tau, c->dtype,
             c->d_work, d_size, c->h_work, h_size, c->d_info));
@@ -483,7 +611,13 @@ call_ensure(VALUE arg)
     cusolver_call_t *c = (cusolver_call_t*)arg;
     if (c->ctx.scratch != NULL) { cudaEventRecord(c->ctx.scratch->done, cumo_cuda_stream()); }
     cumo_cuda_runtime_return_scratch(c->loose_work, c->loose_work != NULL, NULL);
-    ruby_xfree(c->h_work);
+    if (c->h_pinned) {
+        cudaStreamSynchronize(cumo_cuda_stream());
+        cudaGetLastError();
+    } else {
+        ruby_xfree(c->h_work);
+    }
+    if (c->gesvdj != NULL) { cusolverDnDestroyGesvdjInfo(c->gesvdj); }
     return Qnil;
 }
 
@@ -649,13 +783,14 @@ check_square_matrix(VALUE a, const char *name)
   Factorizes a Hermitian positive definite matrix in place as A = U^H U or
   A = L L^H with cusolverDnXpotrf.
 
-  @param a [Cumo::SFloat, Cumo::DFloat, Cumo::SComplex, Cumo::DComplex]
-    contiguous, of shape [n, n]: the column-major matrix, whose uplo
-    triangle is overwritten with the factor
-  @param uplo [String] "U" or "L"
-  @param want_info [Boolean] whether to wait for the info cuSOLVER reports,
-    true by default
-  @return [Integer, nil] the info cuSOLVER reports, or nil if it is not read
+  @overload potrf(a, uplo, want_info = true)
+    @param a [Cumo::SFloat, Cumo::DFloat, Cumo::SComplex, Cumo::DComplex]
+      contiguous, of shape [n, n]: the column-major matrix, whose uplo
+      triangle is overwritten with the factor
+    @param uplo [String] "U" or "L"
+    @param want_info [Boolean] whether to wait for the info cuSOLVER reports,
+      true by default
+    @return [Integer, nil] the info cuSOLVER reports, or nil if it is not read
  */
 static VALUE
 rb_cusolver_potrf(int argc, VALUE *argv, VALUE self)
@@ -916,6 +1051,35 @@ rb_cusolver_gesvd(VALUE self, VALUE a, VALUE s, VALUE u, VALUE vt, VALUE job)
     return INT2NUM(c.info);
 }
 
+static VALUE
+rb_cusolver_gesvdj_batched(VALUE self, VALUE a, VALUE s)
+{
+    cusolver_call_t c = {0};
+    cumo_narray_t *na;
+    cumo_narray_t *ns;
+
+    na = check_contiguous_array(a, Qnil, 3, "a");
+    c.dtype = cusolver_dtype(a);
+    c.k = (int64_t)CUMO_NA_SHAPE(na)[0];
+    c.n = (int64_t)CUMO_NA_SHAPE(na)[1];
+    c.m = (int64_t)CUMO_NA_SHAPE(na)[2];
+    if (c.m > CUSOLVER_BATCHED_SVD_MAX || c.n > CUSOLVER_BATCHED_SVD_MAX) {
+        rb_raise(cumo_na_eShapeError, "a must have at most %d rows and columns", CUSOLVER_BATCHED_SVD_MAX);
+    }
+    ns = check_contiguous_array(s, eigen_real_class(c.dtype), 2, "s");
+    if ((int64_t)CUMO_NA_SHAPE(ns)[0] != c.k || (int64_t)CUMO_NA_SHAPE(ns)[1] != (c.m < c.n ? c.m : c.n)) {
+        rb_raise(cumo_na_eShapeError, "s must have the shape [%"PRId64", %"PRId64"]", c.k, c.m < c.n ? c.m : c.n);
+    }
+    if (c.k == 0 || c.m == 0 || c.n == 0) {
+        return INT2FIX(0);
+    }
+    c.a = cumo_na_get_offset_pointer_for_read_write(a);
+    c.w = cumo_na_get_offset_pointer_for_write(s);
+    c.ctx = cusolver_context();
+    rb_ensure(gesvdj_batched_body, (VALUE)&c, call_ensure, (VALUE)&c);
+    return INT2NUM(c.info);
+}
+
 static int64_t
 check_tau(VALUE tau, VALUE a, int64_t k)
 {
@@ -1006,7 +1170,7 @@ geev_body(VALUE arg)
             wtype, c->w, c->dtype, NULL, c->n, c->dtype, c->u, c->n, c->dtype, &d_size, &h_size));
     c->d_info = c->ctx.scratch->info;
     if (d_size > 0) c->d_work = cusolver_work(c, d_size);
-    if (h_size > 0) c->h_work = ruby_xmalloc(h_size);
+    if (h_size > 0) c->h_work = cusolver_host_work(c, h_size);
     cumo_cuda_cusolver_check_status(cusolverDnXgeev(
             c->ctx.handle, c->ctx.params, CUSOLVER_EIG_MODE_NOVECTOR, c->jobz, c->n, c->dtype, c->a, c->n,
             wtype, c->w, c->dtype, NULL, c->n, c->dtype, c->u, c->n, c->dtype,
@@ -1082,7 +1246,7 @@ sytrf_body(VALUE arg)
                 c->ctx.handle, c->ctx.params, c->uplo, c->n, c->dtype, c->a, c->n, c->ipiv, c->dtype,
                 &d_size, &h_size));
         if (d_size > 0) c->d_work = cusolver_work(c, d_size);
-        if (h_size > 0) c->h_work = ruby_xmalloc(h_size);
+        if (h_size > 0) c->h_work = cusolver_host_work(c, h_size);
         cumo_cuda_cusolver_check_status(cusolverDnXhetrf(
                 c->ctx.handle, c->ctx.params, c->uplo, c->n, c->dtype, c->a, c->n, c->ipiv, c->dtype,
                 c->d_work, d_size, c->h_work, h_size, c->d_info));
@@ -1095,7 +1259,7 @@ sytrf_body(VALUE arg)
             c->ctx.handle, c->ctx.params, c->uplo, c->n, c->dtype, c->a, c->n, c->ipiv, c->dtype,
             &d_size, &h_size));
     if (d_size > 0) c->d_work = cusolver_work(c, d_size);
-    if (h_size > 0) c->h_work = ruby_xmalloc(h_size);
+    if (h_size > 0) c->h_work = cusolver_host_work(c, h_size);
     cumo_cuda_cusolver_check_status(cusolverDnXsytrf(
             c->ctx.handle, c->ctx.params, c->uplo, c->n, c->dtype, c->a, c->n, c->ipiv, c->dtype,
             c->d_work, d_size, c->h_work, h_size, c->d_info));
@@ -1212,6 +1376,8 @@ Init_cumo_cuda_cusolver(void)
     rb_define_singleton_method(mCusolver, "available?", rb_cusolver_available_p, 0);
 #ifdef CUSOLVER_FOUND
     cumo_cuda_thread_local_init(&contexts, sizeof(cusolver_context_t));
+    cumo_cuda_thread_local_init(&scratches, sizeof(cusolver_scratch_t));
+    cumo_cuda_thread_local_on_exit(&scratches, cusolver_scratch_release);
     rb_define_singleton_method(mCusolver, "version", rb_cusolver_version, 0);
     rb_define_singleton_method(mCusolver, "getrf", rb_cusolver_getrf, 1);
     rb_define_singleton_method(mCusolver, "getrs", rb_cusolver_getrs, -1);
@@ -1222,6 +1388,8 @@ Init_cumo_cuda_cusolver(void)
     rb_define_singleton_method(mCusolver, "syevd", rb_cusolver_syevd, 5);
     rb_define_singleton_method(mCusolver, "sygvd", rb_cusolver_sygvd, 6);
     rb_define_singleton_method(mCusolver, "gesvd", rb_cusolver_gesvd, 5);
+    rb_define_singleton_method(mCusolver, "gesvdj_batched", rb_cusolver_gesvdj_batched, 2);
+    rb_define_const(mCusolver, "BATCHED_SVD_MAX", INT2FIX(CUSOLVER_BATCHED_SVD_MAX));
     rb_define_singleton_method(mCusolver, "geqrf", rb_cusolver_geqrf, 2);
     rb_define_singleton_method(mCusolver, "orgqr", rb_cusolver_orgqr, 2);
     rb_define_singleton_method(mCusolver, "sytrf", rb_cusolver_sytrf, 3);
@@ -1229,8 +1397,9 @@ Init_cumo_cuda_cusolver(void)
     rb_define_singleton_method(mCusolver, "geev", rb_cusolver_geev, 3);
     rb_funcall(mCusolver, rb_intern("private_class_method"), 1, ID2SYM(rb_intern("geev")));
 #endif
-    rb_funcall(mCusolver, rb_intern("private_class_method"), 12,
+    rb_funcall(mCusolver, rb_intern("private_class_method"), 13,
                ID2SYM(rb_intern("sytrf")), ID2SYM(rb_intern("geqrf")), ID2SYM(rb_intern("orgqr")), ID2SYM(rb_intern("gesvd")),
+               ID2SYM(rb_intern("gesvdj_batched")),
                ID2SYM(rb_intern("syevd")), ID2SYM(rb_intern("sygvd")), ID2SYM(rb_intern("uplo")),
                ID2SYM(rb_intern("getrf")), ID2SYM(rb_intern("getrs")),
                ID2SYM(rb_intern("potrf")), ID2SYM(rb_intern("potrs")), ID2SYM(rb_intern("potri")));