scm2cpp-lasso 0.11.0__tar.gz → 0.13.0__tar.gz

{scm2cpp_lasso-0.11.0/src/scm2cpp_lasso.egg-info → scm2cpp_lasso-0.13.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: scm2cpp-lasso
-Version: 0.11.0
-Summary: scikit-learn estimators (lasso, elastic net, ridge, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, NMF, Gaussian mixture, FastICA, PCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver included
+Version: 0.13.0
+Summary: scikit-learn estimators (lasso, elastic net, ridge, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
 Author: Hirotaka Niitsuma
 License: MIT
 Project-URL: Homepage, https://github.com/niitsuma/scm2cpp
@@ -712,6 +712,41 @@ GramLassoLarsCV().fit(X, y).alpha_
 Speed, from the repository README's table: the GLMs 1.1-6x, the CV
 estimators 1.6-6.6x, Theil-Sen 1.3-1.8x (its subset drawing dominates).
 
+## Feature selection: `f_regression`, `f_classif`, `SelectKBest`, `SelectPercentile`, `GramRFE`
+
+scikit-learn's univariate F statistics and its selectors over them,
+and `RFE` around `Ridge` / `LinearRegression`, translated.
+`f_regression` also takes an n x q array of targets: the q results in
+one product, on the device when one answers.
+
+```python
+from scm2cpp_lasso import SelectKBest, GramRFE, f_regression
+SelectKBest("f_regression", k=20).fit(X, y).get_support()
+GramRFE(alpha=1.0, n_features_to_select=10).fit(X, y).ranking_
+F, p = f_regression(X, Y)          # Y: n x q, every target at once
+```
+
+The same scores, ranking and features as scikit-learn. RFE is 80-1,000x
+faster (one Gram and one Cholesky factor, downdated, instead of a refit
+per round), and f_regression over many targets 30-50x. The Gram-route
+estimators (`GramLassoLarsIC`, `GramLarsCV`, `GramLassoLarsCV`,
+`GramLars`, `GramLassoLars`, `CovPCA`) form a large X'X on the device
+when one answers. `SCM2CPP_NO_DEVICE=1` keeps everything on the CPU.
+
+## The GPU build
+
+Where the CUDA toolkit is at install time, `ParallelFastICA`,
+`EMGaussianMixture`, `LBFGSLogisticRegression`,
+`LBFGSLogisticRegressionCV`, `LBFGSHuberRegressor` and the `LBFGS*`
+GLMs also get the same Scheme translated with `--cuda`
+(`libscm2cpp_gpu.so`). Its loops are CUDA kernels and its products
+cuBLAS, on arrays that stay on the device. The classes take it when a
+device answers and the data has a million entries or more, and give
+the CPU build's iterations and results to the last digits. Measured
+against scikit-learn: 10-22x for the L-BFGS fits, 6.7x FastICA, 3x the
+mixture and LogisticRegressionCV (the repository README's table).
+`SCM2CPP_NO_DEVICE=1` keeps everything on the CPU.
+
 ## Logistic regression with L1
 
 `CovLogistic` solves L1-penalized logistic regression by

scm2cpp_lasso-0.11.0/PKG-INFO → scm2cpp_lasso-0.13.0/README.md

@@ -1,21 +1,3 @@
-Metadata-Version: 2.4
-Name: scm2cpp-lasso
-Version: 0.11.0
-Summary: scikit-learn estimators (lasso, elastic net, ridge, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, NMF, Gaussian mixture, FastICA, PCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver included
-Author: Hirotaka Niitsuma
-License: MIT
-Project-URL: Homepage, https://github.com/niitsuma/scm2cpp
-Project-URL: Source, https://github.com/niitsuma/scm2cpp/tree/master/python/scm2cpp-lasso
-Keywords: lasso,elastic-net,ridge,svm,logistic-regression,coordinate-descent,cuda,compiler
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: C++
-Classifier: Topic :: Scientific/Engineering :: Mathematics
-Classifier: Intended Audience :: Science/Research
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-Requires-Dist: numpy>=1.17
-
 # scm2cpp-lasso
 
 *[Japanese (README.ja.md)](README.ja.md)*
@@ -712,6 +694,41 @@ GramLassoLarsCV().fit(X, y).alpha_
 Speed, from the repository README's table: the GLMs 1.1-6x, the CV
 estimators 1.6-6.6x, Theil-Sen 1.3-1.8x (its subset drawing dominates).
 
+## Feature selection: `f_regression`, `f_classif`, `SelectKBest`, `SelectPercentile`, `GramRFE`
+
+scikit-learn's univariate F statistics and its selectors over them,
+and `RFE` around `Ridge` / `LinearRegression`, translated.
+`f_regression` also takes an n x q array of targets: the q results in
+one product, on the device when one answers.
+
+```python
+from scm2cpp_lasso import SelectKBest, GramRFE, f_regression
+SelectKBest("f_regression", k=20).fit(X, y).get_support()
+GramRFE(alpha=1.0, n_features_to_select=10).fit(X, y).ranking_
+F, p = f_regression(X, Y)          # Y: n x q, every target at once
+```
+
+The same scores, ranking and features as scikit-learn. RFE is 80-1,000x
+faster (one Gram and one Cholesky factor, downdated, instead of a refit
+per round), and f_regression over many targets 30-50x. The Gram-route
+estimators (`GramLassoLarsIC`, `GramLarsCV`, `GramLassoLarsCV`,
+`GramLars`, `GramLassoLars`, `CovPCA`) form a large X'X on the device
+when one answers. `SCM2CPP_NO_DEVICE=1` keeps everything on the CPU.
+
+## The GPU build
+
+Where the CUDA toolkit is at install time, `ParallelFastICA`,
+`EMGaussianMixture`, `LBFGSLogisticRegression`,
+`LBFGSLogisticRegressionCV`, `LBFGSHuberRegressor` and the `LBFGS*`
+GLMs also get the same Scheme translated with `--cuda`
+(`libscm2cpp_gpu.so`). Its loops are CUDA kernels and its products
+cuBLAS, on arrays that stay on the device. The classes take it when a
+device answers and the data has a million entries or more, and give
+the CPU build's iterations and results to the last digits. Measured
+against scikit-learn: 10-22x for the L-BFGS fits, 6.7x FastICA, 3x the
+mixture and LogisticRegressionCV (the repository README's table).
+`SCM2CPP_NO_DEVICE=1` keeps everything on the CPU.
+
 ## Logistic regression with L1
 
 `CovLogistic` solves L1-penalized logistic regression by

{scm2cpp_lasso-0.11.0 → scm2cpp_lasso-0.13.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scm2cpp-lasso"
-version = "0.11.0"
-description = "scikit-learn estimators (lasso, elastic net, ridge, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, NMF, Gaussian mixture, FastICA, PCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver included"
+version = "0.13.0"
+description = "scikit-learn estimators (lasso, elastic net, ridge, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included"
 readme = "README.md"
 requires-python = ">=3.8"
 license = {text = "MIT"}
@@ -31,4 +31,4 @@ package-dir = {"" = "src"}
 where = ["src"]
 
 [tool.setuptools.package-data]
-scm2cpp_lasso = ["_generated/*.hpp", "_generated/*.cpp", "_generated/derived_*/*.hpp", "_generated/derived_*/*.cpp", "_generated/nmf_*/*.hpp", "_generated/nmf_*/*.cpp", "_generated/gramops_*/*.hpp", "_generated/gramops_*/*.cpp", "*.cu"]
+scm2cpp_lasso = ["_generated/*.hpp", "_generated/*.cpp", "_generated/derived_*/*.hpp", "_generated/derived_*/*.cpp", "_generated/nmf_*/*.hpp", "_generated/nmf_*/*.cpp", "_generated/gramops_*/*.hpp", "_generated/gramops_*/*.cpp", "_generated/accel_*/*.hpp", "_generated/accel_*/*.cpp", "_generated/gpu_*/*.hpp", "_generated/gpu_*/*.cpp", "*.cu"]

{scm2cpp_lasso-0.11.0 → scm2cpp_lasso-0.13.0}/setup.py

@@ -120,6 +120,28 @@ class BuildWithCuda(build_ext):
         except (subprocess.CalledProcessError, OSError) as exc:
             print(f"scm2cpp_lasso: no CUDA batch path ({exc}); "
                   "the CPU solver is unaffected", file=sys.stderr)
+        # the --cuda translation (kernels + device-resident cuBLAS):
+        # FastICA, the mixture, the L-BFGS fits
+        gpu = GEN / "gpu_cuda" / "gpu_capi.cpp"
+        if gpu.exists():
+            out2 = Path(self.build_lib) / "scm2cpp_lasso" / "libscm2cpp_gpu.so"
+            # the installed device's architecture when one answers
+            arch = []
+            try:
+                if subprocess.run(["nvidia-smi"], capture_output=True).returncode == 0:
+                    arch = ["-arch=native"]
+            except OSError:
+                pass
+            cmd2 = [nvcc, "-O3", "-std=c++17", "-shared", "-x", "cu", *arch,
+                    "-Xcompiler", "-fPIC", "-I", str(GEN / "gpu_cuda"), "-I", str(GEN),
+                    str(gpu), "-o", str(out2), "-lcublas",
+                    "-Wno-deprecated-gpu-targets", "-diag-suppress", "174,177,550"]
+            try:
+                subprocess.run(cmd2, check=True)
+                print("scm2cpp_lasso: built the --cuda estimators")
+            except (subprocess.CalledProcessError, OSError) as exc:
+                print(f"scm2cpp_lasso: no --cuda estimators ({exc}); "
+                      "the CPU paths are unaffected", file=sys.stderr)
 
 
 # The cuBLAS variant of the derived kernel: host C++ against the CUDA
@@ -211,4 +233,13 @@ if _cuda is not None:
         extra_compile_args=CXXFLAGS + ["-O3", "-fstrict-overflow"],
     ))
 
+    extensions.append(Extension(
+        "scm2cpp_lasso._accel_cublas",
+        sources=[GEN_REL + "/accel_cublas/accel_capi.cpp"],
+        include_dirs=[GEN_REL + "/accel_cublas", GEN_REL, str(_cuda / "include")],
+        library_dirs=[str(_cuda / "lib64"), str(_cuda / "lib")],
+        libraries=["cublas", "cudart"],
+        extra_compile_args=CXXFLAGS + ["-O3", "-fstrict-overflow"],
+    ))
+
 setup(ext_modules=extensions, cmdclass={"build_ext": BuildWithCuda})

{scm2cpp_lasso-0.11.0 → scm2cpp_lasso-0.13.0}/src/scm2cpp_lasso/__init__.py

@@ -47,10 +47,11 @@ __all__ = ["CovLasso", "CovRidge", "CovBayesianRidge", "CovLogistic", "CovGroupL
            "LBFGSPoissonRegressor", "LBFGSGammaRegressor", "LBFGSTweedieRegressor",
            "GramLarsCV", "GramLassoLarsCV", "GramOMPCV",
            "EMGaussianMixture", "CovPCA", "ParallelFastICA", "CovARDRegression",
-           "L1LinearSVM", "WeiszfeldTheilSen",
+           "L1LinearSVM", "WeiszfeldTheilSen", "f_regression", "f_classif", "SelectKBest",
+           "SelectPercentile", "GramRFE",
            "CovMultiTaskLasso", "CovMultiTaskLassoCV", "DerivedLasso",
            "cuda_available", "derived_backend", "kernel"]
-__version__ = "0.11.0"
+__version__ = "0.13.0"
 
 _BATCH = load_batch_lib()
 _DP = ctypes.POINTER(ctypes.c_double)
@@ -1349,7 +1350,7 @@ class LBFGSLogisticRegression:
         k = nc if multi else 1
         coef = np.zeros(k * (p + 1))
         out = np.zeros(2)
-        it = _gramops_kernels().logistic_lbfgs(
+        it = _gpu_or_cpu(n * p, getattr(self, "backend", None)).logistic_lbfgs(
             X, yi.astype(np.float64), coef, out, float(self.C), int(self.max_iter),
             float(self.tol), multi, 1 if self.fit_intercept else 0, n, p, k)
         self.coef_ = coef[:k * p].reshape(k, p).copy()
@@ -1417,7 +1418,7 @@ class _LBFGSGLM:
         coef = np.zeros(p + 1)
         out = np.zeros(2)
         fam, power = self._family_power()
-        it = _gramops_kernels().glm(X, y, coef, out, fam, float(power), float(self.alpha),
+        it = _gpu_or_cpu(n * p, getattr(self, "backend", None)).glm(X, y, coef, out, fam, float(power), float(self.alpha),
                                     int(self.max_iter), float(self.tol),
                                     1 if self.fit_intercept else 0, n, p)
         self.coef_, self.intercept_ = coef[:p].copy(), float(coef[p])
@@ -1498,7 +1499,7 @@ class LBFGSLogisticRegressionCV:
 
     def _path(self, X, yi, Cs, coef, multi, k):
         n, p = X.shape
-        lib = _gramops_kernels()
+        lib = _gpu_or_cpu(n * p, getattr(self, "backend", None))
         coefs, iters = [], []
         out = np.zeros(2)
         for C in Cs:
@@ -1581,7 +1582,7 @@ class LBFGSHuberRegressor:
         coef = np.zeros(p + 2)
         coef[-1] = 1.0
         out = np.zeros(2)
-        it = _gramops_kernels().huber(
+        it = _gpu_or_cpu(n * p, getattr(self, "backend", None)).huber(
             X, y, coef, out, float(self.epsilon), float(self.alpha), int(self.max_iter),
             float(self.tol), 1 if self.fit_intercept else 0, n, p)
         self.coef_ = coef[:p].copy()
@@ -1642,7 +1643,7 @@ class EMGaussianMixture:
         self.precisions_cholesky_ = np.zeros((k, p, p))
         labels = np.zeros(n)
         out = np.zeros(2)
-        it = _gramops_kernels().gmm(X, resp, self.weights_, self.means_,
+        it = _gpu_or_cpu(n * p, getattr(self, "backend", None)).gmm(X, resp, self.weights_, self.means_,
                                     self.precisions_cholesky_, labels, out,
                                     int(self.max_iter), float(self.tol),
                                     float(self.reg_covar), n, p, k)
@@ -1689,14 +1690,15 @@ class CovPCA:
     n_components_.
     """
 
-    def __init__(self, n_components=None):
-        self.n_components = n_components
+    def __init__(self, n_components=None, *, backend=None):
+        self.n_components, self.backend = n_components, backend
 
     def fit(self, X, y=None):
         X = np.ascontiguousarray(X, dtype=np.float64)
         n, p = X.shape
         comps = np.zeros((p, p)); ev = np.zeros(p); sv = np.zeros(p); mean = np.zeros(p)
-        _gramops_kernels().pca(X, comps, ev, sv, mean, n, p)
+        lib = _accel(float(n) * p * p) if self.backend in (None, "cublas") else None
+        (lib or _gramops_kernels()).pca(X, comps, ev, sv, mean, n, p)
         k = min(n, p) if self.n_components is None else int(self.n_components)
         self.n_components_ = k
         self.mean_ = mean
@@ -1742,7 +1744,7 @@ class ParallelFastICA:
         w0 = np.ascontiguousarray(w0)
         self.components_ = np.zeros((k, p)); self.whitening_ = np.zeros((k, p))
         self._unmixing = np.zeros((k, k)); self.mean_ = np.zeros(p)
-        self.n_iter_ = _gramops_kernels().fastica(
+        self.n_iter_ = _gpu_or_cpu(n * p, getattr(self, "backend", None)).fastica(
             X, w0, self.components_, self.whitening_, self._unmixing, self.mean_,
             int(self.max_iter), float(self.tol), n, p, k)
         self.mixing_ = np.linalg.pinv(self.components_)
@@ -1801,11 +1803,220 @@ class CovARDRegression:
         return np.ascontiguousarray(X, dtype=np.float64) @ self.coef_ + self.intercept_
 
 
+def _gpu_or_cpu(size, backend=None):
+    """The --cuda translation (kernels and device-resident cuBLAS) when it
+    was built, a device answers, the backend allows it and the problem has
+    at least a million entries; the CPU library otherwise."""
+    import os
+    from ._libfind import gpu_built
+    if (backend in (None, "cuda") and size >= 1e6 and not os.environ.get("SCM2CPP_NO_DEVICE")
+            and gpu_built() and cuda_available()):
+        import importlib
+        return importlib.import_module(f"{__name__}._generated._loader_gpu_cuda")
+    return _gramops_kernels()
+
+
+def _accel(work):
+    """The --cublas build of the selection and Gram products, when it was
+    built, a device answers, and WORK (flops) is large enough to pay for
+    the transfer; else None."""
+    import os
+    from ._libfind import accel_built
+    if (work < 2e9 or os.environ.get("SCM2CPP_NO_DEVICE") or not accel_built()
+            or not cuda_available()):
+        return None
+    import importlib
+    return importlib.import_module(f"{__name__}._generated._loader_accel_cublas")
+
+
+def _gram(Xc, backend=None):
+    """X'X: on the device for a large product when one answers, else numpy."""
+    n, p = Xc.shape
+    lib = _accel(float(n) * p * p) if backend in (None, "cublas") else None
+    if lib is None:
+        return np.ascontiguousarray(Xc.T @ Xc)
+    G = np.zeros((p, p))
+    lib.gram_xx(np.ascontiguousarray(Xc), G, n, p)
+    return G
+
+
+def _cpu_select_lib():
+    return _gramops_kernels()
+
+
+def f_regression(X, y, *, center=True, backend=None):
+    """scikit-learn's ``f_regression`` translated (``fscore.scm``): F and
+    p-values of each feature against y.  ``y`` may also be n x q, the q
+    targets in one call (one product X'Y, on the device when one answers
+    and the product is large) -- what a loop of scikit-learn's calls
+    gives, column by column.  Infinite F (perfect correlation) becomes
+    the largest double and NaN F zero with p-value 1, as
+    force_finite=True has it.  p-values need scipy (else None)."""
+    X = np.ascontiguousarray(X, dtype=np.float64)
+    Y = np.asarray(y, dtype=np.float64)
+    one = Y.ndim == 1
+    Y = np.ascontiguousarray(Y.reshape(len(Y), -1))
+    n, p = X.shape
+    q = Y.shape[1]
+    lib = _accel(float(n) * p * q) if backend in (None, "cublas") else None
+    if lib is None:
+        lib = _cpu_select_lib()
+    r = np.zeros((p, q))
+    F = np.zeros((p, q))
+    with np.errstate(divide="ignore", invalid="ignore"):
+        lib.f_regression(X, Y, r, F, 1 if center else 0, n, p, q)
+    dof = n - (2 if center else 1)
+    try:
+        from scipy import stats
+        pv = stats.f.sf(F, 1, dof)
+    except ImportError:
+        pv = None
+    inf = np.isinf(F)
+    F[inf] = np.finfo(np.float64).max
+    nan = np.isnan(F)
+    F[nan] = 0.0
+    if pv is not None:
+        pv[nan] = 1.0
+    if one:
+        return F[:, 0], (None if pv is None else pv[:, 0])
+    return F, pv
+
+
+def f_classif(X, y):
+    """scikit-learn's ``f_classif`` translated (``fscore.scm``: f_oneway's
+    sums of squares, the class sums one product).  p-values need scipy."""
+    X = np.ascontiguousarray(X, dtype=np.float64)
+    classes, lab = np.unique(np.asarray(y), return_inverse=True)
+    n, p = X.shape
+    k = len(classes)
+    F = np.zeros(p)
+    with np.errstate(divide="ignore", invalid="ignore"):
+        _cpu_select_lib().f_classif(X, lab.astype(np.int32), F, n, p, k)
+    try:
+        from scipy import special
+        pv = special.fdtrc(k - 1, n - k, F)
+    except ImportError:
+        pv = None
+    return F, pv
+
+
+def _clean_nans(scores):
+    scores = np.array(scores, dtype=np.float64, copy=True)
+    scores[np.isnan(scores)] = np.finfo(np.float64).min
+    return scores
+
+
+class _SelectBase:
+    def fit(self, X, y):
+        sf = self.score_func
+        fn = {"f_classif": f_classif, "f_regression": f_regression}.get(sf, sf)
+        self.scores_, self.pvalues_ = fn(X, y)
+        return self
+
+    def transform(self, X):
+        return np.asarray(X)[:, self.get_support()]
+
+    def fit_transform(self, X, y):
+        return self.fit(X, y).transform(X)
+
+
+class SelectKBest(_SelectBase):
+    """scikit-learn's ``SelectKBest`` over the translated F statistics
+    (``score_func`` "f_classif", "f_regression", or one of this module's
+    functions): the k best scores by a stable sort, NaN scores last.
+
+    Attributes after fit: scores_, pvalues_.
+    """
+
+    def __init__(self, score_func="f_classif", *, k=10):
+        self.score_func, self.k = score_func, k
+
+    def get_support(self):
+        if self.k == "all":
+            return np.ones(len(self.scores_), dtype=bool)
+        mask = np.zeros(len(self.scores_), dtype=bool)
+        if self.k == 0:
+            return mask
+        mask[np.argsort(_clean_nans(self.scores_), kind="mergesort")[-self.k:]] = True
+        return mask
+
+
+class SelectPercentile(_SelectBase):
+    """scikit-learn's ``SelectPercentile`` over the translated F
+    statistics: scores above the (100 - percentile)th percentile, ties
+    at it kept in order up to the percentile's count.
+
+    Attributes after fit: scores_, pvalues_.
+    """
+
+    def __init__(self, score_func="f_classif", *, percentile=10):
+        self.score_func, self.percentile = score_func, percentile
+
+    def get_support(self):
+        if self.percentile == 100:
+            return np.ones(len(self.scores_), dtype=bool)
+        if self.percentile == 0:
+            return np.zeros(len(self.scores_), dtype=bool)
+        scores = _clean_nans(self.scores_)
+        threshold = np.percentile(scores, 100 - self.percentile)
+        mask = scores > threshold
+        ties = np.where(scores == threshold)[0]
+        if len(ties):
+            max_feats = int(len(scores) * self.percentile / 100)
+            mask[ties[: max_feats - mask.sum()]] = True
+        return mask
+
+
+class GramRFE:
+    """scikit-learn's ``RFE`` around ``Ridge(alpha)`` (``alpha=0``:
+    ``LinearRegression()``) translated whole (``rfe.scm``): the centred
+    Gram formed once (on the device for a large one when one answers),
+    each round's fit from its Cholesky factor with the dropped features
+    taken out by plane rotations, the features ranked by coef^2 in a
+    stable order.  scikit-learn refits from X every round; the ranking
+    and the final coefficients are the same to rounding.
+
+    Attributes after fit: support_, ranking_, n_features_, coef_,
+    intercept_.
+    """
+
+    def __init__(self, alpha=1.0, *, n_features_to_select=None, step=1, backend=None):
+        self.alpha, self.n_features_to_select, self.step = alpha, n_features_to_select, step
+        self.backend = backend
+
+    def fit(self, X, y):
+        X = np.ascontiguousarray(X, dtype=np.float64)
+        y = np.ascontiguousarray(y, dtype=np.float64)
+        n, p = X.shape
+        k = self.n_features_to_select
+        nsel = p // 2 if k is None else (int(k) if isinstance(k, (int, np.integer)) else int(p * k))
+        step = int(max(1, self.step * p)) if 0.0 < self.step < 1.0 else int(self.step)
+        lib = _accel(float(n) * p * p) if self.backend in (None, "cublas") else None
+        if lib is None:
+            lib = _cpu_select_lib()
+        coef = np.zeros(p)
+        support = np.zeros(p, dtype=np.int32)
+        ranking = np.zeros(p, dtype=np.int32)
+        lib.rfe(X, y, coef, support, ranking, float(self.alpha), nsel, step, n, p)
+        self.support_ = support.astype(bool)
+        self.ranking_ = ranking.astype(int)
+        self.n_features_ = int(self.support_.sum())
+        self.coef_ = coef[self.support_].copy()
+        self.intercept_ = float(y.mean() - X.mean(axis=0)[self.support_] @ self.coef_)
+        return self
+
+    def transform(self, X):
+        return np.asarray(X)[:, self.support_]
+
+    def predict(self, X):
+        return self.transform(np.asarray(X, dtype=np.float64)) @ self.coef_ + self.intercept_
+
+
 class _GramLarsBase:
     def _fit_lars(self, X, y, method, alpha, max_iter):
         Xc, yc, xm, ym = _center(X, y, self.fit_intercept)
         n, p = Xc.shape
-        G = np.ascontiguousarray(Xc.T @ Xc)
+        G = _gram(Xc)
         cov = Xc.T @ yc
         self.coef_ = np.zeros(p)
         self.n_iter_ = _classifier_kernels().lars(
@@ -1868,7 +2079,7 @@ class _GramLarsCVBase:
 
     def _lars(self, Xc, yc, alpha_min, rec):
         n, p = Xc.shape
-        G = np.ascontiguousarray(Xc.T @ Xc)
+        G = _gram(Xc)
         cov = Xc.T @ yc
         coef = np.zeros(p)
         path = np.zeros((self.max_iter + 1, p)) if rec else np.zeros(1)
@@ -1889,7 +2100,7 @@ class _GramLarsCVBase:
                 xm, ym = Xtr.mean(axis=0), ytr.mean()
                 Xtr, Xte, ytr, yte = Xtr - xm, Xte - xm, ytr - ym, yte - ym
             _, _, path, alphas = self._lars(np.ascontiguousarray(Xtr), ytr, 0.0, True)
-            folds.append((alphas, path, Xte.T @ Xte, Xte.T @ yte, float(yte @ yte), len(test)))
+            folds.append((alphas, path, _gram(Xte), Xte.T @ yte, float(yte @ yte), len(test)))
         all_alphas = np.unique(np.concatenate([f[0] for f in folds]))
         stride = int(max(1, int(len(all_alphas) / float(self.max_n_alphas))))
         all_alphas = all_alphas[::stride]
@@ -1998,7 +2209,7 @@ class GramLassoLarsIC:
     def fit(self, X, y):
         Xc, yc, xm, ym = _center(X, y, self.fit_intercept)
         n, p = Xc.shape
-        G0 = np.ascontiguousarray(Xc.T @ Xc)
+        G0 = _gram(Xc)
         cov0 = Xc.T @ yc
         yy = float(yc @ yc)
         path = np.zeros((self.max_iter + 1, p))

scm2cpp_lasso-0.13.0/src/scm2cpp_lasso/_generated/_loader_accel_cublas.py

@@ -0,0 +1,140 @@
+# ctypes loader for libaccel.so, generated alongside it.
+# Arrays are numpy arrays of the declared dtype; they are made
+# contiguous on the way in and mutated in place where the translated
+# function mutates them.
+import ctypes
+import numpy as np
+from pathlib import Path
+
+from .._libfind import load_accel_lib as _load_accel_lib
+
+_lib = _load_accel_lib()
+
+_lib.scm2cpp_f_regression.restype = ctypes.c_int
+_lib.scm2cpp_f_regression.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int]
+def f_regression(x, y, r, f, center, n, p, q):
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    y = np.ascontiguousarray(y, dtype=np.float64)
+    r = np.ascontiguousarray(r, dtype=np.float64)
+    f = np.ascontiguousarray(f, dtype=np.float64)
+    return _lib.scm2cpp_f_regression(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), y.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), r.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), f.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), center, n, p, q)
+
+_lib.scm2cpp_f_classif.restype = ctypes.c_int
+_lib.scm2cpp_f_classif.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_int), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int, ctypes.c_int]
+def f_classif(x, lab, f, n, p, k):
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    lab = np.ascontiguousarray(lab, dtype=np.int32)
+    f = np.ascontiguousarray(f, dtype=np.float64)
+    return _lib.scm2cpp_f_classif(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), lab.ctypes.data_as(ctypes.POINTER(ctypes.c_int)), f.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), n, p, k)
+
+_lib.scm2cpp_min_pos.restype = ctypes.c_double
+_lib.scm2cpp_min_pos.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_int]
+def min_pos(v, m):
+    v = np.ascontiguousarray(v, dtype=np.float64)
+    return _lib.scm2cpp_min_pos(v.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), m)
+
+_lib.scm2cpp_round_even.restype = ctypes.c_double
+_lib.scm2cpp_round_even.argtypes = [ctypes.c_double]
+def round_even(y):
+    return _lib.scm2cpp_round_even(y)
+
+_lib.scm2cpp_cholesky_delete.restype = ctypes.c_int
+_lib.scm2cpp_cholesky_delete.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int, ctypes.c_int]
+def cholesky_delete(l, go, n, k):
+    l = np.ascontiguousarray(l, dtype=np.float64)
+    return _lib.scm2cpp_cholesky_delete(l.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), go, n, k)
+
+_lib.scm2cpp_swap_rc.restype = ctypes.c_int
+_lib.scm2cpp_swap_rc.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int, ctypes.c_int]
+def swap_rc(g, a, b, p):
+    g = np.ascontiguousarray(g, dtype=np.float64)
+    return _lib.scm2cpp_swap_rc(g.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), a, b, p)
+
+_lib.scm2cpp_lars.restype = ctypes.c_int
+_lib.scm2cpp_lars.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_double, ctypes.c_double, ctypes.c_int, ctypes.c_int, ctypes.c_double, ctypes.c_int]
+def lars(g, cov, coef, path, alphas, rec, nsamples, alphamin, method, maxiter, eps, p):
+    g = np.ascontiguousarray(g, dtype=np.float64)
+    cov = np.ascontiguousarray(cov, dtype=np.float64)
+    coef = np.ascontiguousarray(coef, dtype=np.float64)
+    path = np.ascontiguousarray(path, dtype=np.float64)
+    alphas = np.ascontiguousarray(alphas, dtype=np.float64)
+    return _lib.scm2cpp_lars(g.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), cov.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), coef.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), path.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), alphas.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), rec, nsamples, alphamin, method, maxiter, eps, p)
+
+_lib.scm2cpp_rfe_chol.restype = ctypes.c_int
+_lib.scm2cpp_rfe_chol.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_int), ctypes.POINTER(ctypes.c_double), ctypes.c_double, ctypes.c_int, ctypes.c_int]
+def rfe_chol(g, idx, l, alpha, k, p):
+    g = np.ascontiguousarray(g, dtype=np.float64)
+    idx = np.ascontiguousarray(idx, dtype=np.int32)
+    l = np.ascontiguousarray(l, dtype=np.float64)
+    return _lib.scm2cpp_rfe_chol(g.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), idx.ctypes.data_as(ctypes.POINTER(ctypes.c_int)), l.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), alpha, k, p)
+
+_lib.scm2cpp_rfe_solve.restype = ctypes.c_int
+_lib.scm2cpp_rfe_solve.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_int), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int]
+def rfe_solve(c, idx, w, l, k, p):
+    c = np.ascontiguousarray(c, dtype=np.float64)
+    idx = np.ascontiguousarray(idx, dtype=np.int32)
+    w = np.ascontiguousarray(w, dtype=np.float64)
+    l = np.ascontiguousarray(l, dtype=np.float64)
+    return _lib.scm2cpp_rfe_solve(c.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), idx.ctypes.data_as(ctypes.POINTER(ctypes.c_int)), w.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), l.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), k, p)
+
+_lib.scm2cpp_rfe.restype = ctypes.c_int
+_lib.scm2cpp_rfe.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_int), ctypes.POINTER(ctypes.c_int), ctypes.c_double, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int]
+def rfe(x, y, coef, support, ranking, alpha, nsel, step, n, p):
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    y = np.ascontiguousarray(y, dtype=np.float64)
+    coef = np.ascontiguousarray(coef, dtype=np.float64)
+    support = np.ascontiguousarray(support, dtype=np.int32)
+    ranking = np.ascontiguousarray(ranking, dtype=np.int32)
+    return _lib.scm2cpp_rfe(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), y.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), coef.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), support.ctypes.data_as(ctypes.POINTER(ctypes.c_int)), ranking.ctypes.data_as(ctypes.POINTER(ctypes.c_int)), alpha, nsel, step, n, p)
+
+_lib.scm2cpp_eig_hypot.restype = ctypes.c_double
+_lib.scm2cpp_eig_hypot.argtypes = [ctypes.c_double, ctypes.c_double]
+def eig_hypot(a, b):
+    return _lib.scm2cpp_eig_hypot(a, b)
+
+_lib.scm2cpp_sym_eigen.restype = ctypes.c_int
+_lib.scm2cpp_sym_eigen.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int]
+def sym_eigen(a, v, ev, e, m):
+    a = np.ascontiguousarray(a, dtype=np.float64)
+    v = np.ascontiguousarray(v, dtype=np.float64)
+    ev = np.ascontiguousarray(ev, dtype=np.float64)
+    e = np.ascontiguousarray(e, dtype=np.float64)
+    return _lib.scm2cpp_sym_eigen(a.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), v.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), ev.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), e.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), m)
+
+_lib.scm2cpp_eig_abs.restype = ctypes.c_double
+_lib.scm2cpp_eig_abs.argtypes = [ctypes.c_double]
+def eig_abs(a):
+    return _lib.scm2cpp_eig_abs(a)
+
+_lib.scm2cpp_eig_order.restype = ctypes.c_int
+_lib.scm2cpp_eig_order.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_int), ctypes.c_int]
+def eig_order(ev, ord, m):
+    ev = np.ascontiguousarray(ev, dtype=np.float64)
+    ord = np.ascontiguousarray(ord, dtype=np.int32)
+    return _lib.scm2cpp_eig_order(ev.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), ord.ctypes.data_as(ctypes.POINTER(ctypes.c_int)), m)
+
+_lib.scm2cpp_pca.restype = ctypes.c_int
+_lib.scm2cpp_pca.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int]
+def pca(x, comps, ev, sv, mean, n, p):
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    comps = np.ascontiguousarray(comps, dtype=np.float64)
+    ev = np.ascontiguousarray(ev, dtype=np.float64)
+    sv = np.ascontiguousarray(sv, dtype=np.float64)
+    mean = np.ascontiguousarray(mean, dtype=np.float64)
+    return _lib.scm2cpp_pca(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), comps.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), ev.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), sv.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), mean.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), n, p)
+
+_lib.scm2cpp_gram_xx.restype = ctypes.c_int
+_lib.scm2cpp_gram_xx.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int]
+def gram_xx(x, g, n, p):
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    g = np.ascontiguousarray(g, dtype=np.float64)
+    return _lib.scm2cpp_gram_xx(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), g.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), n, p)
+
+_lib.scm2cpp_gram_xy.restype = ctypes.c_int
+_lib.scm2cpp_gram_xy.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int, ctypes.c_int]
+def gram_xy(x, y, c, n, p, q):
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    y = np.ascontiguousarray(y, dtype=np.float64)
+    c = np.ascontiguousarray(c, dtype=np.float64)
+    return _lib.scm2cpp_gram_xy(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), y.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), c.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), n, p, q)
+