scm2cpp-lasso 0.5.0.1__tar.gz → 0.5.3__tar.gz

{scm2cpp_lasso-0.5.0.1 → scm2cpp_lasso-0.5.3}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scm2cpp-lasso
-Version: 0.5.0.1
+Version: 0.5.3
 Summary: Lasso by covariance-update coordinate descent, translated from Scheme to C++ by scm2cpp; GPU batch solver included
 Author: Hirotaka Niitsuma
 License: MIT
@@ -70,9 +70,14 @@ from scm2cpp_lasso import cuda_available
 lambdas = model.lambda_grid(num=100)        # from lambda_max down
 path = model.fit_path(lambdas)              # warm-started, sequential
 grid = model.fit_path_batch(lambdas)        # every lambda from zero
+one = model.fit(lambdas[60])                # a single lambda, if you must
 print("GPU:", cuda_available())
 ```
 
+`fit` is the one-element special case of `fit_path` -- the Gram matrix
+was built in the constructor, so a whole warm-started path costs barely
+more than one penalty, which is why the path is the primary interface.
+
 The objective is scikit-learn's, with `fit_intercept=False`:
 
     (1 / 2 nobs) ||y - X b||^2 + lam ||b||_1
@@ -84,6 +89,44 @@ at zero, and `lambda_grid()` walks down from it -- the construction
 scikit-learn uses.  If your columns differ wildly in scale, standardize
 them first; this solver does not do it for you.
 
+## Against the other lassos pip can install
+
+`bench/lasso-compare.py` in the repository runs the same
+cross-validation-shaped grid -- 4096 lambdas, each solved from zero,
+p=200, n=1800 -- against every lasso a `pip install` produces:
+scikit-learn 1.9.0, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
+Every solver runs at its own default tolerance (ours at the tighter
+1e-8), each gets one untimed warm-up fit, cuML's data is on the device
+before the clock starts, and the last column is each solver's
+objective minus ours at the smallest lambda -- a fast row with a loose
+answer would show as one:
+
+| solver                                 | time    | objective gap |
+|----------------------------------------|---------|---------------|
+| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.9 s   | 0             |
+| scm2cpp-lasso, GPU, one thread/lambda  | 0.2 s   | 0             |
+| sklearn `Lasso.fit` per lambda         | 12.5 s  | +1.6e-09      |
+| celer per lambda                       | 17.3 s  | 0             |
+| skglm per lambda                       | 16.4 s  | +2.8e-17      |
+| cuML per lambda, GPU                   | 57.5 s  | +9.1e-07      |
+
+celer and skglm are built for very large sparse designs, where their
+screening rules dominate; at this size they pay their setup per fit.
+cuML parallelises *within* one fit, which wins when a fit is large; at
+this size its launch overhead dominates, while our GPU row
+parallelises *across* lambdas -- one CUDA thread per penalty, the axis
+a cross-validation grid actually offers.
+
+`CovLassoCV` is scikit-learn's `LassoCV` over this machinery: the same
+grid construction, the same contiguous folds, the same
+minimum-mean-MSE choice -- and where the two disagree it is by one
+grid step on a near-tie, which sklearn also resolves our way once its
+tolerance is tightened.  A fold's training Gram is a subtraction
+(G - Xf'Xf; the Gram is additive over rows), and on the GPU every fold
+and every alpha is one thread of a single launch.  At cv=10 with 1000
+alphas -- ten thousand independent problems -- that launch takes
+1.2 s against sklearn's 1.7 s, with GPU and CPU agreeing to 1e-14.
+
 ## Which method
 
 `fit_path` walks a single path, each lambda starting from the previous

{scm2cpp_lasso-0.5.0.1 → scm2cpp_lasso-0.5.3}/README.md

@@ -52,9 +52,14 @@ from scm2cpp_lasso import cuda_available
 lambdas = model.lambda_grid(num=100)        # from lambda_max down
 path = model.fit_path(lambdas)              # warm-started, sequential
 grid = model.fit_path_batch(lambdas)        # every lambda from zero
+one = model.fit(lambdas[60])                # a single lambda, if you must
 print("GPU:", cuda_available())
 ```
 
+`fit` is the one-element special case of `fit_path` -- the Gram matrix
+was built in the constructor, so a whole warm-started path costs barely
+more than one penalty, which is why the path is the primary interface.
+
 The objective is scikit-learn's, with `fit_intercept=False`:
 
     (1 / 2 nobs) ||y - X b||^2 + lam ||b||_1
@@ -66,6 +71,44 @@ at zero, and `lambda_grid()` walks down from it -- the construction
 scikit-learn uses.  If your columns differ wildly in scale, standardize
 them first; this solver does not do it for you.
 
+## Against the other lassos pip can install
+
+`bench/lasso-compare.py` in the repository runs the same
+cross-validation-shaped grid -- 4096 lambdas, each solved from zero,
+p=200, n=1800 -- against every lasso a `pip install` produces:
+scikit-learn 1.9.0, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
+Every solver runs at its own default tolerance (ours at the tighter
+1e-8), each gets one untimed warm-up fit, cuML's data is on the device
+before the clock starts, and the last column is each solver's
+objective minus ours at the smallest lambda -- a fast row with a loose
+answer would show as one:
+
+| solver                                 | time    | objective gap |
+|----------------------------------------|---------|---------------|
+| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.9 s   | 0             |
+| scm2cpp-lasso, GPU, one thread/lambda  | 0.2 s   | 0             |
+| sklearn `Lasso.fit` per lambda         | 12.5 s  | +1.6e-09      |
+| celer per lambda                       | 17.3 s  | 0             |
+| skglm per lambda                       | 16.4 s  | +2.8e-17      |
+| cuML per lambda, GPU                   | 57.5 s  | +9.1e-07      |
+
+celer and skglm are built for very large sparse designs, where their
+screening rules dominate; at this size they pay their setup per fit.
+cuML parallelises *within* one fit, which wins when a fit is large; at
+this size its launch overhead dominates, while our GPU row
+parallelises *across* lambdas -- one CUDA thread per penalty, the axis
+a cross-validation grid actually offers.
+
+`CovLassoCV` is scikit-learn's `LassoCV` over this machinery: the same
+grid construction, the same contiguous folds, the same
+minimum-mean-MSE choice -- and where the two disagree it is by one
+grid step on a near-tie, which sklearn also resolves our way once its
+tolerance is tightened.  A fold's training Gram is a subtraction
+(G - Xf'Xf; the Gram is additive over rows), and on the GPU every fold
+and every alpha is one thread of a single launch.  At cv=10 with 1000
+alphas -- ten thousand independent problems -- that launch takes
+1.2 s against sklearn's 1.7 s, with GPU and CPU agreeing to 1e-14.
+
 ## Which method
 
 `fit_path` walks a single path, each lambda starting from the previous

{scm2cpp_lasso-0.5.0.1 → scm2cpp_lasso-0.5.3}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scm2cpp-lasso"
-version = "0.5.0.1"
+version = "0.5.3"
 description = "Lasso by covariance-update coordinate descent, translated from Scheme to C++ by scm2cpp; GPU batch solver included"
 readme = "README.md"
 requires-python = ">=3.8"

{scm2cpp_lasso-0.5.0.1 → scm2cpp_lasso-0.5.3}/src/scm2cpp_lasso/__init__.py

@@ -31,7 +31,7 @@ from ._libfind import load_batch_lib
 
 __all__ = ["CovLasso", "CovRidge", "CovLogistic", "CovGroupLasso",
            "cuda_available", "kernel"]
-__version__ = "0.5.0.1"
+__version__ = "0.5.3"
 
 _BATCH = load_batch_lib()
 _DP = ctypes.POINTER(ctypes.c_double)
@@ -141,6 +141,17 @@ class CovLasso:
 
     # ---------------------------- fitting ----------------------------
 
+    def fit(self, lam, **kw):
+        """Coefficients at one lambda.
+
+        A single penalty is the one-element special case of the path:
+        the Gram matrix was built in the constructor, so this costs one
+        descent and nothing per row -- and a warm-started path over a
+        whole grid costs barely more, which is why fit_path is the
+        primary interface rather than this convenience.
+        """
+        return self.fit_path([lam], **kw)[0]
+
     def fit_path(self, lambdas, tol=1e-8, chunk=20, max_sweeps=100000,
                  l1_ratio=1.0):
         """Coefficients per lambda, each warm-started from the last.
@@ -246,7 +257,10 @@ def _batch_descend_multi(grams, corrs, lam, nobs, p, tol=1e-8, chunk=20,
     beta = np.zeros((B, p))
     c = np.ascontiguousarray(corrs.copy())
     g = np.ascontiguousarray(grams)
-    lams = np.full(B, float(lam))
+    # lam may be one penalty for every thread or one per thread; the
+    # kernel always reads a per-thread array
+    lams = (np.ascontiguousarray(lam, dtype=np.float64)
+            if np.ndim(lam) else np.full(B, float(lam)))
     if _BATCH is not None and not force_cpu:
         dp = ctypes.POINTER(ctypes.c_double)
         rc = _BATCH.scm2cpp_batch_descend(
@@ -259,11 +273,12 @@ def _batch_descend_multi(grams, corrs, lam, nobs, p, tol=1e-8, chunk=20,
     prev = np.empty(p)
     for b in range(B):
         gb, cb, bb = g[b], c[b], beta[b]
+        lam_b = float(lams[b])
         swept = 0
         while swept < max_sweeps:
             prev[:] = bb
-            kernel_fn(gb, cb, bb, float(lam) * l1_ratio,
-                      float(lam) * (1.0 - l1_ratio), chunk,
+            kernel_fn(gb, cb, bb, lam_b * l1_ratio,
+                      lam_b * (1.0 - l1_ratio), chunk,
                       float(nobs), p)
             swept += chunk
             if np.max(np.abs(bb - prev)) < tol * max(
@@ -272,6 +287,120 @@ def _batch_descend_multi(grams, corrs, lam, nobs, p, tol=1e-8, chunk=20,
     return beta
 
 
+
+class CovLassoCV:
+    """Cross-validated lasso, scikit-learn's LassoCV semantics.
+
+    The lambda grid comes from the full data (lambda_max down, the
+    sklearn construction), the folds are contiguous splits in order
+    (sklearn's KFold default), each fold's whole path is solved from
+    zero -- which is exactly the workload fit_path_batch exists for,
+    one GPU thread per lambda -- and validation error is one matrix
+    product per fold.  alpha_ minimises the mean validation MSE, ties
+    going to the larger penalty as in scikit-learn, and coef_ is the
+    refit on all rows at alpha_.
+
+    Attributes after fit: alphas_, mse_path_ (num, cv), alpha_, coef_.
+    """
+
+    def __init__(self, cv=5, num=100, eps=1e-3, tol=1e-8, l1_ratio=1.0,
+                 force_cpu=False):
+        self.cv = int(cv)
+        self.num = int(num)
+        self.eps = float(eps)
+        self.tol = float(tol)
+        self.l1_ratio = float(l1_ratio)
+        self.force_cpu = force_cpu
+
+    def fit(self, X, y):
+        X = np.ascontiguousarray(X, dtype=np.float64)
+        y = np.ascontiguousarray(y, dtype=np.float64)
+        n = len(y)
+        full = CovLasso(X, y)
+        self.alphas_ = full.lambda_grid(num=self.num, eps=self.eps,
+                                        l1_ratio=self.l1_ratio)
+        # contiguous folds in row order, sizes as even as they come --
+        # sklearn's KFold(shuffle=False)
+        bounds = np.linspace(0, n, self.cv + 1).astype(int)
+        self.mse_path_ = np.empty((self.num, self.cv))
+        gpu = cuda_available() and not self.force_cpu
+        G = full.g.reshape(full.p, full.p)
+        if gpu:
+            return self._fit_gpu(X, y, full, G, bounds, n)
+        for k in range(self.cv):
+            lo, hi = bounds[k], bounds[k + 1]
+            Xf, yf = X[lo:hi], y[lo:hi]
+            # The Gram is additive over rows, so a fold's training Gram
+            # is a subtraction: G_full - Xf'Xf costs O(n_fold p^2) where
+            # rebuilding from the complement costs O((n - n_fold) p^2).
+            # Over five folds that is the whole Gram work twice instead
+            # of four times, and the Gram is the dominant O(n p^2) step.
+            m = CovLasso(gram=(G - Xf.T @ Xf).ravel(),
+                         corr=full.c0 - Xf.T @ yf,
+                         nobs=n - (hi - lo))
+            # On the GPU every lambda is its own thread, so the batch is
+            # the right shape; on the CPU the warm-started path is: after
+            # the Gram, each lambda resumes where the previous stopped
+            # and costs O(p) per moving coordinate. That warm path is
+            # what makes searching the alpha grid cheap -- the grid
+            # costs barely more than its hardest single alpha.
+            if gpu:
+                path = m.fit_path_batch(self.alphas_, tol=self.tol,
+                                        l1_ratio=self.l1_ratio)
+            else:
+                path = m.fit_path(self.alphas_, tol=self.tol,
+                                  l1_ratio=self.l1_ratio)
+            resid = Xf @ path.T - yf[:, None]
+            self.mse_path_[:, k] = np.mean(resid * resid, axis=0)
+        mean = self.mse_path_.mean(axis=1)
+        # alphas_ descends, so argmin's first hit is the larger penalty
+        self.alpha_ = float(self.alphas_[int(np.argmin(mean))])
+        self.coef_ = full.fit(self.alpha_, tol=self.tol,
+                              l1_ratio=self.l1_ratio)
+        return self
+
+    def _fit_gpu(self, X, y, full, G, bounds, n):
+        # Every fold and every alpha is an independent problem, so all of
+        # them go to the device as one batch: cv * num threads, each with
+        # its fold's Gram (the per-thread-Gram path the bootstrap added)
+        # and its own penalty. Fold sizes may differ by a row; the kernel
+        # only ever uses lam through the product lam * nobs, so each
+        # thread's nobs is absorbed into its lambda and nobs is passed
+        # as 1 -- exact, not approximate.
+        p_ = full.p
+        num, cv = self.num, self.cv
+        grams = np.empty((cv, p_ * p_))
+        corrs = np.empty((cv, p_))
+        nobs_k = np.empty(cv)
+        for k in range(cv):
+            lo, hi = bounds[k], bounds[k + 1]
+            Xf, yf = X[lo:hi], y[lo:hi]
+            grams[k] = (G - Xf.T @ Xf).ravel()
+            corrs[k] = full.c0 - Xf.T @ yf
+            nobs_k[k] = n - (hi - lo)
+        gr = np.repeat(grams, num, axis=0)
+        cr = np.repeat(corrs, num, axis=0)
+        lams = (np.tile(self.alphas_, cv) * np.repeat(nobs_k, num))
+        betas = _batch_descend_multi(gr, cr, lams, 1.0, p_,
+                                     tol=self.tol,
+                                     l1_ratio=self.l1_ratio,
+                                     force_cpu=self.force_cpu,
+                                     kernel_fn=kernel.enet_descend)
+        betas = betas.reshape(cv, num, p_)
+        for k in range(cv):
+            lo, hi = bounds[k], bounds[k + 1]
+            resid = X[lo:hi] @ betas[k].T - y[lo:hi, None]
+            self.mse_path_[:, k] = np.mean(resid * resid, axis=0)
+        mean = self.mse_path_.mean(axis=1)
+        self.alpha_ = float(self.alphas_[int(np.argmin(mean))])
+        self.coef_ = full.fit(self.alpha_, tol=self.tol,
+                              l1_ratio=self.l1_ratio)
+        return self
+
+    def predict(self, X):
+        return np.ascontiguousarray(X, dtype=np.float64) @ self.coef_
+
+
 class CovRidge:
     """Ridge regression over a design given by its Gram matrix.
 

{scm2cpp_lasso-0.5.0.1 → scm2cpp_lasso-0.5.3}/src/scm2cpp_lasso.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scm2cpp-lasso
-Version: 0.5.0.1
+Version: 0.5.3
 Summary: Lasso by covariance-update coordinate descent, translated from Scheme to C++ by scm2cpp; GPU batch solver included
 Author: Hirotaka Niitsuma
 License: MIT
@@ -70,9 +70,14 @@ from scm2cpp_lasso import cuda_available
 lambdas = model.lambda_grid(num=100)        # from lambda_max down
 path = model.fit_path(lambdas)              # warm-started, sequential
 grid = model.fit_path_batch(lambdas)        # every lambda from zero
+one = model.fit(lambdas[60])                # a single lambda, if you must
 print("GPU:", cuda_available())
 ```
 
+`fit` is the one-element special case of `fit_path` -- the Gram matrix
+was built in the constructor, so a whole warm-started path costs barely
+more than one penalty, which is why the path is the primary interface.
+
 The objective is scikit-learn's, with `fit_intercept=False`:
 
     (1 / 2 nobs) ||y - X b||^2 + lam ||b||_1
@@ -84,6 +89,44 @@ at zero, and `lambda_grid()` walks down from it -- the construction
 scikit-learn uses.  If your columns differ wildly in scale, standardize
 them first; this solver does not do it for you.
 
+## Against the other lassos pip can install
+
+`bench/lasso-compare.py` in the repository runs the same
+cross-validation-shaped grid -- 4096 lambdas, each solved from zero,
+p=200, n=1800 -- against every lasso a `pip install` produces:
+scikit-learn 1.9.0, celer 0.7.4, skglm 0.5, and RAPIDS cuML 26.8.
+Every solver runs at its own default tolerance (ours at the tighter
+1e-8), each gets one untimed warm-up fit, cuML's data is on the device
+before the clock starts, and the last column is each solver's
+objective minus ours at the smallest lambda -- a fast row with a loose
+answer would show as one:
+
+| solver                                 | time    | objective gap |
+|----------------------------------------|---------|---------------|
+| scm2cpp-lasso, 1 CPU core (tol 1e-8)   | 0.9 s   | 0             |
+| scm2cpp-lasso, GPU, one thread/lambda  | 0.2 s   | 0             |
+| sklearn `Lasso.fit` per lambda         | 12.5 s  | +1.6e-09      |
+| celer per lambda                       | 17.3 s  | 0             |
+| skglm per lambda                       | 16.4 s  | +2.8e-17      |
+| cuML per lambda, GPU                   | 57.5 s  | +9.1e-07      |
+
+celer and skglm are built for very large sparse designs, where their
+screening rules dominate; at this size they pay their setup per fit.
+cuML parallelises *within* one fit, which wins when a fit is large; at
+this size its launch overhead dominates, while our GPU row
+parallelises *across* lambdas -- one CUDA thread per penalty, the axis
+a cross-validation grid actually offers.
+
+`CovLassoCV` is scikit-learn's `LassoCV` over this machinery: the same
+grid construction, the same contiguous folds, the same
+minimum-mean-MSE choice -- and where the two disagree it is by one
+grid step on a near-tie, which sklearn also resolves our way once its
+tolerance is tightened.  A fold's training Gram is a subtraction
+(G - Xf'Xf; the Gram is additive over rows), and on the GPU every fold
+and every alpha is one thread of a single launch.  At cv=10 with 1000
+alphas -- ten thousand independent problems -- that launch takes
+1.2 s against sklearn's 1.7 s, with GPU and CPU agreeing to 1e-14.
+
 ## Which method
 
 `fit_path` walks a single path, each lambda starting from the previous