scm2cpp-lasso 0.14.0__tar.gz → 0.15.1__tar.gz

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scm2cpp-lasso
-Version: 0.14.0
+Version: 0.15.1
 Summary: scikit-learn estimators (lasso, elastic net, ridge, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
 Author: Hirotaka Niitsuma
 License: MIT
@@ -107,7 +107,7 @@ them first; this solver does not do it for you.
 From 0.14.0 every estimator with a `fit(X, y)` (or `fit(X)`) derives
 from scikit-learn's `BaseEstimator` and the matching mixin, so `clone`,
 `get_params` / `set_params`, `GridSearchCV`, `Pipeline` and
-`cross_val_score` take them, and all 39 pass
+`cross_val_score` take them, and all 44 pass
 `sklearn.utils.estimator_checks.check_estimator` (`check-sklearn.py`
 runs it; the one expected failure is `LBFGSLogisticRegressionCV`'s
 `n_iter_` of 0 for a C whose warm start is already converged, which
@@ -122,11 +122,37 @@ from scm2cpp_lasso import GramLasso
 GridSearchCV(GramLasso(), {"alpha": [0.01, 0.1, 1.0]}, cv=5).fit(X, y)
 ```
 
-`GramLasso`, `GramElasticNet` and `GramRidge` are the `fit(X, y)`
-forms, with the intercept, over the Gram-matrix solvers; `CovLasso`,
-`CovRidge`, `DerivedLasso`, `CovLogistic`, `CovGroupLasso` and
-`CovMultiTaskLasso` keep their data-in-the-constructor interface,
-which solves a whole path at once.
+`CovLasso`, `CovRidge`, `DerivedLasso`, `CovLogistic`,
+`CovGroupLasso` and `CovMultiTaskLasso` keep their
+data-in-the-constructor interface, which solves a whole path at once;
+each has a `fit(X, y)` form with the intercept beside it (0.15.0):
+
+| data in the constructor | `fit(X, y)` | scikit-learn |
+|---|---|---|
+| `CovLasso` | `GramLasso`, `GramElasticNet` | Lasso, ElasticNet |
+| `CovRidge` | `GramRidge` | Ridge |
+| `DerivedLasso` | `DesignLasso` | Lasso |
+| `CovLogistic` | `GramLogisticRegression` | LogisticRegression(l1_ratio=1, solver="liblinear") |
+| `CovGroupLasso` | `GramGroupLasso` (`groups=None`: the lasso) | (none) |
+| `CovMultiTaskLasso` | `GramMultiTaskLasso`, `GramMultiTaskElasticNet` | MultiTaskLasso, MultiTaskElasticNet |
+
+Against scikit-learn on 400 x 30 with an intercept: `DesignLasso` and
+`GramGroupLasso(groups=None)` agree with `Lasso` to 1.5e-10, the
+multi-task pair with `MultiTaskLasso` / `MultiTaskElasticNet` to
+5e-14, and `GramLogisticRegression` reaches liblinear's objective to
+eight digits with the same predictions.
+
+Against the class each one wraps (one alpha, one core), which fits no
+intercept and is unchanged: with `fit_intercept=False` the input checks
+add 15 to 30 us a call (up to +50% on a 0.06 ms fit) and the finiteness
+sum 8 to 14% on 20000 x 200 (none under `assume_finite`); with the
+intercept, the column means add 20 to 30% on 20000 x 200 for the
+Gram-matrix ones, which correct X'X
+for the means rather than copy X to centre it, and about 55% for
+`DesignLasso`, whose kernel needs the centred X itself.
+`GramLogisticRegression` is slower than liblinear by its algorithm, 2.5x
+at 20000 x 200 and more on small problems; `L1LogisticRegression` is
+the fast one for that problem.
 
 What it costs is the input checks: X must be a finite 2-D float64
 array with the feature count of the fit.  A plain C-ordered float64
@@ -166,6 +192,11 @@ package picks the fastest one present.
 | `GramLasso` | Lasso(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
 | `GramElasticNet` | ElasticNet(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
 | `GramRidge` | Ridge, fit(X, y) with the intercept, one or many targets | numpy (Cholesky) | CPU | Ridge |
+| `GramMultiTaskLasso` | MultiTaskLasso, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
+| `GramMultiTaskElasticNet` | MultiTaskElasticNet, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
+| `GramGroupLasso` | (none; skglm GroupLasso), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | - |
+| `GramLogisticRegression` | LogisticRegression(l1_ratio=1, solver='liblinear'), by majorization | `lasso-cov.scm` | CPU | no |
+| `DesignLasso` | Lasso, fit(X, y) with the intercept, from the design | `lasso-design.scm (--derive)` | loops / BLAS / cuBLAS | no |
 | `CovLasso` | Lasso, ElasticNet, lasso_path(precompute=True) | `lasso-cov.scm` | CPU; GPU batch (fit_path_batch, bootstrap) | no |
 | `CovLassoCV` | LassoCV, ElasticNetCV | `lasso-cov.scm` | CPU; GPU batch | no |
 | `DerivedLasso` | lasso_path, from the design | `lasso-design.scm (--derive)` | loops / BLAS / cuBLAS | no |
@@ -612,6 +643,15 @@ matrix (`KernelSVM`) or transposing X into columns
 | `L1LogisticRegression`, 20,000 x 500, C=0.05 | 1.20 s | 2.23 s | 1.9x | 106.77913 / 106.77915 | 100% |
 | `L1LogisticRegression`, 100,000 x 200, C=0.05 | 9.06 s | 18.7 s | 2.1x | 349.61154 / 349.61216 | 100% |
 
+On 2,000 x 20,000 neither side reaches the optimum at tol 1e-4 (the
+objective at tol 1e-10 is 0.0519588), and where each stops depends on
+the coordinate order: over eight seeds ours stops at 0.05240 to
+0.05251 (0.9 to 1.1% above the optimum), `LinearSVC`, whose order is
+random unless `random_state` is given, at 0.05242 to 0.05319 (0.9 to
+2.4%). When `LinearSVC` draws one of its worse orders the gap passes
+the script's 0.5% bound and the row prints `<-- disagrees`; ours is
+then the better solution of the two, so the flag there is not a fault.
+
 The regressors, from the same script (C = 1, epsilon 0.1):
 
 | problem | ours | scikit-learn | ratio | agreement |

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/README.md

@@ -89,7 +89,7 @@ them first; this solver does not do it for you.
 From 0.14.0 every estimator with a `fit(X, y)` (or `fit(X)`) derives
 from scikit-learn's `BaseEstimator` and the matching mixin, so `clone`,
 `get_params` / `set_params`, `GridSearchCV`, `Pipeline` and
-`cross_val_score` take them, and all 39 pass
+`cross_val_score` take them, and all 44 pass
 `sklearn.utils.estimator_checks.check_estimator` (`check-sklearn.py`
 runs it; the one expected failure is `LBFGSLogisticRegressionCV`'s
 `n_iter_` of 0 for a C whose warm start is already converged, which
@@ -104,11 +104,37 @@ from scm2cpp_lasso import GramLasso
 GridSearchCV(GramLasso(), {"alpha": [0.01, 0.1, 1.0]}, cv=5).fit(X, y)
 ```
 
-`GramLasso`, `GramElasticNet` and `GramRidge` are the `fit(X, y)`
-forms, with the intercept, over the Gram-matrix solvers; `CovLasso`,
-`CovRidge`, `DerivedLasso`, `CovLogistic`, `CovGroupLasso` and
-`CovMultiTaskLasso` keep their data-in-the-constructor interface,
-which solves a whole path at once.
+`CovLasso`, `CovRidge`, `DerivedLasso`, `CovLogistic`,
+`CovGroupLasso` and `CovMultiTaskLasso` keep their
+data-in-the-constructor interface, which solves a whole path at once;
+each has a `fit(X, y)` form with the intercept beside it (0.15.0):
+
+| data in the constructor | `fit(X, y)` | scikit-learn |
+|---|---|---|
+| `CovLasso` | `GramLasso`, `GramElasticNet` | Lasso, ElasticNet |
+| `CovRidge` | `GramRidge` | Ridge |
+| `DerivedLasso` | `DesignLasso` | Lasso |
+| `CovLogistic` | `GramLogisticRegression` | LogisticRegression(l1_ratio=1, solver="liblinear") |
+| `CovGroupLasso` | `GramGroupLasso` (`groups=None`: the lasso) | (none) |
+| `CovMultiTaskLasso` | `GramMultiTaskLasso`, `GramMultiTaskElasticNet` | MultiTaskLasso, MultiTaskElasticNet |
+
+Against scikit-learn on 400 x 30 with an intercept: `DesignLasso` and
+`GramGroupLasso(groups=None)` agree with `Lasso` to 1.5e-10, the
+multi-task pair with `MultiTaskLasso` / `MultiTaskElasticNet` to
+5e-14, and `GramLogisticRegression` reaches liblinear's objective to
+eight digits with the same predictions.
+
+Against the class each one wraps (one alpha, one core), which fits no
+intercept and is unchanged: with `fit_intercept=False` the input checks
+add 15 to 30 us a call (up to +50% on a 0.06 ms fit) and the finiteness
+sum 8 to 14% on 20000 x 200 (none under `assume_finite`); with the
+intercept, the column means add 20 to 30% on 20000 x 200 for the
+Gram-matrix ones, which correct X'X
+for the means rather than copy X to centre it, and about 55% for
+`DesignLasso`, whose kernel needs the centred X itself.
+`GramLogisticRegression` is slower than liblinear by its algorithm, 2.5x
+at 20000 x 200 and more on small problems; `L1LogisticRegression` is
+the fast one for that problem.
 
 What it costs is the input checks: X must be a finite 2-D float64
 array with the feature count of the fit.  A plain C-ordered float64
@@ -148,6 +174,11 @@ package picks the fastest one present.
 | `GramLasso` | Lasso(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
 | `GramElasticNet` | ElasticNet(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
 | `GramRidge` | Ridge, fit(X, y) with the intercept, one or many targets | numpy (Cholesky) | CPU | Ridge |
+| `GramMultiTaskLasso` | MultiTaskLasso, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
+| `GramMultiTaskElasticNet` | MultiTaskElasticNet, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
+| `GramGroupLasso` | (none; skglm GroupLasso), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | - |
+| `GramLogisticRegression` | LogisticRegression(l1_ratio=1, solver='liblinear'), by majorization | `lasso-cov.scm` | CPU | no |
+| `DesignLasso` | Lasso, fit(X, y) with the intercept, from the design | `lasso-design.scm (--derive)` | loops / BLAS / cuBLAS | no |
 | `CovLasso` | Lasso, ElasticNet, lasso_path(precompute=True) | `lasso-cov.scm` | CPU; GPU batch (fit_path_batch, bootstrap) | no |
 | `CovLassoCV` | LassoCV, ElasticNetCV | `lasso-cov.scm` | CPU; GPU batch | no |
 | `DerivedLasso` | lasso_path, from the design | `lasso-design.scm (--derive)` | loops / BLAS / cuBLAS | no |
@@ -594,6 +625,15 @@ matrix (`KernelSVM`) or transposing X into columns
 | `L1LogisticRegression`, 20,000 x 500, C=0.05 | 1.20 s | 2.23 s | 1.9x | 106.77913 / 106.77915 | 100% |
 | `L1LogisticRegression`, 100,000 x 200, C=0.05 | 9.06 s | 18.7 s | 2.1x | 349.61154 / 349.61216 | 100% |
 
+On 2,000 x 20,000 neither side reaches the optimum at tol 1e-4 (the
+objective at tol 1e-10 is 0.0519588), and where each stops depends on
+the coordinate order: over eight seeds ours stops at 0.05240 to
+0.05251 (0.9 to 1.1% above the optimum), `LinearSVC`, whose order is
+random unless `random_state` is given, at 0.05242 to 0.05319 (0.9 to
+2.4%). When `LinearSVC` draws one of its worse orders the gap passes
+the script's 0.5% bound and the row prints `<-- disagrees`; ours is
+then the better solution of the two, so the flag there is not a fault.
+
 The regressors, from the same script (C = 1, epsilon 0.1):
 
 | problem | ours | scikit-learn | ratio | agreement |

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/pyproject.toml

@@ -4,7 +4,7 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scm2cpp-lasso"
-version = "0.14.0"
+version = "0.15.1"
 description = "scikit-learn estimators (lasso, elastic net, ridge, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included"
 readme = "README.md"
 requires-python = ">=3.8"

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/src/scm2cpp_lasso/__init__.py

@@ -53,8 +53,10 @@ __all__ = ["GramLasso", "GramElasticNet", "GramRidge", "CovLasso", "CovRidge", "
            "L1LinearSVM", "WeiszfeldTheilSen", "f_regression", "f_classif", "SelectKBest",
            "SelectPercentile", "GramRFE",
            "CovMultiTaskLasso", "CovMultiTaskLassoCV", "DerivedLasso",
+           "GramLogisticRegression", "GramGroupLasso", "GramMultiTaskLasso",
+           "GramMultiTaskElasticNet", "DesignLasso",
            "cuda_available", "derived_backend", "kernel"]
-__version__ = "0.14.0"
+__version__ = "0.15.1"
 
 _BATCH = load_batch_lib()
 _DP = ctypes.POINTER(ctypes.c_double)
@@ -361,19 +363,50 @@ class DerivedLasso:
         the column norms (folded into the Gram's diagonal when p < n),
         builds the Gram matrix, walks the penalties, and leaves the
         coefficient at each in a row of the result.  tol is the largest move of
-        one coefficient in a sweep below which that lambda is done.
+        one coefficient in a sweep below which that lambda is done.  The
+        kernel's value, the sweeps run over the whole path, is kept as
+        n_iter_.
         """
         lams = np.ascontiguousarray(lambdas, dtype=np.float64).ravel()
         beta = np.zeros(self.p)
         resid = np.empty(self.nobs)      # filled by the kernel from y
         xnorm = np.empty(self.p)         # the column norms, taken inside
         betas = np.zeros(lams.size * self.p)
-        self._kernel.lasso_design(self.X, self.y, resid, xnorm, beta, lams,
-                                  betas, int(max_sweeps), float(tol),
-                                  self.nobs, self.p, lams.size)
+        self.n_iter_ = int(self._kernel.lasso_design(
+            self.X, self.y, resid, xnorm, beta, lams, betas, int(max_sweeps), float(tol),
+            self.nobs, self.p, lams.size))
         return betas.reshape(lams.size, self.p)
 
 
+class DesignLasso(Tagged, RegressorMixin, BaseEstimator):
+    """scikit-learn's ``Lasso`` interface over DerivedLasso: fit(X, y)
+    centres the data when ``fit_intercept`` and runs the derived kernel
+    at one penalty ``alpha``, same objective as GramLasso.  ``tol`` is
+    the kernel's (the largest absolute move of one coefficient in a
+    sweep), ``max_iter`` its sweep cap and ``backend`` DerivedLasso's.
+
+    Attributes after fit: coef_, intercept_, n_iter_ (the sweeps the
+    kernel ran), n_features_in_.
+    """
+
+    def __init__(self, alpha=1.0, *, fit_intercept=True, tol=1e-8, max_iter=100000,
+                 backend=None):
+        self.alpha, self.fit_intercept, self.tol, self.max_iter = alpha, fit_intercept, tol, max_iter
+        self.backend = backend
+
+    def fit(self, X, y):
+        X, y = fit_xy(self, X, y)
+        Xc, yc, xm, ym = _centered(X, y, self.fit_intercept)
+        m = DerivedLasso(Xc, yc, backend=self.backend)
+        self.coef_ = m.fit(float(self.alpha), tol=float(self.tol), max_sweeps=int(self.max_iter))
+        self.n_iter_ = m.n_iter_
+        self.intercept_ = float(ym - xm @ self.coef_)
+        return self
+
+    def predict(self, X):
+        return pred_x(self, X) @ self.coef_ + self.intercept_
+
+
 def derived_backend():
     """The derived kernel variant DerivedLasso takes by default: 'cublas'
     when built and a device answers, else 'blas', else 'loops'; None
@@ -669,6 +702,28 @@ def _centered(X, y, fit_intercept):
     return np.ascontiguousarray(X - xm), y - ym, xm, ym
 
 
+def _centered_gram(X, y, fit_intercept):
+    """(Xc'Xc, Xc'yc, xm, ym) for the centred data Xc, yc, without
+    forming Xc: X'X - n xm xm' and X'y - n xm ym, which saves the copy
+    of X (as long as the Gram product itself on 20000 x 200 rows, where
+    it took the intercept from free to twice the fit).  The correction
+    cancels digits when a column's mean is large against its spread,
+    about (mean / sd)^2 ulps, so a column with mean over 100 sd (or no
+    spread at all) sends the whole fit through the centred copy."""
+    n = X.shape[0]
+    G, c = X.T @ X, X.T @ y
+    if not fit_intercept:
+        return G, c, np.zeros(X.shape[1]), np.zeros(y.shape[1:]) if y.ndim > 1 else 0.0
+    xm, ym = X.mean(axis=0), y.mean(axis=0)
+    nm2 = n * xm * xm
+    if np.all(np.diagonal(G) - nm2 > 1e-4 * nm2):
+        G -= np.outer(xm, n * xm)
+        c -= np.multiply.outer(n * xm, ym)
+        return G, c, xm, ym
+    Xc, yc = np.ascontiguousarray(X - xm), y - ym
+    return Xc.T @ Xc, Xc.T @ yc, xm, ym
+
+
 class GramElasticNet(Tagged, RegressorMixin, BaseEstimator):
     """scikit-learn's ``ElasticNet(precompute=True)`` interface over
     CovLasso: fit(X, y) centres the data when ``fit_intercept``, forms
@@ -692,10 +747,10 @@ class GramElasticNet(Tagged, RegressorMixin, BaseEstimator):
 
     def fit(self, X, y):
         X, y = fit_xy(self, X, y)
-        Xc, yc, xm, ym = _centered(X, y, self.fit_intercept)
-        n, p = Xc.shape
-        g = np.ascontiguousarray((Xc.T @ Xc).ravel())
-        c = np.ascontiguousarray(Xc.T @ yc)
+        n, p = X.shape
+        G, c, xm, ym = _centered_gram(X, y, self.fit_intercept)
+        g = np.ascontiguousarray(G.ravel())
+        c = np.ascontiguousarray(c)
         lam, r = float(self.alpha), float(self.l1_ratio)
         tol, cap = float(self.tol), int(self.max_iter)
         beta, prev = np.zeros(p), np.empty(p)
@@ -747,11 +802,9 @@ class GramRidge(Tagged, RegressorMixin, BaseEstimator):
 
     def fit(self, X, y):
         X, y = fit_xy(self, X, y, multi_output=True)
-        Xc, yc, xm, ym = _centered(X, y, self.fit_intercept)
-        p = Xc.shape[1]
-        A = Xc.T @ Xc
+        p = X.shape[1]
+        A, b, xm, ym = _centered_gram(X, y, self.fit_intercept)
         A.flat[::p + 1] += float(self.alpha)
-        b = Xc.T @ yc
         try:
             coef = np.linalg.solve(A, b)
         except np.linalg.LinAlgError:
@@ -2686,7 +2739,9 @@ class CovLogistic:
 
     def _fit_one(self, beta, lam, tol, inner_sweeps, max_rounds):
         beta = beta.copy()
+        self._rounds = 0
         for _ in range(max_rounds):
+            self._rounds += 1
             prev = beta.copy()
             # the majorizer at beta: quadratic G/4, working
             # correlations exactly X'(y - mu)
@@ -2699,6 +2754,58 @@ class CovLogistic:
         return beta
 
 
+class GramLogisticRegression(Tagged, ClassifierMixin, BaseEstimator):
+    """scikit-learn's ``LogisticRegression(l1_ratio=1, solver="liblinear")``
+    interface over CovLogistic: the same problem,
+
+        min_w  ||w||_1 + C sum_i log(1 + exp(-y_i (w.x_i + b))),
+
+    solved by majorization over the Gram matrix instead of liblinear's
+    Newton steps.  As in liblinear the intercept is the weight of a
+    constant column of value ``intercept_scaling`` and is penalized
+    with the rest.  ``tol`` is CovLogistic's (the largest coefficient
+    move in a round, relative to their size) and ``max_iter`` caps the
+    rounds.  Two classes.
+
+    Attributes after fit: classes_, coef_, intercept_, n_iter_.
+    """
+
+    _binary_only = True
+
+    def __init__(self, C=1.0, *, fit_intercept=True, intercept_scaling=1.0, tol=1e-7,
+                 max_iter=500):
+        self.C, self.fit_intercept, self.intercept_scaling = C, fit_intercept, intercept_scaling
+        self.tol, self.max_iter = tol, max_iter
+
+    def fit(self, X, y):
+        X, y = fit_xy(self, X, y, classifier=True)
+        n = X.shape[0]
+        self.classes_, ys = _binary_labels(y)
+        s = float(self.intercept_scaling)
+        if self.fit_intercept:
+            X = np.column_stack([X, np.full(n, s)])
+        m = CovLogistic(X, (ys > 0).astype(np.float64))
+        lam = 1.0 / (n * float(self.C))
+        beta = m._fit_one(np.zeros(m.p), lam, float(self.tol), 200, int(self.max_iter))
+        self.n_iter_ = m._rounds
+        if self.fit_intercept:
+            self.coef_, self.intercept_ = beta[:-1].copy(), float(beta[-1]) * s
+        else:
+            self.coef_, self.intercept_ = beta, 0.0
+        return self
+
+    def decision_function(self, X):
+        return pred_x(self, X) @ self.coef_ + self.intercept_
+
+    def predict_proba(self, X):
+        p1 = 1.0 / (1.0 + np.exp(-self.decision_function(X)))
+        return np.column_stack([1.0 - p1, p1])
+
+    def predict(self, X):
+        z = self.decision_function(X)
+        return self.classes_[(z > 0).astype(int)]
+
+
 class CovGroupLasso:
     """Group lasso over a design given by its Gram matrix.
 
@@ -2778,9 +2885,13 @@ class CovGroupLasso:
 
     def _fit_one(self, beta, c, lam, tol, max_sweeps):
         thr = lam * self.nobs
+        self._sweeps = 0
         for _ in range(max_sweeps):
+            self._sweeps += 1
             moved = 0.0
             for g, w, L in zip(self.groups, self.wg, self.Lg):
+                if L <= 0.0:    # all-zero columns: the group stays at zero
+                    continue
                 bg = beta[g]
                 z = bg + c[g] / L
                 nz = float(np.linalg.norm(z))
@@ -2809,6 +2920,41 @@ class CovGroupLasso:
         return quad / self.nobs + lam * pen
 
 
+class GramGroupLasso(Tagged, RegressorMixin, BaseEstimator):
+    """A ``Lasso``-style interface over CovGroupLasso: fit(X, y) centres
+    the data when ``fit_intercept`` and runs the block descent at
+    ``alpha``,
+
+        (1 / 2 n) ||y - X b - b0||^2 + alpha sum_g sqrt(|g|) ||b_g||_2.
+
+    ``groups`` is a list of index arrays partitioning the columns, or
+    None for one group per column (the lasso).  ``tol`` is
+    CovGroupLasso's and ``max_iter`` its sweep cap.
+
+    Attributes after fit: coef_, intercept_, n_iter_, n_features_in_.
+    """
+
+    def __init__(self, alpha=1.0, *, groups=None, fit_intercept=True, tol=1e-8,
+                 max_iter=10000):
+        self.alpha, self.groups, self.fit_intercept = alpha, groups, fit_intercept
+        self.tol, self.max_iter = tol, max_iter
+
+    def fit(self, X, y):
+        X, y = fit_xy(self, X, y)
+        n, p = X.shape
+        G, c, xm, ym = _centered_gram(X, y, self.fit_intercept)
+        groups = [[j] for j in range(p)] if self.groups is None else self.groups
+        m = CovGroupLasso(gram=G, corr=c, nobs=n, groups=groups)
+        beta, _ = m._fit_one(np.zeros(p), m.c0.copy(), float(self.alpha), float(self.tol),
+                             int(self.max_iter))
+        self.coef_, self.n_iter_ = beta, m._sweeps
+        self.intercept_ = float(ym - xm @ self.coef_)
+        return self
+
+    def predict(self, X):
+        return pred_x(self, X) @ self.coef_ + self.intercept_
+
+
 class CovMultiTaskLasso:
     """Multi-task lasso and elastic net over a design's Gram matrix.
 
@@ -2929,6 +3075,69 @@ class CovMultiTaskLasso:
         return out.reshape(lambdas.size, self.p, self.n_tasks)
 
 
+class GramMultiTaskElasticNet(Tagged, RegressorMixin, BaseEstimator):
+    """scikit-learn's ``MultiTaskElasticNet`` interface over
+    CovMultiTaskLasso: fit(X, Y) centres the data when
+    ``fit_intercept``, forms the Gram matrix once and runs the row-block
+    sweep at ``alpha``.  Y is (n, n_tasks); coef_ is (n_tasks, p) as
+    scikit-learn has it.  ``tol`` is CovMultiTaskLasso's and
+    ``max_iter`` its sweep cap.
+
+    Attributes after fit: coef_, intercept_, n_iter_, n_features_in_.
+    """
+
+    _multi_output = "only"
+
+    def __init__(self, alpha=1.0, *, l1_ratio=0.5, fit_intercept=True, tol=1e-8,
+                 max_iter=100000):
+        self.alpha, self.l1_ratio, self.fit_intercept = alpha, l1_ratio, fit_intercept
+        self.tol, self.max_iter = tol, max_iter
+
+    def fit(self, X, y):
+        X, y = fit_xy(self, X, y, multi_output=True)
+        if y.ndim != 2:
+            raise ValueError("For mono-task outputs, use GramElasticNet or GramLasso; "
+                             "y must be (n_samples, n_tasks)")
+        n, p = X.shape
+        t = y.shape[1]
+        G, c, xm, ym = _centered_gram(X, y, self.fit_intercept)
+        g = np.ascontiguousarray(G.ravel())
+        c = np.ascontiguousarray(c.ravel())
+        lam, r = float(self.alpha), float(self.l1_ratio)
+        tol, cap = float(self.tol), int(self.max_iter)
+        W, prev = np.zeros(p * t), np.empty(p * t)
+        swept = 0
+        while swept < cap:
+            chunk = min(20, cap - swept)
+            prev[:] = W
+            kernel.mt_descend(g, c, W, lam * r, lam * (1.0 - r), chunk, float(n), p, t)
+            swept += chunk
+            if np.max(np.abs(W - prev)) < tol * _scale(W):
+                break
+        W = W.reshape(p, t)
+        self.coef_, self.n_iter_ = np.ascontiguousarray(W.T), swept
+        self.intercept_ = ym - xm @ W
+        return self
+
+    def predict(self, X):
+        return pred_x(self, X) @ self.coef_.T + self.intercept_
+
+
+class GramMultiTaskLasso(GramMultiTaskElasticNet):
+    """scikit-learn's ``MultiTaskLasso`` interface over CovMultiTaskLasso:
+    GramMultiTaskElasticNet with l1_ratio 1.
+
+    Attributes after fit: coef_, intercept_, n_iter_, n_features_in_.
+    """
+
+    def __init__(self, alpha=1.0, *, fit_intercept=True, tol=1e-8, max_iter=100000):
+        self.alpha, self.fit_intercept, self.tol, self.max_iter = alpha, fit_intercept, tol, max_iter
+
+    @property
+    def l1_ratio(self):
+        return 1.0
+
+
 class CovMultiTaskLassoCV(Tagged, RegressorMixin, BaseEstimator):
     """scikit-learn's ``MultiTaskLassoCV`` (``MultiTaskElasticNetCV``
     with ``l1_ratio``) over the covariance machinery.

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/src/scm2cpp_lasso/_compat.py

@@ -191,7 +191,8 @@ def _reset_names(est):
 
 def fit_xy(est, X, y, *, y_numeric=True, multi_output=False, classifier=False):
     """X as a float64 C-ordered 2-D array and y, validated, n_features_in_ set."""
-    if (_plain(X) and type(y) is np.ndarray and y.ndim == 1 and y.shape[0] == X.shape[0]
+    if (_plain(X) and type(y) is np.ndarray and y.shape[0:1] == X.shape[0:1]
+            and (y.ndim == 1 or (multi_output and y.ndim == 2 and y.shape[1] > 1))
             and _finite(X)):
         if classifier:
             ok = _labels_ok(y)

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/src/scm2cpp_lasso/_generated/derived_blas/lasso_design.hpp

@@ -30,26 +30,28 @@ soft_threshold( double  z,  double  g )
 
 inline int
 lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::span<double> resid,  scm2cpp::span<double> xnorm,  scm2cpp::span<double> beta,  scm2cpp::cspan<double> lams,  scm2cpp::span<double> betas,  int  iters,  double  tol,  int  n,  int  p,  int  nlam )
-{   if( p < n ) {
+{   int swept = 0;
+    if( p < n ) {
         for( int i = 0 ; !(i == n) ; i = (i+1) ) {
             resid[ i ] = y[i]   ;
         }  ;
         0  ;
-        std::vector<double> b024258(p,0.0);
-        std::vector<double> g24257((p*p),0.0);
-        std::vector<double> c24256(p,0.0);
-        scm2cpp::blas_syrk(g24257, x, 1.0, 0.0, 1, p, n)  ;
-        scm2cpp::blas_gemv(c24256, x, resid, 1.0, 0.0, 1, p, n)  ;
+        std::vector<double> b03446(p,0.0);
+        std::vector<double> g3445((p*p),0.0);
+        std::vector<double> c3444(p,0.0);
+        scm2cpp::blas_syrk(g3445, x, 1.0, 0.0, 1, p, n)  ;
+        scm2cpp::blas_gemv(c3444, x, resid, 1.0, 0.0, 1, p, n)  ;
         for( int l = 0 ; !(l == nlam) ; l = (l+1) ) {
             double lam = lams[l];
             int stop = 0;
             for( int sweep = 0 ; !((sweep == iters || stop == 1)) ; sweep = (sweep+1) ) {
+                swept = (swept+1)  ;
                 double maxmove = 0.0;
                 for( int j = 0 ; !(j == p) ; j = (j+1) ) {
-                    double rho = c24256[j];
+                    double rho = c3444[j];
                     double old = beta[j];
-                    rho = (rho+(old*g24257[((j*p)+j)]))  ;
-                    double bnew = (soft_threshold(rho,double((lam*(1.0*n))))/g24257[((j*p)+j)]);
+                    rho = (rho+(old*g3445[((j*p)+j)]))  ;
+                    double bnew = (soft_threshold(rho,double((lam*(1.0*n))))/g3445[((j*p)+j)]);
                     beta[ j ] = bnew   ;
                     if( !(bnew == old) ) {
                         if( (bnew-old) > maxmove ) {
@@ -62,9 +64,9 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
                         } else {
                             0  ;
                         }  ;
-                        b024258[ j ] = (b024258[j]+(bnew-old))   ;
-                        for( int k24260 = 0 ; !(k24260 == p) ; k24260 = (k24260+1) ) {
-                            c24256[ k24260 ] = (c24256[k24260]-(g24257[((j*p)+k24260)]*(bnew-old)))   ;
+                        b03446[ j ] = (b03446[j]+(bnew-old))   ;
+                        for( int k3448 = 0 ; !(k3448 == p) ; k3448 = (k3448+1) ) {
+                            c3444[ k3448 ] = (c3444[k3448]-(g3445[((j*p)+k3448)]*(bnew-old)))   ;
                         }  ;
                     } else {
                         false  ;
@@ -85,7 +87,7 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
             }  ;
             ;
         }  ;
-        scm2cpp::blas_gemv(resid, x, b024258, -1.0, 1.0, 0, n, p)  ;
+        scm2cpp::blas_gemv(resid, x, b03446, -1.0, 1.0, 0, n, p)  ;
         0  ;
         ;
     } else {
@@ -104,6 +106,7 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
             double lam = lams[l];
             int stop = 0;
             for( int sweep = 0 ; !((sweep == iters || stop == 1)) ; sweep = (sweep+1) ) {
+                swept = (swept+1)  ;
                 double maxmove = 0.0;
                 for( int j = 0 ; !(j == p) ; j = (j+1) ) {
                     double rho = 0.0;
@@ -148,6 +151,6 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
             ;
         }  ;
     }  ;
-    return 0 ;
+    return swept ;
 }
 #endif // LASSO_DESIGN_HPP

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/src/scm2cpp_lasso/_generated/derived_cublas/lasso_design.hpp

@@ -30,27 +30,29 @@ soft_threshold( double  z,  double  g )
 
 inline int
 lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::span<double> resid,  scm2cpp::span<double> xnorm,  scm2cpp::span<double> beta,  scm2cpp::cspan<double> lams,  scm2cpp::span<double> betas,  int  iters,  double  tol,  int  n,  int  p,  int  nlam )
-{   if( p < n ) {
+{   int swept = 0;
+    if( p < n ) {
         for( int i = 0 ; !(i == n) ; i = (i+1) ) {
             resid[ i ] = y[i]   ;
         }  ;
         0  ;
-        std::vector<double> b024258(p,0.0);
-        std::vector<double> g24257((p*p),0.0);
-        std::vector<double> c24256(p,0.0);
-        scm2cpp::device_matrix d24266 = scm2cpp::dmat_upload(x, (n*p), 1);
-        scm2cpp::blas_syrk(g24257, d24266, 1.0, 0.0, 1, p, n)  ;
-        scm2cpp::blas_gemv(c24256, d24266, resid, 1.0, 0.0, 1, p, n)  ;
+        std::vector<double> b03446(p,0.0);
+        std::vector<double> g3445((p*p),0.0);
+        std::vector<double> c3444(p,0.0);
+        scm2cpp::device_matrix d3454 = scm2cpp::dmat_upload(x, (n*p), 1);
+        scm2cpp::blas_syrk(g3445, d3454, 1.0, 0.0, 1, p, n)  ;
+        scm2cpp::blas_gemv(c3444, d3454, resid, 1.0, 0.0, 1, p, n)  ;
         for( int l = 0 ; !(l == nlam) ; l = (l+1) ) {
             double lam = lams[l];
             int stop = 0;
             for( int sweep = 0 ; !((sweep == iters || stop == 1)) ; sweep = (sweep+1) ) {
+                swept = (swept+1)  ;
                 double maxmove = 0.0;
                 for( int j = 0 ; !(j == p) ; j = (j+1) ) {
-                    double rho = c24256[j];
+                    double rho = c3444[j];
                     double old = beta[j];
-                    rho = (rho+(old*g24257[((j*p)+j)]))  ;
-                    double bnew = (soft_threshold(rho,double((lam*(1.0*n))))/g24257[((j*p)+j)]);
+                    rho = (rho+(old*g3445[((j*p)+j)]))  ;
+                    double bnew = (soft_threshold(rho,double((lam*(1.0*n))))/g3445[((j*p)+j)]);
                     beta[ j ] = bnew   ;
                     if( !(bnew == old) ) {
                         if( (bnew-old) > maxmove ) {
@@ -63,9 +65,9 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
                         } else {
                             0  ;
                         }  ;
-                        b024258[ j ] = (b024258[j]+(bnew-old))   ;
-                        for( int k24260 = 0 ; !(k24260 == p) ; k24260 = (k24260+1) ) {
-                            c24256[ k24260 ] = (c24256[k24260]-(g24257[((j*p)+k24260)]*(bnew-old)))   ;
+                        b03446[ j ] = (b03446[j]+(bnew-old))   ;
+                        for( int k3448 = 0 ; !(k3448 == p) ; k3448 = (k3448+1) ) {
+                            c3444[ k3448 ] = (c3444[k3448]-(g3445[((j*p)+k3448)]*(bnew-old)))   ;
                         }  ;
                     } else {
                         false  ;
@@ -86,7 +88,7 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
             }  ;
             ;
         }  ;
-        scm2cpp::blas_gemv(resid, d24266, b024258, -1.0, 1.0, 0, n, p)  ;
+        scm2cpp::blas_gemv(resid, d3454, b03446, -1.0, 1.0, 0, n, p)  ;
         0  ;
         ;
         ;
@@ -106,6 +108,7 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
             double lam = lams[l];
             int stop = 0;
             for( int sweep = 0 ; !((sweep == iters || stop == 1)) ; sweep = (sweep+1) ) {
+                swept = (swept+1)  ;
                 double maxmove = 0.0;
                 for( int j = 0 ; !(j == p) ; j = (j+1) ) {
                     double rho = 0.0;
@@ -150,6 +153,6 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
             ;
         }  ;
     }  ;
-    return 0 ;
+    return swept ;
 }
 #endif // LASSO_DESIGN_HPP

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/src/scm2cpp_lasso/_generated/derived_loops/lasso_design.hpp

@@ -29,43 +29,45 @@ soft_threshold( double  z,  double  g )
 
 inline int
 lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::span<double> resid,  scm2cpp::span<double> xnorm,  scm2cpp::span<double> beta,  scm2cpp::cspan<double> lams,  scm2cpp::span<double> betas,  int  iters,  double  tol,  int  n,  int  p,  int  nlam )
-{   if( p < n ) {
+{   int swept = 0;
+    if( p < n ) {
         for( int i = 0 ; !(i == n) ; i = (i+1) ) {
             resid[ i ] = y[i]   ;
         }  ;
         0  ;
-        std::vector<double> b024258(p,0.0);
-        std::vector<double> g24257((p*p),0.0);
-        std::vector<double> c24256(p,0.0);
+        std::vector<double> b03446(p,0.0);
+        std::vector<double> g3445((p*p),0.0);
+        std::vector<double> c3444(p,0.0);
         for( int y2491 = 0 ; !(y2491 == p) ; y2491 = (y2491+1) ) {
             for( int y2492 = y2491 ; !(y2492 == p) ; y2492 = (y2492+1) ) {
-                g24257[ ((y2491*p)+y2492) ] = ([&]() -> double { int k2493 = 0; double sum2498 = 0.0; while (!(k2493 == n)) {
-                int k2493_next24266 = (k2493+1);
-                    double sum2498_next24267 = (sum2498+(x[((k2493*p)+y2491)]*x[((k2493*p)+y2492)]));
-                    k2493 = k2493_next24266;
-                    sum2498 = sum2498_next24267;
+                g3445[ ((y2491*p)+y2492) ] = ([&]() -> double { int k2493 = 0; double sum2498 = 0.0; while (!(k2493 == n)) {
+                int k2493_next3454 = (k2493+1);
+                    double sum2498_next3455 = (sum2498+(x[((k2493*p)+y2491)]*x[((k2493*p)+y2492)]));
+                    k2493 = k2493_next3454;
+                    sum2498 = sum2498_next3455;
                 } return sum2498; })()   ;
-                g24257[ ((y2492*p)+y2491) ] = g24257[((y2491*p)+y2492)]   ;
+                g3445[ ((y2492*p)+y2491) ] = g3445[((y2491*p)+y2492)]   ;
             }  ;
         }  ;
         for( int y2494 = 0 ; !(y2494 == p) ; y2494 = (y2494+1) ) {
-            c24256[ y2494 ] = ([&]() -> double { int k2495 = 0; double sum2500 = 0.0; while (!(k2495 == n)) {
-            int k2495_next24268 = (k2495+1);
-                double sum2500_next24269 = (sum2500+(x[((k2495*p)+y2494)]*resid[k2495]));
-                k2495 = k2495_next24268;
-                sum2500 = sum2500_next24269;
+            c3444[ y2494 ] = ([&]() -> double { int k2495 = 0; double sum2500 = 0.0; while (!(k2495 == n)) {
+            int k2495_next3456 = (k2495+1);
+                double sum2500_next3457 = (sum2500+(x[((k2495*p)+y2494)]*resid[k2495]));
+                k2495 = k2495_next3456;
+                sum2500 = sum2500_next3457;
             } return sum2500; })()   ;
         }  ;
         for( int l = 0 ; !(l == nlam) ; l = (l+1) ) {
             double lam = lams[l];
             int stop = 0;
             for( int sweep = 0 ; !((sweep == iters || stop == 1)) ; sweep = (sweep+1) ) {
+                swept = (swept+1)  ;
                 double maxmove = 0.0;
                 for( int j = 0 ; !(j == p) ; j = (j+1) ) {
-                    double rho = c24256[j];
+                    double rho = c3444[j];
                     double old = beta[j];
-                    rho = (rho+(old*g24257[((j*p)+j)]))  ;
-                    double bnew = (soft_threshold(rho,double((lam*(1.0*n))))/g24257[((j*p)+j)]);
+                    rho = (rho+(old*g3445[((j*p)+j)]))  ;
+                    double bnew = (soft_threshold(rho,double((lam*(1.0*n))))/g3445[((j*p)+j)]);
                     beta[ j ] = bnew   ;
                     if( !(bnew == old) ) {
                         if( (bnew-old) > maxmove ) {
@@ -78,9 +80,9 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
                         } else {
                             0  ;
                         }  ;
-                        b024258[ j ] = (b024258[j]+(bnew-old))   ;
-                        for( int k24260 = 0 ; !(k24260 == p) ; k24260 = (k24260+1) ) {
-                            c24256[ k24260 ] = (c24256[k24260]-(g24257[((j*p)+k24260)]*(bnew-old)))   ;
+                        b03446[ j ] = (b03446[j]+(bnew-old))   ;
+                        for( int k3448 = 0 ; !(k3448 == p) ; k3448 = (k3448+1) ) {
+                            c3444[ k3448 ] = (c3444[k3448]-(g3445[((j*p)+k3448)]*(bnew-old)))   ;
                         }  ;
                     } else {
                         false  ;
@@ -103,10 +105,10 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
         }  ;
         for( int y2496 = 0 ; !(y2496 == n) ; y2496 = (y2496+1) ) {
             resid[ y2496 ] = (resid[y2496]-([&]() -> double { int k2497 = 0; double sum2502 = 0.0; while (!(k2497 == p)) {
-            int k2497_next24270 = (k2497+1);
-                double sum2502_next24271 = (sum2502+(b024258[k2497]*x[((y2496*p)+k2497)]));
-                k2497 = k2497_next24270;
-                sum2502 = sum2502_next24271;
+            int k2497_next3458 = (k2497+1);
+                double sum2502_next3459 = (sum2502+(b03446[k2497]*x[((y2496*p)+k2497)]));
+                k2497 = k2497_next3458;
+                sum2502 = sum2502_next3459;
             } return sum2502; })())   ;
         }  ;
         0  ;
@@ -127,6 +129,7 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
             double lam = lams[l];
             int stop = 0;
             for( int sweep = 0 ; !((sweep == iters || stop == 1)) ; sweep = (sweep+1) ) {
+                swept = (swept+1)  ;
                 double maxmove = 0.0;
                 for( int j = 0 ; !(j == p) ; j = (j+1) ) {
                     double rho = 0.0;
@@ -171,6 +174,6 @@ lasso_design( scm2cpp::cspan<double> x,  scm2cpp::cspan<double> y,  scm2cpp::spa
             ;
         }  ;
     }  ;
-    return 0 ;
+    return swept ;
 }
 #endif // LASSO_DESIGN_HPP

{scm2cpp_lasso-0.14.0 → scm2cpp_lasso-0.15.1}/src/scm2cpp_lasso.egg-info/PKG-INFO

@@ -1,6 +1,6 @@
 Metadata-Version: 2.4
 Name: scm2cpp-lasso
-Version: 0.14.0
+Version: 0.15.1
 Summary: scikit-learn estimators (lasso, elastic net, ridge, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
 Author: Hirotaka Niitsuma
 License: MIT
@@ -107,7 +107,7 @@ them first; this solver does not do it for you.
 From 0.14.0 every estimator with a `fit(X, y)` (or `fit(X)`) derives
 from scikit-learn's `BaseEstimator` and the matching mixin, so `clone`,
 `get_params` / `set_params`, `GridSearchCV`, `Pipeline` and
-`cross_val_score` take them, and all 39 pass
+`cross_val_score` take them, and all 44 pass
 `sklearn.utils.estimator_checks.check_estimator` (`check-sklearn.py`
 runs it; the one expected failure is `LBFGSLogisticRegressionCV`'s
 `n_iter_` of 0 for a C whose warm start is already converged, which
@@ -122,11 +122,37 @@ from scm2cpp_lasso import GramLasso
 GridSearchCV(GramLasso(), {"alpha": [0.01, 0.1, 1.0]}, cv=5).fit(X, y)
 ```
 
-`GramLasso`, `GramElasticNet` and `GramRidge` are the `fit(X, y)`
-forms, with the intercept, over the Gram-matrix solvers; `CovLasso`,
-`CovRidge`, `DerivedLasso`, `CovLogistic`, `CovGroupLasso` and
-`CovMultiTaskLasso` keep their data-in-the-constructor interface,
-which solves a whole path at once.
+`CovLasso`, `CovRidge`, `DerivedLasso`, `CovLogistic`,
+`CovGroupLasso` and `CovMultiTaskLasso` keep their
+data-in-the-constructor interface, which solves a whole path at once;
+each has a `fit(X, y)` form with the intercept beside it (0.15.0):
+
+| data in the constructor | `fit(X, y)` | scikit-learn |
+|---|---|---|
+| `CovLasso` | `GramLasso`, `GramElasticNet` | Lasso, ElasticNet |
+| `CovRidge` | `GramRidge` | Ridge |
+| `DerivedLasso` | `DesignLasso` | Lasso |
+| `CovLogistic` | `GramLogisticRegression` | LogisticRegression(l1_ratio=1, solver="liblinear") |
+| `CovGroupLasso` | `GramGroupLasso` (`groups=None`: the lasso) | (none) |
+| `CovMultiTaskLasso` | `GramMultiTaskLasso`, `GramMultiTaskElasticNet` | MultiTaskLasso, MultiTaskElasticNet |
+
+Against scikit-learn on 400 x 30 with an intercept: `DesignLasso` and
+`GramGroupLasso(groups=None)` agree with `Lasso` to 1.5e-10, the
+multi-task pair with `MultiTaskLasso` / `MultiTaskElasticNet` to
+5e-14, and `GramLogisticRegression` reaches liblinear's objective to
+eight digits with the same predictions.
+
+Against the class each one wraps (one alpha, one core), which fits no
+intercept and is unchanged: with `fit_intercept=False` the input checks
+add 15 to 30 us a call (up to +50% on a 0.06 ms fit) and the finiteness
+sum 8 to 14% on 20000 x 200 (none under `assume_finite`); with the
+intercept, the column means add 20 to 30% on 20000 x 200 for the
+Gram-matrix ones, which correct X'X
+for the means rather than copy X to centre it, and about 55% for
+`DesignLasso`, whose kernel needs the centred X itself.
+`GramLogisticRegression` is slower than liblinear by its algorithm, 2.5x
+at 20000 x 200 and more on small problems; `L1LogisticRegression` is
+the fast one for that problem.
 
 What it costs is the input checks: X must be a finite 2-D float64
 array with the feature count of the fit.  A plain C-ordered float64
@@ -166,6 +192,11 @@ package picks the fastest one present.
 | `GramLasso` | Lasso(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
 | `GramElasticNet` | ElasticNet(precompute=True), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | no |
 | `GramRidge` | Ridge, fit(X, y) with the intercept, one or many targets | numpy (Cholesky) | CPU | Ridge |
+| `GramMultiTaskLasso` | MultiTaskLasso, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
+| `GramMultiTaskElasticNet` | MultiTaskElasticNet, fit(X, Y) with the intercept | `lasso-cov.scm` | CPU | no |
+| `GramGroupLasso` | (none; skglm GroupLasso), fit(X, y) with the intercept | `lasso-cov.scm` | CPU | - |
+| `GramLogisticRegression` | LogisticRegression(l1_ratio=1, solver='liblinear'), by majorization | `lasso-cov.scm` | CPU | no |
+| `DesignLasso` | Lasso, fit(X, y) with the intercept, from the design | `lasso-design.scm (--derive)` | loops / BLAS / cuBLAS | no |
 | `CovLasso` | Lasso, ElasticNet, lasso_path(precompute=True) | `lasso-cov.scm` | CPU; GPU batch (fit_path_batch, bootstrap) | no |
 | `CovLassoCV` | LassoCV, ElasticNetCV | `lasso-cov.scm` | CPU; GPU batch | no |
 | `DerivedLasso` | lasso_path, from the design | `lasso-design.scm (--derive)` | loops / BLAS / cuBLAS | no |
@@ -612,6 +643,15 @@ matrix (`KernelSVM`) or transposing X into columns
 | `L1LogisticRegression`, 20,000 x 500, C=0.05 | 1.20 s | 2.23 s | 1.9x | 106.77913 / 106.77915 | 100% |
 | `L1LogisticRegression`, 100,000 x 200, C=0.05 | 9.06 s | 18.7 s | 2.1x | 349.61154 / 349.61216 | 100% |
 
+On 2,000 x 20,000 neither side reaches the optimum at tol 1e-4 (the
+objective at tol 1e-10 is 0.0519588), and where each stops depends on
+the coordinate order: over eight seeds ours stops at 0.05240 to
+0.05251 (0.9 to 1.1% above the optimum), `LinearSVC`, whose order is
+random unless `random_state` is given, at 0.05242 to 0.05319 (0.9 to
+2.4%). When `LinearSVC` draws one of its worse orders the gap passes
+the script's 0.5% bound and the row prints `<-- disagrees`; ours is
+then the better solution of the two, so the flag there is not a fault.
+
 The regressors, from the same script (C = 1, epsilon 0.1):
 
 | problem | ours | scikit-learn | ratio | agreement |