scm2cpp-learn 0.18.0__tar.gz → 0.19.0__tar.gz

{scm2cpp_learn-0.18.0/src/scm2cpp_learn.egg-info → scm2cpp_learn-0.19.0}/PKG-INFO

@@ -1,7 +1,7 @@
 Metadata-Version: 2.4
 Name: scm2cpp-learn
-Version: 0.18.0
-Summary: scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
+Version: 0.19.0
+Summary: scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, PLS, CCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
 Author: Hirotaka Niitsuma
 License: MIT
 Project-URL: Homepage, https://github.com/niitsuma/scm2cpp
@@ -257,6 +257,7 @@ package picks the fastest one present.
 | `GramKernelPCA` | KernelPCA (no inverse_transform) | `kernel-pca.scm` | loops / BLAS | no |
 | `GramLDA` | LinearDiscriminantAnalysis (svd, lsqr, eigen) | `lda.scm` | loops / BLAS | no |
 | `GramTruncatedSVD` | TruncatedSVD on dense X (exact, as arpack) | `truncated-svd.scm` | loops / BLAS | no |
+| `GramPLSRegression`, `GramPLSCanonical`, `GramCCA`, `GramPLSSVD` | PLSRegression, PLSCanonical, CCA, PLSSVD | `pls.scm` | loops / BLAS | no |
 | `f_regression, f_classif` | f_regression, f_classif | `fscore.scm` | loops / BLAS / cuBLAS | no |
 | `SelectKBest, SelectPercentile` | SelectKBest, SelectPercentile | `fscore.scm` | loops / BLAS / cuBLAS | no |
 | `GramRFE` | RFE(Ridge), RFE(LinearRegression) | `rfe.scm` | loops / BLAS / cuBLAS | inside: Ridge |
@@ -714,6 +715,48 @@ The exact route costs O(n p^2 + p^3); with few components of many
 features (2,000 here) the randomized one does less work, and on one core
 it is faster, at its accuracy. Sparse X is not taken yet.
 
+## PLS and CCA: `GramPLSRegression`, `GramPLSCanonical`, `GramCCA`, `GramPLSSVD`
+
+scikit-learn's `cross_decomposition` estimators over the cross products
+(`examples/kernel-only/pls.scm`). NIPALS multiplies the blocks only by
+weight vectors, and each such product is one of Cxx = X'X, Cxy = X'Y,
+Cyy = Y'Y times a vector: the Y score is Y c for a coefficient vector c,
+so X'(Y c) = Cxy c. The deflations are rank-one updates of the three
+matrices (with a = Cxx w, b = Cxy'w and s2 = w'Cxx w, regression mode is
+Cxx -= a a'/s2, Cxy -= a b'/s2, Cyy -= b b'/s2). So X and Y are read
+once, for the three products (dsyrk and dgemm), and every power
+iteration after that costs O(p^2 + pq + q^2) where scikit-learn's costs
+O(n(p + q)). CCA's mode B takes pinv(X) y = Cxx^+ X'y, Cxx^+ from its
+eigenproblem; PLSSVD is the SVD of Cxy.
+
+```python
+from scm2cpp_learn import GramPLSRegression, GramCCA
+pls = GramPLSRegression(n_components=5).fit(X, Y)
+pls.predict(X), pls.transform(X), pls.x_scores_
+```
+
+Weights, loadings, rotations, coefficients, scores and transforms agree
+with scikit-learn's to 1e-13, with the same iteration counts. One cut
+differs: a deflated block loses a rank exactly, which its SVD shows as a
+singular value near eps but its Gram as an eigenvalue near eps
+lam_max, so mode B's pinv also drops eigenvalues under 1e3 eps lam_max
+(singular values under about 5e-7 of the largest, where scikit-learn's
+cut is 2.2e-10). Measured (`bench/eigen-compare.py`, the MKL build, best
+of three):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| PLSRegression, 200,000 x 100 -> 5, 5 components | 5.64 s | 0.26 s | 21x | 19x |
+| PLSCanonical, 200,000 x 100 -> 5, 5 components | 5.63 s | 0.14 s | 40x | 59x |
+| CCA, 200,000 x 100 -> 5, 5 components | 31.9 s | 0.14 s | 220x | 190x |
+| PLSSVD, 200,000 x 100 -> 5, 5 components | 0.28 s | 0.14 s | 2.0x | 4.0x |
+| PLSRegression, 50,000 x 500 -> 10, 10 components | 45.5 s | 0.75 s | 61x | 57x |
+| PLSCanonical, 50,000 x 500 -> 10, 10 components | 45.3 s | 0.60 s | 76x | 100x |
+| CCA, 50,000 x 500 -> 10, 10 components | 132 s | 1.21 s | 109x | 151x |
+| PLSSVD, 50,000 x 500 -> 10, 10 components | 0.34 s | 0.34 s | 1.0x | 3.4x |
+
+PLSSVD is one SVD of X'Y on both sides, so only the product differs.
+
 ## Bayesian ridge
 
 `CovBayesianRidge` is scikit-learn's `BayesianRidge` -- the same model,

scm2cpp_learn-0.18.0/PKG-INFO → scm2cpp_learn-0.19.0/README.md

@@ -1,21 +1,3 @@
-Metadata-Version: 2.4
-Name: scm2cpp-learn
-Version: 0.18.0
-Summary: scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included
-Author: Hirotaka Niitsuma
-License: MIT
-Project-URL: Homepage, https://github.com/niitsuma/scm2cpp
-Project-URL: Source, https://github.com/niitsuma/scm2cpp/tree/master/python/scm2cpp-learn
-Keywords: scikit-learn,lasso,elastic-net,ridge,svm,logistic-regression,pca,lda,coordinate-descent,lapack,cuda,compiler
-Classifier: License :: OSI Approved :: MIT License
-Classifier: Programming Language :: Python :: 3
-Classifier: Programming Language :: C++
-Classifier: Topic :: Scientific/Engineering :: Mathematics
-Classifier: Intended Audience :: Science/Research
-Requires-Python: >=3.8
-Description-Content-Type: text/markdown
-Requires-Dist: numpy>=1.17
-
 # scm2cpp-learn
 
 [![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/niitsuma/scm2cpp/blob/master/notebooks/compare-sklearn.ipynb)
@@ -257,6 +239,7 @@ package picks the fastest one present.
 | `GramKernelPCA` | KernelPCA (no inverse_transform) | `kernel-pca.scm` | loops / BLAS | no |
 | `GramLDA` | LinearDiscriminantAnalysis (svd, lsqr, eigen) | `lda.scm` | loops / BLAS | no |
 | `GramTruncatedSVD` | TruncatedSVD on dense X (exact, as arpack) | `truncated-svd.scm` | loops / BLAS | no |
+| `GramPLSRegression`, `GramPLSCanonical`, `GramCCA`, `GramPLSSVD` | PLSRegression, PLSCanonical, CCA, PLSSVD | `pls.scm` | loops / BLAS | no |
 | `f_regression, f_classif` | f_regression, f_classif | `fscore.scm` | loops / BLAS / cuBLAS | no |
 | `SelectKBest, SelectPercentile` | SelectKBest, SelectPercentile | `fscore.scm` | loops / BLAS / cuBLAS | no |
 | `GramRFE` | RFE(Ridge), RFE(LinearRegression) | `rfe.scm` | loops / BLAS / cuBLAS | inside: Ridge |
@@ -714,6 +697,48 @@ The exact route costs O(n p^2 + p^3); with few components of many
 features (2,000 here) the randomized one does less work, and on one core
 it is faster, at its accuracy. Sparse X is not taken yet.
 
+## PLS and CCA: `GramPLSRegression`, `GramPLSCanonical`, `GramCCA`, `GramPLSSVD`
+
+scikit-learn's `cross_decomposition` estimators over the cross products
+(`examples/kernel-only/pls.scm`). NIPALS multiplies the blocks only by
+weight vectors, and each such product is one of Cxx = X'X, Cxy = X'Y,
+Cyy = Y'Y times a vector: the Y score is Y c for a coefficient vector c,
+so X'(Y c) = Cxy c. The deflations are rank-one updates of the three
+matrices (with a = Cxx w, b = Cxy'w and s2 = w'Cxx w, regression mode is
+Cxx -= a a'/s2, Cxy -= a b'/s2, Cyy -= b b'/s2). So X and Y are read
+once, for the three products (dsyrk and dgemm), and every power
+iteration after that costs O(p^2 + pq + q^2) where scikit-learn's costs
+O(n(p + q)). CCA's mode B takes pinv(X) y = Cxx^+ X'y, Cxx^+ from its
+eigenproblem; PLSSVD is the SVD of Cxy.
+
+```python
+from scm2cpp_learn import GramPLSRegression, GramCCA
+pls = GramPLSRegression(n_components=5).fit(X, Y)
+pls.predict(X), pls.transform(X), pls.x_scores_
+```
+
+Weights, loadings, rotations, coefficients, scores and transforms agree
+with scikit-learn's to 1e-13, with the same iteration counts. One cut
+differs: a deflated block loses a rank exactly, which its SVD shows as a
+singular value near eps but its Gram as an eigenvalue near eps
+lam_max, so mode B's pinv also drops eigenvalues under 1e3 eps lam_max
+(singular values under about 5e-7 of the largest, where scikit-learn's
+cut is 2.2e-10). Measured (`bench/eigen-compare.py`, the MKL build, best
+of three):
+
+| case | scikit-learn | package | ratio, one core | ratio, 20 cores |
+|---|---|---|---|---|
+| PLSRegression, 200,000 x 100 -> 5, 5 components | 5.64 s | 0.26 s | 21x | 19x |
+| PLSCanonical, 200,000 x 100 -> 5, 5 components | 5.63 s | 0.14 s | 40x | 59x |
+| CCA, 200,000 x 100 -> 5, 5 components | 31.9 s | 0.14 s | 220x | 190x |
+| PLSSVD, 200,000 x 100 -> 5, 5 components | 0.28 s | 0.14 s | 2.0x | 4.0x |
+| PLSRegression, 50,000 x 500 -> 10, 10 components | 45.5 s | 0.75 s | 61x | 57x |
+| PLSCanonical, 50,000 x 500 -> 10, 10 components | 45.3 s | 0.60 s | 76x | 100x |
+| CCA, 50,000 x 500 -> 10, 10 components | 132 s | 1.21 s | 109x | 151x |
+| PLSSVD, 50,000 x 500 -> 10, 10 components | 0.34 s | 0.34 s | 1.0x | 3.4x |
+
+PLSSVD is one SVD of X'Y on both sides, so only the product differs.
+
 ## Bayesian ridge
 
 `CovBayesianRidge` is scikit-learn's `BayesianRidge` -- the same model,

{scm2cpp_learn-0.18.0 → scm2cpp_learn-0.19.0}/pyproject.toml

@@ -4,8 +4,8 @@ build-backend = "setuptools.build_meta"
 
 [project]
 name = "scm2cpp-learn"
-version = "0.18.0"
-description = "scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included"
+version = "0.19.0"
+description = "scikit-learn estimators (lasso, elastic net, ridge, RidgeCV, ARD, SVM, SVR, logistic regression, GLMs, Huber, Theil-Sen, F-score selection, RFE, NMF, Gaussian mixture, FastICA, PCA, KernelPCA, LDA, TruncatedSVD, PLS, CCA, graphical lasso, RANSAC, SGD, OMP, LARS) translated from Scheme to C++ by scm2cpp; GPU batch solver and CUDA kernels included"
 readme = "README.md"
 requires-python = ">=3.8"
 license = {text = "MIT"}

{scm2cpp_learn-0.18.0 → scm2cpp_learn-0.19.0}/src/scm2cpp_learn/__init__.py

@@ -50,14 +50,14 @@ __all__ = ["GramLasso", "GramElasticNet", "GramRidge", "GramRidgeCV", "CovLasso"
            "LBFGSLogisticRegression", "LBFGSHuberRegressor", "LBFGSLogisticRegressionCV",
            "LBFGSPoissonRegressor", "LBFGSGammaRegressor", "LBFGSTweedieRegressor",
            "GramLarsCV", "GramLassoLarsCV", "GramOMPCV",
-           "EMGaussianMixture", "CovPCA", "GramKernelPCA", "GramLDA", "GramTruncatedSVD", "ParallelFastICA", "CovARDRegression",
+           "EMGaussianMixture", "CovPCA", "GramKernelPCA", "GramLDA", "GramTruncatedSVD", "GramPLSRegression", "GramPLSCanonical", "GramCCA", "GramPLSSVD", "ParallelFastICA", "CovARDRegression",
            "L1LinearSVM", "WeiszfeldTheilSen", "f_regression", "f_classif", "SelectKBest",
            "SelectPercentile", "GramRFE",
            "CovMultiTaskLasso", "CovMultiTaskLassoCV", "DerivedLasso",
            "GramLogisticRegression", "GramGroupLasso", "GramMultiTaskLasso",
            "GramMultiTaskElasticNet", "DesignLasso",
            "cuda_available", "derived_backend", "kernel"]
-__version__ = "0.18.0"
+__version__ = "0.19.0"
 
 _BATCH = load_batch_lib()
 _DP = ctypes.POINTER(ctypes.c_double)
@@ -837,7 +837,10 @@ class GramRidgeCV(Tagged, RegressorMixin, BaseEstimator):
     and diag H and yhat at every alpha are two products.  Z comes from
     the eigenvectors of Xc'Xc (``gcv_mode="svd"``, the default when
     n > p, X'X by dsyrk) or of Xc Xc' (``"eigen"``, the default
-    otherwise); the eigenproblem is LAPACK's dsyevd where the BLAS
+    otherwise; as scikit-learn's, it takes e = c / diag G^-1 for
+    c = G^-1 y, G = Xc Xc' + 11' + alpha I with the intercept's
+    eigenvalue unpenalised, so that a small alpha, where H tends to I,
+    loses no digits to 1 - diag H); the eigenproblem is LAPACK's dsyevd where the BLAS
     build is here.  Only ``cv=None`` and ``scoring=None`` are taken.
 
     Attributes after fit: alpha_, best_score_ (minus the mean squared
@@ -890,14 +893,19 @@ class GramRidgeCV(Tagged, RegressorMixin, BaseEstimator):
         else:
             Xc = np.ascontiguousarray(X - xm)
             K = _gram(np.ascontiguousarray(Xc.T))
-            m = n
-            V, lam, Z, R = np.zeros((n, n)), np.zeros(n), np.zeros((n, n)), np.zeros((n, t))
-            k.ridge_gcv_dual(K, V, lam, yc, Z, R, n, t)
+            if self.fit_intercept:
+                K += 1.0
+            V, lam, R = np.zeros((n, n)), np.zeros(n), np.zeros((n, t))
+            drop = k.ridge_gcv_dual(K, V, lam, yc, R, n, t, int(bool(self.fit_intercept)))
         na = alphas.size
         mse = np.zeros((na, t))
         loo = np.zeros((n, t, na)) if self.store_cv_results else np.zeros(1)
-        k.ridge_gcv_scan(Z, lam, R, yc, alphas, 1.0 / n if self.fit_intercept else 0.0,
-                         mse, loo, int(bool(self.store_cv_results)), n, m, t, na)
+        store = int(bool(self.store_cv_results))
+        if mode == "svd":
+            k.ridge_gcv_scan(Z, lam, R, yc, alphas, 1.0 / n if self.fit_intercept else 0.0,
+                             mse, loo, store, n, m, t, na)
+        else:
+            k.ridge_gcv_scan_dual(V, lam, R, alphas, drop, mse, loo, store, n, t, na)
         if self.alpha_per_target and t > 1:
             best = np.argmin(mse, axis=0)
             al = alphas[best]
@@ -911,9 +919,12 @@ class GramRidgeCV(Tagged, RegressorMixin, BaseEstimator):
         if mode == "svd":
             coef = V @ (R / (lam[:, None] + al[None, :]))
         else:
-            # (Xc Xc' + alpha I)^-1 yc by the eigenvectors, then Xc' of it
-            qy = V.T @ yc
-            coef = Xc.T @ (V @ (qy / (lam[:, None] + al[None, :])))
+            # (Xc Xc' + alpha I)^-1 yc by the eigenvectors, then Xc' of
+            # it; yc has no part along 1, so the 11' added changes nothing
+            w = 1.0 / (lam[:, None] + al[None, :])
+            if drop >= 0:
+                w[drop] = 0.0
+            coef = Xc.T @ (V @ (R * w))
         if y.ndim == 1:
             self.coef_ = coef[:, 0]
             self.intercept_ = float(ym[0] - xm @ coef[:, 0])
@@ -2259,6 +2270,17 @@ class GramKernelPCA(Tagged, TransformerMixin, BaseEstimator):
         return K @ scaled
 
 
+def _lda_svd_dof(n, K):
+    # the divisor of LinearDiscriminantAnalysis's svd solver: n from
+    # scikit-learn 1.9.1, n - K before; n without scikit-learn
+    try:
+        import sklearn
+        v = tuple(int(x) for x in sklearn.__version__.split(".")[:3] if x.isdigit())
+    except Exception:
+        return n
+    return n - K if v < (1, 9, 1) else n
+
+
 class GramLDA(Tagged, ClassifierMixin, TransformerMixin, BaseEstimator):
     """scikit-learn's ``LinearDiscriminantAnalysis`` over the Gram matrix
     (``examples/kernel-only/lda.scm``): X'X by dsyrk and the class means
@@ -2353,6 +2375,10 @@ class GramLDA(Tagged, ClassifierMixin, TransformerMixin, BaseEstimator):
             self.xbar_ = priors @ means
             std, V, S = np.zeros(p), np.zeros((p, p)), np.zeros(p)
             k.lda_svd_first(np.ascontiguousarray(Sw_emp), std, V, S, p)
+            # S is the SVD of Xc D^-1 / sqrt(n); scikit-learn before 1.9.1
+            # divides by sqrt(n - K) instead (#33626), and its tol and
+            # scalings_ see that scale
+            S *= np.sqrt(n / _lda_svd_dof(n, K))
             rank = int(np.sum(S > self.tol))
             scal = (V[:, :rank] / std[:, None]) / S[:rank]
             fac = 1.0 if K == 1 else 1.0 / (K - 1)
@@ -2481,6 +2507,226 @@ class GramTruncatedSVD(Tagged, TransformerMixin, BaseEstimator):
         return np.asarray(X) @ self.components_
 
 
+def _pls_moments(X, Y, scale):
+    """Centred (and, with scale, standardised) Cxx, Cxy, Cyy, with the
+    means and deviations: from X'X, X'Y and Y'Y less the n m m' terms,
+    the centred copies only when a column's mean dwarfs its spread (the
+    correction would cancel its digits)."""
+    n = X.shape[0]
+    xm, ym = X.mean(axis=0), Y.mean(axis=0)
+    Gxx, Gxy, Gyy = _gram(X), _gram_xy(X, Y), _gram(np.ascontiguousarray(Y))
+    Gxy = Gxy.reshape(X.shape[1], Y.shape[1])
+    nx, ny = n * xm * xm, n * ym * ym
+    if np.all(np.diagonal(Gxx) - nx > 1e-4 * nx) and np.all(np.diagonal(Gyy) - ny > 1e-4 * ny):
+        Cxx = Gxx - np.outer(xm, n * xm)
+        Cxy = Gxy - np.outer(xm, n * ym)
+        Cyy = Gyy - np.outer(ym, n * ym)
+    else:
+        Xc, Yc = np.ascontiguousarray(X - xm), np.ascontiguousarray(Y - ym)
+        Cxx, Cyy = _gram(Xc), _gram(Yc)
+        Cxy = _gram_xy(Xc, Yc).reshape(X.shape[1], Y.shape[1])
+    if scale:
+        xs = np.sqrt(np.maximum(np.diagonal(Cxx), 0.0) / (n - 1))
+        ys = np.sqrt(np.maximum(np.diagonal(Cyy), 0.0) / (n - 1))
+        xs[xs == 0.0] = 1.0
+        ys[ys == 0.0] = 1.0
+        Cxx = Cxx / np.outer(xs, xs)
+        Cxy = Cxy / np.outer(xs, ys)
+        Cyy = Cyy / np.outer(ys, ys)
+    else:
+        xs, ys = np.ones(X.shape[1]), np.ones(Y.shape[1])
+    return (np.ascontiguousarray(Cxx), np.ascontiguousarray(Cxy), np.ascontiguousarray(Cyy),
+            xm, ym, xs, ys)
+
+
+class _GramPLS(Tagged, TransformerMixin, BaseEstimator):
+    # scikit-learn's _PLS over the cross products (examples/kernel-only/pls.scm)
+    _multi_output = True
+    _mode, _canon = 0, 0
+
+    def fit(self, X, y):
+        X, y = fit_xy(self, X, y, multi_output=True)
+        n, p = X.shape
+        if n < 2:
+            raise ValueError(f"Found array with {n} sample(s) (shape={X.shape}) while a "
+                             "minimum of 2 is required.")
+        self._predict_1d = y.ndim == 1
+        Y = y.reshape(n, -1)
+        q = Y.shape[1]
+        k = int(self.n_components)
+        upper = min(n, p) if not self._canon else min(n, p, q)
+        if not 1 <= k <= upper:
+            raise ValueError(f"`n_components` upper bound is {upper}. Got {k} instead. "
+                             "Reduce `n_components`.")
+        algo = 1 if getattr(self, "algorithm", "nipals") == "svd" else 0
+        Cxx, Cxy, Cyy, self._x_mean, self._y_mean, self._x_std, self._y_std = \
+            _pls_moments(X, Y, self.scale)
+        xw, yw = np.zeros((p, k)), np.zeros((q, k))
+        xl, yl = np.zeros((p, k)), np.zeros((q, k))
+        niter = np.zeros(k, dtype=np.int32)
+        found = _gramops_kernels().pls_fit(Cxx, Cxy, Cyy, xw, yw, xl, yl, niter, p, q, k,
+                                           self._mode, self._canon, algo,
+                                           int(self.max_iter), float(self.tol))
+        if found < k:
+            warnings.warn(f"y residual is constant at iteration {found}")
+        # scikit-learn keeps the list empty for the svd algorithm
+        self.n_iter_ = [int(t) for t in niter[:found]] if algo == 0 else []
+        if algo == 0:
+            if any(t == int(self.max_iter) for t in self.n_iter_):
+                warnings.warn("Maximum number of iterations reached", UserWarning)
+        self.x_weights_, self.y_weights_ = xw, yw
+        self.x_loadings_, self.y_loadings_ = xl, yl
+        self.x_rotations_ = xw @ np.linalg.pinv(xl.T @ xw)
+        self.y_rotations_ = yw @ np.linalg.pinv(yl.T @ yw)
+        coef = self.x_rotations_ @ yl.T
+        self.coef_ = (coef * self._y_std).T / self._x_std
+        self.intercept_ = self._y_mean
+        self._n_features_out = k
+        self._X, self._Y = X, Y
+        return self
+
+    def transform(self, X, y=None, copy=True):
+        check_is_fitted(self)
+        Xs = (pred_x(self, X) - self._x_mean) / self._x_std
+        xs = Xs @ self.x_rotations_
+        if y is None:
+            return xs
+        Y = np.asarray(y, dtype=np.float64)
+        Y = Y.reshape(-1, 1) if Y.ndim == 1 else Y
+        return xs, ((Y - self._y_mean) / self._y_std) @ self.y_rotations_
+
+    def fit_transform(self, X, y=None):
+        return self.fit(X, y).transform(X, y)
+
+    def inverse_transform(self, X, y=None):
+        check_is_fitted(self)
+        Xr = np.asarray(X, dtype=np.float64) @ self.x_loadings_.T * self._x_std + self._x_mean
+        if y is None:
+            return Xr
+        Yr = np.asarray(y, dtype=np.float64) @ self.y_loadings_.T * self._y_std + self._y_mean
+        return Xr, Yr
+
+    def predict(self, X, copy=True):
+        check_is_fitted(self)
+        yp = (pred_x(self, X) - self._x_mean) @ self.coef_.T + self.intercept_
+        return yp.ravel() if self._predict_1d else yp
+
+
+class GramPLSRegression(RegressorMixin, _GramPLS):
+    """scikit-learn's ``PLSRegression`` over the cross products
+    (``examples/kernel-only/pls.scm``): X'X (dsyrk), X'Y and Y'Y once,
+    then NIPALS and the regression deflation as products and rank-one
+    updates of those p x p, p x q and q x q matrices, never the n rows
+    again.  The scores x_scores_ = X R and y_scores_ (Y less the earlier
+    components' X scores, times the Y weights) are formed at the end.
+
+    Attributes after fit: x_weights_, y_weights_, x_loadings_,
+    y_loadings_, x_rotations_, y_rotations_, x_scores_, y_scores_,
+    coef_, intercept_, n_iter_, n_features_in_.
+    """
+    _mode, _canon = 0, 0
+
+    def __init__(self, n_components=2, *, scale=True, max_iter=500, tol=1e-06, copy=True):
+        self.n_components, self.scale, self.max_iter = n_components, scale, max_iter
+        self.tol, self.copy = tol, copy
+
+    def fit(self, X, y):
+        super().fit(X, y)
+        Xs = (self._X - self._x_mean) / self._x_std
+        Ys = (self._Y - self._y_mean) / self._y_std
+        T = Xs @ self.x_rotations_
+        U = Ys @ self.y_weights_
+        for k in range(U.shape[1]):
+            # Y deflated by the earlier components' X scores (regression mode)
+            U[:, k] -= T[:, :k] @ (self.y_loadings_[:, :k].T @ self.y_weights_[:, k])
+            U[:, k] /= self.y_weights_[:, k] @ self.y_weights_[:, k]
+        self.x_scores_, self.y_scores_ = T, U
+        del self._X, self._Y
+        return self
+
+    def fit_transform(self, X, y=None):
+        return self.fit(X, y).transform(X, y)
+
+
+class GramPLSCanonical(_GramPLS):
+    """scikit-learn's ``PLSCanonical`` (canonical deflation, mode A,
+    ``algorithm`` "nipals" or "svd") over the cross products, as
+    ``GramPLSRegression``."""
+    _mode, _canon = 0, 1
+
+    def __init__(self, n_components=2, *, scale=True, algorithm="nipals", max_iter=500,
+                 tol=1e-06, copy=True):
+        self.n_components, self.scale, self.algorithm = n_components, scale, algorithm
+        self.max_iter, self.tol, self.copy = max_iter, tol, copy
+
+    def fit(self, X, y):
+        if self.algorithm not in ("nipals", "svd"):
+            raise ValueError(f"algorithm={self.algorithm!r}: expected 'nipals' or 'svd'")
+        super().fit(X, y)
+        del self._X, self._Y
+        return self
+
+
+class GramCCA(_GramPLS):
+    """scikit-learn's ``CCA`` (canonical deflation, mode B: pinv of each
+    block from the eigenproblem of its Gram, cut where _pinv2_old cuts)
+    over the cross products, as ``GramPLSRegression``."""
+    _mode, _canon = 1, 1
+
+    def __init__(self, n_components=2, *, scale=True, max_iter=500, tol=1e-06, copy=True):
+        self.n_components, self.scale, self.max_iter = n_components, scale, max_iter
+        self.tol, self.copy = tol, copy
+
+    def fit(self, X, y):
+        super().fit(X, y)
+        del self._X, self._Y
+        return self
+
+
+class GramPLSSVD(Tagged, TransformerMixin, BaseEstimator):
+    """scikit-learn's ``PLSSVD``: the SVD of the centred (scaled) X'Y,
+    formed from X'Y less n mx my' instead of from centred copies."""
+    _multi_output = True
+
+    def __init__(self, n_components=2, *, scale=True, copy=True):
+        self.n_components, self.scale, self.copy = n_components, scale, copy
+
+    def fit(self, X, y):
+        X, y = fit_xy(self, X, y, multi_output=True)
+        n, p = X.shape
+        if n < 2:
+            raise ValueError(f"Found array with {n} sample(s) (shape={X.shape}) while a "
+                             "minimum of 2 is required.")
+        Y = y.reshape(n, -1)
+        k = int(self.n_components)
+        upper = min(n, p, Y.shape[1])
+        if not 1 <= k <= upper:
+            raise ValueError(f"`n_components` upper bound is {upper}. Got {k} instead. "
+                             "Reduce `n_components`.")
+        _, C, _, self._x_mean, self._y_mean, self._x_std, self._y_std = \
+            _pls_moments(X, Y, self.scale)
+        U, _, Vt = np.linalg.svd(C, full_matrices=False)
+        U, Vt = U[:, :k], Vt[:k]
+        # svd_flip, u-based: each column of U's largest entry positive
+        sg = np.sign(U[np.argmax(np.abs(U), axis=0), np.arange(k)])
+        sg[sg == 0] = 1.0
+        self.x_weights_, self.y_weights_ = U * sg, (Vt * sg[:, None]).T
+        self._n_features_out = k
+        return self
+
+    def transform(self, X, y=None):
+        check_is_fitted(self)
+        xs = ((pred_x(self, X) - self._x_mean) / self._x_std) @ self.x_weights_
+        if y is None:
+            return xs
+        Y = np.asarray(y, dtype=np.float64)
+        Y = Y.reshape(-1, 1) if Y.ndim == 1 else Y
+        return xs, ((Y - self._y_mean) / self._y_std) @ self.y_weights_
+
+    def fit_transform(self, X, y=None):
+        return self.fit(X, y).transform(X, y)
+
+
 class ParallelFastICA(Tagged, TransformerMixin, BaseEstimator):
     """FastICA, scikit-learn's ``FastICA`` at its defaults translated
     whole (``examples/kernel-only/fastica.scm``: algorithm="parallel",

{scm2cpp_learn-0.18.0 → scm2cpp_learn-0.19.0}/src/scm2cpp_learn/_generated/_loader_gramops_blas.py

@@ -522,15 +522,25 @@ def ridge_gcv_primal(x, xm, yc, g, v, lam, z, r, n, p, t):
     return _lib.scm2cpp_ridge_gcv_primal(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), xm.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), yc.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), g.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), v.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), lam.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), z.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), r.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), n, p, t)
 
 _lib.scm2cpp_ridge_gcv_dual.restype = ctypes.c_int
-_lib.scm2cpp_ridge_gcv_dual.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int]
-def ridge_gcv_dual(k, q, mu, yc, z, r, n, t):
+_lib.scm2cpp_ridge_gcv_dual.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int, ctypes.c_int]
+def ridge_gcv_dual(k, q, mu, yc, r, n, t, icpt):
     k = np.ascontiguousarray(k, dtype=np.float64)
     q = np.ascontiguousarray(q, dtype=np.float64)
     mu = np.ascontiguousarray(mu, dtype=np.float64)
     yc = np.ascontiguousarray(yc, dtype=np.float64)
-    z = np.ascontiguousarray(z, dtype=np.float64)
     r = np.ascontiguousarray(r, dtype=np.float64)
-    return _lib.scm2cpp_ridge_gcv_dual(k.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), q.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), mu.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), yc.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), z.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), r.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), n, t)
+    return _lib.scm2cpp_ridge_gcv_dual(k.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), q.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), mu.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), yc.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), r.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), n, t, icpt)
+
+_lib.scm2cpp_ridge_gcv_scan_dual.restype = ctypes.c_int
+_lib.scm2cpp_ridge_gcv_scan_dual.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int]
+def ridge_gcv_scan_dual(q, mu, r, alphas, drop, mse, loo, store, n, t, na):
+    q = np.ascontiguousarray(q, dtype=np.float64)
+    mu = np.ascontiguousarray(mu, dtype=np.float64)
+    r = np.ascontiguousarray(r, dtype=np.float64)
+    alphas = np.ascontiguousarray(alphas, dtype=np.float64)
+    mse = np.ascontiguousarray(mse, dtype=np.float64)
+    loo = np.ascontiguousarray(loo, dtype=np.float64)
+    return _lib.scm2cpp_ridge_gcv_scan_dual(q.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), mu.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), r.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), alphas.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), drop, mse.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), loo.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), store, n, t, na)
 
 _lib.scm2cpp_ridge_gcv_scan.restype = ctypes.c_int
 _lib.scm2cpp_ridge_gcv_scan.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_double, ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int]
@@ -690,3 +700,73 @@ def tsvd_wide(x, v, s, z, n, p, k):
     z = np.ascontiguousarray(z, dtype=np.float64)
     return _lib.scm2cpp_tsvd_wide(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), v.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), s.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), z.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), n, p, k)
 
+_lib.scm2cpp_svd_jacobi.restype = ctypes.c_int
+_lib.scm2cpp_svd_jacobi.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int]
+def svd_jacobi(w, q, r, c):
+    w = np.ascontiguousarray(w, dtype=np.float64)
+    q = np.ascontiguousarray(q, dtype=np.float64)
+    return _lib.scm2cpp_svd_jacobi(w.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), q.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), r, c)
+
+_lib.scm2cpp_svd_collect.restype = ctypes.c_int
+_lib.scm2cpp_svd_collect.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_int), ctypes.c_int, ctypes.c_int]
+def svd_collect(w, q, s, l, rr, ord, r, c):
+    w = np.ascontiguousarray(w, dtype=np.float64)
+    q = np.ascontiguousarray(q, dtype=np.float64)
+    s = np.ascontiguousarray(s, dtype=np.float64)
+    l = np.ascontiguousarray(l, dtype=np.float64)
+    rr = np.ascontiguousarray(rr, dtype=np.float64)
+    ord = np.ascontiguousarray(ord, dtype=np.int32)
+    return _lib.scm2cpp_svd_collect(w.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), q.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), s.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), l.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), rr.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), ord.ctypes.data_as(ctypes.POINTER(ctypes.c_int)), r, c)
+
+_lib.scm2cpp_sym_svd.restype = ctypes.c_int
+_lib.scm2cpp_sym_svd.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int]
+def sym_svd(a, u, s, vt, m, n):
+    a = np.ascontiguousarray(a, dtype=np.float64)
+    u = np.ascontiguousarray(u, dtype=np.float64)
+    s = np.ascontiguousarray(s, dtype=np.float64)
+    vt = np.ascontiguousarray(vt, dtype=np.float64)
+    return _lib.scm2cpp_sym_svd(a.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), u.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), s.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), vt.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), m, n)
+
+_lib.scm2cpp_pls_pinv.restype = ctypes.c_int
+_lib.scm2cpp_pls_pinv.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int]
+def pls_pinv(a, pv, m):
+    a = np.ascontiguousarray(a, dtype=np.float64)
+    pv = np.ascontiguousarray(pv, dtype=np.float64)
+    return _lib.scm2cpp_pls_pinv(a.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), pv.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), m)
+
+_lib.scm2cpp_pls_mv.restype = ctypes.c_int
+_lib.scm2cpp_pls_mv.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int]
+def pls_mv(a, x, y, m, n):
+    a = np.ascontiguousarray(a, dtype=np.float64)
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    y = np.ascontiguousarray(y, dtype=np.float64)
+    return _lib.scm2cpp_pls_mv(a.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), y.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), m, n)
+
+_lib.scm2cpp_pls_mtv.restype = ctypes.c_int
+_lib.scm2cpp_pls_mtv.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int, ctypes.c_int]
+def pls_mtv(a, x, y, m, n):
+    a = np.ascontiguousarray(a, dtype=np.float64)
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    y = np.ascontiguousarray(y, dtype=np.float64)
+    return _lib.scm2cpp_pls_mtv(a.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), y.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), m, n)
+
+_lib.scm2cpp_pls_dot.restype = ctypes.c_double
+_lib.scm2cpp_pls_dot.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.c_int]
+def pls_dot(x, y, m):
+    x = np.ascontiguousarray(x, dtype=np.float64)
+    y = np.ascontiguousarray(y, dtype=np.float64)
+    return _lib.scm2cpp_pls_dot(x.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), y.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), m)
+
+_lib.scm2cpp_pls_fit.restype = ctypes.c_int
+_lib.scm2cpp_pls_fit.argtypes = [ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_double), ctypes.POINTER(ctypes.c_int), ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_int, ctypes.c_double]
+def pls_fit(cxx, cxy, cyy, xw, yw, xl, yl, niter, p, q, k, mode, canon, algo, maxit, tol):
+    cxx = np.ascontiguousarray(cxx, dtype=np.float64)
+    cxy = np.ascontiguousarray(cxy, dtype=np.float64)
+    cyy = np.ascontiguousarray(cyy, dtype=np.float64)
+    xw = np.ascontiguousarray(xw, dtype=np.float64)
+    yw = np.ascontiguousarray(yw, dtype=np.float64)
+    xl = np.ascontiguousarray(xl, dtype=np.float64)
+    yl = np.ascontiguousarray(yl, dtype=np.float64)
+    niter = np.ascontiguousarray(niter, dtype=np.int32)
+    return _lib.scm2cpp_pls_fit(cxx.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), cxy.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), cyy.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), xw.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), yw.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), xl.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), yl.ctypes.data_as(ctypes.POINTER(ctypes.c_double)), niter.ctypes.data_as(ctypes.POINTER(ctypes.c_int)), p, q, k, mode, canon, algo, maxit, tol)
+