Skip to content

cca_zoo.linear

Linear CCA methods. All classes are sklearn.base.BaseEstimator subclasses.


Base class

BaseModel

BaseModel(latent_dimensions: int = 1, center: bool = True)

Bases: BaseEstimator, ABC

Abstract base class for all multiview CCA models.

Subclasses must implement :meth:fit. All other public methods (transform, fit_transform, score, pairwise_correlations, average_pairwise_correlations, get_factor_loadings) are provided here using the weights_ attribute set by fit.

This class inherits from :class:sklearn.base.BaseEstimator so that get_params / set_params round-trip correctly and sklearn model selection utilities work out of the box.

Constructor parameters are validated with sklearn's _parameter_constraints mechanism (see :meth:_setup_fit). Subclasses that add their own constructor parameters may extend _parameter_constraints by merging in BaseModel._parameter_constraints; parameters with no declared constraint are left unvalidated, so this is always safe to skip.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions to fit. Default is 1.

1
center bool

Whether to subtract per-view column means before fitting. The means are stored in means_ and applied in transform.

True

weights property

weights: list[ndarray]

Weight matrices post-fit, one per view.

Shape is (n_features_i, latent_dimensions) for each view.

Raises:

Type Description
NotFittedError

If fit has not been called.

fit abstractmethod

fit(views: list[ArrayLike], y: None = None) -> BaseModel

Fit the model to multiview data.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i). All arrays must have the same number of rows.

required
y None

Ignored. Present for scikit-learn API compatibility.

None

Returns:

Name Type Description
self BaseModel

Fitted estimator.

Raises:

Type Description
ValueError

If fewer than 2 views are provided.

ValueError

If views have inconsistent numbers of samples.

transform

transform(views: list[ArrayLike]) -> list[np.ndarray]

Project views into the latent space using the fitted weights.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required

Returns:

Type Description
list[ndarray]

List of arrays, each of shape (n_samples, latent_dimensions).

Raises:

Type Description
NotFittedError

If fit has not been called.

fit_transform

fit_transform(
    views: list[ArrayLike], y: None = None
) -> list[np.ndarray]

Fit and then transform the training data.

Equivalent to self.fit(views).transform(views) but may be more efficient for some subclasses.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Type Description
list[ndarray]

List of arrays, each of shape (n_samples, latent_dimensions).

score

score(views: list[ArrayLike], y: None = None) -> np.ndarray

Return average pairwise canonical correlations for each dimension.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Type Description
ndarray

Array of shape (latent_dimensions,) with the average

ndarray

pairwise correlation for each canonical dimension.

pairwise_correlations

pairwise_correlations(views: list[ArrayLike]) -> np.ndarray

Compute the full pairwise correlation matrix per latent dimension.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required

Returns:

Type Description
ndarray

Array of shape (n_views, n_views, latent_dimensions) where

ndarray

entry [i, j, d] is the Pearson correlation between the

ndarray

d-th canonical variate of view i and view j.

average_pairwise_correlations

average_pairwise_correlations(
    views: list[ArrayLike],
) -> np.ndarray

Return the mean off-diagonal pairwise correlation per dimension.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required

Returns:

Type Description
ndarray

Array of shape (latent_dimensions,) with the average

ndarray

off-diagonal pairwise correlation for each canonical dimension.

get_factor_loadings

get_factor_loadings(
    views: list[ArrayLike],
) -> list[np.ndarray]

Compute canonical factor loadings for each view.

A loading is the Pearson correlation between an original feature and a canonical variate. Loadings indicate which original variables drive each canonical direction.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required

Returns:

Type Description
list[ndarray]

List of arrays, each of shape (n_features_i, latent_dimensions),

list[ndarray]

where entry [j, d] is the correlation between feature j of

list[ndarray]

view i and the d-th canonical variate of view i.


Two-view exact methods

CCA

CCA(latent_dimensions: int = 1, center: bool = True)

Bases: rCCA

Canonical Correlation Analysis.

Finds the pair of linear projections that maximise the Pearson correlation between two views subject to unit within-view variance constraints:

\[ \begin{aligned} \max_{\mathbf{w}_1, \mathbf{w}_2} \mathbf{w}_1^\top X_1^\top X_2 \mathbf{w}_2 \\ \text{subject to } \mathbf{w}_i^\top X_i^\top X_i \mathbf{w}_i = 1 \end{aligned} \]

This is a special case of :class:rCCA with c=0. The solution uses PCA whitening followed by an SVD of the cross-covariance matrix, which is numerically stable even for high-dimensional views.

References

Hotelling, H. (1936). Relations between two sets of variates. Biometrika, 28(3/4), 321–377.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = CCA(latent_dimensions=2).fit([X1, X2]) corrs = model.score([X1, X2])

Source code in cca_zoo/linear/_cca.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        c=0.0,
    )

fit

fit(views: list[ArrayLike], y: None = None) -> CCA

Fit the CCA model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of exactly two arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self CCA

Fitted estimator.

Raises:

Type Description
ValueError

If the number of views is not exactly 2.

ValueError

If views have inconsistent numbers of samples.

Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = CCA(latent_dimensions=2).fit([X1, X2])

Source code in cca_zoo/linear/_cca.py
def fit(self, views: list[ArrayLike], y: None = None) -> CCA:
    """Fit the CCA model.

    Args:
        views: List of exactly two arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If the number of views is not exactly 2.
        ValueError: If views have inconsistent numbers of samples.

    Example:
        >>> import numpy as np
        >>> rng = np.random.default_rng(0)
        >>> X1 = rng.standard_normal((50, 10))
        >>> X2 = rng.standard_normal((50, 8))
        >>> model = CCA(latent_dimensions=2).fit([X1, X2])
    """
    return super().fit(views, y)

rCCA

rCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
)

Bases: BaseModel

Regularised Canonical Correlation Analysis (canonical ridge).

Finds the pair of linear projections of two views that maximise their correlation subject to regularised within-view variance constraints:

\[ \begin{aligned} \max_{\mathbf{w}_1, \mathbf{w}_2} \mathbf{w}_1^\top X_1^\top X_2 \mathbf{w}_2 \\ \text{subject to } \mathbf{w}_i^\top \bigl((1 - c_i) X_i^\top X_i + c_i I\bigr) \mathbf{w}_i = 1 \end{aligned} \]

The solution is found by whitening each view with its regularised covariance matrix and computing the SVD of the resulting cross-covariance.

:class:CCA (c=0) and :class:PLS (c=1) are special cases.

References

Vinod, H. D. (1976). Canonical ridge and econometrics of joint production. Journal of Econometrics, 4(2), 147–166.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
c float | list[float]

Ridge regularisation parameter(s) in [0, 1]. A single float is applied to both views; a list [c1, c2] applies per-view regularisation. Default is 0 (standard CCA).

0.0
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = rCCA(latent_dimensions=2, c=0.1).fit([X1, X2]) scores = model.transform([X1, X2])

Source code in cca_zoo/linear/_rcca.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
) -> None:
    super().__init__(latent_dimensions=latent_dimensions, center=center)
    self.c = c

fit

fit(views: list[ArrayLike], y: None = None) -> rCCA

Fit the rCCA model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of exactly two arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self rCCA

Fitted estimator.

Raises:

Type Description
ValueError

If the number of views is not exactly 2.

ValueError

If views have inconsistent numbers of samples.

Source code in cca_zoo/linear/_rcca.py
def fit(self, views: list[ArrayLike], y: None = None) -> rCCA:
    """Fit the rCCA model.

    Args:
        views: List of exactly two arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If the number of views is not exactly 2.
        ValueError: If views have inconsistent numbers of samples.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    if self.n_views_ != 2:
        raise ValueError(
            f"rCCA requires exactly 2 views, got {self.n_views_}. "
            "Use MCCA for more than 2 views."
        )
    c_ = perview_parameter("c", self.c, 0.0, 2)
    X1, X2 = views_
    # Whiten each view with its regularised covariance
    X1_w, W1 = svd_whiten(X1, c_[0])
    X2_w, W2 = svd_whiten(X2, c_[1])
    # SVD of the cross-covariance of whitened views
    k = min(self.latent_dimensions, X1_w.shape[1], X2_w.shape[1])
    cross_cov = X1_w.T @ X2_w / (X1.shape[0] - 1)
    U, _, Vt = np.linalg.svd(cross_cov, full_matrices=False)
    U = U[:, :k]
    Vt = Vt[:k, :]
    self.weights_: list[np.ndarray] = [W1 @ U, W2 @ Vt.T]
    return self

PLS

PLS(latent_dimensions: int = 1, center: bool = True)

Bases: rCCA

Partial Least Squares (two-view).

Finds the pair of unit-norm weight vectors that maximise the covariance between the projected views:

\[ \begin{aligned} \max_{\mathbf{w}_1, \mathbf{w}_2} \mathbf{w}_1^\top X_1^\top X_2 \mathbf{w}_2 \\ \text{subject to } \|\mathbf{w}_i\|_2 = 1 \end{aligned} \]

This is equivalent to the truncated SVD of the sample cross-covariance matrix \(X_1^\top X_2 / (n - 1)\), and corresponds to :class:rCCA with c=1.

References

Wold, H. (1975). Soft modelling by latent variables: the nonlinear iterative partial least squares (NIPALS) approach. Perspectives in Probability and Statistics, 117–142.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = PLS(latent_dimensions=2).fit([X1, X2]) scores = model.transform([X1, X2])

Source code in cca_zoo/linear/_pls.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        c=1.0,
    )

fit

fit(views: list[ArrayLike], y: None = None) -> PLS

Fit the PLS model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of exactly two arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self PLS

Fitted estimator.

Raises:

Type Description
ValueError

If the number of views is not exactly 2.

ValueError

If views have inconsistent numbers of samples.

Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = PLS(latent_dimensions=2).fit([X1, X2])

Source code in cca_zoo/linear/_pls.py
def fit(self, views: list[ArrayLike], y: None = None) -> PLS:
    """Fit the PLS model.

    Args:
        views: List of exactly two arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If the number of views is not exactly 2.
        ValueError: If views have inconsistent numbers of samples.

    Example:
        >>> import numpy as np
        >>> rng = np.random.default_rng(0)
        >>> X1 = rng.standard_normal((50, 10))
        >>> X2 = rng.standard_normal((50, 8))
        >>> model = PLS(latent_dimensions=2).fit([X1, X2])
    """
    return super().fit(views, y)

Multiview methods

MCCA

MCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    pca: bool = True,
    eps: float = 1e-06,
)

Bases: BaseModel

Multiset Canonical Correlation Analysis.

Finds linear projections of multiple (>=2) views that maximise the sum of pairwise cross-view covariances subject to within-view variance constraints. A ridge regularisation parameter c controls the trade-off between correlation and variance explained.

The primal objective is:

\[ \begin{aligned} \max_{\mathbf{w}} \sum_{i \neq j} \mathbf{w}_i^\top X_i^\top X_j \mathbf{w}_j \\ \text{subject to } \mathbf{w}_i^\top \bigl((1-c_i) X_i^\top X_i + c_i I\bigr) \mathbf{w}_i = 1 \end{aligned} \]

This is solved as a generalised eigenvalue problem:

\[ A \mathbf{v} = \lambda B \mathbf{v} \]

where \(A\) is the between-view block covariance matrix and \(B\) is the block-diagonal regularised within-view covariance matrix.

When pca=True (default), each view is first reduced to its principal components, which makes the problem numerically stable for high-dimensional data and allows an efficient closed-form \(B\).

References

Kettenring, J. R. (1971). Canonical analysis of several sets of variables. Biometrika, 58(3), 433–451.

Vinod, H. D. (1976). Canonical ridge and econometrics of joint production. Journal of Econometrics, 4(2), 147–166.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
c float | list[float]

Ridge regularisation parameter(s). Either a single float applied to all views or a list of per-view floats in [0, 1]. c=0 gives standard CCA constraints; c=1 gives sphering (PLS-like). Default is 0.

0.0
pca bool

Whether to apply full PCA whitening as a pre-processing step before solving the eigenvalue problem. Highly recommended for high-dimensional data. Default is True.

True
eps float

Small constant added to the eigenvalues of B to ensure positive definiteness. Default is 1e-6.

1e-06
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) X3 = rng.standard_normal((50, 6)) model = MCCA(latent_dimensions=2).fit([X1, X2, X3]) scores = model.transform([X1, X2, X3])

Source code in cca_zoo/linear/_mcca.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    pca: bool = True,
    eps: float = 1e-6,
) -> None:
    super().__init__(latent_dimensions=latent_dimensions, center=center)
    self.c = c
    self.pca = pca
    self.eps = eps

fit

fit(views: list[ArrayLike], y: None = None) -> MCCA

Fit the MCCA model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self MCCA

Fitted estimator.

Raises:

Type Description
ValueError

If fewer than 2 views are provided.

ValueError

If views have inconsistent numbers of samples.

Source code in cca_zoo/linear/_mcca.py
def fit(self, views: list[ArrayLike], y: None = None) -> MCCA:
    """Fit the MCCA model.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If fewer than 2 views are provided.
        ValueError: If views have inconsistent numbers of samples.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    c_ = perview_parameter("c", self.c, 0.0, self.n_views_)

    if self.pca:
        pca_models = [PCA().fit(v) for v in views_]
        views_pca = [m.transform(v) for m, v in zip(pca_models, views_)]
        A = self._build_A(views_pca)
        B = self._build_B_pca(pca_models, c_)
    else:
        A = self._build_A(views_)
        B = self._build_B(views_, c_)

    splits = np.cumsum([v.shape[1] for v in (views_pca if self.pca else views_)])
    _, eigvecs = gevp(A, B, self.latent_dimensions)

    raw_weights = np.split(eigvecs, splits[:-1], axis=0)
    if self.pca:
        self.weights_: list[np.ndarray] = [
            m.components_.T @ w for m, w in zip(pca_models, raw_weights)
        ]
    else:
        self.weights_ = raw_weights
    return self

GCCA

GCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    view_weights: list[float] | None = None,
    eps: float = 1e-06,
)

Bases: BaseModel

Generalised Canonical Correlation Analysis.

Finds linear projections of multiple (>=2) views that maximise their joint correlation with a shared auxiliary latent vector:

\[ \begin{aligned} \max_{\mathbf{w}_i, T} \sum_{i=1}^M \mathbf{w}_i^\top X_i^\top T \\ \text{subject to } T^\top T = I \end{aligned} \]

The solution is obtained by constructing the weighted projection matrix:

\[ Q = \sum_{i=1}^M \mu_i X_i \bigl((1-c_i) X_i^\top X_i + c_i I\bigr)^{-1} X_i^\top \]

and computing its top-k eigenvectors \(V\), then recovering the per-view weights as \(\mathbf{w}_i = X_i^+ V\).

References

Tenenhaus, A., & Tenenhaus, M. (2011). Regularized generalized canonical correlation analysis. Psychometrika, 76(2), 257–284.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
c float | list[float]

Ridge regularisation parameter(s) in [0, 1]. Default is 0.

0.0
view_weights list[float] | None

Per-view weights \(\mu_i\) in the GCCA objective. Default is equal weights (1 for all views).

None
eps float

Regularisation floor for within-view matrices. Default is 1e-6.

1e-06
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) X3 = rng.standard_normal((50, 6)) model = GCCA(latent_dimensions=2).fit([X1, X2, X3]) scores = model.transform([X1, X2, X3])

Source code in cca_zoo/linear/_gcca.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    view_weights: list[float] | None = None,
    eps: float = 1e-6,
) -> None:
    super().__init__(latent_dimensions=latent_dimensions, center=center)
    self.c = c
    self.view_weights = view_weights
    self.eps = eps

fit

fit(views: list[ArrayLike], y: None = None) -> GCCA

Fit the GCCA model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self GCCA

Fitted estimator.

Raises:

Type Description
ValueError

If fewer than 2 views are provided.

ValueError

If views have inconsistent numbers of samples.

Source code in cca_zoo/linear/_gcca.py
def fit(self, views: list[ArrayLike], y: None = None) -> GCCA:
    """Fit the GCCA model.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If fewer than 2 views are provided.
        ValueError: If views have inconsistent numbers of samples.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    c_ = perview_parameter("c", self.c, 0.0, self.n_views_)
    mu = perview_parameter("view_weights", self.view_weights, 1.0, self.n_views_)

    # Build Q = sum_i mu_i X_i (cov_i)^{-1} X_i^T
    Q = np.zeros((self.n_samples_, self.n_samples_))
    for i, (v, ci, mi) in enumerate(zip(views_, c_, mu)):
        cov_i = (1.0 - ci) * np.cov(v, rowvar=False) + ci * np.eye(v.shape[1])
        min_eig = np.linalg.eigvalsh(cov_i).min()
        if min_eig < self.eps:
            cov_i += (self.eps - min_eig) * np.eye(cov_i.shape[0])
        Q += mi * (v @ np.linalg.inv(cov_i) @ v.T)

    _, eigvecs = gevp(Q, None, self.latent_dimensions)
    T = eigvecs[:, : self.latent_dimensions]  # (n_samples, k)
    self.weights_: list[np.ndarray] = [np.linalg.pinv(v) @ T for v in views_]
    return self

TCCA

TCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    eps: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Tensor Canonical Correlation Analysis.

Extends CCA to more than two views by exploiting higher-order cross-view correlations via a tensor product structure. The method constructs the order-M cross-moment tensor:

\[ \mathcal{M}_{p_1 p_2 \ldots p_M} = \frac{1}{n} \sum_{i=1}^n \tilde{x}_{1,i}^{(p_1)} \tilde{x}_{2,i}^{(p_2)} \cdots \tilde{x}_{M,i}^{(p_M)} \]

where \(\tilde{X}_j = X_j \Sigma_j^{-1/2}\) are the whitened views, and then decomposes \(\mathcal{M}\) using PARAFAC to recover the canonical directions.

References

Kim, T.-K., Wong, S.-F., & Cipolla, R. (2007). Tensor canonical correlation analysis for action classification. CVPR 2007. IEEE.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
c float | list[float]

Ridge regularisation in [0, 1]. Default is 0.

0.0
eps float

Regularisation floor for within-view covariance matrices.

1e-06
random_state int | None

Seed for reproducibility (passed to PARAFAC).

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 5)) X2 = rng.standard_normal((50, 5)) X3 = rng.standard_normal((50, 5)) model = TCCA(latent_dimensions=2, random_state=0).fit([X1, X2, X3]) scores = model.transform([X1, X2, X3])

Source code in cca_zoo/linear/_tcca.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    eps: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(latent_dimensions=latent_dimensions, center=center)
    self.c = c
    self.eps = eps
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> TCCA

Fit the TCCA model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self TCCA

Fitted estimator.

Raises:

Type Description
ValueError

If fewer than 2 views are provided.

ValueError

If views have inconsistent numbers of samples.

Source code in cca_zoo/linear/_tcca.py
def fit(self, views: list[ArrayLike], y: None = None) -> TCCA:
    """Fit the TCCA model.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If fewer than 2 views are provided.
        ValueError: If views have inconsistent numbers of samples.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    c_ = perview_parameter("c", self.c, 0.0, self.n_views_)
    whitened, cov_invsqrt = self._whiten_views(views_, c_)

    # Build cross-moment tensor via sequential outer products
    M: np.ndarray | None = None
    for i, wv in enumerate(whitened):
        if M is None:
            M = wv
        else:
            for _ in range(len(M.shape) - 1):
                wv = np.expand_dims(wv, 1)
            M = np.expand_dims(M, -1) @ wv
    assert M is not None
    M = np.mean(M, 0)

    tl.set_backend("numpy")
    parafac_result = parafac(
        M,
        self.latent_dimensions,
        verbose=False,
        random_state=self.random_state,
    )
    self.weights_: list[np.ndarray] = [
        cov_invsqrt[i] @ fac for i, fac in enumerate(parafac_result.factors)
    ]
    return self

Confound-adjusted / structured methods

PartialCCA

PartialCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    eps: float = 1e-06,
)

Bases: MCCA

Partial Canonical Correlation Analysis.

Extends CCA to account for confounding variables partials that may drive the correlation between views. Each view is first deconfounded by regressing out partials via least squares, and (ridge-regularised) CCA is then applied to the residuals, subject to the additional constraint that canonical weights are orthogonal to the confounds:

\[ \begin{aligned} w_{opt} = \underset{w}{\mathrm{argmax}}\ w_1^\top X_1^\top X_2 w_2 \\ \text{subject to } w_i^\top X_i^\top X_i w_i = 1, \quad w_i^\top X_i^\top Z = 0 \end{aligned} \]
References

Rao, B. R. (1969). Partial canonical correlations. Trabajos de Estadistica y de Investigacion Operativa, 20(2-3), 211-219.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
c float | list[float]

Ridge regularisation parameter(s) applied to the deconfounded views. Either a scalar or a per-view list. Default is 0.

0.0
eps float

Small constant added to the eigenvalues of B to ensure positive definiteness. Default is 1e-6.

1e-06
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) Z = rng.standard_normal((50, 3)) model = PartialCCA(latent_dimensions=2).fit([X1, X2], partials=Z) scores = model.transform([X1, X2], partials=Z)

Source code in cca_zoo/linear/_partialcca.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    eps: float = 1e-6,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        c=c,
        pca=False,
        eps=eps,
    )

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    partials: ArrayLike | None = None,
) -> PartialCCA

Fit the Partial CCA model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None
partials ArrayLike | None

Confound array of shape (n_samples, n_confounds) to regress out of each view before fitting CCA. Required.

None

Returns:

Name Type Description
self PartialCCA

Fitted estimator.

Raises:

Type Description
ValueError

If partials is not provided.

Source code in cca_zoo/linear/_partialcca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    partials: ArrayLike | None = None,
) -> PartialCCA:
    """Fit the Partial CCA model.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.
        partials: Confound array of shape (n_samples, n_confounds) to
            regress out of each view before fitting CCA. Required.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If ``partials`` is not provided.
    """
    if partials is None:
        raise ValueError("PartialCCA requires `partials` to be provided to fit().")
    views_ = self._setup_fit(views)
    partials_arr = np.asarray(partials, dtype=float)
    self.confound_betas_: list[np.ndarray] = [
        np.linalg.pinv(partials_arr) @ v for v in views_
    ]
    deconfounded = [
        v - partials_arr @ beta for v, beta in zip(views_, self.confound_betas_)
    ]
    c_ = perview_parameter("c", self.c, 0.0, self.n_views_)
    A = self._build_A(deconfounded)
    B = self._build_B(deconfounded, c_)
    _, eigvecs = gevp(A, B, self.latent_dimensions)
    splits = np.cumsum([v.shape[1] for v in deconfounded])
    self.weights_: list[np.ndarray] = np.split(eigvecs, splits[:-1], axis=0)
    return self

transform

transform(
    views: list[ArrayLike],
    partials: ArrayLike | None = None,
) -> list[np.ndarray]

Project views into the latent space, optionally removing confounds.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
partials ArrayLike | None

Confound array matching the one used at fit time. If omitted, no deconfounding is applied (falls back to a plain linear projection), which keeps score/fit_transform usable without threading partials through every call.

None

Returns:

Type Description
list[ndarray]

List of arrays, each (n_samples, latent_dimensions).

Source code in cca_zoo/linear/_partialcca.py
def transform(
    self,
    views: list[ArrayLike],
    partials: ArrayLike | None = None,
) -> list[np.ndarray]:
    """Project views into the latent space, optionally removing confounds.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        partials: Confound array matching the one used at fit time. If
            omitted, no deconfounding is applied (falls back to a plain
            linear projection), which keeps ``score``/``fit_transform``
            usable without threading ``partials`` through every call.

    Returns:
        List of arrays, each (n_samples, latent_dimensions).
    """
    check_is_fitted(self)
    if partials is None:
        return super().transform(views)
    validated = validate_views(views)
    partials_arr = np.asarray(partials, dtype=float)
    centred = [v - m for v, m in zip(validated, self.means_)]
    deconfounded = [
        v - partials_arr @ beta for v, beta in zip(centred, self.confound_betas_)
    ]
    return [v @ w for v, w in zip(deconfounded, self.weights_)]

fit_transform

fit_transform(
    views: list[ArrayLike],
    y: None = None,
    partials: ArrayLike | None = None,
) -> list[np.ndarray]

Fit and then transform the training data.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None
partials ArrayLike | None

Confound array, passed through to both fit and transform.

None

Returns:

Type Description
list[ndarray]

List of arrays, each (n_samples, latent_dimensions).

Source code in cca_zoo/linear/_partialcca.py
def fit_transform(
    self,
    views: list[ArrayLike],
    y: None = None,
    partials: ArrayLike | None = None,
) -> list[np.ndarray]:
    """Fit and then transform the training data.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.
        partials: Confound array, passed through to both ``fit`` and
            ``transform``.

    Returns:
        List of arrays, each (n_samples, latent_dimensions).
    """
    return self.fit(views, y=y, partials=partials).transform(
        views, partials=partials
    )

GRCCA

GRCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    mu: float | list[float] = 0.0,
    eps: float = 1e-06,
)

Bases: MCCA

Group Regularised Canonical Correlation Analysis.

Extends :class:MCCA with structured ridge regularisation that shrinks within-group feature weights toward a shared group-level effect. Each view's features are partitioned into groups via feature_groups; the per-view c parameter controls shrinkage of within-group deviations and mu controls the weighting of the group-level effect.

Each view is internally augmented with group-mean features before solving the generalised eigenvalue problem, then the resulting weights are algebraically collapsed back to the original feature space, so transform operates directly on the un-augmented views.

References

Tuzhilina, E., Tozzi, L., & Hastie, T. (2021). Canonical correlation analysis in high dimensions with structured regularization. Statistical Modelling.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
c float | list[float]

Ridge regularisation parameter(s) controlling within-group shrinkage. Either a scalar applied to all views or a per-view list, each in [0, 1]. c=0 disables grouping for that view (falls back to plain MCCA behaviour). Default is 0.

0.0
mu float | list[float]

Regularisation parameter(s) controlling the group-level effect scale. Either a scalar or a per-view list. Default is 0.

0.0
eps float

Small constant added to the eigenvalues of B to ensure positive definiteness. Default is 1e-6.

1e-06
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) groups1 = rng.integers(0, 3, size=10) groups2 = rng.integers(0, 3, size=8) model = GRCCA(latent_dimensions=2, c=0.5).fit( ... [X1, X2], feature_groups=[groups1, groups2] ... ) scores = model.transform([X1, X2])

Source code in cca_zoo/linear/_grcca.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    c: float | list[float] = 0.0,
    mu: float | list[float] = 0.0,
    eps: float = 1e-6,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        c=c,
        pca=False,
        eps=eps,
    )
    self.mu = mu

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    feature_groups: list[ndarray] | None = None,
) -> GRCCA

Fit the GRCCA model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None
feature_groups list[ndarray] | None

List of integer group-label arrays, one per view, each of shape (n_features_i,). Required for meaningful grouping whenever the corresponding per-view c is nonzero; defaults to a single group per view (equivalent to plain MCCA) if omitted.

None

Returns:

Name Type Description
self GRCCA

Fitted estimator.

Source code in cca_zoo/linear/_grcca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    feature_groups: list[np.ndarray] | None = None,
) -> GRCCA:
    """Fit the GRCCA model.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.
        feature_groups: List of integer group-label arrays, one per
            view, each of shape (n_features_i,). Required for
            meaningful grouping whenever the corresponding per-view
            ``c`` is nonzero; defaults to a single group per view
            (equivalent to plain MCCA) if omitted.

    Returns:
        self: Fitted estimator.
    """
    views_ = self._setup_fit(views)
    c_ = perview_parameter("c", self.c, 0.0, self.n_views_)
    mu_ = perview_parameter("mu", self.mu, 0.0, self.n_views_)

    if feature_groups is None:
        if any(ci > 0 for ci in c_):
            warnings.warn(
                "No feature_groups provided; using a single group per "
                "view, which makes the group regularisation a no-op."
            )
        feature_groups = [np.ones(v.shape[1], dtype=int) for v in views_]
    self.feature_groups_ = feature_groups

    processed = [
        self._augment_view(v, g, m, c)
        for v, g, m, c in zip(views_, feature_groups, mu_, c_)
    ]
    A = self._build_A(processed)
    B = self._build_B(processed, c_)
    _, eigvecs = gevp(A, B, self.latent_dimensions)
    splits = np.cumsum([v.shape[1] for v in processed])
    raw_blocks = np.split(eigvecs, splits[:-1], axis=0)

    self.weights_: list[np.ndarray] = [
        self._collapse_weights(block, g, c, m)
        for block, g, c, m in zip(raw_blocks, feature_groups, c_, mu_)
    ]
    return self

Reduced-rank regression methods

CCAR3

CCAR3(
    latent_dimensions: int = 1,
    center: bool = True,
    lambda_: float = 0.0,
    highdim: bool = True,
    ledoit_wolf: bool = True,
    rho: float = 1.0,
    max_iter: int = 10000,
    tol: float = 0.0001,
    eps: float = 1e-08,
)

Bases: BaseModel

Canonical Correlation Analysis via Reduced Rank Regression.

Recasts two-view CCA as a reduced-rank regression: Y is first whitened by its (optionally Ledoit-Wolf shrunk) covariance,

\[ \tilde{Y} = Y \Sigma_Y^{-1/2}, \]

and a coefficient matrix \(B\) relating \(X\) to \(\tilde{Y}\) is estimated. In the low-dimensional regime (highdim=False) this has the closed form \(B = \Sigma_X^{-1} X^\top \tilde{Y} / n\) (an ordinary reduced-rank regression, distinct from the classical CCA eigenproblem — the two agree only when \(\Sigma_X\) is close to isotropic). In the high-dimensional regime (highdim=True, the default), \(B\) is instead estimated by a row-wise group-lasso-penalised regression, solved by ADMM:

\[ \begin{aligned} \hat{B} = \underset{B}{\mathrm{argmin}}\ \frac{1}{n} \lVert \tilde{Y} - X B \rVert_F^2 + \lambda \sum_{j=1}^{p} \lVert B_{j, :} \rVert_2 \end{aligned} \]

which drives whole rows of \(B\) (whole \(X\) features) to zero, giving a sparse-in-\(X\) solution well-suited to \(p \gg n\). The rank- latent_dimensions SVD of \(\hat{B}\) gives the canonical directions, which are then whitened so that the canonical variates have unit variance, sign-aligned to positive correlation, and sorted in descending order.

Because the penalty acts on rows of \(B\), sparsity is induced only in \(X\); \(Y\) is handled densely via its inverse-square-root covariance. Swap the order of views to regularise the other view instead.

This is a from-scratch NumPy port of the reference R implementation, ccar3; cross-validation utilities and the CVXR/rrpack solver backends are out of scope here — use GridSearchCV from cca_zoo.model_selection to select lambda_ as for any other estimator.

References

Donnat, C., & Tuzhilina, E. (2024). Canonical Correlation Analysis as Reduced Rank Regression in High Dimensions. arXiv:2405.19539.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means before fitting. Default True.

True
lambda_ float

Row-group-lasso regularisation strength used when highdim=True. 0 disables the penalty. Default is 0.

0.0
highdim bool

Whether to estimate the reduced-rank coefficient with the ADMM-solved group-lasso penalty (default, needed when X has more features than samples) or with the closed-form low-dimensional solution (False).

True
ledoit_wolf bool

Whether to shrink the Y covariance matrix with Ledoit-Wolf shrinkage before inverting it. Default True.

True
rho float

ADMM step-size parameter. Default 1.0.

1.0
max_iter int

Maximum number of ADMM iterations. Default 10_000.

10000
tol float

ADMM convergence tolerance on the primal/dual residuals. Default 1e-4.

0.0001
eps float

Small constant added to covariance matrices before inversion, for numerical stability. Default 1e-8.

1e-08
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = CCAR3(latent_dimensions=2, highdim=False).fit([X1, X2]) scores = model.transform([X1, X2])

Source code in cca_zoo/linear/_ccar3.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    lambda_: float = 0.0,
    highdim: bool = True,
    ledoit_wolf: bool = True,
    rho: float = 1.0,
    max_iter: int = 10_000,
    tol: float = 1e-4,
    eps: float = 1e-8,
) -> None:
    super().__init__(latent_dimensions=latent_dimensions, center=center)
    self.lambda_ = lambda_
    self.highdim = highdim
    self.ledoit_wolf = ledoit_wolf
    self.rho = rho
    self.max_iter = max_iter
    self.tol = tol
    self.eps = eps

fit

fit(views: list[ArrayLike], y: None = None) -> CCAR3

Fit the CCAR3 model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of exactly two arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self CCAR3

Fitted estimator.

Raises:

Type Description
ValueError

If the number of views is not exactly 2.

ValueError

If views have inconsistent numbers of samples.

Source code in cca_zoo/linear/_ccar3.py
def fit(self, views: list[ArrayLike], y: None = None) -> CCAR3:
    """Fit the CCAR3 model.

    Args:
        views: List of exactly two arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If the number of views is not exactly 2.
        ValueError: If views have inconsistent numbers of samples.
    """
    views_ = self._setup_fit(views)
    if self.n_views_ != 2:
        raise ValueError(
            f"CCAR3 requires exactly 2 views, got {self.n_views_}. "
            "Use MCCA for more than 2 views."
        )
    X, Y = views_
    n = X.shape[0]

    Sy = LedoitWolf().fit(Y).covariance_ if self.ledoit_wolf else Y.T @ Y / n
    sqrt_inv_Sy = _sqrt_inv_psd(Sy)
    Y_tilde = Y @ sqrt_inv_Sy

    if self.highdim:
        B = _admm_row_sparse_rrr(
            X,
            Y_tilde,
            lambda_=self.lambda_,
            rho=self.rho,
            max_iter=self.max_iter,
            tol=self.tol,
            ridge=self.eps,
        )
    else:
        Sx = X.T @ X / n + self.eps * np.eye(X.shape[1])
        B = np.linalg.solve(Sx, X.T @ Y_tilde / n)

    U, V = _postprocess_rrr_fit(
        B, X, Y, sqrt_inv_Sy, self.latent_dimensions, ridge=self.eps
    )
    self.weights_: list[np.ndarray] = [U, V]
    return self

Gradient-descent methods

PLS_EY

PLS_EY(
    latent_dimensions: int = 1,
    center: bool = True,
    learning_rate: float = 0.01,
    max_iter: int = 1000,
    batch_size: int | None = None,
    tol: float = 1e-06,
    momentum: float = 0.9,
    random_state: int | None = None,
)

Bases: CCA_EY

Stochastic Eckart-Young PLS for large-scale data.

This is equivalent to :class:~cca_zoo.linear.gradient.CCA_EY with c=1: the reward excludes the \(i = j\) terms that CCA_EY's (\(c=0\)) reward includes, and the penalty is purely \(\operatorname{tr}(BB)\) on the weight Gram matrix \(B\), which drives the weights towards (approximate) orthonormality at the optimum on its own — no manifold projection step, and no upfront whitening.

Suitable for high-dimensional or streaming data where forming the full (p x p) cross-covariance matrix is too expensive.

Initial weights have exactly orthonormal columns (unit-norm, mutually orthogonal) before any gradient step, matching the shape of this loss's own penalty on \(B\) — unlike :class:~cca_zoo.linear.gradient.CCA_EY's own data-informed default, which instead orthonormalises the initial projections (see :func:cca_zoo._utils._ey.random_orthonormal_weights vs. :func:cca_zoo._utils._ey.cheap_orthonormal_projection_weights).

References

Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
learning_rate float

Gradient step size. Default is 1e-2.

0.01
max_iter int

Number of gradient steps. Default is 1000.

1000
batch_size int | None

Mini-batch size. None uses the full dataset.

None
tol float

Convergence tolerance on the objective change. Default is 1e-6.

1e-06
momentum float

Momentum coefficient in [0, 1). Default is 0.9.

0.9
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((200, 500)) X2 = rng.standard_normal((200, 400)) model = PLS_EY(latent_dimensions=4, batch_size=64, random_state=0) model = model.fit([X1, X2])

Source code in cca_zoo/linear/gradient/_pls_ey.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    learning_rate: float = 1e-2,
    max_iter: int = 1000,
    batch_size: int | None = None,
    tol: float = 1e-6,
    momentum: float = 0.9,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        c=1.0,
        learning_rate=learning_rate,
        max_iter=max_iter,
        batch_size=batch_size,
        tol=tol,
        momentum=momentum,
        random_state=random_state,
    )

fit

fit(views: list[ArrayLike], y: None = None) -> PLS_EY

Fit PLS_EY by mini-batch momentum gradient descent.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self PLS_EY

Fitted estimator.

Raises:

Type Description
ValueError

If fewer than 2 views are provided.

ValueError

If views have inconsistent numbers of samples.

Source code in cca_zoo/linear/gradient/_pls_ey.py
def fit(self, views: list[ArrayLike], y: None = None) -> PLS_EY:
    """Fit PLS_EY by mini-batch momentum gradient descent.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If fewer than 2 views are provided.
        ValueError: If views have inconsistent numbers of samples.
    """
    return super().fit(views, y)

CCA_EY

CCA_EY(
    latent_dimensions: int = 1,
    center: bool = True,
    c: float = 0.0,
    learning_rate: float = 0.01,
    max_iter: int = 1000,
    batch_size: int | None = None,
    tol: float = 1e-06,
    momentum: float = 0.9,
    random_state: int | None = None,
)

Bases: BaseGradientModel

Eckart-Young CCA for large-scale data, ridge-blended with PLS_EY.

Optimises the unconstrained Eckart-Young (EY) objective by mini-batch momentum gradient descent directly on the raw (centred) views, with no manifold projection step and no upfront whitening: unlike classical CCA, which whitens each view before finding the correlated directions, the EY reformulation folds the orthonormalising pressure into the loss itself, so a full-batch preprocessing pass over the data is never needed. This matches how the same underlying loss is used, unwhitened, by :class:~cca_zoo.linear.gradient.PLS_EY, :class:~cca_zoo.tree.TreeCCA, and :class:~cca_zoo.deep.DCCA_EY.

For embeddings \(Z_i = X_i W_i\), let \(C\) and \(V\) be the mean pairwise cross-covariance and mean auto-covariance across views (see :func:cca_zoo._utils._ey.ey_cross_covariance), and \(B = \frac{1}{M}\sum_i W_i^\top W_i\) the mean weight Gram matrix (see :func:cca_zoo._utils._ey.weight_gram_mean). c blends the within-view normalisation between the data's own auto-covariance and the identity (in weight space, \(W_i^\top I W_i = W_i^\top W_i\)) — exactly the canonical-ridge blend \((1-c)X^\top X + cI\) already used by :class:~cca_zoo.linear.rCCA, translated into this unconstrained, stochastic setting:

\[ V_c = (1 - c) V + c B, \qquad \mathcal{L}_{EY}(c) = -2 \operatorname{tr}(C - c V) + \operatorname{tr}(V_c V_c) \]

c=0 recovers plain (unregularised) CCA_EY exactly; c=1 recovers :class:~cca_zoo.linear.gradient.PLS_EY's loss exactly (its reward excludes the \(i=j\) terms that \(\mathcal{L}_{EY}(0)\) includes, and its penalty is purely \(\operatorname{tr}(BB)\)) — both endpoints, and the gradient at intermediate \(c\), are verified against finite differences and against PLS_EY's own independently verified gradient. This objective has the canonical directions as a stationary point without requiring an explicit orthonormality constraint, unlike a plain squared-projection-distance loss.

Note

Unlike the exact, closed-form :class:~cca_zoo.linear.rCCA (where c=0 is always numerically safe), gradient descent on the raw, unregularised (\(c=0\)) objective can diverge to nan when a mini-batch's samples don't outnumber the number of features by a healthy margin — e.g. n_features approaching or exceeding batch_size — since nothing then bounds the weights in the data's near-null directions. If you see nan weights, increase c (a small value like 0.1-0.3 is usually enough) or batch_size rather than assuming the model doesn't apply to your data. (Initial weights give exactly unit-variance, uncorrelated projections on one mini-batch — see :func:cca_zoo._utils._ey.cheap_orthonormal_projection_weights — a cheap stand-in for classical CCA's full whitening step and the natural match for this loss's own fixed point; empirically this does not postpone the divergence above, since every later mini-batch is an independent fresh draw, so c/batch_size remain the actual remedy.)

References

Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
c float

Ridge blend in [0, 1] between CCA_EY (0) and PLS_EY (1). Default is 0 (standard, unregularised CCA_EY); see the note above on numerical stability for high-dimensional data.

0.0
learning_rate float

Gradient step size. Default is 1e-2.

0.01
max_iter int

Number of gradient steps. Default is 1000.

1000
batch_size int | None

Mini-batch size. None uses the full dataset.

None
tol float

Convergence tolerance. Default is 1e-6.

1e-06
momentum float

Momentum coefficient in [0, 1). Default is 0.9.

0.9
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((5000, 200)) X2 = rng.standard_normal((5000, 150)) model = CCA_EY(latent_dimensions=4, batch_size=128, random_state=0) model = model.fit([X1, X2])

Source code in cca_zoo/linear/gradient/_cca_ey.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    c: float = 0.0,
    learning_rate: float = 1e-2,
    max_iter: int = 1000,
    batch_size: int | None = None,
    tol: float = 1e-6,
    momentum: float = 0.9,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        learning_rate=learning_rate,
        max_iter=max_iter,
        batch_size=batch_size,
        tol=tol,
        momentum=momentum,
        random_state=random_state,
    )
    self.c = c

fit

fit(views: list[ArrayLike], y: None = None) -> CCA_EY

Fit CCA_EY by mini-batch momentum gradient descent.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self CCA_EY

Fitted estimator.

Raises:

Type Description
ValueError

If fewer than 2 views are provided.

ValueError

If views have inconsistent numbers of samples.

Source code in cca_zoo/linear/gradient/_cca_ey.py
def fit(self, views: list[ArrayLike], y: None = None) -> CCA_EY:
    """Fit CCA_EY by mini-batch momentum gradient descent.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If fewer than 2 views are provided.
        ValueError: If views have inconsistent numbers of samples.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    self.weights_ = self._gradient_descent(views_, rng)
    return self

MCCA_EY

MCCA_EY(
    latent_dimensions: int = 1,
    center: bool = True,
    c: float = 0.0,
    learning_rate: float = 0.01,
    max_iter: int = 1000,
    batch_size: int | None = None,
    tol: float = 1e-06,
    momentum: float = 0.9,
    random_state: int | None = None,
)

Bases: CCA_EY

Eckart-Young multiview CCA for large-scale data (>=2 views).

Identical to :class:CCA_EY; the shared Eckart-Young loss and its gradient (see :mod:cca_zoo._utils._ey) are already defined for an arbitrary number of views, so no multiview-specific logic is needed here.

References

Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
c float

Ridge blend in [0, 1] between CCA_EY-like (0) and PLS_EY-like (1) behaviour. Default is 0; see :class:CCA_EY's docstring for the numerical-stability note on high-dimensional data.

0.0
learning_rate float

Gradient step size. Default is 1e-2.

0.01
max_iter int

Number of gradient steps. Default is 1000.

1000
batch_size int | None

Mini-batch size. None uses the full dataset.

None
tol float

Convergence tolerance. Default is 1e-6.

1e-06
momentum float

Momentum coefficient in [0, 1). Default is 0.9.

0.9
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((5000, 200)) X2 = rng.standard_normal((5000, 150)) X3 = rng.standard_normal((5000, 100)) model = MCCA_EY(latent_dimensions=4, batch_size=128, random_state=0) model = model.fit([X1, X2, X3])

Source code in cca_zoo/linear/gradient/_cca_ey.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    c: float = 0.0,
    learning_rate: float = 1e-2,
    max_iter: int = 1000,
    batch_size: int | None = None,
    tol: float = 1e-6,
    momentum: float = 0.9,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        learning_rate=learning_rate,
        max_iter=max_iter,
        batch_size=batch_size,
        tol=tol,
        momentum=momentum,
        random_state=random_state,
    )
    self.c = c

fit

fit(views: list[ArrayLike], y: None = None) -> MCCA_EY

Fit MCCA_EY for 2 or more views.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self MCCA_EY

Fitted estimator.

Raises:

Type Description
ValueError

If fewer than 2 views are provided.

ValueError

If views have inconsistent numbers of samples.

Source code in cca_zoo/linear/gradient/_mcca_ey.py
def fit(self, views: list[ArrayLike], y: None = None) -> MCCA_EY:
    """Fit MCCA_EY for 2 or more views.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If fewer than 2 views are provided.
        ValueError: If views have inconsistent numbers of samples.
    """
    super().fit(views, y)
    return self

Sparse / iterative methods

PLS_ALS

PLS_ALS(
    latent_dimensions: int = 1,
    center: bool = True,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: _BaseIterative

Alternating Least Squares variant of Partial Least Squares.

Maximises the sum of cross-view covariances using simple power-iteration updates, without regularisation:

\[ \mathbf{w}_i \leftarrow \frac{X_i^\top \bar{\mathbf{s}}_{\neg i}} {\|X_i^\top \bar{\mathbf{s}}_{\neg i}\|_2} \]

where \(\bar{\mathbf{s}}_{\neg i}\) is the normalised sum of projected scores from all views except \(i\). This is the multiset generalisation of the NIPALS alternating power iteration.

References

Wold, H. (1975). Soft modelling by latent variables: the nonlinear iterative partial least squares (NIPALS) approach. Perspectives in Probability and Statistics, 117-142.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
max_iter int

Maximum ALS iterations per dimension. Default is 500.

500
tol float

Convergence tolerance. Default is 1e-6.

1e-06
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = PLS_ALS(latent_dimensions=2, random_state=0).fit([X1, X2])

Source code in cca_zoo/linear/_iterative.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(latent_dimensions=latent_dimensions, center=center)
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

SCCA_PMD

SCCA_PMD(
    latent_dimensions: int = 1,
    center: bool = True,
    tau: float | list[float] = 1.0,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: _BaseIterative

Sparse CCA via Penalized Matrix Decomposition.

Maximises the cross-view covariance subject to L1 norm constraints on each weight vector:

\[ \begin{aligned} \max_{\mathbf{w}_1, \mathbf{w}_2} \mathbf{w}_1^\top X_1^\top X_2 \mathbf{w}_2 \\ \text{subject to } \|\mathbf{w}_i\|_1 \leq \tau_i \sqrt{p_i},\quad \|\mathbf{w}_i\|_2 = 1 \end{aligned} \]

The update for each view uses bisection to find the soft-threshold that satisfies the L1 constraint exactly.

References

Witten, D. M., Tibshirani, R., & Hastie, T. (2009). A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics, 10(3), 515–534.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
tau float | list[float]

L1 bound scaling factor(s) in (0, 1]. The actual L1 bound is tau * sqrt(n_features_i). Default is 1 (no sparsity).

1.0
max_iter int

Maximum ALS iterations. Default is 500.

500
tol float

Convergence tolerance. Default is 1e-6.

1e-06
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = SCCA_PMD(tau=0.5, random_state=0).fit([X1, X2])

Source code in cca_zoo/linear/_iterative.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    tau: float | list[float] = 1.0,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.tau = tau

fit

fit(views: list[ArrayLike], y: None = None) -> SCCA_PMD

Fit the SCCA_PMD model.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Name Type Description
self SCCA_PMD

Fitted estimator.

Raises:

Type Description
ValueError

If fewer than 2 views are provided.

Source code in cca_zoo/linear/_iterative.py
def fit(self, views: list[ArrayLike], y: None = None) -> SCCA_PMD:
    """Fit the SCCA_PMD model.

    Args:
        views: List of arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If fewer than 2 views are provided.
    """
    # Store processed tau for use in _update_weight
    self._tau: list[float] = []  # set in super().fit via _setup_fit
    super().fit(views, y)
    return self

SCCA_ADMM

SCCA_ADMM(
    latent_dimensions: int = 1,
    center: bool = True,
    tau: float | list[float] = 0.1,
    mu: float = 1.0,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: _BaseIterative

Sparse CCA via Alternating Direction Method of Multipliers.

Solves the sparse CCA problem using ADMM to enforce both the L1 sparsity constraint on weight vectors and the unit-norm constraint on the projected scores simultaneously. For view \(i\), each outer iteration performs:

\[ \begin{aligned} \mathbf{w}_i &\leftarrow \mathbf{w}_i - \gamma_i \Bigl( X_i^\top X_i \mathbf{w}_i - X_i^\top \bar{\mathbf{s}}_{\neg i} + \mu (\mathbf{w}_i - \mathbf{z}_i + \mathbf{u}_i) \Bigr) \\ \mathbf{z}_i &\leftarrow \Pi_{\|\cdot\|_2 \le 1}\Bigl( \mathcal{S}_{\tau_i / \mu}(\mathbf{w}_i + \mathbf{u}_i) \Bigr) \\ \mathbf{u}_i &\leftarrow \mathbf{u}_i + \mathbf{w}_i - \mathbf{z}_i \end{aligned} \]

where \(\bar{\mathbf{s}}_{\neg i}\) is the summed projected score from all other views, \(\mathcal{S}_\lambda\) is the elementwise soft-threshold operator, \(\Pi_{\|\cdot\|_2 \le 1}\) projects onto the unit ball, \(\mathbf{u}_i\) is the scaled dual variable, and \(\gamma_i = \bigl(\|X_i^\top X_i\| / n + \mu\bigr)^{-1}\).

References

Suo, X., Mineiro, P., & Anandkumar, A. (2017). Sparse canonical correlation analysis. arXiv:1705.10865.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
tau float | list[float]

L1 regularisation weight(s). Default is 0.1.

0.1
mu float

ADMM penalty parameter (step size). Default is 1.0.

1.0
max_iter int

Maximum outer iterations. Default is 500.

500
tol float

Convergence tolerance. Default is 1e-6.

1e-06
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = SCCA_ADMM(tau=0.1, random_state=0).fit([X1, X2])

Source code in cca_zoo/linear/_iterative.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    tau: float | list[float] = 0.1,
    mu: float = 1.0,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.tau = tau
    self.mu = mu

SCCA_IPLS

SCCA_IPLS(
    latent_dimensions: int = 1,
    center: bool = True,
    alpha: float | list[float] = 0.0,
    l1_ratio: float | list[float] = 1.0,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: _BaseIterative

Iterative PLS with elastic net penalty on weight vectors.

Alternates between penalised regression sub-problems. For view \(i\):

\[ \hat{\mathbf{w}}_i = \arg\min_{\mathbf{w}} \frac{1}{2n} \|X_i \mathbf{w} - \bar{\mathbf{s}}_{\neg i}\|_2^2 + \alpha_i \Bigl( l_1 \|\mathbf{w}\|_1 + \tfrac{1-l_1}{2} \|\mathbf{w}\|_2^2 \Bigr) \]

followed by a normalisation step to enforce unit variance of the score.

References

Mai, Q., & Zhang, X. (2019). An iterative penalized least squares approach to sparse canonical correlation analysis. Biometrics, 75(3), 734–744.

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
alpha float | list[float]

Elastic net penalty strength(s). Default is 0.

0.0
l1_ratio float | list[float]

Ratio of L1 to total penalty. 1 = lasso, 0 = ridge. Default is 1.

1.0
max_iter int

Maximum ALS iterations. Default is 500.

500
tol float

Convergence tolerance. Default is 1e-6.

1e-06
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = SCCA_IPLS(alpha=0.1, random_state=0).fit([X1, X2])

Source code in cca_zoo/linear/_iterative.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    alpha: float | list[float] = 0.0,
    l1_ratio: float | list[float] = 1.0,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.alpha = alpha
    self.l1_ratio = l1_ratio

SCCA_Span

SCCA_Span(
    latent_dimensions: int = 1,
    center: bool = True,
    span: int | list[int] | None = None,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: _BaseIterative

SpanCCA — sparse CCA via truncated power iteration.

Solves sparse CCA by a sparse power iteration where each weight update retains only the span entries with the largest absolute values.

References

Asteris, M., Khanna, R., Kyrillidis, A., & Dimakis, A. G. (2016). Bilinear approaches for online learning over large feature spaces. NeurIPS 2016. (SpanCCA algorithm).

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
span int | list[int] | None

Number of non-zero entries to retain per view. Either a single int or a list. Default is None (keep all — no sparsity).

None
max_iter int

Maximum ALS iterations. Default is 500.

500
tol float

Convergence tolerance. Default is 1e-6.

1e-06
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = SCCA_Span(span=5, random_state=0).fit([X1, X2])

Source code in cca_zoo/linear/_iterative.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    span: int | list[int] | None = None,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.span = span

ElasticCCA

ElasticCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    alpha: float | list[float] = 0.0,
    l1_ratio: float | list[float] = 0.5,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: _BaseIterative

Elastic net regularised CCA.

Alternates between elastic net regression sub-problems, regressing each view's score against the sum of all other views' scores:

\[ \hat{\mathbf{w}}_i = \arg\min_{\mathbf{w}} \frac{1}{2n} \|X_i \mathbf{w} - \mathbf{s}_{\text{all}}\|_2^2 + \alpha_i \Bigl( l_1 \|\mathbf{w}\|_1 + \tfrac{1 - l_1}{2} \|\mathbf{w}\|_2^2 \Bigr) \]

where \(\mathbf{s}_{\text{all}} = \sum_j X_j \mathbf{w}_j / \|\cdot\|\).

References

Waaijenborg, S., de Witt Hamer, P. C. V., & Zwinderman, A. H. (2008). Quantifying the association between gene expressions and DNA-markers by penalized canonical correlation analysis. Statistical Applications in Genetics and Molecular Biology, 7(1).

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
alpha float | list[float]

Elastic net regularisation strength. Default is 0.

0.0
l1_ratio float | list[float]

L1 / total penalty ratio. Default is 0.5.

0.5
max_iter int

Maximum ALS iterations. Default is 500.

500
tol float

Convergence tolerance. Default is 1e-6.

1e-06
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = ElasticCCA(alpha=0.1, l1_ratio=0.5, random_state=0).fit([X1, X2])

Source code in cca_zoo/linear/_iterative.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    alpha: float | list[float] = 0.0,
    l1_ratio: float | list[float] = 0.5,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.alpha = alpha
    self.l1_ratio = l1_ratio

ParkhomenkoCCA

ParkhomenkoCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    tau: float | list[float] = 0.1,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: _BaseIterative

Sparse CCA via soft-thresholding power iteration (Parkhomenko 2009).

Uses a fixed soft-threshold \(\tau_i\) rather than the adaptive bisection search of :class:SCCA_PMD:

\[ \mathbf{w}_i \leftarrow S_{\tau_i}(X_i^\top \bar{\mathbf{s}}_{\neg i}) \]

where \(S_\tau\) is the element-wise soft-threshold operator.

References

Parkhomenko, E., Tritchler, D., & Beyene, J. (2009). Sparse canonical correlation analysis with application to genomic data integration. Statistical Applications in Genetics and Molecular Biology, 8(1).

Parameters:

Name Type Description Default
latent_dimensions int

Number of latent dimensions. Default is 1.

1
center bool

Whether to subtract column means. Default True.

True
tau float | list[float]

Soft-threshold parameter(s). Default is 0.1.

0.1
max_iter int

Maximum ALS iterations. Default is 500.

500
tol float

Convergence tolerance. Default is 1e-6.

1e-06
random_state int | None

Seed for reproducibility.

None
Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((50, 10)) X2 = rng.standard_normal((50, 8)) model = ParkhomenkoCCA(tau=0.1, random_state=0).fit([X1, X2])

Source code in cca_zoo/linear/_iterative.py
def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    tau: float | list[float] = 0.1,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(
        latent_dimensions=latent_dimensions,
        center=center,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.tau = tau