Skip to content

cca_zoo.linear

Linear CCA methods. All classes are sklearn.base.BaseEstimator subclasses. Sparse/regularised iterative methods live in cca_zoo.sparse; mini-batch methods live in cca_zoo.stochastic.


Base class

BaseModel

BaseModel(n_components: int = 1, center: bool = True)

Bases: BaseEstimator, ABC

Base class for multiview CCA models.

Subclasses implement :meth:fit, starting with :meth:_setup_fit and ending with :meth:_fit_maps_and_importances. A linear model sets weights_; a nonlinear model overrides :meth:_transform_view, its per-view encoder. transform, predict, inverse_transform, score and feature_importances_per_view_ are built on that encoder.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True

Attributes:

Name Type Description
means_

Per-view feature means subtracted before fitting.

n_features_per_view_

Number of features in each view.

n_components_

Number of latent dimensions fitted: n_components, or fewer where the model prunes dimensions or the data have too few, as sklearn's PCA.n_components_.

feature_names_per_view_

Each view's feature names, set when every view was fitted as a DataFrame with string column names.

n_samples_

Number of training samples.

n_views_

Number of views.

feature_importances_per_view_

Each feature's share of its view's embedding, one array per view. Non-negative and summing to 1 within each view (all zeros if a view's embedding uses no feature). Linear models use Var(x_j) * sum_k w_jk**2; GAMCCA the variance of each smooth, MARSCCA earth's evimp and the tree models split gain. Other models use the mean squared change in the view's scores when the feature is permuted, over at most 500 training rows, which equals twice the variance share for a linear or additive model.

fit abstractmethod

fit(views: list[ArrayLike], y: None = None) -> BaseModel

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
BaseModel

self.

transform

transform(views: list[ArrayLike]) -> list[Any]

Project each view into the latent space.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required

Returns:

Type Description
list[Any]

List of arrays, each of shape (n_samples, n_components), or of

list[Any]

DataFrames under :meth:set_output.

inverse_transform

inverse_transform(
    scores: list[ArrayLike],
) -> list[np.ndarray]

Map each view's latent scores back to that view's feature space.

Each view is reconstructed from its own scores by a least-squares regression fitted on the training data, as in :meth:sklearn.decomposition.PCA.inverse_transform. To reconstruct a view from the other views, use :meth:predict.

Parameters:

Name Type Description Default
scores list[ArrayLike]

One array of shape (n_samples, n_components) per view.

required

Returns:

Type Description
list[ndarray]

List of arrays, each of shape (n_samples, n_features_i).

Raises:

Type Description
ValueError

If scores has the wrong length or width.

predict

predict(views: list[ArrayLike | None]) -> list[np.ndarray]

Reconstruct every view from the views that are observed.

The shared latent scores are estimated from the observed views (their mean, or the posterior mean for probabilistic models) and mapped to each view by a least-squares regression fitted on the training data. Pass None for a view to reconstruct.

Parameters:

Name Type Description Default
views list[ArrayLike | None]

One entry per view: an array of shape (n_samples, n_features_i) or None.

required

Returns:

Type Description
list[ndarray]

List of arrays, each of shape (n_samples, n_features_i).

Raises:

Type Description
ValueError

If views has the wrong length, no view is observed, or the observed views have inconsistent shapes.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import CCA
>>> rng = np.random.default_rng(0)
>>> X1, X2 = rng.standard_normal((50, 10)), rng.standard_normal((50, 8))
>>> model = CCA(n_components=2).fit([X1, X2])
>>> model.predict([X1, None])[1].shape
(50, 8)

fit_transform

fit_transform(
    views: list[ArrayLike], y: None = None
) -> list[np.ndarray]

Fit the model and transform the training views.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Type Description
list[ndarray]

List of arrays, each of shape (n_samples, n_components).

score

score(views: list[ArrayLike], y: None = None) -> float

Mean canonical correlation between the views' latent scores.

The average over latent dimensions of the mean pairwise correlation. Per-dimension values: average_pairwise_correlations(pairwise_correlations(model.transform(views))) from :mod:cca_zoo.metrics.

Parameters:

Name Type Description Default
views list[ArrayLike]

List of arrays, each of shape (n_samples, n_features_i).

required
y None

Ignored.

None

Returns:

Type Description
float

The mean canonical correlation.

get_feature_names_out

get_feature_names_out(
    input_features: list[list[str]] | None = None,
) -> list[np.ndarray]

Names of each view's latent dimensions, <model><k> as in sklearn.

Parameters:

Name Type Description Default
input_features list[list[str]] | None

Ignored, unless given: then each view's must match the names seen in fit.

None

Returns:

Type Description
list[ndarray]

One array of names per view, of length n_components.

Raises:

Type Description
ValueError

If input_features differ from the fitted names.

set_output

set_output(*, transform: str | None = None) -> _Model

Set the container :meth:transform returns, as sklearn's set_output.

Parameters:

Name Type Description Default
transform str | None

"pandas" or "polars" for one DataFrame per view, indexed as the input view when it is a pandas DataFrame; "default" for sklearn's global transform_output configuration; None leaves the setting unchanged.

None

Returns:

Type Description
_Model

self.


Two-view exact methods

CCA

CCA(n_components: int = 1, *, center: bool = True)

Bases: RidgeCCA

Canonical correlation analysis of two views.

\[ \max_{w_1, w_2} w_1^\top X_1^\top X_2 w_2 \quad \text{subject to} \quad w_i^\top X_i^\top X_i w_i = 1. \]

:class:RidgeCCA with shrinkage=0.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

References

Hotelling, H. (1936). Relations between two sets of variates. Biometrika, 28(3/4), 321-377.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import CCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> Z1, Z2 = CCA(n_components=2).fit_transform([X1, X2])
Source code in cca_zoo/linear/_cca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=0.0,
    )

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> CCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
sample_weight ArrayLike | None

Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally.

None

Returns:

Type Description
CCA

self.

Raises:

Type Description
ValueError

If there are not exactly two views.

Source code in cca_zoo/linear/_cca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> CCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        sample_weight: Weight of each sample; an integer weight is the
            same as repeating the sample. None weights samples equally.

    Returns:
        self.

    Raises:
        ValueError: If there are not exactly two views.
    """
    return super().fit(views, y, sample_weight)

RidgeCCA

RidgeCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
)

Bases: BaseModel

Ridge-regularised CCA of two views (canonical ridge).

\[ \max_{w_1, w_2} w_1^\top X_1^\top X_2 w_2 \quad \text{subject to} \quad w_i^\top \bigl((1 - c_i) X_i^\top X_i + c_i I\bigr) w_i = 1, \]

solved by whitening each view and taking the SVD of the cross-covariance. shrinkage=0 is :class:CCA and shrinkage=1 is :class:PLS.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float | list[float]

Shrinkage of each view's covariance towards the identity, in [0, 1]: 0 is CCA and 1 is PLS. Per-view. Default is 0.

0.0

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

References

Vinod, H. D. (1976). Canonical ridge and econometrics of joint production. Journal of Econometrics, 4(2), 147-166.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import RidgeCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> model = RidgeCCA(n_components=2, shrinkage=[0.1, 0.5]).fit([X1, X2])
Source code in cca_zoo/linear/_ridge_cca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> RidgeCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
sample_weight ArrayLike | None

Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally.

None

Returns:

Type Description
RidgeCCA

self.

Raises:

Type Description
ValueError

If there are not exactly two views.

Source code in cca_zoo/linear/_ridge_cca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> RidgeCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        sample_weight: Weight of each sample; an integer weight is the
            same as repeating the sample. None weights samples equally.

    Returns:
        self.

    Raises:
        ValueError: If there are not exactly two views.
    """
    views_: list[np.ndarray] = self._setup_fit(views, sample_weight)
    if self.n_views_ != 2:
        raise ValueError(
            f"{type(self).__name__} requires exactly 2 views, got "
            f"{self.n_views_}. Use MCCA for more than 2 views."
        )
    c_ = perview_parameter("shrinkage", self.shrinkage, 0.0, 2)
    xp, _ = get_namespace(*views_)
    X1, X2 = views_
    # Whiten each view with its regularised covariance, then take the SVD
    # of the whitened views' cross-covariance.
    X1_w, W1 = svd_whiten(X1, c_[0])
    X2_w, W2 = svd_whiten(X2, c_[1])
    k = min(self.n_components, X1_w.shape[1], X2_w.shape[1])
    U, _, Vt = xp.linalg.svd(X1_w.T @ X2_w / (X1.shape[0] - 1), full_matrices=False)
    self.weights_: list[Any] = [W1 @ U[:, :k], W2 @ Vt[:k, :].T]
    self._fit_maps_and_importances(views_)
    return self

PLS

PLS(n_components: int = 1, *, center: bool = True)

Bases: RidgeCCA

Partial least squares of two views.

\[ \max_{w_1, w_2} w_1^\top X_1^\top X_2 w_2 \quad \text{subject to} \quad \|w_i\|_2 = 1, \]

the truncated SVD of the cross-covariance; :class:RidgeCCA with shrinkage=1.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

References

Wold, H. (1975). Soft modelling by latent variables: the nonlinear iterative partial least squares (NIPALS) approach. Perspectives in Probability and Statistics, 117-142.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import PLS
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> Z1, Z2 = PLS(n_components=2).fit_transform([X1, X2])
Source code in cca_zoo/linear/_pls.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=1.0,
    )

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> PLS

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
sample_weight ArrayLike | None

Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally.

None

Returns:

Type Description
PLS

self.

Raises:

Type Description
ValueError

If there are not exactly two views.

Source code in cca_zoo/linear/_pls.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> PLS:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        sample_weight: Weight of each sample; an integer weight is the
            same as repeating the sample. None weights samples equally.

    Returns:
        self.

    Raises:
        ValueError: If there are not exactly two views.
    """
    return super().fit(views, y, sample_weight)

Multiview methods

MCCA

MCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    pca: bool = True,
)

Bases: BaseModel

Multiset CCA: the sum of pairwise covariances under ridge constraints.

\[ \max_{w} \sum_{i \neq j} w_i^\top X_i^\top X_j w_j \quad \text{subject to} \quad w_i^\top \bigl((1 - c_i) X_i^\top X_i + c_i I\bigr) w_i = 1, \]

solved as the generalized eigenproblem \(A v = \lambda B v\), with \(A\) the between-view and \(B\) the regularised within-view covariance blocks. shrinkage=0 is CCA and shrinkage=1 is PLS.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float | list[float]

Shrinkage of each view's covariance towards the identity, in [0, 1]: 0 is CCA and 1 is PLS. Per-view. Default is 0.

0.0
pca bool

Whether to solve in each view's principal components, which is faster and more stable for wide data. Default is True.

True

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

References

Kettenring, J. R. (1971). Canonical analysis of several sets of variables. Biometrika, 58(3), 433-451.

Vinod, H. D. (1976). Canonical ridge and econometrics of joint production. Journal of Econometrics, 4(2), 147-166.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import MCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> X3 = rng.standard_normal((50, 6))
>>> model = MCCA(n_components=2, shrinkage=0.1).fit([X1, X2, X3])
Source code in cca_zoo/linear/_mcca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    pca: bool = True,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.pca = pca

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> MCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
sample_weight ArrayLike | None

Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally.

None

Returns:

Type Description
MCCA

self.

Source code in cca_zoo/linear/_mcca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> MCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        sample_weight: Weight of each sample; an integer weight is the
            same as repeating the sample. None weights samples equally.

    Returns:
        self.
    """
    views_ = self._setup_fit(views, sample_weight)
    c_ = perview_parameter("shrinkage", self.shrinkage, 0.0, self.n_views_)

    if self.pca:
        pca_models = [PCA().fit(v) for v in views_]
        solved_views = [m.transform(v) for m, v in zip(pca_models, views_)]
        A = self._build_A(solved_views)
        B = self._build_B_pca(pca_models, c_)
    else:
        solved_views = views_
        A = self._build_A(views_)
        B = self._build_B(views_, c_)
    _, eigvecs = gevp(A, B, self.n_components)

    edges = list(accumulate((v.shape[1] for v in solved_views), initial=0))
    weights = [eigvecs[a:b, :] for a, b in pairwise(edges)]
    if self.pca:
        weights = [m.components_.T @ w for m, w in zip(pca_models, weights)]
    self.weights_: list[Any] = weights
    self._fit_maps_and_importances(views_)
    return self

GCCA

GCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    view_weights: list[float] | None = None,
)

Bases: BaseModel

Generalized CCA: views correlated with a shared latent variable.

\[ \min_{T,\, w_i} \sum_i \mu_i \Bigl( w_i^\top C_i w_i - \tfrac{2}{n - 1} w_i^\top X_i^\top T \Bigr) \quad \text{subject to} \quad \tfrac{1}{n - 1} T^\top T = I, \]

with \(C_i = (1 - c_i) \Sigma_{ii} + c_i I\): at \(c_i = 0\), the squared error of regressing \(T\) on each view. Each view's weights are its regularised regression onto \(T\), \(w_i = C_i^{-1} X_i^\top T / (n - 1)\), and \(T\) holds the top eigenvectors of \(\sum_i \mu_i X_i C_i^{-1} X_i^\top\), found from the SVD of the stacked whitened views.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float | list[float]

Shrinkage of each view's covariance towards the identity, in [0, 1]: 0 is CCA and 1 is PLS. Per-view. Default is 0.

0.0
view_weights list[float] | None

Weight \(\mu_i\) of each view; None weights them equally. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

References

Carroll, J. D. (1968). Generalization of canonical correlation analysis to three or more sets of variables. Proceedings of the 76th Annual Convention of the American Psychological Association, 3, 227-228. Kettenring, J. R. (1971). Canonical analysis of several sets of variables. Biometrika, 58(3), 433-451.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import GCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> X3 = rng.standard_normal((50, 6))
>>> model = GCCA(n_components=2).fit([X1, X2, X3])
Source code in cca_zoo/linear/_gcca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    view_weights: list[float] | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.view_weights = view_weights

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> GCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
sample_weight ArrayLike | None

Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally.

None

Returns:

Type Description
GCCA

self.

Source code in cca_zoo/linear/_gcca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> GCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        sample_weight: Weight of each sample; an integer weight is the
            same as repeating the sample. None weights samples equally.

    Returns:
        self.
    """
    views_ = self._setup_fit(views, sample_weight)
    xp, _ = get_namespace(*views_)
    c_ = perview_parameter("shrinkage", self.shrinkage, 0.0, self.n_views_)
    mu = perview_parameter("view_weights", self.view_weights, 1.0, self.n_views_)

    # Q = sum_i mu_i X_i cov_i^{-1} X_i^T is H H^T for the stacked
    # whitened views H = [sqrt(mu_i) X_i cov_i^{-1/2}], so its top
    # eigenvectors are H's top left singular vectors: an n x sum(p_i)
    # SVD in place of an n x n eigenproblem.
    whiteners = []
    for v, ci in zip(views_, c_):
        identity = xp.eye(v.shape[1], dtype=v.dtype, device=device(v))
        cov = (1.0 - ci) * covariance(v) + ci * identity
        whiteners.append(psd_inverse_sqrt(cov, self._EPS))
    whitened = [math.sqrt(mi) * v @ r for v, r, mi in zip(views_, whiteners, mu)]
    U = xp.linalg.svd(xp.concat(whitened, axis=1), full_matrices=False)[0]
    # Unit-variance shared latent, so the scores' scale does not depend on
    # the number of samples.
    T = U[:, : self.n_components] * math.sqrt(self.n_samples_ - 1)
    # Each view's weights are its regularised regression onto T,
    # cov_i^{-1} X_i' T / (n - 1): least squares at shrinkage 0.
    self.weights_: list[Any] = [
        r @ (r @ (v.T @ T)) / (self.n_samples_ - 1)
        for v, r in zip(views_, whiteners)
    ]
    self._fit_maps_and_importances(views_)
    return self

TCCA

TCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    init: str = "svd",
    random_state: int | RandomState | None = None,
)

Bases: BaseModel

Tensor CCA: higher-order correlation of three or more views.

Decomposes the cross-moment tensor of the whitened views \(\tilde X_i = X_i \Sigma_i^{-1/2}\),

\[ \mathcal{M} = \frac{1}{n} \sum_s \tilde x_{1s} \otimes \cdots \otimes \tilde x_{Ms}, \]

by PARAFAC; the factors give the canonical directions.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float | list[float]

Shrinkage of each view's covariance towards the identity, in [0, 1]: 0 is CCA and 1 is PLS. Per-view. Default is 0.

0.0
init str

PARAFAC's initialisation: "svd", deterministic, or "random", seeded by random_state; random starts show whether the SVD start found the best of the tensor's local optima. Default is "svd".

'svd'
random_state int | RandomState | None

Seed for init="random", as sklearn's NMF uses it. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

References

Luo, Y., Tao, D., Ramamohanarao, K., Xu, C., & Wen, Y. (2015). Tensor canonical correlation analysis for multi-view dimension reduction. IEEE Transactions on Knowledge and Data Engineering, 27(11), 3111-3124.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import TCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 5))
>>> X2 = rng.standard_normal((50, 5))
>>> X3 = rng.standard_normal((50, 5))
>>> model = TCCA(n_components=2, random_state=0).fit([X1, X2, X3])
Source code in cca_zoo/linear/_tcca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    init: str = "svd",
    random_state: int | np.random.RandomState | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.init = init
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> TCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
TCCA

self.

Source code in cca_zoo/linear/_tcca.py
def fit(self, views: list[ArrayLike], y: None = None) -> TCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    c_ = perview_parameter("shrinkage", self.shrinkage, 0.0, self.n_views_)
    whitened, cov_invsqrt = self._whiten_views(views_, c_)

    M = cross_moment_tensor(whitened)

    # Imported here: importing tensorly prints SyntaxWarnings on Python
    # 3.12, which every user of cca_zoo would otherwise see.
    import tensorly as tl
    from tensorly.decomposition import parafac

    with tl.backend_context("numpy"):
        parafac_result = parafac(
            M,
            self.n_components,
            init=self.init,
            random_state=self.random_state,
            verbose=False,
        )
    self.weights_: list[np.ndarray] = [
        cov_invsqrt[i] @ fac for i, fac in enumerate(parafac_result.factors)
    ]
    self._fit_maps_and_importances(views_)
    return self

Confound-adjusted / structured methods

PartialCCA

PartialCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
)

Bases: MCCA

CCA of the views after regressing out confounds.

\[ \max_{w} w_1^\top X_1^\top X_2 w_2 \quad \text{subject to} \quad w_i^\top X_i^\top X_i w_i = 1, \quad w_i^\top X_i^\top Z = 0, \]

for confounds \(Z\) (partials). The constraints leave MCCA's eigenproblem on the views' partial covariance given the confounds,

\[ \Sigma_{XX|Z} = \Sigma_{XX} - \Sigma_{XZ} \Sigma_{ZZ}^{-1} \Sigma_{ZX}, \]

whose off-diagonal blocks form \(A\) and whose diagonal blocks, with shrinkage, form \(B\), as :class:MCCA forms them from the covariance.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float | list[float]

Shrinkage of each view's covariance towards the identity, in [0, 1]: 0 is CCA and 1 is PLS. Per-view. Default is 0.

0.0

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

confound_betas_

Regression of each view on the centred confounds, shape (n_confounds, n_features_i).

partials_mean_

Mean of the training confounds, shape (n_confounds,).

References

Rao, B. R. (1969). Partial canonical correlations. Trabajos de Estadistica y de Investigacion Operativa, 20(2-3), 211-219.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import PartialCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> Z = rng.standard_normal((50, 3))
>>> model = PartialCCA(n_components=2).fit([X1, X2], partials=Z)
>>> Z1, Z2 = model.transform([X1, X2], partials=Z)
Source code in cca_zoo/linear/_partialcca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=shrinkage,
        pca=False,
    )

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    partials: ArrayLike | None = None,
) -> PartialCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
partials ArrayLike | None

Confounds, shape (n_samples, n_confounds).

None

Returns:

Type Description
PartialCCA

self.

Raises:

Type Description
ValueError

If partials is not given.

Source code in cca_zoo/linear/_partialcca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    partials: ArrayLike | None = None,
) -> PartialCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        partials: Confounds, shape (n_samples, n_confounds).

    Returns:
        self.

    Raises:
        ValueError: If ``partials`` is not given.
    """
    if partials is None:
        raise ValueError("PartialCCA requires `partials` to be provided to fit().")
    views_ = self._setup_fit(views)
    self.partials_mean_: np.ndarray = np.asarray(partials, dtype=float).mean(axis=0)
    z = np.asarray(partials, dtype=float) - self.partials_mean_
    x = np.hstack(views_)
    x = x - x.mean(axis=0)
    betas = np.linalg.pinv(z.T @ z) @ (z.T @ x)
    # The views' partial covariance given the confounds.
    partial = (x.T @ x - (x.T @ z) @ betas) / (len(x) - 1)
    edges = np.cumsum([0, *self.n_features_per_view_])
    within = [partial[a:b, a:b] for a, b in pairwise(edges)]
    c_ = perview_parameter("shrinkage", self.shrinkage, 0.0, self.n_views_)
    A = (partial - block_diag(within)) / self.n_views_
    B = self._floored_blocks(
        [(1.0 - ci) * w + ci * np.eye(len(w)) for w, ci in zip(within, c_)]
    )
    _, eigvecs = gevp(A, B, self.n_components)
    self.weights_: list[np.ndarray] = np.split(eigvecs, edges[1:-1], axis=0)
    self.confound_betas_: list[np.ndarray] = np.split(betas, edges[1:-1], axis=1)
    self._fit_maps_and_importances(views_)
    return self

transform

transform(
    views: list[ArrayLike],
    partials: ArrayLike | None = None,
) -> list[np.ndarray]

Project views into the latent space, regressing out confounds if given.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
partials ArrayLike | None

Confounds, shape (n_samples, n_confounds); None projects the views unadjusted. Default is None.

None

Returns:

Type Description
list[ndarray]

One array of shape (n_samples, n_components) per view.

Source code in cca_zoo/linear/_partialcca.py
def transform(
    self,
    views: list[ArrayLike],
    partials: ArrayLike | None = None,
) -> list[np.ndarray]:
    """Project views into the latent space, regressing out confounds if given.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        partials: Confounds, shape (n_samples, n_confounds); None projects
            the views unadjusted. Default is None.

    Returns:
        One array of shape (n_samples, n_components) per view.
    """
    if partials is None:
        return super().transform(views)
    validated = self._check_views(views)
    z = np.asarray(partials, dtype=float) - self.partials_mean_
    centred = [v - m for v, m in zip(validated, self.means_)]
    deconfounded = [v - z @ beta for v, beta in zip(centred, self.confound_betas_)]
    return [v @ w for v, w in zip(deconfounded, self.weights_)]

fit_transform

fit_transform(
    views: list[ArrayLike],
    y: None = None,
    partials: ArrayLike | None = None,
) -> list[np.ndarray]

Fit, then transform the training views.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
partials ArrayLike | None

Confounds, shape (n_samples, n_confounds).

None

Returns:

Type Description
list[ndarray]

One array of shape (n_samples, n_components) per view.

Source code in cca_zoo/linear/_partialcca.py
def fit_transform(
    self,
    views: list[ArrayLike],
    y: None = None,
    partials: ArrayLike | None = None,
) -> list[np.ndarray]:
    """Fit, then transform the training views.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        partials: Confounds, shape (n_samples, n_confounds).

    Returns:
        One array of shape (n_samples, n_components) per view.
    """
    return self.fit(views, y=y, partials=partials).transform(
        views, partials=partials
    )

GRCCA

GRCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    mu: float | list[float] = 0.0,
    feature_groups: list[ndarray] | None = None,
)

Bases: MCCA

MCCA with ridge penalties that shrink weights towards their group means.

Tuzhilina et al.'s penalty on each view's weights \(w\), for features in groups \(g\) of size \(p_g\) with mean weight \(\bar w_g\),

\[ \sum_g \sum_{j \in g} (w_j - \bar w_g)^2 + \mu \sum_g p_g \bar w_g^2 = w^\top \bigl((I - H) + \mu H\bigr) w, \]

with \(H\) the projection averaging each group, takes the place of :class:MCCA's identity: each view's block of \(B\) is \((1 - c)\Sigma_{ii} + c\,((I - H) + \mu H)\). shrinkage \(c\) sets the strength of the penalty and mu that of the group means relative to the deviations from them: mu=1 is MCCA's ridge, and mu=0 shrinks each weight towards its group's mean alone. Their \(\lambda\) and \(\mu\) are \(c / (1 - c)\) and \(\mu c / (1 - c)\).

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float | list[float]

Strength of the penalty, in [0, 1]: 0 is CCA. Per-view. Default is 0.

0.0
mu float | list[float]

Penalty on the group means, relative to the within-group deviations. Per-view. Default is 0.

0.0
feature_groups list[ndarray] | None

Integer group label of each feature, shape (n_features_i,) per view; None puts each view in one group. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

feature_groups_

Group label of each feature, per view.

Raises:

Type Description
ValueError

If a view's feature_groups does not label each feature.

References

Tuzhilina, E., Tozzi, L., & Hastie, T. (2023). Canonical correlation analysis in high dimensions with structured regularization. Statistical Modelling, 23(3), 203-227.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import GRCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> groups = [rng.integers(0, 3, size=10), rng.integers(0, 3, size=8)]
>>> model = GRCCA(n_components=2, shrinkage=0.5, feature_groups=groups)
>>> model = model.fit([X1, X2])
Source code in cca_zoo/linear/_grcca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    mu: float | list[float] = 0.0,
    feature_groups: list[np.ndarray] | None = None,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=shrinkage,
        pca=False,
    )
    self.mu = mu
    self.feature_groups = feature_groups

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> GRCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
sample_weight ArrayLike | None

Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally.

None

Returns:

Type Description
GRCCA

self.

Source code in cca_zoo/linear/_grcca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> GRCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        sample_weight: Weight of each sample; an integer weight is the
            same as repeating the sample. None weights samples equally.

    Returns:
        self.
    """
    return cast(GRCCA, super().fit(views, y, sample_weight))

Reduced-rank regression methods

CCAR3

CCAR3(
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float = 0.0,
    ledoit_wolf: bool = True,
    max_iter: int = 10000,
    tol: float = 0.0001,
)

Bases: BaseModel

Two-view CCA by row-sparse reduced rank regression.

Whitens \(Y\) by its (optionally Ledoit-Wolf) covariance, regresses \(\tilde{Y} = Y \Sigma_Y^{-1/2}\) on \(X\), and takes the canonical directions from the rank-n_components SVD of the fitted values. The regression has a row-group lasso penalty,

\[ \hat{B} = \underset{B}{\mathrm{argmin}}\ \frac{1}{n} \lVert \tilde{Y} - X B \rVert_F^2 + \alpha \sum_j \lVert B_{j, :} \rVert_2, \]

which drops whole features of \(X\); swap the views to make the other one sparse. alpha=0 is solved by least squares. A port of the R package ccar3.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
alpha float

Row-group lasso strength. Default is 0.

0.0
ledoit_wolf bool

Whether to shrink the covariance of Y. Default is True.

True
max_iter int

Maximum MultiTaskLasso iterations. Default is 10000.

10000
tol float

MultiTaskLasso tolerance. Default is 1e-4.

0.0001

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

n_iter_

Iterations of the MultiTaskLasso; 0 when alpha=0 is solved by least squares.

References

Donnat, C., & Tuzhilina, E. (2024). Canonical Correlation Analysis as Reduced Rank Regression in High Dimensions. arXiv:2405.19539.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import CCAR3
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> model = CCAR3(n_components=2, alpha=0.1).fit([X1, X2])
Source code in cca_zoo/linear/_ccar3.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float = 0.0,
    ledoit_wolf: bool = True,
    max_iter: int = 10_000,
    tol: float = 1e-4,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.alpha = alpha
    self.ledoit_wolf = ledoit_wolf
    self.max_iter = max_iter
    self.tol = tol

fit

fit(views: list[ArrayLike], y: None = None) -> CCAR3

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
CCAR3

self.

Raises:

Type Description
ValueError

If there are not exactly two views.

Source code in cca_zoo/linear/_ccar3.py
def fit(self, views: list[ArrayLike], y: None = None) -> CCAR3:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.

    Raises:
        ValueError: If there are not exactly two views.
    """
    views_ = self._setup_fit(views)
    if self.n_views_ != 2:
        raise ValueError(
            f"{type(self).__name__} requires exactly 2 views, got "
            f"{self.n_views_}. Use MCCA for more than 2 views."
        )
    X, Y = views_
    n = len(X)
    # Regress the whitened response on X.
    cov_y = LedoitWolf().fit(Y).covariance_ if self.ledoit_wolf else Y.T @ Y / n
    whitener = _inverse_sqrt(cov_y)
    if self.alpha == 0.0:
        B = np.linalg.lstsq(X, Y @ whitener, rcond=None)[0]
        self.n_iter_: int = 0
    else:
        B, self.n_iter_ = self._regression(X, Y @ whitener)
    # The canonical directions are the right singular vectors of the
    # fitted values X B, not of B, whose singular vectors ignore the
    # covariance of X.
    k = min(self.n_components, *B.shape)
    q = np.linalg.svd(X @ B, full_matrices=False)[2][:k].T
    u, v = B @ q, whitener @ q
    # Whitening each side's scores leaves the pairs mixed; the SVD of the
    # whitened cross-covariance gives the canonical pairs, in order.
    u = u @ _whitener_of_scores(X @ u)
    v = v @ _whitener_of_scores(Y @ v)
    left, _, right_t = np.linalg.svd((X @ u).T @ (Y @ v) / n)
    self.weights_: list[np.ndarray] = [
        np.zeros((X.shape[1], self.n_components)),
        np.zeros((Y.shape[1], self.n_components)),
    ]
    if np.any(B):
        self.weights_[0][:, :k] = u @ left
        self.weights_[1][:, :k] = v @ right_t.T
    self._fit_maps_and_importances(views_)
    return self

ECCA

ECCA(
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float = 0.0,
    ledoit_wolf: bool = False,
    max_iter: int = 10000,
    tol: float = 0.0001,
)

Bases: CCAR3

Two-view CCA by entrywise-sparse reduced rank regression.

:class:~cca_zoo.linear.CCAR3 with an entrywise lasso penalty on the regression of the whitened \(\tilde{Y} = Y \Sigma_Y^{-1/2}\) on \(X\),

\[ \hat{B} = \underset{B}{\mathrm{argmin}}\ \frac{1}{n} \lVert \tilde{Y} - X B \rVert_F^2 + \alpha \sum_{j,k} \lvert B_{jk} \rvert, \]

and takes the canonical directions from the rank-n_components SVD of the fitted values \(X \hat{B}\). The penalty zeroes entries of \(\hat{B}\), where :class:~cca_zoo.linear.CCAR3's zeroes whole rows; the canonical directions combine its columns, so a feature leaves them only when its whole row is zero. A port of ecca() from the R package ccar3.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
alpha float

Entrywise lasso strength. Default is 0.

0.0
ledoit_wolf bool

Whether to shrink the covariance of Y. Default is False, as the R package's ecca().

False
max_iter int

Maximum Lasso iterations. Default is 10000.

10000
tol float

Lasso tolerance. Default is 1e-4.

0.0001

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

n_iter_

Most iterations of any column's Lasso; 0 when alpha=0 is solved by least squares.

References

Donnat, C., & Tuzhilina, E. (2024). Canonical Correlation Analysis as Reduced Rank Regression in High Dimensions. arXiv:2405.19539.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import ECCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> model = ECCA(n_components=2, alpha=0.1).fit([X1, X2])
Source code in cca_zoo/linear/_ecca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float = 0.0,
    ledoit_wolf: bool = False,
    max_iter: int = 10_000,
    tol: float = 1e-4,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        alpha=alpha,
        ledoit_wolf=ledoit_wolf,
        max_iter=max_iter,
        tol=tol,
    )

Sparse-precision covariance methods

GraphicalLassoCCA

GraphicalLassoCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    alpha: float | list[float | None] | None = 0.01,
    max_iter: int = 1000,
    tol: float = 1e-08,
)

Bases: MCCA

MCCA with each within-view covariance estimated by the graphical lasso.

Replaces each view's block of :class:~cca_zoo.linear.MCCA's \(B\) with the covariance implied by a sparse precision estimate, an L1 penalty on partial correlations rather than shrinkage of the covariance. The lasso is fitted to each view's correlation matrix, so alpha is the same in any units, and solved by :func:graphical_lasso, ADMM: sklearn's coordinate descent fails to converge, or to run, on strongly correlated features. Solved in the original feature space.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float | list[float]

Shrinkage of each view's estimated covariance towards the identity, in [0, 1], as in MCCA. Default is 0.

0.0
alpha float | list[float | None] | None

Graphical-lasso penalty on the correlation scale; None selects it by :class:~sklearn.covariance.GraphicalLassoCV. Per-view. With None, the refit at the selected penalty is by ADMM too. Default is 0.01.

0.01
max_iter int

Maximum ADMM iterations per view. Default is 1000.

1000
tol float

Tolerance on ADMM's residuals, per entry. Default is 1e-8.

1e-08

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

covariance_

Estimated covariance of each view.

precision_

Estimated sparse precision of each view.

n_iter_

Most graphical-lasso iterations of any view.

References

Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3), 432-441.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import GraphicalLassoCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((100, 12))
>>> X2 = rng.standard_normal((100, 9))
>>> model = GraphicalLassoCCA(alpha=0.1).fit([X1, X2])
>>> model.precision_[0].shape
(12, 12)
Source code in cca_zoo/linear/_graphical_lasso_cca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.0,
    alpha: float | list[float | None] | None = 0.01,
    max_iter: int = 1000,
    tol: float = 1e-8,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=shrinkage,
        pca=False,
    )
    self.alpha = alpha
    self.max_iter = max_iter
    self.tol = tol

fit

fit(
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> GraphicalLassoCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None
sample_weight ArrayLike | None

Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally.

None

Returns:

Type Description
GraphicalLassoCCA

self.

Raises:

Type Description
ValueError

If sample_weight is given with alpha=None: cross-validating the penalty splits the rows themselves.

Source code in cca_zoo/linear/_graphical_lasso_cca.py
def fit(
    self,
    views: list[ArrayLike],
    y: None = None,
    sample_weight: ArrayLike | None = None,
) -> GraphicalLassoCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.
        sample_weight: Weight of each sample; an integer weight is the
            same as repeating the sample. None weights samples equally.

    Returns:
        self.

    Raises:
        ValueError: If ``sample_weight`` is given with ``alpha=None``:
            cross-validating the penalty splits the rows themselves.
    """
    alpha_ = perview_parameter("alpha", self.alpha, None, len(views))
    if sample_weight is not None and None in alpha_:
        raise ValueError(
            "alpha=None selects the penalty by cross-validating over the "
            "rows, which cannot use sample_weight; pass alpha."
        )
    return cast(GraphicalLassoCCA, super().fit(views, y, sample_weight))

Robust methods

RANSACCCA

RANSACCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    min_samples: int | float = 0.25,
    residual_threshold: float | None = None,
    max_trials: int = 200,
    stop_probability: float = 0.99,
    random_state: int | None = None,
)

Bases: BaseModel

Robust multiview CCA by random sample consensus.

As :class:~sklearn.linear_model.RANSACRegressor: fits :class:~cca_zoo.linear.MCCA on random subsets of min_samples rows, scores each candidate by the total positive cross-view agreement of every sample, and refits on the inliers of the best one. Targets samples whose cross-view relationship is wrong but whose magnitudes are ordinary, which leverage-based :class:~cca_zoo.linear.HuberCCA cannot see. The number of trials adapts to the inlier fraction found, up to max_trials.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float | list[float]

Shrinkage of every internal MCCA fit. Default is 0.1: each trial fits a quarter of the rows, often fewer than the features.

0.1
min_samples int | float

Subset size, as a fraction in (0, 1] or a count. Default is 0.25.

0.25
residual_threshold float | None

Minimum agreement of an inlier; None is 0. Default is None.

None
max_trials int

Maximum subsets tried. Default is 200.

200
stop_probability float

Stop once a subset at least as clean as the best is found with this probability. Default is 0.99.

0.99
random_state int | None

Seed for the subset draws. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

inlier_mask_

Boolean mask of the consensus set.

n_trials_

Number of subsets tried.

References

Chapman, J., Wang, H.-T., Wells, L., & Wiesner, J. (2021). CCA-Zoo: A collection of Regularized, Deep Learning based, Kernel, and Probabilistic CCA methods in a scikit-learn style framework. Journal of Open Source Software, 6(68), 3823.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import RANSACCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 8))
>>> X2 = rng.standard_normal((200, 6))
>>> model = RANSACCCA(random_state=0).fit([X1, X2])
Source code in cca_zoo/linear/_ransac_cca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    min_samples: int | float = 0.25,
    residual_threshold: float | None = None,
    max_trials: int = 200,
    stop_probability: float = 0.99,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.min_samples = min_samples
    self.residual_threshold = residual_threshold
    self.max_trials = max_trials
    self.stop_probability = stop_probability
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> RANSACCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
RANSACCCA

self.

Source code in cca_zoo/linear/_ransac_cca.py
def fit(self, views: list[ArrayLike], y: None = None) -> RANSACCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    n = self.n_samples_
    k = self.n_components
    min_samples = self._resolve_min_samples(n)
    threshold = 0.0 if self.residual_threshold is None else self.residual_threshold
    rng = np.random.default_rng(self.random_state)

    best_mask: np.ndarray | None = None
    best_score = -np.inf
    dynamic_max_trials = self.max_trials
    trial = 0
    while trial < dynamic_max_trials:
        idx = rng.choice(n, min_samples, replace=False)
        candidate = MCCA(n_components=k, shrinkage=self.shrinkage).fit(
            [v[idx] for v in views_]
        )
        agreement = _cross_view_agreement(
            candidate._transform_arrays(cast("list[ArrayLike]", views_))
        )
        score = float(np.clip(agreement, 0.0, None).sum())
        if score > best_score:
            best_score = score
            best_mask = agreement >= threshold
            w = max(float(best_mask.sum()) / n, 1e-10)
            denom = np.clip(1.0 - w**min_samples, 1e-12, 1 - 1e-12)
            dynamic_max_trials = min(
                self.max_trials,
                int(np.ceil(np.log(1 - self.stop_probability) / np.log(denom))),
            )
        trial += 1

    assert best_mask is not None
    final = MCCA(n_components=k, shrinkage=self.shrinkage).fit(
        [v[best_mask] for v in views_]
    )
    self.weights_: list[np.ndarray] = final.weights_
    self.inlier_mask_: np.ndarray = best_mask
    self.n_trials_: int = trial
    self._fit_maps_and_importances(views_)
    return self

TrimmedCCA

TrimmedCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float = 0.1,
    support_fraction: float = 0.75,
    n_init: int = 10,
    max_iter: int = 30,
    tol: float = 1e-08,
    random_state: int | None = None,
)

Bases: CCAEY

Robust multiview CCA by concentration steps.

:class:~cca_zoo.linear.gradient.CCAEY fitted by concentration steps: as in least trimmed squares, alternates between keeping the h rows with the lowest CCAEY loss and refitting CCAEY on them. Neither step increases the loss. The best of n_init random starts is kept. Suited to heavy contamination when the clean fraction is roughly known. Supports one latent dimension.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions; must be 1. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float

Shrinkage of the covariances towards the identity, one value for every view as in CCAEY, in [0, 1]: 0 is CCA and 1 is PLS. Default is 0.1, since each fit sees only support_fraction of the rows.

0.1
support_fraction float

Fraction of rows kept, in (0, 1]; a prior on the clean fraction. Default is 0.75.

0.75
n_init int

Random restarts. Default is 10.

10
max_iter int

Maximum concentration steps per restart. Default is 30.

30
tol float

L-BFGS-B ftol for each refit. Default is 1e-8.

1e-08
random_state int | None

Seed for the restarts. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, 1).

inlier_mask_

Boolean mask of the kept training rows.

n_iter_

Concentration steps of the best restart.

References

Rousseeuw, P. J., & Van Driessen, K. (1999). A fast algorithm for the minimum covariance determinant estimator. Technometrics, 41(3), 212-223.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import TrimmedCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 8))
>>> X2 = rng.standard_normal((200, 6))
>>> model = TrimmedCCA(support_fraction=0.7, random_state=0).fit([X1, X2])
>>> int(model.inlier_mask_.sum())
140
Source code in cca_zoo/linear/_trimmed_cca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float = 0.1,
    support_fraction: float = 0.75,
    n_init: int = 10,
    max_iter: int = 30,
    tol: float = 1e-8,
    random_state: int | None = None,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=shrinkage,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.support_fraction = support_fraction
    self.n_init = n_init

fit

fit(views: list[ArrayLike], y: None = None) -> TrimmedCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
TrimmedCCA

self.

Source code in cca_zoo/linear/_trimmed_cca.py
def fit(self, views: list[ArrayLike], y: None = None) -> TrimmedCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    n = self.n_samples_
    h = max(2, round(self.support_fraction * n))
    rng = np.random.default_rng(self.random_state)
    best_weights: list[np.ndarray] | None = None
    best_mask: np.ndarray | None = None
    best_obj = np.inf
    best_converged = False
    for _ in range(self.n_init):
        weights = []
        for xv in views_:
            w = rng.standard_normal((xv.shape[1], 1))
            w /= np.linalg.norm(w)
            weights.append(w)

        kept = np.sort(rng.choice(n, h, replace=False))
        xs_kept = [xv[kept] for xv in views_]
        weights, _ = self._minimise(xs_kept, weights, _REFIT_MAX_ITER)
        cur_obj = self._loss(xs_kept, weights)

        converged = False
        for n_iter in range(1, self.max_iter + 1):
            zs = [(xv @ w).ravel() for xv, w in zip(views_, weights)]
            b = weight_gram_mean(weights)[0, 0]
            new_kept = _select(zs, b, self.shrinkage, h)
            if np.array_equal(new_kept, kept):
                converged = True
                break
            xs_new = [xv[new_kept] for xv in views_]
            new_weights, _ = self._minimise(xs_new, weights, _REFIT_MAX_ITER)
            new_obj = self._loss(xs_new, new_weights)
            if new_obj > cur_obj + 1e-10:
                converged = True
                break
            kept, weights, cur_obj = new_kept, new_weights, new_obj

        if cur_obj < best_obj:
            best_obj, best_weights, best_mask = cur_obj, weights, kept
            self.n_iter_: int = n_iter
            best_converged = converged

    assert best_weights is not None
    assert best_mask is not None
    warn_if_not_converged(self, best_converged)
    self.weights_: list[np.ndarray] = best_weights
    self.inlier_mask_: np.ndarray = np.zeros(n, dtype=bool)
    self.inlier_mask_[best_mask] = True
    self._fit_maps_and_importances(views_)
    return self

ProjectionPursuitCCA

ProjectionPursuitCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    n_init: int = 10,
    max_iter: int = 200,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Robust multiview CCA by projection pursuit.

Searches directly for unit directions maximising a robust correlation, the projection index PI, averaged over pairs of views. PI is Spearman's rank correlation, Alfons et al.'s PP-SPM:

\[ \max_{\|a_1\| = \dots = \|a_M\| = 1} \frac{1}{\binom{M}{2}} \sum_{i < j} \operatorname{PI}(X_i a_i, X_j a_j). \]

Each direction is parametrised by angles and the objective, which is not smooth, is maximised by Powell's method from n_init random starts. Later dimensions are fitted on deflated views. Robustness comes from the index, so no rows are flagged as outliers.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
n_init int

Random restarts per dimension. Default is 10.

10
max_iter int

Maximum Powell iterations per restart. Default is 200.

200
tol float

Powell xtol and ftol. Default is 1e-6.

1e-06
random_state int | None

Seed for the restarts. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

n_iter_

Most Powell iterations of any component's best restart.

References

Branco, J. A., Croux, C., Filzmoser, P., & Oliveira, M. R. (2005). Robust canonical correlations: A comparative study. Computational Statistics, 20(2), 203-229.

Alfons, A., Croux, C., & Filzmoser, P. (2017). Robust maximum association estimators. Journal of the American Statistical Association, 112(517), 436-445.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import ProjectionPursuitCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 6))
>>> X2 = rng.standard_normal((200, 5))
>>> model = ProjectionPursuitCCA(random_state=0).fit([X1, X2])
Source code in cca_zoo/linear/_projection_pursuit_cca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    n_init: int = 10,
    max_iter: int = 200,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.n_init = n_init
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(
    views: list[ArrayLike], y: None = None
) -> ProjectionPursuitCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
ProjectionPursuitCCA

self.

Source code in cca_zoo/linear/_projection_pursuit_cca.py
def fit(self, views: list[ArrayLike], y: None = None) -> ProjectionPursuitCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    pairs = list(combinations(range(self.n_views_), 2))

    shape = [(p, self.n_components) for p in self.n_features_per_view_]
    deflated_weights = [np.zeros(s) for s in shape]
    loadings = [np.zeros(s) for s in shape]
    deflated = [v.copy() for v in views_]
    self.n_iter_: int = 0
    for d in range(self.n_components):
        directions, n_iter = self._fit_directions(deflated, pairs, rng)
        self.n_iter_ = max(self.n_iter_, n_iter)
        for i, (view, a) in enumerate(zip(deflated, directions)):
            deflated_weights[i][:, d] = a
            loadings[i][:, d] = loading(view, a)
        deflated = deflate(deflated, directions)
    warn_if_not_converged(self, self.n_iter_ < self.max_iter)
    self.weights_ = [
        undeflated_weights(w, p) for w, p in zip(deflated_weights, loadings)
    ]
    self._fit_maps_and_importances(views_)
    return self

EY-loss methods

PLSEY

PLSEY(
    n_components: int = 1,
    *,
    center: bool = True,
    max_iter: int = 1000,
    tol: float = 1e-08,
    random_state: int | None = None,
)

Bases: CCAEY

Multiview PLS by minimising the Eckart-Young loss.

:class:~cca_zoo.linear.gradient.CCAEY with shrinkage=1, fitted by full-batch L-BFGS-B without forming a covariance matrix, so suited to wide data. See :class:~cca_zoo.linear.gradient.StochasticCCAEY for mini-batches.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
max_iter int

Maximum L-BFGS-B iterations. Default is 1000.

1000
tol float

L-BFGS-B ftol. Default is 1e-8.

1e-08
random_state int | None

Seed for the initial weights. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

n_iter_

L-BFGS-B iterations run.

References

Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import PLSEY
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 500))
>>> X2 = rng.standard_normal((200, 400))
>>> model = PLSEY(n_components=4, random_state=0).fit([X1, X2])
Source code in cca_zoo/linear/gradient/_pls_ey.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    max_iter: int = 1000,
    tol: float = 1e-8,
    random_state: int | None = None,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=1.0,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )

fit

fit(views: list[ArrayLike], y: None = None) -> PLSEY

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
PLSEY

self.

Source code in cca_zoo/linear/gradient/_pls_ey.py
def fit(self, views: list[ArrayLike], y: None = None) -> PLSEY:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    return super().fit(views, y)

CCAEY

CCAEY(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float = 0.0,
    max_iter: int = 1000,
    tol: float = 1e-08,
    random_state: int | None = None,
)

Bases: BaseModel

Multiview CCA by minimising the Eckart-Young loss, with a ridge blend.

For \(Z_i = X_i W_i\), with \(C\) and \(V\) the mean pairwise cross-covariance and mean auto-covariance (:mod:cca_zoo._utils._ey) and \(B = \frac{1}{M} \sum_i W_i^\top W_i\),

\[ V_s = (1 - s) V + s B, \qquad \mathcal{L}_{EY}(s) = -2 \operatorname{tr}(C - s V) + \operatorname{tr}(V_s V_s) \]

for shrinkage \(s\), as in :class:~cca_zoo.linear.RidgeCCA: 0 is CCA and 1 is :class:~cca_zoo.linear.gradient.PLSEY. The loss is minimised by full-batch L-BFGS-B with its exact gradient; see :class:~cca_zoo.linear.gradient.StochasticCCAEY for mini-batches. With few samples per feature, shrinkage=0 is ill-conditioned; use 0.1 to 0.3.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float

Shrinkage of each view's covariance towards the identity, in [0, 1]: 0 is CCA and 1 is PLS. Default is 0.

0.0
max_iter int

Maximum L-BFGS-B iterations. Default is 1000.

1000
tol float

L-BFGS-B ftol. Default is 1e-8.

1e-08
random_state int | None

Seed for the initial weights. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

n_iter_

L-BFGS-B iterations run.

References

Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import CCAEY
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((1000, 20))
>>> X2 = rng.standard_normal((1000, 15))
>>> X3 = rng.standard_normal((1000, 10))
>>> model = CCAEY(n_components=4, random_state=0).fit([X1, X2, X3])
Source code in cca_zoo/linear/gradient/_cca_ey.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float = 0.0,
    max_iter: int = 1000,
    tol: float = 1e-8,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> CCAEY

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
CCAEY

self.

Source code in cca_zoo/linear/gradient/_cca_ey.py
def fit(self, views: list[ArrayLike], y: None = None) -> CCAEY:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    start = self._initial_weights(views_, rng)
    self.weights_, self.n_iter_ = self._minimise(views_, start, self.max_iter)
    warn_if_not_converged(self, self.n_iter_ < self.max_iter)
    self._fit_maps_and_importances(views_)
    return self

HuberCCA

HuberCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float = 0.0,
    delta: float = 4.0,
    max_iter: int = 1000,
    tol: float = 1e-08,
    random_state: int | None = None,
)

Bases: CCAEY

Bounded-influence CCA by Huber reweighting of the EY loss.

:class:~cca_zoo.linear.gradient.CCAEY, with each sample's contribution to the EY moments reweighted by a Huber weight of its leverage, recomputed at every evaluation, so high-leverage samples have bounded influence. Fitted by full-batch L-BFGS-B, with each evaluation's weights held fixed in its gradient, as in iteratively reweighted least squares.

Parameters:

Name Type Description Default
n_components int

Number of latent dimensions. Default is 1.

1
center bool

Whether to centre each view. Default is True.

True
shrinkage float

Shrinkage of each view's covariance towards the identity, in [0, 1]: 0 is CCA and 1 is PLS. Default is 0.

0.0
delta float

Huber cutoff as a multiple of the median sample leverage; smaller is more robust. Values below 1 downweight most of the data. Default is 4.0.

4.0
max_iter int

Maximum L-BFGS-B iterations. Default is 1000.

1000
tol float

L-BFGS-B ftol. Default is 1e-8.

1e-08
random_state int | None

Seed for the initial weights. Default is None.

None

Attributes:

Name Type Description
weights_

Weight matrix of each view, shape (n_features_i, n_components).

n_iter_

L-BFGS-B iterations run.

References

Filzmoser, P., Dehon, C., & Croux, C. (2000). Outlier resistant estimators for canonical correlation analysis. In COMPSTAT: Proceedings in Computational Statistics 2000 (pp. 301-306). Physica-Verlag.

Examples:

>>> import numpy as np
>>> from cca_zoo.linear import HuberCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((1000, 20))
>>> X2 = rng.standard_normal((1000, 15))
>>> model = HuberCCA(n_components=4, random_state=0).fit([X1, X2])
Source code in cca_zoo/linear/gradient/_huber_cca.py
def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float = 0.0,
    delta: float = 4.0,
    max_iter: int = 1000,
    tol: float = 1e-8,
    random_state: int | None = None,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=shrinkage,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.delta = delta

fit

fit(views: list[ArrayLike], y: None = None) -> HuberCCA

Fit the model.

Parameters:

Name Type Description Default
views list[ArrayLike]

Arrays of shape (n_samples, n_features_i), one per view.

required
y None

Ignored.

None

Returns:

Type Description
HuberCCA

self.

Source code in cca_zoo/linear/gradient/_huber_cca.py
def fit(self, views: list[ArrayLike], y: None = None) -> HuberCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    return super().fit(views, y)