cca_zoo.linear¶
Linear CCA methods. All classes are sklearn.base.BaseEstimator subclasses.
Sparse/regularised iterative methods live in cca_zoo.sparse;
mini-batch methods live in cca_zoo.stochastic.
Base class¶
BaseModel ¶
Bases: BaseEstimator, ABC
Base class for multiview CCA models.
Subclasses implement :meth:fit, starting with :meth:_setup_fit and
ending with :meth:_fit_maps_and_importances. A linear model sets weights_; a
nonlinear model overrides :meth:_transform_view, its per-view encoder.
transform, predict, inverse_transform, score and
feature_importances_per_view_ are built on that encoder.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
Attributes:
| Name | Type | Description |
|---|---|---|
means_ |
Per-view feature means subtracted before fitting. |
|
n_features_per_view_ |
Number of features in each view. |
|
n_components_ |
Number of latent dimensions fitted: |
|
feature_names_per_view_ |
Each view's feature names, set when every view was fitted as a DataFrame with string column names. |
|
n_samples_ |
Number of training samples. |
|
n_views_ |
Number of views. |
|
feature_importances_per_view_ |
Each feature's share of its view's
embedding, one array per view. Non-negative and summing to 1
within each view (all zeros if a view's embedding uses no
feature). Linear models use |
fit
abstractmethod
¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
BaseModel
|
self. |
transform ¶
Project each view into the latent space.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
List of arrays, each of shape (n_samples, n_features_i). |
required |
Returns:
| Type | Description |
|---|---|
list[Any]
|
List of arrays, each of shape (n_samples, n_components), or of |
list[Any]
|
DataFrames under :meth: |
inverse_transform ¶
Map each view's latent scores back to that view's feature space.
Each view is reconstructed from its own scores by a least-squares
regression fitted on the training data, as in
:meth:sklearn.decomposition.PCA.inverse_transform. To reconstruct a
view from the other views, use :meth:predict.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
scores
|
list[ArrayLike]
|
One array of shape (n_samples, n_components) per view. |
required |
Returns:
| Type | Description |
|---|---|
list[ndarray]
|
List of arrays, each of shape (n_samples, n_features_i). |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
predict ¶
Reconstruct every view from the views that are observed.
The shared latent scores are estimated from the observed views (their
mean, or the posterior mean for probabilistic models) and mapped to each
view by a least-squares regression fitted on the training data. Pass
None for a view to reconstruct.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike | None]
|
One entry per view: an array of shape (n_samples, n_features_i)
or |
required |
Returns:
| Type | Description |
|---|---|
list[ndarray]
|
List of arrays, each of shape (n_samples, n_features_i). |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Examples:
fit_transform ¶
Fit the model and transform the training views.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
List of arrays, each of shape (n_samples, n_features_i). |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
list[ndarray]
|
List of arrays, each of shape (n_samples, n_components). |
score ¶
Mean canonical correlation between the views' latent scores.
The average over latent dimensions of the mean pairwise correlation.
Per-dimension values:
average_pairwise_correlations(pairwise_correlations(model.transform(views)))
from :mod:cca_zoo.metrics.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
List of arrays, each of shape (n_samples, n_features_i). |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
float
|
The mean canonical correlation. |
get_feature_names_out ¶
Names of each view's latent dimensions, <model><k> as in sklearn.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
input_features
|
list[list[str]] | None
|
Ignored, unless given: then each view's must match the names seen in fit. |
None
|
Returns:
| Type | Description |
|---|---|
list[ndarray]
|
One array of names per view, of length n_components. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
set_output ¶
Set the container :meth:transform returns, as sklearn's set_output.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
transform
|
str | None
|
|
None
|
Returns:
| Type | Description |
|---|---|
_Model
|
self. |
Two-view exact methods¶
CCA ¶
Bases: RidgeCCA
Canonical correlation analysis of two views.
:class:RidgeCCA with shrinkage=0.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
References
Hotelling, H. (1936). Relations between two sets of variates. Biometrika, 28(3/4), 321-377.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import CCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> Z1, Z2 = CCA(n_components=2).fit_transform([X1, X2])
Source code in cca_zoo/linear/_cca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
sample_weight
|
ArrayLike | None
|
Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally. |
None
|
Returns:
| Type | Description |
|---|---|
CCA
|
self. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If there are not exactly two views. |
Source code in cca_zoo/linear/_cca.py
RidgeCCA ¶
Bases: BaseModel
Ridge-regularised CCA of two views (canonical ridge).
solved by whitening each view and taking the SVD of the cross-covariance.
shrinkage=0 is :class:CCA and shrinkage=1 is :class:PLS.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float | list[float]
|
Shrinkage of each view's covariance towards the identity,
in |
0.0
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
References
Vinod, H. D. (1976). Canonical ridge and econometrics of joint production. Journal of Econometrics, 4(2), 147-166.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import RidgeCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> model = RidgeCCA(n_components=2, shrinkage=[0.1, 0.5]).fit([X1, X2])
Source code in cca_zoo/linear/_ridge_cca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
sample_weight
|
ArrayLike | None
|
Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally. |
None
|
Returns:
| Type | Description |
|---|---|
RidgeCCA
|
self. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If there are not exactly two views. |
Source code in cca_zoo/linear/_ridge_cca.py
PLS ¶
Bases: RidgeCCA
Partial least squares of two views.
the truncated SVD of the cross-covariance; :class:RidgeCCA with
shrinkage=1.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
References
Wold, H. (1975). Soft modelling by latent variables: the nonlinear iterative partial least squares (NIPALS) approach. Perspectives in Probability and Statistics, 117-142.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import PLS
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> Z1, Z2 = PLS(n_components=2).fit_transform([X1, X2])
Source code in cca_zoo/linear/_pls.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
sample_weight
|
ArrayLike | None
|
Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally. |
None
|
Returns:
| Type | Description |
|---|---|
PLS
|
self. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If there are not exactly two views. |
Source code in cca_zoo/linear/_pls.py
Multiview methods¶
MCCA ¶
MCCA(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float | list[float] = 0.0,
pca: bool = True,
)
Bases: BaseModel
Multiset CCA: the sum of pairwise covariances under ridge constraints.
solved as the generalized eigenproblem \(A v = \lambda B v\), with \(A\) the
between-view and \(B\) the regularised within-view covariance blocks.
shrinkage=0 is CCA and shrinkage=1 is PLS.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float | list[float]
|
Shrinkage of each view's covariance towards the identity,
in |
0.0
|
pca
|
bool
|
Whether to solve in each view's principal components, which is faster and more stable for wide data. Default is True. |
True
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
References
Kettenring, J. R. (1971). Canonical analysis of several sets of variables. Biometrika, 58(3), 433-451.
Vinod, H. D. (1976). Canonical ridge and econometrics of joint production. Journal of Econometrics, 4(2), 147-166.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import MCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> X3 = rng.standard_normal((50, 6))
>>> model = MCCA(n_components=2, shrinkage=0.1).fit([X1, X2, X3])
Source code in cca_zoo/linear/_mcca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
sample_weight
|
ArrayLike | None
|
Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally. |
None
|
Returns:
| Type | Description |
|---|---|
MCCA
|
self. |
Source code in cca_zoo/linear/_mcca.py
GCCA ¶
GCCA(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float | list[float] = 0.0,
view_weights: list[float] | None = None,
)
Bases: BaseModel
Generalized CCA: views correlated with a shared latent variable.
with \(C_i = (1 - c_i) \Sigma_{ii} + c_i I\): at \(c_i = 0\), the squared error of regressing \(T\) on each view. Each view's weights are its regularised regression onto \(T\), \(w_i = C_i^{-1} X_i^\top T / (n - 1)\), and \(T\) holds the top eigenvectors of \(\sum_i \mu_i X_i C_i^{-1} X_i^\top\), found from the SVD of the stacked whitened views.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float | list[float]
|
Shrinkage of each view's covariance towards the identity,
in |
0.0
|
view_weights
|
list[float] | None
|
Weight \(\mu_i\) of each view; None weights them equally. Default is None. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
References
Carroll, J. D. (1968). Generalization of canonical correlation analysis to three or more sets of variables. Proceedings of the 76th Annual Convention of the American Psychological Association, 3, 227-228. Kettenring, J. R. (1971). Canonical analysis of several sets of variables. Biometrika, 58(3), 433-451.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import GCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> X3 = rng.standard_normal((50, 6))
>>> model = GCCA(n_components=2).fit([X1, X2, X3])
Source code in cca_zoo/linear/_gcca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
sample_weight
|
ArrayLike | None
|
Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally. |
None
|
Returns:
| Type | Description |
|---|---|
GCCA
|
self. |
Source code in cca_zoo/linear/_gcca.py
TCCA ¶
TCCA(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float | list[float] = 0.0,
init: str = "svd",
random_state: int | RandomState | None = None,
)
Bases: BaseModel
Tensor CCA: higher-order correlation of three or more views.
Decomposes the cross-moment tensor of the whitened views \(\tilde X_i = X_i \Sigma_i^{-1/2}\),
by PARAFAC; the factors give the canonical directions.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float | list[float]
|
Shrinkage of each view's covariance towards the identity,
in |
0.0
|
init
|
str
|
PARAFAC's initialisation: |
'svd'
|
random_state
|
int | RandomState | None
|
Seed for |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
References
Luo, Y., Tao, D., Ramamohanarao, K., Xu, C., & Wen, Y. (2015). Tensor canonical correlation analysis for multi-view dimension reduction. IEEE Transactions on Knowledge and Data Engineering, 27(11), 3111-3124.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import TCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 5))
>>> X2 = rng.standard_normal((50, 5))
>>> X3 = rng.standard_normal((50, 5))
>>> model = TCCA(n_components=2, random_state=0).fit([X1, X2, X3])
Source code in cca_zoo/linear/_tcca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
TCCA
|
self. |
Source code in cca_zoo/linear/_tcca.py
Confound-adjusted / structured methods¶
PartialCCA ¶
Bases: MCCA
CCA of the views after regressing out confounds.
for confounds \(Z\) (partials). The constraints leave MCCA's
eigenproblem on the views' partial covariance given the confounds,
whose off-diagonal blocks form \(A\) and whose diagonal blocks, with
shrinkage, form \(B\), as :class:MCCA forms them from the covariance.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float | list[float]
|
Shrinkage of each view's covariance towards the identity,
in |
0.0
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
confound_betas_ |
Regression of each view on the centred confounds, shape (n_confounds, n_features_i). |
|
partials_mean_ |
Mean of the training confounds, shape (n_confounds,). |
References
Rao, B. R. (1969). Partial canonical correlations. Trabajos de Estadistica y de Investigacion Operativa, 20(2-3), 211-219.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import PartialCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> Z = rng.standard_normal((50, 3))
>>> model = PartialCCA(n_components=2).fit([X1, X2], partials=Z)
>>> Z1, Z2 = model.transform([X1, X2], partials=Z)
Source code in cca_zoo/linear/_partialcca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
partials
|
ArrayLike | None
|
Confounds, shape (n_samples, n_confounds). |
None
|
Returns:
| Type | Description |
|---|---|
PartialCCA
|
self. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in cca_zoo/linear/_partialcca.py
transform ¶
Project views into the latent space, regressing out confounds if given.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
partials
|
ArrayLike | None
|
Confounds, shape (n_samples, n_confounds); None projects the views unadjusted. Default is None. |
None
|
Returns:
| Type | Description |
|---|---|
list[ndarray]
|
One array of shape (n_samples, n_components) per view. |
Source code in cca_zoo/linear/_partialcca.py
fit_transform ¶
fit_transform(
views: list[ArrayLike],
y: None = None,
partials: ArrayLike | None = None,
) -> list[np.ndarray]
Fit, then transform the training views.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
partials
|
ArrayLike | None
|
Confounds, shape (n_samples, n_confounds). |
None
|
Returns:
| Type | Description |
|---|---|
list[ndarray]
|
One array of shape (n_samples, n_components) per view. |
Source code in cca_zoo/linear/_partialcca.py
GRCCA ¶
GRCCA(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float | list[float] = 0.0,
mu: float | list[float] = 0.0,
feature_groups: list[ndarray] | None = None,
)
Bases: MCCA
MCCA with ridge penalties that shrink weights towards their group means.
Tuzhilina et al.'s penalty on each view's weights \(w\), for features in groups \(g\) of size \(p_g\) with mean weight \(\bar w_g\),
with \(H\) the projection averaging each group, takes the place of
:class:MCCA's identity: each view's block of \(B\) is
\((1 - c)\Sigma_{ii} + c\,((I - H) + \mu H)\). shrinkage \(c\) sets the
strength of the penalty and mu that of the group means relative to
the deviations from them: mu=1 is MCCA's ridge, and mu=0 shrinks
each weight towards its group's mean alone. Their \(\lambda\) and \(\mu\)
are \(c / (1 - c)\) and \(\mu c / (1 - c)\).
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float | list[float]
|
Strength of the penalty, in |
0.0
|
mu
|
float | list[float]
|
Penalty on the group means, relative to the within-group deviations. Per-view. Default is 0. |
0.0
|
feature_groups
|
list[ndarray] | None
|
Integer group label of each feature, shape (n_features_i,) per view; None puts each view in one group. Default is None. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
feature_groups_ |
Group label of each feature, per view. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If a view's |
References
Tuzhilina, E., Tozzi, L., & Hastie, T. (2023). Canonical correlation analysis in high dimensions with structured regularization. Statistical Modelling, 23(3), 203-227.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import GRCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> groups = [rng.integers(0, 3, size=10), rng.integers(0, 3, size=8)]
>>> model = GRCCA(n_components=2, shrinkage=0.5, feature_groups=groups)
>>> model = model.fit([X1, X2])
Source code in cca_zoo/linear/_grcca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
sample_weight
|
ArrayLike | None
|
Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally. |
None
|
Returns:
| Type | Description |
|---|---|
GRCCA
|
self. |
Source code in cca_zoo/linear/_grcca.py
Reduced-rank regression methods¶
CCAR3 ¶
CCAR3(
n_components: int = 1,
*,
center: bool = True,
alpha: float = 0.0,
ledoit_wolf: bool = True,
max_iter: int = 10000,
tol: float = 0.0001,
)
Bases: BaseModel
Two-view CCA by row-sparse reduced rank regression.
Whitens \(Y\) by its (optionally Ledoit-Wolf) covariance, regresses
\(\tilde{Y} = Y \Sigma_Y^{-1/2}\) on \(X\), and takes the canonical
directions from the rank-n_components SVD of the fitted values. The
regression has a row-group lasso penalty,
which drops whole features of \(X\); swap the views to make the other one
sparse. alpha=0 is solved by least squares. A port of the R package
ccar3.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
alpha
|
float
|
Row-group lasso strength. Default is 0. |
0.0
|
ledoit_wolf
|
bool
|
Whether to shrink the covariance of |
True
|
max_iter
|
int
|
Maximum MultiTaskLasso iterations. Default is 10000. |
10000
|
tol
|
float
|
MultiTaskLasso tolerance. Default is 1e-4. |
0.0001
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
n_iter_ |
Iterations of the MultiTaskLasso; 0 when |
References
Donnat, C., & Tuzhilina, E. (2024). Canonical Correlation Analysis as Reduced Rank Regression in High Dimensions. arXiv:2405.19539.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import CCAR3
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> model = CCAR3(n_components=2, alpha=0.1).fit([X1, X2])
Source code in cca_zoo/linear/_ccar3.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
CCAR3
|
self. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If there are not exactly two views. |
Source code in cca_zoo/linear/_ccar3.py
ECCA ¶
ECCA(
n_components: int = 1,
*,
center: bool = True,
alpha: float = 0.0,
ledoit_wolf: bool = False,
max_iter: int = 10000,
tol: float = 0.0001,
)
Bases: CCAR3
Two-view CCA by entrywise-sparse reduced rank regression.
:class:~cca_zoo.linear.CCAR3 with an entrywise lasso penalty on the
regression of the whitened \(\tilde{Y} = Y \Sigma_Y^{-1/2}\) on \(X\),
and takes the canonical directions from the rank-n_components SVD of
the fitted values \(X \hat{B}\). The penalty zeroes entries of \(\hat{B}\),
where :class:~cca_zoo.linear.CCAR3's zeroes whole rows; the canonical
directions combine its columns, so a feature leaves them only when its
whole row is zero. A port of ecca() from the R package ccar3.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
alpha
|
float
|
Entrywise lasso strength. Default is 0. |
0.0
|
ledoit_wolf
|
bool
|
Whether to shrink the covariance of |
False
|
max_iter
|
int
|
Maximum Lasso iterations. Default is 10000. |
10000
|
tol
|
float
|
Lasso tolerance. Default is 1e-4. |
0.0001
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
n_iter_ |
Most iterations of any column's Lasso; 0 when |
References
Donnat, C., & Tuzhilina, E. (2024). Canonical Correlation Analysis as Reduced Rank Regression in High Dimensions. arXiv:2405.19539.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import ECCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((50, 10))
>>> X2 = rng.standard_normal((50, 8))
>>> model = ECCA(n_components=2, alpha=0.1).fit([X1, X2])
Source code in cca_zoo/linear/_ecca.py
Sparse-precision covariance methods¶
GraphicalLassoCCA ¶
GraphicalLassoCCA(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float | list[float] = 0.0,
alpha: float | list[float | None] | None = 0.01,
max_iter: int = 1000,
tol: float = 1e-08,
)
Bases: MCCA
MCCA with each within-view covariance estimated by the graphical lasso.
Replaces each view's block of :class:~cca_zoo.linear.MCCA's \(B\) with
the covariance implied by a sparse precision estimate, an L1 penalty on
partial correlations rather than shrinkage of the covariance. The lasso
is fitted to each view's correlation matrix, so alpha is the same in
any units, and solved by :func:graphical_lasso, ADMM: sklearn's
coordinate descent fails to converge, or to run, on strongly correlated
features. Solved in the original feature space.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float | list[float]
|
Shrinkage of each view's estimated covariance towards the
identity, in |
0.0
|
alpha
|
float | list[float | None] | None
|
Graphical-lasso penalty on the correlation scale; None selects
it by :class: |
0.01
|
max_iter
|
int
|
Maximum ADMM iterations per view. Default is 1000. |
1000
|
tol
|
float
|
Tolerance on ADMM's residuals, per entry. Default is 1e-8. |
1e-08
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
covariance_ |
Estimated covariance of each view. |
|
precision_ |
Estimated sparse precision of each view. |
|
n_iter_ |
Most graphical-lasso iterations of any view. |
References
Friedman, J., Hastie, T., & Tibshirani, R. (2008). Sparse inverse covariance estimation with the graphical lasso. Biostatistics, 9(3), 432-441.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import GraphicalLassoCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((100, 12))
>>> X2 = rng.standard_normal((100, 9))
>>> model = GraphicalLassoCCA(alpha=0.1).fit([X1, X2])
>>> model.precision_[0].shape
(12, 12)
Source code in cca_zoo/linear/_graphical_lasso_cca.py
fit ¶
fit(
views: list[ArrayLike],
y: None = None,
sample_weight: ArrayLike | None = None,
) -> GraphicalLassoCCA
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
sample_weight
|
ArrayLike | None
|
Weight of each sample; an integer weight is the same as repeating the sample. None weights samples equally. |
None
|
Returns:
| Type | Description |
|---|---|
GraphicalLassoCCA
|
self. |
Raises:
| Type | Description |
|---|---|
ValueError
|
If |
Source code in cca_zoo/linear/_graphical_lasso_cca.py
Robust methods¶
RANSACCCA ¶
RANSACCCA(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float | list[float] = 0.1,
min_samples: int | float = 0.25,
residual_threshold: float | None = None,
max_trials: int = 200,
stop_probability: float = 0.99,
random_state: int | None = None,
)
Bases: BaseModel
Robust multiview CCA by random sample consensus.
As :class:~sklearn.linear_model.RANSACRegressor: fits
:class:~cca_zoo.linear.MCCA on random subsets of min_samples rows,
scores each candidate by the total positive cross-view agreement of
every sample, and refits on the inliers of the best one. Targets samples
whose cross-view relationship is wrong but whose magnitudes are ordinary,
which leverage-based :class:~cca_zoo.linear.HuberCCA cannot see. The
number of trials adapts to the inlier fraction found, up to
max_trials.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float | list[float]
|
Shrinkage of every internal MCCA fit. Default is 0.1: each trial fits a quarter of the rows, often fewer than the features. |
0.1
|
min_samples
|
int | float
|
Subset size, as a fraction in |
0.25
|
residual_threshold
|
float | None
|
Minimum agreement of an inlier; None is 0. Default is None. |
None
|
max_trials
|
int
|
Maximum subsets tried. Default is 200. |
200
|
stop_probability
|
float
|
Stop once a subset at least as clean as the best is found with this probability. Default is 0.99. |
0.99
|
random_state
|
int | None
|
Seed for the subset draws. Default is None. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
inlier_mask_ |
Boolean mask of the consensus set. |
|
n_trials_ |
Number of subsets tried. |
References
Chapman, J., Wang, H.-T., Wells, L., & Wiesner, J. (2021). CCA-Zoo: A collection of Regularized, Deep Learning based, Kernel, and Probabilistic CCA methods in a scikit-learn style framework. Journal of Open Source Software, 6(68), 3823.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import RANSACCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 8))
>>> X2 = rng.standard_normal((200, 6))
>>> model = RANSACCCA(random_state=0).fit([X1, X2])
Source code in cca_zoo/linear/_ransac_cca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
RANSACCCA
|
self. |
Source code in cca_zoo/linear/_ransac_cca.py
TrimmedCCA ¶
TrimmedCCA(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float = 0.1,
support_fraction: float = 0.75,
n_init: int = 10,
max_iter: int = 30,
tol: float = 1e-08,
random_state: int | None = None,
)
Bases: CCAEY
Robust multiview CCA by concentration steps.
:class:~cca_zoo.linear.gradient.CCAEY fitted by concentration steps: as
in least trimmed squares, alternates between keeping the h rows with
the lowest CCAEY loss and refitting CCAEY on them. Neither step increases
the loss. The best of n_init random starts is kept. Suited to heavy
contamination when the clean fraction is roughly known. Supports one
latent dimension.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions; must be 1. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float
|
Shrinkage of the covariances towards the identity, one value
for every view as in |
0.1
|
support_fraction
|
float
|
Fraction of rows kept, in |
0.75
|
n_init
|
int
|
Random restarts. Default is 10. |
10
|
max_iter
|
int
|
Maximum concentration steps per restart. Default is 30. |
30
|
tol
|
float
|
L-BFGS-B |
1e-08
|
random_state
|
int | None
|
Seed for the restarts. Default is None. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, 1). |
|
inlier_mask_ |
Boolean mask of the kept training rows. |
|
n_iter_ |
Concentration steps of the best restart. |
References
Rousseeuw, P. J., & Van Driessen, K. (1999). A fast algorithm for the minimum covariance determinant estimator. Technometrics, 41(3), 212-223.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import TrimmedCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 8))
>>> X2 = rng.standard_normal((200, 6))
>>> model = TrimmedCCA(support_fraction=0.7, random_state=0).fit([X1, X2])
>>> int(model.inlier_mask_.sum())
140
Source code in cca_zoo/linear/_trimmed_cca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
TrimmedCCA
|
self. |
Source code in cca_zoo/linear/_trimmed_cca.py
ProjectionPursuitCCA ¶
ProjectionPursuitCCA(
n_components: int = 1,
*,
center: bool = True,
n_init: int = 10,
max_iter: int = 200,
tol: float = 1e-06,
random_state: int | None = None,
)
Bases: BaseModel
Robust multiview CCA by projection pursuit.
Searches directly for unit directions maximising a robust correlation,
the projection index PI, averaged over pairs of views. PI is Spearman's
rank correlation, Alfons et al.'s PP-SPM:
Each direction is parametrised by angles and the objective, which is not
smooth, is maximised by Powell's method from n_init random starts.
Later dimensions are fitted on deflated views. Robustness comes from the
index, so no rows are flagged as outliers.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
n_init
|
int
|
Random restarts per dimension. Default is 10. |
10
|
max_iter
|
int
|
Maximum Powell iterations per restart. Default is 200. |
200
|
tol
|
float
|
Powell |
1e-06
|
random_state
|
int | None
|
Seed for the restarts. Default is None. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
n_iter_ |
Most Powell iterations of any component's best restart. |
References
Branco, J. A., Croux, C., Filzmoser, P., & Oliveira, M. R. (2005). Robust canonical correlations: A comparative study. Computational Statistics, 20(2), 203-229.
Alfons, A., Croux, C., & Filzmoser, P. (2017). Robust maximum association estimators. Journal of the American Statistical Association, 112(517), 436-445.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import ProjectionPursuitCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 6))
>>> X2 = rng.standard_normal((200, 5))
>>> model = ProjectionPursuitCCA(random_state=0).fit([X1, X2])
Source code in cca_zoo/linear/_projection_pursuit_cca.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
ProjectionPursuitCCA
|
self. |
Source code in cca_zoo/linear/_projection_pursuit_cca.py
EY-loss methods¶
PLSEY ¶
PLSEY(
n_components: int = 1,
*,
center: bool = True,
max_iter: int = 1000,
tol: float = 1e-08,
random_state: int | None = None,
)
Bases: CCAEY
Multiview PLS by minimising the Eckart-Young loss.
:class:~cca_zoo.linear.gradient.CCAEY with shrinkage=1, fitted by
full-batch L-BFGS-B without forming a covariance matrix, so suited to
wide data. See :class:~cca_zoo.linear.gradient.StochasticCCAEY for
mini-batches.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
max_iter
|
int
|
Maximum L-BFGS-B iterations. Default is 1000. |
1000
|
tol
|
float
|
L-BFGS-B |
1e-08
|
random_state
|
int | None
|
Seed for the initial weights. Default is None. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
n_iter_ |
L-BFGS-B iterations run. |
References
Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import PLSEY
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 500))
>>> X2 = rng.standard_normal((200, 400))
>>> model = PLSEY(n_components=4, random_state=0).fit([X1, X2])
Source code in cca_zoo/linear/gradient/_pls_ey.py
CCAEY ¶
CCAEY(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float = 0.0,
max_iter: int = 1000,
tol: float = 1e-08,
random_state: int | None = None,
)
Bases: BaseModel
Multiview CCA by minimising the Eckart-Young loss, with a ridge blend.
For \(Z_i = X_i W_i\), with \(C\) and \(V\) the mean pairwise cross-covariance
and mean auto-covariance (:mod:cca_zoo._utils._ey) and
\(B = \frac{1}{M} \sum_i W_i^\top W_i\),
for shrinkage \(s\), as in :class:~cca_zoo.linear.RidgeCCA: 0 is CCA
and 1 is :class:~cca_zoo.linear.gradient.PLSEY. The loss is minimised by
full-batch L-BFGS-B with its exact gradient; see
:class:~cca_zoo.linear.gradient.StochasticCCAEY for mini-batches. With
few samples per feature, shrinkage=0 is ill-conditioned; use 0.1 to
0.3.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float
|
Shrinkage of each view's covariance towards the identity,
in |
0.0
|
max_iter
|
int
|
Maximum L-BFGS-B iterations. Default is 1000. |
1000
|
tol
|
float
|
L-BFGS-B |
1e-08
|
random_state
|
int | None
|
Seed for the initial weights. Default is None. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
n_iter_ |
L-BFGS-B iterations run. |
References
Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import CCAEY
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((1000, 20))
>>> X2 = rng.standard_normal((1000, 15))
>>> X3 = rng.standard_normal((1000, 10))
>>> model = CCAEY(n_components=4, random_state=0).fit([X1, X2, X3])
Source code in cca_zoo/linear/gradient/_cca_ey.py
fit ¶
Fit the model.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
views
|
list[ArrayLike]
|
Arrays of shape (n_samples, n_features_i), one per view. |
required |
y
|
None
|
Ignored. |
None
|
Returns:
| Type | Description |
|---|---|
CCAEY
|
self. |
Source code in cca_zoo/linear/gradient/_cca_ey.py
HuberCCA ¶
HuberCCA(
n_components: int = 1,
*,
center: bool = True,
shrinkage: float = 0.0,
delta: float = 4.0,
max_iter: int = 1000,
tol: float = 1e-08,
random_state: int | None = None,
)
Bases: CCAEY
Bounded-influence CCA by Huber reweighting of the EY loss.
:class:~cca_zoo.linear.gradient.CCAEY, with each sample's contribution
to the EY moments reweighted by a Huber weight of its leverage,
recomputed at every evaluation, so high-leverage samples have bounded
influence. Fitted by full-batch L-BFGS-B, with each evaluation's weights
held fixed in its gradient, as in iteratively reweighted least squares.
Parameters:
| Name | Type | Description | Default |
|---|---|---|---|
n_components
|
int
|
Number of latent dimensions. Default is 1. |
1
|
center
|
bool
|
Whether to centre each view. Default is True. |
True
|
shrinkage
|
float
|
Shrinkage of each view's covariance towards the identity,
in |
0.0
|
delta
|
float
|
Huber cutoff as a multiple of the median sample leverage; smaller is more robust. Values below 1 downweight most of the data. Default is 4.0. |
4.0
|
max_iter
|
int
|
Maximum L-BFGS-B iterations. Default is 1000. |
1000
|
tol
|
float
|
L-BFGS-B |
1e-08
|
random_state
|
int | None
|
Seed for the initial weights. Default is None. |
None
|
Attributes:
| Name | Type | Description |
|---|---|---|
weights_ |
Weight matrix of each view, shape (n_features_i, n_components). |
|
n_iter_ |
L-BFGS-B iterations run. |
References
Filzmoser, P., Dehon, C., & Croux, C. (2000). Outlier resistant estimators for canonical correlation analysis. In COMPSTAT: Proceedings in Computational Statistics 2000 (pp. 301-306). Physica-Verlag.
Examples:
>>> import numpy as np
>>> from cca_zoo.linear import HuberCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((1000, 20))
>>> X2 = rng.standard_normal((1000, 15))
>>> model = HuberCCA(n_components=4, random_state=0).fit([X1, X2])