cca_zoo.sparse

Sparse linear CCA methods. No optional dependency required. Two mechanism families live here: EY-loss coordinate descent, and Alternating Least Squares (ALS).

---

EY-loss coordinate descent

ElasticNetCCA

ElasticNetCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 1.0,
    l1_ratio: float | list[float] = 0.5,
    max_iter: int = 1000,
    tol: float = 1e-06,
    random_state: int | None = None,
    positive: bool = False,
)

Bases: BaseModel

Sparse multiview CCA by coordinate descent on the elastic-net EY loss.

Minimises, over \(Z_i = X_i W_i\),

\[ \mathcal{L}_{EY}(Z_1, \dots, Z_M) + \sum_i \left( \alpha_i \rho_i \|W_i\|_1 + \tfrac{1}{2} \alpha_i (1-\rho_i) \|W_i\|_F^2 \right), \]

with \(\rho\) = l1_ratio, by cyclic coordinate descent as in :class:~sklearn.linear_model.ElasticNet; each coordinate's quartic restriction is minimised exactly. The loss is not jointly convex, so the result can depend on random_state.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
alpha	float | list[float]	Penalty strength. Per-view. Default is 1.0.	1.0
l1_ratio	float | list[float]	L1 share of the penalty in [0, 1]. Per-view. Default is 0.5.	0.5
max_iter	int	Maximum coordinate-descent sweeps at each stage of the penalty path. Default is 1000, as sklearn's ElasticNet.	1000
tol	float	Tolerance on the change in the objective. Default is 1e-6.	1e-06
random_state	int | None	Seed for the initial weights. Default is None.	None
positive	bool	Whether to constrain the weights to be non-negative. Default is False.	False

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Coordinate-descent sweeps at the full penalty.

References

Chapman, J., Wang, H.-T., Wells, L., & Wiesner, J. (2021). CCA-Zoo: A collection of Regularized, Deep Learning based, Kernel, and Probabilistic CCA methods in a scikit-learn style framework. Journal of Open Source Software, 6(68), 3823.

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import ElasticNetCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 20))
>>> X2 = rng.standard_normal((200, 15))
>>> model = ElasticNetCCA(n_components=2, alpha=[0.1, 0.5]).fit([X1, X2])

Source code in cca_zoo/sparse/_elasticnetcca.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 1.0,
    l1_ratio: float | list[float] = 0.5,
    max_iter: int = 1000,
    tol: float = 1e-6,
    random_state: int | None = None,
    positive: bool = False,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.alpha = alpha
    self.l1_ratio = l1_ratio
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state
    self.positive = positive

fit

fit(
    views: list[ArrayLike], y: None = None
) -> ElasticNetCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
ElasticNetCCA	self.

Source code in cca_zoo/sparse/_elasticnetcca.py

def fit(self, views: list[ArrayLike], y: None = None) -> ElasticNetCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    alpha = perview_parameter("alpha", self.alpha, 1.0, self.n_views_)
    l1_ratio = perview_parameter("l1_ratio", self.l1_ratio, 0.5, self.n_views_)
    rng = np.random.default_rng(self.random_state)
    weights = cheap_orthonormal_projection_weights(
        views_, self.n_components, None, rng
    )
    for scale in PENALTY_PATH:
        self.n_iter_, converged = self._coordinate_descent(
            views_, weights, [a * scale for a in alpha], l1_ratio
        )
    warn_if_not_converged(self, converged)
    # All-zero weights have loss zero, so a fit ending above it has none.
    if self._objective(views_, weights, alpha, l1_ratio) > 0.0:
        weights = [np.zeros_like(w) for w in weights]
    self.weights_: list[np.ndarray] = weights
    self._fit_maps_and_importances(views_)
    return self

---

MultiTaskElasticNetCCA

MultiTaskElasticNetCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 1.0,
    l1_ratio: float | list[float] = 0.5,
    max_iter: int = 1000,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Row-sparse multiview CCA with a multi-task elastic-net penalty on the EY loss.

As :class:~cca_zoo.sparse.ElasticNetCCA with :class:~sklearn.linear_model.MultiTaskElasticNet's penalty,

\[ \mathcal{L}_{EY}(Z_1, \dots, Z_M) + \sum_i \left( \alpha_i \rho_i \|W_i\|_{2,1} + \tfrac{1}{2} \alpha_i (1-\rho_i) \|W_i\|_F^2 \right), \]

so each feature is used by every component or by none. Each row is updated by a proximal-gradient step with backtracking.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
alpha	float | list[float]	Penalty strength. Per-view. Default is 1.0.	1.0
l1_ratio	float | list[float]	Row-group share of the penalty in [0, 1]. Per-view. Default is 0.5.	0.5
max_iter	int	Maximum coordinate-descent sweeps at each stage of the penalty path. Default is 1000, as sklearn's ElasticNet.	1000
tol	float	Tolerance on the change in the objective. Default is 1e-6.	1e-06
random_state	int | None	Seed for the initial weights. Default is None.	None

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Proximal sweeps at the full penalty.

References

Chapman, J., Wang, H.-T., Wells, L., & Wiesner, J. (2021). CCA-Zoo: A collection of Regularized, Deep Learning based, Kernel, and Probabilistic CCA methods in a scikit-learn style framework. Journal of Open Source Software, 6(68), 3823.

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import MultiTaskElasticNetCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 20))
>>> X2 = rng.standard_normal((200, 15))
>>> model = MultiTaskElasticNetCCA(n_components=2, alpha=0.1).fit([X1, X2])

Source code in cca_zoo/sparse/_multitaskelasticnetcca.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 1.0,
    l1_ratio: float | list[float] = 0.5,
    max_iter: int = 1000,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.alpha = alpha
    self.l1_ratio = l1_ratio
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(
    views: list[ArrayLike], y: None = None
) -> MultiTaskElasticNetCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
MultiTaskElasticNetCCA	self.

Source code in cca_zoo/sparse/_multitaskelasticnetcca.py

def fit(self, views: list[ArrayLike], y: None = None) -> MultiTaskElasticNetCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    alpha = perview_parameter("alpha", self.alpha, 1.0, self.n_views_)
    l1_ratio = perview_parameter("l1_ratio", self.l1_ratio, 0.5, self.n_views_)
    rng = np.random.default_rng(self.random_state)
    weights = cheap_orthonormal_projection_weights(
        views_, self.n_components, None, rng
    )
    for scale in PENALTY_PATH:
        self.n_iter_, converged = self._proximal_descent(
            views_, weights, [a * scale for a in alpha], l1_ratio
        )
    warn_if_not_converged(self, converged)
    # All-zero weights have loss zero, so a fit ending above it has none.
    if self._objective(views_, weights, alpha, l1_ratio) > 0.0:
        weights = [np.zeros_like(w) for w in weights]
    self.weights_: list[np.ndarray] = weights
    self._fit_maps_and_importances(views_)
    return self

---

OrthogonalMatchingPursuitCCA

OrthogonalMatchingPursuitCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    n_nonzero_coefs: int | list[int] | None = None,
    max_iter: int = 10,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Sparse multiview CCA with a fixed number of active features per view.

The EY analogue of :class:~sklearn.linear_model.OrthogonalMatchingPursuit: each view's active set grows one feature at a time, choosing the feature with the largest EY gradient norm, and the active weights are refitted by coordinate descent after each addition. Once the active sets settle, the weights are the exact EY optimum on them, as OMP's coefficients are the least-squares fit on its support. Features outside the active set are zero.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
n_nonzero_coefs	int | list[int] | None	Active features, capped at the view's width. The components share the active set, so it needs at least n_components features for them to be linearly independent. None is max(n_components, n_features_i // 10). Per-view. Default is None.	None
max_iter	int	Maximum rounds of regrowing every view's active set; the fit has converged when the sets repeat. Default is 10.	10
tol	float	Tolerance on the change in the EY loss while refitting an active set. Default is 1e-6.	1e-06
random_state	int | None	Seed for the dense warm start. Default is None.	None

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Rounds of regrowing the active sets.

References

Chapman, J., Wang, H.-T., Wells, L., & Wiesner, J. (2021). CCA-Zoo: A collection of Regularized, Deep Learning based, Kernel, and Probabilistic CCA methods in a scikit-learn style framework. Journal of Open Source Software, 6(68), 3823.

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import OrthogonalMatchingPursuitCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((200, 20))
>>> X2 = rng.standard_normal((200, 15))
>>> model = OrthogonalMatchingPursuitCCA(
...     n_components=2, n_nonzero_coefs=5
... ).fit([X1, X2])

Source code in cca_zoo/sparse/_ompcca.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    n_nonzero_coefs: int | list[int] | None = None,
    max_iter: int = 10,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.n_nonzero_coefs = n_nonzero_coefs
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(
    views: list[ArrayLike], y: None = None
) -> OrthogonalMatchingPursuitCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
OrthogonalMatchingPursuitCCA	self.

Raises:

Type	Description
ValueError	If n_nonzero_coefs has the wrong length or is fewer than n_components.

Source code in cca_zoo/sparse/_ompcca.py

def fit(
    self, views: list[ArrayLike], y: None = None
) -> OrthogonalMatchingPursuitCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.

    Raises:
        ValueError: If ``n_nonzero_coefs`` has the wrong length or is fewer
            than ``n_components``.
    """
    views_ = self._setup_fit(views)
    budgets = self._resolve_n_nonzero_coefs(self.n_features_per_view_)
    rng = np.random.default_rng(self.random_state)
    # A dense start: all-zero scores are a stationary point of the EY
    # loss, with no gradient to select features by.
    weights = cheap_orthonormal_projection_weights(
        views_, self.n_components, None, rng
    )
    scores = [v @ w for v, w in zip(views_, weights)]
    active_sets: list[list[int]] = []
    converged = False
    for n_iter in range(1, self.max_iter + 1):
        previous, active_sets = (
            active_sets,
            [
                self._regrow(views_, weights, scores, i, budget)
                for i, budget in enumerate(budgets)
            ],
        )
        if active_sets == previous:
            converged = True
            break
    self.n_iter_: int = n_iter
    warn_if_not_converged(self, converged)
    # The EY optimum on the active sets: CCA on the selected features, of
    # which one in the span of the others takes no weight.
    independent = [_independent(v, a) for v, a in zip(views_, active_sets)]
    coefficients = penalised_basis_ey_closed_form(
        [v[:, a] - v[:, a].mean(axis=0) for v, a in zip(views_, independent)],
        self.n_components,
        [0.0] * self.n_views_,
    )
    for w, a, c in zip(weights, independent, coefficients):
        w[:] = 0.0
        w[a] = c
    self.weights_: list[np.ndarray] = weights
    self._fit_maps_and_importances(views_)
    return self

---

Alternating Least Squares

PMDCCA

PMDCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    l1_bound: float | list[float] = 1.0,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Sparse CCA by penalized matrix decomposition.

Maximises cross-view covariance subject to L1 and L2 constraints on each weight vector:

\[ \max_{\mathbf{w}_1, \mathbf{w}_2} \mathbf{w}_1^\top X_1^\top X_2 \mathbf{w}_2 \quad\text{subject to}\quad \|\mathbf{w}_i\|_1 \le b_i \sqrt{p_i},\ \|\mathbf{w}_i\|_2 = 1. \]

Each update soft-thresholds with the level found by bisection.

One of three sparse PLS power iterations, with :class:SpanCCA and :class:ParkhomenkoCCA: each multiplies by the other views' summed score, then thresholds, and they differ only in the threshold.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
l1_bound	float | list[float]	Bound \(b_i\) on the L1 norm, as a fraction of sqrt(n_features_i), in (0, 1]; 1 imposes no sparsity. Per-view. Default is 1.	1.0
max_iter	int	Maximum iterations per latent dimension. Default is 500.	500
tol	float	Convergence tolerance on the change in weights. Default is 1e-6.	1e-06
random_state	int | None	Seed for the start. Default is None.	None

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Most iterations run by any component.

References

Witten, D. M., Tibshirani, R., & Hastie, T. (2009). A penalized matrix decomposition, with applications to sparse principal components and canonical correlation analysis. Biostatistics, 10(3), 515-534.

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import PMDCCA
>>> rng = np.random.default_rng(0)
>>> X1, X2 = rng.standard_normal((50, 10)), rng.standard_normal((50, 8))
>>> model = PMDCCA(l1_bound=0.5, random_state=0).fit([X1, X2])

Source code in cca_zoo/sparse/_pmd.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    l1_bound: float | list[float] = 1.0,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.l1_bound = l1_bound
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> PMDCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
PMDCCA	self.

Source code in cca_zoo/sparse/_pmd.py

def fit(self, views: list[ArrayLike], y: None = None) -> PMDCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    bounds = [
        b * np.sqrt(p)
        for b, p in zip(
            perview_parameter("l1_bound", self.l1_bound, 1.0, self.n_views_),
            self.n_features_per_view_,
        )
    ]
    self.n_iter_: int = 0
    converged = True
    deflation = Deflation(views_, self.n_components)
    for deflated in deflation:
        w = pls_direction(deflated, rng)
        for n_iter in range(1, self.max_iter + 1):
            previous = [wi.copy() for wi in w]
            # Power iteration, soft-thresholded to each view's L1 bound.
            for i, (view, bound) in enumerate(zip(deflated, bounds)):
                w[i] = _threshold_to_l1_bound(
                    view.T @ others_score(deflated, w, i), bound
                )
            if max(np.linalg.norm(a - b) for a, b in zip(w, previous)) < self.tol:
                break
        else:
            converged = False
        self.n_iter_ = max(self.n_iter_, n_iter)
        deflation.record(w)
    warn_if_not_converged(self, converged)
    self.weights_: list[np.ndarray] = deflation.weights()
    self._fit_maps_and_importances(views_)
    return self

---

ADMMCCA

ADMMCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 0.1,
    rho: float = 1.0,
    max_iter: int = 500,
    admm_iter: int = 50,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Sparse CCA by linearised ADMM.

For view \(i\), with the other views' weights fixed, solves

\[ \max_{\mathbf{w}_i}\ \tfrac{1}{n} \mathbf{w}_i^\top X_i^\top \textstyle\sum_{j \ne i} X_j \mathbf{w}_j - \alpha_i \|\mathbf{w}_i\|_1 \quad\text{subject to}\quad \tfrac{1}{n} \|X_i \mathbf{w}_i\|_2^2 \le 1, \]

the paper's problem on \(X_i / \sqrt{n}\), so that the constraint is unit variance and alpha means the same at any sample size. It is solved by linearised ADMM on the split \(\mathbf{z}_i = X_i \mathbf{w}_i / \sqrt{n}\), so each step is a soft-threshold and a projection onto the unit ball, with step size \(n / (\rho \|X_i\|_{\mathrm{op}}^2)\). Further components use deflation rather than the paper's orthogonality constraint.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
alpha	float | list[float]	L1 penalty. Per-view. Default is 0.1.	0.1
rho	float	ADMM augmented-Lagrangian penalty. Default is 1.0.	1.0
max_iter	int	Maximum outer iterations per latent dimension. Default is 500.	500
admm_iter	int	Maximum ADMM iterations per view update. Default is 50.	50
tol	float	Convergence tolerance of both loops. Default is 1e-6.	1e-06
random_state	int | None	Seed for the start. Default is None.	None

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Most iterations run by any component.

References

Suo, X., Minden, V., Nelson, B., Tibshirani, R., & Saunders, M. (2017). Sparse canonical correlation analysis. arXiv:1705.10865.

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import ADMMCCA
>>> rng = np.random.default_rng(0)
>>> X1, X2 = rng.standard_normal((50, 10)), rng.standard_normal((50, 8))
>>> model = ADMMCCA(alpha=0.1, random_state=0).fit([X1, X2])

Source code in cca_zoo/sparse/_admm.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 0.1,
    rho: float = 1.0,
    max_iter: int = 500,
    admm_iter: int = 50,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.alpha = alpha
    self.rho = rho
    self.max_iter = max_iter
    self.admm_iter = admm_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> ADMMCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
ADMMCCA	self.

Source code in cca_zoo/sparse/_admm.py

def fit(self, views: list[ArrayLike], y: None = None) -> ADMMCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    alphas = perview_parameter("alpha", self.alpha, 0.1, self.n_views_)
    self.n_iter_: int = 0
    converged = True
    deflation = Deflation(views_, self.n_components)
    for deflated in deflation:
        w = pls_direction(deflated, rng)
        # The paper's problem on X / sqrt(n): unit-variance constraints.
        scaled = [v / np.sqrt(len(v)) for v in deflated]
        # The split z_i = X_i w_i and its scaled dual persist across the
        # outer iterations, as in the paper's Algorithm 1.
        z = [v @ wi for v, wi in zip(scaled, w)]
        dual = [np.zeros(len(v)) for v in scaled]
        steps = [1.0 / (self.rho * np.linalg.norm(v, ord=2) ** 2) for v in scaled]
        for n_iter in range(1, self.max_iter + 1):
            previous = [wi.copy() for wi in w]
            for i, (view, alpha, step) in enumerate(zip(scaled, alphas, steps)):
                others = sum(
                    v @ wj for j, (v, wj) in enumerate(zip(scaled, w)) if j != i
                )
                w[i], z[i], dual[i] = self._admm_block(
                    view, view.T @ others, alpha, step, w[i], z[i], dual[i]
                )
            if max(np.linalg.norm(a - b) for a, b in zip(w, previous)) < self.tol:
                break
        else:
            converged = False
        self.n_iter_ = max(self.n_iter_, n_iter)
        deflation.record(w)
    warn_if_not_converged(self, converged)
    self.weights_: list[np.ndarray] = deflation.weights()
    self._fit_maps_and_importances(views_)
    return self

---

IPLSCCA

IPLSCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 0.0,
    l1_ratio: float | list[float] = 1.0,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Sparse CCA by iterative penalised least squares.

Each view's weights solve an elastic-net regression onto the other views' summed score, then are rescaled to a unit-variance score, as in Waaijenborg et al.'s elastic-net CCA and Mai and Zhang's lasso version:

\[ \hat{\mathbf{w}}_i = \arg\min_{\mathbf{w}} \frac{1}{2n} \|X_i \mathbf{w} - \bar{\mathbf{s}}_{\neg i}\|_2^2 + \alpha_i \Bigl(r \|\mathbf{w}\|_1 + \tfrac{1-r}{2} \|\mathbf{w}\|_2^2\Bigr). \]

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
alpha	float | list[float]	Penalty strength. Per-view. Default is 0.	0.0
l1_ratio	float | list[float]	Share of the penalty that is L1: 1 is Mai and Zhang's lasso, and Waaijenborg et al. use an elastic net. Per-view. Default is 1.	1.0
max_iter	int	Maximum iterations per latent dimension. Default is 500.	500
tol	float	Convergence tolerance on the change in weights. Default is 1e-6.	1e-06
random_state	int | None	Seed for the start and the lasso's coordinate order. Default is None.	None

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Most iterations run by any component.

References

Waaijenborg, S., de Witt Hamer, P. C. V., & Zwinderman, A. H. (2008). Quantifying the association between gene expressions and DNA-markers by penalized canonical correlation analysis. Statistical Applications in Genetics and Molecular Biology, 7(1).

Mai, Q., & Zhang, X. (2019). An iterative penalized least squares approach to sparse canonical correlation analysis. Biometrics, 75(3), 734-744.

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import IPLSCCA
>>> rng = np.random.default_rng(0)
>>> X1, X2 = rng.standard_normal((50, 10)), rng.standard_normal((50, 8))
>>> model = IPLSCCA(alpha=0.1, random_state=0).fit([X1, X2])

Source code in cca_zoo/sparse/_ipls.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 0.0,
    l1_ratio: float | list[float] = 1.0,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.alpha = alpha
    self.l1_ratio = l1_ratio
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> IPLSCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
IPLSCCA	self.

Source code in cca_zoo/sparse/_ipls.py

def fit(self, views: list[ArrayLike], y: None = None) -> IPLSCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    regressions = [
        elastic_net(a, r, self.tol, self.random_state)
        for a, r in zip(
            perview_parameter("alpha", self.alpha, 0.0, self.n_views_),
            perview_parameter("l1_ratio", self.l1_ratio, 1.0, self.n_views_),
        )
    ]
    self.n_iter_: int = 0
    converged = True
    deflation = Deflation(views_, self.n_components)
    for deflated in deflation:
        w = pls_direction(deflated, rng)
        for n_iter in range(1, self.max_iter + 1):
            previous = [wi.copy() for wi in w]
            # Regress each view on the others' score, rescaled to unit variance.
            for i, (view, regression) in enumerate(zip(deflated, regressions)):
                regression.fit(view, others_score(deflated, w, i))
                w[i] = regression.coef_ / max(
                    (view @ regression.coef_).std(), 1e-12
                )
            if max(np.linalg.norm(a - b) for a, b in zip(w, previous)) < self.tol:
                break
        else:
            converged = False
        self.n_iter_ = max(self.n_iter_, n_iter)
        deflation.record(w)
    warn_if_not_converged(self, converged)
    self.weights_: list[np.ndarray] = deflation.weights()
    self._fit_maps_and_importances(views_)
    return self

---

SpanCCA

SpanCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    span: int | list[int] | None = None,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Sparse CCA by hard-thresholded alternating updates.

Each update keeps the span largest-magnitude weights. This is an alternating heuristic in the spirit of Asteris et al.'s SpanCCA, not their randomised low-rank search.

One of three sparse PLS power iterations, with :class:PMDCCA and :class:ParkhomenkoCCA: each multiplies by the other views' summed score, then thresholds, and they differ only in the threshold.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
span	int | list[int] | None	Number of nonzero weights kept; None keeps all. Per-view. Default is None.	None
max_iter	int	Maximum iterations per latent dimension. Default is 500.	500
tol	float	Convergence tolerance on the change in weights. Default is 1e-6.	1e-06
random_state	int | None	Seed for the start. Default is None.	None

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Most iterations run by any component.

References

Asteris, M., Kyrillidis, A., Koyejo, O., & Poldrack, R. (2016). A simple and provable algorithm for sparse diagonal CCA. ICML.

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import SpanCCA
>>> rng = np.random.default_rng(0)
>>> X1, X2 = rng.standard_normal((50, 10)), rng.standard_normal((50, 8))
>>> model = SpanCCA(span=5, random_state=0).fit([X1, X2])

Source code in cca_zoo/sparse/_span.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    span: int | list[int] | None = None,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.span = span
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> SpanCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
SpanCCA	self.

Source code in cca_zoo/sparse/_span.py

def fit(self, views: list[ArrayLike], y: None = None) -> SpanCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    spans = (
        self.n_features_per_view_
        if self.span is None
        else [
            int(s) for s in perview_parameter("span", self.span, 1, self.n_views_)
        ]
    )
    self.n_iter_: int = 0
    converged = True
    deflation = Deflation(views_, self.n_components)
    for deflated in deflation:
        w = pls_direction(deflated, rng)
        for n_iter in range(1, self.max_iter + 1):
            previous = [wi.copy() for wi in w]
            # Power iteration keeping each view's `span` largest weights.
            for i, (view, span) in enumerate(zip(deflated, spans)):
                raw = view.T @ others_score(deflated, w, i)
                if span < len(raw):
                    raw[np.abs(raw) < np.sort(np.abs(raw))[-span]] = 0.0
                w[i] = raw / max(np.linalg.norm(raw), 1e-12)
            if max(np.linalg.norm(a - b) for a, b in zip(w, previous)) < self.tol:
                break
        else:
            converged = False
        self.n_iter_ = max(self.n_iter_, n_iter)
        deflation.record(w)
    warn_if_not_converged(self, converged)
    self.weights_: list[np.ndarray] = deflation.weights()
    self._fit_maps_and_importances(views_)
    return self

---

ParkhomenkoCCA

ParkhomenkoCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 0.1,
    max_iter: int = 500,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: BaseModel

Sparse CCA by soft-thresholded power iteration on standardised views.

Uses the paper's diagonal approximation to the within-view covariances, which amounts to standardising each feature, then iterates

\[ \mathbf{w}_i \leftarrow S_{\alpha_i}\bigl(\tfrac{1}{n} \tilde X_i^\top \bar{\mathbf{s}}_{\neg i}\bigr), \]

with \(S_\alpha\) the soft-threshold, \(\tilde X_i\) the standardised view and \(\bar{\mathbf{s}}_{\neg i}\) the other views' summed score at unit variance: each feature's correlation with that score is soft-thresholded at \(\alpha_i\). Later components deflate the standardised views, so that at alpha=0 this is :class:~sklearn.cross_decomposition.PLSCanonical with scale=True. Weights are returned on the original feature scale.

One of three sparse PLS power iterations, with :class:PMDCCA and :class:SpanCCA: each multiplies by the other views' summed score, then thresholds, and they differ only in the threshold.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
alpha	float | list[float]	Soft-threshold on each feature's correlation with the other views' score; 1 or more zeroes every weight. Per-view. Default is 0.1.	0.1
max_iter	int	Maximum iterations per latent dimension. Default is 500.	500
tol	float	Convergence tolerance on the change in weights. Default is 1e-6.	1e-06
random_state	int | None	Seed for the start. Default is None.	None

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Most iterations run by any component.

References

Parkhomenko, E., Tritchler, D., & Beyene, J. (2009). Sparse canonical correlation analysis with application to genomic data integration. Statistical Applications in Genetics and Molecular Biology, 8(1).

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import ParkhomenkoCCA
>>> rng = np.random.default_rng(0)
>>> X1, X2 = rng.standard_normal((50, 10)), rng.standard_normal((50, 8))
>>> model = ParkhomenkoCCA(alpha=0.1, random_state=0).fit([X1, X2])

Source code in cca_zoo/sparse/_parkhomenko.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    alpha: float | list[float] = 0.1,
    max_iter: int = 500,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.alpha = alpha
    self.max_iter = max_iter
    self.tol = tol
    self.random_state = random_state

fit

fit(
    views: list[ArrayLike], y: None = None
) -> ParkhomenkoCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
ParkhomenkoCCA	self.

Source code in cca_zoo/sparse/_parkhomenko.py

def fit(self, views: list[ArrayLike], y: None = None) -> ParkhomenkoCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    alphas = perview_parameter("alpha", self.alpha, 0.1, self.n_views_)
    self.n_iter_: int = 0
    converged = True
    # The diagonal approximation to each covariance: standardise, once,
    # as sklearn's PLSCanonical(scale=True) does.
    scales = [np.where(s < 1e-12, 1.0, s) for s in (v.std(axis=0) for v in views_)]
    deflation = Deflation(
        [v / s for v, s in zip(views_, scales)], self.n_components
    )
    for deflated in deflation:
        w = pls_direction(deflated, rng)
        for n_iter in range(1, self.max_iter + 1):
            previous = [wi.copy() for wi in w]
            # Each feature's covariance with the others' score, its
            # correlation on the first component, soft-thresholded.
            for i, (view, alpha) in enumerate(zip(deflated, alphas)):
                covariances = view.T @ others_score(deflated, w, i) / len(view)
                raw = soft_threshold(covariances, alpha)
                w[i] = raw / max(np.linalg.norm(raw), 1e-12)
            if max(np.linalg.norm(a - b) for a, b in zip(w, previous)) < self.tol:
                break
        else:
            converged = False
        self.n_iter_ = max(self.n_iter_, n_iter)
        deflation.record(w)
    warn_if_not_converged(self, converged)
    # Back to the original feature scale.
    self.weights_: list[np.ndarray] = [
        w / s[:, None] for w, s in zip(deflation.weights(), scales)
    ]
    self._fit_maps_and_importances(views_)
    return self

---

SAR

SAR(
    n_components: int = 1,
    *,
    center: bool = True,
    n_alphas: int = 100,
    max_iter: int = 500,
    tol: float = 1e-06,
)

Bases: BaseModel

Sparse alternating regression with BIC-selected sparsity.

CCA as alternating regressions of each view's score on the other views' summed score, each a lasso whose penalty is chosen by BIC. Latent dimensions after the first are found on deflated views, then re-fitted against the original views (Wilms and Croux, Section 3). Each component starts from the ridge-regularised CCA direction, from which the first lasso finds a target the other views share. The extension beyond two views is this implementation's own.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
n_alphas	int	Penalties on each lasso path. Default is 100.	100
max_iter	int	Maximum iterations per latent dimension. Default is 500.	500
tol	float	Convergence tolerance of the alternating loop and each lasso path. Default is 1e-6.	1e-06

Attributes:

Name	Type	Description
weights_		Weight matrix of each view, shape (n_features_i, n_components).
n_iter_		Most iterations run by any component.

References

Wilms, I., & Croux, C. (2015). Sparse canonical correlation analysis from a predictive point of view. Biometrical Journal, 57(5), 834-851.

Examples:

>>> import numpy as np
>>> from cca_zoo.sparse import SAR
>>> rng = np.random.default_rng(0)
>>> latent = rng.standard_normal(50)
>>> X1 = np.column_stack([latent, rng.standard_normal((50, 9))])
>>> X2 = np.column_stack([latent, rng.standard_normal((50, 7))])
>>> model = SAR().fit([X1, X2])

Source code in cca_zoo/sparse/_sar.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    n_alphas: int = 100,
    max_iter: int = 500,
    tol: float = 1e-6,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.n_alphas = n_alphas
    self.max_iter = max_iter
    self.tol = tol

fit

fit(views: list[ArrayLike], y: None = None) -> SAR

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
SAR	self.

Source code in cca_zoo/sparse/_sar.py

def fit(self, views: list[ArrayLike], y: None = None) -> SAR:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    self.weights_: list[np.ndarray] = [
        np.zeros((p, self.n_components)) for p in self.n_features_per_view_
    ]
    self.n_iter_: int = 0
    converged = True
    deflation = Deflation(views_, self.n_components)
    for d, deflated in enumerate(deflation):
        w = ridge_cca_direction(deflated)
        for n_iter in range(1, self.max_iter + 1):
            previous = [wi.copy() for wi in w]
            # BIC-selected lasso of each view on the others' score.
            for i, view in enumerate(deflated):
                w[i] = _unit_norm(
                    self._bic_lasso(view, others_score(deflated, w, i))
                )
            if max(np.linalg.norm(a - b) for a, b in zip(w, previous)) < self.tol:
                break
        else:
            converged = False
        self.n_iter_ = max(self.n_iter_, n_iter)
        # A later component's scores, re-fitted by lasso on the original
        # views, where the paper keeps its weights sparse.
        for i, (view, deflated_view, wi) in enumerate(zip(views_, deflated, w)):
            self.weights_[i][:, d] = (
                wi
                if d == 0
                else _unit_norm(self._bic_lasso(view, deflated_view @ wi))
            )
        deflation.record(w)
    warn_if_not_converged(self, converged)
    self._fit_maps_and_importances(views_)
    return self