cca_zoo.stochastic¶

Mini-batch CCA methods, for data too large to fit a full-batch gradient step in memory.

StochasticCCAEY ¶

StochasticCCAEY(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float = 0.0,
    learning_rate: float = 0.2,
    batch_size: int | None = None,
    max_iter: int = 1000,
    n_iter_no_change: int = 5,
    tol: float = 1e-06,
    random_state: int | None = None,
)

Bases: CCAEY

Multiview CCA by mini-batch SGD on the Eckart-Young loss.

The loss of :class:~cca_zoo.linear.gradient.CCAEY, minimised as :class:~sklearn.linear_model.SGDRegressor does with learning_rate="adaptive": each epoch shuffles the data and takes one step per mini-batch, and whenever the epoch loss stalls the step is divided by 5, so that the fit settles rather than hovering at the noise of its mini-batches. For data too large for full-batch gradients.

Parameters:

Name	Type	Description	Default
`n_components`	`int`	Number of latent dimensions. Default is 1.	`1`
`center`	`bool`	Whether to centre each view. Default is True.	`True`
`shrinkage`	`float`	Shrinkage of each view's covariance towards the identity, in `[0, 1]`: 0 is CCA and 1 is PLS. Default is 0.	`0.0`
`learning_rate`	`float`	Step size as a fraction of `1 / L_i` for view i, where `L_i` is the largest eigenvalue of its constraint matrix `(1 - shrinkage) cov_i + shrinkage I`, so that the default suits views in any units. The initial step. Default is 0.2.	`0.2`
`batch_size`	`int \| None`	Mini-batch size; None uses all samples. Default is None.	`None`
`max_iter`	`int`	Number of epochs. Default is 1000.	`1000`
`n_iter_no_change`	`int`	Epochs without an improvement of `tol` on the best epoch loss, the mean of its mini-batches' losses, before the step is divided by 5; the fit stops once the step is below 1e-6. Default is 5, as in sklearn's SGD.	`5`
`tol`	`float`	Improvement in the epoch loss that counts, as for `n_iter_no_change`. Default is 1e-6.	`1e-06`
`random_state`	`int \| None`	Seed for the shuffling and initial weights. Default is None.	`None`

Attributes:

Name	Type	Description
`weights_`		Weight matrix of each view, shape (n_features_i, n_components).
`n_iter_`		Epochs run.

References

Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012.

Examples:

>>> import numpy as np
>>> from cca_zoo.stochastic import StochasticCCAEY
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((5000, 200))
>>> X2 = rng.standard_normal((5000, 150))
>>> model = StochasticCCAEY(n_components=4, batch_size=128, random_state=0)
>>> model = model.fit([X1, X2])

Source code in cca_zoo/stochastic/_stochastic_cca_ey.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float = 0.0,
    learning_rate: float = 0.2,
    batch_size: int | None = None,
    max_iter: int = 1000,
    n_iter_no_change: int = 5,
    tol: float = 1e-6,
    random_state: int | None = None,
) -> None:
    super().__init__(
        n_components=n_components,
        center=center,
        shrinkage=shrinkage,
        max_iter=max_iter,
        tol=tol,
        random_state=random_state,
    )
    self.learning_rate = learning_rate
    self.batch_size = batch_size
    self.n_iter_no_change = n_iter_no_change

fit ¶

fit(
    views: list[ArrayLike], y: None = None
) -> StochasticCCAEY

Fit the model.

Parameters:

Name	Type	Description	Default
`views`	`list[ArrayLike]`	Arrays of shape (n_samples, n_features_i), one per view.	required
`y`	`None`	Ignored.	`None`

Returns:

Type	Description
`StochasticCCAEY`	self.

Raises:

Type	Description
`ValueError`	If the updates diverge.

Source code in cca_zoo/stochastic/_stochastic_cca_ey.py

def fit(self, views: list[ArrayLike], y: None = None) -> StochasticCCAEY:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.

    Raises:
        ValueError: If the updates diverge.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    rng = np.random.default_rng(self.random_state)
    self.weights_ = self._fit_sgd(views_, rng)
    self._fit_maps_and_importances(views_)
    return self