cca_zoo.gp

Gaussian-process (GP) nonlinear CCA methods, fit in closed form by one generalized eigendecomposition. Built entirely on scikit-learn's own GaussianProcessRegressor/RBF/ConstantKernel/KernelCenterer — no optional dependency required.

---

GaussianProcessCCA

GaussianProcessCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    kernel: Kernel | list[Kernel | None] | None = None,
    alpha: float | list[float] = 0.01,
    n_inducing: int | list[int | None] | None = None,
    random_state: int | None = None,
)

Bases: BaseModel

Nonlinear CCA with Gaussian-process encoders.

Each view's encoder is \(f_i(x) = k(x, U_i)^\top B_i\) for a kernel \(k\) over the whole feature vector and inducing points \(U_i\), so it can represent within-view interactions. Every view's \(B_i\) minimises the EY loss (:mod:cca_zoo._utils._ey) plus the RKHS-norm penalty \(\tfrac12 \alpha_i \operatorname{tr}(B_i^\top K_i B_i)\). On the Nyström features of the inducing points the penalty is a ridge, and the global minimiser is a generalized eigenproblem, solved in closed form. Fitting costs \(O(n m^2 + m^3)\) for \(m\) inducing points. Kernel hyperparameters are fixed; tune them by cross-validation.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
kernel	Kernel | list[Kernel | None] | None	A kernel, cloned per view, or one per view; None uses ConstantKernel(1.0) * RBF(sqrt(n_features * X.var() / 2)), the length scale of sklearn's gamma="scale". Default is None.	None
alpha	float | list[float]	RKHS-norm penalty, also the noise level of the posterior variance. Per-view. Default is 0.01.	0.01
n_inducing	int | list[int | None] | None	Number of inducing points; None, or at least n_samples, uses every training row. Per-view. Default is None.	None
random_state	int | None	Seed for the inducing points. Default is None.	None

Attributes:

Name	Type	Description
encoders_		Fitted per-view encoders, with inducing_, kernel_ and coef_.

References

Chapman, J., Wang, H.-T., Wells, L., & Wiesner, J. (2021). CCA-Zoo: A collection of Regularized, Deep Learning based, Kernel, and Probabilistic CCA methods in a scikit-learn style framework. Journal of Open Source Software, 6(68), 3823. Chapman, J., Wells, L., & Lawry Aguila, A. (2024). Unconstrained Stochastic CCA: Unifying Multiview and Self-Supervised Learning. arXiv:2310.01012. Rasmussen, C. E., & Williams, C. K. I. (2006). Gaussian Processes for Machine Learning. MIT Press.

Quiñonero-Candela, J., & Rasmussen, C. E. (2005). A Unifying View of Sparse Approximate Gaussian Process Regression. Journal of Machine Learning Research, 6, 1939-1959.

Examples:

>>> import numpy as np
>>> from cca_zoo.gp import GaussianProcessCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((100, 3))
>>> X2 = rng.standard_normal((100, 3))
>>> model = GaussianProcessCCA(alpha=[0.01, 0.1]).fit([X1, X2])
>>> means, stds = model.transform([X1, X2]), model.posterior_std([X1, X2])

Source code in cca_zoo/gp/_gpcca.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    kernel: Kernel | list[Kernel | None] | None = None,
    alpha: float | list[float] = 0.01,
    n_inducing: int | list[int | None] | None = None,
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.kernel = kernel
    self.alpha = alpha
    self.n_inducing = n_inducing
    self.random_state = random_state

fit

fit(
    views: list[ArrayLike], y: None = None
) -> GaussianProcessCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
GaussianProcessCCA	self.

Source code in cca_zoo/gp/_gpcca.py

def fit(self, views: list[ArrayLike], y: None = None) -> GaussianProcessCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    k = self.n_components

    kernel_ = perview_parameter("kernel", self.kernel, None, self.n_views_)
    alpha_ = perview_parameter("alpha", self.alpha, 0.01, self.n_views_)
    n_inducing_ = perview_parameter(
        "n_inducing", self.n_inducing, None, self.n_views_
    )

    encoders = [
        _GpEncoder(
            X,
            k,
            clone(kern) if kern is not None else _default_kernel(X),
            a,
            n_ind,
            self.random_state,
        )
        for X, kern, a, n_ind in zip(views_, kernel_, alpha_, n_inducing_)
    ]

    features = [enc.features(X) for enc, X in zip(encoders, views_)]
    # Inducing points other than the training rows leave the features
    # uncentred on the training data: the scores are centred there, as
    # the EY loss fitted them.
    offsets = [f.mean(axis=0) for f in features]
    coefficients = penalised_basis_ey_closed_form(
        [f - o for f, o in zip(features, offsets)], k, alpha_
    )
    for enc, coef, offset in zip(encoders, coefficients, offsets):
        enc.coef_ = enc._nystroem @ coef
        enc.offset_ = offset @ coef

    self.encoders_: list[_GpEncoder] = encoders
    self._fit_maps_and_importances(views_)
    return self

posterior_std

posterior_std(views: list[ArrayLike]) -> list[np.ndarray]

Posterior standard deviation of each view's latent scores.

The uncertainty of :meth:transform's scores, larger away from the training data.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required

Returns:

Type	Description
list[ndarray]	One array of shape (n_samples, n_components) per view.

Source code in cca_zoo/gp/_gpcca.py

def posterior_std(self, views: list[ArrayLike]) -> list[np.ndarray]:
    """Posterior standard deviation of each view's latent scores.

    The uncertainty of :meth:`transform`'s scores, larger away from the
    training data.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.

    Returns:
        One array of shape (n_samples, n_components) per view.
    """
    validated = self._check_views(views)
    return [
        enc.predict_std(v - m)
        for v, m, enc in zip(validated, self.means_, self.encoders_)
    ]