cca_zoo.nonparametric

Kernel- and graph-based nonparametric CCA methods.

---

KCCA

KCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    kernel: str | list[str] = "linear",
    gamma: float | list[float | None] | None = None,
    degree: float | list[float] = 3,
    coef0: float | list[float] = 1.0,
    kernel_params: dict[str, object]
    | list[dict[str, object]]
    | None = None,
)

Bases: _BaseKernelModel

Kernel CCA for two or more views.

:class:~cca_zoo.linear.MCCA in each view's kernel feature space. With centred kernels \(K_i\) the dual problem is \(A \alpha = \lambda B \alpha\), with between-view blocks \(K_i K_j / (n - 1)\) and within-view blocks \((1 - c_i) K_i^2 / (n - 1) + c_i K_i\), the covariance and the squared norm of the feature-space direction.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
shrinkage	float | list[float]	Shrinkage of each view's covariance in the kernel feature space towards the identity, in [0, 1]: 0 is kernel CCA and 1 is kernel PLS. Per-view. Default is 0.1: an RBF kernel's feature space has a dimension per sample, where unshrunk kernel CCA correlates any two views perfectly.	0.1
kernel	str | list[str]	Kernel name or callable for :func:~sklearn.metrics.pairwise_kernels. Per-view. Default is "linear".	'linear'
gamma	float | list[float | None] | None	Kernel coefficient for RBF, polynomial and sigmoid kernels. Per-view. Default is None.	None
degree	float | list[float]	Polynomial kernel degree. Per-view. Default is 3.	3
coef0	float | list[float]	Polynomial and sigmoid kernel constant. Per-view. Default is 1.	1.0
kernel_params	dict[str, object] | list[dict[str, object]] | None	Extra kernel keyword arguments. Per-view. Default is None.	None

Attributes:

Name	Type	Description
weights_	list[ndarray]	Dual coefficients of each view, shape (n_samples, n_components).
views_fit_	list[ndarray]	The centred training views, against which the kernel of a new view is evaluated and centred.

References

Hardoon, D. R., Szedmak, S., & Shawe-Taylor, J. (2004). Canonical correlation analysis: An overview with application to learning methods. Neural Computation, 16(12), 2639-2664.

Examples:

>>> import numpy as np
>>> from cca_zoo.nonparametric import KCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((30, 5))
>>> X2 = rng.standard_normal((30, 5))
>>> model = KCCA(n_components=2, kernel="rbf", shrinkage=0.1).fit([X1, X2])

Source code in cca_zoo/nonparametric/_kcca.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    kernel: str | list[str] = "linear",
    gamma: float | list[float | None] | None = None,
    degree: float | list[float] = 3,
    coef0: float | list[float] = 1.0,
    kernel_params: dict[str, object] | list[dict[str, object]] | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.kernel = kernel
    self.gamma = gamma
    self.degree = degree
    self.coef0 = coef0
    self.kernel_params = kernel_params

fit

fit(views: list[ArrayLike], y: None = None) -> KCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
KCCA	self.

Source code in cca_zoo/nonparametric/_kcca.py

def fit(self, views: list[ArrayLike], y: None = None) -> KCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    linear = MCCA(self.n_components, center=False, shrinkage=self.shrinkage)
    features, projections = self._feature_maps(views_)
    self._set_weights(projections, linear.fit(features).weights_)
    self._fit_maps_and_importances(views_)
    return self

---

KGCCA

KGCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    kernel: str | list[str] = "linear",
    gamma: float | list[float | None] | None = None,
    degree: float | list[float] = 3,
    coef0: float | list[float] = 1.0,
    kernel_params: dict[str, object]
    | list[dict[str, object]]
    | None = None,
    view_weights: list[float] | None = None,
)

Bases: _BaseKernelModel

Kernel generalized CCA.

:class:~cca_zoo.linear.GCCA in each view's kernel feature space. With centred kernels \(K_i\), \(T\) holds the top eigenvectors of \(\sum_i \mu_i K_i ((1 - c_i) K_i^2 / (n - 1) + c_i K_i)^{-1} K_i\).

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
shrinkage	float | list[float]	Shrinkage of each view's covariance in the kernel feature space towards the identity, in [0, 1]: 0 is kernel CCA and 1 is kernel PLS. Per-view. Default is 0.1, as for :class:~cca_zoo.nonparametric.KCCA.	0.1
kernel	str | list[str]	Kernel name or callable for :func:~sklearn.metrics.pairwise_kernels. Per-view. Default is "linear".	'linear'
gamma	float | list[float | None] | None	Kernel coefficient for RBF, polynomial and sigmoid kernels. Per-view. Default is None.	None
degree	float | list[float]	Polynomial kernel degree. Per-view. Default is 3.	3
coef0	float | list[float]	Polynomial and sigmoid kernel constant. Per-view. Default is 1.	1.0
kernel_params	dict[str, object] | list[dict[str, object]] | None	Extra kernel keyword arguments. Per-view. Default is None.	None
view_weights	list[float] | None	Weight of each view; None weights them equally. Default is None.	None

Attributes:

Name	Type	Description
weights_	list[ndarray]	Dual coefficients of each view, shape (n_samples, n_components).
views_fit_	list[ndarray]	The centred training views, against which the kernel of a new view is evaluated and centred.

References

Tenenhaus, A., Philippe, C., & Frouin, V. (2015). Kernel generalized canonical correlation analysis. Computational Statistics & Data Analysis, 90, 114-131.

Examples:

>>> import numpy as np
>>> from cca_zoo.nonparametric import KGCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((30, 5))
>>> X2 = rng.standard_normal((30, 5))
>>> X3 = rng.standard_normal((30, 5))
>>> model = KGCCA(n_components=2, kernel="rbf").fit([X1, X2, X3])

Source code in cca_zoo/nonparametric/_kgcca.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    kernel: str | list[str] = "linear",
    gamma: float | list[float | None] | None = None,
    degree: float | list[float] = 3,
    coef0: float | list[float] = 1.0,
    kernel_params: dict[str, object] | list[dict[str, object]] | None = None,
    view_weights: list[float] | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.kernel = kernel
    self.gamma = gamma
    self.degree = degree
    self.coef0 = coef0
    self.kernel_params = kernel_params
    self.view_weights = view_weights

fit

fit(views: list[ArrayLike], y: None = None) -> KGCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
KGCCA	self.

Source code in cca_zoo/nonparametric/_kgcca.py

def fit(self, views: list[ArrayLike], y: None = None) -> KGCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    linear = GCCA(
        self.n_components,
        center=False,
        shrinkage=self.shrinkage,
        view_weights=self.view_weights,
    )
    features, projections = self._feature_maps(views_)
    self._set_weights(projections, linear.fit(features).weights_)
    self._fit_maps_and_importances(views_)
    return self

---

KTCCA

KTCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    kernel: str | list[str] = "linear",
    gamma: float | list[float | None] | None = None,
    degree: float | list[float] = 3,
    coef0: float | list[float] = 1.0,
    kernel_params: dict[str, object]
    | list[dict[str, object]]
    | None = None,
    init: str = "svd",
    random_state: int | None = None,
)

Bases: _BaseKernelModel

Kernel tensor CCA.

:class:~cca_zoo.linear.TCCA in each view's kernel feature space: PARAFAC of the cross-moment tensor of the whitened feature maps.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
shrinkage	float | list[float]	Shrinkage of each view's covariance in the kernel feature space towards the identity, in [0, 1]: 0 is kernel CCA and 1 is kernel PLS. Per-view. Default is 0.1, as for :class:~cca_zoo.nonparametric.KCCA.	0.1
kernel	str | list[str]	Kernel name or callable for :func:~sklearn.metrics.pairwise_kernels. Per-view. Default is "linear".	'linear'
gamma	float | list[float | None] | None	Kernel coefficient for RBF, polynomial and sigmoid kernels. Per-view. Default is None.	None
degree	float | list[float]	Polynomial kernel degree. Per-view. Default is 3.	3
coef0	float | list[float]	Polynomial and sigmoid kernel constant. Per-view. Default is 1.	1.0
kernel_params	dict[str, object] | list[dict[str, object]] | None	Extra kernel keyword arguments. Per-view. Default is None.	None
init	str	PARAFAC's initialisation, "svd" or "random", as for :class:~cca_zoo.linear.TCCA. Default is "svd".	'svd'
random_state	int | None	Seed for init="random". Default is None.	None

Attributes:

Name	Type	Description
weights_	list[ndarray]	Dual coefficients of each view, shape (n_samples, n_components).
views_fit_	list[ndarray]	The centred training views, against which the kernel of a new view is evaluated and centred.

References

Luo, Y., Tao, D., Ramamohanarao, K., Xu, C., & Wen, Y. (2015). Tensor canonical correlation analysis for multi-view dimension reduction. IEEE Transactions on Knowledge and Data Engineering, 27(11), 3111-3124.

Examples:

>>> import numpy as np
>>> from cca_zoo.nonparametric import KTCCA
>>> rng = np.random.default_rng(0)
>>> X1 = rng.standard_normal((20, 5))
>>> X2 = rng.standard_normal((20, 5))
>>> X3 = rng.standard_normal((20, 5))
>>> model = KTCCA(random_state=0).fit([X1, X2, X3])

Source code in cca_zoo/nonparametric/_ktcca.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    kernel: str | list[str] = "linear",
    gamma: float | list[float | None] | None = None,
    degree: float | list[float] = 3,
    coef0: float | list[float] = 1.0,
    kernel_params: dict[str, object] | list[dict[str, object]] | None = None,
    init: str = "svd",
    random_state: int | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.kernel = kernel
    self.gamma = gamma
    self.degree = degree
    self.coef0 = coef0
    self.kernel_params = kernel_params
    self.init = init
    self.random_state = random_state

fit

fit(views: list[ArrayLike], y: None = None) -> KTCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
KTCCA	self.

Source code in cca_zoo/nonparametric/_ktcca.py

def fit(self, views: list[ArrayLike], y: None = None) -> KTCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_: list[np.ndarray] = self._setup_fit(views)
    linear = TCCA(
        self.n_components,
        center=False,
        shrinkage=self.shrinkage,
        init=self.init,
        random_state=self.random_state,
    )
    features, projections = self._feature_maps(views_)
    self._set_weights(projections, linear.fit(features).weights_)
    self._fit_maps_and_importances(views_)
    return self

---

ManifoldCCA

ManifoldCCA(
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    method: str = "laplacian",
    n_neighbors: int | list[int] = 10,
    affinity: str | list[str] = "nearest_neighbors",
    gamma: float | list[float | None] | None = None,
    reg: float | list[float] = 0.001,
    n_operator_components: int
    | list[int | None]
    | None = None,
)

Bases: BaseModel

Multiview CCA with a graph-operator constraint per view.

Maximises cross-view covariance of the training embeddings, restricted to each view's smoothest eigenvectors of a graph Laplacian or LLE operator M_i built from that view's neighbourhoods, subject to a blend of each view's variance and its roughness on its graph:

\[ \max_{Z_1, \dots, Z_M} \sum_{i \neq j} \operatorname{Cov}(Z_i, Z_j) \quad \text{subject to} \quad \sum_i (1 - c_i) \operatorname{Cov}(Z_i, Z_i) + c_i Z_i^\top M_i Z_i = I. \]

The roughness is rescaled so its trace over the kept eigenvectors equals their variance's, so that shrinkage means the same for either operator. This is :class:~cca_zoo.linear.MCCA with shrinkage=c on each view's operator eigenvectors scaled by the inverse square root of their rescaled eigenvalues: shrinkage=0 is CCA between the views' smooth graph coordinates, and shrinkage=1 PLS, which trades correlation for smoothness. With identical views and shrinkage > 0 it reduces to the view's :class:~sklearn.manifold.LocallyLinearEmbedding or, up to centring, :class:~sklearn.manifold.SpectralEmbedding. New points are embedded with each method's standard out-of-sample extension: LLE barycentric weights for method="lle", the Nystrom extension for method="laplacian"; either returns a training point's own embedding. Fitting solves a dense (n M) x (n M) eigenproblem, so it suits moderate sample sizes.

Parameters:

Name	Type	Description	Default
n_components	int	Number of latent dimensions. Default is 1.	1
center	bool	Whether to centre each view. Default is True.	True
shrinkage	float | list[float]	Blend of each view's constraint from its embedding's variance (0, CCA) to its roughness on the view's graph (1, PLS), as :class:~cca_zoo.linear.MCCA's shrinkage. Per-view. Default is 0.1.	0.1
method	str	"laplacian" or "lle", for every view. Default is "laplacian".	'laplacian'
n_neighbors	int | list[int]	Neighbours in each view's graph. Per-view. Default is 10.	10
affinity	str | list[str]	"nearest_neighbors" or "rbf", for method="laplacian". Per-view. Default is "nearest_neighbors".	'nearest_neighbors'
gamma	float | list[float | None] | None	RBF kernel coefficient for affinity="rbf"; None uses 1 / n_features. Per-view. Default is None.	None
reg	float | list[float]	Regularisation of each local reconstruction for method="lle". Per-view. Default is 1e-3.	0.001
n_operator_components	int | list[int | None] | None	Smoothest operator eigenvectors kept per view before the joint solve; fewer is stronger regularisation. None uses max(4 * n_components, 40), clipped to n_samples - 1. For method="laplacian", eigenvectors with eigenvalue above 0.95, which the Nystrom extension cannot carry to new points, are dropped. Per-view. Default is None.	None

Attributes:

Name	Type	Description
embedding_		Training embedding of each view, shape (n_samples, n_components).
views_fit_		The centred training views, from which a new view's embedding is interpolated.

References

Roweis, S. T., & Saul, L. K. (2000). Nonlinear dimensionality reduction by locally linear embedding. Science, 290(5500), 2323-2326.

Belkin, M., & Niyogi, P. (2003). Laplacian eigenmaps for dimensionality reduction and data representation. Neural Computation, 15(6), 1373-1396.

Examples:

>>> import numpy as np
>>> from cca_zoo.nonparametric import ManifoldCCA
>>> rng = np.random.default_rng(0)
>>> X1, X2 = rng.standard_normal((60, 8)), rng.standard_normal((60, 6))
>>> model = ManifoldCCA(n_neighbors=8).fit([X1, X2])
>>> Z1, Z2 = model.transform([X1, X2])

Source code in cca_zoo/nonparametric/_manifold_cca.py

def __init__(
    self,
    n_components: int = 1,
    *,
    center: bool = True,
    shrinkage: float | list[float] = 0.1,
    method: str = "laplacian",
    n_neighbors: int | list[int] = 10,
    affinity: str | list[str] = "nearest_neighbors",
    gamma: float | list[float | None] | None = None,
    reg: float | list[float] = 1e-3,
    n_operator_components: int | list[int | None] | None = None,
) -> None:
    super().__init__(n_components=n_components, center=center)
    self.shrinkage = shrinkage
    self.method = method
    self.n_neighbors = n_neighbors
    self.affinity = affinity
    self.gamma = gamma
    self.reg = reg
    self.n_operator_components = n_operator_components

fit

fit(views: list[ArrayLike], y: None = None) -> ManifoldCCA

Fit the model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	Arrays of shape (n_samples, n_features_i), one per view.	required
y	None	Ignored.	None

Returns:

Type	Description
ManifoldCCA	self.

Source code in cca_zoo/nonparametric/_manifold_cca.py

def fit(self, views: list[ArrayLike], y: None = None) -> ManifoldCCA:
    """Fit the model.

    Args:
        views: Arrays of shape (n_samples, n_features_i), one per view.
        y: Ignored.

    Returns:
        self.
    """
    views_ = self._setup_fit(views)
    n = self.n_samples_
    m = self.n_views_

    c_ = perview_parameter("shrinkage", self.shrinkage, 0.1, m)
    n_neighbors_ = perview_parameter("n_neighbors", self.n_neighbors, 10, m)
    affinity_ = perview_parameter("affinity", self.affinity, "nearest_neighbors", m)
    gamma_ = perview_parameter("gamma", self.gamma, None, m)
    reg_ = perview_parameter("reg", self.reg, 1e-3, m)
    n_operator_components_ = perview_parameter(
        "n_operator_components", self.n_operator_components, None, m
    )

    C = _centering_matrix(n)

    k_ops = [
        self._resolve_n_operator_components(c, n) for c in n_operator_components_
    ]
    full_bases = []
    eigenvalue_blocks = []
    laplacian_parts: list[_LaplacianViewState] = []
    lle_nn: list[NearestNeighbors] = []
    for v, nn_i, aff_i, gamma_i, reg_i, k_op in zip(
        views_, n_neighbors_, affinity_, gamma_, reg_, k_ops
    ):
        if self.method == "laplacian":
            operator, state = _laplacian_parts(v, nn_i, aff_i, gamma_i)
            null_vector = np.sqrt(state.degrees)
            laplacian_parts.append(state)
        else:
            operator = _lle_operator(v, nn_i, reg_i)
            null_vector = np.ones(n)
            lle_nn.append(NearestNeighbors(n_neighbors=nn_i + 1).fit(v))

        P = _orthonormal_complement(null_vector)
        basis, eigenvalues = _smooth_basis(P.T @ operator @ P, k_op, self._EPS)
        if self.method == "laplacian":
            basis, eigenvalues = _extendable(basis, eigenvalues, self.n_components)
        full_bases.append(P @ basis)
        eigenvalue_blocks.append(eigenvalues)

    # Between-view covariances of the smooth bases, as MCCA's A.
    stacked = np.hstack(full_bases)
    A = stacked.T @ C @ stacked
    A -= block_diag(*[b.T @ C @ b for b in full_bases])
    # Each view's variance blended with its roughness, U' M U = diag(ev),
    # the roughness rescaled to the variance's trace so that shrinkage
    # means the same for operators of any scale.
    variances = [b.T @ C @ b for b in full_bases]
    B = block_diag(
        *[
            (1 - c) * v + c * np.trace(v) / ev.sum() * np.diag(ev)
            for v, ev, c in zip(variances, eigenvalue_blocks, c_)
        ]
    )
    _, eigvecs = gevp(A / m, B / m, self.n_components)
    sizes = [b.shape[1] for b in full_bases]
    blocks = np.split(eigvecs, np.cumsum(sizes)[:-1], axis=0)
    embedding = [fb @ blk for fb, blk in zip(full_bases, blocks)]
    # The Laplacian's basis is orthogonal to D^{1/2} 1, not to the
    # constant: centre the embedding, and new points by the same means.
    self._embedding_means_: list[np.ndarray] = [e.mean(axis=0) for e in embedding]
    self.embedding_: list[np.ndarray] = [
        e - m for e, m in zip(embedding, self._embedding_means_)
    ]
    self.views_fit_: list[np.ndarray] = views_
    self._n_neighbors_: list[int] = n_neighbors_
    self._reg_: list[float] = reg_

    for state, basis, eigenvalues, block in zip(
        laplacian_parts, full_bases, eigenvalue_blocks, blocks
    ):
        state.full_basis, state.mu, state.block = basis, 1.0 - eigenvalues, block
    self._laplacian_state_: list[_LaplacianViewState] | None = (
        laplacian_parts if self.method == "laplacian" else None
    )
    self._lle_state_: list[NearestNeighbors] | None = (
        lle_nn if self.method == "lle" else None
    )
    self._fit_maps_and_importances(views_)
    return self