cca_zoo.tree

Gradient-boosted-tree nonlinear CCA methods. Requires pip install cca-zoo[tree].

---

TreeCCA

TreeCCA(
    latent_dimensions: int = 1,
    center: bool = True,
    backend: str = "xgboost",
    n_estimators: int = 50,
    max_depth: int = 5,
    learning_rate: float = 0.1,
    subsample: float = 0.8,
    colsample_bytree: float = 0.8,
    min_child_weight: float = 5,
    gauss_seidel: bool = True,
    random_state: int = 0,
)

Bases: BaseModel

TreeCCA — nonlinear multiview CCA with gradient-boosted-tree encoders.

Learns one nonlinear encoder \(f_i\) per view (a gradient-boosted tree ensemble per latent dimension) that jointly maximise the Eckart-Young (EY) unconstrained-CCA objective:

\[ \mathcal{L}_{EY} = -2 \operatorname{tr}(C) + \operatorname{tr}(V V) \]

where, for embeddings \(Z_i = f_i(X_i)\), \(C\) is the mean pairwise cross-covariance (including \(i = j\) terms) and \(V\) the mean auto-covariance across all views (see :mod:cca_zoo._utils._ey, the same shared EY-loss machinery used by :class:~cca_zoo.linear.gradient.CCA_EY and :class:~cca_zoo.deep.DCCA_EY). The encoders are fit by alternating (Gauss-Seidel) gradient boosting: each round, for every view in turn, one tree is added to each of its latent_dimensions boosters using the EY-loss gradient (rescaled to a fixed target standard deviation for well-conditioned tree leaves) as a custom regression objective, and — when gauss_seidel=True — the gradient is recomputed from the freshest embeddings before moving to the next view. Training starts from a random-orthogonal, unit-variance initial embedding per view. Because each latent component is a boosted-tree ensemble, per-component feature importance (split gain) is available directly, without a separate interpretability method such as SHAP.

This is a from-scratch reimplementation, as a scikit-learn-style :class:~cca_zoo._base.BaseModel, of the "Design A" (sequential, scalar-booster) training procedure from the TreeCCA research codebase, generalised from two views to an arbitrary number of views.

References

Chapman, J. (2026). TreeCCA: Canonical Correlation Analysis via Gradient-Boosted Trees. arXiv:2607.27027.

Parameters:

Name	Type	Description	Default
latent_dimensions	int	Number of latent components. Must not exceed the number of features in any view. Default is 1.	1
center	bool	Whether to subtract per-view column means before fitting. Default is True.	True
backend	str	Gradient-boosting library used for the per-component encoders: "xgboost" (default) or "lightgbm". The "lightgbm" backend requires the optional lightgbm package.	'xgboost'
n_estimators	int	Number of boosting rounds (trees added per booster). Default is 50.	50
max_depth	int	Maximum depth of each tree. Default is 5.	5
learning_rate	float	Boosting learning rate. Default is 0.1.	0.1
subsample	float	Row subsampling ratio per tree. Default is 0.8.	0.8
colsample_bytree	float	Column subsampling ratio per tree. Default is 0.8.	0.8
min_child_weight	float	Minimum sum of instance weight (xgboost) / minimum number of samples (lightgbm) needed in a child. Default is 5.	5
gauss_seidel	bool	If True, re-predict view 1's embedding after updating its boosters and use the fresh values when computing view 2's gradient (Gauss-Seidel); if False, both gradients are computed from the same stale embeddings (Jacobi). Default is True.	True
random_state	int	Seed for the boosters and for drawing the random-orthogonal initial embedding. Default is 0.	0

Example

import numpy as np rng = np.random.default_rng(0) X1 = rng.standard_normal((100, 5)) X2 = rng.standard_normal((100, 5)) model = TreeCCA(latent_dimensions=2, n_estimators=10).fit([X1, X2]) scores = model.transform([X1, X2])

Source code in cca_zoo/tree/_treecca.py

def __init__(
    self,
    latent_dimensions: int = 1,
    center: bool = True,
    backend: str = "xgboost",
    n_estimators: int = 50,
    max_depth: int = 5,
    learning_rate: float = 0.1,
    subsample: float = 0.8,
    colsample_bytree: float = 0.8,
    min_child_weight: float = 5,
    gauss_seidel: bool = True,
    random_state: int = 0,
) -> None:
    super().__init__(latent_dimensions=latent_dimensions, center=center)
    self.backend = backend
    self.n_estimators = n_estimators
    self.max_depth = max_depth
    self.learning_rate = learning_rate
    self.subsample = subsample
    self.colsample_bytree = colsample_bytree
    self.min_child_weight = min_child_weight
    self.gauss_seidel = gauss_seidel
    self.random_state = random_state

weights property

weights: list[ndarray]

Not implemented for TreeCCA.

Raises:

Type	Description
NotFittedError	If fit has not been called.
NotImplementedError	TreeCCA encoders are boosted-tree ensembles, not linear weight matrices. Use boosters_ instead, e.g. model.boosters_[view][component].get_score(importance_type="gain") (xgboost backend) or model.boosters_[view][component].feature_importance(importance_type="gain") (lightgbm backend) for per-component feature importance.

fit

fit(views: list[ArrayLike], y: None = None) -> TreeCCA

Fit the TreeCCA model.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	List of 2 or more arrays, each (n_samples, n_features_i).	required
y	None	Ignored.	None

Returns:

Name	Type	Description
self	TreeCCA	Fitted estimator.

Raises:

Type	Description
ValueError	If fewer than 2 views are provided.
ValueError	If views have inconsistent numbers of samples.
ValueError	If backend is not "xgboost" or "lightgbm".
ImportError	If backend="lightgbm" but lightgbm is not installed.

Source code in cca_zoo/tree/_treecca.py

def fit(self, views: list[ArrayLike], y: None = None) -> TreeCCA:
    """Fit the TreeCCA model.

    Args:
        views: List of 2 or more arrays, each (n_samples, n_features_i).
        y: Ignored.

    Returns:
        self: Fitted estimator.

    Raises:
        ValueError: If fewer than 2 views are provided.
        ValueError: If views have inconsistent numbers of samples.
        ValueError: If ``backend`` is not ``"xgboost"`` or ``"lightgbm"``.
        ImportError: If ``backend="lightgbm"`` but lightgbm is not
            installed.
    """
    if self.backend not in ("xgboost", "lightgbm"):
        raise ValueError(
            f"backend must be 'xgboost' or 'lightgbm', got {self.backend!r}."
        )
    if self.backend == "lightgbm" and not _LGBM_AVAILABLE:
        raise ImportError(
            "backend='lightgbm' requires the lightgbm package. "
            "Install with: pip install lightgbm"
        )
    views_ = self._setup_fit(views)
    k = self.latent_dimensions
    n_views = len(views_)

    rng = np.random.default_rng(self.random_state)
    base_margins = []
    projections = []
    for X in views_:
        bm, proj = _random_orthogonal_base_margin(X, k, rng)
        base_margins.append(bm)
        projections.append(proj)
    self._projections_: list[np.ndarray] = projections

    params = self._booster_params()
    encoders = [_Encoder(self.backend, X, k, params) for X in views_]

    for _ in range(self.n_estimators):
        representations = [
            bm + enc.predict() for bm, enc in zip(base_margins, encoders)
        ]
        grads = _rescale_to_target_std(ey_grad_z(representations))

        for view_idx in range(n_views):
            encoders[view_idx].boost(grads[view_idx])
            if self.gauss_seidel and view_idx < n_views - 1:
                representations[view_idx] = (
                    base_margins[view_idx] + encoders[view_idx].predict()
                )
                grads = _rescale_to_target_std(ey_grad_z(representations))

    self.boosters_: list[list[Any]] = [enc.boosters for enc in encoders]
    return self

transform

transform(views: list[ArrayLike]) -> list[np.ndarray]

Project views into the latent space using the fitted boosters.

Parameters:

Name	Type	Description	Default
views	list[ArrayLike]	List of arrays, each (n_samples, n_features_i), matching the number of views passed to fit.	required

Returns:

Type	Description
list[ndarray]	List of arrays, each (n_samples, latent_dimensions).

Raises:

Type	Description
NotFittedError	If fit has not been called.
ValueError	If fewer than 2 views are provided.

Source code in cca_zoo/tree/_treecca.py

def transform(self, views: list[ArrayLike]) -> list[np.ndarray]:
    """Project views into the latent space using the fitted boosters.

    Args:
        views: List of arrays, each (n_samples, n_features_i), matching
            the number of views passed to ``fit``.

    Returns:
        List of arrays, each (n_samples, latent_dimensions).

    Raises:
        sklearn.exceptions.NotFittedError: If ``fit`` has not been called.
        ValueError: If fewer than 2 views are provided.
    """
    check_is_fitted(self)
    validated = validate_views(views)
    centred = [v - m for v, m in zip(validated, self.means_)]
    result = []
    for v, boosters, projection in zip(centred, self.boosters_, self._projections_):
        bm = v @ projection
        result.append(bm + self._predict_boosters(boosters, v))
    return result