CorrelationNorm: Correlation normalisation

class hypercoil.nn.activation.CorrelationNorm(factor: Tensor | Tuple[Tensor, Tensor] | None = None, grad_path: Literal['input', 'both'] = 'both')

Correlation normalisation module.

Divide each entry \(A_{ij}\) of the input matrix \(A\) by the product of the signed square roots of the corresponding diagonals:

\(\bar{A}_{ij} = A_{ij} \frac{\mathrm{sgn}(A_{ii} A_{jj})}{\sqrt{A_{ii}}\sqrt{A_{jj}} + \epsilon}\)

This default behaviour, which maps a covariance matrix to a Pearson correlation matrix, can be overriden by providing a factor argument (detailed below). This activation function is also similar to a signed version of the normalisation operation for a graph Laplacian matrix.

Dimension:

input : \((*, P, P)\)

P denotes the row and column dimensions of the input matrices. * denotes any number of additional dimensions.

output : \((*, P, P)\)

As above.

Parameters:

inputTensor

Tensor to be normalised.

factorTensor, iterable(Tensor, Tensor), or None (default None)

Normalisation factor.

• If this is not explicitly specified, it follows the default behaviour (division by the product of signed square roots.)

• If this is a tensor, input is directly divided by the provided tensor. This option is provided mostly for compatibility with non-square inputs.

• If this is a pair of tensors, then input is divided by their outer product.

gradpathstr 'input' or 'both' (default 'both')

If this is set to 'input' and the default normalisation behaviour is used, then gradient will be blocked from flowing backward through the computation of the normalisation factor from the input.

Returns:

Tensor

Normalised input.

Attributes:

factor

Methods

__call__(input[, factor, key])

Call self as a function.