EliminationSelector

class hypercoil.nn.confound.EliminationSelector(num_columns: int, infimum: float = -1.5, supremum: float = 2.5, or_dim: int = 1, and_dim: int = 1, *, key: jax.random.PRNGKey)

Model selection by elimination of variables.

Begin with a full complement of model vectors, then eliminate them by placing an L1 penalty on the weight of this layer.

Danger

Do not use this model! It once performed well as a fluke of initialisation, but testing it across multiple random seeds has shown that it is not a good model. It is included here as part of the synthetic data experiments, but should not be used in practice.

The internal weights of this module are passed through a parameterised sigmoid function and then thresholded at 0. Any variables corresponding to a 0 weight are eliminated in the forward pass.

Dimension:

Input : \((*, I, T)\)

* denotes any number of preceding dimensions, \(I\) denotes number of candidate model vectors, \(T\) denotes number of time points or observations per vector.

Output : \((*, I, T)\)

Parameters:

n_columnsint

Number of candidate vectors for the model.

infimumfloat (default -1.5)

Infimum of the thresholded sigmoid function, pre-thresholding. Note that an infimum closer to 0 results in a gentler slope close to the elimination threshold.

supremumfloat (default 2.5)

Supremum of the thresholded sigmoid function.

or_dimint

If this is greater than 1, then or_dim separate vectors are learned, and a variable is only eliminated if every one learns a 0 weight for that variable. During testing, we did not find a practical use for this. We didn’t look very carefully, and it’s possible that someone might find a use.

and_dimint

If this is greater than 1, then and_dim separate vectors are learned, and a variable is eliminated if any one learns a 0 weight for that variable. If both or_dim and and_dim are set, then the selector first takes the union across the or dimension and then takes the intersection across the and dimension. During testing, we did not find a practical use for this. We didn’t look very carefully, and it’s possible that someone might find a use.

initcallable

Initialisation function for the layer weight. Defaults to values randomly sampled from Uniform(0, 1).

Methods

__call__(x, *[, key])

Call self as a function.