pyhgf.model.hybrid.layer_norm_adapter

pyhgf.model.hybrid.layer_norm_adapter(layer_norm)

Build a frozen LayerNorm around an eqx.nn.LayerNorm’s parameters.

Forward, each row is centred, divided by its spread, then scaled and shifted by the (frozen) weight/bias. Backward, the error is scaled by weight, then the centring and rescaling are undone: the error’s own row-average and its overlap with the normalised input are subtracted before dividing by the spread — the two subtractions account for the fact that shifting or stretching a whole row leaves its normalisation unchanged.

Parameters:

layer_norm (LayerNorm)

Return type:

EquinoxAdapter