net_blocks

Functions

init_feed_forward(n_factors, ...[, bias])

Initialise Feed-Forward network with one of activation functions: "swiglu", "relu", "gelu".

Classes

LearnableInversePositionalEncoding(...[, ...])	Class to introduce learnable positional embeddings.
PointWiseFeedForward(n_factors, ...[, bias])	Feed-Forward network to introduce nonlinearity into the transformer model.
PositionalEncodingBase(*args, **kwargs)	Base class for positional encoding.
PreLNTransformerLayer(n_factors, n_heads, ...)	Pre-LN Transformer Layer as described in "On Layer Normalization in the Transformer Architecture" https://arxiv.org/pdf/2002.04745
PreLNTransformerLayers(n_blocks, n_factors, ...)	Pre-LN Transformer blocks.
SwigluFeedForward(n_factors, n_factors_ff, ...)	Feed-Forward network to introduce nonlinearity into the transformer model.
TransformerLayersBase(*args, **kwargs)	Base class for transformer layers.