init_feed_forward

rectools.models.nn.transformers.net_blocks.init_feed_forward(n_factors: int, ff_factors_multiplier: int, dropout_rate: float, ff_activation: str, bias: bool = True) → Module[source]

Initialise Feed-Forward network with one of activation functions: “swiglu”, “relu”, “gelu”.

Parameters

  • n_factors (int) – Latent embeddings size.

  • ff_factors_multiplier (int) – How many hidden units to use in the network.

  • dropout_rate (float) – Probability of a hidden unit to be zeroed.

  • ff_activation ({"swiglu", "relu", "gelu"}) – Activation function to use.

  • bias (bool, default True) – If True, add bias to linear layers.

Returns

Feed-Forward network.

Return type

nn.Module