LiGRLayer

class rectools.models.nn.transformers.ligr.LiGRLayer(n_factors: int, n_heads: int, dropout_rate: float, ff_factors_multiplier: int = 4, bias_in_ff: bool = False, ff_activation: str = 'swiglu')

Bases: Module

Transformer Layer as described in “From Features to Transformers: Redefining Ranking for Scalable Impact” https://arxiv.org/pdf/2502.03417

Parameters

• n_factors (int) – Latent embeddings size.

• n_heads (int) – Number of attention heads.

• dropout_rate (float) – Probability of a hidden unit to be zeroed.

• ff_factors_multiplier (int, default 4) – Feed-forward layers latent embedding size multiplier.

• bias_in_ff (bool, default False) – Add bias in Linear layers of Feed Forward

• ff_activation ({"swiglu", "relu", "gelu"}, default "swiglu") – Activation function to use.

Methods

forward(seqs, attn_mask, key_padding_mask)

Forward pass through transformer block.

Attributes

forward(seqs: Tensor, attn_mask: Optional[Tensor], key_padding_mask: Optional[Tensor]) → Tensor

Forward pass through transformer block.

Parameters

• seqs (torch.Tensor) – User sequences of item embeddings.

• attn_mask (torch.Tensor, optional) – Optional mask to use in forward pass of multi-head attention as attn_mask.

• key_padding_mask (torch.Tensor, optional) – Optional mask to use in forward pass of multi-head attention as key_padding_mask.

Returns

User sequences passed through transformer layers.

Return type

torch.Tensor