LiGRLayers

class rectools.models.nn.transformers.ligr.LiGRLayers(n_blocks: int, n_factors: int, n_heads: int, dropout_rate: float, ff_factors_multiplier: int = 4, ff_activation: str = 'swiglu', bias_in_ff: bool = False)[source]

Bases: TransformerLayersBase

LiGR Transformer blocks.

Parameters

  • n_blocks (int) – Number of transformer blocks.

  • n_factors (int) – Latent embeddings size.

  • n_heads (int) – Number of attention heads.

  • dropout_rate (float) – Probability of a hidden unit to be zeroed.

  • ff_factors_multiplier (int, default 4) – Feed-forward layers latent embedding size multiplier. Pass in transformer_layers_kwargs to override.

  • ff_activation ({"swiglu", "relu", "gelu"}, default "swiglu") – Activation function to use. Pass in transformer_layers_kwargs to override.

  • bias_in_ff (bool, default False) – Add bias in Linear layers of Feed Forward. Pass in transformer_layers_kwargs to override.

Methods

forward(seqs, timeline_mask, attn_mask, ...)

Forward pass through transformer blocks.

Attributes

forward(seqs: Tensor, timeline_mask: Tensor, attn_mask: Optional[Tensor], key_padding_mask: Optional[Tensor], **kwargs: Any) → Tensor[source]

Forward pass through transformer blocks.

Parameters

  • seqs (torch.Tensor) – User sequences of item embeddings.

  • timeline_mask (torch.Tensor) – Mask indicating padding elements.

  • attn_mask (torch.Tensor, optional) – Optional mask to use in forward pass of multi-head attention as attn_mask.

  • key_padding_mask (torch.Tensor, optional) – Optional mask to use in forward pass of multi-head attention as key_padding_mask.

  • kwargs (Any) –

Returns

User sequences passed through transformer layers.

Return type

torch.Tensor