MRR

class rectools.metrics.ranking.MRR(k: int, debias_config: Optional[DebiasConfig] = None)[source]

Bases: _RankingMetric

Mean Reciprocal Rank at k (MRR@k).

MRR calculates as mean value of reciprocal rank of first relevant recommendation among all users.

Estimates relevance of recommendations taking in account their order.

\[MRR@K = \frac{1}{|U|} \sum_{i=1}^{|U|} \frac{1}{rank_{i}}\]
where

  • \({|U|}\) is a number of unique users;

  • \(rank_{i}\) is a rank of first relevant recommendation starting from 1.

If a user doesn’t have any relevant recommendation then his metric value will be 0.

Parameters

  • k (int) – Number of items at the top of recommendations list that will be used to calculate metric.

  • debias_config (DebiasConfig, optional, default None) – Config with debias method parameters (iqr_coef, random_state).

Examples

>>> reco = pd.DataFrame(
...     {
...         Columns.User: [1, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4],
...         Columns.Item: [7, 8, 1, 2, 2, 1, 3, 4, 7, 8, 3],
...         Columns.Rank: [1, 2, 1, 2, 1, 2, 3, 4, 1, 2, 3],
...     }
... )
>>> interactions = pd.DataFrame(
...     {
...         Columns.User: [1, 1, 2, 3, 3, 3, 4, 4, 4],
...         Columns.Item: [1, 2, 1, 1, 3, 4, 1, 2, 3],
...     }
... )
>>> # Here
>>> #    - for user ``1`` we return non-relevant recommendations;
>>> #    - for user ``2`` we return 2 items and relevant is first;
>>> #    - for user ``3`` we return 4 items, 2nd, 3rd and 4th are relevant;
>>> #    - for user ``4`` we return 3 items and relevant is last;
>>> MRR(k=1).calc_per_user(reco, interactions).values
array([0., 1., 0., 0.])
>>> MRR(k=3).calc_per_user(reco, interactions).values
array([0.        , 1.        , 0.5       , 0.33333333])
Inherited-members

Parameters

Methods

calc(reco, interactions)

Calculate metric value.

calc_from_merged(merged[, is_debiased])

Calculate metric value from merged recommendations.

calc_per_user(reco, interactions)

Calculate metric values for all users.

calc_per_user_from_merged(merged[, is_debiased])

Calculate metric values for all users from merged recommendations.

Attributes

calc_from_merged(merged: DataFrame, is_debiased: bool = False) → float[source]

Calculate metric value from merged recommendations.

Parameters

  • merged (pd.DataFrame) – Result of merging recommendations and interactions tables. Can be obtained using merge_reco function.

  • is_debiased (bool, default False) – An indicator of whether the debias transformation has been applied before or not.

Returns

Value of metric (average between users).

Return type

float

calc_per_user(reco: DataFrame, interactions: DataFrame) → Series[source]

Calculate metric values for all users.

Parameters

  • reco (pd.DataFrame) – Recommendations table with columns Columns.User, Columns.Item, Columns.Rank.

  • interactions (pd.DataFrame) – Interactions table with columns Columns.User, Columns.Item.

Returns

Values of metric (index - user id, values - metric value for every user).

Return type

pd.Series

calc_per_user_from_merged(merged: DataFrame, is_debiased: bool = False) → Series[source]

Calculate metric values for all users from merged recommendations.

Parameters

  • merged (pd.DataFrame) – Result of merging recommendations and interactions tables. Can be obtained using merge_reco function.

  • is_debiased (bool, default False) – An indicator of whether the debias transformation has been applied before or not.

Returns

Values of metric (index - user id, values - metric value for every user).

Return type

pd.Series