calc_metrics

rectools.metrics.scoring.calc_metrics(metrics: Dict[str, MetricAtK], reco: DataFrame, interactions: Optional[DataFrame] = None, prev_interactions: Optional[DataFrame] = None, catalog: Optional[Collection[Union[str, int]]] = None) → Dict[str, float][source]

Calculate metrics.

Parameters

  • metrics (dict(str -> Metric)) – Dict of metric objects to calculate, where key is metric name and value is metric object.

  • reco (pd.DataFrame) – Recommendations table with columns Columns.User, Columns.Item, Columns.Rank.

  • interactions (pd.DataFrame, optional) – Interactions table with columns Columns.User, Columns.Item. Obligatory only for some types of metrics.

  • prev_interactions (pd.DataFrame) – Table with previous user-item interactions, with columns Columns.User, Columns.Item. Obligatory only for some types of metrics.

  • catalog (collection, optional) – Collection of unique item ids that could be used for recommendations. Obligatory only if ClassificationMetric or SerendipityMetric instances present in metrics.

Returns

Dictionary where keys are the same with keys in metrics and values are metric calculation results.

Return type

dict(str->float)

Raises

ValueError – If obligatory argument for some metric not set.

Examples

>>> from rectools import Columns
>>> from rectools.metrics import Accuracy, NDCG
>>> reco = pd.DataFrame(
...     {
...         Columns.User: [1, 1, 2, 2, 3, 3, 3, 3, 4, 4, 4],
...         Columns.Item: [7, 8, 1, 2, 1, 2, 3, 4, 1, 2, 3],
...         Columns.Rank: [1, 2, 1, 2, 1, 2, 3, 4, 1, 2, 3],
...     }
... )
>>> interactions = pd.DataFrame(
...     {
...         Columns.User: [1, 1, 2, 3, 3, 3, 4, 4, 4],
...         Columns.Item: [1, 2, 1, 1, 3, 4, 1, 2, 3],
...         Columns.Datetime: [1, 1, 1, 1, 1, 2, 2, 2, 2],
...     }
... )
>>> split_dt = 2
>>> df_train = interactions.loc[interactions[Columns.Datetime] < split_dt]
>>> df_test = interactions.loc[interactions[Columns.Datetime] >= split_dt]
>>> metrics = {
...     'ndcg@1': NDCG(k=1),
...     'accuracy@1': Accuracy(k=1)
... }
>>> calc_metrics(
...     metrics,
...     reco=reco,
...     interactions=df_test,
...     prev_interactions=df_train,
...     catalog=df_train[Columns.Item].unique()
... )
{'accuracy@1': 0.3333333333333333, 'ndcg@1': 0.5}