_AUCMetric

class rectools.metrics.auc._AUCMetric(k: int, debias_config: DebiasConfig = None, insufficient_handling: str = 'ignore')[source]

Bases: DebiasableMetrikAtK

ROC AUC based metric base class.

Warning: This class should not be used directly. Use derived classes instead.

Parameters

  • k (int) – Number of items at the top of recommendations list that will be used to calculate metric.

  • insufficient_handling ({“ignore”, “raise”, “exclude”}, default “ignore”) – Method of handling users with insufficient recommendations for metric calculation. ROC AUC based metrics with k parameter often need more then k recommendations for each user. This happens because this metrics calculate ROC AUC score for specific number of user false positives and ranked test positives that is derived from provided k parameter but is not equal to it. The following methods are available: - ignore - don’t check for insufficient recommendations lists, handle all of insufficient cases as if algorithms are not able to retrieve users unpredicted test positives on any k level. This will understate the metric value; - exclude - exclude all users with insufficient recommendations lists from metrics computation; - raise - raise error if there are any users with insufficient recommendations lists. Use this option very carefully because some of the algorithms are unable to provide full required lists because of their inference logic. So can get errors even if you requested enough recommendations in recommend method. For example, ItemKNN generates recommendations only until the model has non-zero scores for the item in item-item similarity matrix. So with small K for neighbours in ItemKNN and big K for recommend and AUC based metric you will still get an error when insufficient_handling is set to raise.

  • debias_config (DebiasConfig, optional, default None) – Config with debias method parameters (iqr_coef, random_state).

Inherited-members

Methods

calc(reco, interactions)

Calculate metric value.

calc_from_fitted(fitted[, is_debiased])

Calculate metric value from fitted data.

calc_per_user(reco, interactions)

Calculate metric values for all users.

calc_per_user_from_fitted(fitted[, is_debiased])

Calculate metric values for all users from from fitted data.

fit(reco, interactions, k_max, ...)

Prepare intermediate data for effective calculation.

Attributes

insufficient_handling

calc(reco: DataFrame, interactions: DataFrame) → float[source]

Calculate metric value.

Parameters

  • reco (pd.DataFrame) – Recommendations table with columns Columns.User, Columns.Item, Columns.Rank.

  • interactions (pd.DataFrame) – Interactions table with columns Columns.User, Columns.Item.

Returns

Value of metric (average between users).

Return type

float

calc_from_fitted(fitted: AUCFitted, is_debiased: bool = False) → float[source]

Calculate metric value from fitted data.

Parameters

  • fitted (AUCFitted) – Meta data that got from .fit method.

  • is_debiased (bool, default False) – An indicator of whether the debias transformation has been applied before or not.

Returns

Value of metric (average between users).

Return type

float

calc_per_user(reco: DataFrame, interactions: DataFrame) → Series[source]

Calculate metric values for all users.

Parameters

  • reco (pd.DataFrame) – Recommendations table with columns Columns.User, Columns.Item, Columns.Rank.

  • interactions (pd.DataFrame) – Interactions table with columns Columns.User, Columns.Item.

Returns

Values of metric (index - user id, values - metric value for every user).

Return type

pd.Series

calc_per_user_from_fitted(fitted: AUCFitted, is_debiased: bool = False) → Series[source]

Calculate metric values for all users from from fitted data.

Parameters

  • fitted (AUCFitted) – Meta data that got from .fit method.

  • is_debiased (bool, default False) – An indicator of whether the debias transformation has been applied before or not.

Returns

Values of metric (index - user id, values - metric value for every user).

Return type

pd.Series

classmethod fit(reco: DataFrame, interactions: DataFrame, k_max: int, insufficient_handling_needed: bool) → AUCFitted[source]

Prepare intermediate data for effective calculation.

You can use this method to prepare some intermediate data for later calculation. It can optimize calculations if you want calculate different AUC based metrics with different k parameter.

Parameters

  • reco (DataFrame) –

  • interactions (DataFrame) –

  • k_max (int) –

  • insufficient_handling_needed (bool) –

Return type

AUCFitted