API Reference

Samplers for Evaluation

Classes

OASISSampler(alpha, predictions, scores, oracle)	Optimal Asymptotic Sequential Importance Sampling (OASIS) for estimation of the weighted F-measure.
ImportanceSampler(alpha, predictions, …[, …])	Importance sampling for estimation of the weighted F-measure
DruckSampler(alpha, predictions, scores, oracle)	Stratified sampling for estimation of the weighted F-measure
PassiveSampler(alpha, predictions, oracle[, …])	Passive sampling for estimation of the weighted F-measure

Stratification

Classes

stratification.Strata(allocations)

Represents a collection of strata and facilitates sampling from them

Functions

stratification.stratify_by_scores(scores[, …])	Stratify by binning the items based on their scores
stratification.stratify_by_features(…)	Stratify by clustering the items in feature space
stratification.auto_stratify(scores, **kwargs)	Generate Strata instance automatically