R/measures.R
A performance measure is evaluated after a single train/predict step and returns a single number to assess the quality of the prediction (or maybe only the model, think AIC). The measure itself knows whether it wants to be minimized or maximized and for what tasks it is applicable.
All supported measures can be found by listMeasures or as a table in the tutorial appendix: https://mlr.mlr-org.com/articles/tutorial/measures.html.
If you want a measure for a misclassification cost matrix, look at makeCostMeasure. If you want to implement your own measure, look at makeMeasure.
Most measures can directly be accessed via the function named after the scheme measureX (e.g. measureSSE).
For clustering measures, we compact the predicted cluster IDs such that they form a continuous series starting with 1. If this is not the case, some of the measures will generate warnings.
Some measure have parameters. Their defaults are set in the constructor makeMeasure and can be overwritten using setMeasurePars.
featperc timetrain timepredict timeboth sse measureSSE(truth, response) mse measureMSE(truth, response) rmse measureRMSE(truth, response) medse measureMEDSE(truth, response) sae measureSAE(truth, response) mae measureMAE(truth, response) medae measureMEDAE(truth, response) rsq measureRSQ(truth, response) expvar measureEXPVAR(truth, response) arsq rrse measureRRSE(truth, response) rae measureRAE(truth, response) mape measureMAPE(truth, response) msle measureMSLE(truth, response) rmsle measureRMSLE(truth, response) kendalltau measureKendallTau(truth, response) spearmanrho measureSpearmanRho(truth, response) mmce measureMMCE(truth, response) acc measureACC(truth, response) ber measureBER(truth, response) multiclass.aunu measureAUNU(probabilities, truth) multiclass.aunp measureAUNP(probabilities, truth) multiclass.au1u measureAU1U(probabilities, truth) multiclass.au1p measureAU1P(probabilities, truth) multiclass.brier measureMulticlassBrier(probabilities, truth) logloss measureLogloss(probabilities, truth) ssr measureSSR(probabilities, truth) qsr measureQSR(probabilities, truth) lsr measureLSR(probabilities, truth) kappa measureKAPPA(truth, response) wkappa measureWKAPPA(truth, response) auc measureAUC(probabilities, truth, negative, positive) brier measureBrier(probabilities, truth, negative, positive) brier.scaled measureBrierScaled(probabilities, truth, negative, positive) bac measureBAC(truth, response) tp measureTP(truth, response, positive) tn measureTN(truth, response, negative) fp measureFP(truth, response, positive) fn measureFN(truth, response, negative) tpr measureTPR(truth, response, positive) tnr measureTNR(truth, response, negative) fpr measureFPR(truth, response, negative, positive) fnr measureFNR(truth, response, negative, positive) ppv measurePPV(truth, response, positive, probabilities = NULL) npv measureNPV(truth, response, negative) fdr measureFDR(truth, response, positive) mcc measureMCC(truth, response, negative, positive) f1 measureF1(truth, response, positive) gmean measureGMEAN(truth, response, negative, positive) gpr measureGPR(truth, response, positive) multilabel.hamloss measureMultilabelHamloss(truth, response) multilabel.subset01 measureMultilabelSubset01(truth, response) multilabel.f1 measureMultilabelF1(truth, response) multilabel.acc measureMultilabelACC(truth, response) multilabel.ppv measureMultilabelPPV(truth, response) multilabel.tpr measureMultilabelTPR(truth, response) cindex cindex.uno iauc.uno ibrier meancosts mcp db dunn G1 G2 silhouette
| truth | (factor) |
|---|---|
| response | (factor) |
| probabilities | (numeric | matrix) |
| negative | ( |
| positive | ( |
none
He, H. & Garcia, E. A. (2009) Learning from Imbalanced Data. IEEE Transactions on Knowledge and Data Engineering, vol. 21, no. 9. pp. 1263-1284.
H. Uno et al. On the C-statistics for Evaluating Overall Adequacy of Risk Prediction Procedures with Censored Survival Data Statistics in medicine. 2011;30(10):1105-1117. https://doi.org/10.1002/sim.4154.
H. Uno et al. Evaluating Prediction Rules for T-Year Survivors with Censored Regression Models Journal of the American Statistical Association 102, no. 478 (2007): 527-37. https://www.jstor.org/stable/27639883.