Perform a posthoc Friedman-Nemenyi test.

Source: R/friedmanPostHocTestBMR.R

Performs a [PMCMR::posthoc.friedman.nemenyi.test] for a [BenchmarkResult] and a selected measure. This means *all pairwise comparisons* of `learners` are performed. The null hypothesis of the post hoc test is that each pair of learners is equal. If the null hypothesis of the included ad hoc [stats::friedman.test] can be rejected an object of class `pairwise.htest` is returned. If not, the function returns the corresponding friedman.test. Note that benchmark results for at least two learners on at least two tasks are required.

friedmanPostHocTestBMR(bmr, measure = NULL, p.value = 0.05,
  aggregation = "default")

Arguments

bmr

(BenchmarkResult)
Benchmark result.
measure

(Measure)
Performance measure. Default is the first measure used in the benchmark experiment.
p.value

(`numeric(1)`)
p-value for the tests. Default: 0.05
aggregation

(character(1))
“mean” or “default”. See getBMRAggrPerformances for details on “default”.

Value

([pairwise.htest]): See [PMCMR::posthoc.friedman.nemenyi.test] for details. Additionally two components are added to the list:

f.rejnull (`logical(1)`)

Whether the according friedman.test rejects the Null hypothesis at the selected p.value

crit.difference (`list(2)`)

Minimal difference the mean ranks of two learners need to have in order to be significantly different

See also

Other benchmark: BenchmarkResult, batchmark, benchmark, convertBMRToRankMatrix, friedmanTestBMR, generateCritDifferencesData, getBMRAggrPerformances, getBMRFeatSelResults, getBMRFilteredFeatures, getBMRLearnerIds, getBMRLearnerShortNames, getBMRLearners, getBMRMeasureIds, getBMRMeasures, getBMRModels, getBMRPerformances, getBMRPredictions, getBMRTaskDescs, getBMRTaskIds, getBMRTuneResults, plotBMRBoxplots, plotBMRRanksAsBarChart, plotBMRSummary, plotCritDifferences, reduceBatchmarkResults

Examples

# see benchmark