DiagROC is a Jamovi module for ROC analysis combined with diagnostic accuracy metric tools.

• ROC Analysis: Run ROC analysis and assess diagnostic accuracy metrics
• Optimal Cut-off Selection: Assess diagnostic accuracy metrics for selected cut-off points
• Manual Calculator: Assess diagnostic accuracy metrics based on the given contingency table
• Sample Size Estimation for Diagnostic Accuracy Studies

This module performs ROC analysis using the pROC R package. The classification variable must be dichotomous. If the variable is coded as "0" and "1" in the dataset, the level "1" is automatically considered as the positive class.

Unless explicitly specified, the pROC package automatically determines the direction of threshold comparison for univariate ROC curves (which is the type of analysis performed in this module), selecting the direction that maximizes the AUC. This module adopts that default automatic selection strategy. According to the pROC documentation ¹:

in which direction to make the comparison? “auto” (default for univariate curves): automatically define in which group the median is higher and take the direction accordingly. Not available for multivariate curves. “>” (default for multivariate curves): if the predictor values for the control group are higher than the values of the case group (controls > t >= cases). “<”: if the predictor values for the control group are lower or equal than the values of the case group (controls < t <= cases)

The confidence interval for the AUC is calculated using DeLong's method.

The summary coordinate table presents selected points based on the local maxima method. Alternatively, users may choose to display all available coordinates. For diagnostic accuracy metrics, the threshold corresponding to the maximum Youden index is selected by default. (If you wish to evaluate diagnostic accuracy at alternative thresholds, you can use the "Optimal Cutoff Selection" menu.)

You may include multiple dependent (outcome) variables in the analysis. In this case, a separate ROC analysis will be performed for each variable, and the optimal cut-off values and diagnostic accuracy measures will be calculated individually.

ROC curves can be displayed either in a single combined plot or as separate plots for each variable. If more than one variable is selected, the AUC comparison option becomes available.

For plotting, this module uses the ggplot2 package instead of the native plotting functions of the pROC package. This allows for full integration with Jamovi's theme and visual style.

Confidence intervals for diagnostic accuracy measures are calculated using the following methods

Begin the analysis by selecting your index test or exposure variable and your reference standard or disease status variable. The index variable can be either numerical or nominal. If a nominal variable is selected, it must be dichotomous. For dichotomous variables, a level selector will become active; for ordinal or continuous numerical variables, a cut-off configuration box will be enabled.

When specifying a threshold, first select the direction of comparison, then enter the desired cut-off value. Based on the chosen direction and cut-off, cases that meet the condition will be labeled as test/exposure positive, while those that do not will be considered negative. A contingency table is then constructed, and diagnostic accuracy metrics are computed accordingly.

You may define multiple cut-off values. In such cases, diagnostic analyses will be performed separately for each threshold.

The sample size calculation tool for diagnostic accuracy studies in this module is adapted from the online calculator developed by Haldun Akoğlu, MD. For methodological background, see Akoğlu (2022).²