This function will calculate two major types of evaluation metrics in terms of presence-only data. The first type is presence-only customized metrics, such as Contrast Validation Index (CVI), continuous Boyce index (CBI), and ROC_ratio. The second type is presence-background evaluation metrics by extracting background points as pseudo absence observations.

  bg_pred = NULL,
  threshold = NULL,
  visualize = FALSE



(isolation_forest) The extended isolation forest SDM. It could be the item model of POIsotree made by function isotree_po.


(vector of numeric) A vector contains predicted values at occurrence locations.


(vector of numeric) the vector contains predicted values with same number of background points.


(vector of numeric) the vector contains predicted values of the whole area. The reason to take a vector is to keep this function flexible for multiple types of output.


(numeric or NULL) The threshold to calculate threshold-based evaluation metrics. If NULL, a recommended threshold will be calculated based on optimal TSS value. The default is NULL.


(logical) If TRUE, plot the evaluation figures. The default is FALSE.


(POEvaluation) A list of

  • po_evaluation is presence-only evaluation metrics. It is a list of

    • cvi (list) A list of CVI with 0.25, 0.5, and 0.75 as threshold

    • boyce (list) A list of items related to continuous Boyce index (CBI)

    • roc_ratio (list) A list of ROC ratio and AUC ratio

  • pb_evaluation is presence-background evaluation metrics. It is a list of

    • confusion matrix (table) A table of confusion matrix. The columns are true values, and the rows are predicted values.

    • sensitivity (numeric) The sensitivity or TPR

    • specificity (numeric) The specificity or TNR

    • TSS (list) A list of info related to true skill statistic (TSS)

      • cutoff (vector of numeric) A vector of cutoff threshold values

      • tss (vector of numeric) A vector of TSS for each cutoff threshold

      • Recommended threshold (numeric) A recommended threshold according to TSS

      • Optimal TSS (numeric) The best TSS value

    • roc (list) A list of ROC values and AUC value

    • Jaccard's similarity index (numeric) The Jaccard's similarity index

    • Sørensen's similarity index (numeric) The Sørensen's similarity index or F-measure

    • Overprediction rate (numeric) The Overprediction rate

    • Underprediction rate (numeric) The Underprediction rate


  • CVI is the proportion of presence points falling in cells having a threshold (0.5 for example) habitat suitability index minus the proportion of cells within this range of threshold of the model. Here we used varied thresholds: 0.25, 0.5, and 0.75.

  • continuous Boyce index (CBI) is made with a 100 resolution of moving windows and Kendall method.

  • ROC_ratio curve plots the proportion of presences falling above a range of thresholds against the proportion of cells falling above the range of thresholds. The area under the modified ROC curve was then called AUC_ratio.

  • Sensitivity (TPR) = TP/(TP + FN)

  • Specificity (TNR) = TN/(TN + FP)

  • True skill statistic (TSS) = Sensitivity + specificity - 1

  • Jaccard's similarity index = TP/(FN + TP + FP)

  • Sørensen's similarity index (F-measure) = 2TP/(FN + 2TP + FP)

  • Overprediction rate = FP/(TP + FP)

  • Underprediction rate = FN/(TP + FN)


  • Peterson, A. Townsend, Monica Papeş, and Jorge Soberón. "Rethinking receiver operating characteristic analysis applications in ecological niche modeling." Ecological modelling 213.1 (2008): 63-72. doi:10.1016/j.ecolmodel.2007.11.008

  • Hirzel, Alexandre H., et al. "Evaluating the ability of habitat suitability models to predict species presences." Ecological modelling 199.2 (2006): 142-152. doi:10.1016/j.ecolmodel.2006.05.017

  • Hirzel, Alexandre H., and Raphaël Arlettaz. "Modeling habitat suitability for complex species distributions by environmental-distance geometric mean." Environmental management 32.5 (2003): 614-623. doi:10.1007/s00267-003-0040-3

  • Leroy, Boris, et al. "Without quality presence-absence data, discrimination metrics such as TSS can be misleading measures of model performance." Journal of Biogeography 45.9 (2018): 1994-2002. doi:10.1111/jbi.13402


# Using a pseudo presence-only occurrence dataset of
# virtual species provided in this package

obs_df <- occ_virtual_species %>% filter(usage == "train")
eval_df <- occ_virtual_species %>% filter(usage == "eval")
x_col <- "x"
y_col <- "y"
obs_col <- "observation"

# Format the observations
obs_train_eval <- format_observation(
  obs_df = obs_df, eval_df = eval_df,
  x_col = x_col, y_col = y_col, obs_col = obs_col,
  obs_type = "presence_only")

env_vars <- system.file(
  package = 'itsdm') %>% read_stars() %>%
  slice('band', c(1, 5, 12, 16))

# With perfect_presence mode,
# which should be very rare in reality.
mod <- isotree_po(
  obs_mode = "perfect_presence",
  obs = obs_train_eval$obs,
  obs_ind_eval = obs_train_eval$eval,
  variables = env_vars, ntrees = 10,
  sample_size = 0.8, ndim = 2L,
  seed = 123L, nthreads = 1,
  response = FALSE,
  spatial_response = FALSE,
  check_variable = FALSE)

# Without background samples or absences
eval_train <- evaluate_po(
  occ_pred = mod$pred_train$prediction,
  var_pred = na.omit(as.vector(mod$prediction[[1]])))

# With background samples
bg_pred <- st_extract(
  mod$prediction, mod$background_samples) %>%
eval_train <- evaluate_po(
  occ_pred = mod$pred_train$prediction,
  bg_pred = bg_pred$prediction,
  var_pred = na.omit(as.vector(mod$prediction[[1]])))