ECOL 8910: Perspectives in Computational Ecology

The authors set out to test the performance of five presence-only and presence-background SDM methods on sparsely recorded species in the neotropics. They use presence data on the common anuran Hypsiboas bischoffi to test the building of SDMs for species in the Brazilian Atlantic Forest (BAF). The models that they compare include: BIOCLIM (envelope based method in environmental space), DOMAIN (distance based method in environmental space), SVM (non-probabilistic statistical pattern recognition algorithms for estimating the boundary of a set), OM-GARP (a genetic algorithm which selects a set of rules that best predicts the distribution), and MAXENT. In addition to simply comparing the performance of these models in this region they also compare two alternative “calibration” domains for their models, the BAF and all of South America. “Calibration” domains defined the region from which pseudoabsence points were chosen for evaluation as well for training of presence-background models (OM-GARP and MAXENT). For those models that use pseudoabsence (OM-GARP and MAXENT) they examine how increasing calibration area (background) changes predictions in environmental space. These questions are relevant particularly in the Neotropics and more broadly because of the importance of modeling relatively unknown species with distributions of unidentified extents. Evaluation was performed using 5 random 75% training -25% testing partitions with 10 random pseudoabsences per occurrence point and assessed via AUC. Mean AUCs of models ranged from .77 (BIOCLIM/BAF) to .99 (MAXENT/SA). AUCs were always higher for models calibrated in SA than those in BAF. SVM had the highest AUC for BAF (.95) while MAXENT had the highest for SA (.99). BIOCLIM had the lowest AUC for both domains. The highest difference in predicted area for BAF vs. SA was OM-GARP’s. MAXENT was substantially more robust in its environmental predictions than OM-GARP to changes in extent of background area. As a result both SVM and MAXENT seem to be good choices for SDMs regardless of the unknown extent of the true distribution while OM-GARP may be a good option when the “calibration” area concurs relatively well with the extent of the “true” distribution. The most interesting results of this paper are the differences between OM-GARP and MAXENT in the way that their predictions are affected by a change in the extent of background points. This serves as yet another strong argument for MAXENT as a great choice model. Most of the remainder of the results seem to distill to the fact that evaluating in an increased area ought to lead to higher AUC because there are more obvious absence areas that are easily identified.

Raes, N. and H. ter Steege (2007). “A null-model for significance testing of presence-only species distribution models.” Ecography 30(5): 727-736.

Validation of SDMs is both important and potentially difficult and problematic, especially in the case of presence-only data. Using pseudo-absence (background) points to calculate the AUC of trained models can create a number of problems, particularly that the maximum AUC is no longer 1 but rather 1-a/2 where a is the fraction of the landscape of interest genuinely covered by the species’ true distribution. As a result the interpretation of AUC values becomes muddied and typically “good” AUC values may not be what they seem. The authors propose a null model based approach to significance testing using “collection” localities, randomly drawn from the background, in equal number to the actual number of collection localities. This method relies, however, on the often unmet assumption that researchers have sampled all localities equally well. They propose to address this issue by drawing the random “collection” localities exclusively from the set of known collection localities. The authors illustrate this method using occurrences of the genus Shorea on the island Borneo and the SDM MaxEnt. They build both background based random null-models and null-models based on 1837 known collection localities for all plants across Borneo. Both of these types of null-models were used to construct a 95% confidence interval for AUC values such that AUCs outside of the interval implied that the model performed significantly differently from the random null model. 91% of species had a higher AUC than the random null-model CI, while only 61% had an AUC higher than the bias adjusted CI.  In so doing, they show the importance of correcting for bias in your presence points when building null models, while recognizing that often researchers will not have an accessible set of nearly 2000 potential collection points to use for correcting their null-model. They suggest using distance to features such as roads, rivers, cities, and nature reserves, as a proxy for this data. They found that the AUCs of their true models and both null-models decreased with increasing number of presence points (and growing predicted range size) which they interpret to mean that as the unknown true distribution of the species increases only the maximum AUC decreases and not the actual predictive ability of models. Though the concept of null-model based hypothesis testing has been used previously in SDM and elsewhere of course as well, the addition of the bias-corrected null model seems valuable for effective evaluation. It does ignore, however, the fact that the actual predictive accuracy of a given model will be affected by bias in the presence data. This method, though, may make our AUC evaluations more fair and accurate and so seems worth further study.