Do custom classifications outperform global tree maps?
A case study in Ranomafana National Park, Madagascar
Global-scale tree cover maps such as the Hansen Global Forest Change dataset (Hansen et al. 2013) are valuable tools for assessing land cover change at large scales and making standardized comparisons between regions. However, customized classifications are likely to be more accurate at the local level for fine-scale analysis of smaller regions. To investigate the differences between these classification methods, we will examine tree cover in Madagascar’s Ranomafana National Park and its surrounding zone of interaction (ZOI). The ZOI extends beyond the boundaries of the park to encompass contiguous habitat, species migration corridors, relevant watersheds, and strong human interactions, and in this case is delineated as a 10-km buffer around the park (DeFries et al. 2010). Since protected areas are not isolated islands, it is critical to monitor biodiversity within the entire ZOI rather than just the park boundaries (DeFries et al 2010). We use a random forest classifier, a type of supervised machine learning algorithm, to create a custom classification for this region and the entire Landsat scene overlapping it.
This classification uses only three land cover categories, summarized in Table 1 below, for the sake of using the least number of classes necessary to distinguish forest cover. The primary objective was to differentiate tree cover from other types of vegetation, such as shrubs, grassland, and agriculture. These non-forest vegetation types are often confused for one other, and were thus combined into a single class. Figure 2 provides visual examples of each land cover class.
Class Name | Class Definition | Class Justification |
Tree Cover | Forested landscape with dense canopy cover. | Rigorously mapping this land cover type is the objective of the analysis. |
Shrubland and Agriculture | Vegetated landscape consisting of low shrubs, grasses, or agricultural fields. | This land cover type is most likely to be confused with tree cover and must be distinguished accordingly. |
Bare Earth and Rock | Landscape with little to no vegetation, consisting of bare soil or rock. | This land cover type is dominant in the western portion of the image. |
Figure 2. Example of each land cover type from Google Earth satellite imagery.
Upon visual inspection, this classification map has achieved the stated goal of distinguishing forest from other land cover types. Figure 3 demonstrates the classifier’s ability to accurately pick out clusters of forest cover in a patchwork landscape.
Figure 3. A landscape of patchy forest within the study region (left) and its corresponding classification (right). Located at (47.35144, -21.68077).
However, some inaccuracies arise when it comes to differentiating non-forest classes from one another. For instance, Figure 4 illustrates the classifier’s overestimation of shrubland and agriculture in the western portion of the Landsat scene. In this case, swaths of bare soil and rock were classified as vegetation. Although this level of error is not ideal, it has no impact on the classifier’s ability to distinguish tree cover and therefore does not affect our subsequent analysis.
Figure 4. A landscape of primarily bare earth within the study region (left) and its corresponding classification (right). Located at (46.42991, -21.8154).
Next, we compare this custom classification with the Hansen Global Forest map for the study region. Figure 5 displays the agreements and disagreements in tree cover classification between the two maps, using a 50% tree cover threshold for the Hansen map.
It is immediately clear that the custom map classifies forest cover more conservatively than the Hansen map. At this 50% threshold, false positives are much more common than false negatives. Zooming in on a portion of Ranomafana National Park (Figure 6), it becomes evident that much of what the custom map classified as shrubland was identified as tree cover by the Hansen map.
Figure 6. A portion of Ranomafana National Park (left) and its corresponding map of agreements and disagreements between the custom classification and the Hansen map (right). Located at (47.5142, -21.1364).
However, it should be noted that we are comparing Hansen’s tree cover map from 2000 with a Landsat image from 2020. Forest cover could have been lost over those 20 years, explaining some portion of this discrepancy.
Since the custom map was produced using more rigorous and localized methodology than the Hansen map for this particular study region, let us assume that the custom map is the more accurate reference against which the Hansen thresholds can be evaluated. Figure 7 summarizes the agreement and disagreement between the two maps within the ZOI at various tree cover thresholds, characterizing pixels as true positives (TP), false negatives (FN), false positives (FP), or true negatives (TN).
Figure 7. Agreement and disagreement in tree cover classification between the custom map and the Hansen map within the ZOI at various tree cover thresholds.
As the threshold increases, true and false positives decrease while true and false negatives increase. This trend matches the results of a similar study by Adjognon et al. (2019). Table 2 summarizes the accuracy metrics of the Hansen map at different thresholds. True positive rate (TPR) is calculated as TP/(TP+FN), true negative rate (TNR) is calculated as TN/(TN+FP), and balanced accuracy rate is calculated as (TPR+TNR)/2.
Threshold | 0.45 | 0.60 | 0.75 | 0.90 |
TPR | 0.96 | 0.86 | 0.79 | 0.72 |
TNR | 0.43 | 0.73 | 0.85 | 0.90 |
Balanced | 0.70 | 0.80 | 0.82 | 0.81 |
Table 2. Accuracy metrics of the Hansen map evaluated against the custom map at various tree cover thresholds.
Based on these accuracy metrics, it is clear that the Hansen map best matches the forest cover classification in the custom map at a threshold of about 75% tree cover. It is important to note, however, that a supervised classification is entirely dependent on the selection of training points and is therefore subjective to the researcher’s class definitions.
Overall, this analysis provides a valuable comparison between a global tree cover map and a customized, local classification for a specific region. In an ideal world, the next step would be to obtain real-world ground truth points to use as a “gold standard” reference against which we could evaluate both maps.
References
Adjognon, G. S., Rivera-Ballesteros, A., & van Soest, D. (2019). Satellite-based tree cover mapping for forest conservation in the drylands of Sub Saharan Africa (SSA): Application to Burkina Faso gazetted forests. Development Engineering, 4: 10039. https://doi.org/10.1016/j.deveng.2021.100059
DeFries, R., et al. (2010). From plot to landscape scale: linking tropical biodiversity measurements across spatial scales. Frontiers in Ecology and the Environment, 8(3): 153-160. https://doi.org/10.1890/080104Hansen, M., et al. (2013). High-resolution global maps of 21st-century forest cover change. Science, 342(6160): 850-853. https://doi.org/10.1126/science.1244693
Hansen, M., et al. (2013). High-resolution global maps of 21st-century forest cover change. Science, 342(6160): 850-853. https://doi.org/10.1126/science.1244693