Portfolio Entry V — December 11, 2020

How do tree detection practices vary across analyses?

Bringing together analyses from students assessing land cover in western Rwanda, we are able to better understand best practices and opportunities for future improvements in measuring tree cover in this landscape.

How accurate were our classifications of tree cover?

Figure 1: Study region around Gishwati Forest National Park, Rwanda

In this analysis, we used a random-forest classification model to detect tree cover patterns across a 100 x 100 km square region surrounding the Gishwati Forest National Park in western Rwanda (Fig. 1). Each classification included between 10 and 20 decision trees, and tree cover was assessed using cloud-masked composite images taken by the LANDSAT-8 and Sentinel-2 images. These analyses were conducted using images centered around June to September in 2019 (sometimes including images from this same month range in 2018 or 2020), as this is the season with the least rainfall and cloud cover in western Rwanda.

The overall accuracy of this classification varied amongst the different analyses, ranging from 31% to 80%, with an average accuracy of 66% (73% excluding the 31% outlier) for Landsat-8 and 67% (73% excluding the 34% accuracy outlier) for Sentinel-2 (Table 1). Variation within these accuracies is likely influenced by how training points were determined in each analysis, and assessing the errors of omission and commission can help us investigate how training points impacted the classification. Despite these variances between analyses, it does not appear that land cover classifications using Landsat-8 or Sentinel-2 yielded significantly different overall accuracies, errors of commission, errors of omission, nor kappa coefficients (Table 1). Though the visualization of Landsat-8 and Sentinel-2 appears to show significant differences in tree cover mapping between the two satellites, this is likely due to pixel size (Landsat-8 = 30m; Sentinel-2 = 10m) (Fig. 2). Our confusion matrices confirm that using images from different satellites did not significantly alter accuracies of tree cover mapping (Table 1). 

Table 1: Summary of accuracies, errors of omission and commission, and kappa coefficient from Landsat-8 and Sentinel-2a analyses
  • 2a: Tree cover using Landsat-8 (30m pixel)
  • 2b: Tree cover using Sentinel-2 (10m pixel)
  • 2c: Base image from Google Earth/Google Maps
Figure 2: Comparison of tree cover in Landsat-8 and Sentinel-2 images

Primary errors of omission and commission

Figure 3: Error of omission – tree cover left out of tree cover class

The primary issues in our tree cover analyses were a misclassification of land cover features in relationship to tree cover. The majority of errors of omission came from smaller woodlots and patches of forest not being included in the tree cover class. While dense forests (often within national parks) were consistently and accurately classified as tree cover, woodlots and smaller patches of natural forest were frequently omitted across all analyses (Fig. 3). For errors of commission, where points that were not actually trees were classified as trees, the most common feature that incorrectly fell into the tree cover class was high-density agriculture. Making up a large portion of the landscape and exhibiting different spectral signatures for varying species, portions of the heterogenous agriculture class were easily absorbed into the tree cover classification (Fig. 4). Similarly, small, isolated woodlots were sometimes excluded from the tree cover class as they blended into the different agricultural land cover features. All land cover features were compared in relationship to tree cover, meaning that it was only pertinent for the confusion matrix whether agricultural land was confused with tree cover, not if agricultural land was classified into other land cover classes (barren or grassland, for example). 

  • 4a: Tree cover and agriculture, L8
  • 4b: Tree cover and agriculture, S2
  • 4c: Base satellite image
Figure 4:  Error of commission – Agricultural lands being included in tree cover class
Figure 5: Confusion between volcanic and tree land cover classes

While not resulting in enormous errors of commission within the tree cover class, a number of analysts reported that there were difficulties in distinguishing volcanic rock and brush from tree cover when creating training points. The volcanic rock flows from Mount Nyiragongo in the Virunga National Park are a distinct land cover feature deserving of their own class (Fig. 5). However, due to successional vegetation growth on many of these rock flows, it is possible that some training points classified as “barren” inadvertently included shrubby vegetation and trees, leading to some barren pixels being included throughout the study region (Fig. 6a-6c). Further, tree cover was sometimes confused with shrubby or grassland areas, leading to some woodlots being incorrectly committed to the grassland land cover class (Fig. 6b). These scenarios of tree cover being committed to brush/grassland and volcanic rock/barren classes were noticed through visual assessment. As seen in Figure 6b, these misclassifications come across as errors of omission in our tree cover analysis, and we can only specify how they got confused through a visual assessment. For future studies that are concerned with broader land cover classifications, it may be important to create additional training points for each unique land cover feature, as the training points here were only focused on distinguishing tree cover from other land cover classes. 

  • 6a: Misclassified barren and shrubland pixels
  • 6b: Tree cover and errors of omission
  • 6c: Base satellite image
Figure 6: Visual error of commission – tree cover being committed to volcanic/barren and grassland land cover classes

Ways forward

Amongst the group members who conducted analyses of tree cover in western Rwanda, there is a general consensus that our tree cover analyses were not so good at picking up isolated forests and woodlots due to high rates of tree cover omission. Of the eight analyses conducted, the average rate of omission was over 50%, meaning that a large number of trees on-the-ground were not included as trees in this land cover analysis (Table 1). In comparison, the error of commission rested around 30%, reflecting the lower amount of other land cover features being misclassified as trees. Nevertheless, while a visual assessment shows some significantly misclassified regions near woodlots and natural forests outside of parks (Fig. 4; Fig. 6b), most analyses appear to pick up large forests and many tree patches across the region (Fig. 2). 

To improve this analysis in the future, it would be recommended that the user increases the number of training points distinguishing woodlots within the overall “tree cover” class. By overtraining on woodlots, it is hoped that fewer small patches of trees will be omitted from the tree classification. This overtraining could have expanded benefits with increased training points for other land cover classes, as there will be fewer errors of commission where woodlots are incorrectly classified as agriculture, barren/volcanic, or shrub land.

Link to code: https://code.earthengine.google.com/f3584723f65e1c638e26319b11e41399