Identifying and extracting key features from bark images of 42 tree species using convolutional neural networks and class activation mapping

Background/Question/Methods

Among the various parts that can be used for the classification of trees, barks have several advantages in terms of accessibility and high constancy throughout the season. Consequently, deep learning-based approaches that only utilize bark images for tree species identification are increasing recently. In previous studies, the classification accuracy has been achieved over 90% for 22 to 53 tree species, indicating the decent performance of machines. Despite the high classification performance of deep learning models, unlike other organs including leaves and flowers, the key identification features of barks remain veiled. Recently, however, with the invention of Class Activation Mapping (CAM) methods, it has been possible to grasp the key features where the neural network ‘pays attention’ for each prediction. In this study, bark classification dataset of 42 tree species with higher quality and quantity was constructed and Convolutional Neural Networks (CNNs) were trained and tested on the dataset. Then CAM methods including Grad-CAM and Grad-CAM++ were applied to extract key features of barks from classifications of CNNs.

Results/Conclusions

After training the entire dataset with more than 70 epochs, two different CNNs models, EfficientNetB7 and VGG-16, achieved the highest accuracy of 92.62% and 92.22%, respectively. Despite of the excellent performances of models, high intra-genus similarities were also found in Betula, Picea, and Quercus genera by showing high ratio of misclassification into another species in the same genus: 4.5% in Betula alleghanlensis to Betula papyrifera; 11.2% in Picea abies to Picea glauca; and 9.8% in Quercus serrata to Quercus acutissima. The CAM methods could successfully extract distinct visual features including blisters of resin, horizontal and vertical cracks in the bark, and high contrast region near the crack. We expect that further research using additional feature extraction methods combined with conventional classification algorithm such as decision trees will enable to establish solid identification keys for barks.