Improving the Neural Network Image Processing Accuracy by a Dental Robot-Simulator Using the Training Dataset Augmentation Method

In modern automated systems and robotics, machine vision is widely used to solve various types of applied problems without human participation. This is a scientific field of artificial intelligence, and related technologies for obtaining images of real-world objects, processing them and using the results obtained. Most machine vision subsystems use artificial neural networks to solve problems such as detection, classification and segmentation of objects. The effectiveness of machine vision subsystems can be assessed using many criteria, the key one of which is the accuracy of the corresponding problem solution, for example, the accuracy of object classification. The use of traditional methods for increasing accuracy based on optimizing the structure of the neural network and neurons, selecting hyperparameters, does not guarantee the stability of the results when image obtaining conditions change, for example, when changing lighting, shooting angle, noise. The number of false detections of objects in unacceptable positions, or the number of missed objects and classification and segmentation errors increases and becomes unacceptable. An alternative way to increase accuracy is to use data augmentation methods for network training obtaining synthetic images that provide special properties of the training sample. However, studies devoted to augmentation methods do not analyze the image structure in terms of saliency maps, and it does not allow to produce effective augmented data for training.
To solve the problem of increasing the accuracy of neural network image processing, an iterative augmentation method based on new principles of image synthesis has been developed. The method will allow countering false activations of neurons by reducing the influence of non-key features on the image saliency map on the result of object classification. The proposed method was used to train neural networks of a dental robot simulator. Software has been developed that allows synthesizing new images automatically and training the network. When using the proposed augmentation method, an increase in the accuracy of object classification by 2-10 % is observed.

1. Verbitsky M. The beginning study of Topology. Problems and theorems. Moscow, МCNMO, 2017, pp. 163—164 (in Russian).

2. Kokoulin A. N., Kokoulin R. A. The Hierarchical Approach for Image Processing in Objects Recognition System. Proceedings of the 2022 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus), 2022, Jan. 25—28, 2022, St. Petersburg, Russia, pp. 340—344.

3. Astashina N. B., Baydarov A. A., Arutunov S. D., Yuzhakov A. A., Valiahmetova K. R. Development of "artificial intellect based dental android system" for simulation of medical manipulations and doctor-patient communication, Permsky medicinsky jurnal, 2022, vol. 39, no. 6, pp. 62—70 (in Russian).

4. Kokoulin A. N., Yuzhakov A. A. Two-stage object detection scheme in computer vision systems of service robots, Vestnik permskogo nacionalnogo issledovatelskogo politechnicheskogo universiteta: elektrotechnika, informacionnye technologii, sistemy upravlenija, 2024, no. 49, pp. 176—199 (in Russian).

5. Yang S., Xiao W., Zhang M., Guo S., Zhao J., Shen F. Image Data Augmentation for Deep Learning: A Survey, arXiv eprints, 2022, doi:10.48550/arXiv.2204.08610.

6. Redmon J., Divvala S., Girshick R., Farhadi A. You only look once: Unified, realtime object detection", Proceedings of the IEEE conference on computer vision and pattern recognition, 2016, pp. 779—788.

7. Liu W., Anguelov D., Erhan D., Szegedy C., Reed S., Fu C. Y., Berg A. C. Ssd: Single shot multibox detector, European conference on computer vision, Springer, Cham, 2016, pp. 21—37.

8. Deng L., Yang M., Li T., He Y., Wang C. RFBNet: deep multimodal networks with residual fusion blocks for RGB-D semantic segmentation, 2019, arXiv: 1907.00135.

9. Zhao Z. Q., Zheng P., Xu S. T., Wu X. Object detection with deep learning: A review, IEEE transactions on neural networks and learning systems, 2019, vol. 30, no. 11, pp. 3212—3232.

10. Kniazev A., Slivnitsin P., Mylnikov L., Schlechtweg S. Influence of synthetic image datasets on the result of neural networks for object detection, Proceedings of the 9th International Conference on Applied Innovation in IT, 2021, vol. 9, iss. 1.

11. Tur A. I., Kokoulin A. N. Recyclable Waste Optical Sorting System Based on Neural Networks Ensemble. Proceedings of the 2022 IEEE Conference of Russian Young Researchers in Electrical and Electronic Engineering (ElConRus), 2022, Jan. 25—28, 2022, St. Petersburg, Russia, pp. 345—348.

12. Vries T., Taylor G. W. Learning Confidence for Out-ofDistribution Detection in Neural Networks, 2018, available at: http://bit.ly/2XZHpH1.

13. Liu T., Mirzasoleiman B. Data-efficient augmentation for training neural networks, Advances in Neural Information Processing Systems, 2022, doi: 10.48550/arXiv.2210.08363.

14. Shen R., Bubeck S., Gunasekar S. Data augmentation as feature manipulation, Proccedings of International conference on machine learning, 2022, doi: 10.48550/arXiv.2203.01572.

15. Zou D., Cao Y., Li Y., Gu Q., Krause A., Brunskill E. et al. The benefits of mixup for feature learning, Proceedings of the 40th International Conference on Machine Learning, 2023, doi: 10.5555/3618408.3620236, available at: https://arxiv.org/abs/2005.00695.

16. Warr K. Strengthening Deep Neural Networks, SPb., Piter, 2021, 272 p. (in Russian).

17. Tramer F. Ensemble Adversarial Training: Attacks and Defenses, International Conference on Learning Representations, 2018, available at: http://bit.ly/2XldcFh.

18. Nicolae M.-I. Adversarial Robustness Toolbox v1.0.0", arXiv e-prints, 2018, doi: 10.48550/arXiv.1807.01069.

19. Carlini N., Wagner D. Towards Evaluating the Robustness of Neural Networks", arXiv e-prints, 2016, doi: 10.48550/arXiv.1608.04644.

20. Alber M., Müller K., Erhan D., Kim B., D hne S. et al. Learning how to explain neural networks: Patternnet and pattern attribution, Proceedings of the International Conference on Learning Representations, 2018, available at: https://openreview.net/forum?id=Hkn7CBaTW (дата обращения: 14.03.2024).

21. Adebayo J., Gilmer J., Muelly M., Goodfellow I., Hardt M., Kim B. Sanity checks for saliency maps, Advances in neural information processing systems, 2020, doi: 10.48550/arXiv.1810.03292.