Application of training data synthesis methods for recognition of partially hidden faces in images
https://doi.org/10.17586/0021-3454-2022-65-11-842-850
Abstract
A new approach to solving the problem of automatic face recognition of people using personal protective equipment such as a medical mask has been proposed and tested. This approach is based on the use of methods of generating synthetic images of partially hidden faces and the face recognition model ArcFace. A strategy for training data sets formation is proposed and a number of corresponding recognition models are derived. A series of experiments aimed at assessing the quality of predictions of the obtained solution are carried out, and a relationship between the resulting quality of predictions implemented by recognition models and the volume of synthetic images in training datasets is established. According to the results of experimental studies, neural network models, further trained on datasets with volume of artificially synthesized images of 40-60%, demonstrate values of recognition accuracy above 87% on the AAc quantitative metric (Average Accuracy). Using the proposed approach makes it possible to significantly improve the quality of recognition of partially hidden faces compared to the basic approach.
Keywords
face recognition,
neural network recognition models,
ArcFace,
BRAVE-MASKS,
synthetic image generation,
personal protective equipment,
deep learning
Supporting agencies: the research was carried out at the expense of the Russian Foundation for Basic Research (project N 20-04-60529-viruses), as well as partially within the framework of a leading scientific school (grant N NS-17.2022.1.6).
About the Authors
M. A. Letenkov
St. Petersburg Federal Research Center of the RAS
Russian Federation
Russian Federation
Maхim A. Letenkov — St. Petersburg Institute for Informatics and Automation of the RAS, Laboratory of Big Data Technologies in Socio-Cyberphysical Systems; Junior Researcher
St. Petersburg
R. N. Iakovlev
St. Petersburg Federal Research Center of the RAS
Russian Federation
Russian Federation
Roman N. Iakovlev — St. Petersburg Institute for Informatics and Automation of the RAS, Laboratory of Big Data Technologies in Socio-Cyberphysical Systems; Junior Researcher
St. Petersburg
M. V. Markitantov
St. Petersburg Federal Research Center of the RAS
Russian Federation
Russian Federation
Maxim V. Markitantov — St. Petersburg Institute for Informatics and Automation of the RAS, Speech and Multimodal Interfaces Laboratory; Junior Researcher
St. Petersburg
D. A. Ryumin
St. Petersburg Federal Research Center of the RAS
Russian Federation
Russian Federation
Dmitry A. Ryumin — PhD; St. Petersburg Institute for Informatics and Automation of the RAS, Speech and Multimodal Interfaces Laboratory; Senior Researcher
St. Petersburg
A. A. Karpov
St. Petersburg Federal Research Center of the RAS
Russian Federation
Russian Federation
Alexey A. Karpov — Dr. Sci., Associate Professor; St. Petersburg Institute for Informatics and Automation of the RAS, Speech and Multimodal Interfaces Laboratory; Chief Researcher
St. Petersburg
References
1. Zhang K., Zhang Z., Li Z., Qiao Y. IEEE Signal Processing Letters, 2016, no. 10(23), pp. 1499–1503, DOI: 10.1109/LSP.2016.2603342.
2. Zhang F., Fan X., Ai G., Song J., Qin Y., Wu J. arXiv preprint arXiv:1905.01585, 2019, рр. 1–9.
3. Schroff F., Kalenichenko D., Philbin J. Proceedings of the IEEE conference on computer vision and pattern recognition, 2015, рр. 815–823, DOI: 10.1109/CVPR.2015.7298682.
4. Deng J., Guo J., Xue N., Zafeiriou S. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition, 2019, рр. 4690–4699.
5. He Y., Xu D., Wu L., Jian M., Xiang S., Pan C. arXiv preprint arXiv:1904.10633, 2019, рр. 1–10, DOI: 10.48550/arXiv.1904.10633.
6. Parkhi O. M., Vedaldi A., Zisserman A. Deep face recognition, 2015, рр. 1–12. DOI: 10.5244/C.29.41.
7. Rab S., Javaid M., Haleem A., Vaishya R. Diabetes & Metabolic Syndrome: Clinical Research & Reviews, 2020, no. 6(14), pp. 1617–1619.
8. Martínez-Díaz Y., Méndez-Vázquez H., Luevano L. S., Nicolás-Díaz M., Chang L., González-Mendoza M. IEEE Access., 2021, vol. 10, рр. 7341–7353.
9. Anwar A., Raychowdhury A. arXiv preprint arXiv:2008.11104, 2020, рр. 1–8.
10. Cao Q., Shen L., Xie W., Parkhi O.M., Zisserman A. 13th IEEE International Conference on Automatic Face & Gesture Recognition (FG 2018), IEEE, 2018, рр. 67–74.
11. Guo Y., Zhang L., Hu Y., He X., Gao J. European conference on computer vision, Springer, Cham, 2016, рр. 87–102.
12. Wang Z., Wang G., Huang B., Xiong Z., Hong Q., Wu H., Liang J. arXiv preprint arXiv:2003.09093, 2020, рр. 1–3.
13. Liu W., Wen Y., Yu Z., Li M., Raj B., Song L. Proceedings of the IEEE conference on computer vision and pattern recognition, 2017, рр. 212–220.
14. Wang H., Wang Y., Zhou Z., Ji X., Gong D., Zhou J., Liu W. Proceedings of the IEEE conference on computer vision and pattern recognition, 2018, рр. 5265–5274
Review
For citations:
Letenkov M.A., Iakovlev R.N., Markitantov M.V., Ryumin D.A., Karpov A.A. Application of training data synthesis methods for recognition of partially hidden faces in images. Journal of Instrument Engineering. 2022;65(11):842-850. (In Russ.) https://doi.org/10.17586/0021-3454-2022-65-11-842-850
Views:
13
Email this article 