THE EFFECT OF FACIAL ACCESSORY AUGMENTATION ON THE ACCURACY OF DEEP LEARNING-BASED FACIAL RECOGNITION SYSTEMS
Abstract
Abstract: Face recognition based on deep learning has become an important technology in many areas. However, these systems often face challenges in real-world conditions, such as when the face is partially covered by accessories such as masks or glasses. This study aims to evaluate the effect of data augmentation by adding facial accessories (masks, glasses, and a combination of both) and geometric augmentation on the accuracy of face recognition systems. There are three types of datasets used in this method: the original dataset (category 1), the dataset with facial accessories augmentation (category 2), and the dataset with geometric augmentation (category 3). Data augmentation was performed on the training dataset to increase diversity, followed by the face detection process using SCRFD and feature extraction with ArcFace. The model was then trained using Multi-Layer Perceptron (MLP). Based on the results, adding face accessories (category 2) made the model a lot more accurate, hitting 99% accuracy. In category 3, adding geometric features improved accuracy to 91%. Other evaluation metrics, such as precision, recall, and F1-score, also showed improvement after augmentation. This study concludes that facial accessories augmentation is more effective in improving the accuracy and robustness of face recognition models compared to geometric augmentation.
Keywords: augmentation; deep learning; face recognition; glasses.
Abstrak: Pengenalan wajah berbasis deep learning telah menjadi salah satu teknologi penting dalam berbagai aplikasi. Namun, sistem ini sering kali menghadapi tantangan dalam kondisi dunia nyata, seperti saat wajah tertutup sebagian oleh aksesori seperti masker atau kacamata. Penelitian ini bertujuan untuk mengevaluasi pengaruh augmentasi data dengan menambahkan aksesori wajah (masker, kacamata, dan kombinasi keduanya) serta augmentasi geometris terhadap akurasi sistem pengenalan wajah. Metode yang digunakan melibatkan tiga kategori dataset: dataset asli tanpa augmentasi (kategori 1), dataset dengan augmentasi aksesoris wajah (kategori 2), dan dataset dengan augmentasi geometris (kategori 3). Augmentasi data dilakukan pada dataset pelatihan untuk meningkatkan keberagaman, diikuti dengan proses deteksi wajah menggunakan SCRFD dan ekstraksi fitur dengan ArcFace. Model kemudian dilatih menggunakan Multi-Layer Perceptron (MLP). Hasil penelitian menunjukkan bahwa augmentasi aksesoris wajah (kategori 2) memberikan peningkatan signifikan pada akurasi model, mencapai 99%, sedangkan kategori 3 dengan augmentasi geometris mencapai akurasi 91%. Metrik evaluasi lainnya, seperti precision, recall, dan F1-score, juga menunjukkan peningkatan setelah augmentasi. Penelitian ini menyimpulkan bahwa augmentasi aksesoris wajah lebih efektif dalam meningkatkan akurasi dan ketahanan model pengenalan wajah dibandingkan dengan augmentasi geometris.
Kata kunci: augmentasi; deep learning; kacamata; pengenalan wajah.References
X. Chen, “The Advance of Deep Learning and Attention Mechanism,” in Proceedings - 2022 International Conference on Electronics and Devices, Computational Science, ICEDCS 2022, 2022. doi: 10.1109/ICEDCS57360.2022.00078.
I. H. Sarker, “Deep Learning: A Comprehensive Overview on Techniques, Taxonomy, Applications and Research Directions,” SN Comput Sci, vol. 2, no. 6, p. 420, Nov. 2021, doi: 10.1007/s42979-021-00815-1.
J. Chai, H. Zeng, A. Li, and E. W. T. Ngai, “Deep learning in computer vision: A critical review of emerging techniques and application scenarios,” Machine Learning with Applications, vol. 6, p. 100134, Dec. 2021, doi: 10.1016/j.mlwa.2021.100134.
S. Sethi, M. Kathuria, and T. Kaushik, “Face mask detection using deep learning: An approach to reduce risk of Coronavirus spread,” J Biomed Inform, vol. 120, p. 103848, Aug. 2021, doi: 10.1016/j.jbi.2021.103848.
R. A. S. Naseri, A. Kurnaz, and H. M. Farhan, “Optimized face detector-based intelligent face mask detection model in IoT using deep learning approach,” Appl Soft Comput, vol. 134, p. 109933, Feb. 2023, doi: 10.1016/j.asoc.2022.109933.
B. Kocacinar, B. Tas, F. P. Akbulut, C. Catal, and D. Mishra, “A Real-Time CNN-Based Lightweight Mobile Masked Face Recognition System,” IEEE Access, vol. 10, pp. 63496–63507, 2022, doi: 10.1109/ACCESS.2022.3182055.
M. Umer et al., “Face mask detection using deep convolutional neural network and multi-stage image processing,” Image Vis Comput, vol. 133, p. 104657, May 2023, doi: 10.1016/j.imavis.2023.104657.
H. Yang and X. Han, “Retracted: Face Recognition Attendance System Based on Real-Time Video Processing,” IEEE Access, vol. 8, pp. 159143–159150, 2020, doi: 10.1109/ACCESS.2020.3007205.
M. A. Ahmed, M. D. Salman, R. A. Alsharida, Z. T. Al-Qaysi, and M. M. Hammood, “An Intelligent Attendance System Based On Convolutional Neural Networks For Real-Time Student Face Identifications,” Journal of Engineering Science and Technology, vol. 17, no. 5, 2022.
X. Wang, K. Wang, and S. Lian, “A survey on face data augmentation for the training of deep neural networks,” Neural Comput Appl, vol. 32, no. 19, pp. 15503–15531, Oct. 2020, doi: 10.1007/s00521-020-04748-3.
Waleed Ayad, Siraj Qays, and Ali Al-Naji, “Generating and Improving a Dataset of Masked Faces Using Data Augmentation,” Journal of Techniques, vol. 5, no. 2, pp. 46–51, Jun. 2023, doi: 10.51173/jt.v5i2.1140.
M. R. Prusty, V. Tripathi, and A. Dubey, “A novel data augmentation approach for mask detection using deep transfer learning,” Intell Based Med, vol. 5, p. 100037, 2021, doi: 10.1016/j.ibmed.2021.100037.
M. F. Ozdemir and D. Hanbay, “Deep Learning Based Real-Time Face Tracking System in Multi-Camera,” SSRN Electronic Journal, 2022, doi: 10.2139/ssrn.4098733.
M. F. Ozdemir and D. Hanbay, “Deep Learning Based Real-Time Face Tracking System in Multi-Camera,” SSRN Electronic Journal, 2022, doi: 10.2139/ssrn.4098733.
R. Arun Kumar, V. A. Solayappan, S. S. T, and R. P. K, “Masked Deep Face Recognition using ArcFace and Ensemble Learning,” in 2021 IEEE 2nd International Conference on Technology, Engineering, Management for Societal impact using Marketing, Entrepreneurship and Talent (TEMSMET), IEEE, Dec. 2021, pp. 1–6. doi: 10.1109/TEMSMET53515.2021.9768777.
P.-S. Pham, M.-K. Nguyen, M.-H. Le, M.-T. Tran, and T.-N. Le, “Masked Face Recognition on Limited Training Data,” in 2024 International Conference on Multimedia Analysis and Pattern Recognition (MAPR), IEEE, Aug. 2024, pp. 1–6. doi: 10.1109/MAPR63514.2024.10661046.
T. M. S. N. Alamsyah, T. F. Abidin, R. Ferdhiana, M. Dirhamsyah, and M. Chaidir, “Analysis of Face Data Augmentation in Various Poses for Face Recognition Model,” in 2022 7th International Conference on Informatics and Computing, ICIC 2022, 2022. doi: 10.1109/ICIC56845.2022.10006997.
P. Rahmatullah, T. F. Abidin, A. Misbullah, and Nazaruddin, “Effectiveness of Data Augmentation in Multi-class Face Recognition,” in 2021 5th International Conference on Informatics and Computational Sciences (ICICoS), IEEE, Nov. 2021, pp. 64–68. doi: 10.1109/ICICoS53627.2021.9651780.
A. Ma, N. Dvornik, R. Zhang, L. Pishdad, K. G. Derpanis, and A. Fazly, “SAGE: Saliency-Guided Mixup with Optimal Rearrangements,” in BMVC 2022 - 33rd British Machine Vision Conference Proceedings, 2022.
S. Zeng, B. Zhang, Y. Zhang, and J. Gou, “Dual sparse learning via data augmentation for robust facial image classification,” International Journal of Machine Learning and Cybernetics, vol. 11, no. 8, 2020, doi: 10.1007/s13042-020-01067-w.
E. Ghorbel, G. Maddouri, and F. Ghorbel, “Face Blending Data Augmentation for Enhancing Deep Classification,” in International Conference on Pattern Recognition Applications and Methods, 2024. doi: 10.5220/0012357900003654.
R. Shah, M. Purohit, S. Gadhe, K. Mehrotra, and S. Mehta, “FaceReZero Transformer for Unbiased Occlusion Invariant Deep Face Recognition with Train-Time Augmentations,” in 2023 Seventh International Conference on Image Information Processing (ICIIP), IEEE, Nov. 2023, pp. 73–78. doi: 10.1109/ICIIP61524.2023.10537650.
H. A. Sung, S. Kim, and E. C. Lee, “Masked Face Recognition Model with Explainable AI,” 2023, pp. 154–159. doi: 10.1007/978-3-031-27199-1_16.
Z. Ye, H. Zhang, X. Li, and Q. Zhang, “DeMaskGAN: a de-masking generative adversarial network guided by semantic segmentation,” Vis Comput, vol. 40, no. 8, pp. 5605–5618, Aug. 2024, doi: 10.1007/s00371-023-03125-0.
M. Anusha and P. Kiruthika, “A Comparative Study on Augmented Analytics Using Deep Learning Techniques,” in Smart Innovation, Systems and Technologies, 2022. doi: 10.1007/978-981-16-3675-2_9.
N. J. Shoumy, L. M. Ang, D. M. M. Rahaman, T. Zia, K. P. Seng, and S. Khatun, “Improving Human Emotion Recognition from Emotive Videos Using Geometric Data Augmentation,” in Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), 2021. doi: 10.1007/978-3-030-79463-7_13.
