VIRTUAL MOUSE WITH HAND GESTURE RECOGNITION BASED ON HAND LANDMARK MODEL FOR POINTING DEVICE
Abstract
Abstract: Technology is growing rapidly and has become one of the human needs that must be owned to solve the problems being faced. The development of touchless input devices or hand gesture recognition using a camera is a form of machine learning. Gestures can define as physical movements of the hands, arms, or body as expressive messages, besides that this hand gesture system can explain the contents of commands that have meaning. In this research, a virtual mouse system will be developed using hand gesture recognition based on the hand landmark model for pointing devices. The resulting application can be run on a desktop device using a webcam. The results of the tests carried out to analyze the implementation of the hand landmark model into the system show that the average system accuracy reaches 96% and the speed reaches 0.05 seconds.
Keywords: hand gesture recognition, hand landmark models, machine learning, virtual mouse
Abstract: Teknologi semakin pesat dan sudah menjadi salah satu kebutuhan manusia yang harus dimiliki untuk menyelesaikan permasalahan yang sedang dihadapi. Perkembangan piranti masukan tanpa sentuhan atau hand gesture recognition menggunakan kamera adalah salah satu bentuk dari machine learning. Gestur mampu mendefinisikan sebagai gerakan fisik dari tangan, lengan, maupun badan sebagai pesan yang ekspresif, selain itu sistem gestur tangan ini mampu menjelaskan isi perintah yang memiliki arti. Dalam penelitian ini akan dikembangkan sebuah sistem virtual mouse menggunakan hand gesture recognition berbasis hand landmark model untuk pointing device. Aplikasi yang dihasilkan dapat dijalankan pada perangkat desktop dengan menggunakan webcam. Hasil dari pengujian yang dilakukan untuk menganalisa penerapan hand landmark model kedalam sistem menunjukkan rata-rata akurasi sistem mencapai 96% dan kecepatan mencapai 0.05 second.
Keywords: hand gesture recognition, hand landmark models, machine learning, virtual mouse
References
M. Ngafifi, “Kemajuan Teknologi Dan Pola Hidup Manusia Dalam Perspektif Sosial Budaya,†J. Pembang. Pendidik. Fondasi dan Apl., vol. 2, no. 1, pp. 33–47, 2014, doi: 10.21831/jppfa.v2i1.2616.
S. N. Supal, M. Hayati, and M. Yatim, “Kemahiran meta kognitif kanak-kanak dalam penggunaan perisian dan peranti input tanpa sentuhan leap motion,†J. ICT Educ., vol. 7, no. 2, pp. 1–13, 2020.
Murthy;Jadon, “A Review of Vision Based Dynamic Hand Gestures Recognition,†Comput. Sci. Appl., vol. 10, no. 05, pp. 990–1001, 2009, doi: 10.12677/csa.2020.105102.
Falabiba, “Modul Interaksi Manusia dan Komputer,†2019.
A. Sunyoto and A. Harjoko, “Review Teknik, Teknologi, Metodologi dan Implementasi Pengenalan Gestur Tangan Berbasis Visi,†Semin. Nas. Apl. Teknol. Inf., p. H-7, 2014, [Online]. Available: http://jurnal.uii.ac.id/Snati/article/view/3290
S. Anwar, S. K. Sinha, S. Vivek, and V. Ashank, “Hand gesture recognition: A survey,†Lect. Notes Electr. Eng., vol. 511, no. 3, pp. 365–371, 2019, doi: 10.1007/978-981-13-0776-8_33.
P. Garg, N. Aggarwal, and S. Sofat, “Vision based hand gesture recognition,†Proc. Int. Conf. Serv. Sci. ICSS, pp. 260–265, 2009, doi: 10.1109/ICSS.2013.40.
A. Mulder, “Hand gestures for HCI,†Hand Centered Stud. Hum. Mov. Proj. …, no. February, pp. 1–19, 1996, [Online]. Available: http://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Hand+Gestures+for+HCI#3%5Cnhttp://scholar.google.com/scholar?hl=en&btnG=Search&q=intitle:Hand+gestures+for+HCI#3
Y. Velaskar, A. Dulam, S. Sureliya, S. Shenoy, and C. Chouhan, “Computer Vision based Hand Gesture Interfaces,†Int. J. Eng. Res. Technol., vol. 05, no. 01, pp. 1–4, 2017, [Online]. Available: www.ijert.org
T. Simon, H. Joo, I. Matthews, and Y. Sheikh, “Hand keypoint detection in single images using multiview bootstrapping,†Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 4645–4653, 2017, doi: 10.1109/CVPR.2017.494.
H. Yunita and E. Setyati, “Hand Gesture Recognition Sebagai Pengganti Mouse Komputer Menggunakan Kamera,†J. ELTIKOM, vol. 3, no. 2, pp. 64–76, 2019, doi: 10.31961/eltikom.v3i2.114.
A. Halder and A. Tayade, “Sign Language to Text and Speech Translation in Real Time Using Convolutional Neural Network,â€,†Int. J. Res. Publ. Rev., no. 2, pp. 9–17, 2021.
N. Yuliana, K. Ratri, and R. Wardani, “Metode Convex Hull dan Convexity Defects untuk Pengenalan Isyarat Tangan,†J. Telemat., vol. 11, no. 2, pp. 81–88, 2016.
M. M. Islam, S. Siddiqua, and J. Afnan, “Real time Hand Gesture Recognition using different algorithms based on American Sign Language,†2017 IEEE Int. Conf. Imaging, Vis. Pattern Recognition, icIVPR 2017, 2017, doi: 10.1109/ICIVPR.2017.7890854.
N. S. Rokade, H. R. Jadhav, S. A. Pathan, and U. Annamalai, “Hand Gesture Recognition System For Multimedia Applications,†Int. Res. J. Eng. Technol., pp. 1894–1899, 2016, [Online]. Available: www.irjet.net
D. K V, H. E, N. Jain D, and N. Reddy B, “Hand Gesture Recognition and Voice Conversion for Hearing and Speech Aided Community,†Int. J. Sci. Res. Comput. Sci. Eng. Inf. Technol., vol. 6, no. 3, pp. 223–228, 2020, doi: 10.32628/cseit206346.
R. Munawarah, O. Soesanto, and M. R. Faisal, “Penerapan Metode Support Vector Machine,†Kumpul. J. Ilmu Komput., vol. 04, no. 01, pp. 103–113, 2016.
W. Liu et al., “SSD: Single shot multibox detector,†Lect. Notes Comput. Sci. (including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics), vol. 9905 LNCS, pp. 21–37, 2016, doi: 10.1007/978-3-319-46448-0_2.
T. M. Kadarina and M. H. Ibnu Fajar, “Pengenalan Bahasa Pemrograman Python Menggunakan Aplikasi Games Untuk Siswa/I Di Wilayah Kembangan Utara,†J. Abdi Masy., vol. 5, no. 1, p. 11, 2019, doi: 10.22441/jam.2019.v5.i1.003.
T. C. A.-S. Zulkhaidi, E. Maria, and Y. Yulianto, “Pengenalan Pola Bentuk Wajah dengan OpenCV,†J. Rekayasa Teknol. Inf., vol. 3, no. 2, p. 181, 2020, doi: 10.30872/jurti.v3i2.4033.
S. S. Makahaube, A. M. Sambul, and S. R. U. Sompie, “Implementation of Gesture Recognition Technology for Automated Education Service Kiosk,†vol. 16, no. 4, pp. 465–472, 2021.
Z. Ren, J. Meng, and J. Yuan, “Depth Camera Based Hand Gesture Recognition and its Applications in Human-Computer-Interaction,†pp. 1–5, 2011.
