COMPARISON OF NAÏVE BAYES, SVM, K-NN, DECISION TREE, AND RANDOM FOREST IN SENTIMENT ANALYSIS BASED ON SEABANK  APPLICATION ASPECTS

Muhammad Al Fachrozi; Ken Ditha Tania

doi:10.33330/jurteksi.v12i1.4189

Muhammad Al Fachrozi Universitas Sriwijaya
Ken Ditha Tania Universitas Sriwijaya

DOI: https://doi.org/10.33330/jurteksi.v12i1.4189

Keywords: Analisis Sentimen, Aspek, Bank Digital, SeaBank, CRISP-DM

Abstract

Abstract: The increasing use of digital banking applications has led to the need for a deeper understanding of user perceptions, especially through aspect-based sentiment analysis. This study aims to classify the sentiment of SeaBank app users by focusing on four main aspects: learnability, efficiency, technical issues or errors, and satisfaction. Review data totaling 1,971 comments were collected from the Google Play Store and labeled with sentiments based on the scores (ratings) given by users. The CRISP-DM approach serves as the methodological framework for this study, which includes five classification algorithms: Naïve Bayes, Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Decision Tree, and Random Forest. The evaluation results show that the SVM algorithm provides the best performance with the highest average value of the four aspects achieving accuracy of 93.91%, Precision of 91.16%, recall of 97.96% and F1-Measure of 94.33%. According to the research findings, the Support Vector Machine (SVM) algorithm provides the best performance when performing aspect-based sentiment analysis on text data from digital banking application reviews. The findings are expected to serve as a reference for the development of automated evaluation systems that rely on user opinions as the basis for decision making.

Keywords: aspects; CRISP-DM; digital Banking; seabank; sentiment analysis

Abstrak: Peningkatan pemakaian aplikasi perbankan digital mendorong perlunya pemahaman yang lebih dalam mengenai persepsi pengguna, terutama melalui analisis sentimen berbasis aspek. Penelitian ini bertujuan untuk mengklasifikasikan sentimen pengguna aplikasi SeaBank dengan berfokus pada empat aspek utama: kemudahan dipelajari (learnability), efisiensi penggunaan (efficiency), kendala atau kesalahan teknis (error), serta tingkat kepuasan (satisfaction). Data ulasan berjumlah 1.971 komentar dikumpulkan dari Google Play Store dan diberi label sentimen berdasarkan skor (rating) yang diberikan oleh pengguna. Pendekatan CRISP-DM berfungsi sebagai kerangka metodologis untuk penelitian ini, yang mencakup lima algoritma klasifikasi: Naïve Bayes, Support Vector Machine (SVM), k-Nearest Neighbor (k-NN), Decision Tree, dan Random Forest. Hasil evaluasi menunjukkan bahwa algoritma SVM memberikan performa terbaik dengan nilai rata-rata dari ke empat aspek tertinggi yang mencapai accuracy sebesar 93.91%, Precision sebesar 91.16%, recall sebesar 97.96% dan F1-Measure sebesar 94.33%. Menurut temuan penelitian, algoritma Support Vector Machine (SVM) memberikan kinerja terbaik saat melakukan analisis sentimen berbasis aspek pada data teks dari ulasan aplikasi Seabank. Temuan ini diharapkan dapat menjadi referensi bagi pengembangan sistem evaluasi otomatis yang mengandalkan opini pengguna sebagai dasar pengambilan keputusan.

Kata kunci: Analisis Sentimen, Aspek, Bank Digital, SeaBank, CRISP-DM

References

[1] D. W. Bhatara and R. R. Suryono, “Analisis Sentimen Aplikasi BCA Mobile Menggunakan Algoritma Naive Bayes dan Suport Vector Machine,” JIPI (Jurnal Ilm. Penelit. dan Pembelajaran Inform., vol. 9, no. 4, pp. 1907–1917, Nov. 2024, doi: 10.29100/jipi.v9i4.5536.
[2] I. Tahyudin, A. Rafi Hananto, S. Anggun Rahayu, R. M. Anjani, and A. Nurhopipah, “Sentiment Analysis Model Development on E-Money Service Complaints,” TEM J., pp. 2050–2055, Nov. 2023, doi: 10.18421/TEM124-15.
[3] N. Sakir, J. N. U. Jaya, and N. Wahyuni, “Penerapan Metode Pieces Framework Sebagai Evaluasi Tingkat Kepuasan Pengguna Aplikasi Seabank di Balikpapan,” JURIKOM (Jurnal Ris. Komputer), vol. 9, no. 2, p. 344, Apr. 2022, doi: 10.30865/jurikom.v9i2.4047.
[4] H. A. Alnemer, “Determinants of digital banking adoption in the Kingdom of Saudi Arabia: A technology acceptance model approach,” Digit. Bus., vol. 2, no. 2, p. 100037, 2022, doi: 10.1016/j.digbus.2022.100037.
[5] S. J and K. U, “Sentiment analysis of amazon user reviews using a hybrid approach,” Meas. Sensors, vol. 27, p. 100790, Jun. 2023, doi: 10.1016/j.measen.2023.100790.
[6] A. Alsokkar, M. Otair, H. E. Alfar, A. Y. Nasereddin, K. Aldiabat, and L. Abualigah, “Sentiment analysis for Arabic call center notes using machine learning techniques,” J. Auton. Intell., vol. 7, no. 3, Jan. 2024, doi: 10.32629/jai.v7i3.940.
[7] V. Novalia, K. Ditha Tania, A. Meiriza, and A. Wedhasmara, “Knowledge Discovery of Application Review Using Word Embedding’s Comparison with CNN-LSTM Model on Sentiment Analysis,” in 2024 International Conference on Electrical Engineering and Computer Science (ICECOS), IEEE, Sep. 2024, pp. 234–238. doi: 10.1109/ICECOS63900.2024.10791113.
[8] A. Sabir, H. A. Ali, and M. A. Aljabery, “ChatGPT Tweets Sentiment Analysis Using Machine Learning and Data Classification,” Informatica, vol. 48, no. 7, May 2024, doi: 10.31449/inf.v48i7.5535.
[9] Dhina Nur Fitriana and Yuliant Sibaroni, “Sentiment Analysis on KAI Twitter Post Using Multiclass Support Vector Machine (SVM),” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 4, no. 5, pp. 846–853, Oct. 2020, doi: 10.29207/resti.v4i5.2231.
[10] S. Masturoh and A. B. Pohan, “ANALISIS SENTIMEN TERHADAP E-WALLET DANA PADA ULASAN GOOGLE PLAY MENGGUNAKAN ALGORITMA K-NEAREST NEIGHBOR,” J. Pilar Nusa Mandiri, vol. 17, no. 1, pp. 53–58, Apr. 2021, doi: 10.33480/pilar.v17i1.2182.
[11] R. M. Harun and F. Fahmi, “Analisis Sentimen Menggunakan Algoritma Naïve Bayes, KNN, dan Decision Tree Terhadap Ulasan Aplikasi KitaLulus,” Build. Informatics, Technol. Sci., vol. 6, no. 3, pp. 2033–2042, Dec. 2024, doi: 10.47065/bits.v6i3.6367.
[12] T. A.M and A. Yaqin, “Perbandingan Algoritma Naïve Bayes, K-Nearest Neighbors dan Random Forest untuk Klasifikasi Sentimen Terhadap BPJS Kesehatan pada Media Twitter,” InComTech J. Telekomun. dan Komput., vol. 12, no. 1, p. 01, Apr. 2022, doi: 10.22441/incomtech.v12i1.13642.
[13] G. E. Chanchí Golondrino, M. A. Ospina Alarcón, and L. M. Sierra Martínez, “Determination of the satisfaction attribute in usability tests using sentiment analysis and fuzzy logic,” Int. J. Comput. Commun. Control, vol. 18, no. 3, May 2023, doi: 10.15837/ijccc.2023.3.4901.
[14] A. J. Syahid and D. Mahdiana, “Perbandingan Algoritma Untuk Klasifikasi Analisis Sentimen Terhadap Genose Pada Media Sosial Twitter,” semanTIK, vol. 7, no. 1, p. 9, 2021, doi: 10.55679/semantik.v7i1.18087.
[15] J. W. Iskandar and Y. Nataliani, “Perbandingan Naïve Bayes, SVM, dan k-NN untuk Analisis Sentimen Gadget Berbasis Aspek,” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 5, no. 6, pp. 1120–1126, Dec. 2021, doi: 10.29207/resti.v5i6.3588.
[16] R. Q. Rohmansa, N. Pratiwi, and M. J. Palepa, “Analisis Sentimen Ulasan Pengguna Aplikasi Discord Menggunakan Metode K-Nearest Neighbor,” JIPI (Jurnal Ilm. Penelit. dan Pembelajaran Inform., vol. 9, no. 1, pp. 368–378, 2024, doi: 10.29100/jipi.v9i1.4943.
[17] R. Puspita and A. Widodo, “Perbandingan Metode KNN, Decision Tree, dan Naïve Bayes Terhadap Analisis Sentimen Pengguna Layanan BPJS,” J. Inform. Univ. Pamulang, vol. 5, no. 4, p. 646, Dec. 2021, doi: 10.32493/informatika.v5i4.7622.
[18] Rusydi Umar, Imam Riadi, and Purwono, “Comparison of SVM, RF and SGD Methods for Determination of Programmer’s Performance Classification Model in Social Media Activities,” J. RESTI (Rekayasa Sist. dan Teknol. Informasi), vol. 4, no. 2, pp. 329–335, Apr. 2020, doi: 10.29207/resti.v4i2.1770.