COMPARISON SVM, RF, BERT PUBLIC SENTIMENT DATA MBG IN X

Abstract

Abstract: MBG is a strategic program of the Prabowo-Gibran administration. This program has become a widely discussed issue in the public. To better understand public perception of this program, sentiment analysis is necessary. This study aims to compare the performance of algorithms machine learning SVM, RF, And BERT with preprocessing data analyzing public sentiment of the MBG program in media X. The total dataset for this study was 39,858 out of 42,465 successfully crawled tweets. The research methods included data collection, preprocessing data (cleaning, case folding, word normalization, stopword removal and stemming), feature extraction, model training (fine-tuning), handling class imbalance with SMOTE, and evaluation using accuracy, precision, recall, and f1-score. The research results show that without SMOTE, the best performing models are BERT with 89% accuracy, SVM 87%, and RF 78.4%. After SMOTE, the best algorithms were SVM with 92.94%, BERT with 88.3%, and RF with 86.59%. The results confirmed that SVM is the best algorithm if at leastclass imbalance. BERT is the best algorithm before and after SMOTE, because BERT is more effective in capturing the nuances of language on social media, so BERT is the most recommended in MBG sentiment analysis.

           
Keywords: sentiment analysis; machine learning; SVM, RF, and BERT

Abstrak: MBG merupakan program strategis pemerintahan Prabowo - Gibran. Program ini menjadi isu yang banyak diperbincangkan publik. Untuk mengetahui lebih dalam persepsi masyrakat tentang program ini, perlu dilakukan analisis sentiment. Penelitian ini bertujuan membandingkan kinerja algoritma machine learning SVM, RF, dan BERT dengan preprocessing data menganalisis sentiment public program MBG di media X. Total dataset penelitian ini adalah 39.858 dari 42.465 tweet yang berhasil di crawling. Metode penelitian mencakup pengumpulan data, preprocessing data (cleaning, case folding, normalisasi kata, stopword removal dan stemming), ekstraksi fitur, pelatihan model (fine-tuning), penanganan class imbalance dengan SMOTE, dan evaluasi menggunakan akurasi, presisi, recall, dan f1-score. Hasil peneltian menunjukkan, tanpa SMOTE model dengan kinerja terbaik adalah BERT dengan akurasi 89%, SVM 87%, dan RF 78,4%. Setelah SMOTE algoritma terbaik adalah SVM 92,94%, BERT 88,3% dan RF 86,59%. Hasil penelitian menegaskan bahwa SVM adalah algoritma terbaik jika minimal class imbalance. BERT adalah algoritma terbaik sebelum dan sesudah SMOTE, karena BERT lebih efektif dalam menangkap nuansa bahasa pada media sosial, sehingga BERT paling di rekomendasikan dalam analisis sentimen MBG.

Kata kunci: analisis sentimen; machine learning; SVM, RF, dan BERT