ANALISIS KLASIFIKASI ULASAN APLIKASI SIREKAP 2024 MENGGUNAKAN EKSTRAKSI FITUR DISTILBERT DAN METODE SUPPORT VECTOR MACHINE

Reno Ridhoi; Naufal Azmi Verdikha; Fendy Yulianto

doi:10.33884/jif.v13i01.9753

Authors

Reno Ridhoi Universitas Muhammadiyah Kalimantan Timur
Naufal Azmi Verdikha Universitas Muhammadiyah Kalimantan Timur
Fendy Yulianto Universitas Muhammadiyah Kalimantan Timur

DOI:

https://doi.org/10.33884/jif.v13i01.9753

Keywords:

DistilBERT, Support Vector Machine, Riview Classification Sirekap, Cross-Validation, Data Imbalance.

Abstract

This study aims to classify reviews of the SIREKAP 2024 application automatically using the DistilBERT feature extraction method and the Support Vector Machine (SVM) algorithm. The data used includes 8,538 user reviews from the Google Play Store with five Rating categories as the target variable. After undergoing 10-Fold cross-validation, the average F1-Score obtained was 36.62%, with the highest performance reaching 37.16%. The analysis indicates that data imbalance is the main obstacle in improving the model's accuracy, particularly in the minority class. The study concludes that the combination of DistilBERT and SVM yields suboptimal results and requires further optimization. Recommendations are provided to improve model accuracy and enhance the quality of the application based on user reviews.

References

M. F. Y. Herjanto and C. Carudin, “Analisis Sentimen Ulasan Pengguna Aplikasi Sirekap Pada Play Store Menggunakan Algoritma Random Forest Classifer,” J. Inform. dan Tek. Elektro Terap., vol. 12, no. 2, pp. 1204–1210, 2024, doi: 10.23960/jitet.v12i2.4192.

K. D. Pratama, D. W. Brata, and W. Purnomo, “Analisis Sentimen Ulasan Pengguna Aplikasi Info BMKG pada Google Play Store di Indonesia,” J. Pengemb. Teknol. Inf. dan Ilmu Komput., vol. 7, no. 4, pp. 1826–1834, 2023, [Online]. Available: https://j-ptiik.ub.ac.id/index.php/j-ptiik/article/view/12539%0Ahttps://j-ptiik.ub.ac.id/index.php/j-ptiik/article/download/12539/5722

F. K. Ihtada, “No TitleΕΛΕΝΗ,” Αγαη, vol. 15, no. 1, pp. 37–48, 2024.

L. Galke et al., “Are We Really Making Much Progress in Text Classification? A Comparative Review,” 2022, [Online]. Available: http://arxiv.org/abs/2204.03954

F. Yulianto, R. Arifando, and A. A. Supianto, “Improved Eliminate Particle Swarm Optimization on Support Vector Machine for Freshwater Fish Classification,” Proc. 2019 4th Int. Conf. Sustain. Inf. Eng. Technol. SIET 2019, pp. 38–43, 2019, doi: 10.1109/SIET48054.2019.8986119.

F. Fajri, B. Tutuko, and S. Sukemi, “Membandingkan Nilai Akurasi BERT dan DistilBERT pada Dataset Twitter,” JUSIFO (Jurnal Sist. Informasi), vol. 8, no. 2, pp. 71–80, 2022, doi: 10.19109/jusifo.v8i2.13885.

A. Akbar Hidayat, “Analisis Ekstraksi Fitur Pada Klasifikasi Teks Menggunakan Algoritma K-Nearest Neighbor (Studi Kasus: Berita Hoaks),” 2022.

G. Darmawan, S. Alam, and M. I. Sulistyo, “Analisis Sentimen Berdasarkan Ulasan Pengguna Aplikasi Mypertamina Pada Google Playstore Menggunakan Metode Naïve Bayes,” STORAGE – J. Ilm. Tek. dan Ilmu Komput., vol. 2, no. 3, pp. 100–108, 2023.

H. B. Assyafah and D. T. Yulianti, “Analisis Dataset menggunakan Sentiment Analysis (Studi Kasus Pada Tripadvisor) ,” Strategi, vol. 3, no. 2, pp. 320–331, 2021, [Online]. Available: https://mail.strategi.it.maranatha.edu/index.php/strategi/article/view/287

M. A. Rohman, “Teknik Ekstraksi Fitur dalam NLP - Sekolah Statistics, Data and Technical Analytics.” Accessed: Dec. 01, 2024. [Online]. Available: https://sekolahstata.com/teknik-ekstraksi-fitur-dalam-nlp/

M. Jojoa, P. Eftekhar, B. Nowrouzi-Kia, and B. Garcia-Zapirain, “Natural language processing analysis applied to COVID-19 open-text opinions using a distilBERT model for sentiment categorization,” AI Soc., vol. 39, no. 3, pp. 883–890, 2024, doi: 10.1007/s00146-022-01594-w.

N. Khoirunnisaa, K. Nabila, N. Kesuma, S. Setiawan, A. Yunizar, and P. Yusuf, “KLASIFIKASI TEKS ULASAN APLIKASI NETFLIX PADA GOOGLE PLAY STORE MENGGUNAKAN ALGORITMA NAÏVE BAYES DAN SVM,” SKANIKA Sist. Komput. dan Tek. Inform., vol. 7, no. 1, pp. 64–73, Jan. 2024, doi: 10.36080/SKANIKA.V7I1.3138.

啓造朝長, “済無No Title No Title No Title,” Angew. Chemie Int. Ed. 6(11), 951–952., vol. 2016, no. 1, pp. 1–2, 1967.

A. Ilham, N. Azmi Verdikha, and A. J. Latipah, “Klasifikasi Ujaran Kebencian di Twitter Menggunakan Fitur Ekstraksi Glove dengan Support Vector Machine (SVM),” J. Explor. IT, vol. 5, no. 36, pp. 64–72, 2023, [Online]. Available: https://doi.org/10.35891/explorit

R. Q. Wang, F. R. Ouedraogo, and S. S. Pathapati, “Preprocessing,” Comput. Fluid Dyn. Appl. Water, Wastewater Stormwater Treat., vol. 3, no. 3, pp. 53–56, 2019, doi: 10.1061/9780784415313.ch06.