Improving Bioethanol Sentiment Analysis Performance using SMOTE in Machine Learning Model Comparison

Rajhu Ilham Pradana, Jasmir Jasmir, Gunardi Gunardi

Abstract


Sentiment analysis of public policies on social media is crucial for government evaluation; however, it is often challenged by highly imbalanced datasets. This study aims to address this issue through a case study on public sentiment toward bioethanol fuel policies on YouTube, where the cleaned dataset after preprocessing consisted of 2,409 comments dominated by negative sentiment (1,430 comments), followed by neutral sentiment (734 comments), and only a small number of positive sentiments (245 comments). The performance of classical Machine Learning (ML) models was severely degraded due to this imbalance, particularly in detecting the minority class. This study applied TF-IDF weighting for feature extraction, followed by the Synthetic Minority Oversampling Technique (SMOTE) to balance the training data (1,927 samples) before comparing the performance of three ML algorithms: Logistic Regression, Support Vector Machine (SVM), and LightGBM. The evaluation results on the testing dataset (482 samples) demonstrate that the implementation of SMOTE significantly improved the models’ ability to recognize the “Positive” class. The LightGBM model combined with SMOTE achieved the best performance, with an accuracy of 64.11%. In particular, the application of SMOTE successfully increased the minority-class F1-score from a baseline of 18.18% to 35.29%. These findings confirm that handling imbalanced data is a critical step in producing valid and reliable sentiment analysis results.

Keywords


sentiment analysis; bioethanol; imbalanced data; LightGBM; SMOTE

Full Text:

PDF

References


H. Gonaygunta, G. S. Nadella, and K. Meduri, “Utilizing Logistic Regression in Machine Learning for Categorizing Social Media Advertisement,” Indonesian Journal of Electrical Engineering and Computer Science, Vol. 37, No. 3, p. 1954, Mar. 2025, DOI: 10.11591/ijeecs.v37.i3.pp1954-1963.

W. Maharani and A. Z. Latifa, “Analyzing Public Sentiment on the Relocation of Indonesia’s Capital to Kalimantan as the Ibu Kota Nusantara using Logistic Regression,” Jurnal Teknik Informatika (Jutif), Vol. 6, No. 2, pp. 575–592, Apr. 2025, DOI: 10.52436/1.jutif.2025.6.2.4230.

B. Darmawan, A. Dwi Laksito, M. Resa, A. Yudianto, and A. Sidauruk, “Analisis Perbandingan Ekstraksi Fitur Teks pada Sentimen Analisis Kenaikan Harga BBM,” Krea-TIF: Jurnal Teknik Informatika, Vol. 11, No. 1, pp. 53–63, 2023, DOI: 10.32832/krea-tif.v11i1.13819.

A. Handayanto, K. Latifa, N. D. Saputro, and R. R. Waliyansyah, “Analisis dan Penerapan Algoritma Support Vector Machine ( SVM ) dalam Data Mining untuk Menunjang Strategi Promosi,” JUITA: Jurnal Informatika, Vol. 7, No. 2, pp. 71–79, 2019, DOI: 10.30595/juita.v7i2.4378.

I. M. Karo Karo and H. Hendriyana, “Klasifikasi Penderita Diabetes menggunakan Algoritma Machine Learning dan Z-Score,” Jurnal Teknologi Terpadu, Vol. 8, No. 2, pp. 94–99, 2022, DOI: 10.54914/jtt.v8i2.564.

U. Khaira, R. Johanda, P. E. P. Utomo, and T. Suratno, “Sentiment Analysis of Cyberbullying on Twitter using SentiStrength,” Indonesian Journal of Artificial Intelligence and Data Mining, Vol. 3, No. 1, p. 21, 2020, DOI: 10.24014/ijaidm.v3i1.9145.

E. Rohaeti and A. Andriyati, “Comparative Study of Predictive Classification Models on Data with Severely Imbalanced Predictors,” JUITA : Jurnal Informatika, Vol. 12, No. 1, p. 121, May 2024, DOI: 10.30595/juita.v12i1.21491.

B. Solihah, A. Azhari, and A. Musdholifah, “The Empirical Comparison of Machine Learning Algorithm for the Class Imbalanced Problem in Conformational Epitope Prediction,” JUITA: Jurnal Informatika, Vol. 9, No. 1, p. 131, May 2021, DOI: 10.30595/juita.v9i1.9969.

N. L. W. S. R. Ginantra, C. P. Yanti, G. D. Prasetya, I. B. G. Sarasvananda, and I. K. A. G. Wiguna, “Analisis Sentimen Ulasan Villa di Ubud menggunakan Metode Naive Bayes, Decision Tree, dan K-NN,” Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), Vol. 11, No. 3, pp. 205–215, Dec. 2022, DOI: 10.23887/janapati.v11i3.49450.

I. Fadil, R. Surya Manggala, E. Firmansyah, and M. A. Helmiawan, “Performance Optimization of Support Vector Machine with SMOTE for Multiclass Stunting Prediction in Sumedang District, Indonesia,” Jurnal Teknik Informatika (Jutif), Vol. 6, No. 4, pp. 2917–2928, Sep. 2025, DOI: 10.52436/1.jutif.2025.6.4.4843.

M. Parhusip, S. Sudianto, and T. G. Laksana, “Sentiment Analysis of the Public Towards the Kanjuruhan Tragedy with the Support Vector Machine Method,” JUITA : Jurnal Informatika, Vol. 11, No. 2, p. 241, Nov. 2023, DOI: 10.30595/juita.v11i2.17405.

H. Atsqalani, N. Hayatin, and C. S. K. Aditya, “Sentiment Analysis from Indonesian Twitter Data Using Support Vector Machine And Query Expansion Ranking,” Jurnal Online Informatika, Vol. 7, No. 1, pp. 116–122, Jun. 2022, DOI: 10.15575/join.v7i1.669.

A. Irianti, H. Halimah, S. Sutedi, and M. Agariana, “Integration of BERT and SVM in Sentiment Analysis of Twitter/X Regarding Constitutional Court Decision No. 60/PUU-XXII/2024,” Jurnal Teknik Informatika (Jutif), Vol. 6, No. 2, pp. 469–482, Apr. 2025, DOI: 10.52436/1.jutif.2025.6.2.4068.

G. V. Gopal and G. R. M. Babu, “An Ensemble Feature Selection Approach using Hybrid Kernel based SVM for network intrusion detection system,” Indonesian Journal of Electrical Engineering and Computer Science, Vol. 23, No. 1, p. 558, Jul. 2021, DOI: 10.11591/ijeecs.v23.i1.pp558-565.

C. Adiwinata and A. Afiyati, “Sentiment Analysis of X Users Toward Electric Motorcycles Using SVM and BERT Algorithms,” JUITA: Jurnal Informatika, Vol. 13, No. 2, pp. 119–126, Aug. 2025, DOI: 10.30595/juita.v13i2.26152.

Indra, Agus Umar Hamdani, Suci Setiawati, Zena Dwi Mentari, and Mauridhy Hery Purnomo, “Comparison of K-NN, SVM, and Random Forest Algorithm for Detecting Hoax on Indonesian Election 2024,” Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), Vol. 13, No. 1, pp. 166–179, Mar. 2024, DOI: 10.23887/janapati.v13i1.76079.

A. S. Sunge, S. Suzanna, and H. M. Mardi Putra, “Interpretable Machine Learning for Employee Recruitment Prediction using Boruta, CatBoost, Lasso, Logistic Regression, NLP, and RFE Feature Selection,” Jurnal Teknik Informatika (Jutif), Vol. 6, No. 4, pp. 2153–2170, Aug. 2025, DOI: 10.52436/1.jutif.2025.6.4.4810.

K. L. Ong, C. P. Lee, H. S. Lim, and K. M. Lim, “Speech Emotion Recognition with Light Gradient Boosting Decision trees machine,” International Journal of Electrical and Computer Engineering (IJECE), Vol. 13, No. 4, p. 4020, Aug. 2023, DOI: 10.11591/ijece.v13i4.pp4020-4028.

W. Wiharto, A. Abdurrahman, and U. Salamah, “Detection of COVID-19 based on Cough Sound and Accompanying Symptom using LightGBM Algorithm,” Indonesian Journal of Electrical Engineering and Computer Science, Vol. 38, No. 2, p. 940, May 2025, DOI: 10.11591/ijeecs.v38.i2.pp940-949.

M. Mar’atuttahirah, K. Tunnisa, D. F. R. Ra, H. Najwa, and J. Fahrisal, “MSMEs Recommendation System using Item-Based Collaborative Filtering and LightGBM Machine Learning,” Jurnal Teknik Informatika (Jutif), Vol. 6, No. 5, pp. 3832–3843, Oct. 2025, DOI: 10.52436/1.jutif.2025.6.5.5292.

S. Uyun and E. Sulistyowati, “Feature Selection for Multiple Water Quality Status: Integrated Bootstrapping And SMOTE Approach in Imbalance Classes,” International Journal of Electrical and Computer Engineering (IJECE), Vol. 10, No. 4, p. 4331, Aug. 2020, DOI: 10.11591/ijece.v10i4.pp4331-4339.

D. Z. Abidin, M. Rosario, A. Sadikin, N. Nurhadi, and J. Jasmir, “Improving Term Deposit Customer Prediction using Support Vector Machine with SMOTE and Hyperparameter Tuning in Bank Marketing Campaigns,” Jurnal Teknik Informatika (Jutif), Vol. 6, No. 3, pp. 1267–1278, Jun. 2025, DOI: 10.52436/1.jutif.2025.6.3.4585.

A. F. Rozi, A. Wibowo, and B. Warsito, “Resampling Technique for Imbalanced Class Handling on Educational Dataset,” JUITA : Jurnal Informatika, Vol. 11, No. 1, p. 77, May 2023, DOI: 10.30595/juita.v11i1.15498.

I. K. A. Purnawan, A. D. Wibawa, A. Kurniawati, and M. H. Purnomo, “Optimizing Diabetic Neuropathy Severity Classification using Electromyography Signals Through Synthetic Oversampling Techniques,” Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), Vol. 13, No. 3, pp. 681–690, Dec. 2024, DOI: 10.23887/janapati.v13i3.85675.

A. Ridho Lubis, M. K. M. Nasution, O. Salim Sitompul, and E. Muisa Zamzami, “The Effect of the TF-IDF Algorithm in Times Series in Forecasting Word on Social Media,” Indonesian Journal of Electrical Engineering and Computer Science, Vol. 22, No. 2, p. 976, May 2021, DOI: 10.11591/ijeecs.v22.i2.pp976-984.

A. D. Sripamuji, I. Ramadhanti, R. R. Amalia, J. Saputra, and B. Prihatnowo, “Penerapan Algoritma Support Vector Machine dan Multi-Layer Perceptron pada Klasifikasi Topik Berita,” Jurnal Nasional Pendidikan Teknik Informatika (JANAPATI), Vol. 11, No. 2, pp. 84–91, 2022, DOI: 10.23887/janapati.v11i2.44151.

S. Defit and G. W. Nurcahyo, “Product Codefication Accuracy with Cosine Similarity and Weighted Term Frequency and Inverse Document Frequency (TF-IDF),” Journal of Applied Engineering and Technological Science, Vol. 2, No. 2, pp. 14–21, 2021, DOI: 10.37385/jaets.v2i2.

F. Muttakin, N. Andrika, and S. Salsabila, “Sentiment Analysis of Shoe Product Reviews on Indonesian E-Commerce Platform Using Lexicon Based and Support Vector Machine,” Jurnal Teknik Informatika (Jutif), vol. 6, no. 2, pp. 839–854, Apr. 2025, doi: 10.52436/1.jutif.2025.6.2.3800.

M. Sopiyan, F. Fauziah, and Y. F. Wijaya, “Fraud Detection using Random Forest Classifier, Logistic Regression, and Gradient Boosting Classifier Algorithms on Credit Cards,” JUITA: Jurnal Informatika, Vol. 10, No. 1, p. 77, May 2022, DOI: 10.30595/juita.v10i1.12050.

N. A. Syam, N. Arifin, W. Firgiawan, and M. F. Rasyid, “Comparison of SVM and Gradient Boosting with PCA for Website Phising Detection,” Jurnal Teknik Informatika (Jutif), Vol. 6, No. 2, pp. 691–708, Apr. 2025, DOI: 10.52436/1.jutif.2025.6.2.4344.

A. R. P. Dewi, S. Riyadi, C. Darmajati, N. A. M. Isa, and A. D. Andriyani, “Sentiment Analysis of Pro-Israel Product Boycott Action using IndoBERT Method on Unbalanced Data,” JUITA: Jurnal Informatika, Vol. 13, No. 2, pp. 187–197, Aug. 2025, DOI: 10.30595/juita.v13i2.25976.

N. Hadi and D. Sugiarto, “Analisis Sentimen Pembangunan IKN pada Media Sosial X menggunakan Algoritma SVM, Logistic Regression dan Naïve Bayes,” Jurnal Informatika: Jurnal Pengembangan IT, Vol. 10, No. 1, pp. 37–49, 2025, DOI: 10.30591/jpit.v10i1.7106.

S. Lonang, A. Yudhana, and M. K. Biddinika, “Analisis Komparatif Kinerja Algoritma Machine Learning untuk Deteksi Stunting,” Jurnal Media Informatika Budidarma, Vol. 7, No. 4, p. 2109, Oct. 2023, DOI: 10.30865/mib.v7i4.6553.

H. Imaduddin, F. Y. A’la, and Y. S. Nugroho, “Sentiment Analysis in Indonesian Healthcare Applications using IndoBERT Approach,” International Journal of Advanced Computer Science and Applications, Vol. 14, No. 8, pp. 113–117, 2023, DOI: 10.14569/IJACSA.2023.0140813.

C. G. L. Pringandana and K. Kusnawi, “A Comparative Analysis of Hyperparameter-Tuned XGBoost and LightGBM for Multiclass Rainfall Classification in Jakarta,” Jurnal Teknik Informatika (Jutif), Vol. 6, No. 4, pp. 2467–2483, Aug. 2025, DOI: 10.52436/1.jutif.2025.6.4.4965.

S. Widodo, F. S. Utomo, and Berlilana, “A Comprehensive Evaluation of CatBoost and LightGBM Algorithms for Honorarium Prediction on Categorical Datasets with Class Imbalance,” JUITA: Jurnal Informatika, Vol. 13, No. 3, pp. 359–370, Nov. 2025, DOI: 10.30595/juita.v13i3.27363.




DOI: https://doi.org/10.32520/stmsi.v15i5.6300

Article Metrics

Abstract view : 0 times
PDF - 0 times

Refbacks

  • There are currently no refbacks.


Creative Commons License
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.