Optimization of Artificial Intelligence (AI) Dependency Level Classification Among Students Using Random Forest with SMOTE Oversampling Technique to Address Data Imbalance

Optimization of Artificial Intelligence (AI) Dependency Level Classification Among Students Using Random Forest with SMOTE Oversampling Technique to Address Data Imbalance

• By Yadi Kuntoro, Antonius;
• 6
• 380-385

The development of generative artificial intelligence (AI) has significantly transformed the way students complete academic
tasks. Excessive reliance on AI has the potential to diminish students' creativity and independent thinking abilities. This study
aims to classify the level of students' dependency on AI in relation to academic creativity using the Random Forest algorithm
optimized with the Synthetic Minority Over-sampling Technique (SMOTE) to address class imbalance in the data. Data were
collected through a structured questionnaire using a Likert scale of 1–5 consisting of 24 statement items from 74 student
respondents at Politeknik 'Aisyiyah Pontianak, covering variables of AI usage intensity, dependency behavior, creativity,
independent thinking, and academic motivation. Respondents were classified into three classes: Low (n=30), Moderate (n=19),
and High (n=25). The results indicate that the Random Forest + SMOTE model evaluated with Cross Validation achieved an
accuracy of 97.14% (±6.02%), a precision of 97.06% (micro average), and recall values of 100% for the Low class, 95.83% for
the Moderate class, and 95.00% for the High class. The most dominant feature was the tendency to think of AI-generated answers when questioned by lecturers (B4, importance=0.1097), followed by AI usage intensity of more than 3 hours per day (A2,
importance=0.0872). These findings contribute to the development of an early detection system for AI dependency among students in higher education institutions.

• Katakunci Data Mining, AI Dependency, Data Classification, Student Creativity, Random Forest.
• URL https://ejournal.ikado.ac.id/index.php/teknika/index
• Penerbit Institut Informatika Indonesia - TEKNILA - Journal of Information and Communication Technology, Surabaya Jawa Timur

REFERENSI

[1] M. Farrokhnia, S. K. Banihashem, O. Noroozi, and A. Wals, "A SWOT analysis of ChatGPT: Implications for educational practice and research," Innovations Educ. Teach. Int., vol. 61, no. 3, pp. 460–474, 2023. https://doi.org/10.1080/14703297.2023.2195846

[2] C. Zhai, S. Wibowo, and L. D. Li, "The effects of over reliance on AI dialogue systems on students’ cognitive abilities: A systematic review," Smart Learning Environments, vol. 11, no. 1, p. 28, 2024. https://doi.org/10.1186/s40561-024-00316-7

[3] E. Kasneci et al., "ChatGPT for good? On opportunities and challenges of large language models for education," Learning and Individual Differences, vol. 103, p. 102274, 2023. https://doi.org/10.1016/j.lindif.2023.102274

[4] H. He and E. A. Garcia, "Learning from imbalanced data," IEEE Trans. Knowl. Data Eng., vol. 21, no. 9, pp. 1263–1284, 2009. https://doi.org/10.1109/TKDE.2008.239

[5] B. Charbuty and A. Abdulazeez, "Classification based on decision tree algorithm for machine learning," J. Appl. Sci. Technol. Trends, vol. 2, no. 01, pp. 20–28, 2021. https://doi.org/10.38094/jastt20165

[6] T. M. Amabile, Creativity in Context: Update to the Social Psychology of Creativity. Westview Press, 1996

[7] Q. V. Liao and S. S. Sundar, "Designing for responsible trust in AI systems: A communication perspective," in Proc. 2022 ACM Conf. Fairness, Accountability, Transparency (FAccT ’22), pp. 1257–1268, 2022. https://doi.org/10.1145/3531146.3533182

[8] L. Breiman, "Random forests," Machine Learning, vol. 45, no. 1, pp. 5–32, 2001. https://doi.org/10.1023/A:1010933404324

[9] M. Ndiaye, O. Sarr, and D. Seck, "Oversampling methods for imbalanced datasets in classification: A systematic review," Artif. Intell. Rev., vol. 56, pp. 13057–13081, 2023. https://doi.org/10.1007/s10462023-10492-8

[10] H. Hermanto, A. Mustopa, and A. Y. Kuntoro, "Algoritma Klasifikasi Naive Bayes Dan Support Vector Machine Dalam Layanan Komplain Mahasiswa," JITK (Jurnal Ilmu Pengetahuan Dan Teknologi Komputer), vol. 5, no. 2, pp. 211–220, Feb. 2020. https://doi.org/10.33480/jitk.v5i2.1181’

[11] Hermanto, R. Fahlapi, A. Y. Kuntoro, and T. Asra, "Perbandingan Algoritma Klasifikasi Analisis Sentimen Pengguna Aplikasi Getcontact Dalam Pencegahan Penipuan Online," J-INTECH (Journal of Information and Technology), pp. 158–167, 2024

[12] N. V. Chawla, K. W. Bowyer, L. O. Hall, and W. P. Kegelmeyer, "SMOTE: Synthetic minority over sampling technique," J. Artif. Intell. Res., vol. 16, pp. 321–357, 2002. https://doi.org/10.1613/jair.953

[13] E. P. Torrance, Torrance Tests of Creative Thinking: Norms-Technical Manual. Personnel Press, 1966

[14] S. Grassini, "Shaping the future of education: Exploring the potential and consequences of AI and ChatGPT in educational settings," Education Sciences, vol. 13, no. 7, p. 692, 2023. https://doi.org/10.3390/educsci13070692

[15] Y. Wardat, M. A. Tashtoush, R. AlAli, and A. M. Jarrah, "ChatGPT: A revolutionary tool for teaching and learning mathematics," EURASIA J. Math. Sci. Technol. Educ., vol. 19, no. 7, p. em2286, 2023. https://doi.org/10.29333/ejmste/13272

[16] A. Noviriandini, Hermanto, and Yudhistira, "Klasifikasi Support Vector Machine Berbasis Particle Swarm Optimization Untuk Analisa Sentimen Pengguna Aplikasi Pedulilindungi," JIKA (Jurnal Informatika), vol. 6, no. 1, pp. 50–56, https://doi.org/10.31000/jika.v6i1.5681A