Impact of Banning AI in IT Classes on Students'

Academic Performance Perception

Louis Jason, Evelyn Chin, Gavin Bongso, Elin
Information Technology & Management
Ming Chuan University

Introduction
The influence of AI in the education sector is rapidly increasing alongside digitalization and affects students' help-seeking behavior in this era. Artificial intelligence-powered chatbots can provide more personalized learning experiences
and help students become better learners by evaluating their responses and providing feedback (Liu et al., 2022). Schulsche et al. (2023) found that features on smartphones are also appealing to students looking for online-based
academic support since smartphones are common for students to contact each other. However, it negatively impacts programming students’ behavior. Instead of attempting to correct the errors in their programming code themselves,
certain students opted to look up solutions from multiple internet sources (Muller et al., 2020).

AI affects IT students’ study habits. Programming is a sophisticated field that demands proficiency in analyzing algorithms, data structure, mathematics, and formal logic, as well as performing tasks like debugging and testing (Sagar et al.,
2016). Artificial intelligence tries to replicate human assistance by offering tips and guidance to students who encounter programming difficulties (Price et al., 2017). Furthermore, as a teaching tool, engaging new technology can provide
programming students with more rapid results while also increasing their motivation to invest more time in their studies (Konecki et al., 2015). On the other hand, artificial intelligence has the capacity to differentiate software engineering
and data management learning materials for different skill levels (Gulwani, 2022).

Previous research mostly focused on how AI affects and influences IT students in their school life. Khare et al. (2018) stated that in response to the growing demand for student services and assistance in studying, educational institutions
have started using artificial intelligence-powered chatbots. Additionally, significant improvements in AI technology have enabled software engineers to generate programming code from any language (Narasimhan et al., 2021). Moreover,
machine learning has also led to a growing desire to automate various processes, particularly in creating websites automatically based on user intentions (Kaluarachchi & Wickramasinghe, 2023). As a result, advanced website builders
that utilize machine learning algorithms are being developed so that even students with no programming skill are also able to create their own websites. However, it is unclear whether these automatic website builders will meet the same
accessibility standards as websites created by human designers (Pillai et al., 2022). The AI sophistication makes such IT abilities as code programming, creating websites, managing databases, and designing user interface questionable.
Therefore, this paper aims to investigate how banning AI tools in IT classes impacts students’ perception of their academic performance.

Research Problem method

Participants
RQ1: What are the potential benefits and drawbacks of banning the use of AI for IT students?
30 Taiwanese IT students
14 Males and 16 Females, aged 19 to 24
RQ2: How confident are students with their ability and how helpful is AI for them?
Material
RQ3: How does banning AI use in IT classes impact students’ perception of their academic performance?
Demographic scale which gathered background information
AI experience section which probed students’ engagement with and use of AI
AI banning perceptions section which examined students' perceived academic outcomes.

Procedure
Surveys were distributed via social media.
Participants were recruited via online form.

REsult Participants were asked to complete the survey.

The findings indicates that students recognize the potential benefits of incorporating AI tools in their IT
classes, as it can enhance their academic performance and efficiency in completing assignments. Conversely,
when AI is disallowed, students express concerns about a possible decrease in their academic performance, Table 2. Student AI experience
feeling less productive and efficient in their work. These perceptions highlight the significant role that AI plays
in shaping students' educational experiences and outcomes in the IT field (Table 1). The survey result provided
student’s AI use experience in terms of use frequency, self-confidence, and AI helpfulness perception (Table
2). The survey revealed that the majority of the participants (N=22) claimed to frequently (often or always) use
AI tools. Additionally, the majority of the participants (N=22) claimed that they are slightly confident or not
confident with their ability if AI is not allowed. Morover, almost every participant (N=26) perceived AI is helpful
in IT coursework.

Table 1. Student academic performance perception

Table 3. Student AI banning perception

The survey results provide insights into participants' perceptions on various aspects related to the use of AI in
IT education (Table 3). Firstly, it was found that the majority of respondents agreed that AI-powered chatbots
have a positive impact on academic performance in IT courses (M(SD)=3.27(0.78), closer to Agree). Similarly,
respondents expressed agreement regarding the positive influence of AI-powered tools on confidence in
programming abilities (M(SD)=3.03(0.93), closer to Agree). However, opinions were more varied when
considering the potential effects of banning AI. Participants showed a slight leaning towards agreement on the
potential increase in workload for IT students if AI were to be banned (M(SD)=2.97(0.89), closer to Agree). The
perception of an increase in stress due to the ban also garnered moderate agreement (M(SD)=3.10(0.8), closer
to Agree). Regarding the potential benefits of banning AI, respondents tended to agree that it could lead to
improvements in critical thinking and problem-solving skills (M(SD)=2.87(0.9), closer to Agree). However, there
was some variability in opinions, suggesting a range of perspectives on this matter. Finally, participants, on
average, slightly agreed that the banning of AI could result in a decrease in efficiency and speed of certain
tasks in IT courses (M(SD)=2.83(0.79), closer to Agree).

conclusion
The survey results provide valuable insights into the research problem, addressing the potential benefits and drawbacks of banning
Ai use for IT students (RQ1), how confident students are in their abilities and how helpful they perceive AI to be (RQ2), and the
impact of banning AI use on students' perception of their academic performance (RQ3).
Note. There are 4 levels of agreement for this table; strongly disagree (1),
Regarding RQ1, the study shows that participants generally agree that AI-powered chatbots have a positive impact on academic disagree (2), agree (3), and strongly agree (4).
achievement and increase confidence in programming capabilities. This suggests that it can have been wrong in banning AI, as it can
hinder the perceived improvement in IT students’ academic achievement and confidence. However, there is some difference in
perception of the potential increase in workload and stress due to the AI ban. This suggests that banning AI can have both positive
and negative effects on IT students’ performance and stress levels.

Regarding RQ2, the survey results indicate that students have mixed levels of confidence in their abilities and perceive AI as helpful
References
in their IT coursework. While the majority of participants expressed slightly confident or not confident views if AI is not allowed, they Gulwani, S. (2022). AI-assisted programming: Applications, user experiences, and neuro-symbolic techniques (keynote). In Proceedings of the 30th ACM Joint European Software
also acknowledged the helpfulness of AI tools in their studies. This suggests that AI plays a significant role in supporting students' Engineering Conference and Symposium on the Foundations of Software Engineering. [Link]
Kaluarachchi, T., & Wickramasinghe, M. (2023). A systematic literature review on automatic website generation. Journal of Computer Languages, 75, 101202.
learning and bolstering their confidence in the IT field. [Link]
Khare, K., Stewart, B. G., & Khare, A. (2018). Artificial intelligence and the student experience: An institutional perspective. IAFOR Journal of Education, 6(3), 63–
[Link]://[Link]/10.22492/ije.6.3.04
Regarding RQ3, the results of the study indicate that the ban on AI use in IT classrooms may affect students’ perceptions of
Konecki, M., Kadoić, N., & Piltaver, R. (2015). Intelligent assistant for helping students to learn programming. International Convention on Information and Communication Technology,
academic achievement. Respondents agreed that AI-powered chatbots make a positive contribution to improving learning Electronics and Microelectronics (MIPRO). [Link]
outcomes in IT courses. Therefore, if AI is banned, students’ perceptions of academic performance are likely to be negatively Li, L., Subbareddy, R., & Raghavendra, C. G. (2022). AI intelligence chatbot to improve students learning in the higher education platform. Journal of Interconnection Networks,
22(Supp02). [Link]
impacted. However, there are various perspectives on the impact of banning AI use on critical thinking and problem-solving skills.
Muller, S., Babes-Vroman, M., Emenike, M. E., & Nguyen, T. D. (2020). Exploring novice programmers’ homework practices: Initial observations of information seeking behaviors. SIGCSE
This suggests that the impact on these skills may vary among students if AI is banned. '20: Proceedings of the 51st ACM Technical Symposium on Computer Science Education [Link]
Narasimhan, A. (2021, August 23). CGEMs: A metric model for automatic code generation using GPT-3. [Link]
Pillai, A., Shinohara, K., & Tigwell, G. W. (2022). Website builders still contribute to inaccessible web design. ASSETS '22: Proceedings of the 24th International ACM SIGACCESS
In conclusion, the results highlight the importance of AI in enhancing academic achievement, confidence, and problem-solving skills Conference on Computers and Accessibility. [Link]
among IT students. It suggests that a ban on AI could potentially hinder students' learning experience and negatively impact their Price, T. M., Liu, Z., Cateté, V., & Barnes, T. (2017). Factors influencing students’ help-seeking behavior while programming with human and computer tutors. ICER '17: Proceedings of the
2017 ACM Conference on International Computing Education. Research[Link]
perception of their academic performance. Considering the multifaceted nature of the research questions, it is clear that a balanced
Qayyum, A. (2018). Student help-seeking attitudes and behaviors in a digital era. International Journal of Educational Technology in Higher Education, 15(1). [Link]
approach is necessary when considering the use of AI in IT education. While the benefits of AI in improving academic outcomes and 018-0100-7
confidence are evident, the potential drawbacks such as increased workload and stress also need to be considered. Future Sagar, M., Gupta, A., & Kaushal, R. (2016). Performance prediction and behavioral analysis of student programming ability. International Conference on Advances in Computing,
Communications and Informatics (ICACCI). [Link]
research should take into account these findings to strike a balance between leveraging the benefits of AI and addressing its
Schlusche, C., Schnaubert, L., & Bodemer, D. (2022). Understanding students’ academic help-seeking on digital devices – a qualitative analysis. Research and Practice in Technology
potential challenges, ultimately optimizing the educational experience for IT students. Enhanced Learning, 18, 017. [Link]