Machine Learning for Student Performance Prediction
Machine Learning for Student Performance Prediction
ISSN 1660-6795
[Link]
Many students at higher education institutions have to work hard to finish different courses since
there is no specialized help supplied to students who require particular attention in the registered
courses. Students' grades may be predicted using machine learning approaches in a wide range of
courses. Such strategies would encourage students to improve their performance based on expected
grades, and teachers would be able to identify those students who would benefit from extra guidance
in the classroom. Students' conduct reveals the link between students and a Web-based education
system's Learning Management System. In this research, we propose a novel model for students to
forecast their performance based on machine learning approaches that include new characteristics
known as student behavior and a sequential feature selection used to identify important features.
The suggested output model is being evaluated using ANN, SVM, KNN, Logistic regression, and
a Decision Tree. We also used Bagging, Boosting, and Random Forest as collaborative approaches
to improve classification accuracy. The results show that the student's activities and academic
progress are linked. Using collaborative ways to improve academic performance in the classifier
resulted in a 96% accuracy rate. As a result, the ML techniques are found to be better than the other
techniques used in predicting the students’ performance in the classroom learning.
1. Introduction
Students' retention in prestigious colleges and universities is a major problem because of their
status as sites of higher learning [1]. According to [2], a majority of students drop out of college
within their first year owing to a lack of assistance in undergraduate courses [3]. First year of
college is referred to be a "make or break year" because of this. In the absence of any
assistance, a student may get demotivated and decide to drop out of the course. To help
students stay on track in colleges and universities, there is a pressing need to find a suitable
solution. As a result of early grade prediction, the University's degree courses will be
monitored more closely and the students' learning process will be improved based on their
expected grades.
This is because computers and portable gadgets have evolved in the last three decades,
allowing access to a wide range of electronically available materials. Educators may now
connect with students in new ways because to these innovations [2]. Modern schooling has
becoming increasingly technologically oriented [3]. Human-machine collaboration is
necessary to develop a learning environment that is both successful and efficient for the human
learner [4] because of the technology's involvement with humans. Higher education
institutions have begun incorporating blended learning approaches into their standard teaching
methods due to the relevance of both instructor-generated and learner-generated material [2].
There is also a rise in the popularity of massive open online courses (MOOCs). As mentioned
in [5,] researchers are now able to tackle issues that were previously inaccessible. There has
been a startling development of Artificial Intelligence (AI) in the educational system recently.
One of the ten most common reasons for using new technology in education is to assist
students accomplish their educational goals. The importance of AI-enabled educational
technology has grown to draw attention to the improvement of educational quality and the
enhancement of traditional teaching and learning methods [6]. Research shows that artificial
intelligence (AI) technology is allowing computers to execute a variety of activities by
emulating human problem-solving skills [7]. AI technology has also been identified as one of
the most significant uses in the field of special educational needs (SEN) [8].
Organization for Economic Co-operation and Development (OECD) members have a dropout
rate of around 45% [2]. In response to this issue, higher education institutions are establishing
and implementing interventions. A student's first year of college is the best time to implement
these methods, according to researchers and practitioners. Because of this, a lot of attention
has been paid to figuring out which students are most likely to drop their classes as early as
possible [2,3]. Machine Learning has recently been used to enhance corporate decision-
making in predictive analysis. The importance of Machine Learning research in a variety of
business tasks may be seen in the fields of finance, operations, and risk management, to name
a few. It has been shown that machine learning may help forecast business process
performance [4] and identify credit card fraud [5]. The application of Machine Learning in
higher education management is increasing. Educators are increasingly interested in applying
Machine Learning to forecast student performance and identify students at risk based on their
initial data acquired during their years of study [6]. Fewer studies have been done on how to
predict a student's academic achievement based on their past academic experience [2,3]. As
seen in Figure 1, some of the most important statistics from the literature study are shown
graphically.
Nanotechnology Perceptions Vol. 20 No. S8 (2024)
1381 A. S. Gousia Banu et al. An Enhanced Machine Learning-Based...
Figure 1. Basic data about the approaches, goals, and methods used.
In order to better forecast student exam success, this research examined and compared the
latest state-of-the-art machine learning algorithms. For this type of prediction, supervised
machine learning approaches are preferred over unsupervised ones since they don't need the
use of an unlabeled dataset. Similarity-based, model-based, and probabilistic machine learning
techniques were all considered in this study. The technique based on similarity predicts exam
achievement by finding students with comparable historical results. The second technique of
this strategy is driven by an estimate of the implicit correlation between the input learning data
and the underlying model. Using the input datasets' distributional features, the third approach
is probabilistic. Student "high risk" was detected utilising all three state-of-the-art techniques
and representative processes, as well as for executing test performance classifications and
regressions. It was believed that a student's performance on the categorization assignment
would determine whether or not they passed the exam. The estimated regression coefficient
was generated using the final test results of the students. The accuracy of student test
performance predictions was also examined in relation to the different forms of data provided
for optimisation of a specific machine learning technique. The ideal combination of input data
types was determined by taking into consideration a variety of student attributes. Data input
includes demographics, prior academic performance, and student participation. Different
viewpoints on student test performance prediction were explored in the analysis to ensure
complete examination and comparison of underpinning approaches. When it came to
predicting final test scores, for example, students were asked to supply a variety of input data
at various points in time leading up to the final exam. In this manner, each technique's
performance patterns were analysed. Each approach was also evaluated in terms of its
computing needs in terms of the amount of time it takes to forecast. Data from the Open
University Learning Analytics Dataset, one of the largest and most diverse collections of
learning analytics and student-related data, was mined for use in testing and evaluation
(OULAD).
1.1. Motivation
Recently, predictive analysis has relied on Machine Learning to support business decision-
making. Applications in finance, operations and risk management are good attestations of the
relevance of Machine Learning research in various business functions. Hassan Zeineddine et
al., for example, used machine learning to predict business process performance [4], and
McLean et al. to spot credit-card fraud [5].
More and more, Machine Learning is used in the field of higher education management.
Specifically, there has been an increased interest in adopting Machine Learning to predict
student performance and identify students at risk based on initial data gathered during their
years of study, as surveyed in the work of Ding L et al. [6]. Fewer work addressed the
prediction of student performance using data prior to starting their academic journey [2,3].
Given the complexity of choosing an optimal prediction model for a given dataset from a wide
pool of predictive methods and different hyper-parameter values per model, the automation of
this process can help increase the prediction accuracy [7–9]. In relation to that, Automated
Machine Learning (ML) is a technique meant to derive the best classification model and
corresponding hyper-parameters for a given decision-making problem. This technique can add
value if used in predicting student performance. Yet, the review of the literature in this area
shows a lack of empirical work using ML. Our research work relies on ML to help increase
the accuracy of predicting student performance using data available upon entering an academic
program.
The rest of the paper is organized as follows. Section 2 will discuss the literature survey.
Section 3 describes the dataset. Section 4 presents the Methodology and Section 5 highlights
the results and discussion. Section 6 concludes the proposed work.
2. Literature Survey
There have been a number of relevant research published in the field of student test
performance analytics. Exam performance analysis at educational institutions was the primary
focus of these investigations, with the classification challenge of classifying individuals as
either "passing" or "failing." The purpose of these research was to identify students who were
"at-risk" of failing a particular class. According to a study published in 2003 [9], k-Nearest
Neighbor (k-NN) and Nave Bayes were used to classify students into dropout and non-dropout
categories. Dropout risk can be identified by a student's demographic data or a brief description
of where they were born. In a web-based course, [10] examined many learning algorithms for
predicting final test marks for students enrolled. Students' grades were categorised into two
classes (pass and fail), three (high, middle, and poor), and nine (out of a total of 10) for this
study's research purposes (according to the achieved grade). It was shown that a k-NN
algorithm could be used to classify students attending an online touch-typing course into two
groups: those who would "drop out" and those who would finish the final test. Using the
regression exercise, they were able to make predictions about how students will do on the final
exam. A technique known as the k-NN algorithm was used by [11] to determine the
Nanotechnology Perceptions Vol. 20 No. S8 (2024)
1383 A. S. Gousia Banu et al. An Enhanced Machine Learning-Based...
appropriate level of study material for each learner. Despite the fact that their goals are distinct,
the results of performance analysis have a considerable impact on prediction studies as well.
In some cases, statistical approaches may not be adequate to establish the relationship between
numerous factors and performance [12]. Educators and students alike may benefit from the
deployment of advanced algorithms [13]. Many scholars have been encouraged by the
progress of data mining tools to examine deeper insights into the information diffusion
process. In the past, some of them have used data mining techniques to accomplish this goal
[14]. There were, however, relatively few of these investigations at the time. A rise in
educational quality has been spurred on by the use of digital data management systems and
instructional software [15], all educational research will be based on data mining and analytics.
Multiple surveys on EDM studies have been undertaken in the past by the researchers. Some
of them use a broader definition of educational processes to include a wide range of topics. In
addition, a small number of these programmes concentrate on analysing and forecasting
student performance. Let's take a quick look at what these polls are all about. More than a
decade ago, [16] released a paper in which they synthesised previous studies published in the
period 1995-2005 and outlined the most likely goals of EDM. Another comprehensive
assessment [17] of over 300 studies from 1993 to 2009 was published in 2010. Student
performance prediction is one of eleven main sub-areas it splits the EDM research area into.
It was also at that time that the field of learning analytics had begun to take off. It's important
to point out that the initial impetus for educational analytics research came from two research
organisations: the International Educational Data Mining Society and the Society for Learning
Analytics Research. According to [18, these two research communities have a lot in common
when it comes to their areas of interest. A study by [19] in the same year describes the
motivations for and difficulties associated with studying learning analytics. They conducted
an in-depth investigation on student modelling methodologies in 2013. Possibly, this is the
first of its kind to focus on a specific facet of electronic dance music. In [20] analysed 240
EDM trials between 2010 and the first quarter of 2013 and issued yet another thorough analysis
in 2014, which was published in 2014. Student performance modelling has seen a steady
increase in papers published since 2010, according to this research. Additionally, researchers
performed an EDM survey on the psychology of learning and the usage of clustering methods
[21]. Educators and administrators may profit greatly from the prediction of students'
performance, according to [22], the most recent appraisal of EDM research.
2.1. Techniques
Students' behaviour may be predicted using several sorts of data, such as demographics and
grades from previous assignments, using methods like machine learning (ML) and supervised
learning (SL). The Hellenic Open University's work on machine-supervised learning methods
applied to a dataset provided a solid beginning point for this investigation. Findings from this
research show a correlation between the Nave Bayes (NB) algorithm and the likelihood of
student dropout [23]. As a result, several strategies may be used to anticipate students'
behaviour depending on the nature and characteristics of each case study. As a result of this
effort, the various approaches have been organised into four broad categories: ML with
supervision, unsupervised ML, CF, and ANN. In order to incorporate those works with
comparable goals but utilised alternative DM approaches, a new category has been formed.
Figure 1 indicates how many problems and instances each of these categories of approaches
is better suited to, as shown by the amount of weight they each have in the literature. An
overwhelming majority of instances are handled by supervised ML, followed by CF, which
accounts for less than a quarter. However, unsupervised ML has been used in a very small
number of situations.
2.1.1. Machine Learning
Machine Learning is a set of techniques that enables computers to learn without the need for
human intervention [24]. Predictive analysis, medical diagnostics, stock market research and
DNA sequencing are just some of the many uses for ML. In particular, we are interested in
predictive analysis, where ML may be used to create sophisticated models that can be used to
forecast future behaviour. By supplying consumers with appropriate data, these models may
be of considerable benefit. ML algorithms may be divided into two primary categories:
supervised and unsupervised.
2.1.2. Logistic regression
Students' performance may be predicted using regression techniques, which make use of a
finite set of relationships between the dependent and independent variables to generate an
inferential function. There are a variety of different ways to characterise the relationships
between independent variables in a logistic regression model, and they include binary,
categorical, and continuous [25, 26]. According to the logistic regression, the degree of
prediction accuracy is about 70% when utilising factors such as professional goals, CGPA,
psychological scores, and personal interests as predictors of future success.
2.1.3. Decision tree
There have been several studies that have made use of the Decision Tree prediction approach,
due to its simplicity and convenience of use in analysing both small and big data sets and
anticipating their worth. If-then statements can be used to simplify the logic of decision tree
approaches, making this method easier to grasp. Students' grades in specific courses and their
current CGPA have been utilised in various articles to forecast their performance using this
strategy [27]. It is possible to predict a student's success in an academic programme using this
technique with a level of accuracy of roughly 70 % when utilising data collected prior to the
student's enrollment in the programme [28].
2.1.4. Artificial neural network
An Artificial Neural Network (ANN) is capable of detecting all of the relationships between
the variables that it is trained on. In educational data mining, it is a commonly utilised
approach. The ANN is an effective tool for forecasting student performance because of its
capacity to discover complicated relationships between independent and dependent factors
with high confidence [29]. Many factors go into predicting student achievement with a neural
network, including the student's attitude toward learning, the applicant's CGPA, and their
grades in individual courses. To forecast student performance using data collected after
students began their academic journey, this method had an accuracy of up to 98%, and had an
accuracy of about 70% [30].
2.1.5. Naive Bayes
Another approach used to forecast student achievement called naive Bayes. In order to
Nanotechnology Perceptions Vol. 20 No. S8 (2024)
1385 A. S. Gousia Banu et al. An Enhanced Machine Learning-Based...
demonstrate the relevance and influence of each of these predictors, it does comparisons
among independent variables using all of the qualities included in the data. For the most part,
studies that employed this strategy relied on data from sources other than test scores and
standardised test scores. According to research conducted with the help of Nave Bayes, which
relied mostly on data acquired after students had begun their academic journeys, the model
had a maximum accuracy of 76% [16].
2.1.6. K-nearest neighbors
The K-Nearest Neighbors methodology is a simple way to classify data points based on the
dominating class of their K-Nearest Neighbors. The K-Nearest Neighbors methodology
[13,21,23] is a fast method for predicting student success based on a variety of multivariate
criteria (slow, medium, good, and outstanding). If data from internal evaluations, CGPA, and
extracurricular activities were used, the accuracy rate increased to 83% [16].
2.1.7. Support vector machine
An SVM is a supervised learning approach that uses an N-dimensional hyperplane to classify
data points. N is the number of characteristics that characterize a data point. When working
with small samples, this strategy has been useful for researchers [6,12,13,21,23,25] in
forecasting student performance. Overlapped data can also be effectively handled by the SVM.
Student performance was predicted with an accuracy of roughly 80 percent using CGPA,
extracurricular activities, psychomotor tests, and internal assessments in previous study [19].
3. Dataset Description
This data is based on the performance of students in secondary education at two schools in
Portugal. It was gathered through school reports and surveys that included student grades,
demographics, social and school-related aspects. Mathematics (mat) and Portuguese language
performance are examined in two separate datasets. A binary/five-level classification and
regression task were used to model the two datasets in [31]. G3's target property G2 and G1
have a high association with each other. Final-year grades G3 (given at 3rd period) are
different from grades G1 and G2, which are awarded at the beginning and end of the semester,
respectively. Predicting G3 without G2 and G1 is more difficult, but the benefits of such a
forecast are far greater.
4. Methodology
As distinguished institutions of higher learning, universities place a great value on student
retention. Many students drop out of college during their first year because they lack enough
help in their undergraduate courses. Consequently, many students refer to their freshman year
as their "make or break" year. In the absence of any assistance, a student may get demotivated
and decide to drop out of the course. There is a tremendous need to find a way to keep college
students in their programmed longer. As a result of early grade prediction, the University's
degree courses will be monitored more closely and the students' learning process will be
improved based on their expected grades. Students' learning outcomes may be enhanced by
the application of machine learning and Educational Data Mining. Students' grades in the
courses they've enrolled in may be predicted using a variety of models, which can aid in the
retention of those students. Using this data, a system can propose that teachers give extra
attention to kids who are at danger of falling behind in class. Students' grades in various
courses may be predicted with the use of this data, and the universities' retention rates can be
improved as a result. Automated machine learning used a set of prediction approaches and
their associated hyper-parameter values to find the most accurate model, as shown in Figure
5.
factor will be used as a weight. It's possible to standardise the output by using a transfer
function called S. The sigmoid function, which modulates values between 0 and 1, was chosen
as the study's transfer function:
1
S(x) = 1+e−x (2)
The ANN is a hierarchical model with several layers. Nodes (neurons) in each layer are linked
in a single way to all nodes in the layer below them through unidirectional linkages. There is
no way to communicate with other nodes in the same layer or upstream. Fig. 6 depicts a typical
data structure with three levels: an input layer, a collection of intermediate layers, and an
output layer. The ANN design used in this work consists of an input layer that represents the
various category values of the adopted data characteristics, two intermediate layers with 12
and 7 neurons each, and an output layer with one neuron representing the binary result.
according to Cover's Theorem. The bordering data points, referred known as support vectors,
will be used to reference the boundary hyperplane. A certain distance should separate the
support vectors from the border. As a result of this kernel function, distances may be computed
more quickly by taking the dot product of two points (x and y). Polynomial kernel F of degree
d [30] was employed in this study:
F(x, y) = (∑ni=1 xi . yi + c)d (3)
tree-like topology, making it easier to answer queries by traversing the tree from root to branch.
Two or more downstream nodes, including the root node, are connected to each decision node
(all representing answers to decision questions). As the last response to a sequence of questions
collected by the preceding nodes, a leaf node serves as the final node.
Accuracy may be measured by calculating the probability of making the correct forecast (Po).
A classifier pair's chance of accidentally agreeing on a classification is called the probability
of agreement (Pe). The chance that classifier I would correctly forecast class n in a binary
system with two predictors is given by the formula Pe = P1(a) • P2(a) + P1(b) • P2(b).
According to Fleiss' Scale, a kappa value of between 0.4 and 0.75 is excellent. A kappa of less
than 0.4 is considered bad, while a kappa of more than 0.75 is considered good. Kapa of 0.5
for our Machine learning Model is roughly two-thirds greater than the Kapa for any individual
prediction model (using the same data). As a consequence, our machine learning Model
derived from the automated search has a lower possibility of exhibiting the observed process
and random guessing errors.
academic Programme can be used to increase the accuracy of student performance projections
using pre-start data. Predicting student achievement using pre-start data has never had an
accuracy higher than 83%, according to existing research. With the use of machine learning,
we were able to achieve an overall accuracy of 96%, with a Kapa of 0.3. If you're looking for
an optimal student performance prediction model, you're advised to employ machine learning
(ML). Students who are at risk of failing and need early help from academic institutions'
specialized departments must be identified more accurately. There is a maximum accuracy of
80% in the literature on new-start student failure predictions. An accuracy of 96% was
achieved after employing the machine learning model.
Students at risk may now be identified with greater precision, which saves time and money for
educational institutions. In future, descriptive statistics may be utilized to analyses the role and
effect of various psychographic elements on the prediction model. In the future, it is possible
that autogenerated ensemble models will be examined to determine if academic and
psychographic data can predict student performance in the workplace.
References
1. Sannyuya Liu, Shiqi Liu, Zhi Liu, Xian Peng, Zongkai Yang, Automated detection of emotional
and cognitive engagement in MOOC discussions to predict learning achievement, Computers &
Education, Volume 181, 2022, 104461.
2. Puranam Revanth Kumar, Rajesh Kumar Jha, P Akhendra Kumar, and B Deevena Raju
“Improved neurological diagnoses and treatment strategies via automated human brain tissue
segmentation from clinical magnetic resonance imaging”, Intelligent Medicine, vol. 4, Issue 4,
pp. 1-21, 2024.
3. Bahareh Javadizadeh, Matthew Aplin-Houtz, Maureen Casile, Using SCARF as a motivational
tool to enhance students′ class performance, The International Journal of Management
Education, Volume 20, Issue 1, 2022, 100594.
4. Hassan Zeineddine, Udo Braendle, Assaad Farah, Enhancing prediction of student success:
Automated machine learning approach, Computers & Electrical Engineering, Volume 89, 2021,
106903.
5. Puranam Revanth Kumar, Rajesh Kumar Jha, and P Akhendra Kumar “BrainHyperintensities:
Automatic Segmentation of White Matter Hyperintensities in Clinical Brain MRI Images using
Improved Deep Neural Network”, The Journal of Supercomputing, vol. 80, pp. 15545–15581,
2024.
6. Ding, L., Er, E., & Orey, M. (2018). An exploratory study of student engagement in gamified
online discussions. Computers & education, Vol. 120, Elsevier213–226.
7. Dobson, A. J., & Barnett, A. G. (2008). An introduction to generalized linear models. Chapman
and Hall/CRC. Taylor & Francis Group.
8. Dringus, L. P. (2012). Learning analytics considered harmful. Journal of Asynchronous Learning
Networks, 16(3), 87–100.
9. Puranam Revanth Kumar, B Shilpa, Rajesh Kumar Jha, B Deevena Raju, and Thayyaba Khatoon
Mohammed, “Inpainting Non-Anatomical Objects in Brain Imaging using Enhanced Deep
Convolutional Autoencoder Network”, Sadhana, vol. 49, article number 181, pp. 1-13, 2024.
10. Advances in neural information processing systems (pp. 155–161). Cambridge, MA: MIT Press.
11. Dudani, S. A. (1976). The distance-weighted k-nearest neighbour rule. IEEE Transactions on
Systems, Man and Cybernetics, 6(4), 325–327.
12. van Gerven, M. A. J., & Bohte, S. M. (2017). Editorial: Artificial neural networks as models of
Nanotechnology Perceptions Vol. 20 No. S8 (2024)
An Enhanced Machine Learning-Based… A. S. Gousia Banu et al. 1394
outcome in educational sector by using decision tree. Int. J. Technol. Res. Eng. 2017, 4, 2347–
4718.
29. Gouse Baig Mohammad, Shitharth, and Puranam Revanth Kumar, “Integrated Machine
Learning Model for an URL Phishing Detection”, International Journal of Grid and Distributed
Computing, vol. 14, Issue 1, pp: 513-529, 2021.
30. Gray, C.C.; Perkins, D. Utilizing early engagement and machine learning to predict student
outcomes. Comput. Educ. 2019, 131, 22–32.
31. Iatrellis, O.; Savvas, I.K.; Fitsilis, P.; Gerogiannis, V.C. A two-phase machine learning approach
for predicting student outcomes. Educ. Inf. Technol. 2020, 1–20.
32. Kuzilek, J.; Vaclavek, J.; Zdrahal, Z.; Fuglik, V. Analysing Student VLE Behaviour Intensity
and Performance. In Proceedings of the European Conference on Technology Enhanced
Learning, Delft, The Netherlands, 16–19 September 2019; Springer: Cham, Switzerland, 2019;
pp. 587–590.
33. Puranam Revanth Kumar, and T Ananthan “Machine Vision using LabVIEW for Label
Inspection”, Journal of Innovation in Computer Science and Engineering (JICSE), Vol.9, Issue
1, pp: 58 - 62, 2019.