FUNDAMENTALS OF LEARNING ANALYTICS AND EDUCATIONAL DATA MINING
1. Data Mining
Uncovering hidden patterns in data is known as Knowledge Discovery in Databases (KDD).
This concept encompasses data mining and is widely used in enterprises to facilitate
accurate and expedient decision-making. Knowledge Discovery in Databases (KDD) and
data mining are often used interchangeably, with data mining referring to extracting
previously unknown valuable information, trends, and patterns from large databases. Data
mining encompasses knowledge extraction, information discovery, exploratory data
analysis, and unsupervised pattern recognition. The process comprises the following steps:
a) defining problems or formulating hypotheses, b) targeting the database data, c) pre-
processing the data (e.g. de-noising, conversion, scaling, dimension reduction, feature
extraction), d) utilising data mining algorithms, and e) recognising patterns/correlations.
This process is illustrated in Figure 1.
Figure 1: Schematic process of data mining
2. Educational Data Mining (EDM)
Educational data mining focuses on analysing data from educational settings to understand
students and their learning environments better. It helps improve educational practices
and learning materials through data-driven decision-making. This field also plays a vital role
in designing better learning technology by uncovering how people learn and predicting
learning patterns. Despite posing a challenge for educational institutions, the growing
availability of educational data presents an opportunity to provide valuable insights for
students, teachers, and administrators. The key application areas of educational data
Dr. Saswati Mukherjee, School of Education Technology, JU 1
mining are categorised into four domains:
• Student Model: Modeling students based on personal characteristics, including prior
knowledge, attitude, motivation, and meta-cognition strategies.
• Domain Model: Discovering and improving models of the knowledge structure within a
specific domain.
• Pedagogical Support: Improving and testing applications that provide pedagogical
support, such as determining which pedagogical support is most effective for different
students.
• Investigating key factors that deeply impact learning to design better learning
environments and provide empirical evidence.
3. Learning Analytics (LA)
E-learning in higher education has led to using LMS platforms like Moodle, Blackboard, and
Canvas to support online learning. These platforms generate data on student interactions,
which can be analysed to identify learning behaviour patterns, a process known as Learning
Analytics.
Learning Analytics involves measuring, collecting, analysing, and reporting data about
learners and their environments to understand better and improve learning. It also involves
assessing the behaviour of educational communities. A major focus of learning analytics is
gathering and analysing data and implementing interventions to enhance the learning
experience. Online learners leave behind data trails, which can be collected from various
sources and analysed to provide valuable insights and visualisations for institutions,
teachers, and learners. The potential benefit of using learning analytics is that it provides a
more detailed view of how students engage with learning content, their learning
approaches, and how they set learning goals. Figure 2 illustrates student, teacher,
administrator, and government learning analytics.
Benefits of LA
• Evidence - Proof of practice (success or failure), justification, removal of doubt or
assumptions
• Nimble - Quickly adjust practice, immediate feedback
• Grounded - Make changes based on evidence rather than assumption or intuition
• Revealing - Provide information in areas we did not know or were not aware of / or potential
• Student-Centered - Inform learner experience, help guide the learner
• Predictive education - Predictions based on preference, performance, and ability
• Change practice - With evidence, change what does not work, informed decisions
Dr. Saswati Mukherjee, School of Education Technology, JU 2
4. Inhibitors to Learning Analytic Use
Approaching students’ Learning Management System (LMS) interaction involves
understanding why Learning Analytics (LA) is not used as a potential method in LMS. In
many e-learning and online student courses, teachers, managers, and administrators are
not focusing on how students engage with the curriculum. Understanding the internal and
external factors influencing students' choices provides valuable insights into the learning
process and helps connect it to pedagogical design. The use of learning analytics by teachers
is hindered by the following barriers:
• Lack of Training: Uncertainty about what data to collect, how to process, and how to use
it
• Fear of Exposure: Concerns about judgement from peers and fear of revealing weaknesses
• Too Much Data: Feeling overwhelmed by the large volume of data and struggling to
make sense of it
• Too Little Data: Not having enough data in a specific area that is meaningful
• Lack of Ability: Inability to make changes based on the data and knowledge gathered
• Cultural vs Procedural: Handling data related to cultural norms, teachers or behaviours
• Intentionality: Ensuring that empirical practices and data are connected to the research
question
• Lack of Resources: Limited guidance in literature, lack of examples and resources, and
constrained time."
Figure 2: Snapshot of LA possibilities that correspond to the education system
Dr. Saswati Mukherjee, School of Education Technology, JU 3
Educational data mining discovers hidden patterns in educational data, while learning
analytics utilises these patterns to improve learning environments. Learning analytics
integrates technical and social/pedagogical aspects of learning using predictive models,
while educational data mining looks for new patterns in data and develops algorithms and
models. Both are interdisciplinary, combining computer science, education, and statistics.
The intersection of these three areas also forms other subareas, such as computer-based
education (CBE), data mining, machine learning, and educational statistics (Figure 3).
Figure 3: Areas of EDM/Learning Analytics
Impact on Learning Science
Educational data mining and learning analytics have had several impacts on the learning
sciences.
Engagement and Disengagement
Before the development of analytics, measuring disengagement was challenging. However,
educational data mining (EDM) and learning analytics (LA) have created models that
accurately identify various forms of disengagement, including behaviours like gaming in the
system. Interestingly, some actions that may seem disengaged, such as clicking hints,
finding an answer, and then explaining it, can indicate engagement. These detectors are
integrated into intelligent tutoring systems, allowing automatic detection and response to
frustration to improve learning outcomes.
Collaboration and Learning
EDM and LA methods help understand student learning in collaborative and group settings.
Analysing collaborative learning behaviours in various contexts can determine successful
groups and learners in computer-mediated learning activities and nursing education. Off-
topic discussions are more harmful when learning basic facts than discussing problem-
solving alternatives.
Dr. Saswati Mukherjee, School of Education Technology, JU 4
5. Educational Environments
Various educational environments exist, including traditional education, computer-based
educational systems, and blended learning, each providing different data sources (Figure
4).
Traditional Face-to-Face Education
The traditional education system is widely used and involves direct interaction between
educators and students, including lectures, class discussions, small group activities, and
individual assignments. These systems collect information about student attendance,
grades, curriculum objectives, and class schedules and can incorporate computer-based
educational tools.
Computer-Based Educational Systems (CBE)
CBE uses computers in education to provide guidance, instruction, and management of
educational content for students. These systems range from simple standalone applications
to advanced web-based intelligent educational systems. Table 1 lists and describes
examples of computer-based educational systems.
Blended Learning Systems
Blended learning environments combine face-to-face instruction with computer-mediated
instruction. Blended courses increase access and convenience, transforming many online
face-to-face sessions and providing more flexibility and freedom than face-to-face courses.
These systems gather information from both face-to-face and computer-based systems.
Figure 4: Types of educational environments
Dr. Saswati Mukherjee, School of Education Technology, JU 5
Table 1: Computer-based educational systems
System Description
Intelligent Tutoring Customized instruction or feedback is provided directly to
System (ITS) students by modelling their behaviour and adjusting the
interaction mode based on individual needs. This involves
domain, student, and pedagogical models. Intelligent tutoring
systems (ITSs) record all student-tutor interactions.
Adaptive and Intelligent Adaptive learning builds a model of each student's goals,
Hypermedia System preferences, and knowledge to personalise interactions. The
(AIHS)
data from these systems is more detailed, allowing for a more in-
depth analysis than traditional web-based education systems.
The data collected by AIHs is similar to that of Intelligent
Tutoring Systems (ITS).
Learning LMSs (Learning Management Systems) handle course-delivery
Management functions like administration, documentation, tracking, and
System (LMS) reporting of training programs for classroom, online, and e-
learning settings. They also store student information in a
database and support activities like reading, writing, testing, and
peer interaction. Examples of LMSs include Blackboard, Virtual-
U, Moodle, Ilias, Sakai, and Claroline.
Massive Open Online Web-based classes, such as MOOCs (Massive Open Online
Course (MOOC) Courses) like Udacity, MITx, EdX, Coursera, and Udemy, can
accommodate a large number of participants and deliver course
content online without attendance limits. MOOCs collect usage
data from many students, generating large amounts of data for
analysis using data mining techniques.
Test and quiz system Measure the students' level of knowledge with respect to one or
more concepts or subjects by using a series of questions/items
and other prompts to gather information from respondents.
They store great information about students' answers,
calculated scores, and statistics.
Other types Wearable learning systems, learning object repositories, concept
maps, social networks, wikis, forums, virtual and augmented
reality systems.
Dr. Saswati Mukherjee, School of Education Technology, JU 6
6. Educational Data
Relational data stores data in tables with unique names. Each table contains columns that store
records. These records represent objects identified by unique keys and described by attribute
values. Relational data can be accessed using queries like SQL. Moodle uses relational databases
with table names starting with mdl_ followed by descriptive words. Relational data mining finds
patterns across multiple tables in the database. Educational data mining (EDM) is used to
discover association rules in student behaviour in e-learning. Typically, relational data is
transformed into transaction data before data mining.
Transactional data comprises records of individual transactions, typically identified by a unique
transaction number and a list of items involved. Although Moodle doesn't have a built-in
transactional database, transactional data can be derived within it and used for data mining
methods such as classification, clustering, and association rule mining. These techniques have
been applied to Moodle student usage data in educational data mining (EDM) to provide
feedback for course improvement and student learning.
Temporal, Sequence and Time Series Data consist of sequences of values or events changing
over time. A temporal database typically stores relational data that include time-related
attributes. These attributes may involve several time- -stamps, each involving different
semantics. A sequence database stores sequences of ordered events, with or without a concrete
notion of time. A time-series database stores sequences of values or events obtained over
repeated time measurements (e.g., hourly, daily, weekly). An example of a sequential database
Moodle uses is a student’s log. A log is a list of a student’s events, in which each line or record
contains a time stamp plus one or more fields that hold information about an activity at that
instant. A Moodle log (Figure 5) consists of the time and date it was accessed, the Internet
Protocol (IP) address accessed from, the name of the student, each action (view, add, update
and delete) performed in the different modules (forum, resource, assignment, etc.) and the
action.
Figure 5: Moodle Log File
Dr. Saswati Mukherjee, School of Education Technology, JU 7
Text data comprises large collections of documents from various sources, such as news articles,
research papers, books, digital libraries, email messages, chat and forum messages, and web
pages. Text databases may be highly unstructured, such as some Hypertext Markup Language
(HTML) web pages, or somewhat structured/semi-structured, such as email messages and
eXtensible Markup Language (XML) web pages. Moodle provides a large amount of text
information, such as students' messages in forums, chats, emails, and anything students can
read or write in the system. Text mining, or text data mining, is roughly equivalent to text
analytics and can be defined as applying data mining techniques to unstructured textual data.
Typical text mining tasks include text categorisation, text clustering, concept/entity extraction,
production of granular taxonomies, sentiment analysis, document summarisation, and entity
relation modelling (i.e., learning relations between named entities).
Multimedia Data stores image, audio and video data. Multimedia data objects such as video
require gigabytes of storage, so specialised storage and search techniques are required. Moodle
stores a large amount of multimedia data, for example, an instructor’s presentations (in MS
PowerPoint or PDF format, etc.), an instructor’s images (in JGP or GIF format, etc.), a student’s
work and exercises (in MS Word or PDF format, etc.), instructor’s videos (in AVI, MOV or FLASH
format), etc. All these files are stored in a Moodle data directory. Instructors can browse all
these files directly using the Moodle files interface (Figure 6) or download them to their own
local disk as a backup ZIP file. As seen in Figure 6, the Moodle data directory has a root
directory where files are uploaded by the instructor and several default directories, such
as the Moddata directory, which contains data submitted by the students. The backup data
directory contains backup files of the entire course.
Figure 6: Moodle Files
Dr. Saswati Mukherjee, School of Education Technology, JU 8
World Wide Web Data
The World Wide Web (WWW) provides four main types of source data: texts, graphics, videos,
and sound files. Intra-page structure describes the organisation of the content. Intra-page
structure information includes arranging various HTML or XML tags on a page. Inter-page
structure information consists of hyperlinks that connect one page to another. User data
describes the patterns of Web page usage. Web-based systems record user actions in Weblogs,
known as click-stream records, which provide raw tracking of user navigation on the site.
7. EDM/LA Knowledge Discovery Cycle
Applying EDM/LA is a cycle application of the general knowledge discovery and data mining
(KDD) process, with specific characteristics in each step.
Educational Environment
Different kinds of data are collected from LMS ITS, MOOC, etc., to resolve different educational
problems.
Educational Data
Educational data is collected from various sources, including interactions between instructors,
students, and the educational platform (such as navigation behaviour, quiz inputs, interactive
exercises, forum messages, etc.), administrative data (school and teacher information),
demographic data (gender, age, etc.), student affectivity (motivation, emotional states), and
more.
Preprocessing and Transformation
Educational data (raw, original, or primary data) are not available in appropriate form, so it is
necessary to convert the data to an appropriate form for solving each educational problem. This
includes choosing what data to collect, focusing on the questions to be answered, and ensuring
the data aligns with the questions. Traditional preprocessing tasks, such as feature engineering,
are applied to educational data to select attributes/variables with information. Continuous
attributes are normally transformed /discretised into categorical attributes to improve
comprehensibility. Consideration of ethical issues, data privacy, informed consent, etc., when
using educational data is also mandatory.
Methods and Techniques
Traditional data mining techniques include visualisation, classification prediction, clustering,
outlier detection, relationship mining, causal mining, social network analysis, process mining,
and text mining. Other methods are human judgment, knowledge tracing, and non-negative
matrix factorisation.
Interpretation and Application of New Knowledge
The ultimate goal of any learning analytics process and the results of follow-up actions will
determine the success or failure of the analytical efforts. The new knowledge discovered by the
EDM/LA methods is used by instructors and academic authorities to improve student learning
performance.
Dr. Saswati Mukherjee, School of Education Technology, JU 9
Benefit to Learners, Teachers and Course Designers
Figure 7 presents the basic setting of EDM with a few stakeholders (learners, teachers, study
advisers, directors of education, and educational researchers) who can benefit from EDM in
different ways. Students can receive personalized advice and recommendations about courses
Fig 7: Educational data mining in a nutshell
and tasks based on their needs, while teachers can assess material effectiveness and student
performance. Educational data mining (EDM) goes beyond logs of intelligent tutoring and
learning management systems, aiming to address various learning process phases. Examples of
problems include reorganizing classes based on performance data and identifying students who
need feedback or assistance.
Summary
EDM (Educational Data Mining) and LA (Learning Analytics) share a common goal of enhancing
learning and improving performance. They have similar components but are now distinguished
by clear boundaries. EDM involves automated systems with minimal human intervention, while
LA includes system designs with an instructor. Decision-making based on data patterns differs
between the two, with EDM involving machine decision-making and LA involving human
decision-making. Efforts are underway to integrate Learning Management Systems (LMS) with
Massive Open Online Courses (MOOCs) and Intelligent Tutoring Systems to better support
learners in problem-solving.
Dr. Saswati Mukherjee, School of Education Technology, JU 10