1.
What is Data Analytics
Data Analytics is the process of examining, cleaning, transforming, and interpreting data to discover
useful information, draw conclusions, and support decision-making. It helps organisations understand
their data better, identify patterns, solve problems, and improve overall performance.
At its core, data analytics converts raw, often messy, numbers and observations into structured insight. In
an engineering context this could mean converting hundreds of sensor readings from a machine into a
clear statement about when it is likely to fail; in a business context it could mean converting millions of
customer transactions into a recommendation about which product to promote next. In every case, the
underlying goal is the same — to reduce uncertainty and support a decision with evidence rather than
guesswork.
How Data Analytics Works
The working of data analytics can be understood as a six-step cycle. It begins by defining clear objectives
for the analysis, followed by collecting relevant data from appropriate sources. The raw data is then
cleaned and processed to remove errors and inconsistencies, after which suitable analysis techniques are
applied to extract meaning. The results are visualised and communicated through charts, dashboards, or
reports, and finally interpreted to support practical decision-making.
These six steps are rarely a strict one-way path. In practice, the cycle is iterative: interpretation of an
early result often sends the analyst back to collect more data or redefine the objective, and cleaning is
frequently repeated as new anomalies surface during analysis. Treating the process as a loop rather than a
straight line is what allows data analytics to remain accurate and relevant as new data becomes available.
Step Description
Define Objectives Clarify the goal and questions the analysis should answer.
Data Collection Gather relevant data from surveys, sensors, databases, or literature.
Data Cleaning & Processing Remove errors, duplicates, and inconsistencies from the raw data.
Analysis Techniques Apply statistical or machine learning methods to the cleaned data.
Visualisation & Present findings using charts, graphs, and dashboards.
Communication
Interpretation of Results Draw conclusions and translate results into decisions.
Technical Skills for Data Analytics
A data analyst typically relies on a mix of statistical, mathematical, and software skills. These include
SQL for querying databases, core statistics and mathematics for building sound analyses, Excel for quick
calculations, dedicated data visualisation tools, and programming languages such as Python or R for
automation and modelling.
Beyond these technical skills, an effective analyst also needs domain knowledge — an understanding of
the field the data comes from — since the same numbers can be interpreted very differently depending on
the underlying engineering or business context. This combination of technical capability and contextual
judgement is what separates a purely mechanical calculation from genuinely useful analytics.
Applications of Data Science
Data science techniques are applied across nearly every industry to solve domain-specific problems, as
summarised below. The breadth of these applications illustrates why data analytics is now considered a
foundational skill across engineering, business, and scientific disciplines rather than a niche specialism
confined to computer science.
Domain Representative Applications
E-commerce Identifying consumers, recommending products, analysing reviews
Manufacturing Predicting potential problems, monitoring systems, automating units, maintenance
scheduling, anomaly detection
Banking Fraud detection, credit risk modelling, customer lifetime value
Healthcare Medical image analysis, drug discovery, bioinformatics, virtual assistants
Transport Self-driving cars, enhanced driving experience, car monitoring, passenger safety
Finance Customer segmentation, strategic decision-making, algorithmic trading, risk
analytics
Notably for a mechanical engineering learner, the manufacturing applications listed above — predictive
maintenance, monitoring, and anomaly detection — are the ones most directly relevant to this course,
since they connect data analytics back to physical systems, machine health, and component reliability
rather than purely commercial data.
2. Experiment No. 1 — Data Science, Data Analytics, and AI-ML-
DL
This experiment is a theoretical study aimed at building a clear conceptual foundation of data science,
data analytics, and the AI–ML–DL hierarchy, before these concepts are applied in later, computation-
based experiments.
Rather than involving any coding or dataset, this first experiment is deliberately conceptual. Its purpose is
to make sure that before a learner writes a single line of Python or fits a single regression model, they
understand what these terms actually mean, how the underlying fields relate to one another, and where
mechanical engineering applications fit into the broader data science landscape. This grounding makes
the later, hands-on experiments — regression, classification, hypothesis testing, and visualisation — far
easier to interpret correctly.
3. Data Science
Data science is a multidisciplinary field that uses scientific methods, algorithms, and systems to extract
knowledge and insights from data. It involves collecting, cleaning, analysing, and interpreting data to
identify patterns, trends, and relationships, ultimately leading to better decision-making.
Data science sits at the intersection of three broad areas: domain knowledge (understanding client
requirements and business context), computer science (databases, data governance, preprocessing, and
infrastructure), and mathematical/statistical aptitude (designing methods, tuning models, and applying
programming languages such as Python and R). Where computer science and statistics overlap, machine
learning emerges; where domain knowledge and statistics overlap, research and analysis take place.
This three-way intersection is important because it explains why data science is described as
multidisciplinary rather than purely technical. A person who is only strong in programming but has no
domain knowledge may build a technically correct model that answers the wrong question, while a
person who deeply understands the domain but lacks statistical grounding may draw the wrong
conclusion from correct data. Genuine competence in data science requires enough overlap across all
three circles.
Common Data Science Tools
Widely used tools include Tableau, Excel, RapidMiner, DataRobot, and Qlikview on the analytics and
visualisation side, and H2O, TensorFlow, BigML, Snowflake, and Trifacta for large-scale modelling and
data infrastructure.
These tools broadly split into two categories: those aimed at business users who need quick visual insight
without writing code (such as Tableau and Qlikview), and those aimed at data scientists and engineers
who need to build, train, and deploy models at scale (such as TensorFlow and Snowflake). Choosing the
right tool depends on the size of the dataset, the complexity of the modelling task, and the technical
background of the person carrying out the analysis.
Top Data Science Roles
Typical career roles in this field include Data Scientist, Data Architect, Data Engineer, Statistician, Data
Science Manager, Machine Learning Engineer, and Decision Scientist.
These roles are complementary rather than interchangeable: data engineers build and maintain the
pipelines that move and store data, data architects design the overall structure of data systems,
statisticians and data scientists focus on drawing valid conclusions and building predictive models, and
decision scientists and data science managers translate technical findings into business or engineering
action. A single project usually draws on several of these roles working together.
4. Data Analytics
Data analytics is the process of examining raw data to uncover patterns, draw conclusions, and make
informed decisions. It involves collecting, transforming, and organising data to derive insights and drive
business intelligence.
The typical data analytics pipeline begins with defining the parameters of data collection, followed by
data search and generation, and storage of raw data. This is followed by data arrangement and filtering,
after which analysis is carried out using tools such as R, Python, or Excel. The results are visualised,
commonly through pie charts and other graphics, and finally used for forecasting and decision-making.
It is worth distinguishing data analytics from data science at this point, since the two terms are often used
loosely. Data science is the broader discipline, encompassing everything from data engineering to
advanced machine learning; data analytics is best understood as one important application area within
that discipline, focused specifically on examining existing data to answer defined questions and support
reporting, rather than on building new predictive systems from scratch.
Analytics itself is commonly split into four types: descriptive analytics, which summarises what has
happened; diagnostic analytics, which investigates why it happened; predictive analytics, which forecasts
what is likely to happen next; and prescriptive analytics, which recommends what action should be taken.
A mature analytics process typically moves through these four types in order, since each stage builds on
the understanding developed in the one before it.
5. Big Data
Big data refers to extremely large and complex datasets, which may be qualitative or quantitative in
nature. Big data analytics refers to the process of discovering patterns, trends, and correlations among
raw data to make data-driven decisions.
Analysing big data involves applying familiar statistical techniques such as clustering and regression, but
at a much larger scale, using new-age tools capable of handling volume, velocity, and variety that
traditional systems cannot.
Big data is often characterised using the '3 Vs': volume (the sheer amount of data generated, often too
large for a single machine to process), velocity (the speed at which new data is generated and must be
processed, such as continuous sensor streams from a machine), and variety (the range of formats
involved, from structured spreadsheets to unstructured images, text, and video). A dataset that is large in
size but simple in structure is not necessarily 'big data' in the technical sense unless it also challenges
conventional storage, processing speed, or format-handling capabilities.
6. Relation Between AI, ML, and DL
Artificial Intelligence, Machine Learning, and Deep Learning are nested concepts, with each successive
field being a specialised subset of the one before it.
Field Description
Artificial Intelligence (AI) Software techniques intended to enable computers to mimic human
intelligence, using algorithmic logic, if-then statements, decision trees,
machine learning, and more.
Machine Learning (ML) A subset of AI that uses complex mathematics coupled with programming to
enable computers to improve with experience; includes regression and
clustering.
Deep Learning (DL) A subset of ML used for very complex tasks such as speech and image
processing; relies on layered neural networks to analyse extremely large
datasets.
A simple way to remember the nesting is to picture three concentric circles: the outermost and largest
circle is AI, which includes any technique — even a simple set of hand-written rules — designed to
mimic intelligent behaviour. Inside it sits machine learning, restricted to techniques that learn from data
rather than following fixed rules. Inside machine learning sits deep learning, restricted further still to
techniques based specifically on multi-layered neural networks. Every deep learning system is therefore
also a machine learning system and an AI system, but the reverse is not true.
7. Artificial Intelligence (AI)
AI refers to the ability of computer systems to perform tasks that typically require human intelligence,
such as learning, problem-solving, and decision-making. AI uses predictions and automation to optimise
and solve complex tasks that humans have historically performed, such as facial and speech recognition,
decision-making, and translation.
It is helpful to think of AI not as a single technology but as a goal — the goal of building machines that
behave intelligently. That goal can be pursued through many different techniques, ranging from simple
rule-based expert systems and decision trees to sophisticated neural networks. What unites them under
the label 'AI' is the intent to replicate some aspect of human cognitive ability, not any single underlying
method.
Categories of AI
Category Strength Description / Examples
Artificial Narrow Weak Ability to complete a specific task, e.g., Alexa, Siri, chess-
Intelligence (ANI) playing engines, facial recognition, NLP, self-driving cars
Artificial General Strong Would perform at par with a human across general tasks;
Intelligence (AGI) incorporates human-like behaviour
Artificial Super Intelligence Strong Would surpass human intelligence and ability; neither AGI nor
(ASI) ASI exists yet, though research is ongoing
Virtually every AI system in everyday use today — voice assistants, recommendation engines, spam
filters, and self-driving car perception systems — falls into the Artificial Narrow Intelligence category.
Each of these systems is highly capable within its specific task but cannot transfer that capability to an
unrelated task; a chess engine, however strong, cannot drive a car. AGI and ASI remain research goals
rather than deployed technologies, and their eventual development, if it occurs, is expected to raise
significant technical and ethical questions.
8. Machine Learning
Machine learning is a branch of artificial intelligence that focuses on enabling computer systems to learn
from data and improve their performance on specific tasks without explicit programming. ML allows
machines to identify patterns, make predictions, and improve their decision-making processes based on
the data they analyse, rather than relying on pre-defined rules.
Traditional ML methods (flat algorithms) cannot be applied directly to raw data such as .csv files,
images, or text; they require a preprocessing step known as feature extraction.
Feature extraction is the process of converting raw data into a set of numerical measurements — features
— that a model can actually use. For example, rather than feeding a raw image of a machined surface
into a traditional ML model, an engineer might first extract features such as average roughness, peak
height, and spacing between defects. The quality of this feature extraction step often determines the
accuracy of the final model, which is one of the main reasons deep learning, discussed later, has become
so influential — it removes much of the manual effort this step traditionally required.
Types of Machine Learning
Type Techniques Example Applications
Unsupervised Learning Dimensionality reduction, Big data visualisation, meaningful compression,
clustering structure discovery, recommender systems, customer
segmentation
Type Techniques Example Applications
Supervised Learning Classification, regression Identity fraud detection, image classification,
customer retention, diagnostics, weather and market
forecasting
Reinforcement Real-time decisions, robot Skills acquisition, game AI, learning tasks
Learning navigation
These three types differ mainly in what kind of feedback the model receives while learning. Supervised
learning is trained on data that already has known, labelled outcomes, so the model learns by comparing
its predictions against the correct answer. Unsupervised learning works with unlabelled data and instead
looks for inherent structure, such as natural groupings or reduced representations. Reinforcement learning
differs from both: rather than learning from a fixed dataset, an agent learns by interacting with an
environment and receiving rewards or penalties for its actions, gradually improving its strategy through
trial and error. For mechanical engineering applications, supervised learning — particularly regression
and classification — is generally the most directly applicable of the three, since failure time prediction
and risk-level classification are both supervised problems with known historical outcomes.
9. Deep Learning
Deep learning is a specialised subset of machine learning that utilises artificial neural networks with
multiple layers to analyse data, extract features, and learn complex patterns.
In traditional machine learning, feature extraction is a manual, separate step performed before
classification. Deep learning excels at automatic feature extraction, combining it with classification
within the same network, which makes it especially effective in areas such as image recognition, natural
language processing, and speech recognition. Given an input such as an image of a car, a deep learning
model recognises its unique characteristics and makes correct predictions without human intervention.
A neural network achieves this by passing data through successive layers, where each layer learns to
detect progressively more abstract patterns. In an image recognition task, for instance, the earliest layers
might learn to detect simple edges and colour gradients, middle layers might combine these into shapes
such as wheels or windows, and the final layers combine those shapes into a full concept such as 'car'.
This layered, hierarchical structure is what gives deep learning its name and its ability to handle raw,
unstructured data directly.
10. Deep Learning (continued) — Effect of Data Volume
Deep learning does not require manual effort to perform or optimise the feature extraction process, and it
is powered by massive amounts of data (big data), which opens up significant opportunities for new
innovations.
A useful way to picture this is the example of teaching a robot to understand human language using data
from the entire internet: the internet supplies massive big data — billions of articles, comments, and posts
— and the deep learning model acts as the robot that reads all of it to learn patterns, grammar, context,
and meaning. As shown in the figure below, performance for traditional machine learning plateaus early
even as more data is supplied, while small and large neural networks continue to improve, with large
neural networks benefiting the most from increasing data volume.
This relationship has an important practical implication: the choice of algorithm should depend on how
much data is actually available. When a dataset is small — as is often the case in mechanical engineering
laboratory experiments with only a few dozen data points — a traditional machine learning model, or
even a simple statistical method, will often outperform a deep neural network, because the network has
too few examples to learn from and no advantage over simpler methods. Deep learning's benefits become
apparent only once datasets grow into the thousands or millions of samples, which is why it is far more
common in domains such as image or speech processing than in small, controlled engineering
experiments.
Figure: Performance improves with data volume, with larger neural networks benefiting most (based on Ng, deep
learning performance curves).
11. Relationship Between Data Science, Data Analytics, and AI-
ML-DL
These fields form a nested hierarchy rather than separate, disconnected domains. Artificial Intelligence is
the broadest domain, mimicking human intelligence and encompassing machine learning, natural
language processing, and vision. Machine learning is a subset of AI that learns from data and includes
supervised, unsupervised, and reinforcement learning. Deep learning, built on neural networks, is a
further subset of ML specialised in vision and NLP tasks.
Data science draws on machine learning and statistics to provide end-to-end decision support, covering
data cleaning, visualisation, and modelling. Data analytics, in turn, sits within data science and focuses
more narrowly on reporting and trends — descriptive and diagnostic analysis of data that has already
been prepared. Big data acts as the connective layer that fuels all of these fields, since large volumes of
data are what make advanced AI, ML, and DL techniques effective in practice.
Bringing these ideas together, a typical mechanical engineering data analytics project — such as the mini
project required later in this course — moves through this entire hierarchy in miniature: data is collected
from a physical process, cleaned and explored using data analytics techniques, modelled using a machine
learning method appropriate to the size of the dataset, and, if the dataset is large enough, potentially
refined further using deep learning. Understanding how these fields nest inside one another makes it
much easier to choose the right tool for a given engineering problem, rather than defaulting to the most
advanced-sounding technique regardless of whether the data actually supports it.