Introduction to Machine

Learning Models
Understanding the Basics

Wajih ur Rehman, PhD

 What is Machine Learning?
• Machine learning is a branch of artificial
intelligence that allows systems to learn from
data, identify patterns, and make decisions with
minimal human intervention.
• Key Concepts:
• Data-driven: Models learn from data instead of
being programmed with rules.
• Algorithms: Mathematical formulas that process
data to extract patterns.
• Patterns: Recognizing trends and relationships in
the data that can be used for predictions.
 Types of Machine Learning
• Supervised Learning:
– Involves training a model on a labeled dataset (input-output
pairs).
– The model learns to predict outputs from given inputs.
• Unsupervised Learning:
– Deals with datasets without labeled responses.
– The model tries to find hidden structures or groupings in the
data.
• Reinforcement Learning:
– The model learns by taking actions in an environment to
maximize cumulative rewards.
– Uses feedback to reinforce good actions and penalize bad ones.
 Supervised Learning
• Learning from a labeled dataset to make predictions or
classify data.
• Key Algorithms:
– Linear Regression: Used for predicting continuous values; finds
the best-fitting line through data points.
– Decision Trees: A flowchart-like structure that makes decisions
based on feature values.
– Support Vector Machines: Finds the hyperplane that best
separates different classes in the feature space.
– Neural Networks: Mimics the human brain's structure, used for
complex patterns and deep learning.
• Applications: Email spam detection (classification),
predicting weather, house/stock prices (regression), and
medical diagnosis.
Linear Regression
Decision Trees
Support Vector Machines
Artificial Neural Network
 Unsupervised Learning
• Learning from data without labels to discover underlying
patterns.
• Key Algorithms:
– K-Means Clustering: Groups data points into K clusters based on
feature similarity.
– Hierarchical Clustering: Builds a tree of clusters by either
merging or splitting them.
– Principal Component Analysis (PCA): Reduces dimensionality of
data while preserving variance, useful for visualization.
• Applications: Market segmentation (grouping customers),
recommendation systems (finding similar items), and
anomaly detection (identifying unusual patterns or outliers
etc.).
K-means clustering
Hierarchical Clustering
Principal Component Analysis
 Reinforcement Learning
• Learning by interacting with an environment to
maximize some notion of cumulative reward.
• Key Concepts:
– Agent: The learner or decision-maker.
– Environment: The context in which the agent operates.
– Actions: Choices made by the agent to affect the
environment.
– Rewards: Feedback from the environment based on
actions taken.
• Applications: Game playing (like AlphaGo), robotics
(navigation), self-driving cars (real-time decision
making) and Nuclear Fusion Reactor.
 Machine Learning Workflow/Steps
1. Data Collection: Gathering relevant data from various sources
(databases, etc.).
2. Data Preprocessing: Cleaning data (handling missing values,
outliers) and transforming it (normalization, encoding categorical
variables).
3. Model Selection: Choosing the right algorithm based on the
problem type (classification, regression, clustering).
4. Training the Model: Feeding the training data into the model to
learn patterns.
5. Model Evaluation: Testing the model on unseen data to assess
performance using metrics like accuracy, precision, recall.
6. Deployment: Integrating the trained model into a production
environment where it can make real-time predictions.
7. Monitoring and Maintenance: Continuously tracking model
performance and updating it as needed to adapt to new data.
 Statistical measures
• Here’s a brief explanation of R², MAE, and RMSE — commonly used metrics for
evaluating regression models:
• 1. R² Score (Coefficient of Determination)
• Definition: Indicates how well the model explains the variance in the target
variable.
• Range: 0 to 1 (can be negative if model performs worse than a horizontal line).
• Formula:

• Where:
– SSresSS_{res}: Sum of squared residuals
– SStotSS_{tot}: Total sum of squares
• Interpretation:
– 1 → perfect prediction
– 0 → model predicts no better than mean
– <0 → model is worse than just predicting the mean
 Mean Absolute Error
• 2. MAE (Mean Absolute Error)
• Definition: Average of the absolute differences
between predicted and actual values.
• Formula:

• Interpretation:
– Directly measures average error in the units of the
target variable
– Less sensitive to outliers than RMSE
 Root Mean Squared Error
• 3. RMSE (Root Mean Squared Error)
• Definition: Square root of the average of squared
differences between predicted and actual values.
• Formula:

• Interpretation:
– Penalizes larger errors more than MAE
– Useful when large errors are particularly undesirable
 Evaluating Models
• Key Metrics:
• Accuracy: The ratio of correctly predicted instances to the total
instances.
• Precision and Recall: Precision measures the accuracy of positive
predictions; recall measures the model's ability to find all relevant
instances.
• F1 Score: The harmonic mean of precision and recall, useful for
imbalanced classes.
• Confusion Matrix: A table that describes the performance of a
classification model by showing true vs. predicted values.
• ROC-AUC Curve: Evaluates the trade-off between true positive rate
and false positive rate, helping to choose the optimal model
threshold.
• Importance of Validation: Cross-validation helps prevent overfitting
by testing the model on different subsets of the data.
 Accuracy and Precision
Correct Predictions
• Accuracy =
𝑇𝑜𝑡𝑎𝑙 𝐼𝑛𝑠𝑡𝑎𝑛𝑐𝑒𝑠
 Recall

There is a potential trade-off between precision and recall.

A higher precision (filter avoids flagging important emails) might come at the cost
of lower recall (missing some spam emails).
You’ll need to decide which metric is more important for your needs. For instance,
if missing important emails is a bigger concern, you might adjust the filter to prioritize
higher recall, even if it means catching some non-spam emails in the process.
F1 score
The F1 score is the harmonic mean of
precision and recall, ranging from 0 to
1, with 1 representing perfect
precision and recall.
Of a classification model
Confusion Matrix
ROC-AUC Curve

Receiver Operating Characteristic (ROC) curve

 Challenges in Machine Learning
• Common Issues:
• Overfitting: When a model learns noise and details in the
training data to the extent that it performs poorly on
unseen data.
• Underfitting: When a model is too simple to capture the
underlying trend in the data.
• Bias-Variance Tradeoff: Balancing model complexity
(variance) with accuracy (bias) to achieve optimal
performance.
• Data Quality and Quantity: Ensuring that data is sufficient,
relevant, and of high quality for training.
• Interpretability: Many models (especially deep learning) are
complex and can be challenging to interpret.
 Bias-Variance trade off
• The bias-variance tradeoff in machine learning
refers to the inherent conflict between a model's
ability to accurately represent the underlying
data patterns (low bias) and its sensitivity to
variations in the training data (high variance).
• In essence, it's about finding the right balance
between a model that is too simple (underfitting,
high bias) and a model that is too complex
(overfitting, high variance).
Bias Variance Trade Off
 Future of Machine Learning
• Trends:
• Explainable AI: Developing models that provide
transparency and interpretability to users.
• Automated Machine Learning (AutoML): Tools that
automate the process of applying machine learning to
real-world problems.
• Advancements in Neural Networks: Innovations in
architectures, such as transformers, that enhance
model capabilities.
• Ethical Considerations: Addressing biases in models
and ensuring fairness, accountability, and transparency
in AI systems.
 Conclusion and further study
• Machine learning models are transforming
industries by automating decision-making and
uncovering insights from data.
• Study Machine Learning models from various
sources
 Questions and Discussion
Ask questions, share your thoughts, or
discuss specific interests in machine learning
 An opportunity in UK
• As part of the project, you will utilise in-
process monitoring techniques employing a
variety of sensing technologies to generate
data that will be used to measure and monitor
the process parameters. You will utilise sensor
data capture and analytics to identify links
between process data and the production
outcomes and utilise control methods that can
make use of the identified links to ensure
successful production outcomes.