Exploring Advanced
Machine Learning
Models
Dr. Saima Khosa
Computer science
Decision Trees
• The Anatomy of a Decision Tree
• Exploring the components of a decision tree.
Points:
• Nodes: Decision points where data is split.
• Edges/Branches: Paths between nodes based on decision
outcomes.
• Leaves: Terminal nodes that represent the prediction
outcome.
Decision
Tree
Constructing a Decision Tree
• Step-by-step guide to building a decision tree.
• Selecting the Best Split: Using measures like Gini impurity
or entropy to choose the most effective feature at each node.
• Recursive Splitting: Continue splitting data at each node
until a stopping criterion (like max depth or minimum
samples) is met.
• Tree Pruning: Reducing the size of the tree after building it
to avoid overfitting.
Practical Considerations for
Decision Trees
How to enhance the effectiveness of decision trees.
• Handling Missing Data: Strategies for dealing with
incomplete data during tree construction.
• Feature Importance: Evaluating which features contribute
most to the decision-making process.
Random Forest
• Breakdown of the Random Forest architecture and its
ensemble nature.
• Ensemble Learning: Combines predictions from multiple
machine learning algorithms to produce better results than
could be obtained from any of the individual models alone.
• Base Models: Random Forest uses decision trees as its base
models, each trained on a random subset of the overall
training data.
Step-by-Step Creation of a
Random Forest
• Detailed steps in building a Random Forest model.
• Impotent Points:
• Bootstrap Aggregating (Bagging): Random selection of
data points and features to train each tree.
• Training Individual Trees: Each tree learns from its subset
independently to predict outcomes.
• Aggregation of Results: Combining tree predictions
through voting or averaging.
Tuning a Random Forest Model
• Key parameters and their effects on model
performance.
• Key Points:
• Number of Trees: More trees can improve accuracy but
increase computational cost.
• Max Depth of Trees: Controls the depth of each tree;
deeper trees learn finer details but can overfit.
• Min Samples Split: The minimum number of samples
required to split an internal node.
Applications of Random Forest
• Discusses where Random Forest is effectively used.
• Points:
• Finance for credit scoring.
• Biomedical fields for disease prediction.
• E-commerce for recommending products.
Naïve Bayes
• Principles Behind Naïve Bayes
• Fundamental assumptions and formulae driving Naïve
Bayes.
• Key Points:
• Bayes’ Theorem: Describes the probability of an event,
based on prior knowledge of conditions that might be related
to the event.
• Feature Independence: Assumes all features are
independent of each other, simplifying calculation.
Training a Naïve Bayes Classifier
• Steps involved in developing a Naïve Bayes model.
• Key Points:
• Data Preparation: Convert data into a format suitable for
model training (e.g., frequency counts for categorical data).
• Probability Estimation: Calculate probabilities for each
class based on training data.
• Decision Rule: Classify new instances based on the class
with the highest posterior probability.
Enhancing Naïve Bayes
Performance
• Tips for improving accuracy and effectiveness.
• Key Points:
• Smoothing Techniques: Such as Laplace smoothing to
handle zero probabilities in the dataset.
• Feature Selection: Identifying the most relevant features to
reduce noise and improve model efficiency.