ADVANCE MACHINE LEARNING
Unit I : Machine Learning Paradigms and Optimization using Gradient Descent
Definition of Machine Learning, types of learning: Supervised Learning, Unsupervised Learning,
Semi-Supervised Learning, Reinforcement Learning, Self-Supervised Learning (overview), and key
application areas. Learning Framework in ML:Concept of model training, hypothesis space, cost/loss
function, empirical risk minimization, and the objective of learning. Role of Optimization in Machine
Learning: Importance of optimization in training ML models, Convex vs Non-convex optimization,
convergence behavior. Gradient Descent for Regression and Classification: Linear Regression,Logistic
[Link] of Gradient Descent: Batch Gradient Descent, Stochastic Gradient Descent
(SGD), Mini-Batch Gradient Descent, Regularization term in Optimization:L1 Regularization (Lasso)
and L2 Regularization (Ridge).
Unit-II : Introduction to Feature Selection
Definition of feature selection, Importance and benefits:Improves model accuracy,Reduces overfitting,
Types of Feature Selection Techniques: Filter Methods (Univariate selection) , Chi-Square
Test,ANOVA F-test,Mutual Information, Correlation Coefficient (Pearson/Spearman), Feature
Ranking and Thresholding, Significant Feature Selection (SIGF), information Gain Attribute Ranking
(INFG), Gain Ratio (GNR), CFS Subset Evaluator (CFS), OneR Attribute Evaluation (OneR).
Wrapper Methods : Forward Selection, Backward Elimination, Recursive Feature Elimination (RFE),
Embedded Methods: Feature selection is part of model training: Lasso Regression (L1 regularization),
Ridge Regression (L2 regularization), Elastic Net,Tree-based feature importance (Random Forest,
XGBoost).Heuristic and nature-inspired algorithms For Feature selection ,Genetic Algorithm (GA) ,
Particle Swarm Optimization (PSO), Ant Colony Optimization (ACO)
Unit III : Model Evaluation after Feature Selection
Classification models:Accuracy,Precision, Recall, F1-Score,ROC-AUC and PR-AUC,Confusion
Matrix, Log Loss, Cohen’s Kappa, MCC (Matthews Correlation Coefficient). Evaluation metrics for
regression models: Mean Absolute Error (MAE), Mean Squared Error (MSE), Root Mean Squared
Error (RMSE), R-squared (R²) and Adjusted R², Mean Absolute Percentage Error (MAPE). Compare
performance before and after feature selection, Apply cross-validation techniques, visualization tools
for evaluation: Confusion Matrix Heatmap,ROC and Precision-Recall Curves, Learning Curves,
Feature Importance Plots, Residual plots for regression
Unit 4 : Parametric and Non-Parametric Tests
Introduction to Statistical Inference : Hypothesis Testing Framework, Parametric Tests Overview:
Assumptions (Normality, Homogeneity of Variance, etc.),One-sample and Two-sample Parametric
Tests. ypes of Parametric Tests : Z-Test:One Sample Z-Test for Mean,Two Sample Z-Test for
Means,Z-Test for Proportions . T-Test:One Sample T-Test,Two Independent Sample T-Test,Paired T-
Test. ANOVA (Analysis of Variance):One-Way ANOVA,Two-Way ANOVA,Post-hoc tests (Tukey,
Bonferroni).
Non-Parametric Tests: Chi-Square Tests:Test of Independence (Test of Goodness of Fit),, Mann–
Whitney U Test (Alternative to Independent T-Test), Wilcoxon Signed-Rank Test (Alternative to
Paired T-Test), Kruskal-Wallis H Test (Alternative to One-Way ANOVA), Friedman Test (Alternative
to Two-Way ANOVA for repeated measures), Spearman Rank Correlation (Alternative to Pearson
Correlation)
Unit V: Advance Topic
Introduction to Neural Networks and Deep Learning. Recent Advancements in AI: Explainable AI
(XAI), Edge AI, Generative AI, AI for Social Good (Applications of AI in healthcare, education,
climate change, and sustainability), AI Security (Protecting AI systems from attacks and ensuring
secure AI development)
Case studies of AI-driven innovations: Smart Speakers, Self-Driven cars, Smart Agriculture.
Text Books / Reference Books:
1. M. Mohri, A. Rostamizadeh, and A. Talwalkar, Foundations of Machine Learning, 2nd ed. MIT Press, 2018.
2. E. Alpaydin, Introduction to Machine Learning, 4th ed. MIT Press, 2020.