Mathematics for Machine Learning (2025 Focus)
# Mathematics for Machine Learning (2025 Focus: Optimization & Probabilistic ML)
## 1. Linear Algebra
Linear Algebra is the foundation of Machine Learning, especially for handling data transformations
and deep learning models.
### 1.1 Vectors, Matrices, Eigenvalues, Eigenvectors
#### Vectors
- A vector is an ordered set of numbers, often represented as a column or row.
- Example: [2, 3, 5]
- **Operations:** Addition, scalar multiplication, dot product, cross product (for 3D vectors).
#### Matrices
- A matrix is a 2D array of numbers.
- Notation: A matrix A with dimensions mn has m rows and n columns.
- **Operations:** Addition, multiplication, transpose, inverse.
#### Eigenvalues & Eigenvectors
- Given a square matrix A, an eigenvector v satisfies: A v = v
- **Importance in ML:** Used in Principal Component Analysis (PCA) for dimensionality reduction.
### 1.2 Singular Value Decomposition (SVD)
- Factorizes a matrix A into three matrices: A = U V^T
- **Applications in ML:** Dimensionality reduction, latent semantic analysis in NLP.
### 1.3 Tensors
- Generalization of matrices to higher dimensions.
- Used in Deep Learning frameworks like TensorFlow and PyTorch.
## 2. Probability & Statistics
Understanding probability is crucial for probabilistic models and Bayesian inference.
### 2.1 Bayesian Inference
- Based on Bayes Theorem: P(A | B) = (P(B | A) P(A)) / P(B)
- **Applications in ML:** Naive Bayes classifier, probabilistic graphical models.
### 2.2 Probability Distributions
- **Normal Distribution:** Used in modeling continuous variables.
- **Poisson Distribution:** Models the number of events occurring in a fixed interval.
- **Exponential Distribution:** Models the time between events in a Poisson process.
### 2.3 Markov Chains
- A stochastic model describing a sequence of possible events where the probability of each event
depends only on the previous state.
- **Application in AI:** Reinforcement Learning, language modeling.
## 3. Calculus & Optimization
Optimization techniques are essential for training machine learning models.
### 3.1 Gradient Descent & Variants
- **Gradient Descent:** Minimizes a function by moving in the direction of the negative gradient.
- **Variants:** Stochastic Gradient Descent (SGD), Mini-batch Gradient Descent, Adam, RMSprop.
### 3.2 Lagrange Multipliers
- Method for constrained optimization: Used in SVM for maximizing the margin.
### 3.3 Convex & Non-Convex Optimization
- **Convex Functions:** One global minimum, easy to optimize.
- **Non-Convex Functions:** Many local minima, difficult to optimize (common in Deep Learning).
## Conclusion
Mastering these mathematical concepts will help in understanding and implementing machine
learning algorithms efficiently in 2025.