Machine Learning Fundamentals
Core concepts, models, evaluation and practical workflow
Original educational study guide
Machine Learning Fundamentals | Page 1
1. Machine Learning Overview
What is ML?
Machine learning builds models that learn patterns from data to make predictions or decisions. A model is
useful when it generalizes to unseen examples, not merely when it memorizes training data.
Types
Supervised learning uses target labels. Unsupervised learning works without target labels. Reinforcement
learning learns through interactions and rewards.
Pipeline
A typical pipeline includes problem definition, data collection, cleaning, splitting, preprocessing, model training,
validation, evaluation and deployment.
Machine Learning Fundamentals | Page 2
2. Data Preparation
Features and targets
Features are input variables; the target is the quantity to predict. Leakage occurs when information unavailable
at prediction time enters the features.
Train/validation/test
Training data fits the model. Validation data supports model selection and tuning. The test set should be
reserved for final unbiased evaluation.
Preprocessing
Scaling is important for distance-based and gradient-based models. Categorical variables often require
encoding. Preprocessing should be fit on training data only and then applied to validation/test data.
Machine Learning Fundamentals | Page 3
3. Regression
Linear regression
Linear regression predicts a continuous target using a weighted combination of features.
y_hat = β0 + β1x1 + ... + βpxp.
Loss
Mean squared error penalizes squared prediction errors.
MSE=(1/n)Σ(y_i−y_hat_i)^2.
Assumptions and limitations
Linear regression can be sensitive to outliers and correlated features. Good performance depends on the
relationship between features and target.
Machine Learning Fundamentals | Page 4
4. Classification
Logistic regression
Logistic regression models class probability using a sigmoid transformation.
σ(z)=1/(1+e^(−z)).
Confusion matrix
For binary classification, outcomes are TP, TN, FP and FN. Accuracy can be misleading for severe class
imbalance.
Precision and recall
Precision=TP/(TP+FP).
Recall=TP/(TP+FN).
F1 combines precision and recall using their harmonic mean.
Machine Learning Fundamentals | Page 5
5. Trees and Ensembles
Decision trees
Trees split the feature space using rules. They are interpretable but can overfit when allowed to grow too
complex.
Random forests
Random forests average many randomized decision trees, reducing variance compared with a single tree.
Boosting
Boosting builds models sequentially, focusing later models on errors made earlier. It can be highly effective but
requires careful tuning.
Machine Learning Fundamentals | Page 6
6. Nearest Neighbors and Clustering
KNN
K-nearest neighbors predicts using nearby training examples. Feature scaling matters because distance
depends on feature magnitude.
K-means
K-means partitions observations into k clusters by iteratively assigning points to centroids and updating
centroids.
Choosing k
The elbow method and silhouette score are common exploratory tools, but domain knowledge and stability
should also be considered.
Machine Learning Fundamentals | Page 7
7. Model Evaluation
Regression metrics
MAE measures average absolute error. MSE emphasizes large errors. RMSE is the square root of MSE and is
in target units.
MAE=(1/n)Σ|y_i−y_hat_i|.
Classification metrics
Accuracy, precision, recall, F1 and ROC-AUC answer different questions. Choose a metric based on the cost of
false positives and false negatives.
Cross-validation
K-fold cross-validation repeatedly trains and validates on different partitions to estimate generalization
performance.
Machine Learning Fundamentals | Page 8
8. Overfitting and Regularization
Bias and variance
High bias models are too simple; high variance models are too sensitive to training data. The goal is an
appropriate balance.
Regularization
L2 regularization penalizes squared weights; L1 encourages sparse solutions.
Objective = loss + λ penalty.
Practical controls
Use more data, simpler models, regularization, early stopping, feature selection or augmentation where
appropriate.
Machine Learning Fundamentals | Page 9
9. Neural Networks
Basic unit
A neuron computes a weighted sum followed by an activation function. Layers compose these transformations
into a network.
Training
Training typically minimizes a loss using gradient-based optimization. Backpropagation computes gradients
efficiently using the chain rule.
Common activations
ReLU is common in hidden layers. Sigmoid and softmax are often used for probabilities in suitable output
settings.
Machine Learning Fundamentals | Page 10
10. Responsible ML
Data quality
Biased, incomplete or noisy data can produce unreliable models. A high test score does not automatically imply
fairness or safety.
Interpretability
Feature importance, partial dependence and local explanation methods can help analyze models, but
explanations should be interpreted carefully.
Deployment
Monitor performance, data drift, latency and failures after deployment. A model is part of a larger system, not
the entire solution.
Machine Learning Fundamentals | Page 11
Practice Questions
1. Define supervised learning.
2. Why separate validation and test data?
3. What is data leakage?
4. Write the MSE formula.
5. What does precision measure?
6. What does recall measure?
7. Why can accuracy be misleading?
8. What does KNN depend on?
9. What is overfitting?
10. What does regularization do?
11. What is cross-validation?
12. What is backpropagation?
Answer Key / Self-check
1. Learning from labeled input-output examples.
2. Validation supports model selection; the test set estimates final generalization.
3. Using information in features that would not be available at prediction time.
4. (1/n)Σ(y_i−y_hat_i)^2.
5. Of predicted positives, the fraction that are truly positive.
6. Of actual positives, the fraction detected.
7. A majority class can dominate the score.
8. Distances to nearby training examples, so scaling can matter.
9. Fitting training data too closely and generalizing poorly.
10. Adds a penalty that discourages overly complex parameter values.
11. Repeated train/validation splits used to estimate performance.
12. Efficient gradient computation through the network using the chain rule.
Final Revision Checklist
• Review the definitions before memorizing formulas.
• Work through the examples without looking at the solution first.
• Write down assumptions whenever a formula depends on them.
• Check units, dimensions, shapes and boundary cases in numerical work.
• Practice explaining each concept in your own words.
Machine Learning Fundamentals | Page 12