Practical 1

Aim: Introduction to Machine Learning

1. Introduction to Machine Learning

Machine Learning (ML) is a branch of Artificial Intelligence that enables computers to learn
patterns from data and make predictions or decisions without being explicitly programmed
for every task. Instead of following fixed rules, an ML system improves its performance
automatically as it is exposed to more data.
Example:
from sklearn.linear_model import LinearRegression
model = LinearRegression() # a simple ML model

2. Types of Machine Learning

Machine Learning is broadly classified into three types: Supervised Learning, where the
model learns from labeled data (input-output pairs); Unsupervised Learning, where the model
finds hidden patterns in unlabeled data; and Reinforcement Learning, where an agent learns
by interacting with an environment and receiving rewards or penalties.

3. Supervised Learning
In supervised learning, the algorithm is trained on a dataset that contains both input features
and the correct output (label), and it learns to map inputs to outputs. Common tasks include
classification, where the output is a category, and regression, where the output is a continuous
value.
Example:
[Link](X_train, y_train) # learns from labeled data

4. Unsupervised Learning
In unsupervised learning, the algorithm works with data that has no labeled output and tries to
discover hidden structures, groupings, or patterns on its own. Clustering and dimensionality
reduction are common unsupervised techniques.
Example:
from [Link] import KMeans
kmeans = KMeans(n_clusters=3).fit(X)

5. Reinforcement Learning
Reinforcement learning involves an agent that learns to make decisions by performing actions
in an environment and receiving feedback in the form of rewards or penalties. The goal of the
agent is to learn a policy that maximizes the cumulative reward over time, and it is widely
used in robotics and game playing.

6. Machine Learning Terminology

A few key terms are essential to understanding ML: features are the input variables used to
make predictions, labels are the output values being predicted, a dataset is divided into
training data (used to teach the model) and testing data (used to evaluate it), and a model is
the mathematical representation learned from the data.

7. Data Preprocessing
Data preprocessing is the step of cleaning and preparing raw data before feeding it to a
model, since real-world data is often incomplete, inconsistent, or contains errors. Common
preprocessing tasks include handling missing values, removing duplicates, encoding
categorical variables, and scaling numerical features.
Example:
[Link]([Link](), inplace=True) # handle missing values

8. Feature Scaling
Feature scaling is the process of standardizing the range of independent variables so that
features with larger numeric ranges do not dominate the learning process. Common methods
include normalization, which rescales values to a range of 0 to 1, and standardization, which
rescales data to have a mean of 0 and a standard deviation of 1.
Example:
from [Link] import StandardScaler
X_scaled = StandardScaler().fit_transform(X)

9. Train-Test Split
The train-test split is the practice of dividing a dataset into two parts: a training set used to fit
the model, and a testing set used to evaluate how well the model generalizes to new, unseen
data. A common split ratio is 80% training data and 20% testing data.
Example:
from sklearn.model_selection import train_test_split
X_train, X_test, y_train, y_test = train_test_split(X, y,
test_size=0.2)

10. Linear Regression

Linear regression is a supervised learning algorithm used to predict a continuous numeric
output by fitting a straight line that best represents the relationship between the input
feature(s) and the target variable. It works by minimizing the difference between predicted
and actual values.
Example:
model = LinearRegression().fit(X_train, y_train)
prediction = [Link](X_test)

11. Logistic Regression

Despite its name, logistic regression is a classification algorithm used to predict a categorical
outcome, typically binary (yes/no, 0/1). It uses a sigmoid function to convert the model's
output into a probability between 0 and 1, which is then mapped to a class label.
Example:
 from sklearn.linear_model import LogisticRegression
model = LogisticRegression().fit(X_train, y_train)

12. Decision Tree

A decision tree is a classification and regression algorithm that splits data into branches based
on feature values, forming a tree-like structure of decisions that leads to a final prediction at
the leaf nodes. It is easy to interpret and visualize but can overfit if not properly controlled.
Example:
from [Link] import DecisionTreeClassifier
model = DecisionTreeClassifier().fit(X_train, y_train)

13. K-Nearest Neighbors (KNN)

K-Nearest Neighbors is a simple supervised algorithm that classifies a new data point based
on the majority class among its 'k' closest neighbors in the training data, using a distance
measure such as Euclidean distance. It is easy to implement but can become slow on large
datasets.
Example:
from [Link] import KNeighborsClassifier
model = KNeighborsClassifier(n_neighbors=5)

14. Support Vector Machine (SVM)

A Support Vector Machine is a classification algorithm that finds the optimal hyperplane
which best separates data points of different classes with the maximum possible margin.
SVMs can also handle non-linear data using kernel functions.
Example:
from [Link] import SVC
model = SVC(kernel="linear").fit(X_train, y_train)

15. Naive Bayes

Naive Bayes is a classification algorithm based on Bayes' Theorem that assumes all features
are independent of each other. Despite this simplifying ("naive") assumption, it performs well
in practice and is commonly used for text classification tasks like spam detection.
Example:
from sklearn.naive_bayes import GaussianNB
model = GaussianNB().fit(X_train, y_train)

16. K-Means Clustering

K-Means is an unsupervised algorithm that groups data into a specified number (k) of clusters
based on similarity, by iteratively assigning points to the nearest cluster center and
recalculating the center until the clusters stabilize.
Example:
kmeans = KMeans(n_clusters=3).fit(X)
 print(kmeans.labels_)

17. Random Forest and Ensemble Learning

Ensemble learning combines multiple individual models to produce a stronger overall
prediction than any single model alone. Random Forest is a popular ensemble method that
builds many decision trees on random subsets of data and features, then averages their results
(for regression) or takes a majority vote (for classification).
Example:
from [Link] import RandomForestClassifier
model = RandomForestClassifier(n_estimators=100)

18. Overfitting and Underfitting

Overfitting occurs when a model learns the training data too closely, including its noise,
resulting in poor performance on new, unseen data. Underfitting occurs when a model is too
simple to capture the underlying pattern of the data, performing poorly on both training and
test data.

19. Bias-Variance Tradeoff

Bias refers to error caused by overly simplistic assumptions in the model, leading to
underfitting, while variance refers to error caused by excessive sensitivity to small
fluctuations in the training data, leading to overfitting. A good model aims to balance both to
achieve the best generalization on new data.

20. Cross-Validation
Cross-validation is a technique used to evaluate how well a model generalizes to independent
data by splitting the dataset into multiple folds and training/testing the model several times on
different combinations of these folds. K-fold cross-validation is the most commonly used
variant.
Example:
from sklearn.model_selection import cross_val_score
scores = cross_val_score(model, X, y, cv=5)

21. Model Evaluation Metrics

Model evaluation metrics measure how well a trained model performs. For classification
problems, common metrics include accuracy, precision, recall, and F1-score, often
summarized using a confusion matrix. For regression problems, common metrics include
Mean Squared Error (MSE) and Root Mean Squared Error (RMSE).
Example:
from [Link] import accuracy_score
print(accuracy_score(y_test, prediction))

22. Gradient Descent

Gradient descent is an optimization algorithm used to minimize a model's error (loss
function) by iteratively adjusting its parameters in the direction that reduces the error the
most. The size of each adjustment step is controlled by a value called the learning rate.

23. Regularization
Regularization is a technique used to prevent overfitting by adding a penalty term to the
model's loss function that discourages overly complex models. The two most common types
are L1 regularization (Lasso), which can shrink some coefficients to zero, and L2
regularization (Ridge), which shrinks coefficients evenly.
Example:
from sklearn.linear_model import Ridge
model = Ridge(alpha=1.0).fit(X_train, y_train)

24. Hyperparameter Tuning

Hyperparameters are configuration settings of a model that are set before training begins,
such as the number of neighbors in KNN or the depth of a decision tree. Hyperparameter
tuning is the process of finding the best combination of these settings, commonly done using
techniques like Grid Search or Random Search.
Example:
from sklearn.model_selection import GridSearchCV
grid = GridSearchCV(model, param_grid)

25. Introduction to Neural Networks and Deep Learning

A neural network is a machine learning model inspired by the structure of the human brain,
made up of layers of interconnected nodes (neurons) that process and transform input data to
produce an output. Deep learning refers to neural networks with many hidden layers, capable
of learning complex patterns from large amounts of data, and is widely used in image
recognition, speech processing, and natural language processing.
Example:
from sklearn.neural_network import MLPClassifier
model = MLPClassifier(hidden_layer_sizes=(10,))

26. Python Libraries for Machine Learning

Several Python libraries make ML development efficient: NumPy and Pandas are used for
numerical computation and data handling, Matplotlib and Seaborn are used for data
visualization, Scikit-learn provides ready-made implementations of most classical ML
algorithms, and TensorFlow and PyTorch are used for building deep learning models.
Example:
import pandas as pd
df = pd.read_csv("[Link]")

27. Applications of Machine Learning

Machine Learning is used across numerous real-world domains, including email spam
filtering, recommendation systems on platforms like Netflix and Amazon, medical diagnosis,
fraud detection in banking, self-driving cars, voice assistants, and image and speech
recognition.

Conclusion
Thus, we have studied the introduction to Machine Learning, including its types, key
terminology, data preprocessing techniques, common algorithms for classification,
regression, and clustering, model evaluation methods, and an overview of neural networks
and its real-world applications.