ADVANCE PROGRAMMING

LECTURE NO.11
Lecture Content
Machine Learning Libraries:
 Scikit-learn
 XGBoost
 LightGBM
Deep Learning Frameworks:
 TensorFlow
 Keras
 PyTorch

What is Scikit-Learn?
Scikit-Learn (usually written as sklearn) is one of the most popular machine learning
libraries in Python.
It is built on top of NumPy, Pandas, and Matplotlib and provides ready-made tools to:

 Train machine learning models

 Evaluate models
 Preprocess data
 Perform classification
 Perform regression
 Do clustering
 Apply dimensionality reduction

Basically, sklearn gives you everything needed to build ML models quickly without
writing algorithms from scratch.

How to Install and Import Scikit-Learn

Installation

If sklearn is not installed, run:

pip install scikit-learn

Importing

Common imports used in almost every ML project:

import numpy as np
import pandas as pd
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression

You import specific modules depending on the algorithm you need.

Basic Workflow of Using Scikit-Learn

Every ML algorithm in sklearn follows the same 4-step pipeline:

Step 1: Load or create the data

X = [[1], [2], [3], [4], [5]] # inputs
y = [2, 4, 6, 8, 10] # outputs

Step 2: Create the model

model = LinearRegression()

Step 3: Train the model (fit)

[Link](X, y)

Step 4: Predict
prediction = [Link]([[6]])

Simple Scikit-Learn Examples

Example 1: Linear Regression (Supervised Learning)
Goal: Predict a continuous value.

from sklearn.linear_model import LinearRegression

X = [[1], [2], [3], [4]]

y = [3, 6, 9, 12]

model = LinearRegression()
[Link](X, y)

print([Link]([[5]]))

Explanation

 We create simple input–output pairs.

 The model learns a straight-line relation.
 Predicting with input 5 gives something around 15.

Example 2: Train-Test Split

This is used to split data into:

 Train data → used to teach model

 Test data → used to evaluate model
from sklearn.model_selection import train_test_split

X = [[1], [2], [3], [4], [5], [6]]

y = [2, 4, 6, 8, 10, 12]

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.33)

print("Train:", X_train)
print("Test:", X_test)

Explanation

The function automatically creates training and testing datasets.

Example 3: Classification with KNN

from [Link] import KNeighborsClassifier

X = [[1], [2], [3], [10], [11], [12]]

y = ["small", "small", "small", "large", "large", "large"]

model = KNeighborsClassifier(n_neighbors=3)
[Link](X, y)

print([Link]([[4]]))

Explanation

 KNN looks at nearest neighbors.

 Input 4 is close to small numbers → predicts "small".

Example 4: Data Preprocessing (Normalization)

Scaling values to a small range.

from [Link] import MinMaxScaler

import numpy as np

data = [Link]([[10],[20],[30],[40]])
scaler = MinMaxScaler()

scaled = scaler.fit_transform(data)
print(scaled)

Explanation
Values are converted between 0 and 1.
Essential for ML algorithms.

Example 5: Label Encoding

Convert text labels to numbers.

from [Link] import LabelEncoder

le = LabelEncoder()

labels = ["cat", "dog", "cat", "bird"]

encoded = le.fit_transform(labels)

print(encoded)

Explanation

 Text labels → numeric labels

 Required for many ML models

Example 6: Decision Tree Classifier

from [Link] import DecisionTreeClassifier

X = [[1], [2], [3], [10], [11], [12]]

y = ["low", "low", "low", "high", "high", "high"]

model = DecisionTreeClassifier()
[Link](X, y)

print([Link]([[4]]))

Important Modules in Scikit-Learn

Category Module Purpose
Splitting data, finding best
Model Selection train_test_split, GridSearchCV
parameters
StandardScaler, MinMaxScaler, Cleaning and preparing
Preprocessing
LabelEncoder data
Regression Predicting continuous
LinearRegression, Lasso, Ridge
Algorithms values
 Category Module Purpose
Classification SVM, KNN, Naïve Bayes, Decision
Predicting categories
Algorithms Trees
Clustering KMeans, DBSCAN Grouping similar data
Dimensionality Reducing number of
PCA
Reduction features
accuracy_score,
Metrics Measuring performance
mean_squared_error

Evaluating Machine Learning Models

Accuracy (Classification)
from [Link] import accuracy_score
accuracy_score(y_test, predictions)

Mean Squared Error (Regression)

from [Link] import mean_squared_error
mean_squared_error(y_test, predictions)

Mini End-to-End Example

import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression
from [Link] import mean_squared_error

X = [Link]([[1],[2],[3],[4],[5],[6]])
y = [Link]([2,4,6,8,10,12])

# split
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2)

# train
model = LinearRegression()
[Link](X_train, y_train)

# predict
pred = [Link](X_test)

# evaluate
print("Predictions:", pred)
print("MSE:", mean_squared_error(y_test, pred))
Explanation
Step 1 — Prepare Data

We have a simple dataset based on doubling the input.

Step 2 — Train-Test Split

We use 80% data for training and 20% for testing.

Step 3 — Train model

Linear Regression learns the relationship.

Step 4 — Predict

The model predicts outputs for test inputs.

Step 5 — Evaluate

We calculate MSE (Mean Squared Error):

 If MSE is near 0, model is very accurate.

 Higher MSE means more error.

What is XGBoost?
XGBoost stands for Extreme Gradient Boosting.

It is one of the fastest, most accurate, and most powerful machine learning
libraries used in:

 Kaggle competitions
 Real-world production systems
 Large datasets
 High-performance tasks

XGBoost works by building many small decision trees and combining them to make a
strong and accurate model.

Why XGBoost is Popular?

Extremely fast
Very high accuracy
Works well with large datasets
Handles missing values automatically
Supports parallel processing
Avoids overfitting

Installation
pip install xgboost

Importing XGBoost
import xgboost as xgb

For classification:

from xgboost import XGBClassifier

For regression:

from xgboost import XGBRegressor

 Basic Workflow of XGBoost
1. Prepare data
2. Create a model
3. Train (fit)
4. Predict
5. Evaluate

Same as sklearn, but much faster and more powerful.

Example 1: XGBoost Regression

(Predicting a Number)
Code
from xgboost import XGBRegressor

X = [[1], [2], [3], [4], [5]]

y = [2, 4, 6, 8, 10]

model = XGBRegressor()
[Link](X, y)

print([Link]([[6]]))

Explanation

 We give inputs X and outputs y.

 XGBoost builds many decision trees.
 Each new tree corrects the mistakes of the previous one.
 Eventually the model understands the pattern (multiply by 2).
 Prediction for 6 → about 12.

This works even if the pattern is not a straight line.

Example 2: XGBoost Classification

Code
from xgboost import XGBClassifier

X = [[1], [2], [3], [10], [11], [12]]

y = [0, 0, 0, 1, 1, 1]

model = XGBClassifier()
[Link](X, y)

print([Link]([[4]]))

Explanation

Data:

 Small numbers = class 0

 Big numbers = class 1

XGBoost makes many small decision trees to learn this.

When predicting 4:

 Closest to small numbers → predicts 0

Example 3: Using Train-Test Split +

Accuracy Score
Code
import numpy as np
from sklearn.model_selection import train_test_split
from xgboost import XGBClassifier
from [Link] import accuracy_score

X = [Link]([[1],[2],[3],[4],[10],[11],[12],[13]])
y = [Link]([0,0,0,0,1,1,1,1])

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

model = XGBClassifier()
[Link](X_train, y_train)

pred = [Link](X_test)

print("Accuracy:", accuracy_score(y_test, pred))

Explanation

 The data is split into training/testing parts.

 The model is trained on training data.
 Predictions are compared with true labels.
 Accuracy tells how correct the model is.

Example 4: XGBoost with Missing

Values
Code
import numpy as np
from xgboost import XGBRegressor

X = [Link]([[1],[2],[None],[4],[5]])
y = [Link]([2,4,6,8,10])

model = XGBRegressor()
[Link](X, y)
print([Link]([[3]]))

Explanation

 XGBoost supports None/NaN values.

 It automatically learns how to handle missing inputs.
 No need for manual imputation.

This makes it better than most sklearn models.

Example 5: Setting Hyperparameters

Code
model = XGBClassifier(
n_estimators=100, # number of trees
max_depth=3, # depth of each tree
learning_rate=0.1, # how fast model learns
subsample=0.8 # random sampling
)
Explanation of important parameters

Parameter Meaning Use

n_estimators How many trees to build More trees = better accuracy but slow

max_depth Depth of each tree Higher depth = more powerful but may overfit

learning_rate Controls how fast model learns Combines all trees smoothly

subsample Use % of data per tree Prevents overfitting

XGBoost Evaluation Metrics

For Classification:
from [Link] import accuracy_score

For Regression:
from [Link] import mean_squared_error

When to use XGBoost?

Use XGBoost when:

Dataset is large
You need faster training
You want best accuracy
You want automatic handling of missing data
You want high-performance machine learning
What is LightGBM?
LightGBM (from Microsoft) stands for:

Light Gradient Boosting Machine

It is a very fast and efficient machine learning library that uses decision-tree–based
learning.

LightGBM is mainly used for:

 Classification
 Regression
 Ranking
 Handling very large datasets
 High-speed training

Why is LightGBM Popular?

LightGBM is extremely powerful because:

Faster than XGBoost

High accuracy
Uses less memory
Handles large datasets easily
Supports GPU training
Works well with features that have many categories
Can handle missing values automatically

LightGBM uses a special tree-growing strategy called:

Leaf-wise growth (instead of level-wise used by XGBoost)

This makes it faster but requires tuning to avoid overfitting.

Installation
pip install lightgbm
If you want GPU support:

pip install lightgbm --install-option=--gpu

Importing LightGBM
from lightgbm import LGBMClassifier, LGBMRegressor

Basic Workflow
LightGBM follows the same steps as sklearn:

1. Prepare data
2. Create model
3. Train the model
4. Predict
5. Evaluate

Example 1: LightGBM Regression

(Predicting Numbers)
Code
from lightgbm import LGBMRegressor

X = [[1],[2],[3],[4],[5]]
y = [3,6,9,12,15]

model = LGBMRegressor()
[Link](X, y)

print([Link]([[6]]))

Explanation

 The input values (X) and output values (y) show a simple multiplication pattern.
 LightGBM creates many “leaf-wise” boosted decision trees.
 It learns the mathematical relationship.
 When we give input 6, it predicts close to 18.
LightGBM does not need scaling or normalization — it handles raw data very efficiently.

Example 2: LightGBM Classification

Code
from lightgbm import LGBMClassifier

X = [[1], [2], [3], [10], [11], [12]]

y = [0,0,0,1,1,1]

model = LGBMClassifier()
[Link](X, y)

print([Link]([[4]]))

Explanation

 Small values (1–3) → class 0

 Large values (10–12) → class 1
 LightGBM builds boosted trees to learn the pattern.

Prediction:

 Value = 4 → closer to small values

 Output → 0

LightGBM handles this classification even with very large datasets.

Example 3: LightGBM + Train-Test Split

+ Accuracy
Code
import numpy as np
from sklearn.model_selection import train_test_split
from lightgbm import LGBMClassifier
from [Link] import accuracy_score

X = [Link]([[1],[2],[3],[4],[10],[11],[12],[13]])
y = [Link]([0,0,0,0,1,1,1,1])

X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.25)

model = LGBMClassifier()
[Link](X_train, y_train)

pred = [Link](X_test)

print("Accuracy:", accuracy_score(y_test, pred))

Explanation

 Data is split into training/testing.

 The LightGBM model learns the classification pattern.
 Accuracy score checks how many predictions were correct.
 LightGBM usually gives high accuracy because of strong boosting.

Example 4: LightGBM with Missing

Values
Code
import numpy as np
from lightgbm import LGBMRegressor

X = [Link]([[1],[2],[None],[4],[5]])
y = [Link]([3,6,9,12,15])

model = LGBMRegressor()
[Link](X, y)

print([Link]([[3]]))

Explanation

 LightGBM automatically handles missing values.

 It learns how to route missing values in trees.
 No need to do fillna() or imputation.

This makes LightGBM extremely convenient for real-world datasets.

 Example 5: Hyperparameters (Very
Important)
Code
model = LGBMClassifier(
n_estimators=200,
max_depth=4,
learning_rate=0.05,
num_leaves=20,
subsample=0.7,
colsample_bytree=0.8
)

Explanation of parameters

Parameter Meaning Why it's important

n_estimators Number of boosted trees More trees = more accuracy but slower

max_depth Depth of each tree Avoid overfitting

num_leaves Number of leaves Too large → overfitting

learning_rate Step size Controls how fast the model learns

subsample % of data per tree Prevents overfitting

colsample_bytree % of features per tree Adds randomness → better generalization

LightGBM works best when num_leaves < 2^max_depth.

Evaluation Metrics
For Classification:
from [Link] import accuracy_score

For Regression:
from [Link] import mean_squared_error

When to Use LightGBM?

Use LightGBM when:

Dataset is huge
You want very fast training
You have high-dimensional data
You want high accuracy
You want GPU training
You want automatic missing-value handling

LightGBM is especially strong in Kaggle competitions.

Deep Learning Framework

What is TensorFlow?
TensorFlow is an open-source deep learning and machine learning framework
created by Google Brain Team.
It is mainly used for:

 Building neural networks

 Training deep learning models
 Running models on CPU, GPU, and TPU
 Handling huge datasets
 Production-grade model deployment

In simple words:
TensorFlow helps you create and train models that can learn from data — like
recognition, prediction, classification, or generating outputs.

Why TensorFlow? (Key Advantages)

Easy Model Building

Provides high-level APIs like Keras, which makes designing neural networks simple
and clean.

Highly Scalable

Supports CPU, GPU, multiple GPUs, distributed training, and cloud environments.

Fast Execution with Graphs

TensorFlow uses computational graphs to optimize execution and increase speed.

Production Ready

Can be deployed on:

 Mobile (TensorFlow Lite)

 Web browsers ([Link])
 Servers and cloud

Huge Community & Resources

Lots of tutorials, pre-trained models, and documentation.

Installing TensorFlow
For CPU version (lighter):
pip install tensorflow

For GPU version (requires CUDA & cuDNN):

pip install tensorflow-gpu

You can check installation:

import tensorflow as tf
print(tf.__version__)

Importing TensorFlow
Basic import:
import tensorflow as tf

Importing Keras (High-level API):

from tensorflow import keras

Importing Layers:
from [Link] import layers

TensorFlow Basic Concepts

(A) Tensors

A tensor is just a multi-dimensional array.

Example shapes:

 0D tensor → scalar
 1D tensor → vector
  2D tensor → matrix
 3D, 4D → for images, batches, videos

Example 1 — Creating Tensors

Code
import tensorflow as tf

# Creating constants
a = [Link](5)
b = [Link]([1, 2, 3])
c = [Link]([[1, 2], [3, 4]])

print(a)
print(b)
print(c)

Explanation

 creates a fixed tensor.

[Link]()
 TensorFlow automatically gives:
o shape
o datatype
o value

Useful for building mathematical operations inside models.

Example 2 — Basic Tensor Operations

Code
import tensorflow as tf

x = [Link]([10, 20, 30])

y = [Link]([1, 2, 3])

add = [Link](x, y)
mul = [Link](x, y)

print("Addition:", add)
print("Multiplication:", mul)
Explanation

 TensorFlow performs element-wise math.

 [Link]() → adds each element
 [Link]() → multiplies each element
 Automatically uses GPU/CPU for fast execution.

Example 3 — Building a Simple Neural

Network (Keras)
We build a model for simple classification.

Code
import tensorflow as tf
from [Link] import layers, Sequential

model = Sequential([
[Link](16, activation="relu", input_shape=(4,)),
[Link](8, activation="relu"),
[Link](3, activation="softmax")
])

[Link]()

Explanation

 Sequential() builds a model layer by layer.

 Dense layer → fully connected neural layer.
 Activation functions:
o ReLU: for hidden layers
o Softmax: for multi-class output
 input_shape=(4,) means input has 4 features.

[Link]() shows:

 Layers
 Parameters
 Output shapes
Example 4 — Training a Neural Network
on Dummy Data
Code
import tensorflow as tf
from [Link] import layers, Sequential
import numpy as np

# Dummy data
X = [Link](100, 4)
y = [Link](0, 3, 100)

# Build model
model = Sequential([
[Link](16, activation="relu", input_shape=(4,)),
[Link](8, activation="relu"),
[Link](3, activation="softmax")
])

# Compile model
[Link](optimizer="adam",
loss="sparse_categorical_crossentropy",
metrics=["accuracy"])

# Train model
[Link](X, y, epochs=10, batch_size=16)

Explanation

 optimizer="adam": popular optimization algorithm.

 loss="sparse_categorical_crossentropy": for integer labels.
 Training runs through 10 cycles (epochs).
 batch_size = 16 → model trains 16 samples at a time.

Example 5 — Making Predictions

Code
new_data = [Link](1, 4)
prediction = [Link](new_data)
print("Predicted probabilities:", prediction)
print("Predicted class:", [Link](prediction, axis=1).numpy())

Explanation
  [Link]() gives probability of each class.
 argmax() returns the class with the highest probability.

Example 6 — Loading a Pretrained

Model (MobileNetV2)
Code
from [Link] import MobileNetV2

model = MobileNetV2(weights="imagenet")
[Link]()

Explanation

 Loads Google’s pretrained model.

 Already trained on 1.4M images.
 Useful for:
o Image classification
o Transfer learning

Example 7 — Saving & Loading Models

Save model
[Link]("mymodel.h5")

Load model
loaded_model = [Link].load_model("mymodel.h5")

Explanation

 .h5 format keeps:

o architecture
o weights
o optimizer state

Allows easy reuse and deployment.

Where TensorFlow is Used?
Popular Applications

 Image recognition
 Speech recognition
 NLP (ChatGPT-style models)
 Fraud detection
 Medical predictions
 Autonomous vehicles
 Recommendation systems (Netflix, YouTube)

Key TensorFlow Modules

Module Purpose

[Link] High-level neural network API

[Link] Handling datasets efficiently

[Link] Training utilities

[Link] Image preprocessing

[Link] Mobile deployment

[Link] Distributed training

[Link] Mathematical operations

TensorFlow vs PyTorch (Short

comparison)
Feature TensorFlow PyTorch

Best For Production Research

API Style Graph + Eager Pythonic, dynamic

Deployment TensorFlow Lite, TF Serving Less deployment options

 Feature TensorFlow PyTorch

Visualization TensorBoard Third-party tools

What is Keras?
Keras is a high-level deep learning API used to build neural networks in a simple,
readable, human-friendly way.

Originally it worked on top of TensorFlow, Theano, and CNTK — but now it is fully
integrated into TensorFlow.

Today we use it like:

from tensorflow import keras

In simple words

Keras makes it easy to design, train, evaluate, and deploy neural networks without
dealing with complicated low-level code.

Why Keras? (Main Advantages)

Beginner Friendly

Readable, clean, and simple syntax.

Fast Experimentation

Build models quickly using:

 Sequential API
 Functional API
 Model subclassing

TensorFlow Powered

Keras runs on TensorFlow, so you get:

 GPU acceleration
  TPU acceleration
 Distributed training

Large Community

Many examples, tutorials, pretrained models.

Modular Design

Everything is a module:

 layers
 losses
 metrics
 optimizers

Installing Keras
Keras comes with TensorFlow. Install TensorFlow:

pip install tensorflow

Confirm installation:

import tensorflow as tf
print(tf.__version__)

Importing Keras
Main import:
from tensorflow import keras

Import layers:
from [Link] import layers

Import Sequential API:

from [Link] import Sequential
Keras Building Blocks
Keras consists of:

Component Explanation

Layers building blocks (Dense, Conv2D, LSTM, etc.)

Models neural network architectures

Optimizers Adam, SGD, RMSprop

Loss functions MSE, cross-entropy

Metrics accuracy, MAE

Callbacks early stopping, checkpoints

Example 1 — Creating a Simple Dense

Neural Network
Code
from [Link] import Sequential
from [Link] import Dense

model = Sequential([
Dense(32, activation='relu', input_shape=(4,)),
Dense(16, activation='relu'),
Dense(3, activation='softmax')
])

[Link]()

Explanation

 We used Sequential API → layer-by-layer model building.

 input_shape=(4,) means 4 features in input.
 Two hidden layers with ReLU activation.
 Output has 3 neurons → for 3 classes.

[Link]() prints architecture.

Example 2 — Compiling the Model
Code
[Link](
optimizer='adam',
loss='sparse_categorical_crossentropy',
metrics=['accuracy']
)

Explanation

 optimizer: controls learning; Adam is most popular.

 loss: how wrong the model is.
 metrics: measures performance, e.g., accuracy.

Example 3 — Training the Model

Code
import numpy as np

X = [Link](100, 4)
y = [Link](0, 3, 100)

history = [Link](X, y, epochs=10, batch_size=16)

Explanation

 fit() trains the model.

 epochs=10 means data is repeated 10 times.
 batch_size=16 → trains on 16 samples at a time.
 history stores loss + accuracy per epoch.

Example 4 — Making Predictions

Code
test = [Link](1, 4)
pred = [Link](test)
print(pred)
print("Predicted class:", [Link](pred))

Explanation

 predict() returns probability for each class.

 argmax() returns the class with highest probability.

Example 5 — Functional API

Functional API is used for complex models like:

 multi-input
 multi-output
 branching networks

Code
from [Link] import Input, Dense
from [Link] import Model

inputs = Input(shape=(4,))
x = Dense(32, activation='relu')(inputs)
x = Dense(16, activation='relu')(x)
outputs = Dense(3, activation='softmax')(x)

model = Model(inputs, outputs)

[Link]()

Explanation

 Allows building more flexible models.

 Functions like a graph — not strictly linear.
 Good for advanced architectures.

Example 6 — Convolutional Neural

Network (CNN)
Code
from [Link] import Sequential
from [Link] import Conv2D, MaxPooling2D, Flatten, Dense

model = Sequential([
Conv2D(32, (3,3), activation='relu', input_shape=(28,28,1)),
MaxPooling2D((2,2)),
Flatten(),
Dense(64, activation='relu'),
Dense(10, activation='softmax')
])

[Link]()

Explanation

 Conv2D extracts image features.

 MaxPooling reduces size.
 Flatten converts 2D → 1D for Dense layer.
 Output layer has 10 classes (like MNIST digits).

Example 7 — Saving & Loading Models

Save:
[Link]("keras_model.h5")

Load:
from [Link] import load_model
loaded = load_model("keras_model.h5")

Explanation

 Saves architecture + weights.

 Can reload anytime to continue training or predict.

Keras Callbacks
Callbacks are automatic actions during training.

Popular callbacks:

 EarlyStopping → stop when model stops improving

  ModelCheckpoint → save best model
 TensorBoard → visualize training

Example
from [Link] import EarlyStopping

cb = EarlyStopping(patience=3, restore_best_weights=True)

[Link](X, y, epochs=50, callbacks=[cb])

Keras Popular Layers (Quick Overview)

Layer Use

Dense Classic fully connected layer

Conv2D Image feature extraction

MaxPooling Downsampling images

Flatten Change shape to 1D

Dropout Reduce overfitting

LSTM Sequence/time series

GRU Faster LSTM alternative

Embedding NLP, word vectors

Keras Use-Cases in Real Life

 Image classification
 Object detection
 Text classification
 Spam detection
 Sentiment analysis
 Speech recognition
 Forecasting and prediction
 Medical scan diagnostics
 Recommendation systems
Keras vs TensorFlow (Difference)
Feature Keras TensorFlow

Role High-level API Full ML framework

Difficulty Easy Medium/advanced

Model Building Very fast More flexible

Code Short Longer

Use case Beginners, rapid prototyping Deep customization, research

What is PyTorch?
PyTorch is an open-source machine learning and deep learning framework
developed by Facebook AI Research (FAIR).

It is extremely popular because it is:

 Pythonic (feels like regular Python)

 Very flexible
 Great for research and experimentation
 Supports dynamic computation graphs
 Powerful for building neural networks of any kind

In simple words:

PyTorch allows you to create and train neural networks with simple, readable Python
code.

Why PyTorch? (Advantages)

1. Dynamic Computation Graphs

Graph is created on the fly → easier debugging, more flexibility.

2. Pythonic
Feels like NumPy but with GPU support.

3. Strong GPU Acceleration

Easily runs on:

 CPU
 GPU (CUDA)
 Multi-GPU

4. Great for Research

Many research papers use PyTorch due to flexibility.

5. Autograd

Automatic gradient computation for training neural networks.

6. TorchVision, TorchAudio, TorchText

Built-in libraries for:

 images
 audio
 text

Installing PyTorch
Go to official website and select your OS + CUDA version.

Standard installation (CPU version):

pip install torch torchvision torchaudio

GPU version example:

pip install torch torchvision torchaudio --index-url [Link]

Importing PyTorch
import torch
import [Link] as nn
import [Link] as optim

Tensors in PyTorch (Core building block)

Tensors = multi-dimensional arrays

Like NumPy arrays but can run on GPU.

Example 1 — Creating Tensors

Code:
import torch

a = [Link](5)
b = [Link]([1, 2, 3])
c = [Link]([[1, 2], [3, 4]])

print(a)
print(b)
print(c)

Explanation:

 creates a tensor.
[Link]()
 Tensors work like NumPy arrays but support GPU acceleration.

Example 2 — Tensor Operations

Code:
import torch

x = [Link]([10, 20, 30])

y = [Link]([1, 2, 3])

print("Addition:", x + y)
print("Multiplication:", x * y)
Explanation:

 PyTorch supports normal Python operators: +, *, @, etc.

 Performs element-wise operations.

Example 3 — Move Tensor to GPU

Code:
device = "cuda" if [Link].is_available() else "cpu"

x = [Link]([1, 2, 3], device=device)

print(x)

Explanation:

 Automatically checks if GPU is available.

 Runs tensor operations on GPU → faster training.

Example 4 — Building a Simple Neural

Network ([Link])
Code:
import [Link] as nn

class SimpleNN([Link]):
def __init__(self):
super().__init__()
self.fc1 = [Link](4, 16)
self.fc2 = [Link](16, 3)

def forward(self, x):

x = [Link](self.fc1(x))
x = self.fc2(x)
return x

model = SimpleNN()
print(model)

Explanation:
  Neural networks are created by subclassing [Link].
 [Link] creates fully connected layers.
 forward() defines computation.
 [Link]() applies activation.

Example 5 — Loss Function & Optimizer

Code:
criterion = [Link]()
optimizer = [Link]([Link](), lr=0.001)

Explanation:

 CrossEntropyLoss is used for classification.

 Adam optimizer updates weights during training.

Example 6 — Training the Model

Code:
import torch
import [Link] as F

X = [Link](100, 4)
y = [Link](0, 3, (100,))

for epoch in range(10):

optimizer.zero_grad()

outputs = model(X)
loss = criterion(outputs, y)

[Link]()
[Link]()

print(f"Epoch {epoch+1}, Loss: {[Link]()}")

Explanation:

Training process steps:

1. optimizer.zero_grad() → clears previous gradients

 2. Forward pass → model(X)
3. Compute loss
4. [Link]() computes gradients
5. [Link]() updates weights

Example 7 — Making Predictions

Code:
test = [Link](1, 4)
prediction = model(test)
predicted_class = [Link](prediction)

print(prediction)
print("Class:", predicted_class.item())

Explanation:

 Get raw scores (logits)

 argmax() picks the class with highest score.

Example 8 — Using DataLoader (Batch

Loading)
Code:
from [Link] import DataLoader, TensorDataset

dataset = TensorDataset(X, y)
loader = DataLoader(dataset, batch_size=16, shuffle=True)

for batch_X, batch_y in loader:

print(batch_X.shape, batch_y.shape)
break

Explanation:

 handles batching, shuffling, iteration.

DataLoader
 Used heavily in training deep learning models.
Example 9 — Saving & Loading Models
Save:
[Link](model.state_dict(), "[Link]")

Load:
model = SimpleNN()
model.load_state_dict([Link]("[Link]"))

Explanation:

 is common PyTorch model format.

.pth
 Saves only parameters (weights).

PyTorch Ecosystem
Library Usage

torchvision Images, pretrained models

torchaudio Audio tasks

torchtext NLP tasks

lightning Training automation

fastai High-level library on top of PyTorch

PyTorch vs TensorFlow (Simple

Comparison)
Feature PyTorch TensorFlow

Graph Type Dynamic Static + Eager

Easy for Beginners Yes Medium

Flexibility High Medium

Research Very popular Also popular

 Feature PyTorch TensorFlow

Production Medium Strong

Syntax Pythonic TensorFlow-style

Where PyTorch Is Used?

 AI research labs
 Universities
 NLP research
 Computer vision
 Robotics
 Reinforcement learning
 Image generation (GANs)
 Large language models (LLMs)