AI for Computer Vision - Detailed Notes
(RGPV 7th Sem)
Unit I: Introduction to Image Formation and Processing
Detailed notes will be added here...
Unit II: Feature Detection, Matching and Segmentation
Detailed notes will be added here...
Unit III: Feature-based Alignment & Motion Estimation (2D/3D)
Detailed notes will be added here...
Unit IV: 3D Reconstruction Techniques
Detailed notes will be added here...
Unit V: Image-based Rendering and Recognition
Detailed notes will be added here...
✅ **Unit I: Introduction to Image Formation and Processing**
--- THEORY NOTES ---
• Image Formation Basics: An image is a 2D projection of a 3D scene captured by a camera
using lens and light.
• Camera Model: Describes how light rays map from the world to the image plane.
- Pinhole Camera Model: Simplest geometry-based projection model.
- Perspective Projection: Objects farther away appear smaller.
• Radiometry & Photometry: Measurement of light energy affecting pixel intensity values.
• Image Types: Binary, Grayscale, Color, RGB, HSV.
--- IMAGE PROCESSING PIPELINE ---
Image Acquisition ➝ Preprocessing ➝ Feature Extraction ➝ Analysis ➝
Recognition/Interpretation
--- IMAGE FILTERING TECHNIQUES ---
1 Spatial Domain Filters: Operate directly on pixels
1️⃣
• Smoothing Filters → Noise Reduction (Mean, Gaussian)
• Sharpening Filters → Highlight Edges (Laplacian, High-Boost)
2️⃣Frequency Domain Processing: Apply Fourier Transform
• Low-Pass Filters → Remove noise
• High-Pass Filters → Detect edges
--- EDGE DETECTION ---
• Purpose: Detect boundaries of objects
• Operators: Sobel, Prewitt, Roberts, Canny
• Canny Edge Detector Steps: Smoothing → Gradient → Non-max suppression → Hysteresis
thresholding
--- IMAGE TRANSFORMS ---
• Fourier Transform → represents image in frequency domain
• Wavelets → Multi-resolution image analysis
--- PYRAMIDS ---
• Gaussian Pyramid: Repeated smoothing + downsampling
• Laplacian Pyramid: Edge information pyramid
--- OPTIMIZATION IN VISION ---
Used in feature matching, energy minimization, segmentation etc.
--- SHORT EXAM QUESTIONS ---
Q1: Define Pinhole Camera Model.
Q2: Differentiate between spatial and frequency domain filters.
Q3: Write steps of Canny edge detection.
--- LONG EXAM QUESTIONS ---
Q1: Explain image formation process with camera model and geometry.
Q2: Explain image filtering with suitable examples and diagrams.
✅ **Unit II: Feature Detection, Matching & Segmentation**
--- THEORY NOTES ---
⭐ IMP: Feature Detection identifies key points in an image that are invariant to changes in
scale, rotation or lighting.
Common detectors: Harris Corner, SIFT, SURF, FAST, ORB
➡ Harris Corner Detector
• Based on detecting corners where there is a large change in all directions
• Uses auto-correlation matrix
➡ SIFT (Scale Invariant Feature Transform) ⭐ IMP
• Detects scale and rotation invariant features
• Steps: Scale-space → Keypoint localization → Orientation assignment → Descriptor
generation
➡ SURF (Speeded-Up Robust Features)
• Faster than SIFT using box filters and integral images
➡ FAST & ORB
• FAST is extremely fast corner detector
• ORB combines FAST + BRIEF descriptors for real-time apps (e.g., robotics)
--- FEATURE MATCHING ---
⭐ IMP: Used to find correspondences between images
Methods: SSD (Sum of Squared Differences), NCC (Normalized Cross Correlation), Hamming
distance for ORB
➡ RANSAC (Random Sample Consensus) ⭐ Most Asked
• Removes false matches by estimating the best model through random sampling
--- IMAGE SEGMENTATION ---
⭐ IMP: Process of dividing image into meaningful regions
➡ Thresholding-based Segmentation
• Otsu Method: Finds optimal threshold by maximizing variance between classes
➡ Region-based Segmentation
• Region growing & splitting, merging
➡ Clustering-based Segmentation ⭐ IMP
• K-Means clustering: groups pixels based on similarity
➡ Graph-based Segmentation ⭐ IMP
• Graph Cut: minimizes cut cost to separate foreground & background
➡ Watershed Segmentation ⭐ Important for diagrams
• Visualizes gradient of image as a topographic surface
--- SHORT EXAM QUESTIONS ---
Q1: Define feature detection (IMP)
Q2: Difference between SIFT and SURF
Q3: What is RANSAC? Why used? ⭐
--- LONG EXAM QUESTIONS (Repeated in RGPV) ---
Q1: Explain SIFT algorithm with steps ⭐⭐
Q2: Explain image segmentation techniques with examples ⭐⭐
✅ **Unit III: Feature-based Alignment & Motion Estimation (2D/3D)**
--- THEORY NOTES ---
➡ ⭐ IMP: Pose Estimation
• Determines camera location + orientation relative to the object
• Uses feature correspondences and geometric constraints
➡ Triangulation
• Uses 2D projections from multiple views to recover 3D points
➡ ⭐ IMP: Structure from Motion (SfM)
• Recovers 3D scene + camera motion from multiple images
• Used in 3D mapping, AR, drones
➡ ⭐ Most Asked: Optical Flow
• Motion estimation by pixel intensity changes between frames
• Assumption: intensity constant over motion
Popular Methods:
• Lucas-Kanade → Sparse estimation
• Horn-Schunck → Dense estimation
➡ Bundle Adjustment ⭐⭐ Highly Asked in Exams
• Optimization technique in SfM to minimize reprojection error
➡ ⭐ Most Repeated: Camera Calibration
• Estimation of intrinsic + extrinsic parameters of camera
➡ Layered Motion Estimation
• Separates motion into different layers for better tracking
--- ✅ OpenCV Code Examples (IMP for Viva) ---
➡ Lucas-Kanade Optical Flow
import cv2
cap = [Link](0)
lk_params = dict(winSize=(15, 15))
while True:
ret, frame = [Link]()
gray = [Link](frame, cv2.COLOR_BGR2GRAY)
[Link]("Optical Flow", gray)
if [Link](1) & 0xFF == ord('q'):
break
[Link]()
[Link]()
➡ Camera Calibration (Pseudo Example)
import cv2
import numpy as np
# Termination criteria
criteria = (cv2.TERM_CRITERIA_EPS + cv2.TERM_CRITERIA_MAX_ITER, 30, 0.001)
objp = [Link]((6*7, 3), np.float32)
objp[:, :2] = [Link][0:7, 0:6].[Link](-1, 2)
# Used when clicking chessboard patterns
--- SHORT EXAM QUESTIONS ---
Q1: What is optical flow? ⭐
Q2: Define structure from motion (SfM).
Q3: What is camera calibration? ⭐
--- LONG EXAM QUESTIONS (Repeated in RGPV) ---
Q1: Explain optical flow with its different methods ⭐⭐
Q2: Explain Structure from Motion with example diagrams ⭐⭐