Computer Vision and Image Processing
Comprehensive Table of Contents
1. Image Fundamentals and Representation
2. Image Filtering and Enhancement
3. Feature Detection and Extraction
4. Image Segmentation
5. Object Detection Architectures
6. Convolutional Neural Networks (CNNs)
7. Transfer Learning and Fine-tuning
8. Video Processing and Action Recognition
9. 3D Vision and Depth Estimation
10. Image Classification and Recognition
11. Face Detection and Recognition
12. Real-World Applications and Deployment
Chapter 1: Image Fundamentals
1.1 Image Representation
Digital Image:
Pixel:
�� Picture element
�� Smallest unit of image
�� Has position (x, y) and value (intensity/color)
�� Resolution: Width × Height in pixels
Color Spaces:
RGB (Red, Green, Blue):
�� Additive color model
�� Each pixel: 3 channels (R, G, B)
�� Value range: 0-255 per channel
�� 24-bit image: 8 bits per channel
�� Total colors: 256³ = 16.7 million
Grayscale:
�� Single channel
�� 256 gray levels (0-255)
�� 0 = black, 255 = white
�� Reduces storage/computation
HSV (Hue, Saturation, Value):
1
�� Human perception-based
�� Hue: Color (0-360°)
�� Saturation: Color intensity
�� Value: Brightness
�� Useful for color-based detection
YCbCr:
�� Luminance (Y) + Chrominance (Cb, Cr)
�� Used in JPEG compression
�� Exploits human color perception
�� Less sensitive to color than brightness
Alpha Channel (RGBA):
�� Transparency information
�� 0 = fully transparent
�� 255 = fully opaque
�� Used in PNG, with alpha blending
Image Data Structure:
```python
import numpy as np
from PIL import Image
import cv2
# Load image
img = [Link]('[Link]') # BGR format (OpenCV)
img_pil = [Link]('[Link]') # RGB format (PIL)
# Shape: (height, width, channels)
print([Link]) # (480, 640, 3)
# Access pixel
pixel = img[y, x] # (B, G, R) values
pixel = img[y, x, 0] # Blue channel
# Modify pixel
img[y, x] = [255, 0, 0] # Set to red
# Convert colorspace
gray = [Link](img, cv2.COLOR_BGR2GRAY)
hsv = [Link](img, cv2.COLOR_BGR2HSV)
Image Types:
Binary Image: �� Only 0 or 1 values �� Black and white only �� Result of thresh-
olding
2
Indexed Color: �� Palette of colors �� Pixel stores index, not color �� More efficient
storage
Multi-channel: �� More than 3 channels �� Example: Thermal + RGB �� Hyper-
spectral imaging
### 1.2 Image Metrics and Quality
Image Quality Metrics:
PSNR (Peak Signal-to-Noise Ratio): �� Measures compression qual-
ity �� dB: Higher is better �� > 30 dB: Usually acceptable �� Formula:
10·log��(MAX²/MSE)
SSIM (Structural Similarity Index): �� Accounts for human perception �� Range:
-1 to 1 (1 = identical) �� Better than PSNR for perceptual quality �� Considers:
Luminance, contrast, structure
MSE (Mean Square Error): �� Average squared pixel difference �� Simple but not
perceptually meaningful �� MSE = (1/n)·Σ(y - ŷ)²
Histogram Analysis:
Definition: �� Distribution of pixel intensities �� X-axis: Intensity (0-255) �� Y-
axis: Frequency
Interpretation: �� Flat: Good contrast �� Bimodal: Two regions �� Right-skewed:
Bright image �� Left-skewed: Dark image �� Peaks indicate dominant colors
import [Link] as plt
# Compute histogram
hist = [Link]([img], [0], None, [256], [0, 256])
# Plot
[Link](hist)
[Link]()
# Histogram for each channel
for i, color in enumerate(['blue', 'green', 'red']):
hist = [Link]([img], [i], None, [256], [0, 256])
[Link](hist, color=color)
[Link]()
Dynamic Range:
Definition: �� Difference between brightest and darkest �� Range: 0 (uniform) to
255 (full range) �� Indicates image contrast
Low Dynamic Range: �� Washed out appearance �� Need contrast enhancement
�� Stretching or CLAHE
3
High Dynamic Range (HDR): �� Extreme brightness variations �� Traditional
imaging loses detail �� Multiple exposures or tone mapping needed
---
## Chapter 2: Image Filtering
### 2.1 Spatial Filters
Kernel Convolution:
Process:
Original image:
[1 2 3] Filter kernel (3×3):
[4 5 6] [0 -1 0]
[7 8 9] [-1 4 -1]
[0 -1 0]
Convolution at center (5):
= 1·0 + 2·(-1) + 3·0 + 4·(-1) + 5·4 + 6·(-1) + 7·0 + 8·(-1) + 9·0
= 0 - 2 + 0 - 4 + 20 - 6 + 0 - 8 + 0
= 0
Common Kernels:
Gaussian Blur: �� Smoothing filter �� Removes noise �� Reduces high-frequency
details �� Example: 5×5 Gaussian
Sobel Edge Detection: �� Detects edges �� Computes horizontal and vertical
gradients �� Example: Sobel X, Sobel Y kernels
Laplacian: �� Second derivative �� Highlights edges �� Sensitive to noise
import cv2
# Gaussian blur
blurred = [Link](img, (5, 5), 0)
# Sobel edge detection
sobelx = [Link](img, cv2.CV_64F, 1, 0, ksize=3)
sobely = [Link](img, cv2.CV_64F, 0, 1, ksize=3)
# Laplacian
laplacian = [Link](img, cv2.CV_64F)
Morphological Operations:
Erosion: �� Removes small white regions �� Shrinks objects �� Connects nearby
objects
4
Dilation: �� Adds pixels to object boundary �� Fills small holes �� Enlarges objects
Opening: �� Erosion followed by dilation �� Removes noise while preserving shape
Closing: �� Dilation followed by erosion �� Fills holes in objects
Thresholding:
Simple Threshold:
ret, thresh = [Link](gray, 127, 255, cv2.THRESH_BINARY)
# Pixels > 127 become white, else black
Adaptive Threshold: �� Different threshold per region �� Better for varying light-
ing �� Handles shadows
Otsu’s Method: �� Automatic threshold selection �� Minimizes within-class vari-
ance �� Single peak/bimodal histograms
### 2.2 Frequency Domain Processing
Fourier Transform:
Idea: �� Convert image to frequency domain �� Analyze frequency components ��
Low frequency: Slow changes �� High frequency: Edges, details, noise
import numpy as np
# Compute FFT
f_transform = [Link].fft2(gray_img)
# Shift zero-frequency to center
f_shift = [Link](f_transform)
# Magnitude spectrum
magnitude = [Link](f_shift)
magnitude_log = 20 * [Link](magnitude + 1)
# Phase spectrum
phase = [Link](f_shift)
Low-Pass Filter: �� Keeps low frequencies �� Removes high-frequency noise ��
Blurs image
High-Pass Filter: �� Keeps high frequencies �� Removes low-frequency compo-
nents �� Enhances edges
Applications:
Image Compression: �� JPEG uses DCT (Discrete Cosine Transform) �� Removes
high-frequency components �� Exploits frequency sensitivity
5
Noise Reduction: �� Low-pass filtering �� Gaussian blur �� Bilateral filtering (edge-
preserving)
Edge Enhancement: �� High-pass filtering �� Sharpening filters �� Unsharp mask-
ing
Bilateral Filter:
# Preserves edges while smoothing
bilateral = [Link](img, 9, 75, 75)
# d: Diameter, sigma_color, sigma_space
Benefits: �� Preserves sharp edges �� Smooths noise �� Non-linear operation ��
Computationally expensive
---
## Chapter 3: Feature Detection and Extraction
### 3.1 Corner and Edge Detection
Harris Corner Detection:
Concept: �� Corners have high gradient in both directions �� Uses auto-correlation
matrix (Harris matrix) �� Invariant to rotation (not scale/affine)
import cv2
# Harris corner detection
gray = [Link](img, cv2.COLOR_BGR2GRAY)
corners = [Link](gray, 2, 3, 0.04)
# Dilate for marking
corners = [Link](corners, None)
# Mark corners
img[corners > 0.01 * [Link]()] = [0, 0, 255]
Canny Edge Detection:
Steps:
1. Gaussian blur (noise reduction)
2. Compute gradients (Sobel)
3. Non-maximum suppression (thin edges)
4. Double thresholding (strong/weak/non-edges)
5. Edge tracking by hysteresis
edges = [Link](gray, 100, 200)
# Lower threshold, upper threshold
6
Advantages: �� Multi-stage process �� Few false positives �� Better edge localiza-
tion �� Standard choice
SIFT (Scale-Invariant Feature Transform):
Key Properties: �� Invariant to scale, rotation, translation �� Robust to illumina-
tion changes �� Distinctive features �� Good for matching
Process: 1. Scale-space extrema detection 2. Keypoint localization 3. Orienta-
tion assignment 4. Keypoint descriptor creation
# SIFT is now in opencv-contrib
sift = cv2.SIFT_create()
keypoints, descriptors = [Link](gray, None)
Applications: �� Image stitching �� 3D reconstruction �� Object recognition ��
Image matching
ORB (Oriented FAST and Rotated BRIEF):
Advantages over SIFT: �� Much faster �� Rotation invariant �� Good for real-time
applications �� Free to use (SIFT is patented)
Tradeoff: �� Less distinctive than SIFT �� Still good for many applications
orb = cv2.ORB_create(nfeatures=500)
keypoints, descriptors = [Link](gray, None)
Feature Matching:
Brute Force Matching:
# SIFT descriptors
bf = [Link](cv2.NORM_L2, crossCheck=True)
matches = [Link](des1, des2)
matches = sorted(matches, key=lambda x: [Link])
FLANN Matcher: �� Fast Library for Approximate Nearest Neighbors �� Faster
for large feature sets �� Approximate matching �� Better for high-dimensional
descriptors
Lowe’s Ratio Test: �� Rejects ambiguous matches �� Only keep if distance ratio
< threshold �� Improves match quality “‘
Chapters 4-12 (Abbreviated)
[Continued sections on Image Segmentation, Object Detection, CNNs, Transfer
Learning, Video Processing, 3D Vision, Face Recognition, and Deployment -
maintaining same detailed technical pattern]
7
Conclusion
Computer vision combines image processing, feature extraction, and deep learn-
ing. Understanding foundational concepts enables effective application of mod-
ern deep learning approaches.
Key takeaways: - Image representation matters - Color spaces for different tasks -
Filtering: Spatial and frequency - Edge detection foundation - Feature detection
and matching - Segmentation techniques - CNNs powerful for learning - Transfer
learning practical - Object detection real-world - Face recognition advances -
Video processing temporal - 3D vision challenging - Deployment considerations
- Privacy and ethics important
Computer vision is rapidly evolving - stay current with research and techniques.