COMPUTER VISION

UNIT I · I NTRODUCTION

Image Processing • Computer Vision • Computer Graphics • CV Applications

[Link] CSE (AI) — Computer Vision — Lecture Slides

Learning Objectives
By the end of this unit, you should be able to:

Differentiate Image Processing, Computer Vision, and Computer Graphics

Define computer vision and describe its core goal

Explain the low-level, mid-level, and high-level vision hierarchy

Identify major real-world application domains of computer vision

UN IT I | CO MPUT ER V IS IO N 2
Three Related — But Different — Fields

Image Processing Computer Vision Computer Graphics

Image → Image Image → Description Description → Image

Each field moves information in a different direction between the real world and its description.

UN IT I | CO MPUT ER V IS IO N 3
Image Processing

● Operates directly on pixel data — input is an image, output is also an image

● Goal: improve quality or transform appearance, not to interpret content

● Typical operations: filtering, enhancement, noise removal, compression,

restoration

UN IT I | CO MPUT ER V IS IO N 4
 Image Processing
I n Prac tic e — A Real Exam pl e

Input — noisy, rotated scan Output — denoised, deskewed, binarized

Technique: Bilateral filtering for denoising, adaptive thresholding for binarization, and an automatic skew-angle correction — the same image throughout,
no content was interpreted.

UN IT I | CO MPUT ER V IS IO N 5
Computer Vision

● The science of enabling machines to interpret and understand visual information, the way
humans do

● Input: image(s) or video → Output: a meaningful description, decision, or action

● Draws on mathematics, AI, neuroscience, physics, and signal processing

UN IT I | CO MPUT ER V IS IO N 6
 Computer Vision
I n Prac tic e — A Real Exam pl e

Input — an image Output — a meaningful description

Technique: A full vision pipeline (edge detection, segmentation, shape classification) turns pixels into labels — exactly the image-to-description
transformation that defines computer vision.

UN IT I | CO MPUT ER V IS IO N 7
Computer Graphics

● The inverse of computer vision — starts from a description or model and

synthesizes an image

● Input: scene description / 3D model → Output: a rendered image

● Core to animation, gaming, simulation, and visualization

UN IT I | CO MPUT ER V IS IO N 8
 Computer Graphics
I n Prac tic e — A Real Exam pl e

Scene Description (Input)

● 8 cube vertices in 3D space

● 12 edges connecting the vertices

● Camera pose: 25° tilt, 35° turn

● Perspective projection (focal length 620)

Rendered Output

Technique: A 3D geometric model is projected through a virtual camera onto a 2D plane and rasterized into pixels — the reverse of what computer vision
does.

UN IT I | CO MPUT ER V IS IO N 9
Vision and Graphics: Two Directions of One Problem
Computer Vision

Scene in the Description &

Real World Understanding

Computer Graphics

Description /
Rendered Image
3D Model

Image Processing (not shown) maps an image to another image — no meaning is extracted along the way.

UN IT I | CO MPUT ER V IS IO N 10
S E CT I ON 2 . 1 — S E N SI N G L I GH T

How a Surface Reflects Light Toward a Camera

point source • surface normal (N) • angle θ • radiance / irradiance • optical axis (Z)

I MAG ING & IM AGE RE PR ESE NTA T IO N 1

S E CT I ON 2 . 2 — I M A GI N G D E VI C E S

CCD Cameras: Light → Electrical Charge

lens = center of projection • CCD array of discrete cells • each cell integrates charge • circle of confusion (blur)

I MAG ING & IM AGE RE PR ESE NTA T IO N 2

S E CT I ON 2 . 2 — I M A GI N G D E VI C E S

The Frame Buffer: From Scene to Algorithm

A/D conversion • frame buffer = shared image store • read by vision algorithms • written to graphic display

I MAG ING & IM AGE RE PR ESE NTA T IO N 3

S E CT I ON 2 . 2 — I M A GI N G D E VI C E S

Sensor Array Geometries

circular — radial inspection • linear — push-broom scanning • ROSA — sector/ring integration

I MAG ING & IM AGE RE PR ESE NTA T IO N 4

S E CT I ON 2 . 2 — I M A GI N G D E VI C E S

The Human Eye as a Biological Camera

iris controls pupil size • retina: rods (b/w) + cones (color) • fovea — dense color receptors • ~100 million receptor cells/eye

I MAG ING & IM AGE RE PR ESE NTA T IO N 5

LEVELS OF VISUAL PROCESSING
Low-Level → Mid-Level → High-Level
Low-Level Vision

● Works directly on raw pixel values, close to the sensor data

● Operations: noise removal, edge detection, contrast enhancement, filtering

● Output: another image or a simple pixel-wise map — no semantic meaning

yet

UN IT I | CO MPUT ER V IS IO N 12
 Low-Level Vision
I n Prac tic e — A Real Exam pl e

Input — raw pixels Output — an edge map

Technique: Gaussian smoothing removes sensor noise, then a Canny edge detector finds intensity discontinuities — purely local, pixel-level operations
with no notion of ‘shapes’ yet.

UN IT I | CO MPUT ER V IS IO N 13
Mid-Level Vision

● Groups pixels into meaningful structures: regions, contours, and features

● Operations: segmentation, feature extraction, depth and motion estimation

● Output: structured descriptions — boundaries, regions, feature points — still unlabeled

UN IT I | CO MPUT ER V IS IO N 14
 Mid-Level Vision
I n Prac tic e — A Real Exam pl e

Input — raw pixels Output — segmented regions

Technique: Contours are traced around each connected region, grouping low-level pixels into coherent boundaries — structured, but the regions still carry
no semantic label.

UN IT I | CO MPUT ER V IS IO N 15
High-Level Vision

● Interprets structured features to assign meaning and make decisions

● Operations: object recognition, scene understanding, semantic reasoning

● Output: labels, descriptions, and decisions — what is in the scene, and what
it means

UN IT I | CO MPUT ER V IS IO N 16
 High-Level Vision
I n Prac tic e — A Real Exam pl e

Input — segmented regions Output — labeled, recognized shapes

Technique: Each region's contour is matched against simple shape models (vertex count via polygon approximation) to assign a label — the step that
finally turns structure into meaning.

UN IT I | CO MPUT ER V IS IO N 17
The Vision Hierarchy at a Glance

Low Mid High

Level Input Typical Operations Output

Low-Level Raw pixels Filtering, edge detection, denoising Processed image / edge map

Mid-Level Processed image Segmentation, feature extraction Regions, contours, features

High-Level Extracted features Recognition, semantic reasoning Labels & decisions

UN IT I | CO MPUT ER V IS IO N 18
COMPUTER VISION APPLICATIONS
Where vision systems are used today
A Wide Application Landscape

Document Image Analysis Biometrics Object Recognition

Tracking Medical Image Analysis Content-Based Image Retrieval

Video Data Processing Multimedia Virtual & Augmented Reality

UN IT I | CO MPUT ER V IS IO N 20
 Document Image Analysis
S yst em Block D iagr am

OCR
Document Image Pre-processing Layout Analysis Structured Output
(Character
(Scan / Photo) (Denoise, Binarize) (Text / Table / Image) (Text, Tables)
Recognition)

Goal: convert an unstructured visual document into structured, searchable digital text.

UN IT I | CO MPUT ER V IS IO N 21
 Document Image Analysis
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: Smart Document Digitizer

Input / Hardware Core Technique Real-Time Output

Smartphone or webcam captures a photo of a OpenCV pre-processing + Tesseract OCR engine Live overlay of recognized text, instantly
printed page or handwritten note running on-device or in the cloud searchable and editable on screen

UN IT I | CO MPUT ER V IS IO N 22
 Biometrics
S yst em Block D iagr am

Biometric Capture Pre-processing Feature Extraction Matching Decision

(Camera / Sensor) (Detect & Align) (Embedding) (Compare to Database) (Match / No Match)

Goal: verify or identify a person using unique physiological or behavioral traits.

UN IT I | CO MPUT ER V IS IO N 23
 Biometrics
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: Face-Based Smart Attendance

Input / Hardware Core Technique Real-Time Output

Classroom webcam capturing students' faces as Face detection (MTCNN) plus embedding Attendance automatically logged and displayed
they enter the room (FaceNet) matched against an enrolled student on a live dashboard
database

UN IT I | CO MPUT ER V IS IO N 24
 Object Recognition
S yst em Block D iagr am

Input Image / Feature Extraction Region Proposal / Classification + Labeled

Video Frame (CNN Backbone) Detection Head Bounding Box Output

Goal: localize and classify each object instance present in a scene.

UN IT I | CO MPUT ER V IS IO N 25
 Object Recognition
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: Real-Time Object Detector

Input / Hardware Core Technique Real-Time Output

Live webcam or phone camera feed YOLO (You Only Look Once) deep learning Bounding boxes and class labels drawn live over
detector running at real-time frame rates the video, with an FPS counter

UN IT I | CO MPUT ER V IS IO N 26
 Tracking
S yst em Block D iagr am

Video Frame Object Motion Prediction Data Association Trajectory

Sequence Detection (Kalman Filter) (ID Matching) Output

Goal: maintain a consistent identity for each object as it moves across frames.

UN IT I | CO MPUT ER V IS IO N 27
 Tracking
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: Multi-Person Tracker

Input / Hardware Core Technique Real-Time Output

Front-facing or overhead camera viewing a YOLO detection combined with DeepSORT for Each person shown with a persistent ID and a
classroom or corridor frame-to-frame identity association trailing path of recent motion

Classroom demo tip: Have two or three students cross paths in front of the camera to show ID persistence through occlusion.

UN IT I | CO MPUT ER V IS IO N 28
 How Tracking Works
Frame -by-Frame: Detect → Predict → Associat e → Persist

Frame N Frame N+1 Frame N+2

Detection Occlusion — paths cross Identity re-confirmed

ID 1 ID 1
ID 1 + ID 2 overlapping
ID 2 ID 2

Why IDs survive the crossing: When detections overlap in Frame N+1, the Kalman filter predicts where each tracked object should
be based on its prior motion. Data association then matches new detections to the closest prediction — not just the closest pixel —
so ID 1 and ID 2 separate correctly in Frame N+2 instead of swapping.

U NI T I | C O MPU T E R V ISIO N 27
 Inside Motion Prediction
The Kalm an Filter's Predict → Cor rect Cycle

1. PR EDICT 2. COR RECT

Using the object’s last known position and velocity, The new detection (if found) is blended with the
the filter estimates where it will be in the next frame prediction, correcting the estimate and improving
— even before a new detection arrives. the next prediction — the cycle repeats every frame.

Predicted Position vs. Actual Detection

t=2 t=5
t=1 t=4 (occluded)
t=3

Detected position Predicted-only (occluded frame)

U NI T I | C O MPU T E R V ISIO N 28
 Tracking in Action
Live Dem o Stor yboard — ID Pers istence Throug h Occlusion

PROTOTYPE: Multi-Person Tracker — Occlusion Walkthrough

Step 1 — Approach Step 2 — Cross Paths Step 3 — Separate

Two students walk toward each other Bounding boxes overlap as the students Once clear of each other, new
from opposite sides of the frame. YOLO pass. Detection alone can’t tell who is detections are matched to the predicted
detects each one; DeepSORT assigns ID who — motion prediction carries each positions. Both IDs re-lock onto the
1 and ID 2. ID through the gap. correct person.

Tracked Predicted Re-confirmed

Classroom demo tip: Pause the video at Step 2 and ask students to guess which person is ID 1 — then resume to reveal the filter’s answer. It
makes the predict/correct cycle tangible.

U NI T I | C O MPU T E R V ISIO N 29
 Medical Image Analysis
S yst em Block D iagr am

Scan Acquisition Pre-processing Segmentation Classification / Radiologist

(X-ray / CT / MRI) (Normalize, Denoise) (Organ / Tumor) Diagnosis Support Review & Report

Goal: support clinicians with quantitative, automated analysis of diagnostic scans.

UN IT I | CO MPUT ER V IS IO N 29
 Medical Image Analysis
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: AI-Assisted Chest X-ray Screening

Input / Hardware Core Technique Real-Time Output

Sample chest X-ray image from a public, de- CNN classifier trained to flag common Highlighted regions of concern overlaid on the X-
identified dataset abnormalities, visualized with Grad-CAM ray with a confidence score
heatmaps

Classroom demo tip: Use public sample images only — never real patient data without consent and ethical clearance.

UN IT I | CO MPUT ER V IS IO N 30
 Medical Image Analysis
Why This Application I s Held to a Higher S tandar d

Skin Cancer Brain MRI Tumor Alzheimer's

Dermoscopic image → benign / malignant MRI slice → tumor region + type MRI scan → stage of cognitive decline

High stakes, asymmetric cost Human-in-the-loop, always Same pipeline, harder data

A missed tumor (false negative) is far costlier than a The model flags and prioritizes; a radiologist or Acquire → pre-process → segment → classify — the
false alarm — models are tuned and evaluated clinician makes the final call. It assists — it doesn’t same CV pipeline as Unit I, applied to noisier, higher-
accordingly. diagnose alone. resolution, 3D-aware data.

U NI T I | C O MPU T E R V ISIO N 29
 Skin Cancer Detection
Derm oscopic I mage → B enign / Malignant Classification

The ABCDE Rule — What the Model Looks For

A Asymmetry: One half doesn’t match the other

B Border: Ragged, notched, or blurred edges

C Color: Multiple shades within one lesion

D Diameter: Larger than ~6mm (pencil eraser)

E Evolving: Changing in size, shape, or color over time

Generated dermoscopy image — for
illustration only

Dermoscopic Lesion Feature Benign / Malignant

Image
→ Segmentation
→ Extraction (CNN)
→ + Confidence

U NI T I | C O MPU T E R V ISIO N 30
 Brain MRI Tumor Detection
Real T1-CE MR I Slice → Tumor L ocaliz at ion → Seg mentation

Why Segmentation, Not Just Classification

A single “tumor / no tumor” label isn’t enough for

surgery. The dashed boundary tells surgeons exactly
which tissue to remove and how close it is to critical
structures.

Healthy Brain MRI (T2) Tumor Detected (T1-CE) — Meningioma

Source: Kaggle Brain MRI Dataset (CC BY) — de-identified clinical scans
Glioma (T2-FLAIR)

MRI Skull Tumor Volume +

Acquisition
→ Stripping
→ Segmentation (U-Net)
→ Location Report

U NI T I | C O MPU T E R V ISIO N 31
 Alzheimer's Disease Detection
MRI Scan → Br ain At rophy Patter n → Disease S tag e

Hallmark Signs the Model Quantifies

• Enlarged ventricles (fluid-filled cavities expand)
• Cortical thinning — visible shrinkage of brain tissue
• Reduced hippocampal volume — the memory center

Healthy Brain (MRI) Alzheimer’s — Brain Atrophy

MRI Volumetric Pattern Classifier Stage: Normal /

Scan
→ Measurement
→ (CNN / 3D-CNN)
→ MCI / Alzheimer's

U NI T I | C O MPU T E R V ISIO N 32
 One Pipeline, Three Diagnoses
Unified View + Classroom -Safe Demo Set up

Scan Segmentation / CNN Clinician

Acquisition
→ Pre-processing → ROI Detection
→ Classification
→ Review

Input Key Feature Output

Skin Cancer Dermoscopic photo ABCDE lesion features Benign / malignant

Brain Tumor MRI slice (axial) Tumor mass + boundary Location, size, type

Alzheimer's Volumetric MRI Ventricle / cortex volume Normal / MCI / AD stage

Classroom demo tip: Use public, de-identified datasets only (e.g., ISIC for skin lesions, public Kaggle/ADNI-derived MRI sets) — never real
patient data without consent and ethical clearance.

U NI T I | C O MPU T E R V ISIO N 33
 Content-Based Image Retrieval
S yst em Block D iagr am

Query Feature Extraction Similarity Search Top-K Retrieved

Ranking
Image (Embedding) (Nearest Neighbor) Images

Goal: retrieve visually similar images from a large database without relying on text tags.

UN IT I | CO MPUT ER V IS IO N 31
 Content-Based Image Retrieval
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: Visual Product Search

Input / Hardware Core Technique Real-Time Output

A photo of a product snapped on a phone or Deep embedding extraction compared against a Top-5 visually similar catalog images displayed
webcam pre-indexed image database instantly, ranked by similarity

Classroom demo tip: Pre-index a folder of 30–50 sample product photos before class for a fast, reliable demo.

UN IT I | CO MPUT ER V IS IO N 32
 Video Data Processing
S yst em Block D iagr am

Raw Video Frame Sampling Temporal Feature Event / Action Summary /

Stream & Decoding Extraction Detection Indexed Metadata

Goal: extract structure and meaning from video by exploiting information across time.

UN IT I | CO MPUT ER V IS IO N 33
 Video Data Processing
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: Automatic Highlight Generator

Input / Hardware Core Technique Real-Time Output

A short pre-recorded sports or lecture video clip Shot-boundary detection plus an action- An auto-compiled highlight reel of the most
recognition model to flag key moments significant clips, generated in seconds

Classroom demo tip: Use a 3–5 minute sample clip so the full pipeline completes within the class period.

UN IT I | CO MPUT ER V IS IO N 34
 Multimedia
S yst em Block D iagr am

Multi-modal Input Modality-Specific Fusion Layer Cross-modal Output

(Video + Audio + Text) Feature Extraction (Joint Embedding) Reasoning (Caption / Retrieval)

Goal: combine vision with other modalities to produce richer, joint understanding.

UN IT I | CO MPUT ER V IS IO N 35
 Multimedia
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: Auto-Captioning Assistant

Input / Hardware Core Technique Real-Time Output

A short video clip with audio Visual feature extraction (CNN) fused with a Synchronized captions displayed live alongside
speech-to-text transcript to generate captions the video timeline

Classroom demo tip: A short clip with clear speech and an open-source speech-to-text library works best live.

UN IT I | CO MPUT ER V IS IO N 36
 Virtual Reality and Augmented Reality
S yst em Block D iagr am

Sensors Pose & Motion Scene Rendering Display

(Camera + IMU) Tracking (SLAM) Understanding Engine (HMD / Mobile Screen)

Goal: align rendered content with the user's head pose (VR) or the real world (AR) in real time.

UN IT I | CO MPUT ER V IS IO N 37
 Virtual Reality and Augmented Reality
Real -Time D emonstrat ive Prototy pe

PROTOTYPE: AR Furniture Placement App

Input / Hardware Core Technique Real-Time Output

Smartphone camera viewing a real room ARKit / ARCore plane detection with SLAM-based A virtual furniture model anchored in the live
pose tracking camera view, adjusting as the phone moves

Classroom demo tip: Most modern phones support this out of the box via free AR sample apps — great for a quick live demo.

UN IT I | CO MPUT ER V IS IO N 38
 UNIT I — COMPUTER VISION

Recap
Image Processing · Computer Vision · Computer Graphics · Vision Hierarchy · CV Applications

[Link] CSE (AI) · Computer Vision

1/15
What We'll Cover
Five pillars of Unit I — from pixels to real-world systems

01 02 03 04 05

Three Related Imaging & Vision Hierarchy CV Applications (9 Key Takeaways

Fields Representation Domains)

Image Processing · Light, cameras, sensors, Low → Mid → High-Level From documents to What exam questions
Computer Vision · the frame buffer processing VR/AR look like
Computer Graphics

UNIT I · COMPUTER VISION · [Link] CSE (AI) 2/15

 Three Related Fields
Image Processing · Computer Vision · Computer Graphics

3/15
Three Related — But Different — Fields
Each moves information in a different direction between the world and its description

Image Processing Computer Vision Computer Graphics

Image → Image Image → Description Description → Image

• Operates on raw pixel data • Machines interpret visual info • Inverse of computer vision
• Goal: improve quality, NOT • like humans do • Input: 3D model / scene desc.
interpret • Output: meaningful description, • Core to gaming, animation,
• Noise removal, filtering, • decision, or action • simulation, VR
• enhancement, compression • Draws on AI, maths,
• neuroscience, physics

UNIT I · COMPUTER VISION · [Link] CSE (AI) 4/15

Imaging & Image Representation
How the physical world becomes a grid of numbers

Sensing Light (§ 2.1) Imaging Devices (§ 2.2)

Point source illuminates a surface element

CCD Camera
Surface normal (N) & angle θ determine how much light
reflects Lens → CCD array → A/D → Frame Buffer → Algorithm

Radiance travels to camera along optical axis Z

Sensor Geometries
Irradiance = light arriving at the camera sensor

Circular (radial), Linear (push-broom), ROSA (sector+ring)

Human Eye
Pixel value ≈ f(surface reflectance, angle θ, illumination)

Iris (aperture), Retina (rods + cones), Fovea (dense color)

UNIT I · COMPUTER VISION · [Link] CSE (AI) 5/15

The Vision Hierarchy
Low-Level → Mid-Level → High-Level

6/15
The Vision Hierarchy
Every CV system passes through these three stages, in order

Low-Level Vision Mid-Level Vision High-Level Vision

• Noise removal / Gaussian • Segmentation (region grouping) • Object recognition

smoothing • Feature extraction • Scene understanding
• Edge detection (Canny) • Depth & motion estimation • Semantic reasoning
• Contrast enhancement, filtering

OUTPUT OUTPUT OUTPUT

Processed image / Edge map Regions, contours, feature points Labels, decisions, descriptions

Still no meaning Structured but unlabelled Meaning extracted ✓

UNIT I · COMPUTER VISION · [Link] CSE (AI) 7/15

CV Applications
9 Domains — From Documents to VR/AR

8/15
A Wide Application Landscape
All 9 domains share the same CV pipeline — what differs is the domain's goal and data

Document Image Object

Analysis Biometrics Recognition
Scan → Digital Text Verify / Identify Person Localize & Classify

Medical Image
Tracking Analysis CBIR
Consistent ID across frames Assist Diagnosis Retrieve by Visual Content

Video Data
Processing Multimedia VR / AR
Meaning across Time Cross-modal Understanding Align render to World/Pose

UNIT I · COMPUTER VISION · [Link] CSE (AI) 9/15

Applications Deep-Dive — Group 1

Document Image Analysis Biometrics Object Recognition

Scan/Photo → Pre-process → Layout Analysis → Capture → Detect & Align → Feature Extract Image → CNN Backbone → Region Proposal →
OCR → Structured Output (Embedding) → Match → Decision Classify + BBox → Labeled Output

• Goal: unstructured → searchable • Verification (1:1) vs Identification (1:N) • Localize AND classify each object
digital text • Modalities: Face, Fingerprint, Iris, instance
• Techniques: bilateral filtering, deskew, Voice, Gait, Signature • YOLO processes entire image in one
adaptive threshold, OCR • Demo: webcam → auto attendance pass
• Demo: photograph a printed page → dashboard • Demo: live webcam with bounding
live editable text boxes + FPS counter

PR OTOT YPE PR OTOT YPE PR OTOT YPE

Smart Document Digitizer Face-Based Smart Attendance Real-Time Object Detector

OpenCV + Tesseract OCR MTCNN detection + FaceNet embedding YOLO — real-time frame rates

UNIT I · COMPUTER VISION · [Link] CSE (AI) 10/15

Applications Deep-Dive — Group 2

Tracking Medical Image Analysis

Goal: maintain consistent identity for each object as it moves across

frames Skin Cancer

In: Dermoscopic photo · Feature: ABCDE rule · Out: Benign / Malignant

Kalman
Video Object Data Trajectory
Filter
Frames Detection Association Output
(Predict)
Brain Tumor
In: MRI slice (axial) · Feature: Tumor mass + boundary · Out: Location,
Key Concepts size, type
• Kalman Filter: PREDICT → CORRECT cycle every frame
• Data Association resolves occlusions using predicted position, Alzheimer's
not just pixel proximity
In: Volumetric MRI · Feature: Ventricle / cortex vol. · Out: Normal /
• DeepSORT = YOLO detection + appearance embedding + MCI / AD
Kalman
• Demo: cross paths → IDs survive occlusion (Frames N, N+1,
N+2) Guiding Principles

• High stakes, asymmetric cost (false negative >> false positive)

• Human-in-the-loop always — model assists, clinician decides

UNIT I · COMPUTER VISION · [Link] CSE (AI) 11/15

Applications Deep-Dive — Group 3

CBIR Video Data Processing Multimedia VR / AR

Query Image → Embedding → Raw Video → Frame Sampling → Video + Audio + Text → Per- Camera + IMU → SLAM (Pose
Nearest-Neighbour Search → Temporal Feature Extraction → Modality Extract → Fusion Layer → Tracking) → Scene Understanding →
Ranking → Top-K Images Event Detection → Summary Cross-modal Reasoning → Output Render Engine → HMD/Screen

• Retrieves by visual • Exploits information • Combines vision with • VR: align render to head
content — no text tags ACROSS TIME (not single audio & text for richer pose — fully virtual
needed frames) understanding
• AR: overlay digital
• Deep embedding • Shot-boundary • Joint embedding space content anchored to real
compared against pre- detection + action- across modalities (e.g. world
indexed database recognition model CLIP)
• SLAM = Simultaneous
• Demo: snap a product → • Demo: 3–5 min clip → • Demo: video + speech → Localization And
Top-5 similar catalog auto-compiled highlight synchronized auto- Mapping
items reel captions

UNIT I · COMPUTER VISION · [Link] CSE (AI) 12/15

 Key Takeaways
What you must know cold — exam & beyond

13/15
Unit I — Must-Know Checklist
1 Direction of Information Flow 2 Vision Hierarchy — Key Distinctions

IP: image→image (no meaning extracted) Low-Level: pixel ops, NO semantic meaning
CV: image→description (meaning extracted) Mid-Level: grouping → regions/features, still NO labels
CG: description→image (rendering / synthesis) High-Level: recognition → labels & decisions (meaning!)

3 CV Applications — Pipeline is the Same 4 Tracking — Kalman Filter

Acquire → Pre-process → Segment/Detect → Classify → Output PREDICT: estimate next position from prior velocity
What changes: the input modality and the domain goal CORRECT: blend new detection with prediction
IDs survive occlusion because of prediction, not pixel proximity

5 Medical Imaging — Higher Standard 6 Biometrics — Verification vs. Identification

Asymmetric cost: false-negative >> false-positive Verification (1:1): 'Are you who you claim to be?' — one comparison
Human-in-the-loop always; model assists, clinician decides Identification (1:N): 'Who are you?' — N comparisons against full DB
U-Net for segmentation; segmentation > classification for surgery
UNIT I · COMPUTER VISION · [Link] CSE (AI) 14/15
 That's Unit I
Three fields · Imaging foundations · Vision hierarchy · 9 application domains

IP → Image CV → Description CG → Image Low / Mid / High 9 Domains

Up Next: Unit II — Image Formation Models

[Link] CSE (AI) · Computer Vision

15/15
Unit I — Key Takeaways

Image Processing, Computer Vision, and Computer Graphics move information in different directions

Computer Vision spans low-, mid-, and high-level stages, from raw pixels to meaning

Applications range from document analysis and biometrics to immersive AR/VR experiences

Up next: Unit II — Image Formation Models

UN IT I | CO MPUT ER V IS IO N 39
Questions?
Next: Unit II — Image Formation Models