Introduction to Artificial
Intelligence
Module 5 Part 2
Overview:
• Images: Millions of noisy, data-rich
pixels (e.g., 5 million 3-byte
pixels).
• Challenge: Efficiently analyze this
EARLY IMAGE- large volume of raw data.
PROCESSING Key Early (Low-Level)
Operations:
OPERATIONS [Link] Detection – Identify object
boundaries.
[Link] Analysis – Understand
surface patterns.
[Link] Flow – Measure motion
between frames.
Characteristics:
• Local Processing – Operates on
EARLY IMAGE- small regions of the image.
PROCESSING • Object-Agnostic – No prior
knowledge of scene contents
OPERATIONS needed.
• Hardware-Friendly – Suitable for
parallel processing (e.g., GPU or
biological vision systems).
• Edges are straight lines or curves across which
there is a significant change in brightness.
• Represent transitions in depth, surface
orientation, reflectance, or illumination.
Purpose of Edge Detection:
• Reduce a complex image into a compact,
Edge abstract representation.
• Help identify important scene contours.
detection Scene Discontinuity Types:
[Link] Discontinuity
[Link]-Normal Discontinuity
[Link] Discontinuity
[Link] Discontinuity (Shadows)
Real vs. Ideal Output:
• Real edge detection outputs are
often noisy and incomplete.
• Contains gaps and spurious
edges that don’t correspond to
actual scene features.
Example:
Edge detection • A scene with a stapler → edge-
detection output differs from a
clean line drawing.
• Post-processing is needed to
refine edges.
Edge detection
Brightness Profile Across an
Edge:
• A sharp change in brightness
indicates a potential edge.
Method:
• Compute the derivative of image
intensity
• Look for large peaks in the
derivative.
Edge Detection Problem:
• Noise creates false peaks
(subsidiary peaks).
Solution:
• Smooth the image before
differentiating to reduce noise.
• Result: Clearer edge detection with
fewer false positives.
• Visual feel of a surface (linked to
touch).
• Computer Vision: A spatially
repeating pattern detectable by
visual cues.
Examples of Texture Patterns:
• Periodic: Stitches on a sweater,
Texture bricks on a wall.
• Statistical: Pebbles on a beach,
Detection people in a stadium.
Key Insight:
• Texture is a multipixel property —
unlike brightness, which belongs to
individual pixels.
Why It Matters:
• Texture remains robust under
varying illumination, making it
valuable for object recognition.
• For a patch of pixels:
• Compute orientation at each
pixel.
• Create a histogram of
orientations.
Interpretation Examples:
• Wall of bricks → Two strong orientation
peaks (vertical + horizontal).
Texture • Leopard spots → Uniform orientation
distribution.
Detection Challenges:
• Edge detection fails in textured scenes
(e.g., tiger’s stripes hide its shape).
Solution:
• Use differences in texture statistics (e.g.,
orientation histograms) to detect object
boundaries.
• Helps segment objects from textured
backgrounds (e.g., tiger vs. grass).
• Optical Flow is the apparent motion of
objects, surfaces, or edges in a video,
caused by movement of objects or the
camera.
Key Properties:
• Describes direction and speed of motion
in pixels per second.
• Applies to any video sequence, not
static images.
• Reveals scene structure (e.g., closer
Optical Flow objects move faster across the image).
• Helps in action recognition (e.g.,
identifying fast-moving limbs in sports
footage).
Example:
• In a video of a tennis player:
• Optical flow vectors show the
motion of the racket and legs.
• Faster movement = longer vectors.
Vector Representation:
At each pixel (x,y) optical flow is
defined by
• Match corresponding points across
frames.
• Assumption: Image patches at
matching points have similar
Optical Flow intensity patterns.
• Accurate flow estimation requires
dealing with:
• Occlusions
• Illumination changes
• Ambiguities in uniform regions
• Motion tracking, depth estimation,
activity recognition, video
compression.
Optical Flow
• Segmentation divides an image
into regions of similar pixels based
on visual attributes like:
Brightness, Color, Texture
• Pixels within an object share
similar properties.
Segmentation • Sharp changes in these attributes
usually occur at object
of images boundaries.
Two Main Approaches:
[Link] Detection – Find edges
between regions.
[Link] Detection – Group similar
pixels directly.
• Model boundary detection as a
machine learning task.
• Define Pb(x, y, θ) = Probability of
a boundary at pixel (x, y) and
orientation θ.
• Use a circular disk around the
pixel, split into two halves along θ:
Segmentation • Compare histograms of
brightness, color, texture in the
of images two halves.
Training:
• Use labeled datasets with human-
marked boundaries.
• Train classifiers to predict those
exact boundaries.
• More accurate than basic edge
detection methods.
No Closed Regions:
• Thresholding Pb(x, y, θ) does not
guarantee closed boundary
curves.
Segmentation • This means regions aren't directly
formed.
of images Only Local Context:
• Decisions use features from small
neighborhoods.
• Lacks global consistency, so larger
scene structure is not considered.
• Appearance refers to what an
object generally looks like.
• Some categories, like baseballs,
show little variation — most
Object instances look the same.
Recognition by • For such cases, we can compute a
set of features and apply a
Appearance classifier to detect the object.
• Other categories, like houses or
ballet dancers, vary widely in size,
color, shape, or pose.
• A useful abstraction is to
represent objects using local
patterns that shift in position
• We can detect such variable
objects using local histograms of
detector responses.
• These histograms reveal whether
certain parts are present, but
Object ignore exact locations.
• Classifiers trained on such
Recognition by features work well, especially for
object categories with low
Appearance variation.
• A good example is frontal face
detection under normal lighting
and resolution conditions.
• Despite small changes, the overall
face pattern — bright skin, dark
eyes, dark mouth — remains
consistent.
• Face detection is now routine in
even inexpensive cameras.
• For now, we focus on upright faces
Object with vertical noses.
• A common strategy: sweep a
Recognition by fixed-size circular window over the
image and compute features.
Appearance • Present these features to a
classifier — this is known as the
sliding window approach.
• Features must handle lighting
variation, such as shadows or
brightness changes.
• To detect different face sizes,
repeat the sliding window across
different image scales.
• After scanning, we postprocess
Object overlapping detections to produce
final results.
Recognition by • Classifiers like logistic regression
or SVMs can output confidence
Appearance scores.
• These scores help merge
overlapping windows into one
strong detection.
• This creates a system that can
search across both positions and
scales.
• To detect rotated faces, we
estimate the face's orientation
using regression.
• Then, reorient the window and
Object apply the vertical face classifier.
Recognition by • Training data is easy to generate
from datasets with labeled faces.
Appearance • Data augmentation improves
performance: rotate, shift, or scale
training windows slightly.
• This approach now gives robust
results for detecting frontal faces
— though side views remain
harder.
Object Recognition by
Appearance
• Many objects show far more
complex image patterns than
faces.
Complex • This is due to visual effects that
shift or distort features in the
appearance image.
and pattern • Foreshortening distorts patterns
when viewed at a slant.
elements • Aspect causes the same object to
look different from different
directions (e.g., a doughnut looks
like an oval from the side, a ring
from above).
• Occlusion hides parts of objects —
either from other objects or from
themselves (self-occlusion).
Complex appearance and
pattern elements
• Objects can deform, changing
appearance as parts move (e.g.,
human limbs in different poses).
Complex • Despite variation, object
recognition is still possible by
appearance detecting key pattern elements.
• If most of the expected elements
and pattern are present in approximately the
right location, we can infer the
elements object is present.
• An object recognizer must check
for the presence and relative
position of parts.
• Using simple histograms of
pattern elements is not enough —
they lose too much spatial
information.
• Histogram-only approaches
confuse different objects with
similar element counts (e.g., flags
with same colors but different
Complex layouts).
• Better results come from
appearance histograms with spatial structure,
preserving rough layout (e.g.,
and pattern headlights at front, wheels at
bottom of a car).These enhanced
elements features have led to success in
tasks like pedestrian detection.
• Even with variation, searching
across location and scale works if
enough structure is preserved.
• This supports a model of objects
as collections of parts that can
move slightly but retain a
recognizable configuration.
• Simply placing a box around
objects is useful for detection, but
OBJECT not enough to understand actions.
• To infer actions, we need to locate
RECOGNITION arms, legs, head, and body
accurately in the image.
FROM • Detecting individual body parts
STRUCTURAL using sliding windows is difficult —
they’re often small and vary in
INFORMATION appearance.
• Arms and legs may be just 2–3
pixels wide in images, making
them hard to detect directly.
• Understanding relationships
between parts (e.g., matching left
and right limbs) can improve
recognition.
• A structured approach assumes
we know the appearance of each
OBJECT body part.
• The body is modeled as a tree of
RECOGNITION segments (e.g., upper/lower limbs,
torso, face, hair), known as a
FROM “cardboard person” model.
STRUCTURAL • Each segment is rectangular and
has a defined relationship to its
INFORMATION parent segment — e.g., forearm
depends on upper arm, upper arm
depends on torso.
• These relationships form a tree-
structured Bayesian network,
usually rooted at the torso.
• Inference methods on this tree
can estimate the best
configuration in an image.
OBJECT RECOGNITION FROM
STRUCTURAL INFORMATION
OBJECT RECOGNITION FROM
STRUCTURAL INFORMATION
• To manage complexity, the image
is discretized into a grid of
OBJECT candidate rectangles at multiple
orientations.
RECOGNITION • Each segment has a stack of
image patches representing
FROM different positions and angles.
STRUCTURAL • Matching all segments
simultaneously can be
INFORMATION computationally expensive —
torso matching alone may be
O(M⁶).
• Speedups are possible through
clever choices of scoring functions
and efficient search.
• This system is called a pictorial
structure model, and it works
well in practice.
• Body parts usually appear in
connected configurations, not in
OBJECT isolation.
• Detecting one easy-to-find part
RECOGNITION can help locate others (e.g.,
finding the torso may suggest arm
FROM locations).
STRUCTURAL • Deformable template models
define which body configurations
INFORMATION are valid — elbows bend, but
heads don’t connect to feet.
• The simplest deformable models
link lower arms to upper arms,
upper arms to torso, and so on.
• Richer models handle more
variation and allow more accurate
human pose and activity
recognition.
• Tracking people in video is a key practical
problem for game interfaces and surveillance
systems.
• Reliable detection of arms, legs, torso, and
Tracking head positions over time improves
understanding of human activity.
people in • Traditional filtering methods struggle due to
rapid and unpredictable human motion
video between frames.
• At 30 Hz video, body configuration in one frame
doesn’t strongly constrain the next frame’s
pose.
• Current best methods leverage the fact that
appearance changes slowly over consecutive
frames.
• The video is treated as a large stack of images
showing the person to track.
• Appearance models explain many frames by
detecting consistent body segments across
Tracking them.
people in • Detectors identify segments (e.g., torso, limbs)
by looking for features like roughly parallel
video edges.
• Starting with the torso, the biggest and most
reliably detected segment, helps bootstrap
detection of other parts (e.g., upper legs near
the torso).
• Clustering detector responses over frames finds
segments that appear consistently.
• Challenges arise when people appear against
near-static backgrounds, causing false positives
from segment detectors.
• One approach: iteratively reestimate both body
configuration and appearance to refine the
Tracking appearance model.
• Another practical method: use a fixed body
people in configuration detector applied to all frames.
video • Choosing a common, easy-to-detect pose
(e.g., lateral walking) helps reduce false
positives.
• A detector tuned for low false positive rate
ensures that when it responds, the detected
body parts (torso, arms, legs, head) are likely
correct.
Computer Vision
Computer
Vision
• A simple camera model that forms
an image without lenses.
• Light passes through a tiny
pinhole and projects an inverted
image onto the opposite surface
inside the camera.
• No lens means no optical
Computer distortion or focusing issues.
• Very small aperture limits the
Vision amount of light, making images
dim and requiring long exposure
times.
• Produces a sharp image if the
pinhole is sufficiently small.
• Applications: Basis for
understanding image formation;
used in some low-tech imaging
devices.
• Definition: Optical systems using one or more lenses to
focus light and form images.
• Types: Convex (converging), concave (diverging),
compound lenses.
Lens • Cameras, microscopes, telescopes, eyeglasses.
System • Image Formation by a Convex Lens:
• Real inverted image (if object beyond focal length).
• Virtual upright image (if object within focal length).
• Concave Lens: Always forms a virtual, upright, diminished
image.
• Lens Equation
(where f = focal length, v= image distance, u =
object distance)
Lens • Magnification
System
• hi= Height of the image (formed by the lens).
• ho= Height of the object (actual size of the
object)
• Chromatic Aberration: Different wavelengths focus at
different points.
• Spherical Aberration: Blurring due to lens curvature.
• Distortion: Barrel or pincushion effects.
• Correction Methods: Achromatic lenses, aspheric designs,
Lens multiple lens elements.
System Advanced Lens Systems
• Compound Lenses: Multiple lenses to reduce aberrations
(e.g., camera lenses).
• Zoom Lenses: Adjustable focal length.
• Applications: Photography (DSLR, smartphone cameras).