Chapter 2:Multimedia
information representation
Chapter 2:Multimedia
information representation
2.1 Introduction
2.2 Digitization principles
2.3 Text
2.4 Images
2.5 Audio
2.6 Video
Introduction
The conversion of an analog signal into
a digital form
Signal encoder, sampling, signal
decoder
2.2 Digitization principles
2.2.1 Analog signals
Fourier analysis can be used to show that any
time-varying analog signal is made up of a
possibly infinite number of single-frequency
sinusoidal signals whose amplitude and phase
vary continuously with time relative to each other
Signal bandwidth
Fig2.1
The bandwidth of the transmission channel
should be equal to or greater than the bandwidth
of the signal─bandlimiting channel
2.2.2 Encoder design
A bandlimiting filter and an analog-to-digital
converter(ADC), the latter comprising a sample-
and-hold and a quantizer
Fig2.2
Remove selected higher-frequency components
from the source signal (A)
(B) is then fed to the sample-and-hold circuit
Sample the amplitude of the filtered signal at
regular time intervals (C) and hold the sample
amplitude constant between samples (D)
2.2.2 Encoder design
Quantizer circuit which converts each
sample amplitude into a binary value
known as a codeword (E)
The signal to be sampled at a rate which
is higher than the maximum rate of
change of the signal amplitude
The number of different quantization
levels used to be as large as possible
2.2.2 Encoder design
Nyquist sampling theorem states that:
in order to obtain an accurate
representation of a time-varing analog
signal, its amplitude must be sampled
at a minimum rate that is equal to or
greater than twice the highest
sinusoidal frequency component that is
present in the signal
2.2.2 Encoder design
Nyquist rate: samples per second (sps)
The distortion caused by sampling a signal
at a rate lower than the Nyquist rate
Fig2.3
Alias signals: they replace the
corresponding original signals
Figure 2.3 Alias signal generation
due to undersampling.
2.2.2 Encoder design
Quantization intervals
A finite number of digits is used, each sample can
only be represented by a corresponding number
of discrete levels
Fig2.4
If Vmax is the maximum positive and negative
signal amplitude and n is the number of binary
bits used, then the magnitude of each
quantization interval, q
2Vmax
q n
2
2.2.2 Encoder design
Each codeword corresponds to a nominal
amplitude level which is at the center of the
corresponding quantization interval
The difference between the actual signal
amplitude and the corresponding nominal
amplitude is called the quantization error
(Quantization noise)
The ratio of the peak amplitude of a signal to its
minimum amplitude is known as the dynamic
range of the signal, D (decibels or dB)
2.2.2 Encoder design
D 20 log dB
Vmax
10 Vmin
It is necessary to ensure that the level of
quantization noise relative to the smallest
signal amplitude is acceptable
Example 2.2
2.2.3 Decoder design
Fig2.5
Reproduce the original signal, the output
of the DAC is passed through a low-pass
filter which only passes those frequency
components that made up the original
filtered signal (C)
Audio/video encoder-decoder or
audio/video codec
2.3 Text
Three types of text
Unformatted text
Formatted text
hypertext
2.3.1 Unformatted text
American Standard Code for Information
Interchange (ASCII character set)
Fig2.6
Mosaic characters create relatively simple
graphical images
2.3.2 Formatted text
Produced by most word processing
packages
Each with different headings and with
tables, graphics, and pictures inserted at
appropriate points
Fig2.8
WYSIWYG: an acronym for what-you-see-
is-what-you-get
Figure 2.8 Formatted text: (a) an example formatted
text string;(b) printed version of the string.
2.3.3 Hypertext
Formatted text that enables a related set
of documents─normally referred to as
pages─to be created which have defined
linkage points─referred to as hyperlinks
─between each other
Fig2.9
2.4 Images
Image are displayed in the form of a two-
dimensional matrix of individual picture
elements─known as pixels or pels
2.4.1 Graphics
Fig 2.10
Two forms of representation of a computer
graphic: a high-level version (similar to the source
code of a high-level program) and the actual
pixel-image of the graphic (similar to the byte-
string corresponding to the low-level machine
code─bit-map format)
Standardized forms of representation such as GIF
(graphical interchange format) and TIFF (tagged
image file format)
2.4.2 Digitized documents
Fig 2.11
A single binary digit to represent each
pel, a 0 for a white pel and a 1 for a
black pel
2.4.3 Digitized pictures
Color principles
A whole spectrum of colors─known as a color
gamut ─can be produced by using different
proportions of red(R), green(G), and blue (B)
Fig 2.12
Additive color mixing producing a color image on
a black surface
Subtractive color mixing for producing a color
image on a white surface
Fig 2.13
2.4.3 Digitized pictures
Raster-scan principles
Progressive scanning
Each complete set of horizontal scan is
called a frame
The number of bits per pixel is known as
the pixel depth and determines the range
of different colors
2.4.3 Digitized pictures
Aspect ratio
Both the number of pixels per scanned line and
the number of lines per frame
The ratio of the screen width to the screen height
National Television Standards Committee (NTSC),
PAL(UK), CCIR(Germany), SECAM (France)
Table 2.1
2.4.3 Digitized pictures
2.4.3 Digitized pictures
Digital cameras and scanners
An image is captured within the
camera/scanner using an image sensor
A two-dimensional grid of light-sensitive
cells called photosites
A widely-used image sensor is a charge-
coupled device (CCD)
Fig 2.16
2.5 Audio
The bandwidth of a typical speech signal is
from 50Hz through to 10kHz; music signal
from 15 Hz through to 20kHz
The sampling rate: 20ksps (2*10kHz) for
speech and 40ksps (2*20kHz) for music
Music stereophonic (stereo) results in a bit
rate double that of a monaural(mono)
signal
Example 2.4
2.5.2 CD-quality audio
Bit rate per channel
=sampling rate*bits per sample
3
44.110 16 705.6kbps
Total bit rate = 2*705.6=1.411Mbps
Example 2.5
2.6 Video
2.6.1 Broadcast television
Scanning sequence
It is necessary to use a minimum refresh
rate of 50 times per second to avoid flicker
A refresh rate of 25 times per second is
sufficient
Field:the first comprising only the odd scan
lines and the second the even scan lines
2.6.1 Broadcast television
The two field are then integrated together
in the television receiver using a technique
known as interlaced scanning
Fig 2.19
The three main properties of a color source
Brightness
Hue:this represents the actual color of the
source
Saturation:this represents the strength or
vividness of the color
2.6.1 Broadcast television
The term luminance is used to refer to the
brightness of a source
The hue and saturation are referred to as
its chrominance
Ys 0.299 Rs 0.587Gs 0.144 Bs
Where Ys is the amplitude of the luminance
signal and Rs,Gs and Bs are the magnitudes
of the three color component signals
2.6.1 Broadcast television
The blue chrominance (Cb), and the red
chrominance (Cr) are then used to
represent hue and saturation
The two color difference signals:
Cb Bs Ys Cr Rs Ys
2.6.1 Broadcast television
In the PAL system, Cb and Cr are referred to
as U and V respectively
PAL : Y 0.299 R 0.587G 0.114 B
U 0.493( B Y )
V 0.877( R Y )
The NTSC system form two different signals
referred to as I and Q
NTSC : Y 0.299 R 0.587G 0.114 B
I 0.74( R Y ) 0.27( B Y )
Q 0.48( R Y ) 0.41( B Y )
2.6.2 Digital video
Eye have shown that the resolution of the eye is
less sensitive for color than it is for luminance
4 ： 2 ： 2 format
The original digitization format used in
Recommendation CCIR-601
A line sampling rate of 13.5MHz for luminance
and 6.75MHz for the two chrominance signals
The number of samples per line is increased to
720
2.6.2 Digital video
The corresponding number of samples for each
of the two chrominance signals is 360 samples
per active line
This results in 4Y samples for every 2Cb, and 2Cr
samples
The numbers 480 and 576 being the number of
active (visible) lines in the respective system
Fig. 2.21
Example 2.7
Figure 2.21 Sample positions with
4:2:2 digitization format.
2.6.2 Digital video
4 ： 2 ： 0 format is used in digital video
broadcast applications
Interlaced scanning is used and the
absence of chrominance samples in
alternative lines
The same luminance resolution but half the
chrominance resolution
Fig2.22
Figure 2.22 Sample positions in
4:2:0 digitization format.
2.6.2 Digital video
525-line system Y 720 480
Cb Cr 360 240
625-line system Y 720 480
Cb Cr 360 288
13.5 10 8 2 3.375 10 8 162Mbps
6 6
2.6.2 Digital video
HDTV formats: the resolution to the newer
16/9 wide-screen tubes can be up to
1920*1152 pixels
The source intermediate format (SIF) give a
picture quality comparable with video
recorders(VCRs)
2.6.2 Digital video
The common intermediate format (CIF) for
use in videoconferencing applications
Fig 2.23
The quarter CIF (QCIF) for use in video
telephony applications
Fig 2.24
Table 2.2
Figure 2.23 Sample positions for
SIF and CIF.
Figure 2.24 Sample positions for QCIF.
2.6.3 PC video