0% found this document useful (0 votes)

4 views239 pages

Understanding Multimedia Concepts

Multimedia refers to the integration of various forms of media, including text, graphics, audio, and video, into a single digital format. It encompasses multimedia applications that utilize multiple media sources and multimedia systems that process and manage these data types. Key challenges in multimedia include maintaining temporal relationships, data representation, and large data requirements for storage and bandwidth.

Uploaded by

Rituraj Chaudhary

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

0% found this document useful (0 votes)

4 views239 pages

Understanding Multimedia Concepts

Uploaded by

Rituraj Chaudhary

We take content rights seriously. If you suspect this is your content, claim it here.

Available Formats

Download as PDF, TXT or read online on Scribd

What is Multimedia?

Multimedia means that computer

information can be represented
through audio, video, and animation
in addition to traditional media (i.e.,
(i e
text, graphics/drawings, images).

Downloaded from CSIT Tutor

General Definition

A good general working definition for this

module is:
 Multimedia is the field concerned with the
computer controlled integration of text,
graphics,
g p drawings,
g still and moving g images
g
(Video), animation, audio, and any other
media where every type of information can
b represented,
be t d stored,
t d ttransmitted
itt d and
d
processed digitally.

Downloaded from CSIT Tutor

M lti di Application
Multimedia A li ti Definition
D fi iti

 A Multimedia Application is an
application
pp which uses a collection of
multiple media sources e.g. text, graphics,
images,
g , sound/audio,, animation and/or
video

Downloaded from CSIT Tutor

What is HyperText
yp and HyperMedia?
yp

 Hypertext is a text
which contains links
to other texts.
 The term was
i
invented
t dbby T
Tedd
Nelson around 1965.

Downloaded from CSIT Tutor

Hypermedia
yp
 HyperMedia is not constrained to be text-based. It can
include other media
media, ee.g.,
g graphics
graphics, images
images, and
especially the continuous media – sound and video.

Downloaded from CSIT Tutor

Example Hypermedia Applications?

 The World Wide Web (WWW) is the best

example
p of a hypermedia
yp application.
pp
 Powerpoint
 Adobe Acrobat

Downloaded from CSIT Tutor

M lti di Systems
Multimedia S t
 A Multimedia System is a system
capable
p of p
processingg multimedia data
and applications
 A Multimedia System is characterised by
the processing, storage, generation,
manipulation of Multimedia information
information.

Downloaded from CSIT Tutor

Characteristics of a Multimedia System

A Multimedia system has four basic

characteristics:
 Multimedia systems must be computer
controlled.
 Multimedia systems are integrated.
 The information they handle must be
represented digitally.
 The interface to the final presentation of media is
usually interactive.

Downloaded from CSIT Tutor

Challenges for Multimedia Systems

 Distributed Networks
 Temporal relationship between data
– Render
R d diff differentt d
data
t att same titime -
continuously.
– Sequencing within the media
playing
p y g frames in correct order/time
frame in video
– Synchronisation — inter inter-media
media scheduling

Downloaded from CSIT Tutor

Key
y Issues for Multimedia Systems
y

The keyy issues multimedia systems

y need to deal
with here are:
 How to represent and store temporal information.
 How to strictly maintain the temporal relationships
on play back/retrieval
 What
Wh t process are involved
i l d iin th
the above.
b
 Data has to represented digitally — Analog–
Digital Conversion
Conversion, Sampling etcetc.
 Large Data Requirements — bandwidth, storage,
compression

Downloaded from CSIT Tutor

Components of a Multimedia System
Now let us consider the Components (Hardware and
S ft
Software) ) required
i d for
f a multimedia
lti di system:
t
 Capture devices — Video Camera, Video Recorder,
Audio Microphone, Keyboards, mice, graphics tablets, 3D
i
inputt devices,
d i ttactile
til sensors, VR ddevices.
i Di
Digitising
iti i
Hardware
 Storage Devices — Hard disks, CD-ROMs, DVD-ROM,
etc
 Communication Networks — Local Networks, Intranets,
Internet, Mulitmedia or other special high speed networks.
 Computer
C t S Systems
t — Multimedia
M lti di D Desktop
kt machines,
hi
Workstations, MPEG/VIDEO/DSP Hardware
 Display Devices — CD-quality speakers, HDTV,SVGA,
Hi R
Hi-Resolution
l ti monitors,it C
Colour
l printers
i t etc.
t

Downloaded from CSIT Tutor

Applications
Examples of Multimedia Applications include:
 World Wide Web
 Hypermedia courseware
 Video conferencing
 Video-on-demand
 Interactive TV
 Groupware
 Home shopping
 G
Games
 Virtual reality
 Digital video editing and production systems

Downloaded from CSIT Tutor

Practical
Lab 1 and 2

Adobe Photoshop
 Welcome to Adobe Photoshop, the professional image-editing
standard for p
photographers,
g p ,pprofessional designers,
g , and g
graphics
p
producers.
 Photoshop provide a consistent work environment with other Adobe
applications including Adobe Illustrator, Adobe InDesign , Adobe
GoLive Adobe LiveMotion
GoLive, LiveMotion, Adobe After Effects
Effects, and Adobe
Premiere.
Objectives:
 By doing this Lab student will be able to edit various format of
I
Images andd give
i ththe various
i effects
ff t in
i iimages.
 We can use various exciting tools of Adobe Photoshop to edit
image, which includes

Downloaded from CSIT Tutor

Lab 3 and 4

Macromedia FreeHand
 Macromedia FreeHand is a vector-based drawing application. With
FreeHand, you can create vector graphics that can be scaled and printed at
anyy resolution,, without losing
g detail or clarity.
y
 We can use FreeHand to create print and web illustrations such as logos
and advertising banners. We can also use FreeHand to turn our artwork into
Macromedia Flash animations.
 The FreeHand user interface contains a workspace p and Tools ppanel that
are consistent with other Macromedia products such as Macromedia
Dreamweaver, Fireworks, and Flash, to give us a true integrated print and
web solution. In addition, now we can view and test our freeHand
documents in a Macromedia Flash Player window without ever leaving the
freeHand environment
environment.
Objectives:
 By doing this Lab student will able to design a various type of logos using
the different tools of Macromedia freehand.

Downloaded from CSIT Tutor

Lab 5 and 6

Macromedia Flash
 Flash is an authoring tool that designers and developers use to create presentations,
applications, and other content that enables user interaction. Flash projects can
include simple animations, video content, complex presentations, applications, and
everything in between
between. In general
general, individual pieces of content made with Flash are
called applications, even though they might only be a basic animation. You can make
media-rich Flash applications by including pictures, sound, video, and special effects.
 Flash is extremely well suited to creating content for delivery over the Internet
because its files are very small. Flash achieves this through its extensive use of
vector
t graphics.
hi V t graphics
Vector hi require
i significantly
i ifi tl lless memory andd storage
t space
than bitmap graphics because they are represented by mathematical formulas
instead of large data sets. Bitmap graphics are larger because each individual pixel in
the image requires a separate piece of data to represent it.
Objectives:
j
 By doing this Lab student will able to create different types of animation, use the
action script to control the various objects.

Downloaded from CSIT Tutor

Lab 7

Swish Max
 SWiSHmax is a new addition to the SWiSHzone family
of Flash authoring tools
tools. If you want to create stunning
and powerful Flash animations without using Flash, then
SWiSHmax is the tool for you.

Objectives:
 By doing this Lab student will able to create different
t
types off animation
i ti and d give
i diff
differentt ttype off effect
ff t to
t an
object.

Downloaded from CSIT Tutor

Lab 8 and 9
Adobe Premiere
 Adobe Premiere, a revolutionary nonlinear video-editing
pp
application that delivers a breakthrough g render-free
experience. Its high-performance toolset takes video and
audio production to a new level, giving a professional
edge. Adobe Premiere Pro delivers the power and
precision
i i you need d tto ttellll a story
t b
better
tt and d ffaster
t ththan
ever before.
Objectives:
 By
B doing
d i thithis Lab
L b student
t d t will ill able
bl tto edit
dit andd publish
bli h th
the
movie in various formats.

Downloaded from CSIT Tutor

Lab 10 and 11
Macromedia Director

 Macromedia Director MX is the most popular authoring tool of choice for

legions of web and multimedia developers
developers. With Director
Director, you can create
movies for websites, kiosks, and presentations as well as movies for
education and entertainment. Movies can be as small and simple as an
animated logo or as complex as an online chat room or game. Director
movies can include a variety of media, such as sound, text, graphics,
animation,
i ti and
d di
digital
it l video.
id A Di
Director
t moviei can lilink
k tto external
t l media
di or
be one of a series of movies that refer to one another actions.

Objective:
 This lab should be done after using all the tools of the multimedia. By doing
this lab student will be able to integrate all the multimedia objects like audio,
video , images etc and will able to create different interactive presentations.

Downloaded from CSIT Tutor

Project

Multimedia related project

Downloaded from CSIT Tutor

References books;

 Tay Vaughan, Multimedia, Making it work

 Ralf Steinmetz and Klara Nahrstedt , Multimedia:
Computing communication & applications
Computing,
 John Villamil-casanova and Louis Molina , Multimedia
production, Planning Delivery
 Lee Allis, Inside Macromedia Director
 John R. Nyquist and Robert Martin, Director 8 and Lingo
Bible
 Gary Rosenzweig and John Thompson, using
Macromedia Director 8

Downloaded from CSIT Tutor

Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Downloaded from CSIT Tutor
Video and Animation

Prof. S. Shakya

Downloaded from CSIT Tutor

Motion

 Both video and animation give us a sense of

motion
 They exploit some properties of human eye’s
ability of viewing pictures
 Motion video is the element of multimedia
that can hold the interest of viewers in a
presentation

Downloaded from CSIT Tutor

Visual Representation
 The visual effect of motion is due to a biological
phenomenon known as persistence of vision
 An object seen by the human eye remains mapped on the eye’s
retina for a brief time after viewing (approximately 25 ms)
 Another
A th phenomenon
h contributing
t ib ti to
t the
th vision
i i off motion
ti is
i
known as phi phenomenon
 When two light sources are close by and they are
illuminated in quick succession,
succession what we see is not two
lights but a single light moving between the two points
 Due to the above two phenomena of our vision system, a
discrete sequence of individual pictures can be perceived as
a continuous sequence

Downloaded from CSIT Tutor

Visual Representation
 Temporal aspect of Illumination—To represent
 visual reality, two conditions must be met the rate of
repetition of the images must be high enough to guarantee
smooth motion from frame to frame
 the rate must be high enough so that the persistence of
vision extends over the interval between flashes
 The frequency at which the flicking light source must be
repeated before it appears continuous is known as the
f i
fusion frequency
f
• This depends on the brightness of the light source
• The brighter the light source the higher the fusion frequency

 It is known that we perceive a continuous motion to happen

at any frame rate faster than 15 frames per second
 PAL television system
y has a frame rate of 25 frames/s

Downloaded from CSIT Tutor

Visual Representation

 Another problem known as flicker occurs

due to a periodic fluctuation of brightness
p
perception
p
 A technique known as interleaving
improves the view by
-dividing a frame into two fields, each contains
t ea
the alternative
te at e sca
scan lines,
es, a
and
d
- displaying the field in twice of the frames rate

Downloaded from CSIT Tutor

Video resolution
 The smallest detail that can be reproduced in the image is a
pixel
i l
 Practically, some of the scene inevitably fall between
scanning lines, so that two lines are required for such
picture elements
 Only about 70% of the vertical detail is presented by
the scanning lines
 Aspect ratio is the ratio of the picture width to height.
 It is 4:3 for conventional TV
 The picture width, horizontal resolution and the total detail
content of the image can be calculated

Downloaded from CSIT Tutor

Video resolution
 Conventional video systems have relative low resolution
- compare to computer screens: typical resolution of
 640 x 480, even up to 1024 X 768

 One consequence
q of this low resolution is that video played
p y
on computer screen are usually in a small window
 On the other hand, even with this low resolution, the amount
of data in video is huge
Consider PAL TV at 25 frames per second, if we sample at
352 x 288 with 16 bits per pixel, the raw video size is
352 x288 x16 x 25 = 40.55Mbit/s = 5Mbytes/s
-Compare this with a typical Ethernet bandwidth of 10Mbit/s
-or a double speed CD-ROM drive of 300Kbyte/s
 Therefore we need to compress the video data
Therefore,

Downloaded from CSIT Tutor

Digitalising Video

 We need to capture or digitize video for playing back on

comp ters or integrating into m
computers multimedia
ltimedia applications
 We need to take a lot of samples
 At 25 frames per second, each frame requires 1/25 = 40ms
 There 625 scan lines in each frame
frame, giving each scan line is
40ms/625 = 64µs
 At a horizontal resolution of 425 pixel, the time for sampling
each ppixel is 64µs/425
µ = 0.15µs
µ , i.e.,, sampling
p g rate is at least
7Mhz
 This requires very fast hardware
 Hardware required to capture video:
• Video sources: TV, VCR, LaserDisc player, Camcorder
• Video capture card
• Storage space: large hard disk

Downloaded from CSIT Tutor

Video capture cards
 There are many different video capture cards on the market
The common features in these cards are:
 Can accept composite video or S-VHS in NTSC or PAL;
high-end capture cards can accept digital video (DV)
 Video input mixer and ADC— to select/combine video
sources, to convert analog video signal to digital samples
 Video frame buffer— temporary storage for video frame

 Video processor— to filter or enhance the video frame, e.g.,

reduce noise, adjust brightness, contrast and colour
 Compressor/encoder—
p to compress
p and encode the digital
g
video into a required format
 Interface to the system PCI bus

Downloaded from CSIT Tutor

Video capture cards

Downloaded from CSIT Tutor

Video formats
 AVI (Audio Video Interleaved) format was defined by
Microsoft for its Video for Windows systems
 It supports video playback at up to 30 frames per second on
a small window (typical size 300X200 with 8 or 16 bit colour)
 It is a software-only system
 It supports a number of compression algorithms
 QuickTime was originally developed by Apple for storing
audio and video in Macintosh systems
 It supports video playback at up to 30 frames per second on
a small window (typical size 300X200 with 8 or 16 bit colour)
 It is a software-only system
 It supports a number of compression algorithms

Downloaded from CSIT Tutor

Animation
 To animate something is, literally, to bring it to life
 An animation covers all changes that have a
visual effect
 Visual effect can be of two major kinds:
 motion dynamic— time varying positions
 update dynamic— time varying shape, colour,
texture or even lighting,
texture, lighting camera position,
position etc
etc.
 The visual effects is the result of exploiting the
properties of human vision system as described
above (in the section about video)
 A computer animation is an animation performed
by a computer using graphical tools to provide
visual effects
12

Downloaded from CSIT Tutor

Input process
 The first step in producing computer animation is input
process
 Key frames have to be created and input into the computer
 Key frames are the frames in which the objects being
animated are at extreme or characteristic positions
 They can be drawn using traditional artistic tools, such as
pen and brush, and then digitised
 The digital images may need to be cleaned up
 They can also be created using drawing or painting tools
directly
 In composition stage, the foreground and background
figures are combined to generate the individual frames

Downloaded from CSIT Tutor

Inbetween process

 The animation of movement from one

position to another needs a composition
of frames with intermediate p
positions in
between the key frames
 The process of inbetweening is performed
in computer animation through
interpolation
p
 The system is given the starting and ending positions
 It calculates the positions in between

Downloaded from CSIT Tutor

Inbetween process
 The easiest interpolation is linear interpolation
 It has many limitations: the object does no move smoothly, look
unreal
 Spline interpolation can make object move more smoothly
 I b
Inbetweening
i also
l involves
i l interpolating
i l i the
h shapes
h off
objects
 Some animation involves changing the colour of objects
 This is
Thi i usually
ll done
d using
i colour
l look-up
l k table
bl (CLUT)
 By cycling through the colours in the CLUT, the objects’ colours will
change
 Morphing is a popular effect in which one image transforms
into another

Downloaded from CSIT Tutor

Controlling animation
 Full explicit control —the animator provides a description of
everything that occurs in the animation
 either by specifying simple changes, such as scaling,
transformation
 or by providing key frames
 Procedural control —using a program to calculate the position,
angle, etc. of the objects
 In physical systems, the position of one object may influence the
motion of another
 Constraint-based systems —movement of objects that are in
contact with each other is constraint by physical laws
 An animation can be specified by these constraints
 Tracking live action —
 People or animals act out the parts of the characters in the
animation
 The animator trace out the characters

Downloaded from CSIT Tutor

Controlling animation
 Kinematics refers to the position and velocity of points
 The ball is at the origin at time t = 0. It moves with a constant
acceleration in the direction (1,1,5) thereafter.
 The final result of an animation is the sum of all the steps. If
it does not fit
fit, the animator has to try again
again. This is known
as forward kinematics.
 Inverse kinematics (IK) is concerned with moving a
skeleton from one pose to another.
another
 The animator specifies the required position of the end effecter, the IK
algorithm will calculate the joint position, angle, etc.
 Dynamics
y takes into account the p
physical
y laws that g
govern
the masses and forces acting on the objects
 The ball is at the origin at time t = 0 second. It has a mass of 200 grams. The
force of gravity acts on it.

Downloaded from CSIT Tutor

Displaying animation
 The rules governing the showing of video apply to
animation as well
ell
 The frame rate should be at least 10, preferably 15 to 20, to
give a reasonably smooth effect
 There are basically three common ways to display
animation
 Generate a digital video clip
 Manyy Animation tools will export
p an animation in common digitalg
video format, e.g., QuickTime
 Create a package including runtime system of the animation
tool
 F example,
For l DiDirector
t can createt a projector
j t including
i l di allll casts.
t
The projector can then be distributed and play the animation.
 Show the animation in the animation tool

Downloaded from CSIT Tutor

Animation tools

 Macromedia Director and Flash

 It is one of the most popular interactive animation
tool for generating interactive multimedia
applications
 MetaCreations Poser
 It understands human motion and inverse kinematics, e.g., move an
arm the shoulders will follow.
 Discreet 3D Studio Max
 Very
y popular
p p for creating
g 3D animations
 Animation language—VRML (Virtual Reality
Modeling Language)

Downloaded from CSIT Tutor

Video formats
MPEG (Motion Picture Expect Group) is a working group under
ISO
 There are several versions of mpeg standard.
 The most commonly used now is mpeg-1

 It requires hardware support for encoding and decoding (on

slow systems)
 The maximum data rate is 1.5Megabit/sec
 The next generation mpeg-2 is now getting popular

 Mpeg-2 improves mpeg-1 by increasing the maximum data

rate to 15Mbit/sec
 It can interleave audio and video

Downloaded from CSIT Tutor

DATA COMPRESSION

Downloaded from CSIT Tutor

Wh C
Why Compress?
pr ?
 To reduce the volume of data to be
transmitted (text, fax, images)
 Too reduce
educe the
t e bandwidth
ba d dt required
equ ed for
o
transmission and to reduce storage
requirements (speech, audio, video)

Downloaded from CSIT Tutor

Data compression implies sending or storing a smaller
number of bits. Although many methods are used for this
purpose, in general these methods can be divided into two
broad categories: lossless and lossy methods.

Figure1 Data compression methods

Downloaded from CSIT Tutor

C pr i
Compression
 How is compression possible?
 Redundancy in digital audio, image, and video
data
 Properties of human perception
 Digital audio is a series of sample values;
image is a rectangular
l array off pixell
values; video is a sequence of images
played out at a certain rate
 Neighboring sample values are correlated

Downloaded from CSIT Tutor

R d d
Redundancy
 Adjacent audio samples are similar
(predictive encoding); samples
corresponding to silence (silence removal)
 In digital image, neighboring samples on a
scanning line are normally similar (spatial
redundancy)
 In digital video, in addition to spatial
redundancy,
d d neighboring
hb images in a
video sequence may be similar (temporal
redundancy)

Downloaded from CSIT Tutor

H
Human P
Perception
r pti F Factors
t r
 Compressed version of digital audio,
image, video need not represent the
original information exactly
 Perception sensitivities are different for
different signal patterns
 Human eye is less sensitive to the higher
spatial frequency components than the
l
lower ffrequencies (transform
( f coding)
d )

Downloaded from CSIT Tutor

Cl ifi ti
Classification
 Lossless compression
 lossless compression for legal and medical
documents, computer programs
 exploit only data redundancy
 Lossy compression
 digital audio, image, video where some errors
or loss can be tolerated
 exploit both data redundancy and human
perception properties
 Constant bit rate versus variable bit rate
coding

Downloaded from CSIT Tutor

E tr p
Entropy
 Amount of information I in a symbol of
occurring probability p : I = log2(1/p)
 Sy
Symbols
bo s that
t at occur
occu rarely
a e y convey
co ey a large
a ge
amount of information
 Average
g information p per symbol
y is called
entropy H
H = pix log2(1/pi) bits per codeword
 Average number of bits per codeword =
Nipi where Ni is the number of bits for
the symbol generated by the encoding
algorithm
Downloaded from CSIT Tutor
H ff
Huffman Coding
C di
 Assigns fewer bits to symbols that appear
more often and more bits to the symbols
that appear less often
 Efficient when occurrence probabilities
vary widely
 Huffman codebook from the set of
symbols and their occurring probabilities
 Two properties:
 generate compact codes
 prefix property

Downloaded from CSIT Tutor

R l th Coding
Run-length C di
 Repeated occurrence of the same
character is called a run
 Number
u be of
o repetition
epet t o is
s called
ca ed the
t e length
e gt of
o
the run
 Run of any
y length
g is represented
p byy three
characters
 eeeeeee7tnnnnnnnn
 @e7t@n8

Downloaded from CSIT Tutor

15.11 Figure 15.2 Run-length encoding example

Downloaded from CSIT Tutor

L p l Zi W l h (LZW) Coding
Lempel-Ziv-Welch C di
 Works by building a dictionary of phrases
from the input stream
 A to
token
e or
o an
a index
de is s used to identify
de t y
each distinct phrase
 Number of entries in the dictionary y
determines the number of bits required for
the index -- a dictionary with 25,000
words
d requires 15 b bits to encode
d the
h index
d

Downloaded from CSIT Tutor

Arith ti Coding
Arithmetic C di
 String of characters with occurrence
probabilities make up a message
 A co
complete
p ete message
essage may
ay be fragmented
ag e ted
into multiple smaller strings
 A codeword corresponding
p g to each string
g
is found separately

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
Cod g Requirements
Coding equ e e ts
 Let us consider the general
requirements imposed on most
multimedia
l i di systems:
 Storage — multimedia elements
require much more storage space
than simple text. For example, a full
screen true colour image is 640 x 480
X 3 = 921600 bytes

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 The size of one second of
uncompressed CD quality stereo
audio is 44.1kHz X 2 X 2 = 176400
bytes
 The size of one second of
uncompressed PAL video is
384X288X 3 X 25 frames = 8294400
bytes

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
Throughput — continuous media require
very large throughput. For example, an
uncompressed CD quality stereo audio
stream needs 176400 bytes/sec. A raw
digitized PAL TV signal needs(13.5MHz +
6 75MH + 6
6.75MHz 6.75MHz)
75MH ) X 8bits
8bit
= 216 X106bits/sec
= 27
2 X10 06Bytes/sec
/

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 Interaction — to support
pp fast interaction,, the
end-to-end delay should be small. A ‘face-to-face’
application, such as video conferencing, requires
the delayy to be less than 50ms. Furthermore,,
multimedia elements have to be accessed
randomly.
 Conclusion:
 Multimedia elements are very large.
 We need to reduce the data size using
compression.i

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
Kinds
ds o
of cod
coding
g methods
et ods
 Lossless — the compression process
does not reduce the amount of
i f
information.
i
- The original can be reconstructed
exactly
 Lossy — the compression process
reduces the amount of information.
-Only an approximation of the original
can be reconstructed.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
Categories of Compression Techniques

 Entropy coding is lossless

 Source coding and hybrid coding are
lossy.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
Coding techniques
 Vector Quantization — a data stream is divided into
blocks of n bytes (where n > 1). A predefined table
contains a set of patterns is used to code the data
blocks.
blocks
 LZW —a general compression algorithm capable of
working on almost any type of data. It builds a data
y of data occurring
dictionary g in an uncompressed
p data
stream. Patterns of data are identified and are
matched to entries in the dictionary. When a match is
found the code of the entry is output.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 Since the code is shorter than the data
pattern, compression is achieved. The
popular zip application used this method to
compress
p files.
 Differential coding — (also know as
prediction or relative coding) The most
known coding of this kind is DPCM (
Differential Pulse Code Modulation). This
method encodes the difference between the
consecutive samples instead of the sample
values. For example,

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 PCM 215 218 210 212 208 . . .
 DPCM 215 3 -8 2 -4 . . .
 DM ( Delta Modulation) is a
modification of DPCM. The difference
is coded with a single
g bit.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
Huffman coding
 The principle of Huffman coding is to
assign
ass g shorter
s o te code for
o symbol
sy bo that
t at
has higher probability of occurring in
the data stream.
 The length of the Huffman code is
optimal.

Downloaded from CSIT Tutor

A Huffman code tree is created using the
f ll i procedures
following d
 Two characters with the lowest p probabilities are
combined to form a binary tree.
 The two entries in the probability table is
replaced
l d by
b a new entryt whose
h value
l is
i the
th sum
of the probabilities of the two characters.
 Repeat the two steps above
 Assign 0 to be left branches and 1 to the right
branches of the binary tree.
 The Huffman code of each character can be read
from the tree starting from the root.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
JPEG
 JPEG (stands for Joint Photographic Experts
Group) is a joint ISO and CCITT (Comité
Consultatif International
Téléphonique et Télégraphique),
working group for developing standards for
compressing still images
 The JPEG image compression standard
became an international standard in 1992
 JPEG can be applied
pp to colour or g
grayscale
y
images

Downloaded from CSIT Tutor

LOSSY COMPRESSION METHODS

Our eyes
y and ears cannot distinguish
g subtle changes
changes.
g . In
such cases, we can use a lossy data compression method.
method.
These methods are cheaper
cheaper— —they take less time and
space when it comes to sending millions of bits per
second for images and video.
video. Several methods have
been developed using lossy compression techniques
techniques..
JPEG (Joint Photographic Experts Group) encoding is
used to compress pictures and graphics, MPEG
(Moving Picture Experts Group) encoding is used to
compress video, and MPMP33 (MPEG audio layer 3) for
audio compression
compression..

Downloaded from CSIT Tutor

Image
g compression
p – JPEG encodingg
an image can be represented by a two-dimensional array
(table) of picture elements (pixels).
A grayscale picture of 307,200 pixels is represented by
2,457,600 bits, and a color picture is represented by
7,372,800 bits.

IIn JPEG,
JPEG a grayscale l picture
i t i divided
is di id d into bl k off 8 × 8
i t blocks
pixel blocks to decrease the number of calculations because,
as we will see shortly,
shortly the number of mathematical
operations for each picture is the square of the number of
units.

Downloaded from CSIT Tutor

Figure 2 JPEG grayscale example, 640 × 480 pixels

Downloaded from CSIT Tutor

The whole idea of JPEG is to change the picture into a linear
(vector) set of numbers that reveals the redundancies. The
redundancies (lack of changes) can then be removed using
one of the lossless compression methods we studied
previously.
i l A simplified
i lifi d version
i off the
h process isi shown
h i
in
Figure 3.

Figure 3 The JPEG compression process

Downloaded from CSIT Tutor

Video compression – MPEG encoding
The Moving Picture Experts Group (MPEG) method is used
to compress video. In principle, a motion picture is a rapid
sequence of a set of frames in which each frame is a picture.
In other words, a frame is a spatial combination of pixels,
and a video is a temporal combination of frames that are sent
one after
ft another.
th Compressing
C i video,
id th
then, means spatially
ti ll
compressing each frame and temporally compressing a set of
frames.
frames

Downloaded from CSIT Tutor

Figure 4 MPEG frames

Downloaded from CSIT Tutor

Audio compression

Audio compression
p can be used for speech
p or music. For
speech we need to compress a 64 kHz digitized signal,
while for music we need to compress a 1.411 MHz signal.
Two categories of techniques are used for audio
compression: predictive encoding and perceptual
encoding.
encoding

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 By changing appropriate parameters,
the user can select

– the quality of the reproduced image

– compression processing time
– the size of the compressed image

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 The JPEG standard have three levels of definition
as follows:
 Baseline system — must reasonably decompress
colour images, maintain a high compression
ratio, and handle from 4bits/pixel to 16bits/pixel.
 Extended system — covers the various encoding
aspects such as variable length encoding,
encoding
progressive encoding, and hierarchical mode of
encoding.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 Special lossless function
function— ensures that at
the resolution at which the image is
compressed, decompression results in no
loss of any detail the was in the original
image.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 JPEG — Preparation
p
 A source image consists of at least one and
at most 255 planes.
 Each plane Ci may have different number of
pixels in the horizontal (Xi) and vertical (Yi)
dimension.
 Th resolution
The l ti off the
th individual
i di id l plane
l may
be different.
 Each ppixel is represented
p byy a number of
bits p where 2 ≤p ≤ 12.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 The meaning of the value in these
planes is not specified in the
standard.
 The image is divided into 8 X 8 blocks.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
MPEG
 MPEG (stands for Moving Picture Experts
Group) is also a joint ISO and CCITT
working
ki group for
f d
developing
l i standards
t d d forf
compressing still images
 The MPEG video compression standard
became an international standard in 1993
 MPEG uses technology defined in other
standards,
t d d such h as JPEG and d H.261
H 261

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 Itt defines
de es a bas
basicc data rate
ate o
of
1.2Mbits/sec
 It is suitable for symmetric as well
asymmetric i compression
i It
I follows
f ll
the reference scheme that consists of
four stages of processing:
1. Preparation
2. Processing
3. Quantization
4. Entropypy Encodingg

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 In the p
preparation
p stage,
g , unlike JPEG,, MPEG
defines the format of the images
 Each image consists of three components — YUV
 The luminance component has twice as many
samples in the horizontal and vertical axes as
the other two components (known as colour sub-
sampling
 The resolution of the luminance component
should not exceed 768 pixels

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 for each component, a pixel is coded with
eight bits

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
How MPEG encode the video stream
 In order to achieve higher compression
ratio, MPEG uses the fact the image on
consecutive
ti fframes diff
differ relative
l ti small.
ll It
uses a temporal prediction technique to
encode the frame so that the storage
requirement is greatly reduced.
 Common MPEG data stream consists of four
kinds of frames:

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 I-frame (
(Intra-frame)) — it is a self
contained frame, and it is coded without
reference to any other frames.
 P-frame
P frame (Predictive
(Predictive-coded
coded frame) — It is
coded using the predictive technique with
reference to the previous I-frame and/or
previous P
P-frame.
frame.
 B-frame (Bi-directionally predictive coded
frame) — It requires information of the
previous and following I-
I and P-frames
P frames for
encoding and decoding.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
 D frame (DC
D-frame (DC-coded
coded frame) Only the
lowest frequency component of image is
encoded. It is used in fast forward or fast
rewind.

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
MPEG-2
 MPEG-2 is a newer video encoding standard
which builds on MPEG-1
 It supports
pp higher
g video q
quality
y and higher
g
data rate (up to 80 Mbits/sec)
 It supports several resolutions:
 pixels/line line/frame frames/sec
 352 288 30
 720 576 30
 1920 1152 60

Downloaded from CSIT Tutor

D t Compression
Data C mpr i n
Summary
 Compression methods — lossless vs.
lossy
ossy
 Entropy coding — run-length
encoding,
g, Huffman encoding
g
 Source coding — prediction (DPCM,
DM), transformation (DCT)
 hybrid coding — JPEG, MPEG

Downloaded from CSIT Tutor

User interface design

Downloaded from CSIT Tutor

Obj ti
Objectives
 To suggest some general design principles for user interface design
 To explain different interaction styles and their use
 To explain when to use graphical and textual information
presentation
 To explain the principal activities in the user interface design
process
 To introduce usability attributes and approaches to system
evaluation

Downloaded from CSIT Tutor

T i covered
Topics d
 Design issues
 The user interface design process
 User analysis
 User interface prototyping
 Interface evaluation

Downloaded from CSIT Tutor

Th user iinterface
The t f
 User interfaces should be designed to match the skills,
experience
i andd expectations
t ti off itits anticipated
ti i t d users.
 System users often judge a system by its
interface rather than its functionality.
y
 A poorly designed interface can cause a user to make
catastrophic errors.
 Poor user interface
f design
d is the
h reason why h so many
software systems are never used.

Downloaded from CSIT Tutor

H
Human factors
f t in
i interface
i t f design
d ig
 Limited short-term memory
 People
P l can iinstantaneously
t t l remember
b about
b t 7 it
items off iinformation.
f ti If you
present more than this, they are more liable to make mistakes.
 People make mistakes
 When
Wh people l makek mistakes
i k andd systems go wrong, iinappropriatei alarms
l
and messages can increase stress and hence the likelihood of more mistakes.
 People are different
 People have a wide range of physical capabilities. Designers should not just
design for their own capabilities.
 People have different interaction preferences
 Some like pictures, some like text.

Downloaded from CSIT Tutor

UI design
d ig principles
i i l
 UI design must take account of the needs, experience and
capabilities
biliti off th
the system
t users.
 Designers should be aware of people’s physical and mental
limitations ((e.g.g limited short-term memory) y) and should
recognise that people make mistakes.
 UI design principles underlie interface designs although not
allll principles
i i l are applicable
li bl to
t allll designs.
d i

Downloaded from CSIT Tutor

U
User iinterface
t f design
d ig principles
i i l
Principle Description
User familiarity The interface should use terms and concepts which are drawn
from the experience of the people who will make most use of the
system.
system
Consistency The interface should be consistent in that, wherever possible,
comparable operations should be activated in the same way.
Minimal surprise
p Users should never be surprised
p byy the behaviour of a system.
y
Recoverability The interface should include mechanisms to allow users to
recover from errors.
User guidance The interface should provide meaningful feedback when errors
occur andd provide
id context-sensitive
t t iti user help
h l facilities.
f iliti
User diversity The interface should provide appropriate interaction facilities for
different types of system user.

Downloaded from CSIT Tutor

D ig principles
Design i i l
 User familiarity
 The
Th interface
i t f should
h ld be b based
b d on user-oriented
i t d
terms and concepts rather than computer concepts. For example, an office
system should use concepts such as letters, documents, folders etc. rather
than directories, file identifiers, etc.
 Consistency
 The system should display an appropriate level
y Commands and menus should have the same format,,
of consistency.
command punctuation should be similar, etc.
 Minimal surprise
 If a command operates in a known way, the user should be
able to predict the operation of comparable commands

Downloaded from CSIT Tutor

D ig principles
Design i i l
 Recoverability
 The
Th system
t should
h ld provide
id some resilience
ili tto
user errors and allow the user to recover from errors. This might include an
undo facility, confirmation of destructive actions, 'soft' deletes, etc.
 User guidance
 Some user guidance such as help systems, on-line manuals, etc. should be
supplied
 User diversity
 Interaction facilities for different types of user should be supported. For
example, some users have seeing difficulties and so larger text should be
available

Downloaded from CSIT Tutor

D ig issues
Design i iin UI
UIs
 Two problems must be addressed in interactive systems design
 How should information from the user be provided to the computer system?
 How should information from the computer system be presented to the
user?
 User interaction and information presentation may be integrated
through a coherent framework such as a user interface metaphor.

Downloaded from CSIT Tutor

I t
Interaction
ti styles
t l
 Direct manipulation
 Menu selection
 Form fill-in
 Command language
 Natural language

Downloaded from CSIT Tutor

Interaction styles
Interaction Main advantages Main disadvantages Application
style examples
Direct Fast and intuitive May be hard to implement. Video games
manipulation interaction Only suitable where there is a CAD systems
Easy to learn visual metaphor for tasks and
objects
objects.
Menu Avoids user error Slow for experienced users. Most general-
selection Little typing required Can become complex if many purpose systems
menu options.
Form fill-in
fill in Simple data entry Takes up a lot of screen space.
space Stock control,
control
Easy to learn Causes problems where user Personal loan
Checkable options do not match the form processing
fields.
Command Powerful and flexible Hard to learn. Operating
p g systems,
y ,
language Poor error management. Command and
control systems
Natural Accessible to casual Requires more typing. Information
language users Natural language und erstanding retrieval systems
E il extended
Easily t d d systems
t are unreliable.
li bl

Downloaded from CSIT Tutor

M lti l user interfaces
Multiple i t f

Downloaded from CSIT Tutor

LIBSYS interaction
 Document search
 Users need to be able to use the search facilities to find the
documents that they need.
 Document request
 Users request that a document be delivered to their machine or
to a server for printing.

Downloaded from CSIT Tutor

Web based interfaces
Web-based
 Many web-based systems have interfaces based on web
forms.
 Form field can be menus, free text input, radio buttons, etc.
 In
I the
h LIBSYS example,l users makek a choice
h off where
h to
search from a menu and type the search phrase into a free
text field..

Downloaded from CSIT Tutor

LIBSYS search form

Downloaded from CSIT Tutor

Information presentation
 Information presentation is concerned with presenting
system
t iinformation
f ti tot system
t users.
 The information may be presented directly (e.g. text in a
word pprocessor)) or mayy be transformed in some wayy for
presentation (e.g. in some graphical form).
 The Model-View-Controller approach is a way of supporting
multiple
lti l presentations
t ti off ddata.
t

Downloaded from CSIT Tutor

I f
Information
ti presentation
t ti

Downloaded from CSIT Tutor

Model view controller
Model-view-controller

Downloaded from CSIT Tutor

I f
Information
ti presentation
t ti
 Static information
 Initialised at the beginning of a session. It does not change
during the session.
 May be either numeric or textual.
textual
 Dynamic information
 Changes
g duringg a session and the changes
g must be
communicated to the system user.
 May be either numeric or textual.

Downloaded from CSIT Tutor

I f
Information
ti display
di l factors
f t
 Is the user interested in precise information or
ddata relationships?
l i hi ?
 How quickly do information values change?
Must the changeg be indicated immediately?y
 Must the user take some action in response to
a change?
 Is there a direct manipulation interface?
 Is the information textual or numeric? Are relative values
important?

Downloaded from CSIT Tutor

Alt
Alternative
ti information
i f ti presentations
t ti

Downloaded from CSIT Tutor

Analogue or digital presentation?
 Digital presentation
 Compact - takes up little screen space;
 Precise values can be communicated.
 Analogue
A l presentation
t ti
 Easier to get an 'at a glance' impression of a value;
 Possible to show relative values;
 Easier to see exceptional data values.

Downloaded from CSIT Tutor

P
Presentation
t ti methods
th d

Downloaded from CSIT Tutor

Di l i g relative
Displaying l ti values
l

Downloaded from CSIT Tutor

D t visualisation
Data i li ti
 Concerned with techniques for displaying large amounts of
information
information.
 Visualisation can reveal relationships between entities and trends
in the data.
 Possible data visualisations are:
 Weather information collected from a number of sources;
 The state of a telephone
p network as a linked set of nodes;
 Chemical plant visualised by showing pressures and temperatures in a linked
set of tanks and pipes;
 A model of a molecule displayed in 3 dimensions;
 Web pages displayed as a hyperbolic tree.

Downloaded from CSIT Tutor

C l
Colour di
displays
l
 Colour adds an extra dimension to an interface and can help
the user understand complex information structures.
 Colour can be used to highlight exceptional events.
 Common
C mistakes
i k iin theh use off colour
l iin
interface design include:
 The use of colour to communicate meaning;
 The over-use of colour in the display.

Downloaded from CSIT Tutor

C l
Colour use gguidelines
id li
 Limit the number of colours used and be conservative in
their
th i use.
 Use colour change to show a change in system status.
 Use colour coding to support the task that users are trying to
perform.
 Use colour coding in a thoughtful and consistent way.
 Be careful about colour pairings.

Downloaded from CSIT Tutor

E
Error messages
g
 Error message design is critically important.
Poor error messages can mean that a user
rejects rather than accepts a system.
 Messages should be polite,
polite concise,
concise consistent and
constructive.
 The background
g and experience
p of users
should be the determining factor in message
design.

Downloaded from CSIT Tutor

Design factors in message wording
Factor Description
Context Wherever possible, the messages generated by the system should reflect the current
user context. As far as is possible, the system should be aware of what the user is doing
and should generate messages that are relevant to their current activity.
Experience As u sers become familiar with a s ystem they become irritated by long,
long ÔmeaningfulÕ
messages. However, beginners find it difficult to understand short terse statements of a
problem. You should provide both types of message and allow the user to control
message conciseness.
Skill level Messages
g should be tailored to the userÕsskills as well as t heir experience.
p Messages
g
for the different classes of user may be ex pressed in d ifferent ways depending on the
terminology that is familiar to the reader.
Style Messages should be positive rather than negative. They should use the active rather
than the passive mode of address. They should never be insulting or try to be funny.
Culture Wherever possible, the designer of messages should be familiar with the culture of the
country where the system is sold. There are distinct cultural differences between
Europe, Asia and America. A su itable message for one culture might be unacceptable
in another.

Downloaded from CSIT Tutor

User error
 Assume that a nurse misspells the name of a patient whose
records he is trying to retrieve.

Downloaded from CSIT Tutor

Good and bad message design

Downloaded from CSIT Tutor

The UI design process
 UI design is an iterative process involving close liaisons
bbetween
t users andd designers.
d i
 The 3 core activities in this process are:
 User analysis.
y Understand what the users will do with the
system;
 System prototyping. Develop a series of prototypes for
experiment;
 Interface evaluation. Experiment with these prototypes with
users.

Downloaded from CSIT Tutor

The design process

Downloaded from CSIT Tutor

User analysis
 If you don’t understand what the users want to do with a
system, you have no realistic prospect of designing an
effective interface.
 User analyses have to be described in terms that users and
other designers can understand.
 Scenarios where yyou describe typical
yp episodes
p of use,, are one
way of describing these analyses.

Downloaded from CSIT Tutor

User interaction scenario

Jane is a student of Religious Studies and is working on an essay

on Indian architecture and how it has been influenced by religious
practices To help her understand this
practices. this, she would like to access
some pictures of details on notable buildings but can’t find
anything in her local library.

She approaches the subject librarian to discuss her needs and he

suggests some search terms that might be used. He also suggests
some libraries in New Delhi and London that might have this
material so they log on to the library catalogues and do some
searching using these terms. They find some source material and
place a request for photocopies of the pictures with architectural
detail to be posted directly to Jane.

Downloaded from CSIT Tutor

Requirements from the scenario
 Users may not be aware of appropriate search terms so need
a way of helping them choose terms.
 Users have to be able to select collections to search.
 Users
U needd to be
b able
bl to carry out searches
h andd request
copies of relevant material.

Downloaded from CSIT Tutor

Analysis techniques
 Task analysis
 Models the steps involved in completing a task.
 Interviewing and questionnaires
 Asks
A k the
h users about
b theh workk they
h ddo.
 Ethnography
 Observes the user at work.
work

Downloaded from CSIT Tutor

Hierarchical task analysis

Downloaded from CSIT Tutor

Interviewing
 Design semi-structured interviews based on open-ended
questions.
 Users can then provide information that they think is
essential; not just information that you have thought of
collecting.
 Groupp interviews or focus groups
g p allow users to discuss with
each other what they do.

Downloaded from CSIT Tutor

Ethnography
 Involves an external observer watching users at work and
questioning them in an unscripted way about their work.
 Valuable because many user tasks are intuitive and they find
these very difficult to describe and explain.
explain
 Also helps understand the role of social and organisational
influences on work..

Downloaded from CSIT Tutor

Ethnographic records

Air traffic control involves a number of control ‘suites’ where the suites
controlling adjacent sectors of airspace are physically located next to
each other. Flights in a sector are represented by paper strips that are
fitted into wooden racks in an order that reflects their position in the
sector. If there are not enough slots in the rack (i.e. when the airspace
is very busy), controllers spread the strips out on the desk in front of the
rack.
rack
When we were observing controllers, we noticed that controllers
regularly glanced at the strip racks in the adjacent sector. We pointed
this out to them and asked them why they did this.
this They replied that
that, if
the adjacent controller has strips on their desk, then this meant that
they would have a lot of flights entering their sector. They therefore tried
to increase the speed of aircraft in the sector to ‘clear space’ for the
i
incoming
i aircraft.
i ft

Downloaded from CSIT Tutor

Insights from ethnography
 Controllers had to see all flights in a sector. Therefore,
scrolling displays where flights disappeared off the top or
bottom of the display should be avoided.
 The interface had to have some way of telling controllers
how many flights were in adjacent sectors so that they could
pplan their workload.

Downloaded from CSIT Tutor

User interface prototyping
 The aim of prototyping is to allow users to gain direct
experience
i with
ith the
th iinterface.
t f
 Without such direct experience, it is impossible to judge the
usabilityy of an interface.
 Prototyping may be a two-stage process:
 Early in the process, paper prototypes may be used;
 The
Th ddesign is then
h refined
f d andd increasingly
l sophisticated
h d
automated prototypes are then developed.

Downloaded from CSIT Tutor

Paper prototyping
 Work through scenarios using sketches of the interface.
 Use a storyboard to present a series of interactions with the
system.
 Paper
P prototyping is an effective
ff way off getting user
reactions to a design proposal.

Downloaded from CSIT Tutor

Prototyping techniques
 Script-driven prototyping
 Develop
l a set off scripts andd screens using a tooll suchh as
Macromedia Director. When the user interacts with these, the
screen changes to the next display.
 Visual programming
 Use a language designed for rapid development such as Visual
Basic. See Chapter 17.
 Internet-based prototyping
 Use a web browser and associated scripts.

Downloaded from CSIT Tutor

U
User interface
i t f evaluation
l ti
 Some evaluation of a user interface design
should be carried out to assess its suitability.
 Full scale evaluation is very expensive and
impractical for most systems.
systems
 Ideally, an interface should be evaluated against a usability
specification.
p . However,, it is rare for such specifications
p to be
produced.

Downloaded from CSIT Tutor

U bilit attributes
Usability tt ib t

Attribute Description
Learnability How long does it ta ke a new user to become productive with
the system?
Speed of operation How well does the system response match the userÕs work
practice?
Robustness How tolerant is the system of user error?
Recoverability How good is the system at recovering from user errors?
Adaptability How closely is the system tied to a single model of work?

Downloaded from CSIT Tutor

Si l evaluation
Simple l ti ttechniques
h i
 Questionnaires for user feedback.
 Video recording of system use and subsequent tape
evaluation.
 Instrumentation
I i off code
d to collect
ll iinformation
f i about
b facility
f ili
use and user errors.
 The provision of code in the software to collect on-line
on line user
feedback.

Downloaded from CSIT Tutor

K points
Key i t
 User interface design principles should help guide the design of
user interfaces.
i f
 Interaction styles include direct manipulation, menu systems form
fill-in,, command languages
g g and natural language.
g g
 Graphical displays should be used to present trends and
approximate values. Digital displays when precision is required.
 Colour should be used sparingly and consistently.

Downloaded from CSIT Tutor

K points
Key i t
 The user interface design process involves user analysis, system
prototyping and prototype evaluation.
evaluation
 The aim of user analysis is to sensitise designers to the ways in
which users actually work.
 UI prototyping should be a staged process with early paper
prototypes used as a basis for automated prototypes of the
interface.
 The goals of UI evaluation are to obtain feedback on how to
improve the interface design and to assess if the interface meets its
usability requirements.
requirements

Downloaded from CSIT Tutor

User Interfaces
• Multimedia user interfaces are computer
interfaces that communicate with users
using multiple media, sometimes using
multiple
p modes such as written text together
g
with spoken language.
• GUI
GUI-using
using the mouse as the main input
device-have greatly simplified human-
machine interaction
interaction.

Downloaded from CSIT Tutor

User Interfaces
• General design
g issues
• The main emphasis in the design of multimedia user
interfaces is multimedia presentation. There are several
issues which must be considered:
1. To determine the appropriate information content to be
communicated
2. To represent the essential characteristics of the
information.
3. To represent
p the communicative intent.
4. To chose the proper media for information presentation.
5. To coordinate different media and assembling
t h i
techniques within
ithi a presentation.
t ti
6. To provide interactive exploration of information
presented.
Downloaded from CSIT Tutor
Abstractions for Programming

By
Prof. S. Shakya

Downloaded from CSIT Tutor

Overview

The state of the art of programming

• Most of the current commercially available multimedia
applications are implemented in procedure‐oriented
programming languages
• Application code is still highly dependent on hardware
• Change of multimedia devices still often requires re‐
i l
implementation
t ti
• Common operating system extensions try to attack
these problems
p
• Different programming possibilities for accessing and
representing multimedia data

Downloaded from CSIT Tutor

Overview of different abstraction levels

• Libraries
• System software
• Toolkits
lki
• Higher Programming languages
• Object‐oriented approaches

Downloaded from CSIT Tutor

Abstraction for Programming

Abstraction Levels of the Programming of Multimedia Systems

Downloaded from CSIT Tutor

Abstractions from Multimedia
Hardware
d
Strong hardware dependency may cause
problems with:
• Portability
bili
• Reusability
• Coding efficiency

Downloaded from CSIT Tutor

Abstraction Levels
• Common operating system extensions try to
solve this problem
• Different programming possibilities for
accessing and representing multimedia data

Downloaded from CSIT Tutor

Libraries
Processing of continuous media based on
functions embedded in libraries
• Libraries differ in their degree of abstraction

Downloaded from CSIT Tutor

Libraries ‐ OpenGL
2D and 3D graphics API developed by Silicon Graphics
• Basic idea: “write applications once, deploy across many platforms”:
 PCs
 Workstations
 Super Computers
• Benefits:
 Stable
 Reliable and Portable
 Evolving
 Scalable (Features like Zoom, Rectangle handling ...)
 Well documented and easyy to use
• Integrated with:
 Windows 95/NT/2000/XP
 UNIX X Window System

Downloaded from CSIT Tutor

System Software
 Device access becomes part of the operating system:
 D t as time
Data ti capsules
l (file
(fil extensions)
t i )
• Each Logical Data Unit (LDU) carries in its time capsule its data type, actual
• value and valid life span
• Useful conceptp for video,, where each frame has a valid life span
p of 40ms (rate
(
• of read access during a normal presentation)
• Presentation rate is changed for VCR (Video Cassette Recorder) functions like
• fast forward, slow forward or fast rewind by
‐ Changing the presentation life span of a LDU
‐Skipping of LDUs or repetition of LDUs
 Data as streams
‐ a stream denotes the continuous flow of audio and video data between a
source and a sink
‐ Prior to the flow the stream is established equivalent to the setup of a
connection in a networked environment

Downloaded from CSIT Tutor

System Software: Windows Media
Controll Interface
f ((MCI):
)

MMSYSTEM library for

extensibility and device
independence

Downloaded from CSIT Tutor

System Software ‐ DirectX
• Low‐level
o e e APIss aand d libraries
b a es for
o high‐performance
g pe o a ce
applications
p y ggames ‐ formerlyy known as the "Game
• Especially
SDK"
• Direct access to hardware services
• E.g. audio & video cards, hardware accelerators
• “DirectX” = “direct access”
• Strong relationship/interaction with
ActiveX/DCOM

Downloaded from CSIT Tutor

System Software ‐ DirectX

Downloaded from CSIT Tutor

System Software ‐ DirectX
Components:
• DirectDraw ‐ 2 dimensional graphics capabilities
• Direct3D ‐ extensively functional 3D graphics programming API
• DirectSound ‐ (3D) sound, mixing and playback of multiple streams
• DirectPlay ‐ for network multiplayer game development
• DirectInput ‐ input from various peripherals, e.g. joysticks, data gloves
Implementation Strategy:
• Hardware Abstraction Layer (HAL)
• Hardware Emulation Layer (HEL)
• Media Layer (for aggregated “high level” functionality)
 Animations
 Media streaming
 Synchronization

Downloaded from CSIT Tutor

Toolkits
Simpler approach than the system software
interface from the users point of view are
toolkits:
• Abstract from the actual physical layer
• Allow a uniform interface for communication
with all different devices of continuous media
• Introduce the client‐server
client server paradigm
• Can be embedded into programming
languages or object‐oriented
object oriented environments

Downloaded from CSIT Tutor

Higher Programming Languages
Media as data types:
• Definition of appropriate data types (e.g. for
video and audio)
• Smallest unit can be a LDU
• Example of merging a text and a motion
picture:

Downloaded from CSIT Tutor

Higher Programming Languages
Media as files:
• instead of considering continuous media as data types they
can be considered as files:
file_h1 = open(MICROPHONE_1,...)
fil h2 = open(MICROPHONE_2,...)
file_h2 (MICROPHONE 2 )
file_h3 = open(SPEAKER, ...)
...
read(file h1)
read(file_h1)
read(file_h2)
mix(file_h3, file_h1, file_h2)
activate(file_h1, file_h2, file_h3)
...
deactivate(file_h1, file_h2, file_h3)
...
rc1 = close(file_h1)
close(file h1)
rc2 = close(file_h2)
rc3 = close(file_h3)

Downloaded from CSIT Tutor

Programming Language Requirements
• The high‐level language should support parallel processing, because
th processing
the i off continuous
ti d
data
t iis
• controlled by the language through pure asynchronous instructions
• an integral part of a program through the identification of media
Different
ff processes must bbe able
bl to communicate through
h h an inter‐
process communication mechanism, which must be able to:
• Understand a priori and/or implicitly specified time requirements
(QoS parameters or extracted from the data type)
• Transmit the continuous data according to the requirements
• Initiate the processing of the received continuous process on time

Downloaded from CSIT Tutor

Object‐Oriented
Object Oriented Approaches
Basic ideas of object‐oriented programming is data
encapsulation in connection with class and object
definitions
 Abstract Type Definition (definition of data types
through abstract interfaces)
 Class (implementation of a abstract data type)
 Object
Obj t (instance
(i t off a class)
l )
Other important properties of object‐oriented systems
are:
 Inheritance
 Polymorphism

Downloaded from CSIT Tutor

Object‐Oriented
Object Oriented Approaches
• Devices as classes: devices are assigned
g to objects
j which
represent their behavior and interface

Downloaded from CSIT Tutor

Devices as classes

Downloaded from CSIT Tutor

Object‐Oriented
Object Oriented Approaches
Processing units as classes:
• Three main objects:
 Source objects
 Destination objects
 Combined source‐destination objects allows the creation of data
flow paths through connection of objects

• Multimedia object
 Basic Multimedia Classes (BMCs) / Basic Multimedia Objects
(BMOs)
 Compound
C dM
Multimedia
l i di ClClasses (CMC
(CMCs)) / Compound
C d Multimedia
M l i di
Objects (CMO), which are compound of BMCs / BMOs and other
CMCs/CMOs
 BMOs and CMOs can be distributed over different computer nodes

Downloaded from CSIT Tutor

Object‐Oriented
Object Oriented Approaches
Media as classes:
• Media Class Hierarchies define hierarchical
relations for different media
• Different class hierarchies are better suited for
diff
different applications
li i