Fundamentals of Multimedia, 2nd ed.

Chapter 2

Chapter 2
A Taste of Multimedia
2.1 Multimedia Tasks and Concerns
2.2 Multimedia Presentation
2.3 Data Compression
2.4 Multimedia Production
2.5 Multimedia Sharing and Distribution
2.6 Some Useful Editing and Authoring Tools

1 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

2.1 Multimedia Tasks and Concerns

• More basic concerns will impact multimedia as it now appears in
products e.g., we expect impact from Computer Vision: a camera
owner might be encouraged to think like a computer scientist and ask:
- “What is going on in an image?”
- A less high-level question is “Where has this image been taken?”
(scene recognition)
- “Does the image contain a particular object?” (object classification)
- “Where is an object of interest?” (object detection)
- “Which object does each pixel belong to?” (image segmentation)

== a classic Computer Vision hierarchy of high-level to detailed

description of an image, with scene recognition at the top and image
segmentation at the bottom.

2 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

2.2 Multimedia Presentation

• Graphics Styles: Human visual dynamics impact how

presentations must be constructed.

a) Color principles and guidelines: Some color schemes

and art styles are best combined with a certain theme or
style. A general hint is to not use too many colors, as
this can be distracting.

b) Fonts: For effective visual communication in a

presentation, it is best to use large fonts (i.e., 18 to 36
points), and no more than 6 to 8 lines per screen (fewer
than on this screen!). Fig. 2.1 shows a comparison of
two screen projections:

3 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Fig. 2.1: Colours and fonts [from Ron Vetter].

4 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

c) A color contrast program: If the text color is some triple

(R,G,B), a legible color for the background is that color
subtracted from the maximum (here assuming max=1):

(R, G, B)  (1 − R, 1 − G, 1 − B) (2.1)

• Some color combinations are more pleasing than

others; e.g., a pink background and forest green
foreground, or a green background and mauve
foreground.

• Fig. 2.2 shows a small Java program in operation:

Link to TextColor [Link]

Link to [Link]
5 Li, Drew, & Liu © Springer 2021
Fundamentals of Multimedia, 2nd ed. Chapter 2

Fig. 2.2: Program to investigate colors and readability.

6 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

- Fig. 2.3, shows a “colour wheel”, with opposite colors

equal to (1-R, 1-G, 1-B)

Fig. 2.3: Colour wheel

7 Li, Drew, & Liu © Springer 2021
Fundamentals of Multimedia, 2nd ed. Chapter 2

Sprite Animation
• The basic idea: Suppose we have an animation figure,
as in Fig. 2.4 (a). Now create a 1-bit mask M, as in Fig.
2.4 (b), black on white, and accompanying sprite S, as
in Fig. 2.4 (c).

(a) (b) (c)

Fig. 2.4: Sprite creation: Original, mask image M, and sprite S

(“Duke” figure courtesy of Sun Microsystems.)

8 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

• We can overlay the sprite on a colored background B,

as in Fig. 2.5 (a) by first ANDing B and M, and then
ORing the result with S, with final result as in Fig. 2.5
(e).

(a) (b) (c)

(d) (e)
Fig. 2.5: Sprite animation: (a): Background B. (b): Mask M
(c): B AND M. (d): Sprite S (e): B AND M OR S
9 Li, Drew, & Liu © Springer 2021
Fundamentals of Multimedia, 2nd ed. Chapter 2

Video Transitions

• Video transitions: to signal “scene changes”.

• Many different types of transitions:

1. Cut: an abrupt change of image contents formed by
abutting two video frames consecutively. This is the
simplest and most frequently used video transition.

10 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

2. Wipe: a replacement of the pixels in a region of the

viewport with those from another video. Wipes can be
left-to-right, right-to-left, vertical, horizontal, like an
iris opening, swept out like the hands of a clock, etc.

3. Dissolve: replaces every pixel with a mixture over

time of the two videos, gradually replacing the first by
the second. Most dissolves can be classified as two
types: cross dissolve and dither dissolve.

11 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Type I: Cross Dissolve

• Every pixel is affected gradually. It can be
defined by:

(1 − α (t ))·A + α (t )·B
D= (2.2)

where A and B are the color 3-vectors for video A

and video B. Here, α(t) is a transition function,
which is often linear:

=α (t ) k·t ,    with k·tmax ≡ 1 (2.3)

12 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Type II: Dither Dissolve

• Determined by α(t), increasingly more and more pixels

in video A will abruptly (instead of gradually as in Type
I) change to video B.

13 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

• Fade-in and fade-out are special types of Type

I dissolve: video A or B is black (or white).
Wipes are special forms of Type II dissolve in
which changing pixels follow a particular
geometric pattern.

• Build-your-own-transition: Suppose we wish to

build a special type of wipe which slides one
video out while another video slides in to
replace it: a slide (or push).

14 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

(a) Unlike a wipe, we want each video frame not be

held in place, but instead move progressively
farther into (out of) the viewport.

(b) Suppose we wish to slide VideoL in from the left,

and push out VideoR. Figure 2.6 shows this process:

(a) (b) (c)

Fig. 2.6: (a): VideoL. (b): VideoR. (c): VideoL sliding into
place and pushing out VideoR.

15 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Slide Transition (Cont’d)

• As time goes by, the horizontal location xT for the
transition boundary moves across the viewport from xT =
0 at t = 0 to xT = xmax at t = tmax. Therefore, for a transition
that is linear in time,
xT = (t/tmax)xmax

• So for any time t the situation is as shown in Fig. 2.7

(a). Let’s assume that dependence on y is implicit since
we use the same y as in the source video. Then for the
red channel (and similarly for the green and blue), R =
R(x, t).

16 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

• Suppose that we have determined that pixels should

come from VideoL.
Then the x-position xL in the unmoving video should be,
xL = x + (xmax − xT),where x is the position we are trying to
fill in the viewport, xT is the position in the viewport
that the transition boundary has reached, and xmax is the
maximum pixel position for any frame.

• From Fig. 2.7(b), we can calculate the position xL in

VideoL’s coordinate system as the sum of the distance x,
in the viewport, plus the difference xmax − xT .

17 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Fig. 2.7:
a) Geometry of VideoL pushing out VideoR.
b) Calculating position in VideoL from where pixels are copied to
the viewport.
18 Li, Drew, & Liu © Springer 2021
Fundamentals of Multimedia, 2nd ed. Chapter 2

Slide Transition (Cont’d)

• Substituting the fact that the transition moves linearly

with time, xT = xmax(t/tmax), a pseudocode solution in
shown in Fig. 2.8.

19 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

2.3 Data Compression

20 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Fig. 2.9: JPEG compression: (a) original uncompressed

image; (b) JPEG compression with Quality Factor qf = 75
(the typical default).

21 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Fig. 2.9: JPEG compression: (c) Quality Factor qf = 25

and (d) Quality Factor qf = 5.

22 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Table 2.2: JPEG file sizes (bytes) and percentage

size of data for JPEG compression with Quality Factor
qf = 75, 25, and 5.

23 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

• How expensive image and video processing is in terms of

CPU cycles?

• Suppose we have an image whose pixels we wish to

darken, by a factor of 2:

On a RISC machine, 12 instructions per pixel, i.e., per 3 bytes.

24 Li, Drew, & Liu © Springer 2021
Fundamentals of Multimedia, 2nd ed. Chapter 2

2.4 Multimedia Production

• Multimedia production can easily involve an art

director, graphic designer, production artist,
producer, project manager, writer, user
interface designer, sound designer,
videographer, and 3D and 2D animators, as well
as programmers.

25 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

2.5 Multimedia Sharing and Distribution

Fig. 2.10: The webpage

for uploading a YouTube
video. The video, titled
“Edison Phonograph
Multimedia Textbook 3rd
Edition” is open to all
users (Privacy settings:
Public) and can be
searched in the YouTube
homepage using the
title or tags. Note that
the video thumbnails
are automatically
generated by YouTube.

26 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Fig. 2.11: The YouTube

page for the video
uploaded. The list of
related videos are
shown on the right side,
and users can post their
comments, as well.

27 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

2.6 Some Useful Editing and Authoring Tools

• One needs real vehicles for showing understanding

principles of and creating multimedia. And straight
programming in C++ or Java is not always the best way
of showing your knowledge and creativity.

• Some popular authoring tools include the following:

- Adobe Premiere
- HTML5 Canvas
- Adobe Director
- Adobe XD

• Hint for studying this section: Hands-on work in a

laptop or Lab environment, with reference to the text.
28 Li, Drew, & Liu © Springer 2021
Fundamentals of Multimedia, 2nd ed. Chapter 2

Adobe Premiere
• Used for video editing and exporting. Features a
“score” authoring metaphor, placing video clips in
horizontal tracks that resemble a musical score.

Fig. 2.12: Adobe

Premiere Screen.

29 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Adobe Premiere
• As a professional video editor, Premiere offers
many controls for exporting and compressing the
final product.

Fig. 2.13: Adobe

Premiere export
setting screen.

30 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Adobe Premiere
• Premiere allows imports of .psd (Photoshop
files), which can have alpha channels which can
crop specific pixel in the picture.

Fig. 2.14: Adobe Premiere clip viewer.

31 Li, Drew, & Liu © Springer 2021
Fundamentals of Multimedia, 2nd ed. Chapter 2

HTML5 Canvas
• HTML5 canvas is an element used to draw
animated graphics on a webpage.
• The <canvas> element is a container for
graphics.
• A rectangular area that generated by HTML code
with width and height attributes.
• Using JavaScript, we can actually draw the
graphical objects.

32 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Create a Canvas
<canvas id=“canvas” width=“300” height=“300” style=“background-color:
red”> </canvas>

var canvas = [Link](‘canvas’);

var ctx = [Link](‘2d’);

• The background color is transparent by default; here

we make it red to be visible.
• The canvas is filled by grids, acting as an x-y
coordinate system.

33 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Draw Graphical Objects

• You can draw simple shapes, text, images, path or
other objects on the canvas.

// a rectangle with filled color

[Link](x, y, width, height)

// a rectangle with border

[Link](x, y, width, height)

// clear a rectangle area

[Link](x, y, width, height)

34 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Transform and Animation

• The shapes of the graphics can be transformed
by
• Translate

• Rotate

• Scale

35 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Steps to Create an Animation

1. clear the canvas: [Link](x, y, width, height)

2. save the current state of canvas: [Link]()

3. draw the graphics

4. restore the canvas state: [Link]()

36 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Adobe Director
• Director is a complete environment for creating
interactive “movies”.

37 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Director Windows
• Main window
• Stage, on which the action takes place.
• Cast window
• Consists of resources a movie may use, such
as bitmaps, sounds, vector-graphics shapes,
Flash movies, digital videos, and scripts.
• Score window
• Organized in horizontal lines, each for one of
the sprites, and vertical frames.

38 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Animation
• Animation is created by showing slightly different
images over time.
• Using different cast members in different frames.
• To control this process more easily, Director permits
combining many cast members into a single sprite.

Fig. 2.17: A tweened sprite.

Because several instances
may be used for a single cast
member, each instance is
called a sprite. Typically, cast
members are raw media,
whereas sprites are objects
that control where, when, and
how cast members appear on
the stage and in the movie.

39 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Adobe XD
• Used for making prototypes of websites and mobile
apps without any code.

Fig. 2.18: The

Design Mode in
Adobe XD.

40 Li, Drew, & Liu © Springer 2021

Fundamentals of Multimedia, 2nd ed. Chapter 2

Adobe XD
• Screens are linked by making elements clickable,
giving the “feel” of a real app.
• This is commonly used for UI design to be finalized
before sent for programming.

Fig. 2.20: The

Prototype Mode
in Adobe XD.

41 Li, Drew, & Liu © Springer 2021